Robotics... hmm... You're right, what I'm trying to do and what roboticists try to do is very similar. Print media, though?

There's a few interesting differences between robotics and simulation. That's not to say they render robotics knowledge inapplicable, but I feel like enumerating them anyway:

• I don't have to deal with imperfect sensors. I have complete knowledge of the state of the system.
• I don't have to deal with imperfect motors. My motors will always exert exactly the torque I tell them to; it's just a matter of telling them to do what will bring about the best results.
• With IRL robotics, the "simulation" and the control logic are asynchronous, and the control logic runs slower than the simulation. In my simulation, on the other hand, the simulation and the control logic run in sequence, repeating together at 60hz.

Regarding that old formulation:

That formulation left out temporal stuff. As I mentioned more recently (including in my previous post), a more generalized formulation would be some function that maps...

(inputs, data) --> (outputs, data')

...where "inputs" includes state info as well as the goal description, and "data" is an arbitrary chunk of scratch memory for use in whatever manner the function needs.

Also, in some cases, such as for the purposes of unpowered airborne maneuvering, it seems like it would be better to use a desired vel/rot for torso2 (the bone to which the head and shoulders attach) instead of the pelvis, and let the system affect the spine1 and spine2 joint torques (the joints between the pelvis, torso1, and torso2 bones).

Experiment: thruster nozzles with thrust vectoring

Terminology note: I may use "jet", "thruster", and "rocket" interchangeably. Earlier in the development of this project, I would've bothered to distinguish... but I realized (after a conversation with my father) that "jet" doesn't have to strictly refer to "a jet engine, with an intake". The term "jet" was around long before jet engines were invented. Consider the acronym JATO, jet-assisted take off... with rockets. Or the manner in which cephalopods rapidly expel water to propel themselves. So it's fine.

My Soldier Dood now has a list of thruster nozzle declarations. Each nozzle declaration has: a bone it attaches to, a position in that bone, a 'center' direction, a maximum angle away from that center the produced force vector can deviate, and a maximum force magnitude. I made a solver which takes a desired net force and torque, and attempts to achieve that by picking thrusts for each nozzle, staying within the nozzle's allowed force cones.

I believe the solver is doing the best it can do given the limitations I imposed, but unfortunately, it's not good enough to reproduce the capabilities the Dood had beforehand. A few possible approaches came to mind:

1. Omnidirectional thrusters. This is effectively what I had before implementing this new system, only there were only two thrusters (one in each shoulder bone), and the desired force was split evenly between them (with no consideration for torque). But when I eventually add rendering for the thruster flames, omnidirectional thrusters look bad, because the flames will often be going into the Dood and coming out the other side... especially when jetpacking backwards.
   
2. Thrusters with their "center direction" pointing directions other than downward. Obvious candidates would be putting backward-facing thrusters in both shoulders, leftward-facing ones in the left shoulder, and rightward-facing ones in the right shoulder... front, side, and rear thrusters in the feet might be another possibility. Maybe even front-facing thrusters somewhere on the shoulders or chestpiece?
   
3. Get the pose system to deal with orienting the thruster bones. This would be complicated by the fact that the positions and orientations of the bones the thrusters are mounted to won't update until the next physics tick. Still, in theory... Currently the jetpack will only fire when the Dood is airborne anyway, so if I don't have to worry about an imminent landing, I can get a fair deal of control by moving the feet around. I don't know how I would be able to orient the shoulder bones to control where the shoulder-mounted thrusters point without messing up the aiming of the gun, though.

I feel like some combination of approaches 2 and 3 will ultimately produce the best results. But for now, I've just reverted to the old system (described comparatively in "approach 1").

I really think you should spend some time reviewing Michiel van de Panne's work -- a lot of it overlaps with exactly what you're trying to do (if I understand correctly): physically simulated locomotion and control systems/approaches for physics-driven animation.

Here's a video which you might find inspiring:

Seriously.. he has spent decades researching this field, it is well worth browsing through his work to see what might be useful.

All of his papers can be found here: http://www.cs.ubc.ca/~van/papers/

Robotics... hmm... You're right, what I'm trying to do and what roboticists try to do is very similar. Print media, though?

like most engineering endeavors, if you want real answers, you gotta break down and buy a good book on the subject. white papers can help, but usually are based on the fundamentals in the standard texts.

There's a few interesting differences between robotics and simulation. That's not to say they render robotics knowledge inapplicable, but I feel like enumerating them anyway:
I don't have to deal with imperfect sensors. I have complete knowledge of the state of the system.
I don't have to deal with imperfect motors. My motors will always exert exactly the torque I tell them to; it's just a matter of telling them to do what will bring about the best results.
With IRL robotics, the "simulation" and the control logic are asynchronous, and the control logic runs slower than the simulation. In my simulation, on the other hand, the simulation and the control logic run in sequence, repeating together at 60hz.

all that should just make your job easier.

besides, didn't it ever occur to you that robotics manufacturers might already do this to test their control software in a simulated environment?

i know i 'd run it on a sim first before i DL'd untested code to a multi-thousand dollar industrial robot, turned it on, and hoped it didn't break too may things. <g>

"No! Stupid robot! You're supposed to install the windshield ONTO the car, not THROUGH the hood! Oh, God! Shut it down! Shut it down! Well... clean up this mess, check the code, and go get me another piece of glass..."

@Norman Barrows:

RE: making my job easier: Yes and no. That perfect synchronicity can produce some nasty feedback which wouldn't occur in a real system.

RE: robotics uses simulation: Yes, I know they do.

@raigan: neat!

Feed-forward neural nets learning!

I finally got a feed-forward neural net to learn! The thing that seems to have made it work this time is that instead of doing proper gradient descent, where I compute the entire gradient, and then modify all the values at once, I just compute the partial derivative of the error with respect to one weight at a time, and immediately modify that weight.

I don't know why it didn't work when I tried to do it using proper gradient descent. Has anyone else had a similar problem?

Anyway it's working now.

Let's go! What's the hold-up?

Hypothetically I could generate a "training set", containing (state of the Dood, correct joint torques to output) pairs. However, there are several issues I'm concerned about that have made me hesitate to even attempt this.

Coming up with a training set is going to take a long time. I've estimated that finding the optimal outputs for a given input state may take as much as one or two seconds, if I only care about the state of things in the immediate next physics step. I don't know how many data points will be necessary, but I imagine it will be a lot. Probably at least as two for every input dimension, so e.g. if there were only 3 input dimensions, there would need to be at least data points for +x, -x, +y, -y, +z, and -z... and zero. The number of inputs I've given my setup has changed numerous times; it's been anywhere from 100 to 400. So let's say 800 data points, at 1.5 seconds each... that's 20 minutes.

That's not so bad, right?

Now for the bad part: as soon as I make any changes to the simulation (e.g. change the shape or MassInfos of the Dood), all that data becomes obsolete. Maybe some time could be saved by salvaging something from the trained neural net? But the old training set would be entirely invalid, and most of the the time will be spent generating the training set, not doing the actual training.

Looking further forward?

The "one or two seconds" estimate above assumes I'm only concerned with how well the state of the Dood in the immediate next physics step manages to satisfy some goals, (e.g. "pelvis" bone must have such-and-such linear velocity; assign a penalty proportional to the error squared of this value).

Maybe I should be looking further forward than the immediate next physics step? The best way to do it would probably be to evaluate the function described above for the several next physics steps, accounting for what the goals may be in the future... and maximize the minimum value of "best possible score".

I don't think it would be feasible to do this for more than the first additional physics step, however, because the branching factor is effectively infinity. Can I approximate this? Is "kinetic energy" a good heuristic?

Better inputs / better outputs

Obviously with neural nets it's important to choose the right inputs and outputs.

Earlier I considered having some outputs be feedback (to be fed back in as inputs), but in order to train a feed-forward neural net to do that, I need to be able to specify the correct values for those feedback variables. What sort of things could I use for this?

I wanted to use GA to automate the selection of feedback variables (basically equivalent to "memories"), but... that was a failure.

I might just try to generate a training set for a simple (state & goals, joint torques) mapping. Maybe I'll learn something in the process.