Mathematically any function can be represented with only two layers

Bwaaa???

Really? I had read that most problems are solved with only one hidden layer, but I didn't know of any mathematical law that said it was always possible? I assume that means "with an arbitrarily large hidden layer" then?

RE: my setup:

Yes, it's basically an adjacency matrix (with floats for elements). The columns (inputs) correspond to "strict" inputs and memories. The rows (outputs) correspond to "strict" outputs and the updated values for memories.

I've already got a "SetAllowedCoeffs" method which lets me enforce that certain coefficients must be zero. To make it emulate a traditional 1-hidden-layer ANN, I just need to disallow nonzero coefficients mapping directly from inputs to outputs, or from memories to memories. That should be trivial to do. Emulating passing some outputs back in as inputs... maybe harder, because of the temporal aspect. Although maybe if instead of trying to emulate it, I actually had additional inputs and outputs for it... I could try it. Might, even.

RE: no upper limit:

I can pick an arbitrarily large limit, but the real upper limit is indeed "how many pieces of gravel can I put under the player's foot before it makes the simulation cry?" And even though IRL there's a lot of nerve endings in a foot, I think the way we actually handle the foot/ground interaction is a lot more reactive force feedback than some comprehensive understanding of the external rigid-body physics. Maybe even to the extent that I don't even need to give it any contact point info as inputs... or so I had thought.

I'll think about categorizing the contact points into "sensor areas", and maybe taking some kind of average for everything within a sensor area. Dunno if I'm actually going to implement that yet.

Personal aside:

You (anyone reading this) may notice that I've got a habit of not trying things to see if they work. It's irrational, I know, but it's the result of ~2.5 years of nothing that I can say "works", and often when I try things not even being able to tell whether they're a step in the right direction. If you've got an idea, go ahead and suggest it, and I'll tell you why it cannot be done^[1]. And if you still think your idea has particular merit... idk, convince me?

^[1]A joke.

Please could you give a clear high-level overview of what problem you're trying to solve, as distinct form the techniques you're trying to use. I think that might be helpful to you in getting help/ideas.

Short version of what MrRowl said in PM: "what is it you're ultimately trying to do?"

Several things at once.

Some of the things are already working, under certain circumstances^[1]:

• The head should match a given orientation (or maybe just a forward vector). If I lock the position and orientation of the pelvis bone, and don't cause the arms to flail around too wildly in their attempts to aim the gun, this works pretty well. Even if I don't, using the analytic-ish system (mentioned near the beginning of the thread) for computing joint torques for the two spine joints, it still manages to face the right direction most of the time.
• The arms should hold the gun in a 2-handed grip that a) aims the gun in the right direction, and b) looks reasonable. I've gotten passable(ish) results by locking the position and orientation of the "torso 2" bone (to which the "head", "torso 1", "l shoulder" and "r shoulder" bones attach), or locking the pos & ori of the pelvis. Anyway, the system I have for this seems to work fairly well. If I shake the aim direction around too violently it has trouble; there's definitely still room for improvement.
• The bones "torso 2", "torso 1", and "pelvis" should match whatever orientations I tell them to match. Again, this works as long as the arms aren't doing anything too crazy, and everything below those bones is "cheating" to avoid making it unnecessarily difficult to comply. Presumably I can eventually get them to do their thing legitimately (see below).

Then there's a broad category of "lower body stuff" where I'm less sure of what I want, but have thought of a few possibilities:

• When airborne and not using the jetpack/rocketboots:
  • The pelvis (or maybe for "torso 2" instead) should match a given orientation
  • If a landing is imminent^[2], the feet should match a given pos/ori by that time
• When on the ground normally, I haven't got such a clear idea of what I want, but I have some general ideas:
  • The pelvis (or the upper body? or the CoM?) should match a given linear velocity
  • The pelvis should match a given orientation, and for best appearances, "torso 1"'s orientation should try to stay about halfway between the orientation of "torso 2" and "pelvis"
  • If the feet are grounded and supposed to stay grounded^[2], they should stay grounded. Orientation-wise, I think that will usually mean keeping the local y (up) vector of the foot matching the normal vector of the surface, although I recognize that sometimes it may be necessary to stand on the toe, heel, or sides of the foot. Alternatively, I could work in terms of a force/torque with which they should push against the ground^[2]
  • If the feet are supposed to come off of the ground, or stay off of the ground, they should probably be moving toward a destination pos/ori^[2]

Except for the "torso 1" and "torso 2" stuff (they don't really count as part of the "lower body" anyway), all of those "grounded" goals can be expressed as desired values for each of the "l foot", "r foot", and "pelvis" bones to match, specifiable as either a pos/ori, vel/rot^[3], or force/torque.

Now imagine the numeral 2 written a hundred times, with square brackets around it, in superscript... because as the footnote indicates, I don't know how I'm going to come up with the "goal state" values just yet (though I've got some general ideas). What I've been doing with this GA/ANN stuff recently is trying to come up with a "black box" implementation that can handle any goal states I throw at it... if it can handle as diverse an input set as possible, the implementation of that higher-level stuff shouldn't matter. That said, I do need to make sure I'm not asking it to do something that isn't possible.

Edit: even higher level? I want him to be able to look around, aim, stand, walk, run, jump, and land, on varied terrain.

^[1]See my video titled "Upper Body Stuff Working", wherein the Dood is floating in the air looking/aiming in various directions and "twerking" (or that's the label some of my friends gave it... bleh )

^[2]Details of how the higher-level code to make such determinations will work are TBD

^[2]In my code, my angular velocity variable is named "rot" for brevity (and consistency: all of the names "pos", "vel", "ori", and "rot" are the same length). So when you see "rot" in my posts, that's what it means.

Really? I had read that most problems are solved with only one hidden layer, but I didn't know of any mathematical law that said it was always possible?

It was mathematically proven in 1989.

To make it emulate a traditional 1-hidden-layer ANN, I just need to disallow nonzero coefficients mapping directly from inputs to outputs, or from memories to memories. That should be trivial to do.

I think you should go back and double check this. I strongly suspect that this is not the case for many reasons. I've never seen anyone use adjacency matrices for practical ANNs so I'm hesitant to put this out there but I would expect a correct ANN feedforward with adjacency matrices to be something more of the form:

output vector(Ox1) = sigmoid( layer2weights(OxH) * (layer1Weights(HxI) * inputs(Ix1)) )

where O is number of outputs, I number of Inputs, H number of hidden units and members of layer2Weights and layer1Weights are selected by the GA. The only parameters that should be changing within a simulation should be the inputs and outputs. If anything else is changing before evaluating a new GA individual or you're running your intermediate values through tanh, you're preventing the ANN from doing its job.

(edit) I'm going to say this again though..If you already know the positions and velocities you are looking for you may be better off just using inverse kinematics to develop the movement before trying to work it out through optimization algorithms.

To make it emulate a traditional 1-hidden-layer ANN, I just need to disallow nonzero coefficients mapping directly from inputs to outputs, or from memories to memories. That should be trivial to do.

I think you should go back and double check this. I strongly suspect that this is not the case for many reasons. I've never seen anyone use adjacency matrices for practical ANNs so I'm hesitant to put this out there but I would expect a correct ANN feedforward with adjacency matrices to be something more of the form:

output vector(Ox1) = sigmoid( layer2weights(OxH) * (layer1Weights(HxI) * inputs(Ix1)) )

where O is number of outputs, I number of Inputs, H number of hidden units and members of layer2Weights and layer1Weights are selected by the GA. The only parameters that should be changing within a simulation should be the inputs and outputs. If anything else is changing before evaluating a new GA individual or you're running your intermediate values through tanh, you're preventing the ANN from doing its job.

I'm certain. (Edit: certainty revoked due to not being sure whether you normally put the hidden-value weighted sums through sigmoid function or not)

I define that the rectangular chunks of the matrix mapping inputs to outputs, or memories to memories, must be zero; therefore, the only thing that can affect memories' is inputs, and the only thing that can affect outputs is memories. Do two iterations on the same inputs, and it exactly emulates a traditional hidden-layer ANN.

Oh, OHHHHH. Nononono. I left out some surrounding scope:

For each generation:

For each individual:

For X trials (currently 50):

Add a new Dood to the physics world

Start with M containing all zeros

For Y physics ticks (currently 45, = 3/4 of a second):

Simulate

Pack Dood state into I

For Z iterations (currently 2):

(O, M') = sigmoid((coeff matrix) x (I, M))

Use values of O as joint torques

Accumulate score (if the Dood does "perfectly" in a category, its score for this trial will add up to 1.0)

Remove the Dood from the physics world

Individual's score is the average of the X trials' scores

Pick parents from which to spawn the next generation

Edit: wait, what? Why aren't you putting the hidden-layer weighted sums through the sigmoid function? Isn't that how it's normally done?

Inverse kinematics... as in, to come up with a complete description of a pose or movement (i.e. pos/ori/vel/rot of each bone, on a frame-by-frame basis) based on some properties I want it to satisfy (i.e. what I want the feet and pelvis to be doing)?

The problem I foresee with this is once again what I was telling Buckeye: conservation of linear and angular momentum. Yes, the foot/ground interaction can effectively create or destroy momentum, but it will only do so in the manner the constraint solver dictates. A pose or movement chosen arbitrarily has no guarantee of being achievable.

This is a different definition than what you had before, and is probably closer to what it should actually look like. I'm still having trouble understanding why the iteration would be beneficial, however, since the difference would be that you're doubling the execution time of the algorithm.

Never mind the tanh thing, I just looked back through some of my old code and realized it works either way. I think I had better results using sigmoid in hidden layers.

Assuming the ANN is set up for hidden layer feedforward and you're not running internal information through again on the next timestep, the problem might just be in your GA or fitness function declarations, since that is a whole other world of pain in itself. Have you been able to evolve any simple goals like single joint movement to a desired state, etc?

Also, I meant that you can use some of the information from inverse kinematics to define your goal states or to set benchmarks that the optimization should converge toward, because you seem to have specific desires for how the solution looks and the GA will normally just converge onto anything that satisfies the requirements.

(edit)

Inverse kinematics is normally used in robotics with good success. Given the fact that it works for physical systems I have a hard time believing that it wouldn't work in a virtual one (especially since it is used all the time in game design with physics engines).

Also, definitely look into recurrence if you're trying to get sequences of actions going.

Another thing to think about: I work with a guy who used oscillatory neuron principles developed by Randall Beer to produce locomotion in animal-like robots. If you're feeling particularly daring it is an option: http://scholar.google.com/citations?user=F_J8QyAAAAAJ&hl=en&oi=ao

The reason for the multiple iterations is because it wasn't designed to emulate a traditional hidden-layer setup, it just happens to be capable of it.

Actually, the original idea for it was going to be with only 1 iteration per physics tick, and I was just going to accept the fact that the system's reaction times would be limited by that. In that original form, the "memories" were a lot more like what the name "memories" suggests, because they were always something remembered from the previous physics step, rather than from the previous step of iteration.

Simpler goal states... hm... *tries*

Edit: RE: recurrence: You say "look into" it... do you mean just try what you described (having outputs that get used as inputs), or do you mean there's significantly more to it than what you described and I should do some research? I'm already trying feeding outputs back in as inputs (separate from the "memories" system), in conjunction with a hidden-layer-ish coefficient matrix.

RE: oscillatory, etc.: That's a lot of papers! From the titles, it looks like he's done a lot with more "realistic" neurons? I've given it some thought too... Like, a real NN's learning is effectively agent-driven, and real neuronal activation is more like an event-driven system than the matrix math it's usually modeled with... I did a search yesterday and encountered the term "integrate and fire", but from the context it sounds like that's a field that hasn't been seriously worked on in years?

...papers scare me. One, because if I don't read them and continue asking for help, I look like a jerk. And two, because if I read them and get good ideas from them, Intellectual Property issues may arise.

What can I do about my fitness function being noisy?

It's "noisy" for two reasons:

1. Deliberate randomness, so that it can learn to deal with as many diverse situations as possible
2. Discrepancies caused by the order the constraint solver evaluates things being affected by the ordering of the elements of unordered sets

The first kind of noise can be dealt with by just throwing more trials at it. The second isn't really "noise" but "hidden variables", so throwing more trials at it (especially consecutive trials) doesn't help quite as much.

Other than "rewrite your constraint solver to be completely deterministic", what can I do?

Edit: I recorded some stats on how noisy the scores of a single individual can be, when the scores are taken as the average of 1, 10, or 100 trials... here are the results:

y = score

x = index after sorting from smallest to largest

It seems like maybe I just expected the Law of Large Numbers to kick in faster than it wanted to. So never mind that "hidden variables" stuff... probably. There are discrepancies, but they're tiny compared to the deliberate randomness. And that theory about "consecutive trials" was probably bogus.

Update! Abort!

I have given up on this whole GA/ANN business. There's five months of my life I won't be getting back.

My dream of using GA and ANN to create a system where I could simply tell it:

You have I inputs, O outputs, and M floats worth of memory to do scratch work in; give me a function that maps (inputs, memories) --> (outputs, memories'), and I'll tell you how well that functions does; find the (roughly) optimal function.

...is dead.

So what now? I don't know.

Possible "black box" formulation:
Inputs:
Anything and everything about the state of the Dood
(Maybe) info about objects the Dood is in contact with (or just his feet)
"Goal Description", possibly in the form of a desired linear and angular velocity (or net force and torque) on the left foot, the right foot, and the pelvis
Outputs:
Joint torques (3-component vectors) for all of the joints of the Dood's lower body: { left, right } x { hip, knee, ankle }

been a long time since i did this stuff, but i believe that's a problem in: statics and dynamics, dynamics, kinematics, rectilinear kinematics, multiple change of frame of reference.

sounds like what you need is a good robotics book.