hi guys, since a while, i had problems concerning how an ai agent could efficiently control a vehicle. Until now, i hadn't convincing results. So now i've something higher levelled in mind, but i would like to have your expert opinion before digging deeper into. Inspired from vehicle control with neural networks i thought i could use this as a good basis, however i lack of knoweldge about NN. So i've two questions: 1. In the article, they get decimal numbers ranging from 0 to 1 as output. However, in my case, the agent has 3 choices concerning turning for example: it turns left, it turns right, or does nothing. Obviously, the agent could decide 'turn left' when 0 < output < 0.33 and so on. But doesn't it affect the whole system? I mean, when training the network, you give input-output pairs if i understood right. So when you don't turn the "perfect output" is 0.5 and when you turn left it is 0, so when you'll get 0.3 as output you'll be closer to don't turn, so the limit should more be 0.25 and 0.75, isn't it? And now, consider you turn faster when you turn on the right than on the left, should these limits stay the same? How to decide them, is it arbitrary? 2. To improve it, i would like that the used NN would train itself with experience. Consider the vehicles passes checkpoints from time to time. Now the question is: could the respective training have importance/effect on the NN proportional to how good the travel time was? (So if it was very quick, the training has a big effect, and if it was poor the effect is very light) Is it doable, concerning complexity and consumed CPU time? that's it, thanks.

Hi,

First, you should read the article again.

1) For the steering, the output function is :

Steering = (Output - 0.5) * 2 * MaxSteeringAngle;

So, for an Output of 0, you'll have :

0 - 0.5 = -0.5
-0.5 * 2 * MaxSteeringAngle = -MaxSteeringAngle

For an output of 1, you'll have :

1 - 0.5 = 0.5
0.5 * 2 * MaxSteeringAgnle = MaxSteeringAngle

And for an Output of 0.5:

0.5 - 0.5 = 0... so, no steering at all.

Between 0 to 0.5 and 0.5 to 1, you'll have a smooth linear ramp from 0 steering to the maxsteeringangle in either direction.

2) You might want to consider encoding your neural nets in genetic algorithms and use the travel time as the fitness value of the solution. I would recommend doing it offline though, as it requires training.

Hope this helps

Eric

thanks for input, i still have some questions...

1) what i meaned was that for the output in the article, the ship turns proportionally to the output number, if it is 1, it'll turn very strongly, when it's 0.51 it'll turn very slightly. And what i wanted to point out is that in my case, you either turn or you don't don't, you cannot turn slightly or strongly. Therefore, the output must be a precise choice, not a number and thus the number needs to be converted into a choice. So this rises the questions of where the limits should best be placed.

2) what do GA have to do with NN?!

Oh, sorry, I didn't understand that you meant that there was no degree of turning.

I believe outputting a number between [0,1] isn't what you need then... have you thought of returning discrete values, say -1 for turning left, 0 for going straight and 1 for going right, without the possibility to return values in between these values?

As for the GAs... no they are not related to NNs, but they're a tool you can use to train them. An easy way would be to encode the weights in your solution and use the lap time as a fitness value... the best agents should reproduce themselves more often, etc...

Hope this helps

Eric

so neurons can also output discrete values within a finite set? like 1,2 or 3 for example?

Well, you're the programmer, you can make them do what ever you want them to do :)

You can write an activation function that outputs -1 for any negative value, 0 for 0, and 1 for any positive value... though you'll probably (in my opinion) get better results if you add padding on both sides of 0, because in float numbers calculations chances that you get an exact 0 are pretty low.

I'm no neural net expert thought. I read about them, messed with SOMs, but from my understanding it would be pretty easy for you to write your own "custom" output function.

Hope this helps

Eric

thanks

Genetic Algorithms are really powerful when combined with Neural Nets. Especially when you don't know what the correct answer should be.

the thing is...
i thought GA were used to "mutate" continouesly some structures according to some fitness criteria to obtain better ones. In other words: modify randomly some structure, keep the best ones, repeat the same again.
On the other hand, i see my brain as a fixed graph of neurons, where the neurons and synapses are unchanged forever, just the weights of the different parts are modified when trained.
Am I having a wrong view of these? So how can you combine these?! :/

The weights are mutated by the GA.

The fitness is the "lap time" of the racer.

Here you go :)