Suppose I have two types of bots, A and B. Bots of type A learn what they need to learn in whatever way, do what they need to do, and generally go on about their buisiness. Bots of type B need to learn to reproduce the type-A behavior as close as possible, i.e. they need to immitate type-A bots. But they need to do this in a real-time setting, and they have no access to the internal processing of type-A. What are the possible approaches I could take to implement type-B bots? Does it help if there are large numbers of bots of each type, and/or if type-B bots can somehow exchange what they learn? EDIT: One possible approach would be to use a nervous system similar to the one in Karl Sims' creatures, and let it evolve by exchanging genes between the type-B bots, rather than passing them from one generation to the next. But then, how would I define the fitness function to direct this evolution? In other words, how to evaluate how similar is the type-B bot's behavior with that of the type-A bots? For that matter, how can I characterize the type-A behavior at all? [Edited by - technobot on November 4, 2004 9:53:16 AM]

Well, doesn't anyone have any idea? I know the question is very general, but I'm only looking for a direction.. I don't mind even if you post suggestions for more specific cases, as long as you mention what case it applies to.


Btw, by "realtime" I only meant that the application runs at reltime frame-reates. The learning still goes on gradually (although it shouldn't take excessively long).

I would say you could use some kind of genetic algorithm. I am real into genetic programming, but it is fairly involved depending on how free bot A is to changing its algorithm.

About testing how well B bots do, simply run an A bot and a B bot and sum the error. The could be anything. Distance between A and B, etc. Whatever it is you want to minimize

Dwiel

First, if you have acess to bot a's inputs and outputs then:

You can either
1.train a neural network or
2.Train a bayens net

This is simple(ish)

If you don't, then i haven't a clue how to help you.

Good day.

From,
Nice coder

do you want the type-B bots to eventually be created with previously learned knowledge (i.e. "instinct" ) or will new type-B bots have to re-learn stuff (or be taught it by other type-B bots)?

I don't know if this would work for bots or not, but on the topic of immitating, you might look into markov chains. Using markov chains you can immitate English looking text, given real English text as input. Immitation is one thing. Finding an improvement is another. You might be able to use markov chains to generate new pseudo-random decisions for your bots, and then combine them with ANNs, GAs, TD based learning, or any other method of learning.

Quote: Original post by Tazzel3D
I would say you could use some kind of genetic algorithm. I am real into genetic programming, but it is fairly involved depending on how free bot A is to changing its algorithm.

I suppose that A bots will be learning quite slowly, in any case not faster than B bots.

Quote:
About testing how well B bots do, simply run an A bot and a B bot and sum the error. The could be anything. Distance between A and B, etc. Whatever it is you want to minimize

That would work if I could give A's and B's the same sequence of inputs, but what if that's not an option?

lucky_monkey:
Yes, I want new B bots to inherit their knowledge from the ones before them. Although assumming I take the genetic approach, each B bot will have a slightly different behavior, so I can't just copy it to new B bots. So I guess I could take the few existing bots that give the best results, mix their genes, mutate, and then set that as the new B's starting point, just like in regular GA.

Russell:
Ok, I'll have a look into those a bit later.

Quote: Original post by technobot
Yes, I want new B bots to inherit their knowledge from the ones before them. Although assumming I take the genetic approach, each B bot will have a slightly different behavior, so I can't just copy it to new B bots. So I guess I could take the few existing bots that give the best results, mix their genes, mutate, and then set that as the new B's starting point, just like in regular GA.

I'm not completely sure that genetic algorithms are the best solution to the problem, but here's a couple of things:

If you want emergent behaviour in a bot's lifetime then you could periodically regenerate the bot's genome at the expense of possibly creating an inconsistent personality change, this variation in behaviour could detract from the illusion of intelligence.
This evolution could optionally be done with the bot itself as one of the parents, creating a less radical change (and encouraging more diversity in the population).

Don't worry too much about mutation, it's not a very good method of getting emergent behaviour.

Quote: Original post by lucky_monkey
[...] then you could periodically regenerate the bot's genome [...] This evolution could optionally be done with the bot itself as one of the parents, creating a less radical change (and encouraging more diversity in the population).

This is approximately/similar-to what I meant by:

Quote:
EDIT:
One possible approach would be to use a nervous system similar to the one in Karl Sims' creatures, and let it evolve by exchanging genes between the type-B bots, rather than passing them from one generation to the next.

In other words:
1. Take N B bots.
2. Let them run for t time, and evaluate performance during that time.
3. Select the best n<<N bots, and combine their genes.
4. Replace the genes of all N bots with the resulting gene set, and apply a minor mutation to each bot, to avoid identical behavior and allow for evolution.
5. Repeat from 1.
When a new bot is built, its gene set is set according to step 4.

But as I wrote then, there is the problem of how to do step 2, i.e. how to evaluate th bots' performance..

EDIT: to avoid sudden changes of behavior, one could interpolate the genes from the previous set to the new one, for some transitional period dt.

One interesting idea that I've read a little about is using genetic algorithms to improve an artificial neural net.

Quote: 4. Replace the genes of all N bots with the resulting gene set, and apply a minor mutation to each bot, to avoid identical behavior and allow for evolution.

There have been some interesting experiments showing that mutation doesn't really contribute much to evolution, some experiments did without mutation altogether and got very good results. Genetic crossover does most of the work of propelling the population towards a better solution.