A year or two back, I found the A.L.I.C.E. bot ( http://www.alicebot.org/ ) and found it quite fascinating. A short while later, I was thinking that it could be possible to make those bots "smarter." ALICE has no recollection of previous conversations, limiting herself a lot. The AIML code is very static, she cannot add code in herself. But what if she could? If, the bot could understand and then categorize information that is given to it, it could use more and more information as the basis for everything it says. Being a computer program, it would have vast amounts of space to store the information it learns. I had forgotten about this for quite a while, but decided to throw it up amongst a bunch of programmers and see if they think it is a plausible new face of AI. I'll try to explain my ideas more indepth about the "brain" of the bot. It would have a couple different categories of information in xml databases, for example, an English dictionary, fact dictionary, opinion, user specific dictionary and questions/unknown facts. Here's an example of how a conversation could work... User (Matt): I just got a new dog today. I named him Oliver. Bot: (think)From Dictionary- Dog=Furry animal; From Fact Dictionary- Dog=commonly held as pet, Stinks when wet, many different species; Into User Dictionary- Matt=Got pet dog on Oct 3/05, Dog's name is Oliver; Into Question Dictionary- What species of dog? Where did you get it? What colour? (REPLY): That's cool, what kind of dog is it? User (Matt): It's a Black Lab, we also got a red collar and leash for it. Bot: (think) From Dictionary- Black Lab=Species of Dog; Into User DIctionary- Matt=Dog is Black Lab, has red collar and red leash; Question queue- Where did Matt get dog from? What colour is Matt's dog? (REPLY): What colour is the dog and where did you get it from? That's an example of a pretty basic conversation, but I think the basic theory could work in something larger scale. Eventually it oculd get to the point of being able to have a semi intelligent conversation with someone or be able to have an article or something like that be pasted and pull facts out of it... I just want to know what some of you guys think about it, particularly if it could work or not. I've got quite limited programming knowledge (HTML and a tiny bit of C++), so I wouldn't be able to follow up on this by myself. Thanks for any input

It *can work*, but only for very specific types of input and where your English phrases are not ambiguous. For instance:

(1) Take your example -- Black Lab. You used an abbreviation! Will your dictionary have all possible abbreviations as well (this is getting to be pretty large!)? Is this a species of dog, or is it a black laboratory (you could say this can be inferred from the question, but what if the user just types he has a black lab without context?)?

(2) Note that you've defined all of your words in terms of other words! How can the computer "understand" a word if each is defined by similar words that it doesn't know? Now you might refute this by saying that it will match words together; however, it is very likely that a different set of words are used in the definition than are needed to actually tell that it fits that definition, correct? If you then decided to apply a thesaurus, we'll easily run into a situation where we are likely recursive and take forever to make decisions!

(3) Parsing sentences is also a big probelm. Sure, we can deal with "Bill is red." (just assign the quality 'red' to 'Bill'), but we can't deal with natural language. Even the sentences you posted have imprecise grammar (bad comma usage mostly).

(4) How does the computer deal with actions which require visualizing the situation? Or with things which require human experiences to really understand?

... and so on. The point is, yes it is possible for a very limited subset of the English language. I think the more interesting problem lies in designing a language close to English but more amenable for computer parsing.

It's possible to parse grammars even if there would be ambiguity in constructing parse trees - google "GLR parser". The nastiest ambiguous parts of English are prepositional phrases; the rest isn't bad. The problem I had when I was writing this kind of parser is that I couldn't find any reference material that describes English grammar in anything remotely close to BNF.

Ok, so let's say you manage to get past that part, and have a parse forest output by a GLR parser that's stored in a graph (by folding the common parts of each tree - there is a good description of this algorithm somewhere, but it's been years since I've dealt with it).

The main problem is: what the heck do you do with the graph? I think probably the most productive thing you can do is compare the graph to other preconstructed graphs (the knowledge base of your program). My idea was that you can take two graphs, connect the matching leaves end-to-end, and start marking any nodes that match, reducing both graphs to what did and didn't match. (I use this same technique for finding intrinsic functions like strcpy in optimized assembly). Ideally, you can associate some kind of metadata with each graph (or possibly metadata for each major section of the graph) that more effectively represents concepts for your AI.

Then you could finally do some grammar-based analysis of the matching and remainder sections of the graphs to figure out what the sentence means in terms of your knowledge base. You would still have to come up with a way to add new graphs to your knowledge base, do some decision making to come up with a reply/question to the user, and be able to formulate sentences that don't seem "stale" to a person.

Here's my partial grammar which I was using to test graph generation...
Note - I came up with this thing myself, so if it looks like a piece of crap, it's because I'm not a linguistics expert.

It currently completely ignores punctuation.



And if the list of terminals and nonterminals didn't scare you, here are some productions (nonfinal experimental list that works decently for a lot of my testing input):



Yeah... And that's incomplete and really horribly unoptimized (parsing with that grammar results in horribly bloated parse graphs).

But that should give you a general idea about how freaking horrible even a partial natural language grammar looks.

Quote: Original post by tetelestia
It would have a couple different categories of information in xml databases, for example, an English dictionary, fact dictionary, opinion, user specific dictionary and questions/unknown facts. Here's an example of how a conversation could work...

You probably want to stay away from XML for data storage on this one. You don't want to have to wait an hour for your program to do matching on a single graph just because it's having to parse the XML multiple times because you can't store the knowledge database in RAM, would you?

I have a system like this I've been developing for the NPC brains in our game. It's similar in concept to the ALICE bots you speak of. It has the same goal you've mentioned of being able to create and save out constructs so it 'learns' over time.

However, I use JSON instead of XML, which is much faster/easier to work with! JSON can do anything XML does. http://www.crockford.com/JSON/index.html

Cheers,
Roy
http://www.p2pmud.com

@mnangsar - The basic idea of all of your points is that it would have trouble understanding the complex nature of the english language and replying correctly. Have you looked at the ALICE bot? My (not so) new idea would basically expand on that program's base understanding of english. Including abbreviations in the dictionary wouldn't add much space at all, ALICE understands "black lab" and even "lol." It's knowledge base could potentially be enormous...

@Nypren - Yes, XML would be pretty inefficent, it could run at a tolerable speed if it were sorted/categorized enough, still a bit of a problem (maybe it'll just have to wait for quantum computing ;))

@fur - That's pretty cool, how far has it come along? Would you mind letting me see what you've got done?

ALICE doesn't understand anything. It seems to understand "lol" because the programmers specifically told it to look out for that word and how to respond. Otherwise, ALICE is just a clever scheme which looks for simple sentence structures and responds with preformed sentences. The AIML code is basically a list of templates to watch out for. It would be very difficult for the computer to generalize random English sentences which are spoken to it when if you look around, very few sentences have the EXACT same structure (and those which do are probably already coded in!).

One problem I came across when contemplating this was that it’s one thing to have the program understand a “perfect sentence”, but the problem is, people don’t use perfect sentences, and even fewer use them while typing!

As far as not understanding imperfect grammar, spelling, and even abbrieviations, is there any reason that the bot/ai can't do what a human could do when he/she doesn't understand something?

That is, why can't you make it ask questions about what it doesn't understand, based on what it _can_ understand about the context and structure of the sentance? Granted, I understand some sentances are completely unrecoverable. ;)

Still, allowing it to ask a series of questions might help (and letting it just give up eventually could be an option). Also, for abbrieviantions, you are left with the option of scanning the dictionary for the abbrieviated part, and finding options that may match, and asking more specific questions. "Lab? Are you talking about a dog, or a place where you blow things up?"

Granted, these options may not be _fast_, but I hope this helps you out somehow!