Dear GameDev.net community,
I want to share my idea to make "Knowledge Visualization" software.
And I hope I may also find some help from you, and maybe some people may also join this project.

Idea is simple: everyone watches movies - even adults, not only children (they watch cartoons more, but anyway...),
and this means that movies are very important part of everyday culture. Some people may have said hundred years ago: when television/cinema will appear, then people will stop reading books. But we know that theatre always existed, and now it just changed form.
Well, anyway, computer technologies brought new ideas - knowledge can be represented not only by actors playing in theatre/cinema,
but knowledge can be modelled using some 3d animation software, so many computer games appeared. And at the same time, there are still cartoons, movies, and those films which both use computer graphics (or drawing, special effects) and real video, which is taken usually simply by cameramen. But idea is that movie industry is a very expensive one, and there are millions dollars in that market: one can make a movie which costs (its production from A to Z) - say, million dollars, but get 10 times more. So movies are profitable.
On other hand, computer games are different - they are usually considered to be "time-killing" products, and mostly for children or not serious men. So my idea is to solve this contradiction: people can have power of computer graphics (just like in computer games), but use it to represent some important knowledge (which is usually displayed using movies). But here is a difference: on one hand, to make a movie, one would need some actors: and that actually is both plus and minus of movies. If actor has to play someone who is dead, or some hero (or even some god/God of mythology or religion or history), then it may create a problem. On other hand, arts are used to represent ideas which are either eternal and/or are present in some other time and/or space: movie cannot represent that which is on other planet, and camera cannot capture something which was present even on Earth in past: since that time objects for capturing could have been changed. And on other hand, scientific/computer 3d modeling/animation, can represent just everything - not only present on Earth, but past and future, and also on other planets or in some other realms...

Well, it was all philosophy. Now something more practical.

I've made a small program -
http://sourceforge.n...ojects/vedaviz/
it is on sourceforge, and free for download.
(it requires .NET Framework for MS Windows)
One can load any text in it (open rtf file), and add different images for any word s/he wants/needs. Program searches all words/images/ideas in text, and saves their locations. Now one can read a text, but at the same time see some visual help on side - images/illustrations which can be helpful to read text, to understand it deeper visually etc...

Now, the challenge which I face is the following: I have some (hobby) knowledge of Visual Basic .NET (though I learned programming since DOS - Pascal, QB, VB DOS, some C, Java, and even Assembler etc), and it has some limitations: I can place on screen only simple graphics - simple 2d images - raster images, not even vector. .NET supports some vectors, but they are too limited and cannot be added during runtime.
So I was thinking that 2d raster frames can be changed into some 3d animation, and for that I may need DirectX/OpenGL (or some other system). Problem is that I do not have any knowledge on how to write for DirectX, and also I ma not sure which exactly knowledge is more needed here.
I know that there are different game engines, graphics engines, game SDKs etc - I tried NeoAxis, Unity3d - I see that they are modern and can represent everything in good 3d modern animated graphics, using 3d acceleration etc.
So I do not have any idea how to connect Visual Studio (I use free Visual Basic Express 2010) to some modern-3d-graphical system.

Now, what I want to do in the program:
Program may use something like Artificial intelligence, but I hope this is not some utopia like "perpetuum mobile" or "time machine". No, something real.

I've got some scheme what program should do.
It may have three parts:
1. Linguistic part. Here text is opened, and it is analyzed using some parsing (part-of-speech tagging technologies, just like in translation from one language to another) techniques. When text is analyzed, it should generate some abstract model, similar to one in brain of humans. These abstract words correspond to real subjects/objects in real life (or in imaginary life, doesn't matter here - that's my beliefs). There is a program - Widgit - it automatically places images for each word in text (it can do so for simple texts - mostly for children). And there is a game - scribblenauts - when one types a word - object corresponding to it appears on screen and acts. One can type not just noun, but even some adjectives or other modifiers, just like in real language. And there are still few other programs like ALICE and Scratch. These two are simple game programming environments for children, but anyway, they can do something close to what can be done by powerful programming languages, but they are simplified in language - closer to human language. So, my idea is that if text is analized in this way, then there are two kinds of words - data and actions/scripts, and this would mean that any text itself is something like a program, which can be interpreted by brain or computer to generate visual representation in mind or on screen.
2.Second part is to use pairs of words and graphical images or 2d/3d objects. It may simply require some people to create 3d animated models/objects for words (of any part of speech). To simplify this, some abstract ideas can be used - object "human" can be used for any of 6 000 000 000 people living or people who lived in past or would live in future. So time is saved when one object is made well - it can be reused many times. I know one program - DAZ3d - it was free for some time, and there one can make different 3d objects of living and non-living beings, and also do animation using some Animate extensions etc. So idea is that I would like to combine all these ideas into one. Result should be similar to some pictorial dictionary, but animated one. It should be something like database, where each object is represented in db with some word. If that is animation, action (verb, and time words) - that can be shown as script in db. When they are combined, all that create multiple games/movies which cost millions of $ (!). Say, if we take 'second life' - there one can create some 3d object, do some animation on it etc, and there one can both sell something, and buy as well. But there are also many free objects on internet - if they are available, they can be included into program, so anyone can visually represent words of text just like movie or game or cartoon without spending so much time and money for that (well, this program, if done, would take time and efforts, so it would be probably not free, but hope not very expensive). Now, let's see that there different mind-maps - they also represent text, knowledge, but use just simple graphical 2d still raster icons, but why not use 3d animation, just like in modern games and movies???! That may boost the mind and imagination and creativity, wouldn't it? Or suppose there are many knowledge expert bases, expert systems, but why not combine all that together with 3d animation graphics? So, 2nd module is about graphics programming. (1st module - computer linguistics)
3. 3rd module is just a visualization. Final result should be similar to what is done by converting texts to movies. Different movies can be done for same texts. It depends on people who make those movies. So program may allow further creativity and imagination - skins, extensions whatever, extended rendering, or even playing in a game which is made based on some text, converted to 3d-animated-graphics (vector) model. One can imagine himself a part of a book he is reading, by watching everything in that book as not just words but as real 3d objects. Can you imagine that if we look outside windows, and see only words instead of real objects? It would have been rather boring. So there is art, there is creativity, there are other visual forms of representation of information. And it thus text can be transformed into 3d-visual objects using some visualization program.


If anyone is interested, has any comments, suggestions, criticism, whatever thought on subject - welcome here.
I posted it to Artificial intelligence, because it is really a challenge - how to make a computer game (or movie) from a text/book - (semi)automatically. Usually this may need different people to participate. But if that is more or less done automatically - just like there some schemes on how to make a movie from text (that might be learned in some cinematic/theatre-universities), so there can be a way out how to make a 3d-animated game out of text/book.

Well, if idea is still not clear - ask here and I will clarify what is it all about.


© VedaViz 2012

You might find this interesting. It's older, but has some elements of what you described.