Many things seem to be accomplished using NNs, but I don't quite understand how they work. I looked for some documentation, but is basicaly just advanced math and that's not my strong point [lol]. So, for example, how does a character recognition system look like? Is it like a pyramid of neurons? I mean, if the input is a matrix, what will be the output, a single neuron telling true or false? I know there are some "weights" to rule the system, but how are they formed? If I move or change the input character a bit, how can it still be recognized? Can all this be explained in plain english? Is there maybe a nice tutorial to explain all this?

I started with this great tut:
Neural Networks in Plain English

The "Neural Networks in Plain English" tutorial is what I began with as well. After going through all of that, I wanted something that discussed everything in more detail, and I eventually found the following free PDF book on the topic:

An Introduction to Neural Networks

That document contains a thorough treatment of neural networks, including derivations of algorithms such as back-propagation.