Gotta Scrut that Hut! Sorry all my projects for the last 10 years use Scrut Termonology, if its distracting then i will refrain best unless referencing code.
Complicated + overly Sophisticated Artificial Neural Network Designs and Implementation for what I was aiming for. Yes i know all this stuff is soooooooo trivial like takes no thinking and very little design if any( just to keep it somewhat professional ) Following picture is of a neural network design I was working on. I already have working artificial neural networks that have same features used, actually made one that had more advanced, way more than this. This network is used because it would take to long to draw a real one.
CSoma - Holds pointer to two dendrites, how ever the post synaptic neurons dendrite i named as DendritePointer m_pcrAxon;( give appearance of more realistic design, as far as implementation goes pretty much the same )
CDendrite - contains two Pointers, one to presynaptic neuron one to post synaptic neuron. Has information about propagation which includes sensitivity, and interface for propagation rules allowing for Competative Propagation by having the implementation of your propagation rule contain all properties local to that functionality. Competative Propagation keeps track of how ofter an activation of each neuron takes place so if one does not fire for to long of a time it is offset in favorability eventually causing it to fire so you do not kill entire parts of your network from some reason maybe training what ever.
CArtificialNeuralNetwork - Maintains\owns of CSoma objects. Holds\owns a table of all dendrites within your network. I keep two lists here because you can have a CDendrite without a CSoma( i know makes no sense ) and you can have a CSoma with no dendrites( makes no sense ) but it is this way for graphical manipulation of the systems so you can create dendrites or breaking conections and moving them around preserving the sensitivity(weight of the dendrite - also known as a result of Receptor Density located on the membrane your dendrite, and no I didnt model any presynaptic receptors( why would I as I do not need feedback unless specified and yes the network can Run BACKWARDS just expirimenting )
Activation Functions
CStepwiseActivation - if you seek true\false answers no fuzzy logic in the output here
CLinearActivation - experimenting on optimum usage
CHyperbolicTangentActivation - experimenting on optimum usage yet have not got the results described by other people. Disapointed...
CSigmoidActivation - Most common activation when starting a new project, once running( not trained ) I just research as hard as i can about my current network and the ANN Editors for my engine are just so hardcore. I made them in visual basic real fast, but going to port to c++ as i have a home made engine with GUI so i could make much more complex editor than ever possable in vb. And already have the plans for pushing the tools further as vb was too slow and didnt want to waste to much time.
CSignalToActivation - Copies the signal to the activation variable. No per-soma processing here, only manipulation is the dendrites acting on the data with sensitivities. You can also select in your propagation Rules object weither to enable\disable sensitivities(No reason for this just wanted to do it)
CShadowActivation - if you select this then you must choose the activation function used on the shadowed signal as CSoma calls and jumps after copying over the signal property inside of CShadowSignal as the Soma only knows there is 1 Activation so without this design you would shadow but never reach activation on ANY LEVEL. This is not a good design however changing it on final draft, which is next one. References Activation of another Neuron(does not have to be held in the same network but if no then on shut down u need to be careful.
CShadowSignal - Same as CShadowActivation as it contains the Activation function that follows and is responible for setting the activation value. References Signal of another Neuron(does not have to be held in the same network but if no then on shut down u need to be careful.
Default Artificial Neural Network Configuration upon System Startup
-Recurrent Neural Network - Determined by Propagation and Connections
-Activation Function - Sigmoid by default\Example above also used Step-wise
and Signal to Activation on incoming signals.
-Propagation Rules - Full Propagation by default
-Confidence Intervals - disabled by default
-Learning Algorithm - Math capable to tearing apart this black box and
somehow works to set sensitivities closer( hopeful ) on each iteration in
the training phase. If you actually look at the code from different
ways( oop looks different than c implementation and that different than
matrices compared to double\single linked lists or arrays and std:vector
plus etc... ) Set be default to Either Q-Learning or Back Propagation of
Error
Activation Functions so you can read them better, quick( bad ) definitions above
-CStepwiseActivation
-CLinearActivation
-CHyperbolicTangentActivation
-CSigmoidActivation - Default for Middle\Output Neurons
-CSignalToActivation - Default for Input Neurons( rarely see them else where )
-CShadowActivation
-CShadowSignal
Propagation Rules
-CCompetativePropagation - Propagation to only one post synaptic neuron
-CFriendlyPropagation - Treats all the connects as the same, distributing to
every connection in reach
Confidence Intervals
-CDualBoundsCheck - Simple confidence intervals using bounds checking with
user's values. The neural network does no aid in bounds checking atm, this
is what I am working on next time I code.
-Researching Credible interval oriented at the moment around Bayesian
intervals. Left out much needed research on Confidence Intervals so I am
making haste.
Training Algorithm - also note I am Playing with genetic algorithms currently
Supervised Leraning
-QLearning - Reinforcement Learning
-Back Propagation of Error
Unsupervised
-Next training Algorithm I attempt.
ok once again this is so simple design. Only reason i say its a little more advanced than a n00bs( although this is from experience as n00b programers just do the most illogical things ) but no state of the art or even close. Its just a homebrew ANN that can "Attempt" the more complex( this is the complex part ) configuration of the engine with substructures\recurent\Feed Back\Bias( 1 for entire network with one unique sensitivity for each axon to dendrite connection ) giving rise to the behaviors i was looking for.
I thought ANNs were just to delicate, and they kinda are at some times. But the more you use them, its like the world opens up. This is running a Behavioral Pattern Analyzer. It just learns from you telling it what you do during the day, and there is functionality to train a portion of the network to give rise to something that resembles moral. The filters do checking on the information and resolve any problems. say you have not slept in 9 days and you told it that you are going to drink an entire pot of coffee. It should, unless broken to all ****, give outputs, they turned into sentences so the end user can read easier, but should say NO WAY GOOD SIR! Go to Sleep Mr.
---------------
earlier was the picture of some design i went through. this is not the exact same one used but this one is MUCH SIMPLER and took way less time than the real one would of. So Are these types of networks advanced? And btw if you attempt this and have trouble training, well then go ahead and message me as I had the same problems at first. Special way to train these monsters, so go ahead and message me i will be glad to help you out.
[Edited by - InnocuousFox on October 23, 2010 4:48:07 PM]
