Recently in a discussion in my Neural Networks class, the instructor talked about Hopfield Networks (Associative memories), and how we can write Energy Functions for those nets. These energy functions are actually Lyapunov functions, which when negative semi-definite show the model to be stable. I have been trying for a long time to get hold of more literature on the topic (applying Lyapunov functions to Hopfield nets to show their stability). I have come across only one paper (google/citeseer) on the topic, which hasn't been very helpful. Can someone drop me a link to something that can tell me more on Lyapunov functions/stability of Hopfield nets? Thanks!!

Did you want literature just on Lyapunov functions and their use in stability proofs (see Control Theory literature) or on Hopfield network stability, or on just the use of Lyapunov functions for the proof of stability in Hopfield networks? The latter you should find in any decent book on associative memory. I pulled 3 off my shelf and two of them had at least a page covering the basics of this topic. You really shouldn't have any problems with going to your local college library and finding more information. Alternatively, try here as a starting point.

Cheers,

Timkin

Quote: Original post by Timkin
Did you want literature just on Lyapunov functions and their use in stability proofs (see Control Theory literature) or on Hopfield network stability, or on just the use of Lyapunov functions for the proof of stability in Hopfield networks? The latter you should find in any decent book on associative memory. I pulled 3 off my shelf and two of them had at least a page covering the basics of this topic. You really shouldn't have any problems with going to your local college library and finding more information. Alternatively, try here as a starting point.

Cheers,

Timkin

Yes, I had seen the proof of stability in Hopfield networks using Lyapunov functions in my textbook, but I hadnt explored any Control Theory Literature, maybe that is where I should be looking. Thanks for the link.

Quote: Original post by sidhantdash
I hadnt explored any Control Theory Literature, maybe that is where I should be looking.

Lyapunov functions are a staple of dynamical systems research, so wherever you encounter literature dealing with issues of stability of dynamical systems, you're bound to come across Lyapunov functions. If you want a deep understanding of them, turn to books on Nonlinear Ordinary Differential Equations (such as Grimshaw's tome of the same title)... there's plenty of literature in Control Theory covering the topic, since stability of controlled processes is one side of the coin (performance being the other) and thus central to the field. The application of Lyapunov functions to other dynamical systems (such as the parameter processes in machine learning architectures) is more of a recent development, but by no means any less relevant.

If you're interested in stability, you might also want to look at Popov stability.

Cheers,

Timkin