Hehe...after posting this I realised how long it had become... my apologies...
Quote:
Original post by mnansgar
I'm pursuing a doctorate in computational neuroscience.
Off Topic: What's your thesis on? I've worked in CN previously, most particularly on seizure prediction algorithms and image segmentation and registration algorithms. It's a fascinating field. 8)
Back on topic...
Quote:
but I still stand by my previous statement since linear systems are/have been so important to engineering.
There's a huge difference between mathematics and engineering, between what we know and what we can make/sell. For example, we teach undergraduate engineering students that the gradient of a vector field is a vector. It isn't. It's a one form. It just so happens that in Euclidean space, one forms and vectors have equivalent properties. Step outside of Euclidean space and this equality doesn't hold, meaning analysis based on this assumption would fail. Why do we teach these student a 'lie'? Because in general, they'll never need to know they weren't taught the truth. ;) If they do, then we teach them the truth of the matter (which generally only happens when they learn the error of their ways and become mathematicians! ;) )
Quote:
Practical engineering techniques heavily rely on linearization
Yes, they do. In part, because we don't bother to teach engineers advanced mathematical analysis and design techniques and because linearisation works on many real world problems... but that's because most of the real world problems we deal with in every day life are quasi-linear. That's more of a statement about the domain over which engineering presides (and can survive while presiding over), rather than our inability to deal with tougher problems.
There are obviously exceptions to this in which the problems we are trying to engineer solutions for are highly nonlinear. In these cases, linearisation is often applied, but only because the engineer involved doesn't know a better technique, doesn't have the time/money to develop a better implementation or doesn't want to implement anything 'new'. I face this attitude regularly when dealing with in-house engineers of our industry partners. The tools are there though and if you look at engineering R&D, you'll certainly see nonlinear analysis and design techniques being implemented, particularly in areas like Control Theory.
Quote:
especially if you'll agree that numerical analyses are typically just iteratively small linearizations
Most certainly not. If you restrict your view to only finite difference analyses of differential models/systems, then perhaps so... but you're ignoring a wealth of techniques that don't rely on any linearisation of a system model. For example: phase space analysis, spectral analysis, statistical analysis and functional analysis, to name but a few.
Quote:
but I have the impression that your nonlinear work is more of the exception than the norm given the limited resources often allocated to controllers.
Certainly industry still relies on simple solutions (because they're easy to understand and sell as ideas to management), but nonlinear methods have existed since the earliest days of control. Today, there are certainly many more linear devices (such as PIDs) than nonlinear ones, but thats more to do with the inertia involved in shifting industry than any lack of knowledge regarding nonlinear methods.
Quote:
I argue that these are typically (1) limited to classes of equations which have been rigorously studied
So because it has been rigorously studied, that makes it exempt in the consideration of nonlinear vs linearisation? I think perhaps you should have argued along the lines of the "size of the class of problems that have been rigorously studied". Indeed, anything beyond second order is normally the realm of mathematicians (engineers often deal with so called 'ideal second order systems')... and third order systems become tough to analyse, requiring advanced tools such as Lie Algebra and asymptotic methods... but this is only if you're trying to predict state evolution exactly. If you want to analyses the system for its performance (which is quite often all we require of engineered systems) then you don't need to know the exact state; only that it is stable and under what conditions it might traverse to instability... and that it meets a performance criteria.
Quote:
(2) require iterative methods which often require intuition (e.g. number of layers/hidden nodes)
There's nothing wrong with a good iterative learning method, so long as you have the time and the data! ;) Structural learning is certainly possible, but what you find is that human intuition is often quite a good first guess.
Quote:
(3) require vast computing resources for accuracy, drastically limiting their usefulness in practical systems
Yes, you have a point here, when you compare the resources required to analyse a nonlinear system compared to a simple set-point linearisation (which can be achieved by something as simple as linear regression of the local data). Certainly, nonlinear analysis techniques are data intensive and resource consuming. However, used appropriately, on many problems the increased performance far outweighs the cost.
Quote:
(4) even so are typically limited to qualitative/numerical analyses.
I disagree with that. I can (almost) just as easily fit a second order polynomial model to anything you can linearise. That's neither qualitative nor numerical. If, however, I estimate the parameters of my model online, then it certainly is numerical... but then so is the linearisation.
Quote:
I will leave on a positive note though, saying that as you indicate, their use is definitely increasing nowadays perhaps largely due to the availability of fast computing resources.
...and broader acceptance of techniques... which is sort of the point of Rodney's statements... that if we can reduce the science of complex adaptive systems to a description that a secondary school student can understand, then we will see revolutionary change in how we perceive and control our environment around us. Of course, this would also be true if we could teach quantum mechanics to pre-schoolers.
Quote:
I for one would be very interested in hearing your reasons to believe paridigm shifts may occur.
Actually this has a lot to do with my pessimism about science and scientific method, particularly with regards to how we approach problems. Complex systems research is a good example. We bring many of our preconceptions about dynamic systems (developed from years of linear analysis techniques ;) ) to the table when we try and analyse these systems. Like much of science, we try and atomise the problem to understand it... and in parallel systems, that simply doesn't work. Yet most of these systems are comprised from simple elements, interacting with simple rules. It's the internal balance and harmony of the interplay of the components that enables these systems to survive and be observed. I firmly believe we
will find a way to mathematically describe these systems, because we already have some of the important tools that open our eyes to what is going on within these systems. I just believe we're trying to describe these systems in the wrong way. We haven't worked out exactly what our tools are telling us and what we're not seeing yet. My optimisim also stems from the simplicity of the substructure of these systems. Simple elements can be described in simple ways. The complexity arises when you try and describe the global properties in terms of local properties. I think we'll find a way of doing that and that it will, at its heart, be simple and beautiful, just like the systems it describes.
Cheers,
Timkin
