I recently attended the American Institute of Aeronautics and Astronautics (AIAA) Guidance, Navigation, and Control (GNC) conference. I've been to this conference before and I really enjoy it. It's not the top-of-the-line academic performance, but there's a lot of great papers, and most of the top engineers attend. Plus, at least for aerospace engineering in dynamics and controls, in particular, it is probably the flagship conference.

This year I noticed something interesting. But first a bit of background. During the time of WWII, a lot of important advancements were made in signal processing, control, ans stochastic processes. Obtaining optimal solutions (particularly stochastically, in a MMSE sense) were very important. Norbert Weiner had discovered the MMSE filter, but it was not easily applicable to problems of a sequential nature (which all control systems are). Rudolf Kalman, and Richard Bucy had developed a sequentialized version of the Weiner filter, which revolutionized the signal processing, and more particularly the controls industry.

Thus the Kalman filter was born. It has some major limitations, most notably in that it only works for linear time-invariant (LTI) systems. As everyone knows, most systems aren't linear. But it turns out that with some rough approximations (and a helping of the central limit theorem) most systems are almost linear, and the Kalman filter has been applied to many many systems and been surprisingly, and somewhat remarkably good. Extensions to the Kalman filter have been numerous: the extended Kalman filter (EKF, attempts to handle non-linear systems), the unscented Kalman filter (UKF, also attempts to handle non-linear systems more elegantly than the EKF), the consider Kalman filter, iterated extended kalman filter (IEKF), and the list goes on and on. But even with all the advances, the ability to truly handle complex non-linear systems has been a challenge to the controls community for a long time.

As computers became faster, monte carlo methods have become an important tool in signal processing and controls as they can solve problems for ANY system, even highly non-linear ones. In particular, the particle filter (PF) is a sequential monte carlo method that has found a great deal of success in this area. It runs very quickly, and can produce near optimal results (though for the PF there is no guarantee of true optimality like there is for a KF).

In the controls community, KFs, PFs, EKFs, and UKFs have obtained a sort of notoriety for attracting people. In fact, if a paper uses one of these filters, or otherwise pontificates on the strengths or weaknesses of these filters, it is sure to attract a room-busting audience.

Sometimes this gets ridiculous, even to the point of approaching a religion. I attended one paper presentation in which a PF, EKF, and UKF were compared in filtering a particular non-linear system for an Unmanned Air Vehicle (UAV). After the presentation, the discussion bordered on absurd. One guy is quick to point out that the authors missed a technical detail on their implementation of the PF and should have gotten better results for the PF. Another guy points out the theoretical superiority of the UKF and insists that they must have done something wrong since it was not clearly the winner. And on and on it goes each setting forth the reasoning for why his favorite filter is superior. By the end, all I could think was "look people, it's just a filter!"

It reminds me of the dreaded Windows vs. Linux vs. Mac debate that plagues the pages of Slashdot and Engadget. Each filter, like each operating system, has its strength and weaknesses. Rather than viewing it as some competition of optimality in which one gets the nod of academicians while the others get a hairy eyeball, I view each as a tool in my toolbox to be selected with prudence and wisdom in applying a control algorithm to my system. But despite my confusion people will continue to justify, validate, and argue for their filter of choice as if the filters cared about our loyalty to them. Who says scientitst and engineers are above religious arguments anyway? Seems to me they just have different religious arguments. Seriously people, it's just a filter!

I find your lack of faith disturbing.

ReplyDelete[makes squeezing motions with his hand roughly in the direction of Michigan]

Wait, what's wrong with pointing out Windows a the tool crafted by satan to bring people down to everlasting destruction? Just kidding.

ReplyDeleteI'm glad to see engineers have their idiosyncrasies as well. :)

Why is it that a scientist always looks down on engineers? Those guys have always known the art of imperfection, while scientist is always frustrated because he can't know it all!

ReplyDelete"[makes squeezing motions with his hand roughly in the direction of Michigan]"

ReplyDeleteLove it!!

"Wait, what's wrong with pointing out Windows a the tool crafted by satan to bring people down to everlasting destruction?"

Well, in all fairness I have done my fair share of Windows bashing. It's just so easy!!

"Why is it that a scientist always looks down on engineers? Those guys have always known the art of imperfection, while scientist is always frustrated because he can't know it all!"

ReplyDeleteHear hear!!! ;-)

::as I smile wryly and jump back behind the barracks to await enemy fire::

Actually Anonymous, it definitely happens (though I don't think the scientists on this blog are guilty of this, after all they did invite me to be an author here). When I worked at LLNL it definitely had that type of culture. The engineers were sort of subservient to the nuclear physicists and often belittled by them. Seems strange when you really think about it.

I find my lack of understanding with regard to your post disturbing. If I say its all Greek to me, does that mean I need to do more reading or that you are a genius? Or both? Hm!

ReplyDelete"I find my lack of understanding with regard to your post disturbing. If I say its all Greek to me, does that mean I need to do more reading or that you are a genius? Or both? Hm!"

ReplyDeleteUh oh! I find your lack of understanding disturbing, because it probably means it is written poorly! Care to elaborate on the "Greek" and I'll try to clear it up.

Here are some definitions:

MMSE = Minimum Mean Squared Error. This is very common optimal criterion for filtering out the noise of a specific signal.

Linear system = A system of the form Ax=b where x and b are vectors, and A is a matrix.

Time-invariant = a property of a system that indicates that the system behaves the same way (i.e. output is the same for every input) no matter what time I start the system.

Weiner Filter = The MMSE filter for a linear time-invariant system.

Kalman Filter = The sequentialized MMSE filter for a linear time-invariant system.

Inverse Problems = Essentially solving the equation Ax=b when A is either non-invertible or ill conditioned.

Monte Carlo = (multiple definitions depending on field, but for Electrical Engineering) a class of methods of solving inverse problems by repeatedly performing the forward problem Ax=b

Particle Filter = A specific sequentialized Monte Carlo filter capable of handling non-linear systems using a collection of particles which are weighted and resampled according to a specific distribution.

BTW, just out of curiosity, is my vernacular totally inaccessible to the physicists in the crowd (which is probably the primary audience right now)?

ReplyDeleteIf so, I'll do a better job of giving background and proper definitions.

jmb275,

ReplyDeleteI enjoyed it. There were a couple of terms I didn't understand but my posts are probably even worse. :)

LOL

ReplyDeleteJmb275,

ReplyDeleteAppreciate your candor about engineers!

For engineers, all systems are linear, at time=T, for duration of deltaT, and they march on perfecting the art of imperfection, for that deltaT times at a time.

"For engineers, all systems are linear, at time=T, for duration of deltaT, and they march on perfecting the art of imperfection, for that deltaT times at a time."

ReplyDeleteInteresting you should bring this up. This is, in fact, a pretty good description. This is exactly what the EKF tries to do. It doesn't really "handle" non-linear systems (contrary to what I said). In reality, it simply linearizes the system at time step.

It's an important point because the truth is, if the system is non-linear there's VERY little we can do to control it, filter it, or otherwise analyze it. There are a few controllers for non-linear systems but they're not that good, and proving stability for non-linear systems is NOT trivial.

jmb275,

ReplyDeleteI did that for nearly 40 years, and you are correct, stability is nothing to sneeze at. Actually, we are always trading on islands of instabilities, and we can do that as long as they remain small. So when that tank blew up on Apollo 13, it was deltaT at a time, that brought them to a fighting chance to enter the Earth atmosphere back again. Enjoy!

Forgot to take a friendly dig at the scientist! He/she would never do that, but will have litteny of things that will go wrong, and wrong, and wrong...

ReplyDelete