The defining attribute "non-linear" of our subject might suggest
that we are dealing with deviations from the "natural" state of
linearity, which pervades much of our mathematics education and basic
scientific modeling. However, without non-linearity there can be no
change of stability, no hysteresis, no chaotic iteration, and most
importantly in the present context: no separated local optima.
On the other hand linearity is clearly more than just a special
case. Linear models closely reflect the properties of general
optimization problems in the vicinity of their local optima, provided
there is enough smoothness and regularity. This gives rise to a whole
host of nonlinear programming methods based on successive
linearizations and thus the work horses developed for linear
programming. For these Newton-like algorithms one may then establish
rates of convergence and complexity estimates on at least locally
convex problems. The only practical issue in this kind of nearly-linear
programming is the evaluation or approximation of the derivatives that define the local linearizations.
More genuinely nonlinear issues arise when the starting guesses
are further afield or the user demands the computation of global
optima. Then there is usually no guarantee of convergence and
algorithms behave like racehorses: very fast under some conditions,
hopelessly slow under others. Moreover, judging the conditions and
picking the right horse is no easy task. It often has to be based on
numerical experiments rather than mathematical results. Nevertheless,
evermore complicated and highly nonlinear computer models are
successfully being adapted for systematic optimization calculations
rather than ad hoc search procedures. The talk will highlight some of
the techniques and their application in various areas.