The use of differential equations in optimization

A new approach for unconstrained optimization of a function f(x) has been investigated. The method is based on solving the differential equation dx/dt = ± ∇f(x) which defines orthogonal trajectories in Rⁿ-space. A number of numerical integration techniques have been used for solving the above differential equation, the most powerful one which gives rise to a very efficient optimization algorithm is the generalization of the Trapezoidal rule. The interaction between the parameters which appear as a result of using the numerical integration has been investigated. In the above approach factorization of the positive definite matrix (θG + λI), allowing some control over the diagonal elements of the matrix. is presented. A Liapunov function approach has been used in constructing a number of different differential equations of the above form. It is well known that if a Liapunov function which satisfies certain conditions can be found for a given system of differential equations then the origin of the system is stable. Pursuing this idea further we constructed a Liapunov function and then the corresponding differential equation. Application of this differential equation to the problem of finding a minimum of a function f is shown to yield a vector that converges to a point where ∇f = 0. The use of differential equations is also extended to the optimal control problem. The technique is only applicable to unconstrained optimal control problems. If a terminal condition and inequality constraints are presented, the problem should be converted to unconstrained form, e.g. by the use of penalty functions. The method tends to converge, even from a poor approximation point to the minimum without using line searches.