Dual control of exploration and exploitation for auto-optimization control with active learning

The quest for optimal operation in environments with unknowns and uncertainties is highly desirable but critically challenging across numerous fields. This paper develops a dual control framework for exploration and exploitation (DCEE) to solve an auto-optimization problem in such complex settings. In general, there is a fundamental conflict between tracking an unknown optimal operational condition and parameter identification. The DCEE framework stands out by eliminating the need for additional perturbation signals, a common requirement in existing adaptive control methods. Instead, it inherently incorporates an exploration mechanism, actively probing the uncertain environment to diminish belief uncertainty. An ensemble based multi-estimator approach is developed to learn the environmental parameters and in the meanwhile quantify the estimation uncertainty in real time. The control action is devised with dual effects, which not only minimizes the tracking error between the current state and the believed unknown optimal operational condition but also reduces belief uncertainty by proactively exploring the environment. Formal properties of the proposed DCEE framework like convergence are established. A numerical example is used to validate the effectiveness of the proposed DCEE. Simulation results for maximum power point tracking are provided to further demonstrate the potential of this new framework in real world applications. Note to Practitioners —In numerous engineering applications, it is highly desirable to operate a system to improve the efficiency, enhance performance or save energy. However, attaining this optimal control is a challenging task, due to the presence of unknown system and/or environment parameters. We develop a principled approach to balance between exploration and exploitation, involving active learning to estimate unknown parameters and tracking the optimal operational condition based on current estimation. This paper provides a unified framework to solve general auto-optimization control problems. The simulation results demonstrate that the proposed method outperforms existing methods in terms of efficiency and optimality for maximum power point tracking problem, and it can be readily implemented for many other engineering problems. Future research include generalizing the proposed method to nonlinear systems, as well as exploring novel applications to facilitate the widespread adoption of our method.