|
This dissertation is concerned with the improved iterative dynamic programming algorithms for solving optimal control problems. First, a survey of iterative dynamic programming is proposed for the optimal control problem. The use of iterative dynamic programming (IDP) for optimal control of nonlinear dynamical systems is based on treating the control as a stagewise operation, gridding the state and control variables and contracting the grid size iteratively. In applying the backward IDP to obtain optimal control, it is required at each time stage to perform time integration of system dynamic equations for each allowable control starting from each state grid. This will result in wasting considerable computation time in such state grids that are not connected to a state grid at the previous time stage by a control. To overcome such a drawback of backward IDP, a forward IDP technique is proposed. Due to its forward operation, the proposed IDP provides substantial savings in computation time by eliminating the integrations starting from those state grids that are not approximately reachable with an allowable control. Since the conventional iterative dynamic programming (IDP) performs the stage- wise optimization in a backward manner, starting from the last time stage, the initial profiles for delayed variables in all but the first time stages are absent. The lack of initial profiles leads to poor convergence and inaccurate solution in obtaining optimal control policy for time-delay systems. It is demonstrated in this dissertation that the forward IDP technique can be used along with an interpolated integration scheme to overcome the curse of lacking initial profiles associated with the backward IDP in dealing with time-delay systems. The benifits of fast convergence and accurate solutions of using forward IDP are presented by several examples. In solving a high dimensional optimal control problems, two approaches are employed to enhance the possibility of obtaining the true optimum while reducing the required computation efforts. One approach employs Sobol''s quasi-random sequence generator to generate allowable controls and the other utilizes multi-pass computations. Numerical examples show that the use of multi-pass IDP computations with small numbers of state grid points and allowable control values can indeed enhance the possibility of obtaining a true optimum. This is particularly true when the allowable control values are generated by using Sobol''s quasi- random sequence generator. When the optimal control policy is relatively smooth, a small number of stages give excellent results when piecewise continuous control policy is used. To obtain convergence, a good initial control policy may be necessary. Such a starting control policy is readily obtained by low order piecewise continuous control policy. In the last section of this dissertation, we discuess the problem of optimal periodic control problems in which the initial state and periodic time are free parameters, and the periodic operation condition must satisfied for the state and control variables. To treate this problems, we extended the iterative dynamic programming to handle this particular requirements.
|