: Huaguang Zhang, Derong Liu, Yanhong Luo, Ding Wang
: Adaptive Dynamic Programming for Control Algorithms and Stability
: Springer-Verlag
: 9781447147572
: 1
: CHF 139.10
:
: Elektronik, Elektrotechnik, Nachrichtentechnik
: English
: 424
: Wasserzeichen/DRM
: PC/MAC/eReader/Tablet
: PDF
There are many methods of stable controller design for nonlinear systems. In seeking to go beyond the minimum requirement of stability, Adaptive Dynamic Programming in Discrete Time approaches the challenging topic of optimal control for nonlinear systems using the tools of  adaptive dynamic programming (ADP). The range of systems treated is extensive; affine, switched, singularly perturbed and time-delay nonlinear systems are discussed as are the uses of neural networks and techniques of value and policy iteration. The text features three main aspects of ADP in which the methods proposed for stabilization and for tracking and games benefit from the incorporation of optimal control methods:
• infinite-horizon control for which the difficulty of solving partial differential Hamilton-Jacobi-Bellman equations directly is overcome, and  proof provided that the iterative value function updating sequence converges to the infimum of all the value functions obtained by admissible control law sequences;
• finite-horizon control, implemented in discrete-time nonlinear systems showing the reader how to obtain suboptimal control solutions within a fixed number of control steps and with results more easily applied in real systems than those usually gained from infinite-horizon control;
• nonlinear games for which  a pair of mixed optimal policies are derived for solving games both when the saddle point does not exist, and, when it does, avoiding the existence conditions of the saddle point.
Non-zero-sum games are studied in the context of a single network scheme in which policies are obtained guaranteeing system stability and minimizing the individual performance function yielding a Nash equilibrium.
In order to make the coverage suitable for the student as well as for the expert reader, Adaptive Dynamic Programming in Discrete Time:
• establishes the fundamental theory involved clearly with each chapter devoted to a clearly identifiable control paradigm;
• demonstrates convergence proofs of the ADP algorithms to deepen understanding of the derivation of stability and convergence with the iterative computational methods used; and
• shows how ADP methods can be put to use both in simulation and in real applications.
This text will be of considerable interest to researchers interested in optimal control and its applications in operations research, applied mathematics computational intelligence and engineering. Graduate students working in control and operations research will also find the ideas presented here to be a source of powerful methods for furthering their study.
Adaptive Dynamic Programming for Control2
Preface4
Background of This Book4
Why This Book?5
The Content of This Book5
Acknowledgments8
Contents10
Chapter 1: Overview15
1.1 Challenges of Dynamic Programming15
1.2 Background and Development of Adaptive Dynamic Programming17
1.2.1 Basic Structures of ADP18
1.2.1.1 Heuristic Dynamic Programming (HDP)18
1.2.1.2 Dual Heuristic Programming (DHP)19
1.2.2 Recent Developments of ADP20
1.2.2.1 Development of ADP Structures20
1.2.2.2 Development of Algorithms and Convergence Analysis23
1.2.2.3 Applications of ADP Algorithms24
1.3 Feedback Control Based on Adaptive Dynamic Programming25
1.4 Non-linear Games Based on Adaptive Dynamic Programming31
1.5 Summary33
References33
Chapter 2: Optimal State Feedback Control for Discrete-Time Systems40
2.1 Introduction40
2.2 In nite-Horizon Optimal State Feedback Control Based on DHP40
2.2.1 Problem Formulation41
2.2.2 In nite-Horizon Optimal State Feedback Control via DHP43
2.2.3 Simulations57
2.3 In nite-Horizon Optimal State Feedback Control Based on GDHP65
2.3.1 Problem Formulation65
2.3.2 In nite-Horizon Optimal State Feedback Control Based on GDHP67
2.3.2.1 NN Identi cation of the Unknown Nonlinear System67
2.3.2.2 Derivation of the Iterative ADP Algorithm70
2.3.2.3 Convergence Analysis of the Iterative ADP Algorithm71
2.3.2.4 NN Implementation of the Iterative ADP Algorithm Using GDHP Technique77
2.3.3 Simulations80
2.4 In nite-Horizon Optimal State Feedback Control Based on GHJB Algorithm84
2.4.1 Problem Formulation84
2.4.2 Constrained Optimal Control Based on GHJB Equation86
2.4.3 Simulations91
2.5 Finite-Horizon Optimal State Feedback Control Based on HDP93
2.5.1 Problem Formulation95
2.5.2 Finite-Horizon Optimal State Feedback Control Based on HDP97
2.5.2.1 Derivation and Properties of the Iterative ADP Algorithm97
2.5.2.2 The epsilon-Optimal Control Algorithm104
2.5.3 Simulations115
2.6 Summary119
References119
Chapter 3: Optimal Tracking Control for Discrete-Time Systems121
3.1 Introduction121
3.2 In nite-Horizon Optimal Tracking Control Based on HDP121
3.2.1 Problem Formulation122
3.2.2 In nite-Horizon Optimal Tracking Control Based on HDP123
3.2.2.1 System Transformation123
3.2.2.2 Derivation of the Iterative HDP Algorithm124
3.2.2.3 Summary of the Algorithm129
3.2.2.4 Neural-Network Implementation for the Tracking Control Scheme130
3.2.3 Simulations130
3.3 In nite-Horizon Optimal Tracking Control Based on GDHP132
3.3.1 Problem Formulation135
3.3.2 In nite-Horizon Optimal Tracking Control Based on GDHP138
3.3.2.1 Design and Implementation of Feedforward Controller138
3.3.2.2 Design and Implementation of Optimal Feedback Controller139
3.3.2.3 Convergence Characteristics of the Neural-Network Approximation Process147
3.3.3 Simulations149
3.4 Finite-Horizon Optimal Tracking Control Based on ADP150
3.4.1 Problem Formulation153
3.4.2 Finite-Horizon Optimal Tracking Control Based on ADP156
3.4.2.1 Derivation of the Iterative ADP Algorithm156
3.4.2.2 Convergence Analysis of the Iterative ADP Algorithm158
3.4.2.3 The epsilon-Optimal Control Algorithm162
3.4.2.4 Summary of the Algorithm163
3.4.2.5 Neural-Network Implementation of the Iterative ADP Algorithm via HDP Technique163
3.4.3 Simulations166
3.5 Summary170
References171
Chapter 4: Optimal State Feedback Control of Nonlinear Systems with Time Delays173
4.1 Introduction173
4.2 In nite-Horizon Optimal State Feedback Control via Delay Matrix174
4.2.1 Problem Formulation174
4.2.2 Optimal State Feedback Control Using Delay Matrix175
4.2.2.1 Model Network184
4.2.2.2 The M Network185
4.2.2.3 Critic Network185
4.2.2.4 Action Network186
4.2.3 Simulations187
4.3 In nite-Horizon Optimal State Feedback Control via HDP189
4.3.1 Problem Formulation189
4.3.2 Optimal Control Based on Iterative HDP192
4.3.3 Simulations198
4.4 Finite-Horizon Optimal State Feedback Control for a Class of Nonlinear Systems with Time Delays200
4.4.1 Problem Formulation200
4.4.2 Optimal Control Based on Improved Iterative ADP202
4.4.3 Simulations208