| Adaptive Dynamic Programming for Control | 2 |
---|
| Preface | 4 |
| Background of This Book | 4 |
| Why This Book? | 5 |
| The Content of This Book | 5 |
| Acknowledgments | 8 |
| Contents | 10 |
| Chapter 1: Overview | 15 |
---|
| 1.1 Challenges of Dynamic Programming | 15 |
| 1.2 Background and Development of Adaptive Dynamic Programming | 17 |
| 1.2.1 Basic Structures of ADP | 18 |
| 1.2.1.1 Heuristic Dynamic Programming (HDP) | 18 |
| 1.2.1.2 Dual Heuristic Programming (DHP) | 19 |
| 1.2.2 Recent Developments of ADP | 20 |
| 1.2.2.1 Development of ADP Structures | 20 |
| 1.2.2.2 Development of Algorithms and Convergence Analysis | 23 |
| 1.2.2.3 Applications of ADP Algorithms | 24 |
| 1.3 Feedback Control Based on Adaptive Dynamic Programming | 25 |
| 1.4 Non-linear Games Based on Adaptive Dynamic Programming | 31 |
| 1.5 Summary | 33 |
| References | 33 |
| Chapter 2: Optimal State Feedback Control for Discrete-Time Systems | 40 |
---|
| 2.1 Introduction | 40 |
| 2.2 In nite-Horizon Optimal State Feedback Control Based on DHP | 40 |
| 2.2.1 Problem Formulation | 41 |
| 2.2.2 In nite-Horizon Optimal State Feedback Control via DHP | 43 |
| 2.2.3 Simulations | 57 |
| 2.3 In nite-Horizon Optimal State Feedback Control Based on GDHP | 65 |
| 2.3.1 Problem Formulation | 65 |
| 2.3.2 In nite-Horizon Optimal State Feedback Control Based on GDHP | 67 |
| 2.3.2.1 NN Identi cation of the Unknown Nonlinear System | 67 |
| 2.3.2.2 Derivation of the Iterative ADP Algorithm | 70 |
| 2.3.2.3 Convergence Analysis of the Iterative ADP Algorithm | 71 |
| 2.3.2.4 NN Implementation of the Iterative ADP Algorithm Using GDHP Technique | 77 |
| 2.3.3 Simulations | 80 |
| 2.4 In nite-Horizon Optimal State Feedback Control Based on GHJB Algorithm | 84 |
| 2.4.1 Problem Formulation | 84 |
| 2.4.2 Constrained Optimal Control Based on GHJB Equation | 86 |
| 2.4.3 Simulations | 91 |
| 2.5 Finite-Horizon Optimal State Feedback Control Based on HDP | 93 |
| 2.5.1 Problem Formulation | 95 |
| 2.5.2 Finite-Horizon Optimal State Feedback Control Based on HDP | 97 |
| 2.5.2.1 Derivation and Properties of the Iterative ADP Algorithm | 97 |
| 2.5.2.2 The epsilon-Optimal Control Algorithm | 104 |
| 2.5.3 Simulations | 115 |
| 2.6 Summary | 119 |
| References | 119 |
| Chapter 3: Optimal Tracking Control for Discrete-Time Systems | 121 |
---|
| 3.1 Introduction | 121 |
| 3.2 In nite-Horizon Optimal Tracking Control Based on HDP | 121 |
| 3.2.1 Problem Formulation | 122 |
| 3.2.2 In nite-Horizon Optimal Tracking Control Based on HDP | 123 |
| 3.2.2.1 System Transformation | 123 |
| 3.2.2.2 Derivation of the Iterative HDP Algorithm | 124 |
| 3.2.2.3 Summary of the Algorithm | 129 |
| 3.2.2.4 Neural-Network Implementation for the Tracking Control Scheme | 130 |
| 3.2.3 Simulations | 130 |
| 3.3 In nite-Horizon Optimal Tracking Control Based on GDHP | 132 |
| 3.3.1 Problem Formulation | 135 |
| 3.3.2 In nite-Horizon Optimal Tracking Control Based on GDHP | 138 |
| 3.3.2.1 Design and Implementation of Feedforward Controller | 138 |
| 3.3.2.2 Design and Implementation of Optimal Feedback Controller | 139 |
| 3.3.2.3 Convergence Characteristics of the Neural-Network Approximation Process | 147 |
| 3.3.3 Simulations | 149 |
| 3.4 Finite-Horizon Optimal Tracking Control Based on ADP | 150 |
| 3.4.1 Problem Formulation | 153 |
| 3.4.2 Finite-Horizon Optimal Tracking Control Based on ADP | 156 |
| 3.4.2.1 Derivation of the Iterative ADP Algorithm | 156 |
| 3.4.2.2 Convergence Analysis of the Iterative ADP Algorithm | 158 |
| 3.4.2.3 The epsilon-Optimal Control Algorithm | 162 |
| 3.4.2.4 Summary of the Algorithm | 163 |
| 3.4.2.5 Neural-Network Implementation of the Iterative ADP Algorithm via HDP Technique | 163 |
| 3.4.3 Simulations | 166 |
| 3.5 Summary | 170 |
| References | 171 |
| Chapter 4: Optimal State Feedback Control of Nonlinear Systems with Time Delays | 173 |
---|
| 4.1 Introduction | 173 |
| 4.2 In nite-Horizon Optimal State Feedback Control via Delay Matrix | 174 |
| 4.2.1 Problem Formulation | 174 |
| 4.2.2 Optimal State Feedback Control Using Delay Matrix | 175 |
| 4.2.2.1 Model Network | 184 |
| 4.2.2.2 The M Network | 185 |
| 4.2.2.3 Critic Network | 185 |
| 4.2.2.4 Action Network | 186 |
| 4.2.3 Simulations | 187 |
| 4.3 In nite-Horizon Optimal State Feedback Control via HDP | 189 |
| 4.3.1 Problem Formulation | 189 |
| 4.3.2 Optimal Control Based on Iterative HDP | 192 |
| 4.3.3 Simulations | 198 |
| 4.4 Finite-Horizon Optimal State Feedback Control for a Class of Nonlinear Systems with Time Delays | 200 |
| 4.4.1 Problem Formulation | 200 |
| 4.4.2 Optimal Control Based on Improved Iterative ADP | 202 |
| 4.4.3 Simulations | 208 |