| Preface | 6 |
|---|
| Table of Contents | 8 |
|---|
| 1 Introduction | 14 |
|---|
| 1.1 Development of Artificial Intelligence | 14 |
| 1.2 Characteristics of Artificial Intelligent System | 18 |
| 1.3 Computational Intelligence | 22 |
| 1.3.1 Fuzzy Computing | 22 |
| 1.3.2 Neural Computing | 25 |
| 1.3.3 Evolutionary Computing | 25 |
| 1.3.4 Combination of the Three Branches | 28 |
| 1.4 Process Neural Networks | 29 |
| References | 30 |
| 2 Artificial Neural Networks | 33 |
|---|
| 2.1 Biological Neuron | 34 |
| 2.2 Mathematical Model of a Neuron | 35 |
| 2.3 Feedforward/Feedback Neural Networks | 36 |
| 2.3.1 Feedforward/Feedback Neural Network Model | 36 |
| 2.3.2 Function Approximation Capability of Feedforward Neural Networks | 38 |
| 2.3.3 Computing Capability of Feedforward Neural Networks | 40 |
| 2.3.4 Learning Algorithm for Feedforward Neural Networks | 41 |
| 2.3.5 Generalization Problem for Feedforward Neural Networks | 41 |
| 2.3.6 Applications of Feedforward Neural Networks | 43 |
| 2.4 Fuzzy Neural Networks | 45 |
| 2.4.1 Fuzzy Neurons | 45 |
| 2.4.2 Fuzzy Neural Networks | 46 |
| 2.5 Nonlinear Aggregation Artificial Neural Networks | 48 |
| 2.5.1 Structural Formula Aggregation Artificial Neural Networks | 48 |
| 2.5.2 Maximum (or Minimum) Aggregation Artificial Neural Networks | 48 |
| 2.5.3 Other Nonlinear Aggregation Artificial Neural Networks | 49 |
| 2.6 Spatio-temporal Aggregation and Process Neural Networks | 50 |
| 2.7 Classification of Artificial Neural Networks | 52 |
| References | 53 |
| 3 Process Neurons | 56 |
|---|
| 3.1 Revelation of Biological Neurons | 56 |
| 3.2 Definition of Process Neurons | 57 |
| 3.3 Process Neurons and Functionals | 60 |
| 3.4 Fuzzy Process Neurons | 61 |
| 3.4.1 Process Neuron Fuzziness | 62 |
| 3.4.2 Fuzzy Process Neurons Constructed using Fuzzy Weighted Reasoning Rule | 63 |
| 3.5 Process Neurons and Compound Functions | 64 |
| References | 65 |
| 4 Feedforward Process Neural Networks | 66 |
|---|
| 4.1 Simple Model of a Feedforward Process Neural Network | 66 |
| 4.2 A General Model of a Feedforward Process Neural Network | 68 |
| 4.3 A Process Neural Network Model Based on Weight Function Basis Expansion | 69 |
| 4.4 Basic Theorems of Feedforward Process Neural Networks | 71 |
| 4.4.1 Existence of Solutions | 72 |
| 4.4.2 Continuity | 75 |
| 4.4.3 Functional Approximation Property | 77 |
| 4.4.4 Computing Capability | 80 |
| 4.5 Structural Formula Feedforward Process Neural Networks | 80 |
| 4.5.1 Structural Formula Process Neurons | 81 |
| 4.5.2 Structural Formula Process Neural Network Model | 82 |
| 4.6 Process Neural Networks with Time-varying Functions as Inputs and Outputs | 84 |
| 4.6.1 Network Structure | 84 |
| 4.6.2 Continuity and Approximation Capability of the Model | 86 |
| 4.7 Continuous Process Neural Networks | 88 |
| 4.7.1 Continuous Process Neurons | 89 |
| 4.7.2 Continuous Process Neural NetworkModel | 90 |
| 4.7.3 Continuity, Approximation Capability, and Computing Capability of the Model | 91 |
| 4.8 Functional Neural Network | 96 |
| 4.8.1 Functional Neuron | 97 |
| 4.8.2 Feedforward Functional Neural Network Model | 98 |
| 4.9 Epilogue | 99 |
| References | 100 |
| 5 Learning Algorithms for Process Neural Networks | 101 |
|---|
| 5.1 Learning Algorithms Based on the Gradient Descent Method and Newton Descent Method | 102 |
| 5.1.1 A General Learning Algorithm Based on Gradient Descent | 102 |
| 5.1.2 Learning Algorithm Based on Gradient-Newton Combination | 104 |
| 5.1.3 Learning Algorithm Based on the Newton Descent Method | 106 |
| 5.2 Learning Algorithm Based on Orthogonal Basis Expansion | 106 |
| 5.2.1 Orthogonal Basis Expansion of Input Functions | 107 |
| 5.2.2 Learning Algorithm Derivation | 108 |
| 5.2.3 Algorithm Description and Complexity Analysis | 109 |
| 5.3 Learning Algorithm Based on the Fourier Function Transformation | 110 |
| 5.3.1 FourierOrthogonal Basis Expansion of the Function in L2[0, 2rr] | 110 |
| 5.3.2 Learning Algorithm Derivation | 112 |
| 5.4 Learning Algorithm Based on the Walsh Function Transformation | 114 |
| 5.4.1 Learning Algorithm Based on Discrete Walsh Function Transformation | 114 |
| 5.4.2 Learning Algorithm Based on Continuous Walsh Function Transformation | 118 |
| 5.5 Learning Algorithm Based on Spline Function Fitting | 121 |
| 5.5.1 Spline Function | 121 |
| 5.5.2 Learning Algorithm Derivation | 122 |
| 5.5.3 Analysis of the Adaptability and Complexity of a Learning Algorithm | 124 |
| 5.6 Learning Algorithm Based on Rational Square Approximation and Optimal Piecewise Approximation | 125 |
| 5.6.1 Learning Algorithm Based on Rational Square Approximation | 125 |
| 5.6.2 Learning Algorithm Based on Optimal Piecewise Approximation | 132 |
| 5.7 Epilogue | 139 |
| References | 139 |
| 6 Feedback Process Neural Networks | 141 |
|---|
| 6.1 A Three-Layer Feedback Process Neural Network | 142 |
| 6.1.1 Network Structure | 142 |
| 6.1.2 Learning Algorithm | 143 |
| 6.1.3 Stability Analysis | 145 |
| 6.2 Other Feedback Process Neural Networks | 148 |
| 6.2.1 Feedback Process Neural Network with Time-varying Functions as Inputs and Outputs | 148 |
| 6.2.2 Feedback Process Neural Network for Pattern Classification | 149 |
| 6.2.3 Feedback Process Neural Network for Associative Memory Storage | 150 |
| 6.3 Application Examples | 151 |
| References | 155 |
| 7 Multi-aggregation Process Neural Networks | 156 |
|---|
| 7.1 Multi-aggregation Process Neuron | 156 |
| 7.2 Multi-aggregation Process Neural Network Model | 158 |
| 7.2.1 A General Model of Multi-aggregation
|