Peyman Setoodeh, PhD, is Visiting Professor with the Centre for Mechatronics and Hybrid Technologies (CMHT) at McMaster University. He is a Senior Member of the IEEE.

Saeid Habibi, PhD, is Professor and former Chair of the Department of Mechanical Engineering and the Director of the Centre for Mechatronics and Hybrid Technologies (CMHT) at McMaster University. He is a Fellow of the ASME and the CSME as well as a Canada Research Chair and a Senior NSERC Industrial Research Chair.

Simon Haykin, PhD, is Distinguished University Professor with the Department of Electrical and Computer Engineering and the Director of the Cognitive Systems Laboratory (CSL) at McMaster University. He is a Fellow of the IEEE and the Royal Society of Canada. He is a recipient of the Henry Booker Gold Medal from the International Union of Radio Science, the IEEE James H. Mulligan Jr. Education Medal, and the IEEE Denis J. Picard Medal for Radar Technologies and Applications.

List of Figures xiii

List of Table xv

Preface xvii

Acknowledgments xix

Acronyms xxi

1 Introduction 1

1.1 State of a Dynamic System 1

1.2 State Estimation 1

1.3 Construals of Computing 2

1.4 Statistical Modeling 3

1.5 Vision for the Book 4

2 Observability 7

2.1 Introduction 7

2.2 State-Space Model 7

2.3 The Concept of Observability 9

2.4 Observability of Linear Time-Invariant Systems 10

2.4.1 Continuous-Time LTI Systems 10

2.4.2 Discrete-Time LTI Systems 12

2.4.3 Discretization of LTI Systems 14

2.5 Observability of Linear Time-Varying Systems 14

2.5.1 Continuous-Time LTV Systems 14

2.5.2 Discrete-Time LTV Systems 16

2.5.3 Discretization of LTV Systems 17

2.6 Observability of Nonlinear Systems 17

2.6.1 Continuous-Time Nonlinear Systems 18

2.6.2 Discrete-Time Nonlinear Systems 21

2.6.3 Discretization of Nonlinear Systems 22

2.7 Observability of Stochastic Systems 23

2.8 Degree of Observability 25

2.9 Invertibility 26

2.10 Concluding Remarks 27

3 Observers 29

3.1 Introduction 29

3.2 Luenberger Observer 30

3.3 Extended Luenberger-Type Observer 31

3.4 Sliding-Mode Observer 33

3.5 Unknown-Input Observer 35

3.6 Concluding Remarks 39

4 Bayesian Paradigm and Optimal Nonlinear Filtering 41

4.1 Introduction 41

4.2 Bayes’ Rule 42

4.3 Optimal Nonlinear Filtering 42

4.4 Fisher Information 45

4.5 Posterior Cramér–Rao Lower Bound 46

4.6 Concluding Remarks 47

5 Kalman Filter 49

5.1 Introduction 49

5.2 Kalman Filter 50

5.3 Kalman Smoother 53

5.4 Information Filter 54

5.5 Extended Kalman Filter 54

5.6 Extended Information Filter 54

5.7 Divided-Difference Filter 54

5.8 Unscented Kalman Filter 60

5.9 Cubature Kalman Filter 60

5.10 Generalized PID Filter 64

5.11 Gaussian-Sum Filter 65

5.12 Applications 67

5.12.1 Information Fusion 67

5.12.2 Augmented Reality 67

5.12.3 Urban Traffic Network 67

5.12.4 Cybersecurity of Power Systems 67

5.12.5 Incidence of Influenza 68

5.12.6 COVID-19 Pandemic 68

5.13 Concluding Remarks 70

6 Particle Filter 71

6.1 Introduction 71

6.2 Monte Carlo Method 72

6.3 Importance Sampling 72

6.4 Sequential Importance Sampling 73

6.5 Resampling 75

6.6 Sample Impoverishment 76

6.7 Choosing the Proposal Distribution 77

6.8 Generic Particle Filter 78

6.9 Applications 81

6.9.1 Simultaneous Localization and Mapping 81

6.10 Concluding Remarks 82

7 Smooth Variable-Structure Filter 85

7.1 Introduction 85

7.2 The Switching Gain 86

7.3 Stability Analysis 90

7.4 Smoothing Subspace 93

7.5 Filter Corrective Term for Linear Systems 96

7.6 Filter Corrective Term for Nonlinear Systems 102

7.7 Bias Compensation 105

7.8 The Secondary Performance Indicator 107

7.9 Second-Order Smooth Variable Structure Filter 108

7.10 Optimal Smoothing Boundary Design 108

7.11 Combination of SVSF with Other Filters 110

7.12 Applications 110

7.12.1 Multiple Target Tracking 111

7.12.2 Battery State-of-Charge Estimation 111

7.12.3 Robotics 111

7.13 Concluding Remarks 111

8 Deep Learning 113

8.1 Introduction 113

8.2 Gradient Descent 114

8.3 Stochastic Gradient Descent 115

8.4 Natural Gradient Descent 119

8.5 Neural Networks 120

8.6 Backpropagation 122

8.7 Backpropagation Through Time 122

8.8 Regularization 122

8.9 Initialization 125

8.10 Convolutional Neural Network 125

8.11 Long Short-Term Memory 127

8.12 Hebbian Learning 129

8.13 Gibbs Sampling 131

8.14 Boltzmann Machine 131

8.15 Autoencoder 135

8.16 Generative Adversarial Network 136

8.17 Transformer 137

8.18 Concluding Remarks 139

9 Deep Learning-Based Filters 141

9.1 Introduction 141

9.2 Variational Inference 142

9.3 Amortized Variational Inference 144

9.4 Deep Kalman Filter 144

9.5 Backpropagation Kalman Filter 146

9.6 Differentiable Particle Filter 148

9.7 Deep Rao–Blackwellized Particle Filter 152

9.8 Deep Variational Bayes Filter 158

9.9 Kalman Variational Autoencoder 167

9.10 Deep Variational Information Bottleneck 172

9.11 Wasserstein Distributionally Robust Kalman Filter 176

9.12 Hierarchical Invertible Neural Transport 178

9.13 Applications 182

9.13.1 Prediction of Drug Effect 182

9.13.2 Autonomous Driving 183

9.14 Concluding Remarks 183

10 Expectation Maximization 185

10.1 Introduction 185

10.2 Expectation Maximization Algorithm 185

10.3 Particle Expectation Maximization 188

10.4 Expectation Maximization for Gaussian Mixture Models 190

10.5 Neural Expectation Maximization 191

10.6 Relational Neural Expectation Maximization 194

10.7 Variational Filtering Expectation Maximization 196

10.8 Amortized Variational Filtering Expectation Maximization 198

10.9 Applications 199

10.9.1 Stochastic Volatility 199

10.9.2 Physical Reasoning 200

10.9.3 Speech, Music, and Video Modeling 200

10.10 Concluding Remarks 201

11 Reinforcement Learning-Based Filter 203

11.1 Introduction 203

11.2 Reinforcement Learning 204

11.3 Variational Inference as Reinforcement Learning 207

11.4 Application 210

11.4.1 Battery State-of-Charge Estimation 210

11.5 Concluding Remarks 210

12 Nonparametric Bayesian Models 213

12.1 Introduction 213

12.2 Parametric vs Nonparametric Models 213

12.3 Measure-Theoretic Probability 214

12.4 Exchangeability 219

12.5 Kolmogorov Extension Theorem 221

12.6 Extension of Bayesian Models 223

12.7 Conjugacy 224

12.8 Construction of Nonparametric Bayesian Models 226

12.9 Posterior Computability 227

12.10 Algorithmic Sufficiency 228

12.11 Applications 232

12.11.1 Multiple Object Tracking 233

12.11.2 Data-Driven Probabilistic Optimal Power Flow 233

12.11.3 Analyzing Single-Molecule Tracks 233

12.12 Concluding Remarks 233

References 235

Index 253

The New copy of this book will include any supplemental materials advertised. Please check the title of the book to determine if it should include any access cards, study guides, lab manuals, CDs, etc.

The Used, Rental and eBook copies of this book are not guaranteed to include any supplemental materials. Typically, only the book itself is included. This is true even if the title states it includes any access cards, study guides, lab manuals, CDs, etc.