Amit Konar is a Professor of Electronics and Tele-Communication Engineering, Jadavpur University, India, where he offers graduate-level courses on Artificial Intelligence and directs research in Cognitive Science, Robotics and Human-Computer Interfaces. Dr. Konar is the recipient of many prestigious grants and awards and is an author of 10 books and over 350 research publications. He offered consultancy services to Government and private industries. He served editorial services to many journals, including IEEE Transactions on Systems, Man and Cybernetics (Part-A) and IEEE Transactions on Fuzzy Systems.

Aruna Chakraborty is an Associate Professor with the Department of Computer Science and Engineering, St. Thomas' College of Engineering and Technology, India. She is also a Visiting Faculty with Jadavpur University, where she offers graduate-level courses on Intelligent Automation and Robotics, and Cognitive Science. Her research interest includes human-computer interfaces, emotional intelligence and reasoning with fuzzy logic.

Preface xix

Acknowledgments xxvii

Contributors xxix

1 Introduction to Emotion Recognition 1
Amit Konar, Anisha Halder, and Aruna Chakraborty

1.1 Basics of Pattern Recognition, 1

1.2 Emotion Detection as a Pattern Recognition Problem, 2

1.3 Feature Extraction, 3

1.4 Feature Reduction Techniques, 15

1.5 Emotion Classification, 17

1.6 Multimodal Emotion Recognition, 24

1.7 Stimulus Generation for Emotion Arousal, 24

1.8 Validation Techniques, 26

1.9 Summary, 27

References, 28

Author Biographies, 44

2 Exploiting Dynamic Dependencies Among Action Units for Spontaneous Facial Action Recognition 47
Yan Tong and Qiang Ji

2.1 Introduction, 48

2.2 Related Work, 49

2.3 Modeling the Semantic and Dynamic Relationships Among AUs With a DBN, 50

2.4 Experimental Results, 60

2.5 Conclusion, 64

References, 64

Author Biographies, 66

3 Facial Expressions: A Cross-Cultural Study 69
Chandrani Saha, Washef Ahmed, Soma Mitra, Debasis Mazumdar, and Sushmita Mitra

3.1 Introduction, 69

3.2 Extraction of Facial Regions and Ekman’s Action Units, 71

3.3 Cultural Variation in Occurrence of Different AUs, 76

3.4 Classification Performance Considering Cultural Variability, 79

3.5 Conclusion, 84

References, 84

Author Biographies, 86

4 A Subject-Dependent Facial Expression Recognition System 89
Chuan-Yu Chang and Yan-Chiang Huang

4.1 Introduction, 89

4.2 Proposed Method, 91

4.3 Experiment Result, 103

4.4 Conclusion, 109

Acknowledgment, 110

References, 110

Author Biographies, 112

5 Facial Expression Recognition Using Independent Component Features and Hidden Markov Model 113
Md. Zia Uddin and Tae-Seong Kim

5.1 Introduction, 114

5.2 Methodology, 115

5.3 Experimental Results, 123

5.4 Conclusion, 125

Acknowledgments, 125

References, 126

Author Biographies, 127

6 Feature Selection for Facial Expression Based on Rough Set Theory 129
Yong Yang and Guoyin Wang

6.1 Introduction, 129

6.2 Feature Selection for Emotion Recognition Based on Rough Set Theory, 131

6.3 Experiment Results and Discussion, 137

6.4 Conclusion, 143

Acknowledgments, 143

References, 143

Author Biographies, 145

7 Emotion Recognition from Facial Expressions Using Type-2 Fuzzy Sets 147
Anisha Halder, Amit Konar, Aruna Chakraborty, and Atulya K. Nagar

7.1 Introduction, 148

7.2 Preliminaries on Type-2 Fuzzy Sets, 150

7.3 Uncertainty Management in Fuzzy-Space for Emotion Recognition, 152

7.4 Fuzzy Type-2 Membership Evaluation, 157

7.5 Experimental Details, 161

7.6 Performance Analysis, 167

7.7 Conclusion, 175

References, 176

Author Biographies, 180

8 Emotion Recognition from Non-frontal Facial Images 183
Wenming Zheng, Hao Tang, and Thomas S. Huang

8.1 Introduction, 184

8.2 A Brief Review of Automatic Emotional Expression Recognition, 187

8.3 Databases for Non-frontal Facial Emotion Recognition, 191

8.4 Recent Advances of Emotion Recognition from Non-Frontal Facial Images, 196

8.5 Discussions and Conclusions, 205

Acknowledgments, 206

References, 206

Author Biographies, 211

9 Maximum a Posteriori Based Fusion Method for Speech Emotion Recognition 215
Ling Cen, Zhu Liang Yu, and Wee Ser

9.1 Introduction, 216

9.2 Acoustic Feature Extraction for Emotion Recognition, 219

9.3 Proposed Map-Based Fusion Method, 223

9.4 Experiment, 229

9.5 Conclusion, 232

References, 232

Author Biographies, 234

10 Emotion Recognition in Naturalistic Speech and Language—A Survey 237
Felix Weninger, Martin W¨ollmer, and Björn Schuller

10.1 Introduction, 238

10.2 Tasks and Applications, 239

10.3 Implementation and Evaluation, 244

10.4 Challenges, 253

10.5 Conclusion and Outlook, 257

Acknowledgment, 259

References, 259

Author Biographies, 267

11 EEG-Based Emotion Recognition Using Advanced Signal Processing Techniques 269
Panagiotis C. Petrantonakis and Leontios J. Hadjileontiadis

11.1 Introduction, 270

11.2 Brain Activity and Emotions, 271

11.3 EEG-ER Systems: An Overview, 272

11.4 Emotion Elicitation, 273

11.5 Advanced Signal Processing in EEG-ER, 275

11.6 Concluding Remarks and Future Directions, 287

References, 289

Author Biographies, 292

12 Frequency Band Localization on Multiple Physiological Signals for Human Emotion Classification Using DWT 295
M. Murugappan

12.1 Introduction, 296

12.2 Related Work, 297

12.3 Research Methodology, 299

12.4 Experimental Results and Discussions, 306

12.5 Conclusion, 310

12.6 Future Work, 310

Acknowledgments, 310

References, 310

Author Biography, 312

13 Toward Affective Brain–Computer Interface: Fundamentals and Analysis of EEG-Based Emotion Classification 315
Yuan-Pin Lin, Tzyy-Ping Jung, Yijun Wang, and Julie Onton

13.1 Introduction, 316

13.2 Materials and Methods, 323

13.3 Results and Discussion, 327

13.4 Conclusion, 332

13.5 Issues and Challenges Toward ABCIs, 332

Acknowledgments, 336

References, 336

Author Biographies, 340

14 Bodily Expression for Automatic Affect Recognition 343
Hatice Gunes, Caifeng Shan, Shizhi Chen, and YingLi Tian

14.1 Introduction, 344

14.2 Background and Related Work, 345

14.3 Creating a Database of Facial and Bodily Expressions: The FABO Database, 353

14.4 Automatic Recognition of Affect from Bodily Expressions, 356

14.5 Automatic Recognition of Bodily Expression Temporal Dynamics, 361

14.6 Discussion and Outlook, 367

14.7 Conclusions, 369

Acknowledgments, 370

References, 370

Author Biographies, 375

15 Building a Robust System for Multimodal Emotion Recognition 379
Johannes Wagner, Florian Lingenfelser, and Elisabeth André

15.1 Introduction, 380

15.2 Related Work, 381

15.3 The Callas Expressivity Corpus, 382

15.4 Methodology, 386

15.5 Multisensor Data Fusion, 390

15.6 Experiments, 395

15.7 Online Recognition System, 399

15.8 Conclusion, 403

Acknowledgment, 404

References, 404

Author Biographies, 410

16 Semantic Audiovisual Data Fusion for Automatic Emotion Recognition 411
Dragos Datcu and Leon J. M. Rothkrantz

16.1 Introduction, 412

16.2 Related Work, 413

16.3 Data Set Preparation, 416

16.4 Architecture, 418

16.5 Results, 431

16.6 Conclusion, 432

References, 432

Author Biographies, 434

17 A Multilevel Fusion Approach for Audiovisual Emotion Recognition 437
Girija Chetty, Michael Wagner, and Roland Goecke

17.1 Introduction, 437

17.2 Motivation and Background, 438

17.3 Facial Expression Quantification, 440

17.4 Experiment Design, 444

17.5 Experimental Results and Discussion, 450

17.6 Conclusion, 456

References, 456

Author Biographies, 459

18 From a Discrete Perspective of Emotions to Continuous, Dynamic, and Multimodal Affect Sensing 461
Isabelle Hupont, Sergio Ballano, Eva Cerezo, and Sandra Baldassarri

18.1 Introduction, 462

18.2 A Novel Method for Discrete Emotional Classification of Facial Images, 465

18.3 A 2D Emotional Space for Continuous and Dynamic Facial Affect Sensing, 469

18.4 Expansion to Multimodal Affect Sensing, 474

18.5 Building Tools That Care, 479

18.6 Concluding Remarks and Future Work, 486

Acknowledgments, 488

References, 488

Author Biographies, 491

19 Audiovisual Emotion Recognition Using Semi-Coupled Hidden Markov Model with State-Based Alignment Strategy 493
Chung-Hsien Wu, Jen-Chun Lin, and Wen-Li Wei

19.1 Introduction, 494

19.2 Feature Extraction, 495

19.3 Semi-Coupled Hidden Markov Model, 500

19.4 Experiments, 504

19.5 Conclusion, 508

References, 509

Author Biographies, 512

20 Emotion Recognition in Car Industry 515
Christos D. Katsis, George Rigas, Yorgos Goletsis, and Dimitrios I. Fotiadis

20.1 Introduction, 516

20.2 An Overview of Application for the Car Industry, 517

20.3 Modality-Based Categorization, 517

20.4 Emotion-Based Categorization, 520

20.5 Two Exemplar Cases, 523

20.6 Open Issues and Future Steps, 536

20.7 Conclusion, 537

References, 537

Author Biographies, 543

Index 545

The New copy of this book will include any supplemental materials advertised. Please check the title of the book to determine if it should include any access cards, study guides, lab manuals, CDs, etc.

The Used, Rental and eBook copies of this book are not guaranteed to include any supplemental materials. Typically, only the book itself is included. This is true even if the title states it includes any access cards, study guides, lab manuals, CDs, etc.