Video-based Facial Micro-expression Analysis: Datasets, Features, and Algorithm Research

Key Chapter Title List

1. Introduction
2. Differences Between Macro-expressions and Micro-expressions
3. Micro-expression Datasets
4. Micro-expression Features
5. Detection Algorithms
6. Recognition Algorithms
7. Application Scenarios
8. Method Comparison
9. Future Research Directions
10. Conclusion

Document Introduction

Facial micro-expressions, as involuntary and instantaneous facial movements, can reveal the genuine emotions that humans attempt to conceal, holding significant application value in fields such as lie detection and criminal investigation. Unlike easily recognizable macro-expressions, micro-expressions last only 0.065-0.5 seconds, have weak intensity, and are not subject to voluntary control. Their automatic detection and recognition face numerous technical challenges, making traditional macro-expression analysis methods difficult to adapt directly.

This study first clarifies the fundamental differences between macro-expressions and micro-expressions from a neuropsychological perspective, pointing out that they are regulated by the cortical pathway (voluntary control) and the subcortical pathway (involuntary triggering) respectively, exhibiting significant distinctions in duration, action intensity, and involved facial regions. Based on these differences, the study constructs a hierarchical analysis framework that systematically covers seven core modules: neuropsychological basis, datasets, feature extraction, detection algorithms, recognition algorithms, application scenarios, and evaluation systems.

Regarding datasets, the study comprehensively reviews the advantages and disadvantages of nine representative datasets, including MEVIEW, SMIC, the CASME series, and SAMM, pointing out limitations such as small sample sizes, limited emotion categories, and a lack of correspondence between macro- and micro-expressions. To address this, the study proposes and publicly releases a new dataset, MMEW (Micro-and-Macro Expression Warehouse). This dataset contains more video samples, richer emotion label types, and simultaneously provides macro-expression and micro-expression data from the same subjects, laying a foundation for cross-modal research.

At the technical method level, the study systematically classifies and evaluates key technologies for micro-expression analysis: feature extraction covers four major categories—spatiotemporal domain, frequency domain, tensor decomposition, and optical flow features; detection algorithms are divided into optical flow methods and feature descriptor methods; recognition algorithms include traditional classification, deep learning, and transfer learning methods. Through unified evaluation on the CAS(ME)², SAMM, and MMEW datasets, the study finds that the MDMD algorithm performs best in micro-expression detection, while the TLCNN deep learning model achieves the highest accuracy in recognition tasks (69.4% on the MMEW dataset, 73.5% on the SAMM dataset).

The study also verifies that using macro-expression data from the same subject for pre-training can significantly improve micro-expression recognition performance, providing an effective path to address the core issue of insufficient sample sizes in micro-expression datasets. Furthermore, the study clarifies the application value of micro-expression analysis in fields like lie detection, noting that it can serve as an important supplement to traditional physiological detection methods, enhancing the accuracy of deception recognition.

This study provides the first comprehensive and systematic review framework, a unified evaluation benchmark, and a novel dataset for the field of micro-expression analysis. It not only integrates the research achievements in this field but also points out future research directions such as privacy protection, standardized dataset construction, and the development of interpretable algorithms. It holds significant reference value for promoting the academic research and practical application of related technologies.