The Classification of Emotion and Empirical Mode Decomposition (emd).

Abstract

It was aimed to classify emotion for feature extraction in discrete approach method and emotion recognition based on empirical mode decomposition (EMD).By using EMD, EEG signals are decomposed into Intrinsic Mode Functions (IMFs) automatically. Multidimensional information of IMF is utilized as features, the first difference of time series, the first difference of phase, and the normalized energy. These three features are effective for emotion recognition .The role of each IMF is inquired and we find that high frequency component IMF1 has significant effect on different emotional states detection. In addition, the classification accuracy of the proposed method uses several classical techniques, including Support vector machines, Quadratic classifiers ,k-nearest neighbour, Neural networks. Experiment results demonstrate that our method can improve emotion recognition performance.

Introduction

Emotion plays an important role in our daily life and work. Real-time assessment and regulation of emotion will improve people’s life and make it better. For ex, in communication of human-machine interaction, emotion recognition will make the process more easy and natural. Another example, in the treatment of patients, especially those with expression problems, the real emotion state of patients will help doctors to provide more appropriate medical care. In recent years, emotion recognition from EEG has gained mass attention. Also it is a very important factor in brain computer interface (BCI) systems, which will effectively improve the communication between human and machines [1].

Various features and extraction methods have been pro-posed for emotion recognition from EEG signals, including time domain techniques, frequency domain techniques, joint time-frequency analysis techniques, and other strategies. Statistics of EEG series, that is, first and second difference, mean value, and power are usually used in time domain [2]. Nonlinear features, including fractal dimension (FD) [3, 4],sample entropy [5], and non-stationary index [6], are utilized for emotion recognition. Petrantonakis and tiadis Hadjileon were introduced higher order crossings (HOC) features to capture the oscillatory pattern of EEG [10]. Wang et al. extracted frequency domain features for classification[11].Time-frequency analysis is based on the spectrum of EEG signals; then the energy, power, power spectral density (PSD), and differential entropy [12] of certain sub band are usually utilized as features.

Short-time Fourier transform (STFT) [13, 14], Hilbert-Huang transform (HHT) [15, 16], and discrete wavelet transform (DWT) [17–19] are the most commonly used techniques for spectrum calculating. It has been commonly tested and verified that higher frequency sub band such as Beta (16–32 Hz) and Gamma (32–64 Hz) bands outperforms lower sub band for emotion recognition [20, 21].Other features extracted from combination of electrode are utilized too, such as coherence and asymmetry of electrodes in different brain regions [22–24] and graph-theoretic features [25]. Jenke et al. had done a research comparing the performance of different features mentioned above and got a guiding rule for feature extraction and selection [26].Some other strategies such as utilizing deep network to improve the classification performance have also been researched. Yang et al. used hierarchical network with sub network nodes for emotion recognition [28].

EMD is proposed by Huang et al. in 1998 [29]. Unlike DWT, which needs to predetermine transform base function and decomposition level, EMD can decompose signals into IMF automatically. These IMFs represent different frequency components of original signals, with band-limited character-istic. By applying Hilbert transform to IMF, we can get instantaneous phase information of IMF. So EMD is suitable for analysis of nonlinear and non-stationary sequence, such as neural signals.

EMD has been widely used for seizure prediction and detection, but for emotion recognition based on EMD, there is not so much research. Higher order statistics of IMFs [30], geometrical properties of the decomposed IMF in complex plane [31], and the variation and fluctuation of IMF [32] are used as features for seizure prediction and detection. For emotion recognition, Mert and Akan extracted entropy, power, power spectral density, correlation, and asymmetry of IMF as features and then utilized independent component analysis (ICA) to reduce dimension of the feature set [33]. The classification accuracy is computed with all the subjects mixed together. In this paper, we present an emotion recognition method based on EMD.

We utilize the first difference of IMFtime-series,the first difference of the IMF’s phase, and the normalized energy of IMF as features. The motivation of using these three features is that they depict the characteristics of IMF in time, frequency, and energy domain, providing multidimensional information. The first difference of time series depicts the intensity of signal change in time domain.

The first difference of phase measures the change intensity in phase and normalized energy describes the weight of current oscillation component. The three features constitute a feature vector, which is fed into SVM classifier for emotional state detection. The proposed method is studied on a publicly available emotional database DEAP [20]. The effectiveness of the three features is investigated. IMF reduction and channel reduction for feature extraction are both discussed, which aim at improving the classification accuracy with less computation complexity. The performance is compared with some other techniques, including fractal dimension (FD), sample entropy, differential entropy, and time-frequency analysis DWT.

Methodology

To realize emotional state recognition, the EEG signals are decomposed into IMFs by EMD. Three features of IMFs, the fluctuation of the phase, the fluctuation of the time series, and the normalized energy, are formed as a feature vector, which is fed into SVM for classification. The whole process of the algorithm is shown in Figure 1.

Data and Materials

DEAP is a publicly available dataset for emotion analysis, which recorded EEG and peripheral physiological signals of 32 participants as they watched 40 music videos. All the music video clips last for 1 minute, representing different emotion visual stimuli, with grade from 1 to 9. Among the 40 music videos, 20 are high valence visual stimuli and 20 are low valence visual stimuli. The situation is exactly the same for arousal dimension. After watching the music video, participants performed a self-assessment of their levels on arousal, valence, liking, dominance, and familiarity, with ratings from 1 to 9. EEG was recorded with 32 electrodes, placing according to the international 10-20 system. Each electrode recorded 63 s EEG signal, with 3 s baseline signal before the trial. In this ,we used pre processed EEG data for study, with sample rate 128Hz and band range 4–45Hz. EOG artefacts were removed as method in [20].The data was segmented into 60-second trials and a 3-second pre trial baseline removed. The binary classifications of valence and arousal dimension are considered. We utilized the EEG signals are extracted as a sample. So for each subject who watched 40 music videos, we acquire 480 labeled samples. Each music video lasts for 1 minute, and 5 s EEG signals are extracted as a sample. So for each subject who watched 40 music videos, we acquire 480 labeled samples. a

Empirical Mode Decomposition

EMD decomposes EEG signals into a set of IMFs by an automatic shifting process. Each IMF represents different frequency components of original signals and should satisfy two conditions: (1)during the whole data set, the number of extreme points and the number of zero crossings must be either equal or differ at most by one; (2)at each point, the mean value calculated from the upper and lower envelope must be zero [29]. For input signal x(t ), the process of EMD is as follows: (1) Set ℎ(t ) = x(t )and ℎold(t ) = ℎ(t ). (2) Get local maximum and minimum of ℎold(t ). (3) Interpolate the local maximum and minimum with cubic spline function and get the upper envelope max(t )and lower envelope min(t ). (4) Calculate the mean value of the upper and lower envelope as. It is a linear combination of IMF components and the residual part . Figure 2 shows a segment of original EEG signals corresponding to the first five decomposed IMFs. EMD works like an adaptive high pass filter. It shifts out the fastest changing component first and as the level of IMF increases, the oscillation of IMF becomes smoother. Each component is band-limited, which can reflect the characteristic of instantaneous frequency.

Feature Extraction

In this paper, three features of IMF are utilized for emotion recognition, the first difference of time series, the first difference of phase, and the normalized energy. The first difference of time series depicts the intensity of signal change in time domain. The first difference of phase reveals the change intensity of phase, representing the physical meaning of instantaneous frequency. Normalized energy describes the weight of current oscillation component. The motivation of using these three features is that they depict the characteristics of IMF in time, frequency, and energy domain, utilizing multidimensional information.

First Difference of IMF Time Series: The first difference of times series depicts the intensity of signal change in time domain. Previous research has revealed that the variation of EEG time series can reflect different emotion states [2]. For an IMF component with points, IMF{imf1,imf2,...,imfn}, the definition of Dt

First Difference of IMF’s Phase: Based on EMD, EEG is decomposed into multilevel IMFs, each IMF being band limited and representing an oscillation component of original EEG signals. For an????-point IMF,IMF{imf1, imf2, . . . , imf????},Hilbert transform is applied to it, obtaining an analytic signal ????(????) as The analytic signal can be further expressed as follows

Normalized Energy of IMF: For an ????-point IMF, IMF{imf1, imf2, . . . , imf????}, the normalized energy ????norm is defined as follows:where ????(????) is the original EEG signal points. So the numerator is the energy of IMF and the denominator represents the energy of original EEG data set. The normalized energy describes the weight of current oscillation component. When fed into the classifier, log(????norm) is taken as an element of the feature vector according to [26].

SVM Classifier: The extracted features are fed into SVM for classification. SVM is widely used for emotion recognition [34, 35], which has promising property in many fields. In our study, SVM is implemented for SVM classifier with radial basis kernel function and default parameters setting [36].

Conclusion

In this paper, an emotion recognition method based on EMD using three statistics is proposed. Extensive analysis has been carried out to investigate the effectiveness of the features for emotion classification. The results show that the three features are suitable for emotion recognition .Then the effect of each IMF component is inquired. The results reveal that, among the multilevel IMFs, the first component IMF1 plays the most important role in emotion recognition. Also the informative vector based on EMD strategy are investigated and selected for feature extraction. Finally, the proposed method is yields the highest accuracy.