1. Speech Emotion
Recognition
Under the Supervision of :
Dr. Sanjai Singh

2. 1. Satya Prakash kumawat – IEC2020023
2. Rohit Kumar Meena - IEC2020076
3. Arihant Mishra -IEC2020032
Group Members: -

3. OBJECTIVE: -
Developing Machine Learning Models that can predict emotion of given speech.

4. Project Setup:
● Dataset Used – TESS
● Technologies Used – Python, Google Colabs, TensorFlow
● Classified Emotions – Happy, Sad, Angry, disgusted, Pleasant Surprise, Neutral, and
Fear.

5. TESS (Toronto emotional speech set ) DATA set:
There are a set of 200 target words were spoken in the carrier phrase "Say the word _' by two
actresses (aged 26 and 64 years) and recordings were made of the set portraying each of seven
emotions (anger, disgust, fear, happiness, pleasant surprise, sadness, and neutral). There are 2800
data points (audio files) in total.
Data set link : https://drive.google.com/drive/u/0/folders/1i4r4NBJDzXuP8yAz3MC4VapUz4nIcwA7

6. Problem Gap
● As human beings speech is amongst the most natural way to express ourselves, We
depend so much on it that we recognize its importance when resorting to other
communication forms like emails and text messages where we often use emojis to
express the emotions associated with the message. As emotions play a vital role in
communication the detection and analysis of the same are of vital importance in today’s
digital world of remote communication.
● Emotion detection is a challenging task because emotions are subjective. There is no
common consensus on how to measure them. We define a speech Emotion Recognition
system as a collection of methodologies that process and classify speech signals to
detect emotion embedded in them.

7. Approach Used

8. Feature Extraction : -
1. Spectrogram
2 . MFCC (Mel-frequency cepstral coefficients )

9. Waveplots

10. Spectrogram

11. MFCC

12. SVM Classifier
A supervised machine learning model called a
support vector machine (SVM) employs
categorisation techniques to solve two-group
detection problems. An SVM model can classify
new text after being given sets of labeled
training data for each category.

13. WORKING OF SVM
An SVM model is basically a representation of different
classes in a hyperplane in multidimensional space. The
hyperplane will be generated in an iterative manner by SVM
so that the error can be minimized. The goal of SVM is to
divide the datasets into classes to find a maximum marginal
hyperplane (MMH).

14. KNN Classifier
Among the basic machine learning
algorithms, depending on the supervised
learning method, is K-Nearest Neighbor.
Since K-NN is a non-parametric technique,
it makes no assumptions about the base
data. It is also known as a lazy learner
algorithm since it saves the training dataset
rather than learning from it instantaneously.

15. How does K-NN work?
The K-NN working can be explained on the basis of the below algorithm:
● Step-1: Select the number K of the neighbors
● Step-2: Calculate the Euclidean distance of K number of neighbors
● Step-3: Take the K nearest neighbors as per the calculated Euclidean
distance.
● Step-4: Among these k neighbors, count the number of the data points
in each category.
● Step-5: Assign the new data points to that category for which the
number of the neighbor is maximum.
● Step-6: Our model is ready.

16. Deep Learning method used :
• LSTM ( Long short-term memory) :

17. LSTM :
• Conv2D layer followed by Batch Normalization which is followed by Relu activation.
• After that a 2D max pooling layer is applied which is again followed by Conv2D, Batch
Normalization and Relu activation
• After that 2D max pooling layer is used. Finally data is reshaped and input to LSTM
layer is given.
• Finally a Dense layer with softmax activation is used.
•

18. Validation Accuracy Vs Train accuracy :

19. Validation Loss vs Train Loss:

20. Results of LSTM :
•Using MFCC -
•Accuracy >= 96%
•
Using Spectrogram –
•Accuracy >= 93%

21. Final Results
LSTM gives the best result with Spectogram features

22. References
● https://www.researchgate.net/publication/357225911_VGG16_VGQR
● https://ieeexplore.ieee.org/document/7493959
● https://ieeexplore.ieee.org/document/8079924
● https://www.researchgate.net/publication/236952762_Random_Forests
● https://ieeexplore.ieee.org/abstract/document/8805181