Real Time Facial Feature Extraction and Emotion Recognition

Abstract—Facial emotion recognition (FER) is an important topic in the field of computer vision. Deeds, postures, facial expressions and language; these are considered as channels that pass on human feelings. Broad research has being done to investigate the connections between these channels and feelings. This paper proposes a framework which consequently perceives the feeling represented on human face. Neural network results are combined along with image processing results that are used for organizing common emotions: Joy, ambiguous. Shaded front face pictures are given as contribution to the framework. After the face is distinguished, picture handling based component point extraction strategy is utilized to separate a set of chose highlight focuses. At long last, an arrangement of qualities acquired in the wake of preparing those separated component indicates are given as information the neural system to perceive the feeling contained.

Index terms: Emotions, Feature Extraction, Neural Network, Emotion recognition, FER.

INTRODUCTION

Feelings are human behavioural emotions that play a day by day part in our everyday life in activities, for example, basic leadership, learning, inspiration, thinking, mindfulness, arranging and some more. People can identify faces and decipher passionate articulations on them with no genuine trouble. Thinking about the quick expanding enthusiasm for feeling acknowledgment utilizing applications, if robotized frameworks effectively perceive human feeling, it opens the entryways for robotized investigation of human full of feeling conduct and pulls in the consideration of specialists from different trains, for example, brain science, semantics, software engineering, and other related orders what's more, opens up a major research field.

PROBLEM STATEMENT

Human emotions and intentions are expressed via facial expressions and deriving a powerful function is the essential aspect of facial expression system. Automated recognition of facial expressions may be an important thing of natural human-machine interfaces; it may additionally also be used in behavioural technology and in medical exercise. An automated facial expression popularity machine desires to resolve the subsequent issues: detection and region of faces in a cluttered scene, facial characteristic extraction, and facial features class.

PROPOSED SYSTEM

In this work, frameworks which will productively perceive the comprehensive feeling of joy and ambiguous from 2D coloured face pictures. The work has being constrained to the general feelings since order and identification of other surface feelings is tricky. The framework can be widely well-organized in to three phases (Fig.1):

• Face detection
• Feature Extraction
• Facial Expression Classification

Two face identification calculations are executed for the face area assurance arrange. Eyes, mouth and eyebrows are distinguished as the basic highlights and their component indicates are extracted identify the feeling. These component focuses are separated from the chosen include areas with the utilization of a corner point discovery calculation. After element extraction is played out a neural system approach is utilized to perceive the feelings contained inside the face.

IMPLEMENTATION METHODOLOGY

The overall system has been developed using ASP.NET with C#. The flow chart of the system modules is shows in Fig.2

Modules Description

Skin Colour Segmentation:

If the biggest connected region has the probability to become a face, then it'll open a replacement type with the biggest connected region. If the biggest connected regions height is larger or equal than fifty and also the magnitude relation of height/width is between one to a pair of, then it should be face. Then we've got to examine the likelihood to become a face of the biggest connected region. For color segmentation, initial we tend to distinction the image. Then, we've got to search out the biggest connected region. Then we tend to perform color segmentation (Fig.3).

Face Detection:

For face detection, initial we tend to convert binary image from RGB image. Then, we tend to attempt to notice the forehead from the binary image. We tend to begin scan from the centre of the image, then wish to search out an eternal white elements once an eternal black pixel.

Then we would like to search out the utmost dimension of the white element by looking vertical each left and right side. Then, if the new dimension is smaller half the previous most dimensions, then we tend to break the scan as a result of if we tend to reach the brow then this example can arise. Then we tend to cut the face from the beginning position of the forehead and its high are going to be 1.5 multiply of its dimension.

Then we'll have a picture which is able to contain solely eyes, nose and lip. Then we'll cut the RGB image in line with the binary image. (Fig.5)

Eyes Detection:

For eyes detection, we tend to convert the RGB face to the binary face. Now, we tend to contemplate the face dimension by W. we tend to scan from the W/4 to (W-W/4) to search out the center position of the 2eyes. the very best white continuous element on the peak between the ranges is that the middle position of the 2 eyes.

Then we discover the beginning high or higher position of the 2 eyebrows by looking vertical. For left eye, we tend to search w/8 to middle and for right eye we tend to search middle to w – w/8. Here w is that the dimension of the image and middle is that the middle position of the 2 eyes. There is also some white pixels between the brow and also the eye. to form the brow and eye connected, we tend to place some continuous black pixels vertically from brow to the attention. For left eye, the vertical black pixel-lines are placed in between mid/2 to mid/4 and for right eye the lines are in between mid+(w-mid)/ four to mid+3*(w-mid)/ four and height of the black pixel-lines are from the brow beginning height to (h- brow beginning position)/4. Here w is that the dimension of the image and middle is that the middle position of the 2 eyes and h is that the height of the image. Then we discover the lower position of the 2 eyes by looking black element vertically. For left eye, we tend to search from the middle/4 to mid - mid/4 dimension. And for right eye, we tend to search middle + (w-mid)/ four to mid+3*(w- mid)/ four dimension from image lower finish to beginning position of the brow.

Then we discover the correct facet of the left eye by looking black element horizontally from the middle position to the beginning position of black pixels in between the higher position and lower position of the left eye. And left facet for right eye we tend to search middle to the beginning position of black pixels in between the higher position and lower position of right eye. The left facet of the left eye is that the beginning dimension of the image and also the right facet of the correct eye is that the ending dimension of the image. Then we tend to cut the higher position, lower position, left facet and also the right facet of the 2 eyes from the RGB image.

Lip Detection:

For lip detection, we tend to verify the lip box. and that we contemplate that lip should be within the lip box. So, initial we tend to verify the space between the forehead and eyes. Then we tend to add the space with the lower height of the attention to see the higher height of the box which is able to contain the lip. Now, the purpose {start line/place to begin} of the box are going to be the ¼ position of the left eye box and ending point are going to be the ¾ position of the correct eye box. and also the finishing height of the box are going to be the lower end of the face image.

So, this box can contain solely lip and will some a part of the nose. Then we'll cut the RGB image according the box.

So, for detection eyes and lip, we tend to solely got to convert binary image from RGB image and a few looking among the binary image.

Apply Bezier Curve on Lip:

In the lip box, there's lip and will be some a part of nose. So, round the box there's colour or the skin. So, we tend to convert the skin element to white element and different element as black. We tend to conjointly notice those elements that are the same as skin pixels and convert them to white pixel. Here, if 2pixels RGB values distinction is a smaller amount than or equal ten, then we tend to known as them similar element. Here, we tend to use bar chart for locating the space between the lower average RGB worth and better average RGB worth. If the space is a smaller amount than seventy, then we tend to use seven for locating similar element and if the space is better than or equal seventy then we tend to use ten for locating similar element. So, the worth for locating similar element depends on the standard of the image. If the image quality is high, we tend to use seven for locating similar element and if the image quality is low, we use 10.

So, within the binary image, there ar black regions on lip, nose and will another very little half that have a bit totally different than color. Then we tend to apply huge connected region for locating the black region that contain lip in binary image. and that we ar certain that the massive connected region is that the lip as a result of within the lip box, lip is that the largest issue that is totally different than skin.

Then we've got to use Bezier curve on the binary lip. For apply Bezier curve, we discover the beginning and ending element of the lip in horizontal. Then we tend to draw 2 tangents on higher lip from the beginning and ending element and conjointly notice 2 points on the tangent that isn't the a part of the lip. For the lower lip, we discover 2 purpose similar method of the higher lip. we tend to use cubic Bezier curves for draw the Bezier curve of the lip. we tend to draw 2 Bezier curve for the lip, one for higher lip and one for lower lip.

Apply Bezier Curve on Eye:

For apply Bezier curve on eyes, initial we've got to get rid of brow from eye. For take away brow, we tend to search first continuous black element then continuous white element then continuous black element from the binary image of the attention box. Then we tend to take away the first continuous black element from the box then we tend to get the box that solely contains the attention.

Now, the attention box that contains solely eye, has some skin or colour round the box. So, we tend to apply similar colour just like the lip for locating the region of eye. Then we tend to apply huge connect for locating the very best connected region and this can be the attention as a result of within the eye box, eye is that the biggest issue that isn't the same as the colour.

Then we tend to apply the Bezier curve on the attention box, the same as the lip. Then we tend to get the form of the attention.

Database and coaching

In our info, there are 2 tables. One table “Person” is for storing the name of individuals and their index of four varieties of feeling that are hold on in different table “Position”. within the “Position” table, for every index, there are half-dozen management points for lip Bezier curve, half-dozen management points for left eye Bezier curve, half-dozen management points for right eye Bezier curve, lip height and dimension, left eye height and dimension and right eye height and dimension. So, by this technique, the program learns the feeling of the folks.

Emotion Detection:

For feeling detection of a picture, we've got to search out the Bezier curve of the lip, left eye and right eye. Then we tend to convert every dimension of the Bezier curve to one hundred and height in line with its dimension. If the person’s feeling data is offered within the info, then the program can match that feeling’s height is nearest this height and also the program can offer the closest emotion as output.

If the person’s feeling data isn't offered within the info, then the program calculates the typical height for every feeling within the info for all folks then gets a call in line with the typical height.

CONCLUSIONS

The facial expressions are very vital to determine human emotion. Thus we established a system which is capable of determining the human emotion through facial expression. The overall system has been developed using ASP.NET with C# and uses the neural network method. The paper is basically focusing on fixed images which can be stored in the database, and further analysis can be done through the system. The further research can be done on video based image extraction and the algorithms can be developed using the current algorithm as a source referred as genetic algorithm.