Sentiment Analysis of Lms Users Using Support Vector Algorithm

The researchers used qualitative research method to collect and analyse data needed about the perception, expectation and concern in LMS. The questionnaire used was formulated and validated by a social science researcher which is composed of two (2) open-ended questions. These questions were given directly to get the sentiments of both students and faculty regarding the use of LMS as an educational tool. The data gathered by the researchers has a total of 1,321 responses for perception, concerns and expectations of the user regarding education uses of LMS. The method structure of this study is composed of data collection, data cleaning, data annotation through manual classification of perception into positive or negative, and machine learning of the training set which is shown in figure 1. The researchers manually classified the collected responses into negative perception or positive perception. To test the performance evaluation of the classification model, the researchers used 10 stratified cross-validation with a support vector algorithm that is applied in machine learning tools. One of today’s most rapidly growing technical fields is machine learning that lying at the intersection of computer science and statistics and at the core of data science where in this study, it is used to validate the performace of the data model then expectation and concern is categorized through topic modelling [10].

Data Collection and Data Source

Survey questions was posted online through Google Docs/Sheets. Researchers gave the students and faculty the opportunity to respond to two (2) open-ended questions which are: What are your perceptions as in using LMS as tool in instructional delivery; and What are your expectations and concerns in using Learning Management System as tool in learning. Respondents’ data and their responses was saved in spreadsheet application for data cleaning process.

 Data Cleaning

Since the data collected is noisy, the researchers undergo with the process of data cleaning in the form of removing duplicates, symbols, numbers and uppercase words. To remove duplicates, the machine learning tools were used is Waikato Environment for Knowledge Analysis (WEKA). Notepad ++ was used to process symbols, uppercase, lowercase through regular expressions. After data cleaning only 770 responses on perception was retained.

Annotation of Data

The researchers manually classified the collected perception responses data into positive or negative perceptions, while the expectations and concerns will be determined through topic modelling using mallet tool. Negative responses classified by researcher if the response has negative auxiliary words such as “do not”, “not”, etc. Table 1 show the example of cleaned classified positive and negative response:

• Positive Response Negative Response
• helpful tool many topics discussed achieve depth knowledge not good instructional tool
• allows seamless deliverance information students guiding faculty nobody explains lessons read only learn alone
• manageable easy not useful
• another innovative way delivering lessons not applicable mathematics engineering subjects
• user friendly easy manage accurate effective not like cannot absorb lessons given online

Machine Learning Tools and Algorithms for Classification

For identifying and evaluating the manually classified and labelled perceptions of the researcher, the Waikato Environment for Knowledge Analysis (Weka) was used in this study as machine learning tools. In training machine learning tools for classifying positive and negative perceptions the supervised learning was used. Supervised learning means that the researchers already manually classified and labelled the perceptions into postive perceptions and negative perceptions to train the machine learning tools. To test the performance of the classification model of the perceptions, the test options used by the researcher is the stratified cross-validation with ten (10) number of folds. The study titled Automatic Classification of Disaster-Related Tweet stated that The process of the machine learning tools in stratified cross-validation with ten (10) number of folds is the data set is randomly split into 10 equal sizes of subsets, the manually classification model is trained is trained and tested 10 times, with the 9-folds used as the training data set and the remaining 1-fold as the testing data set [11].

In this study algorithms Support Vector Machine (SVM) classifiers were used. One of the most commonly used machine learning algorithms for classification is the support vector machine that is used for binary classification. Finding hyperlane that optimally separates the d-dimensional data into two classes is the main objective of SVM [12]. However, to make data more easily to separate, the SVM integrates the concepts of kernel that mades feature space that makes the data into a higher dimensional space [13].

After classification and labelling of the perception responses, the researcher measure the reliability of classifying and labelling of perceptions using the Intraclass Correlation Coefficient (ICC) or multi-rater Kappa coefficient that is automatically computed by machine learning tools. The intraclass correlation coefficient is the best measure for reliability for nonstop data [14]. The standard metrics such as accuracy, precission, recall and f-measure was used to test the training set performance evaluation which is shown in Table 2.