Psychometric Analysis of The Indonesian Version of The Online Learning Readiness Scale Using Generalized Partial Credit Model

Though there are measures of student online learning readiness published in the behavioral science and educational literature and journal, very few scales are available to measure student online learning readiness, specifically in Indonesia. The purpose of this study was to validate an instrument for high-school students’ readiness for online learning. The Online Learning Readiness Scale aimed to measure readiness of online learning among Indonesian students. Data from 271 senior high-school students (male = 126, female = 145) in Yogyakarta, Indonesia were subjected to an Item Response Theory (IRT) analysis and the psychometric properties of the scale were examined using Generalized Partial Credit Model (GPCM). Results from IRT analysis using GPCM found excellent psychometrics characteristics of the Indonesian version of this scale. Evidence of construct validity is presented. Scores on the student online learning readiness scale were very reliable. Implications for future research are discussed.

Introduction

The Republic of Indonesia is a country that has a very broad and strategic geographical area which is flanked by two oceans and two continents where the conditions of the majority area are oceans and islands which number approximately 17,000 islands. This condition has caused various obstacles in national development including road infrastructure development, education facilities, health facilities and internet networks that have not been evenly distributed from Sabang (far-west) to Merauke (far-east). Development of human resources in this case is the world of education is an important part and cannot be released just like that. The demands of the advancement of the times and technology encourage the government and all parties involved directly or indirectly to do synergy and adaptation and innovation in the face of this millennia era.

In the digital era like today, the use of technology has dominated the world of education. The development of the education world today in Indonesia is required to be able to make changes and innovations in order to catch up with other countries. The first national exam which still uses paper or paper based will be replaced with an online or online based national examination system. This can be proven when Indonesia began organizing computer-based National Exams. The use of computers is like providing an opportunity for ease in administering exams. The intended facilities include administrative ones such as no manual correction, collection of answer sheets manually, slow delivery times to Jakarta where the UN is corrected and the most important is from the Psychometric side where this can be the starting point for administering the exam with Computerize Adaptive Testing (CAT).

The application of online-based national examinations in all regions in Indonesia will certainly face many challenges. One of the challenges is the readiness of students to face the exam. Learning readiness are studied in so many research, none of them in Indonesia. They define readiness for online learning in three aspects: (1) student preference for forms of delivery as face-to-face classroom instruction; (2) students' trust in using electronic communication for learning and, in particular, competence and confidence in Internet use and computer mediated communication; and (3) the ability to be autonomously involved in learning, Although there have been studies focused on development of the student online readiness assessment tools, they seem to have ignored an important detail about the psychometric quality of these instruments .

The participants of this study were high school students with total 271 students, 126 women and 145 male students with an age range of 15-18 years who were selected with nonprobability sampling technique. The consideration in using the sampling technique is due to the limitations in terms of time to create a sampling frame that contains data about active students who are actively going to school, so that the sampling technique that allows to be used is nonprobability. Other characteristics that are taken into consideration in determining research participants are high school students who will face computer-based exams or online. The willingness of respondents to participate in this research is voluntary.

Generalized Partial Credit Model (GPCM) IRT is a powerful modeling approach used to evaluate the psychometric properties of survey questionnaires with categorical (ordered and unordered) responses. IRT models are similar to factor analytic models in that they both provide information about dimensionality and model fit. A key difference between IRT and factor analytic approaches is how the data are treated. While factor analytic methods examine covariances between the individual items, IRT models examine the overall response patterns across all of the items. As a consequence of evaluating item response patterns, the parameter estimates obtained provide insight into how the items function. This type of information can be particularly useful during the process of developing a survey. In addition, factor analytic approaches construct a linear relationship between the factor score and item response. This contrasts the IRT approach, which constructs a nonlinear relationship between latent traits and item responses. Likert scale format it is an extension of the dichotomous response format and is used when more information may be obtained than from a dichotomous format. In elementary applications, the categories in the intended order are scored with successive integers, beginning with zero (0), and the scores of a person on multiple items of a test or questionnaire summed to characterize a person. In more-advanced applications, a probabilistic model, such as the GPCM is applied.

A major advantage of the use of polytomously scored items is that their general coverage of a wider range of the ability scale with a sufficient information relative to an assessment with the same number of dichotomous items. For polytomous response models, the parameter values should be interpreted with the aid of graphical presentations. Inspecting the plots of ICRFs, IRF, and item information functions for each item is an essential item analysis procedure.

Result and discussion

The analysis of 9 items of Online Learning Readiness Scale (OLRS) showed that the model fit with Pearson Chi-square obtained was 4355,967, df = 1952917, p-value = 1,000 and Likelihood Chi-square = 720,460, df = 1952917 and p- value = 1,000 where both of these results indicate that the unidimensionality assumption of Generalized Partial Credit Model (GPCM) in this study has been fulfilled. If the model is not fit, then the use of GPCM needs to be changed into other models such as Partial Credit Model.

Therefore computer-based learning is not a problem if applied in big cities. However, considering the vast territory of Indonesia, the results of the test will produce different results when OLRS is tested to students in eastern Indonesia who still have limitations and there are gaps with big cities such as Yogyakarta or Jakarta.

Conclusion

Based on the results of this study, it can be concluded that the Online Learning Readiness Scale shows good and proven psychometric characteristics that can be used to measure readiness in dealing with computer-based exams. In addition, this measuring instrument is also proven to be one-dimensional with a fit model index that is proven fit. This finding shows that a new measuring tool has been created that is quite useful in the Indonesian context because it has been tested for its psychometric characteristics with a fairly complex GPCM method. This measuring instrument is expected to be used as a consideration to further improve the application of computer-based national exams that have begun to be established in Indonesia.