Scientist-practitioner Divide in Employee Development

The uses of psychometric assessments have grown rapidly over the last 15 years, with great focus on employee selection and development. However, the scientist-practitioner divide had raised different perspectives on the usefulness of such assessments in employee development. The scientists proposed that psychological constructs are unquantifiable and any attempt to measure them is fraught.

On the contrary, the practitioner suggests that anything that adds value to the client’s development has value. The overarching concern is that how the scientist-practitioner model is actually translated into real world behavior is unclear. This paper reviews this debate from both perspectives and discusses the possible future directions to bridge the gap by educating practitioners with more resources to utilize. Industrial Organizational (IO) Psychology by definition is the application of psychological principles, theories, and research to the work setting. With the increase in use of psychometric assessments over the last fifteen years (McDowall & Redman, 2017), the works of IO Psychology have been subjected to extensive scrutiny. More specifically, it was unclear how well the research done by the scientists are translated into applicable terms in the real world setting for practitioners. This relates to the introduction of scientist-practitioner model after the World War II, which has also formed the basis of the work of IO psychologists.

On one hand, the scientists are in continuous search of better methodologies, such as improving validity and reliability of psychometric tests. On the other hand, practitioners have focused on the applicative methods to streamline organizations’ modus operandi. The consequences of having a scientist-practitioner divide are that it often results in irrelevant theories being developed and invalid practices being used. Ones, Kaiser, Chamorro-Premuzic, and Svensson (2017) have also reported trends of unhealthy obsession with publishing journal articles that do not address the practicality element required in the real world. This gap between the focus of scientists and practitioners has been highly probed especially in the area of assessments for employee development (Moyle, & Hackston, 2018; Chamorro-Premuzic, Winsborough, Sherman, & Hogan, 2016; Furnham, 2008). Employee development is defined as “an integrated set of planned programs, provided over a period of time, to help assure that all individuals have the competence necessary to perform to their fullest potential in support of the organization’s goals” (Jacobs & Washington, 2003). In addition, with the recent attempt to challenge the validity of quantifying psychological constructs (Michell, 2008), the usefulness of such assessments is now being questioned. In this paper, psychometrics and training will first be defined, followed by a review of debate from both ends of the scientist-practitioner perspective, and finally a discussion on the role of these assessments in employee development.

Psychometric Assessments and Properties

As mentioned in lecture 4, psychometric assessments are methods of measurement designed to provide a quantitative assessment of one or more behavioral attributes. They often entail collections of questions or items that are administered and scored in a standardized manner, interpreted in a standardized format, and constructed in line with psychometric principles. Psychometric assessments are used to measure job specific skill areas that are not accurately assessed by other methods. Consequently, it enhances objectivity of the selection process, reduces selection errors and long-term organizational costs, and improves the chances of appointing productive and high performing staffs. These in practitioners’ views are strengths of using psychometric assessments, and are justifications to use them in organizational settings where costs are major concerns.

As a result, psychometric assessments have been used in organizations for selection, promotion, realistic job preview, training needs analysis and outcome assessment, team building, and career consultation. There are vast arrays of assessments that measure several psychological attributes, such as personality and cognitive abilities, for various purposes. Just to name a few widely known assessments, there is the Stanford-Binet test and Emotional Intelligence Test, and the Minnesota Multiphasic Personality Inventory (MMPI; Hathaway & McKinley, 1943), Myers-Briggs Type Indicator (MBTI; Myers, 1962; Myers, McCaulley, Quenk, & Hammer, 1998), Big Five model of personality assessments, and 360 feedback instruments.

Training and Development

Training refers to the systematic acquisition of skills, concepts, or attitudes resulting in improved performance in another environment (from lecture 6). It was also mentioned that the ability to learn forms the basic foundation for training programs. There are three broad categories of learning outcomes – cognitive, skill-based, and affective outcomes. In training and development, it is crucial that these outcomes are achievable and convey transferrable skills to their work. However, it is also important to look at the holistic development of an employee, as their overall well being would have an effect on their work performance.

Discussion Scientist’s Perspectives

Scientists have always been thought to be interested only in methodology and hence they constantly come up with improved versions of previous theories and models, in attempt to fill certain gaps. These gaps include low reliability and validity, lack of evidence of norms, and cross-cultural applicability (Furnham, 2018). As such, it can be inferred that scientists are well equipped with knowledge and they know which assessment should be used when the criteria change. Scientists’ criteria for choosing a particular assessment would then mainly revolve around the reliability and validity of the test itself. The current up-to-date assessments, such as MBTI, do demonstrate significant test-retest reliability (Myers, McCaulley, Quenk, & Hammer, 1998) and Five-Factor Model (FFM) do show incremental validity (Furnham, Jensen, & Crump, 2008). However in light of this, scientists have begun to challenge the ability to quantify psychological constructs, as the presumption that psychological attribute is quantitative in structure has never been tested or questioned (Michell, 2008). It is a bold challenge to the world of psychometrics as it is likely to overturn most existing psychometric assessments and as a result, it remains a challenge for the academia. Similarly in science, the existence of arbitrary units demonstrates a lack in ability to quantify all parameters.

For that matter, they have also revised the term and changed it to procedure defined unit. Aubrecht, French and Iona (2011) had also recognized that all units are arbitrary for the fact that numbering systems are anthropocentric. To this nihilistic view that many attributes now seem illusory, Barrett (2011) recognized the value in settling for a fuzzy-order measurement since most psychometrics are represented in terms of orders and not real numbered continuous magnitudes. For as long the hypothesis of ability to quantify psychological attributes remains untested and assumed to be true, acceptance and assumptions have to be made in order to stay relevant (Barrett, 2011).

Practitioner’s Perspectives

On the other hand, it is unclear how practitioners make their choice of assessments – giving probable priority to factors such as costs, ease of administering, and familiarity of a particular test. Furnham and Schofield (1987) found out that most practitioners are contented with the Barnum effect from the tests based on how their clients reacted to their test profiles. McDowall and Redman (2017) did an analysis of the practical and industry trends that reported 80. 5 percent of the participants choose familiarity, which is the highest amongst all other options, as the reason when choosing which psychometric assessment to use. Far behind in second place was basis from literature reviews (McDowall, & Redman, 2017). Practitioners often cite the lack of access to academic literature as the reason to not practice with scientific evidences. However it is not well known to practitioners that research databases such as EBSCO have removed access barriers, with the intention of promoting science-based practices. Also surprisingly, Furnham (2018) in his study has also found that test purchasers, or presumably practitioners, are looking at important psychometric properties such as reliability and validity. This could highlight an important transition of practitioners placing more value on scientific evidence.

Psychometric Assessments’ Role in Employee Development Performance Management

Logically, the natural argument would be to pick out the poorly performing individuals and to send them for training programmes. The key concern would then be who is the right person to choose for development programmes. Psychometric assessments may also serve to measure performances of employees. These include judgmental ratings such as 360-degree feedback. From the feedbacks, it gathers information from the employee’s superiors, peers and subordinates and may be used to identify what training needs this employee may have. Although 360-degree feedbacks have shown to be judged as the least practical amongst twelve other assessments (Burnham, 2008), it was also judged as the top few having higher validity (Burnham, 2008).

In this case it might not be truly reflective of its real practicality, as there could be other factors influencing this results. Furnham and Jackson (2011) found that age of practitioners predicted practical application of tests, and that educational qualifications predicted usefulness of psychological tests. There is value in finding practical steps from literatures that practitioners can adopt. For example, making use of these findings, it is instrumental to make sure that practitioners are trained in using the relevant psychological tests. That way, they can better administer tests onto their employees. In addition, there are two types of measurements – normative measures and ipsative measures. The former produces data that discriminates characteristics across individuals and variables giving us normal distributions, and the latter discriminates within individuals and variables. Both are valid and important for development, where normative measures tell us where an individual stands in comparison to others and ipsative measures evaluates traits within an individual.

Additionally, ipsative measures can be used to compare with one’s previous results to illustrate improvement or progress.

Match between Employee and Training

There are two main categories of assessments – maximum performance and typical performance. The former puts one’s capabilities to test to see what is the best one can do, whereas the latter measures how one prefers to act in a particular situation. With these data collected, practitioners are able to then translate it into employees’ ability to learn and their preferred learning methods. For instance, if the employee tends to extroversion, training programmes that are conducted in groups may appeal to these employees better. Hence, the information may reveal to the organization which employee to send for a particular type of training programme. In other words, it attempts to find the best employee-to-training match.

Self-Awareness Moyle and Hackston (2018) has nicely summarized this in their recent publication that personality tests such as MBTI, and Strengths. Finder are not predictive of performance but rather facilitate self-reflection and hence better self-awareness. These are practical in a way that it helps employees to be clearer of their own behaviors and also realize the implications to performance. Self-awareness was shown to have positive implications such as higher job contentment and enthusiasm, and better relationships and communication with peers (Sutton, Allinson, & Williams, 2013). In a way, self-awareness acts as a starting point and also a catalyst to ensure that development programmes planned are relevant to themselves. Judgmental ratings such as Behaviorally Anchored Rating scales (BARS) can also help employees to recognise which behaviors are the ones that cause them to underperform. Scientists have also argued that 360-degree feedbacks have great limitations such as contamination of feedbacks due to motives of competition, affiliation, collaboration, and revenge (Pavur, & Lepard, 1997), as well as lack of training for raters to give quality feedback. It tends to result in an array of feedbacks that seemingly reflect discrepancies in the individual.

However, Hazucha, Hezlett, and Schneider (1993) has also demonstrated the value of “awareness of discrepancy” as part of self-awareness and hence an opportunity for development. These discrepancies assess more holistically of an individual and reveal gaps that may possibly be present. Regardless of whether the training need is in terms of interpersonal skills or work-related skills, it would nevertheless be beneficial to develop their employees holistically. It demonstrates an organization’s concerns over employees’ well being and also their personal growth, instead of seeing employees as a tool to make money. This may help to build their commitment and loyalty over time. Opportunity for Goal Settings It is fundamental to make sure that the results obtained from the psychometric assessments are well interpreted in ways that are easy to communicate and understand by the employee. As Moyle and Hackston (2018) had suggested, what we do with the results is far more important and has much more value than the results itself. By which, after gaps have been identified, the practitioner and employee can set smarter goals as part of short and long term development plan. From time to time, reviewing these goals would be beneficial to reinforce the developmental outcomes.

Conclusion

There is definitely a realization from the practitioners that they cannot over rely on familiarity as a basis for choosing which psychometric assessment to use (McDowall, & Redman, 2017). This trend is also noticeable in the researchers as well, from their journal articles that highlight the need to train and educate the future graduates with the ability to combine both perspectives and think holistically (Brooks, Grauer, Thornbury, & Highhouse, 2003; Ones et al. 2017; Dorsey, & Harper, 2018; Weiss, 2018). It is worthwhile to note the limitations of psychometrics, and its underlying assumptions. Yet it is futile to overemphasize and be too fixated on the theories such that it becomes inapplicable to real-world situations.

As a practitioner, there will be restrictions limiting us from using the best assessment tools. Many concerns revolve around monetary costs, time required to administer the test, and factors that stem from unfamiliarity with the assessment including time needed to learn how to administer the test and interpret the results in meaningful ways. However, these should not falter us in our beliefs to use best practices. I firmly believe in practicality and that as much as research can lend credibility when practicing IO psychology, there are just some domain that cannot be studied on, given the complexity of person-to-person interaction, and the person-to-environment interaction at the particular time of taking the test and giving feedback. It is not to say that all the more scientists should work on unraveling this complexity, but rather, as many academics have also highlighted, their works should work on narrowing the gap between their literature and the practices.

On this aspect, Malott (2018) has proposed a science-based practitioner model, in contrast to the current scientist-practitioner model. Perhaps given the vast array of assessment tools available now, it is possible to redirect the scientists’ focus to hear the practitioners’ experiences and challenges when using them. From there, modifications to the models can be made to increase the practicality and usefulness of them, as opposed to overemphasizing on the scientific rigor.