Qualitative Rigor in Research

The rejection of reliability and validity in qualitative inquiry in the 1980s has resulted in an interesting shift for ""ensuring rigor"" from the investigator’s actions during the course of the research, to the reader or consumer of qualitative inquiry. The emphasis on strategies that are implemented during the research process has been replaced by strategies for evaluating trustworthiness and utility that is implemented once a study is completed.

Without rigor, research is worthless, becomes fiction, and loses its utility. Challenges to rigor in qualitative inquiry interestingly paralleled the blossoming of statistical packages and the development of computing systems in quantitative research. Rather than explicating how rigor was attained in qualitative inquiry, a number of leading qualitative researchers argued that reliability and validity were terms pertaining to the quantitative paradigm and were not pertinent to qualitative inquiry (Altheide & Johnson, 1998; Leininger, 1994).

In seminal work in the 1980s, Guba and Lincoln substituted reliability and validity with the parallel concept of ""trustworthiness,"" containing four aspects: credibility, transferability, dependability, and conformability. Within these were specific methodological strategies for demonstrating qualitative rigor, such as the audit trail, member checks when coding, categorizing, or confirming results with participants, peer debriefing, negative case analysis, structural corroboration, and referential material adequacy (Guba & Lincoln, 1981; Lincoln & Guba, 1985; Guba & Lincoln, 1982).

Credibility: The credibility criteria involves establishing that the results of qualitative research are credible or believable from the perspective of the participant in the research.

Transferability: Transferability refers to the degree to which the results of qualitative research can be generalized or transferred to other contexts or settings.

Dependability: showing that the findings are consistent and could be repeated.

Confirmabilit : Confirmability refers to the degree to which the results could be confirmed or corroborated by others.

Strategies to ensure rigor inherent in the research process itself were back staged to these new criteria. This shift from constructive (during the process) to evaluative (post hoc) procedures occurred subtly and incrementally. Now, there is often no distinction between procedures that determine validity in the course of inquiry and those that provide research outcomes with such credentials. We are also concerned that by refusing to acknowledge the centrality of reliability and validity in qualitative methods, qualitative methodologists have inadvertently fostered the default notion that qualitative research must therefore be unreliable and invalid, lacking in rigor, and unscientific (Morse, 1999).

Reliability and Validity: The nature of knowledge within the rationalistic (or quantitative) paradigm is different from the knowledge in naturalistic (qualitative) paradigm. Consequently, each paradigm requires paradigm-specific criteria for addressing ""rigor"" (the term most often used in the rationalistic paradigm) or ""trustworthiness"", their parallel term for qualitative "rigor". They noted that, within the rationalistic paradigm, the criteria to reach the goal of rigor are internal validity, external validity, reliability, and objectivity. On the other hand, they proposed that the criteria in the qualitative paradigm to ensure ""trustworthiness"" are credibility, fittingness, auditability, and confirmability (Guba & Lincoln, 1981).

They recommended specific strategies be used to attain trustworthiness such as negative cases, peer debriefing, prolonged engagement and persistent observation, audit trails and member checks. Also important were characteristics of the investigator, who must be responsive and adaptable to changing circumstances, holistic, having processional immediacy, sensitivity, and ability for clarification and summarization (Guba & Lincoln, 1981).

• Techniques for establishing credibility: Prolonged Engagement : Spending sufficient time in the field to learn or understand the culture, social setting, or phenomenon of interest.
• Persistent Observation: the purpose of persistent observation is to identify those characteristics and elements in the situation that are most relevant to the problem or issue being pursued and focusing on them in detail. If prolonged engagement provides scope, persistent observation provides depth"" (Lincoln & Guba, 1985, p. 304).
• Triangulation : Triangulation involves using multiple data sources in an investigation to produce understanding.
• Peer debriefing: Through analytical probing a debriefer can help uncover granted biases, perspectives and assumptions on the researcher's part.
• Negative case analysis: This involves searching for and discussing elements of the data that do not support or appear to contradict patterns or explanations that are emerging from data analysis.
• Referential adequacy: Keeping a portion of raw data and archive it to allow the researcher and other critics to access it later for the purpose of testing analysis of the material.
• Member-checking: This is when data, analytic categories, interpretations and conclusions are tested with members of those groups from whom the data were originally obtained.
• Inquiry audit: It involves having a researcher not involved in the research process examine both the process and product of the research study.
• Confirmability audit : It involves having a researcher not involved in the research process examine both the process and product of the research study.
• Audit trail : An audit trail is a transparent description of the research steps taken from the start of a research project to the development and reporting of findings.
• Triangulation : A single method can never adequately shed light on a phenomenon. Using multiple methods can help facilitate deeper understanding.
• Reflexivity: While some may see these different ways of knowing as a reliability problem, others feel that these different ways of seeing provide a richer, more developed understanding of complex phenomena. )

This resulted in a plethora of terms and criteria introduced for minute variations and situations in which rigor could be applied. Perhaps as a result of this lack of clarity, standards were introduced in the 1980’s for the post hoc evaluation of qualitative inquiry (see Creswell, 1997)

Problems with post-hoc evaluation (development of Standards)

While standards are a comprehensive approach to evaluating the research as a whole, they remain primarily reliant on procedures or checks by reviewers to be used following completion of the research. But using standards on completion of the project at a time is of least importance as by then it is too late to correct problems.

Compounding the problem of duplicate terminology is the trend to treat standards, goals, and criteria synonymously. For example, Yin (1994) describes trustworthiness as a criterion to test the quality of research design, while Guba and Lincoln (1989) refer to it as a goal of the research. While strategies of trustworthiness may be useful in attempting to evaluate rigor, they do not in themselves ensure rigor. While standards are useful for evaluating relevance and utility, they do not in themselves ensure that the research will be relevant and useful. We argue that strategies for ensuring rigor must be built into the qualitative research process per se. These strategies include investigator responsiveness, methodological coherence, theoretical sampling and sampling adequacy, an active analytic stance, and saturation.

Verification Strategies in Qualitative Research

In qualitative research, verification refers to the mechanisms used during the process of research to incrementally contribute to ensuring reliability and validity and, thus, the rigor of a study.

Investigator Responsiveness: It is the researcher’s creativity, sensitivity, flexibility and skill. The lack of responsiveness of the investigator at all stages of the research process is the greatest hidden threat to validity. Lack of knowledge may be due to – overly adhering to instructions, the inability to abstract, working deductively from previously held assumptions.

Ensuring methodological coherence: Congruence between the research question and the components of the method is to be ensured. Data may demand to be treated differently so that the question may have to be changed or methods modified.