Creating a Code Book and inputting Data Entry

69

B. RELIABILITY AND VALIDITY IN QUALITATIVE RESEARCH

Discussions of the terms “reliability” and “validity” have traditionally been attached to

quantitative research. Reactions by qualitative researchers have been mixed regarding whether or not

this concept should be applied to qualitative research. At the extreme, some qualitative researchers

have suggested that the traditional quantitative criteria of reliability and validity are not relevant to

quantitative research. Most qualitative researchers argue that some qualitative research studies are

better than others, and they frequently use the term validity to refer to this difference. When

qualitative researchers speak of “validity” they are usually referring to qualitative research that is

credible and trustworthy, and therefore, defensible. The trustworthiness of the study asks the reader to

assess the extent to which the research findings are believable – was the study sufficiently well done

such that we can believe in what its outcomes purport to tell us? When questions are raised about the

credibility of the study, they are asking if the researchers understand their theoretical framework and

base it on the data generated. This is somewhat comparable to issues related to assessing the internal

validity of a quantitative study, where the question is asked “is the researcher measuring what he

thinks he is measuring?” The term transferability is meant to focus our attention on the application of

the findings to other settings and populations, roughly comparable to the external validity of the

quantitative approach. Confirmability in qualitative studies asks about any biases inherent in the data

collector, seeking to establish the neutrality or auditability of the findings. The concept of reliability

in quantitative studies is applied to qualitative research through its more general interpretation of the

dependability of the observations obtained; in qualitative research it refers to the ability of others to

follow the methods used to collect the data (rather than replicating the findings).

A number of specific techniques have been offered to enhance the trustworthiness and credibility

of a qualitative research study.

A) Triangulation is the process of corroborating evidence from different individuals (e.g., employee

and employer), types of data (e.g., observational field notes and interviews), or methods of data

collection (e.g., documents and interviews). The researcher examines each information source

and finds evidence to support a particular theme or pattern. This ensures that the study will be

accurate because the information is not drawn from a single source, individual, or process of data

collection.

B) Researchers also check their findings with participants in the study to determine if their findings

are accurate. Member checking is a process where the researcher asks one or more participants in

the study to check the accuracy of the account. This check involves taking a study back to

participants and asking them (in writing or in an interview) about the accuracy of the report.

Participants are asked about many aspects of the study, such as whether the description is

complete and realistic, if the themes are accurate to include, and if the interpretations are fair and

representative of those that can be made.

C) Peer debriefing is a process similar to member checking, although the researcher’s colleagues are

brought into assist. Every effort is made to include both colleagues who are experts in the field of

study, as well as those who are not. The researcher enlists the help of colleagues by asking them

to code a few of the transcripts so that they are familiar with the process and some sample

70

responses. These peers are also requested to listen critically to the analysis being developed, and

to offer their feedback and suggestions.

D) The analysis and interpretation of the data should be “thick” in that all of the complexities in the

data should be included. This means that the commonalities among the participants, as well as

their variability, should be captured and adequately described. Thick description reveals the

multiplicity of perspectives among the research participants, often leading to an interpretation

that includes discussion of their diversity under varying contexts and circumstances.

E) In negative case analysis, the researcher focuses on the responses of those who appear to diverge

from more frequent responses. These are not considered as “outliers” as they are in quantitative

research, but rather, variability among the responses from the study participants is embraced. The

qualitative investigator scrutinizes these cases in an attempt to learn from them – why is this

case/person different from the others? This results in a more complex, dense, think analysis and

level of description and ultimately, understanding.

F) Qualitative researchers continue their data collection and analysis until saturation has been

reached. They continue collecting and analyzing data until no new questions are being asked or

observations are being obtained. While quantitative methods typically determine a priori what

their sample size should be to obtain a certain level of confidence in the findings, qualitative

researchers must rely on the data to reveal itself as “saturated” so that they do not prematurely

end data collection or analysis. Qualitative reports almost always indicate questions for further

study that have arisen during the course of the study, indicating what areas have not yet been

adequately addressed.

G) While one of the goals of conducting quantitative research is to strive for objectivity, qualitative

researchers recognize and herald the subjective role of the researcher throughout the process. In

accordance with this, qualitative researchers typically keep a journal throughout their research,

delineating their own ideas, reactions, and “biases” in order to try to separate their own responses

from those of the participants. They address and acknowledge their own biases by locating

themselves in the data, continuing a self-reflective position throughout the study. The

researcher’s acknowledgement of what they bring to the study and the extent of their

involvement is important information for the reader to consider in assessing the trustworthiness

and credibility of the study.

H) Researchers may also ask a person outside the study to conduct a thorough review of the study

and report back, in writing, the strengths and weaknesses of the project. This is the process of

conducting an external audit, where a researcher hires or obtains the services of an individual

outside the study to review different aspects of the research. This auditor reviews the project and

writes or communicates an evaluation of the study. This audit may occur both during and at the

conclusion of the study, and auditors typically ask questions such as: “are the findings grounded

in the data?”, “are inferences logical?”, “are the themes appropriate?”, “can inquiry decisions and

methodological shifts be justified?”, “what is the degree of researcher bias?”, and “what

strategies are used for increasing credibility?”