U10A1-68 - Qualitative Research Plan - ***TUTOR FOLLOW ALL INSTRUCTIONS AS OUTLINED TO COMPLETE THIS WORK. READ ATTACHMENTS.

The Qualitative Report

Volume 4 | Number 3 Article 4

3-1-2000

A Comparative Discussion of the Notion of 'Validity' in Qualitative and Quantitative Research Glyn Winter University of Manchester, [email protected]

Follow this and additional works at: http://nsuworks.nova.edu/tqr

Part of the Quantitative, Qualitative, Comparative, and Historical Methodologies Commons, and the Social Statistics Commons

This Article is brought to you for free and open access by the The Qualitative Report at NSUWorks. It has been accepted for inclusion in The Qualitative Report by an authorized administrator of NSUWorks. For more information, please contact [email protected].

Recommended APA Citation Winter, G. (2000). A Comparative Discussion of the Notion of 'Validity ' in Qualitative and Quantitative Research. The Qualitative Report, 4(3), 1-14. Retrieved from http://nsuworks.nova.edu/tqr/vol4/iss3/4

A Comparative Discussion of the Notion of 'Validity' in Qualitative and Quantitative Research

Abstract The issues surrounding the use and nature of the term 'validity ' in qualitative research are controversial and many. In this paper, the author attempts to establish that 'validity ' is not a single, fixed or universal concept, but rather a contingent construct, inescapably grounded in the processes and intentions of particular research methodologies and projects. The first section of this work deals with the problems faced in defining 'validity ' in both quantitative and qualitative research methods and will briefly review other authors' attempts to categorise it. The work will then proceed to distinguish and compare the claims to 'validity ' made by quantitative and qualitative researchers, highlighting similarities and differences as they emerge. Finally, an attempt will be made to establish that an understanding of nature of 'truth' is central to any theorisation of 'validity.' It will become clear that it is the affiliations of methodologies, concerning truth, that generate varying notions of 'validity.'

Creative Commons License

This work is licensed under a Creative Commons Attribution-Noncommercial-Share Alike 4.0 License.

This article is available in The Qualitative Report: http://nsuworks.nova.edu/tqr/vol4/iss3/4

A Comparative Discussion of the Notion of 'Validity' in

Qualitative and Quantitative Research

by

Glyn Winter +

The Qualitative Report, Volume 4, Numbers 3 & 4, March, 2000

Abstract

The issues surrounding the use and nature of the term 'validity' in qualitative research are

controversial and many. In this paper, the author attempts to establish that 'validity' is not a

single, fixed or universal concept, but rather a contingent construct, inescapably grounded in the

processes and intentions of particular research methodologies and projects. The first section of

this work deals with the problems faced in defining 'validity' in both quantitative and qualitative

research methods and will briefly review other authors' attempts to categorise it. The work will

then proceed to distinguish and compare the claims to 'validity' made by quantitative and

qualitative researchers, highlighting similarities and differences as they emerge. Finally, an

attempt will be made to establish that an understanding of nature of 'truth' is central to any

theorisation of 'validity.' It will become clear that it is the affiliations of methodologies,

concerning truth, that generate varying notions of 'validity.'

Introduction

It is the intention of this work to highlight some of the issues surrounding the use and nature of

the term 'validity' and to establish that 'validity' is not a single, fixed or universal concept, but

rather a contingent construct, inescapably grounded in the processes and intentions of particular

research methodologies and projects. The first section of this work will deal with the problems

faced in defining 'validity' in both quantitative and qualitative research methods and will briefly

review other authors' attempts to categorise it. The work will then proceed to distinguish and

compare the claims to 'validity' made by quantitative and qualitative researchers, highlighting

similarities and differences as they emerge. Finally, an attempt will be made to establish that an

understanding of nature of 'truth' is central to any theorisation of 'validity.' It will become clear

that it is the affiliations of methodologies, concerning truth, that generate varying notions of

'validity.'

Definitions of 'Validity'

The exact nature of 'validity' is a highly debated topic in both educational and social research

since there exists no single or common definition of the term. Therefore, in order to understand

something of the range of meanings attached to 'validity', it is essential to review a selection of

the range of definitions given by leading authors.

A much cited definition of 'validity' is that of Hammersley's (1987, p. 69): "An account is valid

or true if it represents accurately those features of the phenomena, that it is intended to describe,

explain or theorise." Although this would seem to be an all-encompassing and reasonable

description, many other definitions fail to envisage such a 'realist approach' (Denzin & Lincoln,

1998, p. 282). The fact that there are so many possible definitions and replacement terms for

'validity' suggests that it is a concept entirely relative to the person and belief system from which

it stems.

One of the most recurring features in critical discussions of 'validity' is the combination of

'validity' with the term 'reliability' (Campbell & Fisk as cited in Simco & Warin, 1997; Black &

Champion, 1976, p. 222, 234; Kerlinger, 1964, p. 430; Hammersley, 1987, p. 75). Yet, the

definitions for 'reliability' are as varied and as complex as those for 'validity'.

Hammersley (1987) reviews the extent to which 'reliability' and 'validity' are defined by a

selection of different authors writing from different methodological positions. I will avoid

merely repeating those definitions verbatim and instead offer a few key words summarising each

definition.

'validity'

'An agreement between two efforts to measure the same thing with

different methods' -- Campbell and Fisk (as cited in Hammersley, 1987)

'The measure that an instrument measures what it is supposed to' --

Black and Champion (1976, pp. 232-234)

'Accuracy' -- Lehner (1979, p. 130)

'Degree of approximation of 'reality' -- Johnston and Pennypacker (1980,

pp. 190-191)

'Are we measuring what we think we are?' -- Kerlinger (1964, pp. 430,

444-445)

'to the extent that differences in scores yielded…reflect actual

differences' -- Medley and Mitzel (as cited in Hammersley, 1987, p. 150)

Reliability

'An agreement between two efforts to measure the same thing with the

same methods' -- Campbell and Fisk (as cited in Hammersley, 1987)

'Ability to measure consistently' -- Black and Champion (1976, pp. 232-

234)

'Reproductibility of the measurements…stability' -- Lehner (1979, p.

130)

'Capacity to yield the same measurement…stability' -- Johnston and

Pennypacker (1980, pp. 190-191)

'Accuracy or precision of a measuring instrument?' -- Kerlinger (1964,

pp. 430, 444-445)

'To the extent that the average difference between two measures

obtained in the same classroom is smaller than…in different classrooms'

-- Medley and Mitzel (as cited in Hammersley, 1987)

Insofar as 'validity' definitions are concerned, two common strands begin to emerge: Firstly,

whether the means of measurement are accurate. Secondly, whether they are actually measuring

what they are intended to measure. However, it is with the definition of 'reliability', and its

confusion with 'validity' that the greatest level of disagreement appears between authors. The

notions of accuracy, more commonly attributed to 'validity', appear to be associated with

'reliability' also. What authors do seem to attribute to 'reliability' more commonly than to

'validity' is the degree of replicability. From this summary, it is possible to suggest that the

aggregated definition of 'validity' could be that of accuracy, and the definition of 'reliability' that

of replicability. Whatever the differences in definition and classification are amongst the above

authors, it is these two concepts--accuracy and replicability--which appear to underpin their

aggregated goals and means.

However, it immediately becomes apparent that an aggregated definition, whilst representing the

majority of opinion, can prove mutually exclusive to the definitions of certain researchers and

methodologies. I refer of course to qualitative researchers, some of whom vehemently deny that

replicability is either useful or possible in situations concerning highly complex and transient

circumstances: namely those that involve the lives, thoughts and behaviour of actors. Some

researchers would also take issue with the notion of accuracy and claim that precision is of

greater importance. As Hammersley continues:

We can measure the length of a large object in terms of metres, centimetres or millimetres. In

that order, these scales represent an increasing degree of precision. Note that this is independent

of the accuracy of the measurement. On this usage, a score may be very precise but highly

inaccurate. How precise we want our measurement to be will depend on our purposes…Other

things being equal, the more precise the scale, the more difficult it is to achieve high levels of

'validity'…there is often the temptation to be more precise than the level of 'validity' with which

an object is being measured justifies. (Hammersley, 1987, p. 77)

What Hammersley is referring to is the trade-off between 'validity' and precision. It may indeed

be possible to measure the time it takes child 'A' to finish a test (designed to last for an hour) in

milliseconds, but this extra degree of precision is unlikely to generate greater accuracy. In

particular over-precision can threaten 'validity' in statistical tests. Overly precise, discrete

categorisations, even with a sample of thousands, would generate categories by the thousand,

each of which is likely contain only one or two results. In addition, computation, measures of

central tendency and measures of variance become increasingly inaccurate and unworkable as

precision increases. Even in qualitative research, which thrives on accurate description, measures

which are too precise, such as the time in seconds of pauses in conversation or exact measures of

vocal intonation; can confound and obscure the more general purposes of the research and

analysis.

Does 'Validity' Concern the Whole Process of Research, or Certain Key Stages?

'Validity…is a unitary concept.' (Wainer & Braun (1988, p. xvii)

It is the intention of this section to assert that 'validity' is not in any simple sense a unitary

concept. There is no single form, construct or concept that can universally be claimed to define

or encompass the term. Neither, however, can validity be said to be a discretely identifiable

element of any research project, which is capable of being located at multiple and specific stages

within the research. The concept of 'validity' defies extrapolation from, or categorisation within,

any research project.

For some researchers (mainly qualitative), 'validity' is not a singular acid test that can be applied

to the research process as a whole. The 'validity' measure can be applied differently depending

upon the researcher's beliefs as to what stage of the research process is in need of validation.

Such an approach may perceive validity as referring only to measurement, observers, scores,

instruments, relationships between scores or observable variations, rather than to the whole

research process.

Within this approach, 'validity' is claimed either by viewing it as resident in a particular stage of

the research process, or as combinations of certain stages. Maxwell (1992, p. 285) identifies five

typologies of 'validity' as they relate to various stages of the research. I will briefly describe

Maxwell's definitions and offer some critique of them, not because I necessarily conform to his

segmented typologies, but because they raise some interesting concerns.

Descriptive Validity

Descriptive 'validity' is that which is concerned with the initial stage of research, usually

involving data gathering. The central issue is factual accuracy in the informational statements

that describe what was observed and experienced - what Runciman (1983) refers to as

'Reportage.' The choice of language and selection of 'relevant data' are the greatest threat to

'validity'. Maxwell (1992, pp. 287-288) identifies many possible areas of error within this

process concerning data selection and initial interpretative biases. The section concludes with the

following statement:

If different observers or methods produce descriptively different data or accounts of the same

events or situations, this puts into question the descriptive 'validity' (and other types of 'validity'

as well) of the accounts. (Maxwell, 1992, p. 287)

This measure of 'validity' that Maxwell offers should be approached with some caution. Of

course, stark differences in factual statements or events, such as whether person 'A' was present

or not during data gathering, appears a very fundamental error. However, what this rather blatant

difference in description demonstrates, is the highly selective, reductive and subjective processes

involved in all research. If researcher '1' failed to notice the presence of person 'A', while

researcher '2' noted the presence of that particular person as an 'important' factor, then this

discrepancy will undoubtedly raise some concerns. Of course, if the researcher is dishonest or

lacks commitment to the work then the matter becomes one of integrity. In this case, most would

agree that 'validity' has been compromised. However, if both accounts represent the

conscientious efforts of the two researchers, rather than labelling the greatly differing accounts as

evidence of reduced validity and unreliable measures, the discrepancies between them merely

mark the multi-perspective experiences of researchers '1' and '2'. The 'inaccuracy' is an honest

and valid account of the researchers' experiences of the 'realities' that exist within the events

recorded. This perspective acknowledges the essential role of the researcher within the research

process and the events themselves. Of course, this would be far from acceptable within the

quantitative paradigm, but within qualitative research, this would raise interesting questions

worthy of investigation. Contrary to the assumptions governing quantitative research, qualitative

methodologies have come to recognise that research into the lives, personalities and experiences

of people involves the inevitability of contradiction and the existence of parallel and opposing

truths within accounts. To 'cleanse' the data of these personally-oriented discrepancies involves

further subjective action, since it would involve a degree of selection and choice.

Interpretative Validity

Within the qualitative paradigm, interpretation is typically viewed as an inextricable (and,

indeed, unavoidable) element of data collection. On these grounds, Maxwell's segregation of

description and interpretation is not only a false distinction, but effectively impossible.

Interpretation is essentially couched within the rhetorics that the researcher uses to describe a

situation and is mutually constructed between researchers and subjects. Quantitative researchers

do much to disassociate themselves from such interpretations, yet these too are inevitable in their

categorisations and selection of data. To imagine that any 'reasonable' (by reasonable I mean that

which can be justified by any kind of evidence present at one or more stages of the data)

constructed interpretation could ever be proven to be invalid is almost unimaginable. Yet, in

Maxwell's 'realist' approach to 'validity' (1992, p. 290) he ultimately upholds that a 'valid'

account "must respect the perspectives of the actors in that situation" (1992, p. 290). What

Maxwell asserts is that an account is only valid if the actors are able to confirm or recognise the

findings of the research, in particular, he notes, where there is a chance that they may be

disadvantaged by the results. Using this relativistic logic, a convicted rapist with a long history

of convictions for sexual crimes, yet who protests that he is innocent of every charge, would

have to be portrayed as an innocent victim of a series of miscarriages of justice. In fact, we have

no choice but to adopt his own perspective as a test of the 'validity' of our interpretation of his

actions. Regardless of the ethical implications of interpreting meaning from the observations of

others, other than those that they would necessarily agree with, it is worth noting that an

individual may often have no more 'valid' interpretations of their own actions than another might

make.

Theoretical Validity

Maxwell comments that the previous two accounts of 'validity' depend on a consensus on the

application of terms and that disagreements refer only to accuracy and not meaning. We have

already seen that that this analogy is incorrect. Maxwell continues to say 'theoretical validity' is a

more 'abstract' analysis than the 'descriptive' and 'interpretive' 'validities' concerning the

'immediate physical and mental phenomena studied' (1992, p. 291). Maxwell claims that

theoretical 'validity' goes beyond the concrete and descriptive and concerns itself with the

constructions that researchers apply to, or develop, during the research. This of course is a

fallacy, as we have already established that a researcher's theoretical framework and

constructions, whether grounded theory or metatheoretical, intrinsically define both the recording

and interpretation of the data at the initial stage of research. What is interesting about this

typology is that this form of validity applies not only to the research itself, but to the mental and

emotional constructs of the researcher. However, identifying one's own theoretical standpoint, or

even categorising it as essentially 'Marxist', 'Positivist', 'Sructuralist', 'Postmodern', 'Feminist' or

any combination of any theories, would paradoxically necessitate further subjective theorisation

and prove futile.

Generalisability

Maxwell (1992, pp. 293-295) observes that the degree to which an account is believed to be

generalisable is a factor that clearly distinguishes qualitative and quantitative research

approaches. The ability to generalise findings to wider groups and circumstances is one of the

most common tests of 'validity' for quantitative research and yet is considered to be of little, or

even no, importance for many qualitative researchers. Maxwell also notes that sampling, a vital

consideration in establishing the 'validity' of a statistical test, is usually purposeful in qualitative

research as opposed to random. Qualitative research almost exclusively limits itself to 'internal'

generalisations, if indeed it seeks to claim any form of generalisability at all. Quantitative

research, on the other hand, attempts to deal with both 'internal and 'external' generalisations,

referring to these as 'internal validity' and 'external validity' respectively (Maxwell, 1992, p.

294). One possible explanation for this difference in the scope of the claims made by researchers

is tied to the types of situations and phenomena that qualitative and quantitative researchers

investigate. In a very general sense, qualitative research concerns itself with the meanings and

experiences of the 'whole' person, or localised culture. On the other hand, quantitative research

attempts to fragment and delimit phenomena into measurable or 'common' categories that can be

applied to all of the subjects or wider and similar situations. Hence, quantitative research, whilst

able to claim validity for wider populations and not just merely samples, is restricted to

measuring those elements that, by definition and distortion, are common to all. This raises the

question of 'at what cost' are we exchanging accuracy for generalisability. Within the quantitative

definition, an account may be judged 'valid', 'replicable' and 'stable' on the merits of its

generalisability. Yet, one could argue that generalisation in itself is neither 'valid' nor accurate. It

is likely that a 'generalisable' statement, whilst relating to all those to whom it is applied, may not

actually describe the phenomena of any single case with any accuracy, in the same way that a

mean average score need not be the same value as any of the numbers of which it is an average.

Evaluative Validity

As one would expect, this form of validity that Maxwell proposes refers to the application of an

evaluative framework. Maxwell asserts that evaluative frameworks are similar in both qualitative

and quantitative research and that many researchers make no claim to apply any evaluation to

their research whatsoever (Maxwell, 1992, p. 295). However, evaluation is not some conclusive

statement that may or may not be tagged onto the end of a research report, thus determining the

nature, outcome or 'reality' of that research. Similar to the issues raised in response to Maxwell's

categorisation of 'interpretive validity, evaluation is an almost inescapable, and often

unconscious, consequence of the research process itself. Recognising that evaluation of some

sort is an inescapable inevitability within research, enables the control of that evaluation, and

offers a measurement of the research in terms of its overall 'validity.'

As stated at the beginning of this section, it has been my intention to establish that the

deconstruction of the concept of validity into separate types and then relating them to certain

stages of the research process is an unnecessary conceptualisation which may lead one to think

that as long as these parameters are considered, then the research could be said to be wholly

'valid.' No such formulaic approach could ever claim to guarantee validity, and Maxwell's

typologies, whilst being convenient and systematic, are paradoxical and unnecessary.

A Summary of the Differences and Similarities Concerning the Notion of

Validity within Qualitative and Quantitative Research

Although the relative notions of 'validity', as they relate to quantitative and qualitative research

respectively, have been discussed at various stages of this work, it is worth summarising these

ideas for the purpose of comparison.

The traditional criteria for 'validity' find their roots in a positivist tradition, and to an extent,

positivism has been defined by and bolstered along by a systematic theory of 'validity'. Within

the positivist terminology, 'validity' resided amongst, and was the result and culmination of other

empirical conceptions: universal laws, evidence, objectivity, truth, actuality, deduction, reason,

fact and mathematical data to name just a few. It is within this tradition and terminology that

quantitative research is traditionally defined. Qualitative research, arising out of the post-

positivist rejection of a single, static or objective truth, has concerned itself with the meanings

and personal experience of individuals, groups and sub-cultures. 'Reality' in qualitative research

is concerned with the negotiation of 'truths' through a series of subjective accounts. Whereas

quantitative researchers attempt to disassociate themselves as much as possible from the research

process, qualitative researchers have come to embrace their involvement and role within the

research. For quantitative researchers this involvement would greatly reduce the validity of a

test, yet for qualitative researchers denying one's role within research also threatens the validity

of the research.

Some qualitative researchers have argued that the term validity is not applicable to qualitative

research and have at the same time realised the need for some kind of qualifying check or

measure for their research. As a result many researchers have espoused their own theories of

'validity' and have often generated or adopted what they consider to be more appropriate terms,

such as 'trustworthiness', 'worthy', 'relevant', 'plausible', 'confirmable', 'credible' or 'representative'

(Denzin & Lincoln, 1998; Guba & Lincoln, 1989; Hammersley, 1987; Mishler, 1990; Wolcott,

1990). Other qualitative researchers have rejected the notion of 'validity', in any form, as entirely

inappropriate to their work.

Reliability and validity are tools of an essentially positivist epistemology. While they may have

undoubtedly proved useful in providing checks and balances for quantitative methods, they sit

uncomfortably in research of this kind, which is better concerned by questions about power and

influence, adequacy and efficiency, suitability and accountability. (Watling as cited in Simco &

Warin, 1997)

Wainer and Braun describe the validity that concerns quantitative research as 'construct validity'.

The construct is the initial concept, notion, question or hypothesis that determines which data is

to be gathered and how it is to be gathered. Wainer and Braun assert that quantitative researchers

actively cause or affect interplay between construct and data in order to 'validate' their

investigation, usually by the application of a test or other process. Data can either support or

reject this construct which can then be can be put forward as a theory or further hypothesis

(Cronbach & Meech, 1995, p. 20).

In this sense the validity is very specific to the test to which it is applied, or:

The correlation of scores on a test with some other objective measure of that which the test is

used to measure. (Bingham, 1937, p. 214)

Unlike quantitative research, there are no standardised or accepted tests within qualitative

research and often the nature of the investigation is determined and adapted by the research

itself. There may not be any hypothesis or even any findings as such. Instead the 'validity' of the

research resides with the representation of the actors, the purposes of the research and

appropriateness of the processes involved. The only similarity between the two research methods

is that, at some point, questions will be asked and data will be collected. Likewise,

commonalities within definitions of 'validity' only exist within its concern for the research

process and its appropriateness to the phenomena investigated.

As already suggested, there are differences that exist between quantitative and qualitative

research in what they attempt to research. Quantitative research limits itself to what can be

measured or quantified and qualitative research attempts to 'pick up the pieces' of the

unquantifiable, personal, in depth, descriptive and social aspects of the world. Many of the

allegations of invalidity from both sides can be attributed to a failure to recognise the purposes to

which each methodology is suited.

However, there are further issues in the selection of research subjects, in particular those

concerning the selection or use of samples, which ultimately relates to the degree to which the

research is intended to be internally or externally generalisable. Internal validity relates to

whether the findings or results of the research relate to and are caused by the phenomena under

investigation and not other unaccounted for influences. In essence, this essentially comes down

to the issue of correlation and causality. Within quantitative research, causality must be

established for the test to be valid, or indeed, of any use at all. Researchers get around this by

attempting, some say in vain, to isolate the factors under investigation away from any

confounding variables. The pollution of the results caused by other unwanted factors does not

entirely invalidate the test, especially since they cannot really be fully isolated and stripped

away, but a study with notable, and yet unaccounted for, influences is said to have low internal

validity. In a sense what the quantitative researcher is attempting to do is ensure that his/her

explanation for the results can be the only possible or feasible one. There are many factors that

pose a threat to validity in this case, such as the maturation of the individual in a longitudinal

study, previous experiences of the individuals, 'lost data,' the affects of the test itself, or

regression to the mean as a result of outliers or positively or negatively skewed values.

Qualitative research 'validity' also partially requires an 'internal validity', but is not centrally

concerned with issues of cause and effect, and is not so harsh in its isolation and categorisation

of particulars within the phenomena.

External validity is the secondary, yet still very important goal of quantitative research. The

measure of external validity is the extent to which the results can be generalised and thus applied

to other populations. A test can have very high internal validity and very low external validity at

the same time. Threats to external validity are similar to those for internal validity, except that

the test itself is more likely to pose a threat as an alternative explanation for similar results.

External validity is often of no importance to qualitative research and the attempt to achieve it

can seriously hinder its overall validity. However, qualitative findings are best generalisable to

the development of theories and not wider populations.

The necessity for random sampling and other standard design features, which are features of

quantitative research, also have little or no importance in qualitative research, where researchers

may deliberately choose to investigate the unusual and unrepresentative. Unlike quantitative

research whose validity depends upon the established canon of standardised tests and procedures,

qualitative research embodies a vast and evolving body of techniques that can be modified or

developed as the research demands. What these vast range of research methods and techniques

demonstrate is that:

...[A] method in itself is neither valid nor invalid; methods can produce valid data or accounts in

some circumstances and invalid ones in others. (Maxwell, 1992, p. 284)

Therefore, since validity is not a feature of a particular methodology, process or test, within

qualitative research all that remains is how representative the description is and how justifiable

are the findings.

This is why many researchers claim that 'understanding' is more pertinent to qualitative research

than 'validity' (Wolcott, 1992). What is certain is that qualitative research sets itself up for failure

when it attempts to follow the established procedures of quantitative research such as

experimentation, efforts of replication, use of control groups, use of standardised formulas or the

use of the pre-test/post-test method.

One thing that does appear to bear equal importance to the validity of both quantitative and

qualitative enquiries is the reliability of the research, that is, is the research measuring what we

want it to? The means by which this is ensured are different for each methodology, but the

question essentiality remains the same for both.

Some Other Issues and Problems

Valid for Whom?

Another issue that concerns all claims to 'validity' and truth, regardless of the methodology or

issue, is the question for who is the research valid and in whose interest is this claim to truth

being made? The desire to understand and categorise others is implicitly one of power and

control. In 'understanding' we must conflate and model the lives of others and reduce them to a

series of explanations and evaluations. Moreover, these explanations and conceptualisations are

not uniquely formed, but forced to fit our own, pre-existing, conceptual moulds. For many

research projects, it would be of little value or relevance whether the subjects were able to

identify or agree with any of the findings or evaluation constructed beyond the descriptive data.

However, as was established earlier, this can raise ethical problems in itself and requires very

careful handling. It is all too easy for the researcher to act as an omniscient or impartial voice.

Yet, findings and evaluations, however validated by the justification of appropriate methods and

processes, inevitably carry with them the political and philosophical views of the researcher and

the methodology employed. What evolves is a kind of shared or intersubjective truth between the

actual events and the unisubjective 'realities' of the researcher. Any 'reality' perceived or offered

as a result of this process is therefore birthed of the subjective 'reality' of a particular, political

and historical perspective. Theoretical awareness of the social construction of 'reality' has lead to

what Denzin and Lincoln (1998, p. 286) refer to as a 'radical antifunctionalist position.'

This stance claims that knowledge, even the knowledge process, is without grounding, without

authority, and therefore many things 'go.' That is, knowledge itself is no longer the criterion,

because all knowledge claims are based on various assumptions. (Denzin & Lincoln, 1998, p.

286)

However, as the authors continue to point out, 'an assumptionless science is not possible' (Denzin

& Lincoln, 1998, p. 286). Therefore, the purposes of research have had to change in order to

attempt to reduce this problem.

Research is no longer coupled with knowledge, but has been given multiple choices (such as

liberation, emancipation, programmatic politics, expressive 'art'). Depending on one's choice

research is defined accordingly. (Denzin & Lincoln, 1998, p. 286)

This is not to say that certain researchers, in particular in the quantitative field, do not continue to

make assumptions based on what they might consider to be the inevitable results of their

findings.

Another problem raised by the concept of 'validity' centres around the use of existing cultural or

subject oriented terms 'to refer to 'real', natural categories.' (Maxwell, 1992, p. 282). Maxwell

cites the loss of translating these terms to other languages where 'natural' classifications may

connote subtly or wholly different meanings. However, these classifications could also create

problems of subjective misunderstanding within or across native cultures. Terminology presents

problems in the practice and dissemination of research in both qualitative and quantitative

methodologies and can present a serious threat to 'validity', however it is conceptualised.

An Attempt to Understand the Nature of 'Validity' and 'Truth'

As we have seen, establishing a common definition or even core concept of 'validity' is an

immensely difficult process. In order to cut through the surface issues of whether a particular

account is either 'valid' or 'invalid', we must consider the more metaphysical concepts governing

'validity' (regardless of variations in definition or terminology) and propose a thesis as to the

underlying commonality of intention and desire underpinning the attempt to establish or define a

'valid' argument.

Whatever one's choice of terminology, it is the notion of truth and our claim to it that is central to

all perspectives on 'validity'. As I have already established, these notions of truth do not merely

concern the factual events or statements recorded during the data gathering, but the research

process as a whole.

One's notion of truth determines one's definition of accurate representation, reliability and

trustworthiness. Conflicts as to the nature of 'validity' arise from a lack of consensus concerning

a particular theory or philosophy of truth.

However, attempting to establish which truth is more valid, or which 'validity' is more truthful, is

therefore a meaningless exercise and one that would inevitably require adoption of at least one of

the subjective perspectives investigated.

Michel Foucault's theorisation concerning 'truth' elucidates many issues pertinent to this debate.

Contrary to common perception, Foucault does not wholly reject the notion of truth or

objectivity (Seals, 1998). He instead understands 'truth' as elliptical of the much more adequate

phrase 'discourse of truth'. Foucault (1972) argues that statements may be said to be 'truthful' on

three different levels, categorised 'transubjective', 'intersubjective' and 'unisubjective.' In brief,

unisubjective truth concerns truth that is entirely personal to the subject. As Foucault puts it, 'the

conditions of individuation of the subject are in fact very strict, very numerous, and authorise in

this case only one possible subject' (1974, p. 94). Intersubjective truth requires a common

positionality of subjects from which all initiated subjects can share a claim to truthfulness or

falsity. Finally, what Foucault refers to as transubjective truth epitomises what positivists refer to

as objective truth. Foucault offers the example of a mathematical law that exists as true despite

the positionality of any subject. In the realm of transsubjective truth:

The subject of the statement is in a neutral position, indifferent to time space and circumstances,

identical in any linguistic system, and in any code of writing or symbolisation, that any

individual may occupy when affirming such a proposition. (Foucault, 1974, p. 94)

Foucault proceeds to argue that intersubjective truths, when analysed, produce transubjective

truths about subjects and the nature of intersubjectivity (Seals, 1998, p. 61).

The relevance of this for research and in particular claims to 'validity' is that there would appear

to be different forms that truth and 'validity' can take which are relative to the nature or stage of

the research process. If, indeed, a variety of truths exists, then to attempt to classify them all as

merely objective or relative is a mistake at a fundamental level. Seals encapsulates this view:

Statements are distinguishable by objective features and because a different logic or set of rules

for truth determination applies to each type of statement, to treat a given kind of statement in

terms of a logic of truth that does not apply to it is to make a category mistake. (Seals, 1998, p.

62)

It is precisely this 'category mistake' that has created the confusion, misunderstanding and

misapplication of research methods in relation to the concept of 'validity'. By Foucault's

reasoning, it would appear that the correct appropriation of research methods depends on the

nature of the truth that we require, and perhaps more importantly, the truth that is available to us.

Therefore, it would seem that 'validity' relates to the correlation of research methods and the

purposes of the research, rather than any universal or standardised test or procedure. Maxwell

(1992) echoes this correlative relationship:

[T]he ways in which the researchers make these discriminations do not pertain entirely to the

internal coherence, elegance, or plausibility of the account itself, but often refer to the

relationship between the account and something external to it - that is, the phenomena that the

account is about. Validity, in a broad sense, pertains to this relationship between an account and

something outside of that account, whether this thing is construed as objective reality, the

constructions of actors, or a variety of other possible interpretations. (Maxwell, 1992, p. 283)

Whether or not validity is essentially the same concept in qualitative and quantitative research, it

would seem evident that the means by which this is to be achieved are different for each

methodology. However, these means could also be viewed as means to different ends and means

to different 'truths'. From Foucault's definition of the multiplicity of truths, one could suggest that

each different truth inevitably requires different means of validation. It is the means of validation

that should be adjusted depending on the kind of truth that is sought or expected. Therefore,

'validity' appears to reside within the appropriation of research methodologies to those systems of

truth that their processes best represent. As mentioned earlier, one possible test for validity is to

enquire whether the research is measuring what it was intended to measure. Yet this question

may be even more illuminating if we adapt it to enquire, is it measuring the kind of 'truth' it

hoped to measure.

References

Bingham, W. V. D. (1937). Aptitudes and aptitude testing. New York: Harper & Brothers.

Black, J. A., & Champion, D. J. (1976). Methods and issues in social research. New York:

Wiley.

Cronbach, L. J., & Meehl, P. E. (1955). Construct validity in psychological tests.

Psychological Bulletin, 52, 281-302.

Denzin, N. K., & Lincoln, Y. S. (1998). Collecting and interpreting qualitative materials.

London: Sage.

Foucault, M. (1972). The archaeology of knowledge (A. M. Sheridan Smith, Trans). New

York: Harper & Row.

Foucault, M. (1974). Power/knowledge: Selected interviews and other writings. New York:

Pantheon Books.

Guba, E. G., & Lincoln, Y. S. (1989). Fourth generation evaluation. Newbury Park, CA:

Sage.

Hammersley, M. (1987). Some notes on the terms 'validity' and 'reliability. British

Educational Research Journal, 13(1), 73-81.

Johnston, J. M. & Pennypacker, H. S. (1980). Strategies and tactics of human behavioural

research. Hillsdale, NJ: Lawrence Erlbaum Associates.

Kerlinger, F. (1964). Foundations of behavioural research. New York: Holt.

Lehner, P. N. (1979). Handbook of ethological methods. New York: Garland, STPM Press.

Maxwell, J. A. (1992). Understanding and validity in qualitative research. Harvard

Educational Review, 62(3), 279-300.

Mishler, E. G. (1990). Validation in enquiry-guided research: The role of exemplars in

narrative studies. Harvard Educational Review, 60, 415-442.

Runciman, W. G. (1983). A treatise on social theory: Vol 1. The methodology of social

theory. Cambridge: Cambridge University Press.

Seals, G. (1998). Objectively yours, Michel Foucault. Educational Theory, 48(1), 59-66.

Simco, N., & Warin, J. (1997) Validity in image based research: An elaborated illustration of

the issues. British Educational Research Journal, 23(5), 661-673.

Wainer, H., & Braun, H. I. (1988). Test validity. Hilldale, NJ: Lawrence Earlbaum

Associates.

Wolcott, H. F. (1990). On seeking--and rejecting--validity in qualitative research. In E. W.

Eisner & A. Peshkin (Eds.), Qualitative inquiry in education: The continuing debate (pp. 121-

152). New York: Teachers College Press.

Author Note

+ Glyn Winter, MSc, now works as a researcher in outsourcing, business process reengineering

and benchmarking at Manchester Business School - the University of Manchester. At the time of

writing the paper I was completing a MSc in Educational Research within the Education

Department of the Manchester University where I was looking into the use of the linguistic

theory/school of dialogism as a tool to better understand how repondents in an interview

construct reality, truth and attempt to engage with a vast array of dialogues both to make their

views understood and in order to validate them - but even now, this is already a hazy blur to me!

Prior to the MSc I achieved a first class honours BA degree in Education at Liverpool John

Moores University and then worked as a primary school teacher for a while (something which I

miss greatly). Prior to that, I am not ashamed to say, I was either unemployed or else held one of

a string of short-lived and degrading jobs including street cleaning and polishing brass washers

for cookers. The reason for this sudden change in career and intellectual direction remains a

mystery to me! He can be reached at Manchester Business School, Booth Street West,

Manchester, M15 6PB. His telephone number is 0161 275 6322 and his email address is

[email protected] or [email protected].