Research Paradigms
Treveonlh
ResearchParadigms.html.zipResearch Paradigms.html
The term ‘research’ is commonly understood to denote a systematic process of discovering more about a topic, using methods that are transparent and publicly defensible. Within this broad definition lie different strands of activity. A key distinction in education and development is between scholarly research and research conducted for consultancy or monitoring and evaluation purposes. Monitoring and evaluation (sometimes referred to as ‘M & E’) is chiefly used in relation to specific projects and usually focuses on the effectiveness of the project in relation to specified (and often narrow) goals. It is unlikely to be asking deeper questions or engaging with the debates in academic literature that scholarly research tends to be concerned with.
Measurement and indicators 1
TEGINT Project
The TEGINT project (Transforming Education for Girls in Nigeria and Tanzania) sought to address gender inequalities in schools and communities through three interventions: the establishment of ‘girls’ clubs’ in schools, of school committees and of participatory, gender-sensitive teaching methodologies. The monitoring and evaluation process, run at regular points during the programme, sought to discover the impact on particular outcomes such as girls’ enrolment and progression. Data collected annually for M & E included gender-disaggregated figures for enrolment by class, exam entry, exam passes, drop-outs, attendance, numbers of teachers and numbers on the school management committees. However, broader research conducted at the start and the end of the project, in conjunction with university partners, attempted to address more complex issues such as the relationships between girls’ club membership, community attitudes and girls’ empowerment (ActionAid 2012).
Epistemological and political positions; decolonizing methodologies; different research traditions
As a researcher, it is always important to consider which assumptions and political positions underpin one’s work. No research (and no researcher) is completely objective and without a position, no matter how unbiased they believe themselves to be. These are particularly important considerations when working on education and development. If the researcher is from a different educational background to that which is being researched, it may lead to some comparative bias, particularly if the context being researched is a developing country. Researchers may become overwhelmed with the deficiencies of the research context and be blind to positive aspects that may have been undervalued or lacking in their own education. It is also important to note that some methodological approaches explicitly and intentionally contain political positions and aims (e.g. feminist research).
A number of theoretical and methodological developments since the 1970s have aimed to reveal and break down hidden assumptions in research. Postcolonial perspectives seek to question the extent to which certain educational constructions relating to the colonial and postcolonial eras may be legitimizing the continuance of unequal global economic arrangements and aim to reveal the assumptions and power relations which pervade research conducted on ‘developing countries’ by Western researchers (Mazrui and Mazrui 1996, Tikly 1999). Tuhwai Smith, in Decolonising Methodologies (2012), identifies research ‘as a significant site of struggle between the interests and ways of knowing of the West and the interests and ways of resisting of the Other’ (Tuhwai Smith 2012, p. 2). She discusses ways in which research has been openly challenged by communities and indigenous activists about ethnocentric assumptions, racist attitudes and practices, and exploitative research. She, and others, have suggested as a way forward the development of alternative methodological approaches located in the ‘South’ (e.g. Halai and William 2011, Park 2011, Robinson-Pant 2013).
Different approaches: Choices and decisions for the researcher
There are a number of research approaches that involve purposively designed strategies to explore specific educational issues in depth. The first two in this section are primarily quantitative approaches and the remainder, qualitative.
Large-scale surveys
Large-scale surveys aim to capture information about educational circumstances from a wide constituency for comparison and analysis, possibly involving surveys of educational conditions in one location across a number of time periods. They may also have a cross-national element, allowing comparison between different countries: for example, data compiled and published by UNESCO in the EFA Global Monitoring Reports and also data from the OECD’s Programme for International Student Assessment (PISA), which has been running (and expanding) since 19971. Such surveys can cover thousands or even millions of individual children, although other projects may be considerably smaller. They may involve the collection of basic educational data (e.g. enrolment, transition, achievement) or more specific questions tailored to particular research agendas, administered via interviews or questionnaires.
Surveys offer some unique and distinct benefits. The scale of data collection attempts to give an overall picture of a particular context and may allow for greater generalizability as well as comparisons over location and/or time. Information that can suggest research questions can be assembled: for example, data to explore teacher effects on student achievement might be suggested by information on teacher qualifications and exam results.
However, there are limitations. Because of the scale of data collection and the claims to generalizability, it is important to scrutinize the underlying assumptions, especially in terms of sampling (have any significant groups been omitted?); the validity of questions (what factors might influence how questions are understood and answered truthfully?); and reliability (will different groups understand and respond to the questions in the same way?). Surveys are highly demanding in terms of time and resources, and for this reason, longitudinal studies, which trace the same individuals over time, are rare (Foster et al. 2012, p. 722).
Broadfoot (2004) warns about some disadvantages in relying on statistical data in education. There are dangers in placing too much emphasis on league tables and ‘rankings’, which may only be based on relatively small differences in scores, and reflect only certain aspects such as exam achievement, as opposed to scrutinizing the actual quality of education. Statistical data must be analysed and interpreted correctly, and we must be wary of their manipulation for particular political agendas to present something in the best possible light. It is also important to beware of the tendency to recourse to numerical data in the mistaken belief that it is more trustworthy or meaningful; there are many instances where other forms of data would be more appropriate for particular research questions. Comparing such educational data across developing countries may be particularly problematic for a number of reasons, such as the varying reliability of data collection, significantly different political and economic trajectories and varied social and cultural contexts.
Young Lives Project
A prominent example of a large-scale survey is the Young Lives project, an international study of childhood poverty, covering 12,000 children in Ethiopia, India, Vietnam and Peru. It is a rare example of a longitudinal project studying education in developing countries and combines regular quantitative surveys of the children and their care-givers with in-depth qualitative research on a sub-sample. The main themes explored are the nature and extent of inequalities between groups of children; the influences of different aspects of poverty on child development; and the risks and opportunities for children of economic and social change (see http://www.younglives.org.uk/).
Experimental approaches: Quasi-experiments and randomized control trials (RCTs)
Experimental approaches explore the effects of particular interventions on a target population by manipulating one or more independent variables and measuring the degree to which this manipulation accounts for variance in outcome. A wide range of experimental designs exist.
Randomized control trials (RCTs) have been imported from medical research to apply to social policy and offer the opportunity to explore the counterfactual (what would have happened if we hadn’t employed those extra teachers/built those schools/provided free uniforms, etc.). RCTs have recently become popular amongst development agencies as they seem to offer clear results to feed back into practice. They feature as part of an agenda to discover ‘what works’, alongside other measures such as impact evaluation and systematic reviews.
Quasi-experiments are generally regarded as less rigorous than randomized control trials but are often used where it is seen as not practicable or reasonable to randomly assign study participants to the treatment condition. The researcher may have some control over the treatment condition but uses criteria other than randomness; or the researcher may have no control as to which groups are assigned to the treatment condition2. Proponents argue that they are important because policy ideas need rigorous testing.
Measurement and indicators 3
RCTs in Development
One example of how RCTs have been used in assessing an educational intervention is in measuring the success of the ‘Progresa’ Programme which provided poor mothers in Mexico with conditional cash transfers based on children’s enrolment in school and access to healthcare3. RCT studies showed not only that the grants had a positive effect on enrolment but were also able to show which group saw the largest increase (girls who had completed grade 6). It was partly on the strength of these findings that the Mexican government decided to preserve and expand the programme (Shultz 2004).
While RCTs can be helpful in demonstrating whether a programme has had a particular impact, it can be difficult to identify why an effect happened, as often attention is not paid to participants’ motivations and the specifics of the context. Morrison (2009) discusses a hypothetical example of CCTV cameras being shown to reduce crime in schools; yet we do not know why crime has fallen (have the cameras led to more thieves being convicted, or encouraged thieves to look elsewhere, or students to be more vigilant or bring in fewer valuables?). Additional work is needed alongside to explain the ‘mechanisms’ behind the outcomes and give insight into whether the initiative would achieve similar effects when implemented elsewhere.
Ethnographic approaches
Ethnographic approaches involve the collection of qualitative data through researchers embedding themselves in the field research environment. The underlying principle guiding this kind of research is the assumption that people have meaning structures that determine much of their behaviour, and the researcher aims to act as a participant-observer to systematically uncover the meanings behind events and actions of what counts as ‘cultural knowledge’. Much of this methodology originated from anthropology and allows exploration of the day-to-day life of the school, the school’s aims and processes, the meanings that various stakeholders attribute to them and the dynamics of its surrounding social contexts.
Typically, data collection involves observing verbal and non-verbal behaviour; scrutinizing the form and content of verbal interactions; recording patterns of action and non-action; and also may involve looking at documents, archival records and artefacts. Conducting this kind of research in a rigorous manner usually involves lengthy periods of time in the field.
In addition to the depths of understanding that ethnographic analyses can offer, this approach may offer a greater opportunity to investigate issues relating to power relations, postcolonial concerns or neo-liberal agendas by exploring in detail the operations and enculturating functions of schools in particular social and historic contexts.
However, there is also an inevitable trade-off between being able to gain deep insights through very focused data collection by spending a prolonged amount of time in one particular context, and the additional insights that can be gained by comparisons between two or more contexts. Moreover, Brewer (2000) has suggested that ethnographic approaches may have been put under threat by globalization and the ‘disappearance of the local under global processes’ (2000, p. 173).
A good example of ethnographic work in education and development is found in Robinson Pant’s work, Why Eat Green Cucumbers at a Time of Dying? (2001). This provides a rich account of two adult women’s literacy projects in Nepal, examining what kind of literacy is being promoted by international agencies and exploring the women’s own perceptions of their involvement in the classes, contrasting these to those of the organizations delivering them. In doing so, she offers a detailed interpretation of the ways in which development ideologies are transmitted and transformed through programme designs, teaching methodologies, delivery and reception. Another recent example is Sriprakash’s work (2012) on the introduction of child-centred pedagogies in low-income contexts in India. Her study used ethnographic methods to investigate how education reforms have been experienced in the state of Karnataka, to explore the extent to which poverty and inequality in rural communities can be addressed by child-centred models and the politics behind such pedagogic reform.
Critical discourse analysis and policy analysis
Critical discourse analysis explores how social and power relations, identities and knowledge are constructed through written, visual and spoken texts. It is particularly concerned with how power abuse, dominance and inequality are enacted, reproduced and resisted. A key underlying assumption is that language operates as a cultural tool, mediating relationships of privilege and power within institutions, in individual interactions and also through bodies of knowledge, with a recognition that science and scholarly discourse themselves are inherently part of and influenced by social and political structures.
Critical discourse analysis is important in helping to unpack and understand the power relations between individuals. It is also very useful in analysing institutional policies, including how policy is formed and interpreted at the many different levels of implementation, for example between international organizations, governments, and local bodies and stakeholders.
Methods for perusing how government policy operates and wields power can involve the analysis of underlying symbols and imagery of policy in the form of spoken statements, publicity, posters and awareness-raising messages, photos and films. Qualitative data is drawn on, although such analysis also examines how quantitative data is used to gain authority and legitimacy.
Researchers consider their own position in the creation of knowledge and therefore existing power relations; research questions may be implicitly influenced by dominant agendas, and a critical position may itself become a dominant discourse in time. While this approach may be successful at revealing mechanisms that seem objectionable, it generally does not suggest solutions.
Vavrus and Seghers’ study (2010) of poverty reduction policies in Tanzania critically analysed how the concept of ‘partnership’ is conceived of by powerful international organizations; how it is constructed in poverty reduction policies; and whether there are genuine opportunities for agency and change in these strategies. Their study reveals the multiple meanings attributed to ‘partnership’ at different levels of policy, shows how the concept is linked to particular elements of ‘revisionist neo-liberalism’ and questions the extent to which the voices of the poor are genuinely taken on board when they are not consistent with existing reform plans.
Biographical and narrative research
Biographical and narrative approaches explore individual experiences of events from the actor’s point of view. They can investigate socio-historical phenomena, engage in cultural analysis and critically reflect on the processes of research through systematic investigation into individual life perceptions and experiences, enabling multiple interpretations of complex cultural contexts.
Interviews allow for collecting narratives and biographical data from participants. Life histories may have a chronology; but they may not record all events and may focus selectively on critical moments and decisions for an individual, thereby complicating factors, evaluation and reflection on outcomes. The narrative analysis subsequently produced by the researcher may include direct quotations from the participants; by ‘telling a story’, these accounts can be extremely powerful, providing a sense of human experience, authenticity and realism. The analysis may also involve eschewing theory (see Farrell in Martin 2003) in the belief that this can impose particular interpretations and distort the intricacies and the complexity of the context under study. It is interesting to note here also the differing epistemological positions in various qualitative approaches: narrative enquiry often is not concerned with generating or clarifying theory; policy analysis may be deeply concerned with theorizing across multiple contexts in order to draw conclusions about inequality and development (Martin 2003, pp. 112–113).
In a policy and intellectual climate which tends to privilege large-scale quantitative research, narrative and biographical analyses have the potential to foreground voice, reflexivity and inter-subjectivity and can be particularly powerful for researchers keen to engage with the perspectives of colonized, marginalized and disadvantaged groups.
Nonetheless, there are some disadvantages; first, there are difficulties in making comparisons with other contexts, since through exploring the specific and abandoning theory, there is a lack of conceptual apparatus with which to do so. Narrative research may also pose particular dilemmas from a postcolonial perspective: Fox (2008) argues that we need to question the ‘right’ of the researcher to represent, publish and benefit from an individual’s narrative; it is also important to be aware of the potentially inequitable power relationship between the researcher and participant while addressing the dilemma of who is seen as the most authentic and legitimate narrators, particularly if they run counter to mainstream views.
