Appraising

“Young Iranian women as agents of social change”: A Critical Appraisal of Qualitative Research

Models of Critique

Critically appraising research represents a guided process of examining the overall quality of evidence, to form judgement of its trustworthiness, value, and relevance (Umesh et al., 2016). Models of critique vary depending on the type of research being appraised. The two main approaches to appraising qualitative research are by checklists, and frameworks. Both approaches in isolation have been subject to criticism, with checklist approaches described as “positivistic” and nonspecific to the qualitative methodology under review (Williams et al., 2020). And whilst framework approaches focus on overarching concepts of transparency, confirmability, reflexivity, dependency, and transferability – the reader must be familiar with such concepts for the approach to be applied and effectively understood (Williams et al., 2020). In the case of the current paper under review, both approaches will be applied to aid and strengthen the appraisal. The CASP for Qualitative Research (Critical Appraisal Skills Programme [CASP], 2017) 10-question checklist will be used, in combination with the readers understandings and application of, a qualitative research framework. This appraisal was also aided by the Qualitative Assessment & Review Instrument (QARI) (Joanna Briggs Institute, 2020).

Authors

The authors of the article under review can be deemed as reliable. All authors have demonstrated research experience in relevant fields to that of the current paper’s focus. The primary author is a member of the Faculty of Health at a well-known Australian University. The author has demonstrated research experience and is a listed author on journal-published papers dated back to 2008. The primary author is also co-affiliated with other well-known Universities, and in completing the research paper under review was supported by a Queensland-based research institute concerned with individual, family, and community-based health which is relevant to the study aim (Menzies Health Institute Queensland, 2021). Other listed authors have demonstrated experience across related research topics and are widely cited in the literature. The paper clearly states the principal author’s cultural link to the country of Iran and the city whereby study participants were recruited and interviewed, and the author’s involvement with data collection, described as “mutual influence” is mentioned briefly (Salehi et al., 2020). There is no mention of how the author’s cultural position may have influenced different stages of the research and if such influence was acknowledged, recorded, and reflected upon which would have aided reflexivity.

Title/Abstract

The article’s title “Young Iranian women as agents of social change: A qualitative study” pertains to a major theme identified throughout the research (Salehi et al., 2020). The title is not explicitly suggestive of the primary research aim which was to explore the perspectives of young Iranian women to understand how they explore and/or negotiate their social situation (Salehi et al., 2020). The abstract clearly states the identified gap in the literature but does not mention why the research is relevant. A qualitative research design and the research aims are stated however there is no mention of the adopted methodology within the abstract. The abstract includes a succinct summary of the results, and the stated relevance of the research findings are clear and concise.

Journal

The article under review was published in January 2020 by Elsevier, in the journal Women’s Studies International Forum. The journal holds a H-index of 59, meaning the journal has published at least 59 articles that have all been cited at least once (SCImago Journal and Country Rank [SJR], 2020). This is a measure of not only the journal’s productivity, but it’s overall impact (Spicer, 2015). As of 2021, the journal had an impact factor of 1.497 ranking it among the top 40% of journals in the field of social sciences (SJR, 2020).

Introduction/Background

The introduction provides a clear background and effectively gives a wider context to the current research aim by discussing literature related to the topic. The introduction starts with a brief summary stating the link between general wellbeing and social capital (Salehi et al., 2020). The article’s primary author is cited throughout supporting literature used in the introduction, indicating the researcher’s prior experience across related research fields. The introduction then highlights the issue of specific groups being more vulnerable to social exclusion, inequality, and poorer health outcomes typically as a result of poorer connection to their society (Salehi et al., 2020). Iranian women are then introduced as the focus population, with the support of available evidence to state that this subpopulation experience unequal status in their society (Salehi et al., 2020). There is brief mention of some social determinants that are known to bring about this social inequality, which is then tied to the recommendations stated throughout the majority of related literature. Such recommendations are noted to be mostly external and include strategies such as policy change, social learning opportunities and tertiary education to improve social capital (Salehi et al., 2020). The concept of internal experience is then introduced, with evidence to support women as influential agents of social change through channeling their internal power despite the restrictive nature of their societies. The introduction then merges all main points discussed to that point, to illustrate a gap in the literature, that is, there is little evidence investigating the role of Iranian women’s internal experience as builders of their own lives within a socially restrictive context. The aim of the current study is then stated “to explore how young women living in Iran negotiate social and cultural barriers in order to change their social roles and improve the collective status of women” (Salehi et al., 2020).

Methodology

The article clearly states the research to be the qualitative component of a larger mixed method research program (Salehi et al., 2020). Although not clearly stated, the study adopts an approach known as descriptive phenomenology (Matua & Van Der Wal, 2015). As is true for all forms of qualitative research, the chosen approach represents both a research method as well as a philosophical underpinning and it is critical that the researcher(s) understand this (Lopez & Willis, 2004). Again, although not directly stated, the research appears guided by an overarching emancipatory paradigm, that is, the research seeks to produce knowledge that can be of benefit to disadvantaged individuals or groups (Noel, 2016). In proportion to the remaining body of the paper, details pertaining to the methodology are brief. Clear examination of, and a statement linking the method used to the guiding philosophical underpinning reduces ambiguity in the research’s purpose, structure, and findings (Lopez & Willis, 2004).

Congruence between the methodology used and other study aspects

The QARI tool question two addresses congruence between the adopted methodology and the research question (QARI, 2020). Phenomenology seeks to explore in-depth understandings of everyday experience (Matua & Van Der Wal, 2015). As the research seek to explore the phenomena of daily life as experienced by young Iranian women, phenomenology was appropriate and congruent to the research aim. Question three of the QARI tool addresses congruence between research methodology and data collection methods (QARI, 2020). Data was collected via questionnaires using open-ended questions, as well as by semi-structured interviews guided by seeded open-ended questions (Salehi et al., 2020). Collection of data via face-to-face interviews is both preferred, and congruent with phenomenology (Ataro, 2020). Face-to-face interviews are preferred in descriptive phenomenology as they offer “rich, first-person accounts of experience” (Ataro, 2020). Data collection methods are also congruent with a descriptive phenomenological approach whereby interviews were audio-recorded (and not video recorded for interpretation of non-verbal cues). There is no mention of the researcher encouraging deeper descriptions of participant experience throughout interviews (i.e. sensations of the body and experience of time), which again is congruent with descriptive phenomenology (Lopez & Willis, 2004). Data analysis was guided by the Giorgi method, a stepwise descriptive approach to phenomenology thus demonstrating congruence to the methodology as flagged in question four of the QARI tool (Giorgi, 1999; QARI, 2020). Finally, question five of the QARI tool addresses congruence between methodology and the interpretation of results (QARI, 2020). Congruence is demonstrated well whereby the methodology sought to bring about an understanding of “how young women living in Iran negotiate social and cultural barriers in order to change their social roles and improve the collective status of women” (Salehi et al., 2020). The results were interpreted to bring a deeper understanding to Iranian women, other researchers and policy makers about how strong and continuous social changes can occur (Salehi et al., 2020). Interpretation of results also demonstrates congruence to an overarching emancipatory paradigm whereby the research findings may further improve the social conditions experienced by young Iranian women (Noel, 2016).

Is the methodology trustworthy?

Rigor in qualitative research is often referred to as trustworthiness, that is, the degree of confidence found in research findings (Williams et al., 2020). The CASP tool is useful in assessing rigor, or qualitative trustworthiness in the case of the research under review (Long et al., 2020). In addressing question two of the CASP tool (CASP, 2017), a qualitative research design was appropriate as the research sought to explore individual experience – a phenomena which cannot be expressed quantitatively. In addressing question three of the CASP tool (CASP, 2017), whilst the research design was appropriate in addressing the research question (as discussed previously), the researchers do not provide justification of the chosen research design, a strategy which otherwise would have aided dependability. Question four of the CASP tool addresses whether recruitment strategies were appropriate to the research aims (CASP, 2017). Recruitment strategy was appropriate and was discussed thoroughly in the article. Justification of the sampling method is provided and additional information pertaining to recruitment methods are discussed in-depth under the heading “ location, participants and procedures” (Salehi et al., 2020). There is no statement as to why 17 participants were interviewed of the 70 or more individuals who indicated a willingness to participate. Doing so would have again improved rigor by aiding dependability. With reference to question five of the CASP tool (CASP, 2017), data collection is described well. Researchers clearly stated where and how data was collected and for how long interviews ran in duration (Salehi et al., 2020). Interview questions are provided, and data saturation is mentioned. However, with the exception of a brief statement regarding the interview atmosphere promoted for the purpose of data collection, justification of the research methods is very brief. Justification of the research methodology, as well as chosen data collection and analysis methods is not provided.

Results

Data throughout the article is presented in the form of descriptions of participant’s experiences, with heavy use of direct participant quotations to support the identification of themes (Salehi et al., 2020). The results are presented across two main themes: “multiple identities” and “young women as agents of social change” (Salehi et al., 2020). The later theme comprises the majority of the findings and encompasses four sub-themes that signify the concept of women as agents of social change (Salehi et al., 2020).

Data Analysis

Referring to question seven of the CASP tool (CASP, 2017), the researchers state the research’s ethics approval within the methods section, as well as a footnote on page one of the article (Salehi et al., 2020). There is no discussion as to how the research was explained to participants to assess whether ethical standards were maintained, nor is there discussion around how informed consent was gained, how confidentiality was maintained, or how research data was stored and kept secure. Excluding such information throughout the article negatively influences the research’s overall trustworthiness. Question eight of the CASP tool addresses rigour throughout the data analysis process (CASP, 2017). Thematic analysis was conducted, a method Braun and Clarke (2006) define as a way of identifying, analysing, interpreting, and describing themes in the data set. Specifically, researchers adopted the Giorgi method for thematic analysis of which they state clearly and provide background to in the article (Salehi et al., 2020). The Giorgi method for data analysis is congruent with a descriptive phenomenological methodology as illustrated by its four distinct characteristics: (i) it is descriptive, (ii) it uses reduction, (iii) it searches for essences, and (iv) it focuses on intentionality (Giorgi, 1999). There is also sufficient and clear data presented to sufficiently support findings. There is no mention throughout the article of techniques such as bracketing which represents a weakness in the study particularly given the adopted methodology. Bracketing is a central technique to descriptive, qualitative, and exploratory research whereby the researcher(s) acknowledge and mitigate the influence of personal bias, preconceptions, and personal knowledge on the research process as a whole (Lopez & Willis, 2004). Bracketing is critical throughout both data collection and data analysis, even more so in the instance of this research, considering the primary researcher’s cultural background, gender and prior understanding of life and work conditions throughout Iran (Salehi et al., 2020). Failure to address (or include discussion of) potential researcher bias negatively impacts the confirmability of research findings. Further in relation to credibility of the data analysis, there is no statement as to whether research was independently analysed by other members of the research team, or whether participants were later asked to validate findings (member checking).

Data Presentation

Data is presented heavily in the form of direct quotations as expressed by participants throughout interviews, thus aiding credibility of the research. Data presented sufficiently supports the research findings, and quotations are relevant to the overarching themes identified and outlined throughout the article. Whilst not stated in-depth throughout this particular article, detailed information regarding participant demographics is available in a previously published paper (Salehi et al., 2016). Provision of such information aids transferability of the findings. There is a brief statement declaring there was no conflict of interest, however further information to evidence the examination of researcher(s) bias would have aided confirmability and overall rigour. It is worth noting that there is no identification of who represents the presented data, which is usually achieved by allocating participants pseudonyms, letters or numbers (i.e. participant A). Doing so would have made clear to the reader that the data is in fact representative of a range of views among the sample.

Discussion & Conclusion

With reference to question nine of the CASP tool (CASP, 2017), the discussion is lengthy and provides an in-depth overview of the results with adequate and relevant evidence to support the findings. There is no mention of evidence against the findings of this research, however it should be acknowledged there exists a scarcity of evidence studying similar populations, particularly research of a qualitative nature. The discussion is structured similar to that of the results, with first discussion of the initial theme (“multiple identities”), and later, more extensive discussion of the dominant theme (“young women as agents of social change”) (Salehi et al., 2020). The authors do not discuss credibility of their findings; however, triangulation has been demonstrated well throughout the discussion. Triangulation refers to the use of multiple data sources to better understand the studied phenomenon, allowing for interpretation of findings from a range of perspectives (Carter et al., 2014). An example within the discussion is reference to Giele (2003), whereby related research regarding the internal power and influence of women is incorporated to better understand how findings of the current research can be explained (Salehi et al., 2020). The findings are discussed such that there is clear reference back to the original research question, which was: How do young women living in Iran negotiate social and cultural barriers in order to change their social roles and improve the collective status of women? (Salehi et al., 2020). As emphasized heavily throughout the discussion (and as an oversimplified answer to the research question), they [Iranian women] focus on their internal power and influence to be creators and changers of their own lives as well as that of the wider society (Salehi et al., 2020).

Recommendations

There is no subheading for recommendations in the article. Instead, recommendations and overall relevance of the research is woven throughout the discussion. There is also prior support of participant quotations throughout the results section, which in themselves provide rich accounts detailing the need for improved social conditions for young Iranian women (Salehi et al., 2020). The findings are considered well in relation to current policy in Iran, and the discussion refers to recent policy changes in favour of women, drawing links to results of the current study. The overarching recommendation can be interpreted as the need for a shift in view toward Iranian women, from “powerless victims of paternalistic societies” to “builders of social change and rebuilders of their social capital” (Salehi et al., 2020). There are recommendations specific to policy makers regarding change at a government level to complement the already present influence of Iranian women as creators of their own lives and societal change (Salehi et al., 2020). The relevance of the study is discussed briefly, and key stakeholders are identified as Iranian women, social workers, other researchers, and policy makers. Strengths and limitations of the study are discussed, and there is a brief recommendation for future research to involve larger and more culturally diverse samples (Salehi et al., 2020).

Conclusion

Question ten of the QARI tool appraises whether the conclusions drawn are based on the data collected (QARI, 2020). The conclusion again reinforces the overarching theme of viewing Iranian women as creators of their own social capital, a message that is embedded well throughout the data. The conclusion ties back to the building blocks that generate social capital (bonding, bridging, and linking), a concept first mentioned in the introduction (Salehi et al., 2020). The authors reinforce the need to view young Iranian women as intrinsically motivated individuals as builders of their own social capital. The authors provide summary of implications of the research and highlight how change can be supported by key stakeholders in favour of the studied population.

References

Ataro, G. (2020). Methods, methodological challenges and lesson learned from phenomenological study about OSCE experience: Overview of paradigm-driven qualitative approach in medical education. Annals of Medicine and Surgery, 49, 19-23. https://doi.org/https://doi.org/10.1016/j.amsu.2019.11.013

Braun, V., & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative Research in Psychology, 3(2), 77-101. https://doi.org/10.1191/1478088706qp063oa

Carter, N., Bryant-Lukosius, D., DiCenso, A., Blythe, J., & Neville, A. J. (2014). The use of triangulation in qualitative research. Oncology Nursing Forum, 41(5), 545-547. https://doi.org/10.1188/14.Onf.545-547

Critical Appraisal Skills Programme. (2017). CASP Checklists. http://www.casp-uk.net/casp-tools-checklists

Giele, J. Z. (2003). Men and women as agents of change in their own life. Advances in Life Course Research, 8, 299-317. https://doi.org/https://doi.org/10.1016/S1040-2608(03)08014-6

Giorgi, A. (1999). A phenomenological perspective on some phenomenographic results on learning. Journal of Phenomenological Psychology, 30(2), 68-93. https://doi.org/https://doi.org/10.1163/156916299X00110

Joanna Briggs Institute. (2020). Checklist for Qualitative Research [QARI]. https://jbi.global/critical-appraisal-tools

Long, H. A., French, D. P., & Brooks, J. M. (2020). Optimising the value of the critical appraisal skills programme (CASP) tool for quality appraisal in qualitative evidence synthesis. Research Methods in Medicine & Health Sciences, 1(1), 31-42. https://doi.org/10.1177/2632084320947559

Lopez, K. A., & Willis, D. G. (2004). Descriptive versus interpretive phenomenology: their contributions to nursing knowledge. Qualitative Health Research, 14(5), 726-735. https://doi.org/10.1177/1049732304263638

Matua, G. A., & Van Der Wal, D. M. (2015). Differentiating between descriptive and interpretive phenomenological research approaches. Nurse Reserves, 22(6), 22-27. https://doi.org/10.7748/nr.22.6.22.e1344

Menzies Health Institute Queensland. (2021). Translating innovative health research into better health outcomes. https://www.griffith.edu.au/menzies-health-institute-queensland

Noel, L.-A. (2016). Promoting an emancipatory research paradigm in Design Education and Practice DRS 2016, Brighton, UK.

Salehi, A., Harris, N., Coyne, E., & Sebar, B. (2016). Perceived control and self-efficacy, subjective well-being and lifestyle behaviours in young Iranian women. Journal of Health Psychology, 21(7), 1415-1425. https://doi.org/10.1177/1359105314554818

Salehi, A., Sebar, B., Whitehead, D., Hatam, N., Coyne, E., & Harris, N. (2020). Young Iranian women as agents of social change: A qualitative study. Women's studies international forum, 79, 102341. https://doi.org/10.1016/j.wsif.2020.102341

SCImago Journal and Country Rank. (2020). Women's Studies International Forum. Retrieved 3 August from https://www.scimagojr.com/journalsearch.php?q=24359&tip=sid&clean=0

Spicer, A. (2015). Explainer: what is an H-index and how is it calculated? The Conversation. Retrieved 4 August from https://theconversation.com/explainer-what-is-an-h-index-and-how-is-it-calculated-41162

Umesh, G., Karippacheril, J. G., & Magazine, R. (2016). Critical appraisal of published literature. Indian journal of anaesthesia, 60(9), 670-673. https://doi.org/10.4103/0019-5049.190624

Williams, V., Boylan, A.-M., & Nunan, D. (2020). Critical appraisal of qualitative research: necessity, partialities and the issue of bias. BMJ Evidence - Based Medicine, 25(1), 9. https://doi.org/http://dx.doi.org/10.1136/bmjebm-2018-111132