U8D1 & A1 - Answer Questions regarding Sampling Design & Develop a Sampleing Plan Flow Chart ...see instructions below

Acceptable SOPSL Research Methods and Research Designs

Introduction This document will review research methodologies and designs that are acceptable for dissertation research. The scholar-practitioner model must guide all decisions regarding what is and what is not an acceptable research methodology. The research methodologies and designs presented here offer a more general rather than specific discussion of those applicable approaches to dissertation research. The term research methods is identified in this document as the broadest way in which the methodological concepts can be categorized. The term research design is identified as those overarching methodological approaches that are most commonly referred to as belonging to an umbrella of commonly accepted quantitative, qualitative, or mixed methodologies. As such, research designs are more precise than research methods, but still somewhat vague in the precise manner in which data are measured, collected, and analyzed.

Quantitative Methods Quantitative or fixed research designs are one of the most common approaches for sociologically based dissertation research such as found in SOPSL. For a post-positivist perspective – a researcher tests a theory by specifying hypotheses (Creswell, 2009) - there are three main categorizations (Robson, 2002):

• Experimental research design – The assignment of control and test groups and the manipulation of selected variables while controlling all others.

• Quasi-experimental research design – An experimental approach (control and test groups) where participants are not randomly assigned.

• Non-experimental research design – The phenomena studied is not manipulated but requires further explanation or understanding (Robson, 2002).

The bulk of most doctoral studies pertinent to SOPSL specializations use non- experimental research designs, followed by quasi-experimental research designs, and finally, experimental research designs. SOPSL specializations cover a wide range of academic disciplines. Some disciplines use discipline specific research designs. Correlational designs tend to be more common in public safety research than experimental or quasi-experimental designs, for example, as it is often considered unethical to manipulate the types of variables under study, whereas such non- experimental designs are less common in people-based disciplines where it is both desirable and possible to manipulate variables between participant groups. Designs specific to a particular discipline are acceptable as they meet the specified criteria. If the doctoral researcher can justify with scholarly support that their design meets all of the quantitative criteria, then the research design will be accepted. The criteria for acceptable SOPSL quantitative research designs are as follows:

• Tests or verifies theories of explanations

• Identifies variables to study • Relates variables in research questions and hypotheses • Uses standards of validity and reliability • Observes and measures information numerically • Uses unbiased approaches • Employs statistical techniques (Field, 2009, p, 3)

Note: Quantitative descriptive research designs add little to the existing body of knowledge. Such designs are rarely appropriate for SOPSL dissertation research. If questions arise as to the applicability of a particular research design, then the Mentor should contact the SOPSL Research Chair for direction. Given the large number of non-experimental research designs selected for dissertations, doctoral researchers who seek to better understand phenomena (within SOPSL specializations) tend to use the following approaches:

• Survey Research: The use of surveys or questionnaires constitutes non- experimental research if the doctoral researcher is testing for linear relationships, statistical differences or statistical independence. This approach is very common in correlation research designs used in academic disciplines relevant to SOPSL specializations. These data are subject to internal and external validity considerations. • Internal validity – The assessment of trustworthiness of the data

collected from participants. This includes items such as participant experience, knowledge, attitude and emotional well being

• External validity – The assessment of trustworthiness of data as it applies to other groups, settings, and situations (Field, 2009, p. 11).

• Secondary Data Analysis: The use of secondary data is a non-experimental approach and these data are also subject to internal and external validity considerations. This approach is likewise very common in correlation research designs used in academic disciplines relevant to SOPSL specializations.

Qualitative Methods Qualitative (also called flexible) research designs involve the doctoral researcher in exploratory studies that attempt to explain, define, or better understand phenomena. Variation in established theory is a place from which to begin. These research designs follow a constructivist or advocacy approach (Creswell, 2009). The researcher seeks to establish the meaning of phenomena from the viewpoint of participants (Moustakas, 1994). There are four major categories:

• Phenomenological – The researcher identifies lived experiences associated with the phenomena (requires a strong background in psychology).

• Grounded Theory – The researcher derives a general abstract theory grounded in the views of participants.

• Ethnographic – The researcher studies an intact cultural group over a prolonged period of time.

• Case study – The researcher explores in-depth, a program, event, activity, or process of one or more individuals (Creswell, 2009, p. 13).

• Generic qualitative – The researcher examine the perceptions or attitudes of individuals but in a way that does not align with the other qualitative approaches.

No single research design dominates qualitative research. Key to providing a sound rationale and justification for the chosen qualitative research design is to clearly and explicitly connect with authoritative sources on this design, by citing, discussing, and referencing these sources (e.g., Phenomenology - Moustakas; Grounded Theory – Glaser and Strauss; Case study - Yin, Stake, or another acknowledged advocate-expert). If the doctoral researcher can demonstrate or justify through scholarly support that their design meets the qualitative criteria and that they are sufficiently qualified to conduct the research, then the research design will be accepted. The criteria for acceptable SOPSL qualitative research designs are as follows:

• Creates an agenda for change or reform. • Collaborates with the participants. • Adds to the existing body of knowledge.

Qualitative research provides a wealth of information concerning an observed phenomenon. It is intended to better understand, explain, or define aspects of the research problem of interest. The methodology focuses more on open-ended rather than closed- ended questions.

• The most common instrument used for this type of research is an interview guide. However, each type of qualitative research has a unique and defined process for collecting data based on that design’s accepted methods and procedures.

• Gathering this type of data requires planning, researcher experience and a defined strategy.

• The researcher must be aware of participant feelings, emotions, knowledge and experience. In addition, many of these techniques require that data be obtained on facial expressions, intonation and overall participant temperament.

The setting for conducting qualitative research must be conducive to the research intent (e.g., a setting familiar to the participant).

• The researcher is an “active” part of the research when interacting with participants. It is critical to establish the role of the researcher, their overall experience and expertise on the proposed topic and methodology as they will guide and provide oversight for the research. The researcher is the key measurement instrument. Research bias is always a consideration.

• Are additional consultants needed (e.g., counselors, translators, transcriptionists) to provide additional expertise? If so, these individuals must also adhere to confidentiality protocols as dictated by the IRB.

• Multiple sources of data are common (e.g., interview responses, observations, document sources).

• Participant feelings, emotions and experiences must be captured and analyzed.

• Inductive analysis – The researcher must search for emergent themes, explain and interpret convergent and divergent information.

• Credibility and believability of the results are important. • The results may have transferability or be applied to a larger group similar

to the participants. This is different from the quantitative term of generalizability.

• Mixed Methods Mixed methods research designs involve the doctoral researcher in collecting diverse types of data that best provide an understanding of a research problem (Creswell & Plano Clark 2011). This is not a common approach in many of the academic disciplines pertinent to SOPSL specializations. Doctoral researchers who seek to adopt a mixed methods approach to their dissertation must be familiar and up-to-date with the current literature on mixed methods research in order to satisfy current accepted definitions and standards for conducting this type of research. Similar to the requirement for many types of qualitative research, the doctoral researcher is expected to cite, discuss, and reference authoritative sources in support of numerous design elements (e.g., overall design type, sampling methods, data analysis methods, mixed methods validity and applicability, etc.). The doctoral researcher is obligated to demonstrate an acceptable philosophic understanding and appreciation for the merging of positivist and social-constructivist perspectives within a pragmatist paradigm. This merging of philosophic traditions requires the need to collect, organize, analyze, interpret, and merge both quantitative and qualitative data into one study. The criteria for acceptable SOPSL quantitative research designs are as follows:

• Justification for using a mixed methods approach needs to be based on either the current state of knowledge on the topic and that the nature of the phenomenon is sufficiently complex such that examining it through a single methodological approach would fail to describe and explain the phenomenon.

• Doctoral researchers must demonstrate an understanding of and appreciation for both the positivist and social constructivist traditions, either separately or as combined within the pragmatist tradition.

• The chosen design must align with the study’s stated purpose, goals, and objectives, and be capable of answering the study’s research questions.

• The study’s proposed research questions must meet current standards for acceptable research questions, for instance as described by Tashakkori clearly demonstrating the doctoral researcher’s understanding of crafting acceptable quantitative, qualitative, and mixed-methods research questions.

• The study’s proposed data sources, data collection procedures, and sampling methods must meet current accepted standards supporting mixed methods research.

• Authoritative sources on mixed methods analysis must be cited, discussed, and referenced.

• The doctoral researcher must demonstrate knowledge of how the quality and validity of mixed methods research is established and evaluated.

Mixed methods research represents an exciting approach to examining complex phenomenon more holistically. It is quite possible, however, that the quality of a mixed methods study is based at least in part on the researcher’s systems-thinking capabilities, which is rooted in multiple philosophic perspectives, and who can work with both all types of data to uncover a deeper pattern of meaning through the merging of the quantitative and qualitative data.

• Interviews or extensive existing documents are generally acceptable as qualitative data. Open-ended questions included in a questionnaire, or as part of a predominantly quantitative survey are generally not acceptable.

• Surveys or questionnaires that collect quantitative information including secondary data, if appropriate.

• Sampling methods appropriate to each component, qualitative (e.g., purposeful) and quantitative (e.g., random; Teddlie and Yu, 2007).

• The use of a mixed methods design within a small number of scholar- practitioner approaches to disciplined inquiry—for instance, action research, evaluation research, survey or interview feedback, or appreciative inquiry are acceptable, as long the doctoral researcher demonstrates sufficient knowledge of this approach based on a thorough understanding of the literature and authoritative sources.

Undertaking a mixed methods dissertation is no small undertaking, and the decision to do so should be based on the current state of knowledge or the nature of the phenomenon, and familiarity with the literature on mixed methods research. In addition, the doctoral research should consider,

• Personal or Professional Goals: If a doctoral researcher has a desire to continue conducting research, then the completion of a mixed-methods study might constitute experience in conducting both types.

• Time and Resources: A mixed methods study is going to require additional time and resources.

• Current Philosophic Orientation and Methodological Competencies: The doctoral researcher must be competent to conduct both quantitative and qualitative research.

To gain approval of any dissertation proposal, regardless of methodological approach, the doctoral researcher must demonstrate doctoral-level knowledge and expertise in presenting a scientifically meritorious study. There needs to be consistency and alignment among the research problem, purpose, and research questions, based on a critical review of the literature, and supported by a methodologically sound research design. For mixed

methods research, this ultimately requires two methodological approaches: quantitative and qualitative. Finally, undertaking a mixed methods study requires the support of a mentor or perhaps a committee member who has some familiarity with either mixed methods research specifically, or has sufficient knowledge and skills in both quantitative and qualitative research.