Chapter 3

profileDr. Williams
Chapter3.Example.docx

Chapter 3: Methodology

Introduction

The purpose of this quantitative causal-comparative study was to identify if or to what extent there were differences between the MHL of African clergy who have at least a four-year bachelor's degree or higher and those who do not have at least a four-year bachelor's degree. This study's design allowed for the comparisons between participants sorted into two groups; individuals with at least a four-year bachelor's degree or higher and those without a four-year bachelor's degree. The chapter included an overview of the research design and rationale, study participants, sampling method and instrumentation, data collection, analysis, and ethical considerations taken in the design. This study addressed the gap in the literature concerning the MHL of African American clergy. Also, it examines if there was a between African American clergy who have at least a four-year bachelor's degree or higher and those who do not have at least a four-year bachelor's degree.

Chapter 3 contains a descriptive discussion of the conduct of this study and how

it informed the problem. The detailed explanation supports future design replication, data collection, and analysis. The description of the population and sample ensured that the reader could understand the research subjects. The MHLS data collection tool allowed valid and reliable data collection. As described, data analysis procedures followed ethical practices. This chapter's discussion on limitations and delimitations expands the discussion in Chapter 1.

Purpose of the Study

The purpose of this quantitative causal-comparative study was to examine the MHL levels of African American clergy in Arizona and to see if and to what extent a difference exists in A) knowledge of professional help available, B) knowledge of risk factors and causes, C) knowledge of self-treatment, based upon the MHLS Questionnaire (O’Connor & Casey, 2015) between African American clergy who have at least a four-year bachelor’s degree or higher and those who do not have at least a four-year bachelor’s degree. The specific independent variable is (a) level of education (those who did and did not receive a four-year bachelor’s degree). Given the high number of African American individuals who turn to African American clergy for general assistance with their mental health issues and the lack of available research about the MHL of African American clergy, this study is particularly timely. This study added to the current literature by investigating the MHL of African American clergy. As more information and research are made available to the churches and mental health professionals regarding the MHL of African American clergy, it allows them to become better equipped to offer services to individuals in churches dealing with mental health illnesses (Gorczynski et al., 2017).

This study explored the levels of (MHL) between African American clergy with at least a four-year bachelor’s degree and those without a four-year bachelor’s degree. The participants of the study are Black clergy, predominantly African American churches. This group comprises African American or Black clergy, including ministers, elders, and pastors, both male and female, in Arizona. The sample size for this study was calculated through G*Power analysis, then increased by 15 % to account for possible attrition, and an additional 15% for the possible nonparametric test is 36 clergy (see Appendix E). Using sufficient statistical power, the sample size allowed the researcher to determine the differences between groups. Participants completed a 35-question scale-based measure that measured MHL (O’Connor & Casey, 2015). This study used the online platform SurveyMonkey to administer the survey. The specific steps which were taken to address the research questions are described in this chapter. This chapter is written to provide a logical flow from the problem statement and its background to the data analysis. Included in the chapter is a review of the purpose of the study, the identified problem, and how the quantitative method addressed the research questions.

Research Questions and Hypotheses

In the African American community, religion and spirituality serve as a resource for coping with mental health challenges and protective factors against various mental health challenges (White, 2017). For this study, several variables were considered independent and dependent variables. The education of African American clergy was the independent variable. This researcher believed that the key issue that identified the differences between the African American clergy is the level of education. Three dependent variables were considered for this study: the knowledge of professional help available, knowledge of risk factors and causes, and knowledge of self-treatment as seen in below in Table 2. Several issues influence clergy mental health literacy. The purpose of this quantitative causal-comparative study is to determine the mental health literacy of African American clergy. This study dove into the MHL of African American clergy and the level of education made on the MHL. This sample size consisted of 36 African American clergy in Arizona who are pastors of predominantly African American churches. The following research questions guide this quantitative study:

RQ1: If and to what extent does a difference exist in the knowledge of professional help available between African American clergy who completed at least a four-year bachelor’s degree and those who have not completed a least a four-year bachelor’s degree?

H1o: A statistically significant difference does not exist in the knowledge of professional help available between African American clergy who completed at least a four-year bachelor’s degree and those who have not completed a least a four-year bachelor’s degree?

H1a: A statistically significant difference does exist in the knowledge of professional help available between AA clergy who completed at least a four-year bachelor’s degree and those who have not completed a least a four-year bachelor’s degree?

RQ2: If and to what extent does a difference exist in knowledge of risk factors and causes between African American clergy who completed at least a four-year bachelor’s degree and those who have not completed a least a four-year bachelor’s degree?

H1o: A statistically significant difference does not exist in knowledge of risk factors and causes between African American clergy who completed at least a four-year bachelor’s degree and those who have not completed a least a four-year bachelor’s degree.

H1a: A statistically significant difference does exist in knowledge of risk factors and causes between African American clergy who completed at least a four-year bachelor’s degree and those who have not completed a least a four-year bachelor’s degree.

RQ3: If and to what extent does a difference exist in knowledge of self-treatment between African American clergy who completed at least a four-year bachelor’s degree and those who have not completed a least a four-year bachelor’s degree?

H1o: A statistically significant difference does not exist in knowledge of self-treatment between African American clergy who completed at least a four-year bachelor’s degree and those who have not completed a least a four-year bachelor’s degree.

H1a: A statistically significant difference does exist in knowledge of self-treatment between African American clergy who completed at least a four-year bachelor’s degree and those who have not completed a least a four-year bachelor’s degree.

Table 2. Variable Table

Variable

Conceptual Definition

Operational Definition

Measurement Level

Instrument/Data Source

Education

(Independent)

Do you have at least a bachelor’s degree?

Yes or No

Nominal

Survey Response

Knowledge of professional help available

(dependent)

Knowledge of mental health professionals and the services they provide and how to connect

The mean score on the knowledge of professional help available subscale of the MHLS

Ordinal Scale

Mental Health Literacy Scale (MHLS)

Knowledge of risk factors and causes

(dependent)

Knowledge of self-treatment

(dependent)

Knowledge of environmental, social, familial or biological factors that increase the risk of developing a mental illness

Knowledge of self-treatment and where to find mental health services.

The mean score on the Knowledge of risk factors subscale of the MHLS

The mean score on the knowledge of self-treatment subscale of the MHLS

Ordinal Scale

Ordinal Scale

Mental Health Literacy Scale (MHLS)

Mental Health Literacy Scale (MHLS)

Rationale for a Quantitative Methodology

This study used a quantitative methodology. The quantitative approach was chosen based on the gap in the literature and the need to help determine the mental health literacy of African American clergy. The quantitative methodology and the research questions best answered the problem statement presented. The rationale for selecting a quantitative methodology was that this approach allowed the researcher to analyze data statistically to understand a relationship or differences between four variables. The four variables in question were the dependent variables of the following abilities knowledge of self-treatment, knowledge of risk factors and causes, knowledge of professional help available, and the independent variable of education level. The causal-comparative design determines whether individuals in one group differ from individuals in another group based on one or more variables (Umstead & Mayton, 2018). The causal-comparative design seeks to find any present relationships between independent and dependent variables only after the event has already happened (Fraenkel & Wallen, 2009).

The identified gap in the literature calls for an assessment to see if a difference exists between clergy who have completed at least a four-year bachelor's degree and those who have not completed a least a four-year bachelor's degree. Statistical differences that are measured and analyzed are the foundation that quantitative methodologies are built (Apuke, 2017). Determining these clergy's mental health literacy helped address the lack of mental health care support received from African American clergy. Quantitative data would work best for this research because the outcomes from this research are easily measured and seen by objectifying the data. Quantitative methodology is appropriate when the researcher’s collect variables and inferences from particular samples of quantifiable populations (Queiros et al., 2018). The results from a quantitative method are more objective than those from a qualitative method, which makes the quantitative method best suited for foundational research (Doucette, 2017).

Quantitative research allows the researcher to use structured procedures and formal instruments to collect data (Queiros et al., 2018). All quantitative analysis requires instruments to measure how and to what extent variables change when affected by another variable (Howell, 2010; Martin & Bridgmon, 2012). Validated tools address the research questions and hypotheses when conducting quantitative research. Myers and Powers (2017) share that the biases common to qualitative observational research decrease when using validated instruments. Quantitative methods provide for analyzing data gathered through polls, questionnaires, and surveys (Mellinger & Hanson, 2020). Quantitative methods also focus on using data that the researcher can generalize across several groups (Roni et al., 2020). Quantitative data are interpreted by conducting statistical analysis. Statistics are based on mathematic principles and are considered scientifically objective (Ong & Puteh, 2017).

Quantitative research tends to focus on the social sciences (mehrad & Tahriri Zangeneh, 2021). Quantitative research deals with numerical analysis when collecting data (Goertzen, 2017). Researchers who use quantitative methodology aim to discover general patterns or phenomena across various settings. So, a quantitative methodology is the best way to determine how much each cleric knows. Since this study used the mental health literacy scale, it produced quantitative data that can be put into categories to verify or justify the research.

Rationale for Research Design

A research design provides the logic or plan for conducting research (Baran, 2020). Four designs are associated with the quantitative methodology: causal-comparative, correlational, experimental, and quasi-experimental. The causal-comparative design was the best approach to answer the research questions, determine statistically significant differences in the collected survey data, and analyze the data set. A causal-comparative design was chosen for this study, which investigates the effects of an independent variable on a dependent variable by comparing two or more groups (Brewer & Kibn, 2010). Correlative design is another closely related research design option for this study. The difference we find between causal-comparative and correlational designs is that when choosing the causal-comparative, the groups have been formed (Fraenkel & Wallen, 2009). The correlational design for research does not look for differences between various groups but seeks to find relationships within a single group (Fraenkel & Wallen, 2009). According to researchers, true experimental designs are the highest level of research and allow the researcher to establish cause and effect by manipulating the independent variables (Slack & Draugalis, 2001). There are similarities between these design options, which include the inability to manipulate the independent variable and the lack of control groups, limiting the generalizability of the results (Brewer & Kibn, 2010).

The descriptive design was considered for this study. Descriptive studies measure specific population characteristics and determine if there is any association (Kelley et al., 2003). While it was necessary to gather data from the sample, and descriptive statistics were performed, this level of analysis could not answer the research questions. The following section discussed the population and sample section for this study.

The causal-comparative design method was appropriate in this study because it is used to identify cause-effect relationships that may be present. A causal-comparative design is a research methodology; compared to the other designs, the quantitative causal-comparative method explores the relationship between two or more groups and one independent variable. A causal-comparative survey design allows comparing two subjects to assess differences (Nardi, 2018). The causal-comparative design looks retrospectively at whether an event caused something to occur, allowing subjects to be grouped immediately (Brewer & Kibn, 2010). The causal-comparative design also seeks to find a relationship between variables after specific actions or events have occurred. According to Lodico et al. (2010), a causal-comparative design effectively evaluates the relationship or consequences of existing differences between two groups. This design would be best because it allows the researcher to choose participants who already belong to a particular group that the researcher is interested in further studying (Blackstone, 2018).

Causal-comparative designs are best used when the researcher seeks to establish an association between two or more variables (Nardi, 2018) as seen in Table 3. To justify the use of causal-comparative methods, the problem statement and the research questions were considered by the researcher. To address the research questions in this study, the researcher conducted closed-end surveys that examined the MHL of African American clergy. There are a few challenges that are present when the researcher uses the causal-comparative design. When considering the causal-comparative design, issues such as lack of randomization, inability to manipulate independent variables, and focus on standardized performance (such as tests) must be addressed. The possibility is that the groups being studied may differ on some other significant variables besides the target variable of interest. This other variable may be why there is an observed difference between the groups.

One single dependent variable was identified for this study: the education of African American clergy. There were three independent variables for this study; If and to what extent does a difference exist in the knowledge of professional help available between African American clergy who completed at least a four-year bachelor’s degree and those who have not completed a least a four-year bachelor’s degree? If and to what extent does a difference exist in knowledge of risk factors and causes between African American clergy who completed at least a four-year bachelor’s degree and those who have not completed a least a four-year bachelor’s degree? If and to what extent does a difference exist in knowledge of professional help available between African American clergy who completed at least a four-year bachelor’s degree and those who have not completed a least a four-year bachelor’s degree? Qualitative data analysis is the interpretation of data retrieved from other documentation received from validated instruments. This researcher also considered different research designs for this study: descriptive design, experimental design, and correlational design. The descriptive designs are not used to test hypotheses about a phenomenon; however, this design is used to gather information about your target population (Bloomfield & Fisher, 2019).

Table 3. Quantitative Core Designs and Descriptions

Design

Description

Pre-Experimental

Examines the effect/outcome of some form of treatment(s) using either one or two pre-existing group of participants. May use a one-shot comparison group (not a control group measured pre and post). Uses primary data (i.e., data collected by the learner).

Quasi-Experimental

Examines the effect/outcome of some form of treatment(s) using either one or two pre-existing group of participants. May use a control group measured pre and post. Uses primary data (i.e., data collected by the learner). Note: lack of random sampling is key in quasi-experimental designs, differentiating it from a true experimental study design where a random sampling is required.

Correlational or Associative

Examines relationship(s) between pairs of variables using data from a single group of participants with the intent of assessing the direction and strength of a relationship. Can use primary (i.e., collected by the learner) and/or secondary data (i.e., not collected by the learner).

Correlational-predictive

Examines relationship(s) between two or more variables using data from a single group of participants, with the intent of predicting a criterion variable from one or more predictor variables. Can use primary (i.e., collected by the learner) and/or secondary data (i.e., not collected by the learner).

Comparative

Examines differences between two or more groups defined by one or more categorical variables and/or between two or more measurements of a single group. Uses primary data (i.e., collected by the learner) and there is no manipulation of variables.

Ex Post Facto

Examines differences between two or more groups defined by one or more categorical variables and/or between two or more measurements of a single group. Uses secondary data (i.e., not collected by the learner).

Population and Sample Selection

For this study, the general population in Arizona was African American or Black Clergy, including ministers, elders, and pastors, both male and female. All these clerics are associated with congregations in the state of Arizona. Various Christian denominations included Baptist, non–denominational, Methodist, Pentecostal, Apostolic, and other Christian denominations not explicitly named.

The target population of this study consisted of African American clergy located in Arizona who are members of the African American Christian Clergy Coalition. Convenience sampling was used to find participants for this study. As Etikan et al. (2016) noted, convenience sampling is a type of nonprobability or nonrandom sampling where members of the target population meet specific criteria, such as being easily accessible to the researcher or willing to participate in a research study. Convenience sampling is often used in quantitative research studies. However, convenience sampling can sometimes result in lower response rates.

To obtain site authorization (see Appendix B), an authorization request letter was sent to the AACCC director of operations. The authorization request included a brief description of the study, goals for the study, and sources of data to be collected. A timeline for the study was provided along with this researcher's commitment to providing aggregate data from the study after completion.

Quantitative Sample Size

Due to the size of the target population, a smaller sample size was used. The researcher used a convenience sample for this study. Convenience sampling is nonprobability sampling, where members of the target population meet specific practical criteria such as easy accessibility or willingness to participate in a study (Etikan et al., 2016). The convenience sample for this study came from members of the African American Christian Clergy Coalition who met the criteria described in the target population.

When conducting quantitative research, the sample size calculation is based on the researcher's needed effect size, which is the difference between the mean responses of the two groups, the alpha error or false positive error, and the statistical power (Gogtay, 2010). An a priori analysis was conducted utilizing G*Power 3.1.9.4 software to determine the minimum necessary sample size for this study to achieve significance. The analysis was completed using a small effect size .5, a statistical power of 0.80, and an alpha error of 0.05. The minimum sample size for the one-way MANOVA was 28 participants (Figure E2). To account for attrition, 15% was added and 15% for a possible nonparametric test to the minimum sample size, which should reflect a total of 36 in the final sample.

This study included 36 participants. Participants completed a 35-question scale-based measure used to measure the MHLS Questionnaire (O'Connor & Casey, 2015). This study utilized a Web-based administration.

Recruiting and Sampling Strategy

Participants for this research were recruited from the largest organization of African American Clergy in Arizona. These individuals were invited to participate in this study during the monthly meeting. These meetings were currently being held in a hybrid method; the sessions were in-person and via Zoom. This verbal invitation also came with a written flyer asking for their participation in this study and explaining its purpose and overall goal. Emails were also sent out by the director of operations to every organization member, allowing them to participate in this study. This was done to account for those who may not be present in the meeting the day the invitation is given to all the clergy. However, this coalition represents the largest number of organized Black clergy in Arizona. The researcher realized that all African American clergy were not part of this group. To account for those who were not a part of this organization, the invitation to participate in this study was posted on various social media sites, mainly targeting churches and religious organizations in Arizona with African American clergy within their leadership. We designated one month to solicit individuals to participate in this study. The content of the emails included a description of the overall research and a brief description of the actual MHLS that they were asked to complete.

The researcher gathered information from a large and heterogeneous clergy population in this dissertation study. Given the challenges before this researcher in contacting all African American clergy in Arizona (the target population), the researcher applied the non-probability sampling strategies using a specified sampling frame. Non-probability sampling procedures include convenience, purposive, or quota sampling (McEwan, 2020). Although non-probability sampling strategies have some weaknesses in the process compared with probability sampling, several methods are proposed that can help improve the accuracy of nonprobability sample survey estimates, such as quota sampling, sample matching, and weighting techniques (Wiśniowski et al., 2020). Since the researcher would not have access to the population of African American clergy in Arizona, the researcher cannot use probability sampling. Therefore, non-probability convenience sampling procedures were appropriate and feasible for the current investigation.

Convenience sampling was necessary due to the inability to control who responds to the emailed survey and who does not and the fact that the email was only sent to clergy included in the site authorization (Allen, 2017). Site authorization was obtained through contact with African American Christian Clergy Coalition. The main goal of convenience sampling is to target qualified individuals who self-select to participate in a study based on individual motivation, which serves the researcher's convenience (Acharya et al., 2013; Etikan et al., 2016; Leiner, 2014). The assumption made by a researcher utilizing convenience sampling is that the target population is homogenous, or there would be no difference in the research results acquired through a random population sample. Utilizing a convenience sample strategy includes a risk of hidden biases from self-selected participants. Bias cannot be measured or controlled in a convenience sample. According to Acharya et al. (2013), results from a convenience sample cannot be generalized beyond the sample. Convenience sampling is financially reasonable for the novice researcher, easy and self-selected participants are readily available to participate in the research study.

Instrumentation

This causal-comparative research aims to find if any differences occur between dependent and independent variables after an event has already happened. This study had three dependent variables and one independent variable with two groups. This study used a valid instrument to collect the study's data and allow the hypotheses to be tested. This research project's data collection tool is the Mental Health Literacy Scale (MHLS), a 35-question Web-based Mental Health Literacy questionnaire. O'Connor and Casey (2015) developed the first scale-based instrument (Mental Health Literacy Scale) used to assess the knowledge of mental health and prevailing attitudes towards mental health and seeking help for mental health issues. This MHLS can help identify certain groups with low MHL (O'Connor & Casey, 2015). This instrument has a total of 6 subscales, independent of each other. These subscales score independently of each other. Participants completed the full MHLS for this study. However, this study looked specifically at three subscales. Previous studies have used the MHLS to measure the mental health literacy of other groups of clergies in America (Vermaas et al., 2017). These studies also selected subscales that would help identify areas of mental health literacy that needed improvement (Vermaas et al., 2017). Additionally, the MHLS has good internal and test-retest reliability and validity, which was discussed later in more detail.

Scoring for the MHLS can range from 35 to 160, with a higher score indicating higher mental health literacy levels. The use of online surveys for this study is more advantageous than using paper surveys to gather data. Conducting surveys online is more time-efficient and convenient for participating in the study. It requires less time to complete and has less opportunity for interviewer bias (Park et al., 2018). When online surveys are used compared to paper surveys, errors that often appear from data entry and processing can be avoided. The participants were asked to complete a six-question demographic study (Appendix E.).

Research Data

The focus of this quantitative causal-comparative study was to determine if differences exist in A) knowledge of professional help available, B) knowledge of risk factors and causes, C) knowledge of professional help available, based upon the MHLS Questionnaire (O’Connor & Casey, 2015) between African American clergy who have at least a four-year bachelor’s degree or higher and those who do not have at least a four-year bachelor’s degree. The researcher used data collected from administering questions from three attributes of the Mental Health Literacy Scale. Scoring for the MHLS subset can range from 14 to 60, with a higher score indicating higher mental health literacy levels.

Research Data Source #1. The three dependent variables for this study were scored differently. The first attribute, recognition of mental disorders, measures an individual’s “. . .ability to correctly identify features of a [mental] disorder, or a specific disorder” (O’Connor & Casey, 2015, p. 513). The minimum score of this subscale is 8 and the maximum score is 32.

The second attribute of the MHLS for this study is knowledge of risk factors and causes for developing a mental illness, which measures an individual’s “. . .knowledge of environment, social, familial, or biological factors that increase the risk of developing a mental illness” (O’Connor & Casey, 2015, p. 513). The minimum score of this subscale is 2 and the maximum score is 8.

The third attribute of the MHLS for this study is knowledge of professional help available, which measures an individual’s “. . .knowledge of mental health professionals and the services they provide” (O’Connor & Casey, 2015, p. 513). The minimum score of this subscale is 3 and the maximum score is 12. The MHLS is a psychometrically strong measure with a Cronbach’s alpha of .873, indicating good internal consistency (O'Connor & Casey, 2015). Test-retest results for the scale indicate good reliability, r=0.797, p<0.001 (O'Connor & Casey, 2015).

The MHLS was used with diverse populations and different settings or locations with previous research studies (Gorczynski et al., 2017). Authors of the MHLS cite the potential limitation of generalizability as an opportunity for further academic inquiries to explore the use of the MHLS within other samples (O'Connor & Casey, 2015). Dr. O’Connor, the creator of the MHLS, in his email giving the approval to use the MHLS suggest that any questions specific to Australia be modified to address the giving population being studied. Dr. O’Connor suggested the following modification be made to the instrument. Q5: To what extent do you think it is likely that Persistent Depressive Disorder (Dysthymia) is a disorder. Q8: To what extent do you think it is likely that the diagnosis of Substance Abuse Disorder can include physical and psychological tolerance of the drug (i.e., require more of the drug to get the same effect)

Validity

Validity is an important aspect to the research process and was discussed in-depth in this section. That the researcher establishes validity within the research is vital to ensure that the data are robust and replicable, and that the results are accurate (McClelland et al., 2015). Validity is defined as the extent to which a concept is accurately measured by an instrument in a quantitative research study (Heale & Twycross, 2015). Often considered the most significant aspect of research, validity refers to whether a study allows for causal inference or the generalization of results to the population (Zyphur & Pierides, 2017). The research instrument's integrity is assured when there is evidence of validity (Mohajan, 2017). There are two components of validity that researchers must be mindful of, internal validity and external validity. External validity is based on the generalizability of the research study’s results, and internal validity is based on the rigor and depth of the study and how it was conducted. The external validity of this study pertained to the generalizability of the findings. Andrade (2018) shares, external validity determines whether the findings can be generalized to other situations. Internal validity “examines whether how a study was designed, conducted, and analyzed allows trustworthy answers to the research questions in the study” (p. 2).

Validity is the most important factor when choosing the measure to be used for collection of data. The results attained from an instrument must serve the measure’s purpose (Fraenkel & Wallen, 2009). The instrument used for this study was a Likert scale survey called the MHLS Questionnaire (O’Connor & Casey, 2015). “Likert scale has been widely utilized in social science research, especially in business, education, tourism, psychology and other disciplines to measure the opinion or attitude of a respondent." (p. 55, Sangthong, 2020). Even though Likert scales are commonly used, they still need to be tested for validity; this validity illustrates how well these instruments do what they are supposed to do (Andrade, 2018). A systematic review of tools measuring mental health knowledge by Wei et al. (2016) examined the MHLS (through the Consensus- based Standards for the selection of Measurement Instruments (COSMIN). The review found the MHLS to have excellent internal consistency, strong content validity, and good structural validity and reliability. The review also indicated that the MHLS demonstrated fair hypothesis testing and that cultural validity and responsiveness had not yet been examined. The MHLS has been found to be a psychometrically strong measure with a Cronbach’s alpha of .873, indicating good internal consistency (O'Connor & Casey, 2015). Test-retest results for the scale indicate good reliability (r=0.797, p<0.001)’ (O'Connor & Casey, 2015). Additionally, scale developers found there to be adequate assessment in the areas of measurement error, content validity, hypotheses testing, and structural validity’ (O'Connor & Casey, 2015).

Reliability

The reliability of the instruments being used is an essential part of the research study. The reliability concept deals with the assessments' ability to be duplicated while the results are trustworthy across different settings (Rollnick et al., 2019). Threats to reliability exist throughout the entire process of research, and researchers need to be proactive and try to minimize these threats to the research as much as possible (McClelland et al., 2015). The instruments' reliability refers to the level at which the collection tool being used can present stable and consistent results. The study's reliability is always sample dependent, so it may vary from study to study (Scollione & Holdan, 2020). The level of reliability determines the overall accuracy of the results (Mohajan, 2017). When a study has high levels of reliability, another researcher should be able to replicate the study and reassess the outcomes (Rutkowski & Delandshere, 2016). This researcher considered several different types of reliability for this study, and each type is uniquely relevant to the situations where measurements are used (Kamper, 2019).

The participants for this study were different from the individuals used to develop this instrument, which may lead to not all the questions being processed and answered uniformly. This could cause a change in the reliability of the data compared to previous studies. The sample group for this study were specific and unique subset of the larger population being studied; thus, the data collected from this study could measure the MHL of the general population (Scollione & Holdan, 2020).

The data for this study were collected using a well-tested instrument. This instrument has been evaluated for its validity and reliability. To the reliability of the data, the researcher should conduct research using sound methods to collect and analyze the data (Castleberry & Nolen, 2018). The MHLS is a psychometrically strong measure with a Cronbach’s alpha of .873, indicating good internal consistency (O'Connor & Casey, 2015). Test-retest results for the scale indicate good reliability (r=0.797, p<0.001) (O'Connor & Casey, 2015). The scale has demonstrated excellent methodological quality in psychometrics for internal consistency, content and structural validity, and internal and test-retest reliability (O’Connor & Casey, 2015). The researcher took available measures to ensure the reliability of the data.

Data Collection and Management

For this study, the general population was the African American Clergy, including ministers, elders, and pastors, both male and female in Arizona. All of these clerics were associated with congregations in the state of Arizona. Thus, the purpose of this quantitative, causal-comparative study was to determine if, and to what extent, MHL differ among African American clergy in Arizona based upon their education levels.

Following approval from the Grand Canyon University Institutional Review Board (IRB), an electronic survey was deployed via organizational email. To collect data for this study, the researcher used a convenience sample of African American clergy in Arizona. In convenience sampling, “the ease with which potential participants can be located or recruited is the primary consideration” (Sarstedt et al., 2018). Participants had to be African American or Black clergy. The criteria to participate were /based upon descriptions of African American or Black clergy. Additionally, participants had to be clergy in Arizona. To determine the appropriate sample size, the researcher used G*Power 3.1.9.4 software.

Prior to participation individuals who are interested in completing this survey based upon the email received were provided with the informed consent information. The informed consent details the participants’ rights while participating in the study, including explaining how the collected data was used. This informed consent also documented the overall purpose and intent of this study. Finally, the informed consent information included any potential risks and benefits, also information about resources available if the participant is harmed in any way during the study (see Appendix C). Clergy members were informed that there would be no compensation for their participation and that the participation in the research study is completely voluntary. Those who desired to proceed with the survey were asked an inclusionary question to determine if they meet the requirements for participation in the study. These individuals were asked if they are members of the clergy in predominantly African American churches and reside in Arizona. If the individuals did not meet these requirements, they were excluded from participation in this study.

The researcher did not disclose the names of clergy members to any of the participants who are in the study. The names of clergy members remained anonymous from data collection to the reporting of findings. Clergy members were not asked to disclose identifying information in their survey. Each clergy member was given an identifying number that was used strictly for reporting and record keeping purposes.

Before using this survey, the researcher obtained permission to use it from the creators of the survey instrument. The researcher explained why he used it and mention that the study's data would be shared with the creators of the survey if requested. The researcher did not provide actual data results from the study but only shared the summary of the findings. All the data from the surveys, including the surveys themselves, are being kept secure and stored on a flash drive and held in a locked cabinet. Any data collected by a paper form will be stored for three years, after which the data shall be destroyed appropriately, and digital data shall be deleted from the flash drive after three years as well (Redus, 2020).

This researcher submitted a proposal to conduct this study to the Institutional Review Board at Grand Canyon University and wait for acceptance before requesting clergy to participate in the study. The site authorization process includes submitting an online form that includes information about the proposed study such as:

• Name and contact information of the researcher

• Purpose of the research (in this case a dissertation)

• In what way the study relates to the mental health literacy of African American clergy

• Purpose of the study including hypotheses and research questions

• The theoretical framework

• Research design along with data collection and analysis methods

• The resources required for the study

• A list of any instruments used

• A description of how results were used

• A review of the benefits of the study to the district

• Any other legal requirements such as parental consent forms if necessary.

The data collection tool used for purposes of this research was the (MHLS) (Appendix E), a 35-item questionnaire that assesses for knowledge and attitudes towards mental health and associated help seeking’ (O'Connor & Casey, 2015). The authors of this instrument consented for the use in this DNP project (Appendix E). Scores for the MHLS can range from 35 to 160, with a higher score indicating higher levels of mental health literacy. The MHLS is a Likert type scale, with 1 point assigned to the first response choice (very unlikely, strongly disagree, very unhelpful, and definitely unwilling) and 4 points assigned to the last response choice (definitely willing, strongly agree, very likely, and very helpful), except for questions 10, 12, 15, and 20 through 28, which are scored in a reverse pattern (4 or 5 points for first answer and 1 point for the last answer choice).

The researcher used Survey Planet to create the Mental Health Literacy Scale in a digital format that can be taken by all the participants for the research. This allowed the researcher to score and record data in the same format used for the written version of the MHLS. The results of the scoring for each survey are to be downloaded on to a USB disk. A USB flash drive is a data storage device that includes flash memory with an integrated USB interface. The linear sequence of step-by-step of procedures to be used to carry out the major steps for data collection is as follows:

1. Site approval for recruitment and to conduct the research study was obtained from the director of operation at the African American Christian Clergy Coalition.

2. The researcher is to submit to the Institutional Review Board (IRB) the package for review and approval.

3. The researcher also used flyers as well as using the email list provided by the African American Christian Clergy Coalition.

4. After the initial email is sent out, the researcher sent out a follow-up email to seek participation from the clergy. The researcher is to verify the potential participant as an African American clergy. This was accomplished by verifying the individual’s membership in the African American Christian Clergy Coalition. This verification was done by using a list of coalition members provided by the director of operations. If the potential participants desire to be in the study, an email with the survey link was sent.

5. The informed consent document (Appendix D) was sent in a separate email to each participant who has agreed to participate in this study. This must be completed before the participants receive any documentation concerning the overall study.

6. Once the informed consent has been received, notification was sent to research participants to complete the Mental Health Literacy Scale Survey.

7. Collection of results are to occur via the Internet secured and encrypted site.

8. Once the surveys have been completed by the sample, the data was then downloaded by the researcher to an SPSS database for analysis.

9. Data was cleaned for incomplete data records. Data are to be kept under lock and key at the researcher’s home office. Electronic records are to be stored on a password protected computer. Data will be kept for three years in a secure location and then the data will be destroyed (Redus, 2020).

Data Analysis Procedures

It is not known if and to what extent a difference exists in mental health literacy based upon the MHLS Questionnaire (O’Connor & Casey, 2015) between African American clergy who have at least a four-year bachelor’s degree or higher and those who have not completed a least a four-year bachelor’s degree. The following section details the data analyses procedures conducted for this current study. This section presents the statistical analysis of a one-way multivariate analysis of variance (MANOVA), data preparation, instrument scales, and the alignment of the research questions and hypotheses with the data analysis procedures.

The research study was designed to determine whether there was a difference in the MHL score of two groups of African American clergy. In the survey instrument there are two choices that individuals can choose from when answering his or her level of education. The two levels are those who have at least a four-year bachelor’s degree or higher and those who have not completed a least a four-year bachelor’s degree. Based upon the responses, the individuals were placed into one of two groups. Group A are those who have a at least a four-year bachelor’s degree and Group B are those who do not have at least a four-year bachelor’s degree. This study addressed this gap by examining the MHL of African American clergy. There is a lack of existing literature on the MHL levels of African American clergy.

This study used the scores from the MHLS Questionnaire (O’Connor & Casey, 2015). To answer the problem statement, the researcher conducted a quantitative, causal-comparative study to answer the following research questions:

RQ1: If and to what extent does a difference exist in knowledge of professional help available between African American clergy who completed at least a four-year bachelor’s degree and those who have not completed a least a four-year bachelor’s degree?

RQ2: If and to what extent does a difference exist in knowledge of risk factors and causes between African American clergy who completed at least a four-year bachelor’s degree and those who have not completed a least a four-year bachelor’s degree?

RQ3: If and to what extent does a difference exist in knowledge of professional help available between African American clergy who completed at least a four-year bachelor’s degree and those who have not completed a least a four-year bachelor’s degree?

Once the survey closed and data collection end, all responses were exported to

an excel spreadsheet to prepare the data for analysis. First, respondents that did not provide a complete survey were removed from the sample. Once the analysis was complete, the data was summarized so that they could be generalized to the greater population. According to Norris et al. (2015), it is important with groups to interpret and report on the sample size, the mean, standard deviations, as well as other data used to describe groups such as minimum and maximum values. Two of the standard assumptions associated with this collection of data is that data was normally distributed and the variances between the groups being compared were homogeneous.

Data screening of assumptions. Once the data were cleaned, data screening was conducted to assess the underlying assumptions. SPSS was used to evaluate the assumptions of normality, homogeneity of variance-covariance matrices, linearity, and multicollinearity (Tabachnick & Fidell, 2013).

Descriptive and inferential statistics were used to analyze each research question to determine if assumptions were met. The research questions address potential differences between multiples dependent variables; therefore, a one-way multivariate analysis of variance (MANOVA) will be utilized to analyze (French et al., 2008). Descriptive statistics are used to summarize the data and inferential statistics are used to test the hypotheses. One-way MANOVA, which stands for multivariate analysis of variance, was used to determine whether statistically significant differences exist between the groups of participants using the Mental Health Literacy Scale (French et al., 2008). When using MANOVA this allows for the analyzing of the different variables at one time, which provides for a better understating of differences between the groups that could be ignored by a single response variable and also minimizes the Type I error rate (Boslaugh, 2008). MANOVA also allows for a more accurate and comprehensive picture of the phenomena being studied by the researcher (Allen, 2017). Finally, measuring the multiple response variables together will provide more chances at discovering the factor that is central to the investigation (Allen, 2017). The one-way MANOVA, a parametric statistical test, will be run to perform inferential statistical analyses and indicate how likely the current study results could be replicated for an entire population (Fraenkel & Wallen, 2009).

This current study examined the independent variable, education level of the clergy, and the dependent variables, if and to what extent does a difference exist in knowledge of professional help available between African American clergy who completed at least a four-year bachelor’s degree and those who have not completed a least a four-year bachelor’s degree. If and to what extent does a difference exist in knowledge of risk factors and causes between African American clergy who completed at least a four-year bachelor’s degree and those who have not completed a least a four-year bachelor’s degree. If and to what extent does a difference exist in knowledge of professional help available between African American clergy who completed at least a four-year bachelor’s degree and those who have not completed a least a four-year bachelor’s degree? Since this current study involves three dependent variables, the researcher determined that a one-way MANOVA would be the most appropriate statistical test to use since it can determine whether there were any differences between independent groups on more than one continuous dependent variable (French et al., 2008).

A one-way MANOVA answered the three research questions regarding

whether there was a statistically significant difference between the two comparison groups of clergies when measuring the dependent variables simultaneously. The level of significance was p < .05, meaning there was a 5% chance that a difference existed in the two groups of clergies when there was not an actual difference. Also, the current study determined whether a mean difference exists between the two groups of clergies. Conducting an F-test could provide an overall comparison of whether the means of the two groups of clergies differ. If the obtained F is larger than the critical F, the null hypotheses is rejected (Gravetter & Larry, 2016). The one-way MANOVA creates a linear combination of the three dependent variables to generate a grand mean and determine if there were group differences in the dependent variables.

Parametric tests, like one-way MANOVA, are appropriate when the data reveal a normal distribution. A one-way MANOVA test shows whether equal variances and normal score distributions are present (Field, 2013). Therefore, an essential requirement to use a one-way MANOVA test is that the assumptions of normality be met. Normality ensures scores are typically distributed, which would be indicated by a bell-shaped curve (Field, 2013). If normality assumptions are not met, a Mann-Whitney test may be used. The Mann-Whitney U is a non-parametric inferential test that can be used to analyze ranked data when the data are not normally distributed (Fraenkel & Wallen, 2009).

In order to run a one-way MANOVA, ten assumptions needed to be addressed one at a time to ensure the sample could be analyzed using this test, which consisted of (1) two or more dependent variables on a continuous level, (2) one independent variable has two or more categorical, independent groups, (3) independence of observation, (4) no univariate or multivariate outliers, (5) multivariate normality, (6) no multicollinearity, (7) linear relationship between dependent variable for each independent group, (8) adequate sample size, (9) homogeneity of variance-covariance matrices, and (10) homogeneity of variances (Statistics, 2015).

The first assumption requires two or more dependent variables measured at the continuous level (Statistics, 2015). Assumption one was satisfied for the study, as there are three dependent variables measured on a Likert-type scale, which is commonly accepted to be continuous in the field of the social sciences.

The second assumption requires one independent variable with two or more categorical, independent levels (Statistics, 2015). The term level is typically reserved for groups that have an order (Statistics, 2015). The study has one independent variable (Education) with two levels (college graduate, non-graduate). The second assumption was satisfied.

The third assumption requires independence of observation where there is no relationship between the participants in any of the groups. Having different participants in each group is a way to address this assumption (Statistics, 2015). Assumption three was met as the data had different participants in each of the three groups.

The fourth assumption requires no univariate or multivariate outliers (Statistics, 2015). This assumption is commonly tested in SPSS by following the Explore procedures then visually analyzing boxplots to detect outliers. Any data that are more than 1.5 box-lengths from the edge of their box are classified by SPSS as outliers and are noted by circular icons, and data more than three box-lengths away are noted by an asterisk. Although this is not a foolproof method, it is the more straightforward approach (Statistics, 2015).

The fifth assumption requires multivariate normality, which means normally distributed data for each of the groups in the independent variable is expected (Statistics, 2015). This assumption is commonly tested by utilizing the Shapiro-Wilks test for normality in SPSS by following the seven-step Explore procedure. This test is commonly utilized if the sample size is less than 50 participants. There are as many Shapiro-Wilks tests as there are groups of the independent variable multiplied by the number of dependent variables.

The sixth assumption requires that there be no multicollinearity, which means that the dependent variables should be reasonably correlated with each other (Statistics, 2015). If the correlations are too high (greater than 0.9), there is risk for multicollinearity, which is problematic for a MANOVA (Statistics, 2015). Utilizing the Bivariate procedure in SPSS, Pearson correlations between the dependent variables are analyzed to determine correlation between the variables (Statistics, 2015).

The seventh assumption requires a linear association between the dependent variables for each group of independent variables (Statistics, 2015). A scatterplot matrix for each group of the independent variables identifies if there is linear relationship (a straight line) or not (a curved line). If the variables are not linearly related, then there is a loss of ability to identify differences (Statistics, 2015). In SPSS, after splitting the data file to separate out the independent levels, the Chart Builder procedure was utilized to assess linearity through scatterplot (Statistics, 2015).

The eighth assumption requires a sufficient sample size. Laerd (2018) stated that the larger the sample size the better, but at a minimum, there need to be as many participants in each group of the independent variable as there are number of dependent variables. GCU required a minimum of 50 participants per independent variable level. Assumption eight, demonstrating adequate sample size, was satisfied upfront by using a priori power analysis.

The ninth assumption requires homogeneity (similar or comparable) of variance-covariance matrices (matrix of all possible pairs of variables) (Statistics, 2015). After un-splitting the file, the assumption could be tested by utilizing Box’s M test of equality of covariance in SPSS. The important row is the significance level (p-value) of the Box’s M test. If the test is not statistically significant (i.e., p > .001), there is homogeneity of variance-covariance matrices and no assumptions are violated (Statistics, 2015). The assumption was addressed in Chapter 4.

The tenth assumption requires homogeneity (same) of variances. Assuming the assumption of homogeneity of variance-covariance matrices were not violated, a Levene’s test of equality of variances procedure in SPSS is run (Statistics, 2015). The one-way MANOVA assumes that there are equal variances between the groups of the independent variable. The important column is the Sig. which represents the significance level (p-value) of the test. If the test is not statistically significant (greater than .05), there are equal variances and the assumption of homogeneity of variances has not been violated (Statistics, 2015).

Meeting assumptions is a requirement for obtaining accurate results when using a one-way MANOVA as seen below in Table 4; however, it is common for data to violate one or more of these assumptions. When data violate assumptions, the researcher must either correct data, use an alternative test, or proceed with the analysis despite the violation of assumptions (Statistics, 2015).

Table 4. Assumption Strategies for One-Way MANOVA

Assumption

Test

Alternate Fail Procedure

1. Two or more continuous DVs

Design feature

Change design or analysis

2. Two or more categorical IVs

Design feature

Change design or analysis

3. Independence of observations

Design feature

Change design or analysis

4. No univariate or multivariate outliers

Review SPSS box plots; Mahalanobis distance test

Verify data entry or measurement errors; keep and transform or evaluate effect by running one- way MANOVA with and without outliers, or remove

5. Normality of DV distribution or multivariate normality

Shapiro-Wilk test

Transform DVs, run one-way MANOVA; or keep as one-way MANOVA is somewhat robust to normality deviations

6. DVs moderately correlated

Pearson correlation coefficient test between DVs

If low correlation, use multiple one-way ANOVAs. If high correlation, remove DV with high correlation or combine scores for new DV

7. A linear relationship between each pair of DVs for each IV group

Scatterplot matrix

Transform one or more DVs; remove non-linear DV, or keep and accept a loss of power

8. Adequate sample size

Minimum in each IV group as the number of DVs

Increase sample size

9. Homogeneity of variances

Box’s test of Equality of Covariance Matrices

Proceed if equal samples of IVs. If unequal sample sizes, transform or keep and use Pillai’s Trace instead of Wilk’s Lambda

10. Homogeneity of variance- covariance matrices

Levene’s Test of Equality of Error Variances test

Transform to equalize variances or continue and accept lower statistical significance and run different post-hoc tests

To determine the appropriate sample sizes, the researcher used G*Power software. MANOVA is used by the researcher to deduce a relationship between independent and dependent variables using a smaller sample, which can be generalized to a larger population (Allen, 2017). Based upon the number of variables, the minimum sample size for the one-way MANOVA was 28 participants. To account for attrition, 30% was added to the minimum sample size, which should reflect a total of 36 in the final sample. The researcher utilized SPSS version 26 to clean the data by identifying any missing values. Missing data in quantitative research can lead to loss of important information, increase for standard errors, weaken generalization of findings, and reduce statistical power (Dong & Peng, 2013).

Ethical Considerations

In 1974, the National Research Act was signed into law, establishing the National Commission for the protection of Human Subjects of Biomedical and Behavioral Research (Rice, 2008). The ethical standards for the conducting of research have been set by the Belmont Report. Respect for persons, justice, and beneficence are the basis for these ethical standards that have been established. When research is defined as “an activity designed to test a hypothesis, permit conclusions to be drawn, and thereby to develop or contribute to generalize knowledge,” and when it involves human subjects, attention must be paid to any potential ethical issues (Adashi et al., 2018). Topics includes the participant's informed consent, the rights and well-being of the subject, and the management and security of the collected data.

This study's ethical standards were essential to the planning, conducting, and evaluations of data in this research. Ethical considerations for this quantitative, causal-comparative research study were observed to protect all individuals associated with this current study. As participants are answering self-reported, online surveys, subjects did not face risk different from normal everyday activities. Furthermore, these participants were at no risk of exposure to delete physical or psychological harm from their involvement in this study.

When conducting research, there are ethical norms that should be taken into consideration. Norms like ensuring that the researcher is choosing a good research design. So, any method that provides misleading and harmful information would be considered invalid research. The researcher must be able to carry out proper research by following the steps to ensure the study's validity. The requirement of informed consent for this study showed the demonstration of ethical principles and norms for this research. This consent involved informing the participants that their participation in this study was voluntary. These participants also had the right not to answer any question or even withdraw completely from this study. The researcher ensured participants were aware of how the results would be used and explain any known risks of participating in the study. According to Burke-Johnson and Christensen (2014), there are three ethical concerns researchers should consider during the design of a study. These concerns are treatment of the research participants, professional issues, and the relationship between science and society. The most important of these three concerns would be the treatment of research participants (Burke-Johnson & Christensen, 2014).

All participants are informed that at no time during this study would their identities be used. If clergy who start the survey subsequently become uncomfortable, they could withdraw from this study at any time. As the data are collected, although potentially private or personal issues such as age, appear on the demographic survey, they were not discussed. Once the data was collected, measures were taken to ensure the confidentiality of the data further. Codes replaced any identifying information in the data, and other personal information included in the data were removed from any documentation.

To ensure the safeguarding of privacy and confidentiality and minimize any risk, ethical guidelines were followed. All research conducted will be done the following and guided by Grand Canyon University's IRB committee. All electronic versions of the research data were saved on the researcher's personal computer, which has a secure password-protected login. Any paper versions are stored in a locked file drawer that no one can access other than the researcher. All versions will be stored for a minimum of three years and then will be deleted and shredded accordingly.

Assumptions and Delimitations

While Chapter 1 addressed the broad, overall limitations and delimitations of the study, this section discusses, in detail, the limitations and delimitations related to the research methodology and design and potential impacts on the results. The section also describes any limitations and delimitations about the methods, sample, instrumentation, data collection process, and analysis. Limitations and delimitations are significant components of any research conducted. Limitations are design or methodology characteristics that restrict or limit the application of the study findings. Delimitations are boundaries set by the researchers to control the range of the study. When the researcher fails to acknowledge any limitations or delimitations in the survey, it compromises the study (Theofanidis & Fountouki, 2018).

The purpose of this study was to determine if there is a difference in MHL of African American clergy of those who have at least a four-year bachelor’s degree or higher and those who have at least a four-year bachelor’s degree or higher. This study provided valuable information about future training with this population of the clergy; however, many extraneous variables were not considered in this study, which could account for differences between the groups. One example was the type of degree received by each varied clergy by clergy. Having had in classes that dealt with counseling or the psychology of any sort was not a requirement for this study's participants. Participants' demographic information is based upon the survey's response and can be subject to falsification or mistakes. The lack of a useful assessment tool to collect this information can be considered a delimitation for this study.

The delimitations present in any study can limit the generalizability of the results. Delimitations are limitations that have been set by the author (Theofanidis & Fountouki, 2018). For the current study, at least a four-year bachelor’s degree or not at least a four-year bachelor’s degree was selected as the only independent variable. It was recognized that other variables could influence the mental health literacy of the sample population. Therefore, the study's scope could be limited by the nature of the problem statement and research questions. Additionally, the study was only done in one state. Although the sample population was large enough per the G*Power analysis, the findings may not be generalizable to other countries.

Summary

The purpose of this quantitative, causal comparative study was to determine if and to what extent there was a difference in the mental health literacy of African American clergy who have a at least a four-year bachelor’s degree vs African American clergy who have not completed a least a four-year bachelor’s degree. The research considered the difference based upon the Mental Health Literacy Scale survey that was administered to all the participants. The research conducted in this study relied upon a quantitative methodology. The approach does not allow the researcher to alter or even manipulate scores, so this leads to bias free study results.

Chapter 3 describes the specific steps that were taken to address the research questions. The purpose of Chapter 3 is to provide a logical flow from the problem statement and its background to the analysis of the data collected. Each section in the chapter provides an overview of a different aspect of the design intended to convey the study's rationale and logic. The research questions for this study focused on the differences in mental health literacy based on college degrees.

For this study the selected individual variable were not manipulated. For this study a causal-comparative study was chosen. For this study, two independent variables existed. The first includes the clergy who have obtained at least a four-year bachelor’s degree. The second independent variable includes clergy who have not received at least a four-year bachelor’s degree. The dependent variable for this study were the clergy’s mental health literacy that was obtained by the completion of the MHLS.

The descriptions of limitations, delimitations, and ethical considerations ensured the full disclosure of any challenges present to the study results. The limitations and delimitations expanded as new information emerged. The next step is to obtain IRB approval and complete the research. The subsequent chapter focused on data collection, analysis, and present the results. The final chapter summarizes the study and provides evidence of the study's contribution to the literature's extant body.