Unit III Research Essay

RCH 8301, Quantitative Research Methods 1

Course Learning Outcomes for Unit III Upon completion of this unit, students should be able to:

3. Explain the dimensions of research validity. 3.1 Explain sampling and its connection to validity.

5. Evaluate the dimensions of measurement reliability.

5.1 Analyze the measurement techniques and the connection to reliability.

Course/Unit Learning Outcomes

Learning Activity

3.1

Unit Lesson Chapter 9, pp. 137–158 Chapter 10, pp. 162–179 Chapter 11, pp. 182–196 Chapter 12, pp. 200–213 Unit III Essay

5.1

Unit Lesson Chapter 9, pp. 137–158 Chapter 10, pp. 162–179 Chapter 11, pp. 182–196 Chapter 12, pp. 200–213 Unit III Essay

Required Unit Resources Chapter 9: Sampling and Introduction to External Validity, pp. 137–158 Chapter 10: Measurement and Descriptive Statistics, pp. 162–179 Chapter 11: Measurement Reliability, pp. 182–196 Chapter 12: Measurement Validity, pp. 200–213

Unit Lesson

Sampling and Introduction to External Validity

In a research process, the group of people or materials that are to be studied are called the research sample. However, in cases where research applies to a large group, which is referred to as the population, a sample becomes a subset of the population. The sample selection has a great effect on the generalization of the study outcome on the population; thus, the process of selecting the sample is called sampling. Since sampling has a great credibility and validity of the research results, the concept of external validity is very critical in the research process. Notably, validity, which is a measurement of reliability and statistics, considers two aspects in sampling and assignment. Internal validity involves random assignment, whereas external validity involves random sampling. Thus, when sample data is used in the statistical analysis to draw conclusions about the generalized population and an unbiased sampling method is adopted to sample the group, then the unbiasedness of the sampling method is an indication of the external validity.

UNIT III STUDY GUIDE

Sampling and Measurement

RCH 8301, Quantitative Research Methods 2

UNIT x STUDY GUIDE

Title

Random sampling is essential to external validity so that the researcher may use the findings of the study to generalize to the entire population. On the other hand, the nonprobability sampling is a frequently used technique when all of the people (or individuals) in the population of a study are given an equal chance for selection. Since the use of random sampling is always difficult to achieve, most researchers choose the nonprobability sampling technique to save them on time, money, and a workforce. As a result, to realize the external validity, random sampling is adopted to achieve a randomized selection process and thereby minimize the influence of the researcher in choosing the individual sample group from the entire population. There are key concepts of sampling such as the sample design and the steps in electing a sample. As described above, the sample designs include the random sampling technique and the nonprobability sampling. These techniques pave the way to the steps in the selection of the sample, which include the concept of theoretical population, the selected sample, and the accessible population. According to Gliner et al., the general population refers to the “the larger group of interest for the study and from which the sample is drawn” (p. 137). On the other hand, the theoretical or target population refers to “all of the participants of theoretical interest to the researcher and to which he or she would like to generalize” (p. 138). Lastly, the accessible population is the group or population in a study that is a subset of the target population (commonly referred to as the study population), whereby the researcher can effectively apply conclusions and make assumptions (Asiamah et al., 2017).

Overall, the appreciation of the sampling techniques and the population sample requires the understanding of external validity, which is simply the general applicability of the research findings to the entire population. By using a proper sampling technique and by having an accessible population, the researcher can thereby generalize the results beyond the sample to study other people (a process known as population validity) as well as other settings (called the ecological validity). Ecological validity is a subset of external validity since it involves examining whether the study findings can be generalized to real-life settings.

Measurement and Descriptive Statistics

Descriptive statistics are simply considered as brief descriptive coefficients that are used to summarize a

specific data set, which represents either a sample group or the entire population (Holcomb, 2016). The descriptive statistics are categorized into measures of central tendency (mean, median, and mode) as well as the measures of variability standard deviation, minimum and maximum variables, variance, skewness, and kurtosis. The use of descriptive statistics and graphs are used variably depending on the nature of the variables. Descriptive statistics are used in the organization and summarization of the sample data, which then allows for the performance of inferential statistics. The use of populations in a study is characterized by the use of the parameters (descriptive measures) based on sample statistics. The parameters that are used include sample mean and sample variance. Otherwise, all of the descriptive statistics are measures of variability or the measures of central tendency. There are various levels of measurement; nominal measurement, ordinal measurement, ratio scales, and interval measurements, among others. Traditional levels or scales of measurement, which include the level of measurement for variables used in research, have been widely attributed to the differences in the observations (variable). Thus, traditionally, statistical analysis placed their measurement into four categories: nominal, dichotomous, ordinal, and interval (Gliner et al., 2017). The nominal scales are used for the qualitative variables, whereby the observations made are placed into discrete groups, and no inherent quantitative difference can be established amongst the categories. A dichotomous variable only has two categories, such as yes/no or pass/fail. Ordinal scales are used in ranking the order of observations (e.g., the class rank). The ordinal scale is distinct from other measures where there is a specific underlying quantitative measure, which makes the observations differ. In interval measurement (scale), the interval between the adjacent values is constant with no distinct or true points.

Research process sampling from a target population (Iamnee, n.d.)

RCH 8301, Quantitative Research Methods 3

UNIT x STUDY GUIDE

Title

In terms of the nominal measurement, a variable is divided into discrete or separate categories, the categories are named, and then the measurement is done at the nominal level (Gilbert & Prion, 2017). The measurement of the large population gives rise to a smooth polygon, which is called the curve. When the curve is bell-shaped, then it is referred to as the normal curve. The normal curve is particularly important while doing descriptive statistics and research—especially in testing the assumption that the data is normally distributed. A normal distribution is defined by the mean and standard deviation as the main parameters. Since the data in a normal curve is centralized, there is less of a tendency to produce extreme values; however, as the standard deviation increases, the distribution of the data becomes wider. Therefore, an analysis of the distribution and curve is important when making assumptions about the data.

Measurement Reliability

In the measurement of the reliability of research, various methods are used such as test-retest reliability, parallel forms reliability, internal consistency reliability, Cronbach’s alpha, the inter-rater reliability, and others. The measurement reliability involves the measurement of stability, precision, or consistency in the research instrument. The aim of conducting such measurement is to ensure that a reliable instrument is used in the study, which can reliably give similar results for repeated measures. The concept of reliability, especially in statistical analysis, refers to the overall consistency of a scale or measure; therefore, a measure is said to have a high reliability if it only gives similar results in consistent conditions (Trochim, 2006). The internal consistency reliability is used in assessing the consistency of the results across items within a specific test. The internal consistency reliability involves the use of different methods in the assessment of reliability such as the classical reliability theory, which examines the reliability by dividing the observed scores on the test into two components (error and true score). Different types of reliability estimates, such as the inter-rater reliability, are also used to assess the degree of agreement between raters, whereas the test-retest reliability assesses the degree of consistency in the test score from a single administration. The methods to assess inter-observer reliability, such as the single method approach (Kappa-coefficient and Spearman correlation) and the multiple methods (interclass correlation and Kappa), are adopted to ensure that integrity of the data is maintained. Overall, in the measurement of reliability, the standard error of measurement is an important measurement procedure and, thus, affects reliability. The standard error of measurement measures the precision of a test and the accuracy of a score (Tighe et al., 2010). The smaller the standard error of measurement, the more precise the study.

Measurement Validity

The validity of any instrument used in research refers to the measurement validity, which is defined as the “degree to which a measure or test measures that which it was intended to measure” (Gliner et al., 2017, p. 200). To assess the validity of the instrument, various methods are used, such as face validity, content validity, criterion-related validity, and construct validity. The different methods that are used in validity measurement can be drawn based on timelines, which are the 1985 and 1999 standards. Accordingly, the 1985 standards are based on evidence relating to the content, response process, relations to other variables, and consequences. On the other hand, the 1999 standards involved the evidence relate to content, construct, and both criterion-related and construct-related evidence.

Using the 1999 standards, construct validity involves measuring the extent to which the operations of any construct is based on a theory, whereby it subsumes the other types of validity. This implies that construct validity encompasses both empirical and theoretical support to make the valid interpretation of a construct. However, content validity, which is a nonstatistical measure of validity, involves the systematic examination of

(Yekophotostudio, 2017)

RCH 8301, Quantitative Research Methods 4

UNIT x STUDY GUIDE

Title

the test content in order to reveal whether a representative sample of the behavior domain can be measured. In other words, does the test cover the entire domain that it was designed to measure? Furthermore, the face validity is used in the estimation of whether a given test appears to measure a specific criterion. For example, a low face validity implies that the test is more valid.

References

Asiamah, N., Mensah, H. K., & Oteng-Abayie, E. F. (2017). General, target, and accessible population: Demystifying the concepts for effective sampling. The Qualitative Report, 22, 1607–1621. https://nsuworks.nova.edu/tqr/

Gilbert, G. E., & Prion, S. (2017). Making sense of methods and measurement: Nonparametric measures of

association. Clinical Simulation in Nursing, 13(1), 1–2. https://www.nursingsimulation.org/article/S1876-1399(16)30069-X/abstract

Gliner, J. A., Morgan, G. A., & Leech, N. L. (2017). Research methods in applied settings: An integrated

approach to design and analysis (3rd ed.). Routledge. Holcomb, Z. C. (2016). Fundamentals of descriptive statistics. Routledge. Iamnee. (n.d.). Research process sampling from a target population. Probability, measuring (ID 81702923)

[Illustration]. Dreamstime. https://www.dreamstime.com/stock-illustration-research-process-sampling- target-population-business-marketing-social-methods-selecting-sample-elements-to-image81702923

Tighe, J., McManus, I. C., Dewhurst, N. G., Chis, L., & Mucklow, J. (2010). The standard error of

measurement is a more appropriate measure of quality for postgraduate medical assessments than is reliability: An analysis of MRCP (UK) examinations. BMC Medical Education, 10(1), 40–49. https://www.researchgate.net/publication/44651198_The_standard_error_of_measurement_is_a_mor e_appropriate_measure_of_quality_for_postgraduate_medical_assessments_than_is_reliability_An_ analysis_of_MRCPUK_examinations

Trochim, W. M. (2006). Types of reliability. http://www.socialresearchmethods.net/kb/reltypes.php Yekophotostudio. (2017). People in the office. Businesspeople, group (ID 98866369) [Photograph].

Dreamstime. https://www.dreamstime.com/stock-photo-people-office-office-business-people-work- image98866369

Learning Activities (Nongraded) Nongraded Learning Activities are provided to aid students in their course of study. You do not have to submit them. If you have questions, contact your instructor for further guidance and information. Review the “Interpretation Questions” and “Application Problems” at the end of Chapters 9, 10, 11, and 12.