Thesis Proposal

This chapter presents essential steps in designing quantitative methods for research proposal study. The methods section of a quantitative research proposal follows a standard form including participants, materials, procedures and measures. It includes information on the design of the study including the specific type of experiment.

Chapter 8 begins with a discussion of survey design. The chapter also discusses the components of the survey method plan and then moves on to the specifics of the survey design. This is illustrated through the outlining of steps for a survey study beginning with a discussion of the purpose, the identification of a population, the sample, the survey instruments, the relationship between the variables, the research questions, specific items on the survey, and steps be taken in the analysis and interpretation of the data from the survey.

Quantitative Methods

Defining Surveys and Experiments

· A survey design provides a quantitative description of trends, attitudes, or opinions of a population by studying a sample

· The researcher generalizes or draws inferences to the population based on the results obtained from the study sample

· In an experimental design, the basic intent is to test the impact of the treatment on an outcome, controlling for all other factors; this may include random assignment of individuals to groups

Components of a Survey Method Plan

The Survey Design

The first parts of the methods section can introduce readers to the basic purpose and rationale for the research.

· Identify the purpose of survey research; this purpose is to generalize from sample to a population so that inferences can be made

· Indicate why is survey is the preferred type of data collection procedure for the study

· Indicate whether the survey design will be cross-sectional or longitudinal

· Specify the form of data collection; options include telephone, mail, Internet

The Population and Sample

Follow the type of design with characteristics of the population and the sampling procedure. Essential aspects of the population and sample describe include:

· Identify the population in the study; include the size of this population, if the size can be determined

· Identify whether the sampling design for this population a single stage or multistage.

· Identify the selection process; random sampling is ideal; each individual in the population has an equal probability of being selected; this provides the ability to generalize to the population

· Identify whether the study will involve stratification of the population; in order to stratify, characteristics of the population must be known; in stratification, specific characteristics are represented in the sample and the sample reflects the true proportion in the population

· Discuss the procedures for selecting the sample from available lists

· Indicate the number of people in the sample and the procedures used to compute this number; sample size relates to the analysis plan for study

· Determine a margin of error you are willing to tolerate

· Determine the confidence level for this margin of error

· Estimate the percentage of your sample that will respond in a given way; the steps will allow you to determine the sample size needed for each group (Fowler, 2009)

Instrumentation

· Name the survey instrument used to collect data; include information on whether this is an instrument designed for the research, a modified instrument, or an intact instrument developed by someone else

· Describe established validity of the instrument

· Content validity – did the items measure the content they were intended to measure

· Predictive or concurrent validity – do scores predict a criterion measure or do results correlate with other results

· Construct validity – do the items measure hypothetical constructs or concepts

· Mention whether scores resulting from past use of the instrument demonstrate reliability; reliability is the consistency or repeatability of an instrument; the most important form of reliability is internal consistency

· When an instrument is modified, it becomes important to reestablish validity and reliability because they original validity evidence and reliability may not hold

· Includes sample items from the instrument

· Indicate the major content sections in the instrument: cover letter, items, closing instructions, type of scale for responses

· Discuss plans for pilot testing or field testing the survey and provide rationale for these plans; testing helps establish content validity, an initial evaluation of internal consistency, and to improve the survey

· For mailed surveys, identify steps for administration; discuss how you will obtain a good response rate

Variables in the Study

· It may be useful in the methods section to relate the variables to the specific questions on the instrument

· Include a table and discussion that cross-reference the variables, questions or hypotheses, and survey items

Data Analysis and Interpretation

· Identify computer programs used in analysis

· Present information about the steps involved in analyzing the data in the proposal:

· Report information about the number of members of the sample who did and did not return the survey

· Discuss the method by which response bias will be determined; response bias is the effects of non-responses on survey estimates.

· Discuss a plan to provide a descriptive analysis of data for all independent and dependent variables

· If the proposal contains an instrument with scales, identify a procedure to calculate total scale scores

· Identify the statistics and the statistical computer program for testing the major inferential research questions or hypotheses

· Present the results in tables or figures and interpret the results from the statistical tests

· Report how the results answered the research question or hypothesis; also report the effect size and the confidence interval

· Discuss the implications of the results for practice or for future research

Components of an Experimental Method Plan

Participants

· Describe the selection process for participants as either random or nonrandom; when individuals are not randomly assigned, the procedure is called the quasi-experiment

· When individuals can be randomly assigned to groups, the procedure is called a true experiment; discuss how the individual will be randomly assigned to the treatment groups

· Quasi-experiments – the researcher has partial or no control over randomly assigning participants to levels of a manipulated variable

· Conduct and report a power analysis to determine the sample size; the following information is needed to calculate sample sizes

· Alpha level

· The amount of powered desired

· The effect size

· At the end of the section, provide a formal experimental design statement that specifies independent variables and corresponding levels

Variables

· Clearly identified independent variables in the experiment

· Include a manipulation check measure of the intended manipulated variable

· Identify dependent variable (i.e., outcomes)

· Identify other variables measured – participant demographics, variables that contribute noise, potential confounding variables

Instrumentation and Materials

· Thoroughly describe the instruments used in terms of their development, items, scales, and reports and reliability and validity of scores on past uses

· Discuss materials used for the manipulated variable

· If the researcher does not want participants to know what variables are manipulated, draft a cover story; deception requires debriefing and IRB approval

Experimental Procedures

· Identify the type of experimental design, such as pre-experimental designs, quasi-experiments, true experiments, or single-subject designs

· Identify what is being compared and experiment

· When two or more groups are being compared, this is called a between subjects design

· When only one group is being studied this is called within group design

· Provide a diagram or figure to illustrate the specific research designed to be used

· X represents an exposure of a group to an experimental variable or events

· O represents an observation or measurement

· The letter R indicates random assignment

· Left to right indicates the temporal order of procedures

· Horizontal lines indicate comparison groups are not equal

Threats to Validity

· Types are internal validity, external validity, statistical conclusion validity, and construct validity

· Internal validity threats

· History – events can occur that influence the outcome

· Maturation – participants in an experiment may mature or change

· Regression – scores, over time, regress toward the meaning

· Selection – participants can be set collected with certain characteristics that affect the outcomes

· Mortality – participants dropped out during an experiment

· Diffusion of treatment – communication between participants in the control and experimental groups can influence outcomes

· Compensatory/resentful demoralization – benefits of an experiment may be unequal or resented with only the experimental group receives the treatment

· Compensatory rivalry – participants in a control group feel that they are being devalued because they are not receiving the treatment

· Testing – participants become familiar with the outcome measure and remember responses for later testing

· Instrumentation – instrument changes between pretest and posttest

· External validity threats

· Interaction of selection and treatment – narrow characteristics of participants in experiment makes it impossible for the researcher to generalize

· Interaction of setting in treatment – because of the setting the participants are in the researcher cannot generalized individuals in other settings

· Interaction of history and treatment – because results of an experiment are time bound research cannot generalize

The Procedure

The writer describes in detail the procedure for conducting the experiment.

· Discuss a step-by-step approach for the procedure in the experiment

· Administer measures of the dependent variable or a variable closely correlated

· Assign participants to matched pairs

· Randomly assign one member of each pair to the control and experimental group

· Expose experimental group to the treatment

· Administer measures of dependent variables

· Compare performance of the experimental and control groups

· The reader should be able to understand the design, observations, treatment, and timeline of the activities

Data Analysis

· Report the descriptive statistics calculated for observations and measures at the pretest or posttest stage; these will be means, standard deviation, and ranges

· Indicate inferential statistical tests used to examine the hypotheses in the study

· Provide confidence internal and effect sizes

· For single-subject designs, use line graphs for baseline and treatment observations

Interpreting Results and Writing a Discussion Section

· The final step in an experiment is to interpret the findings in light of hypotheses or research questions

· Address whether the hypotheses are questions were supported or refuted

· Consider whether the independent variable manipulation actually made a difference for participants who experienced them

· Suggest why or why not the results were significant; link evidence to literature, theory, and logic

· Address whether the results might have occurred because of unique strengths and weaknesses.

· Discuss generalizability to certain people, settings, and times

· Indicate the implications of the results and future research