BIZZNA TUTOR
kih1125
selecting_subjects_unit_8.docxSELECTING SUBJECTS
Objective: Following completion of this course, the student will be able to explain one method of random sampling and of convenience sampling and will understand the desirability of random assignment of subjects and why convenience sampling is often used in educational research despite the obvious disadvantage.
A population is defined by the researcher. In your pilot study, you might define a population of students that you want to include in your study as seventh graders at Elmhurst Middle School in Cornstalk Mountain, Arkansas. You might define a population of teachers for your study as special education teachers in Stonewall County, Arkansas. You define your population. There are tradeoffs in population size. The more extensive the population (Across a county, across a state, etc.), the greater the generalizability. On the other hand, the more extensive the population, the greater the difficulty of sampling and obtaining subjects. For pilot studies, trial runs so to speak, it is often the case that smaller and more convenient populations are defined. Your next job will be to select a representative sample from the population you define.
Once you have defined your population, you must decide how to select your subjects. Keep in mind that the strongest studies are those in which random sampling is used. This is true in quantitative research whether or not you have a true experimental design. However, it is not true of qualitative research where your sample is both purpose driven and nonrandom. This will be discussed further in the section on qualitative research.
How do you randomly select subjects? The first concern is the composition of your groups. If they are different, e.g. teachers and administrators, parents and teachers, students and teachers, then you immediately go to randomly selecting subjects within their respective groups. But if your study requires an experimental and a control group of students, say, then your first task is randomly assign students to the two groups. There are, of course, many ways to do this. An easy one would be to put names of the whole population on slips of paper in a hat and draw, first name drawn to the experimental group, second to the control group, etc. until the required number of subjects has been attained.
Now, if you have two differing populations for comparison, then you have to randomly select subjects for each population in turn and again there are many ways to do this. One way would be to make a list of subjects, assigning a number to each one. Let’s assume that you have a pool or population of 75 subjects from which to select a representative sample of 20. You could use a random number generator, one that would give you numbers between 1 and 75, and then use the first 20 subject numbers generated. You can do this by entering in an Excel spreadsheet cell the expression =RAND( )*(B-A)+A. This will give you a number that is greater than A and less than B. For the example given, for 75 subjects, you would have =RAND( )*(75-1)+1. Instructions for using this random number generator are set up in a spreadsheet in Blackboard Content and you should use it for selection of subjects in your pilot study. You can also find tables of random numbers and random number generators on the Internet.
Another approach to selection of subjects randomly would be through use of a telephone directory. Decide beforehand which column in the telephone book you will use and then open it at random. Let’s assume that your population is in two digits, with subject numbers less than 100. Look at the telephone number at the top of the column you had decided to use and use the last two digits as your subject number (Telephone number 435-1516 --- you would pick subject number 16). Go to the next number and look at the last two digits for the next subject number. Repeat until you have your desired number of subjects. Obviously, if the last two digits come up more than once you’d ignore the 2nd, 3rd. etc. appearances. One last random sampling procedure termed systematic sampling requires you to start at a random point in a list of subjects and choose every 3rd, 5th, 10th, etc. subject in the list. You could determine this number by dividing the population list size by the number of subjects desired. If you have a total list of 90 potential subjects and want a sample of 10, division gives you a 9 and you’d select every 9th person in the list for inclusion in your sample.
What this randomizing does, of course, is to minimize or eliminate bias, subconscious or conscious, in selecting subjects. This gives you your very best opportunity to see that your samples are equivalent at the outset, a basic requirement that we always try to meet. It must be kept in mind that random sampling efficacy is number-sensitive. In other words, if you assign only four subjects to each group there is a strong possibility that the four in the control group will be very different from the four in the control group. However, as the number per group rises, your confidence in equivalence increases. The probability of your getting nonbiased comparison groups is very dependent on sample size, so don’t scrimp on this aspect of your planning.
When random sampling isn’t possible, convenience sampling can substitute if you are willing to face the fact that it weakens your study. Convenience sampling is just choosing subjects on the basis of their easy availability and obviously this is much weaker than randomizing. Cluster sampling is selecting an intact group to be studied and you see that happening frequently in educational research. In the section on research design, a scenario was described where you’d ask your principal to randomly assign students before school started to two of your science classes for the purpose of doing a quasi-experimental research project. The principal refused and you were then forced into cluster sampling, using two of your classes that were set up by administrative witchcraft or some other method that doesn’t employ random assignment. The primary danger in convenience sampling and cluster sampling is that the groups might be different, but comparison of pretest scores can either allay that fear or prove it to be valid. Please don’t discount cluster sampling. Although it is less than ideal, it is a very frequently used procedure in educational research and has been used in many meaningful projects. Because of the constraints of time, completing a pilot study as you do other assignments in this research class, you’ll probably be forced into selecting your subjects on the basis of their availability and convenience to you (convenience sampling). Still, randomize when possible.
One subject selection procedure that should not be used is use of volunteers. By the very act of volunteering they set themselves apart as being different from non-volunteers. There is great danger of bias in findings, so avoid this procedure like you’d avoid the plague.
For formal research later in your career, consulting a statistician from your school district or one at SAU for advice in sampling would be a good strategy. This person can also evaluate your thoughts concerning statistical analysis and give you ideas for improvement of that aspect of your study if your plan has weaknesses. You should never feel insecure about seeking such advice. Many prolific researchers have a statistics expert who they regularly consult about such problems in the planning stages of research and these statisticians are frequently included as coauthors on the ensuing publications.
Suggested reading
Henry, G. T. (1990). Practical sampling. Newbury Park: Sage.
Thompson, S. K. (2012) Sampling (3rd ed.). New York: Wiley.
THE PROPOSAL
Objective: Following completion of this class, the student will have demonstrated the ability to develop a good quality, well designed proposal for a research project.
It is critical that a formal proposal be presented to your professor for approval or modification prior to commencing your research. To do otherwise is foolish because it often results in having to start over from scratch. This proposal, really just a formal plan, should include a cover sheet like the models in the appendices, perhaps with additional information according to the wishes of the professor, for example his/her name and a date. For this class that additional information is not necessary. The proposal proper consists of five sections and will constitute a large part of the final paper with only minor changes such as tenses. The five sections are: (1) Introduction, (2) Review of Literature (Already submitted and edited), (3) Methods, (4) Reference section (Already done – part of the review of literature), and (5) Appendix A which will contain the survey instrument and informed consent. The proposal will be written largely in future tense (what will be done) while the final paper will require those verbs to be changed to past tense (what was done).
Introduction
The Introduction section should sell the professor or other reader on the importance of the project. It should include, going from general to specific, background information, including perhaps definitions and citation of a few of the most important related studies. It should end with a statement, “The purpose of this study will be to…” (Note the future tense – the study is yet to be done). This section might be a page or two in length. Depending on the professor’s preferences, you might also include at this point a research hypothesis and/or a null hypothesis. For this class, please state a null hypothesis – that there will be no statistical difference between responses of the two groups. Do not address limitations, delimitations, or definition of terms . These are frequently referred to in research methods texts and many supervising professors demand that they be addressed formally and separately. I do not. The latter should occur naturally in the text if necessary and the other two can be implied from the context and do not need to be stated formally in your papers for this class.
Review of Literature
This section, already completed in a previous assignment, edited, and corrected, should be an exhaustive compilation, well organized by topics. The length of this section is often dictated by the professor for whom the paper is written. Some might ask for a certain number of pages; others might require a specific number of references (A minimum of twenty for this pilot study). It is imperative that you make the corrections noted by your editor, in this case the instructor. Negotiation on differences of opinion is fine. To fail to negotiate or follow your editor’s suggestions implies arrogance on your part or shows a disdain for the judgment of the instructor, either of which will be fatal to the future relationship. It is important that the writer and the editor work smoothly together to finish with a paper of which both can be proud. Differences of opinion regarding the manuscript can be accepted by the author or negotiated with the editor but should never be ignored. Be absolutely certain that every editing comment on the review of literature is addressed and all corrections are made prior to inserting it into the proposal.
Methods
The test of the methods section’s success is whether another individual could precisely replicate your study through reading your material. For purposes of your pilot study, we will need to include five subsections.
Introduction. Include the design (Descriptive in this case) and information regarding lack of statistical power (because of small sample sizes). Make a statement that ordinarily insignificant differences are not discussed, but that they will be for this pilot study.
Subjects. Subjects’ pertinent characteristics must be detailed. They might be parents, teachers, administrators, or children. Age and gender might be important. Are they regular students, special education students, art students, or elementary classroom students? What grades are involved? If they are teachers, what is their specialty? Are they experienced or novices? Ask yourself these kinds of questions and extrapolate to parents and administrators if they constitute your sample. For those of you working outside education, you might have specific kinds of clients or others who you need to describe. Counselors, nursing home staff, and VA Hospital patients come to mind. For all of you, explain how subjects will be selected. Do you use convenience sampling, random sampling, or some other procedure (See the section in this Etext on sampling.)?
Instrument. You must give details regarding development of your survey instrument. How many items and what types of items are included? For this pilot study, a mandated minimum includes: (1) three Likert-type items (Assessed via a scale of 5-1 points for strongly agree, agree, neutral, disagree, strongly disagree.), (2) two yes/no or true/false response items, and (3) one open-ended item in which the respondent has no real limits on the length or complexity of the response. You should also add a statement about validation of the instrument which was accomplished by input from an expert panel that consists of at least two fellow students and the class instructor. Give correlation coefficients for your Likert-type items as a measure of reliability. This subsection should be written in past tense, for you will have developed the instrument prior to writing the proposal.
Procedures. Explain how the survey will be administered. Will it be in-person, one-on-one, supervised by the researcher? Will it be in a group setting? When will the survey be taken? At the subjects’ convenience? After a faculty meeting? Where will this occur? Note that a script will be used so as to standardize instructions to each subject. Give enough detail that another person could precisely repeat the process, with no questions remaining.
Statistical analysis. This section should describe the statistical tests to be used and the alpha level on which decisions will be based. This will be highly structured for this class so that each student gets some experience in using some common and useful tools in analysis. This subsection is provided for you and you should simply copy and paste the following as the last part of your methods section ( Change the font.):
Analysis
Likert-type items. Means and standard deviations for both groups will be calculated for each item. Analysis of differences between the means will be accomplished by means of t-tests, with decisions on significance made at alpha=.05.
Yes/No. A table showing the distribution of yes and no responses for both groups will be developed for each item. This will be used to calculate the probability of such occurrences through use of the Fischer Exact Probability Test and if the calculated value is less than alpha, .05, it will be considered that a significant group difference for that item has been detected.
Open-ended item. Analysis of the open-ended survey item will be accomplished by standard qualitative methodology. Repetitive themes will be sought and discussed in view of previous research findings.
References
This section was previously completed, edited, and revised as a part of your review of literature assignment and is simply copied and pasted, intact. It should start at the top of the first full page following the proposal proper. Using a page break at the end of the text on the preceding page will force this to be done properly.
Appendix A
On separate pages following the references, the survey instrument and informed consent should be included as an appendix. The first page should be the title page for Appendix A and should include Appendix A and Survey Instrument and Informed Consent in bold font on two separate double-spaced lines, located about 1/3 of the way down the page. Following that appendix title page, insert your survey instrument and then your informed consent.
Miscellaneous notes
Concerning your proposal, no detail should be missed. Remember the test: Can anyone read your Methods section and precisely replicate the study? It is almost always the case that important modifications of the proposal’s section on methods will result from your professor’s review. It might be that she or he will require clarification of the introduction or perhaps expansion or reorganization of the review of literature. This is to be expected as part of the natural process in students’ research. Be understanding and patient. Expect multiple rewrites to follow the multiple drafts that you went through prior to presenting the proposal. All your professor in this class or any other is trying to do is to teach you about doing research and writing while working towards completion of a project with minimal flaws, a paper that can perhaps be published. Work diligently with him or her and you’ll have a good experience and be proud of your final product!
In your proposal and final manuscript it is important that there be agreement among several key parts. The title of the paper should reflect accurately the statement of the problem. By the same token, the null hypothesis should relate closely to those two. This seems like common sense, but novice writers often change course in midstream, expect reading between the lines, or sometimes just get careless. Here are some examples, both good and bad.
Bad examples
Title. Special education
Purpose of the study. To examine differences in perceptions of parents and teachers.
Null hypothesis. There will be no difference.
Critique of the bad example
Title. This is far too vague and doesn’t tell the reader what the research is about.
Purpose of the study. Differences in perceptions about what? Again, too vague.
Null hypothesis. Reading that alone, you have no clue.
Better examples
Title. Comparison of Parents’ and Teachers’ Perceptions Regarding the effect of No Child Left Behind Legislation on Special Education Students
Purpose of the study. The purpose of this study will be to compare parents’ and teachers’ perceptions regarding the effect of No Child Left Behind Legislation on special education students
Null hypothesis. The null hypothesis will be that there will be no significant differences in parents’ and teachers’ perceptions regarding the effect of No Child Left Behind Legislation on special education students.
Comments
Can you now see how these elements should relate? Any of the three should be able to stand alone and let the reader understand what the research is all about.
Appendix C of this Etext is a model proposal for your emulation. Please note the adherence of the writer to the guidelines set out in this section.
Suggested reading
Hamper, R. & Baugh, L. (2010). Research Proposals: A Guide to Success (3rd ed.).
Ogden, T. E. & Goldberg, I. A. (2002). Handbook for Writing Proposals (2nd ed.).
