BIZZNA TUTOR Survey research

Directions for Unit 7

Survey Research

All the information you need for completing Unit 7 will be found on this directions sheet. Print a copy and please pay careful attention to the directions.

1. Keep the objective relevant to this unit in mind as you proceed: Following completion of this course, the student will understand the process of descriptive research. The student will understand the place of survey research in education and will have a grasp of the fundamentals necessary for survey construction. The student will understand the necessity for use of informed consents in educational research and will be able to construct one for a hypothetical research project.

2. Complete the assigned readings.

3. You have two tasks to complete:

a. Post on the forum a six item survey that you propose to administer. Include a one-page informed consent on a separate page. Refer to the list of criteria on pages 38-41 in your Etext prior to composing your survey items. Please post the survey and informed consent as a single document attachment that your classmates and I can download and open for viewing. The instrument must have three Likert items (strongly agree, agree, neutral, disagree, and strongly disagree). It must also have two true/false or yes/no items and one open-ended item that requires discussion. . Note the use of the term item rather than question. These will be declarative statements rather than questions, so it is not a questionnaire. These minimum requirements are simply to make sure that you get experience in specific analytical procedures. The survey must relate directly to the work you did in Unit 4

b. Survey evaluation. Evaluate your own survey according to the eleven criteria listed in the text. Keep the comments of your classmates and me in mind as you carry out this task. You should have already done a revision based on those critiques prior to carrying out this evaluation. Be sure to attach a copy of the final survey to the survey evaluation that you submit to me. Address each guideline in a paper that is organized around the text’s suggestions for such evaluations. In other words, address each item in sequence. To simplify your paper, type the item in the checklist as a paragraph heading and then simply respond. Example:

4. Use simple, clear, unambiguous language. No jargon was used and the level of language was appropriate for the samples involved.

Commentary on Common Errors to Avoid

I teach a class in sociolinguistics every once in a while and there are a couple of points related to that course that I’d like to make. First, languages evolve constantly and some of the points I’ll make subsequent to this might be challenged. Second, we all tailor our speech to the audience. That is, in professional or formal social settings we tend to speak more correctly, closer to Standard English, while we all naturally move to a lower nonstandard form with friends and family.

The relevance to this class is interesting. Unfortunately, for some, writing professionally isn’t that easy. Some folks tend to write like they talk in informal settings and that just won’t work for technical writing as you must do in research and other professional tasks. I’m going to list a few common errors that I see every semester and hope you’ll take time to study them and thus make your writing more professional.

1. A teacher is able to discipline their students easily. (Their is plural and doesn’t fit properly with a teacher – singular) Proper would be: A teacher is able to discipline her students easily. (Or) Teachers are able to discipline their students easily. (Both singular or both plural)

2. It is important that “if the shoe fits, wear it”. (This is a common misuse of quotation marks. To emphasize a word or phrase, use italics.) Thus, it becomes: It is important that if the shoe fits, wear it. Emphasize with italics and quote with quotation marks.

3. Teacher attitudes are not like student attitudes. (This might just be education jargon, but I’m not sure.) Correct is: Teachers’ attitudes are not like students’ attitudes.

4. In the 1950’s, life was slower. (Misuse of apostrophe…you don’t want a possessive form here.) Correct is: In the 1950s, life was slower.

5. Parents of special needs children are needy themselves. (Use a hyphen between two compound modifiers when they precede a noun.) Correct: Parents of special-needs children are needy themselves. Another bad example: The hot water bottle is full. (Do you mean the water bottle is hot?) Better: The hot-water bottle is full.

6. The author and/or the publisher insisted. (The forward slash is used in informal writing to indicate a choice between two words. It has no place in formal writing.) The author and the publisher insisted. (Or) The author or the publisher insisted.

7. To educate children, one can educate them one-on-one, educate them in a classroom setting, or educate them through use of projects rather than typical education. (Use your thesaurus to avoid repetitious wording.) A little better: To educate children, one can tutor them one-on-one, teach them in a classroom setting, or instruct them through use of projects rather than typical education.

8. Use of commas. This is a terrible problem and too complex to address here. Please study: http://grammar.ccc.commnet.edu/grammar/commas.htm

9. The dog who runs is faster than the girl that walks. (Use who with people and that for other than people.) The dog that runs is faster than the girl who walks. People that study learn more or people who study learn more? The rock who sits is smarter than the rolling stone? What’s wrong with this one?

10. Several methods and tools that have been used in traditional labs do not always meet the needs of learners for various reasons, therefore, there is a need to use the latest technological techniques to achieve effective learning. (Avoid run-on sentences; break them up into separate sentences that stand on their own.) Better: Several methods and tools that have been used in traditional labs do not always meet the needs of learners for various reasons. Therefore, there is a need to use the latest technological techniques to achieve effective learning

11. Handheld devices, particularly the iPad, have been finding their places in schools all over the world. Good schools need parental input. (Neither an iPad nor a school can do anything on its own. Don’t attribute actions or thoughts to inanimate objects.) Better would be: Handheld devices, particularly the iPad, have been used in schools all over the world. Administrators and teachers need parental input.

12. Special-needs children number 200 out of the total population of 1,000, or 20%. Use of the word or is incorrect: 200/1000 is 20%. There’s no choice to make. No either or. It is simply another way to express the data. Better would be: Special-needs children number 200 out of the total population of 1,000 (20%).

13. I could go on, but these are the most frequent. Now that you’ve seen this, I know I’ll never encounter such errors again…makes my effort in putting this together worthwhile!

Additional reading

American Psychological Association. (2007). APA Style Guide to Electronic References. Washington, D.C.: Author.

American Psychological Association. (2010). Publication Manual of the American Psychological Association (6th ed.). Washington, D.C.: Author.

SURVEY RESEARCH

Objective: Following completion of his course, the student will understand the place of survey research in education and will have a grasp of the fundamentals necessary for survey construction.

We see examples of survey research in the newspaper on a daily basis. Political polling is but one example for we are bombarded with requests for our opinions on a wide variety of subjects. Following purchase of a new car, the dealer will want to know how you feel about various aspects of the transaction. Many restaurants give discounts for subsequent visits if you go online and complete their survey. Market research using surveys allows businesses to improve their services, products, and profit margin. In education, it is frequently a good strategy to learn what administrators, teachers, parents, teachers’ aides, students, and others believe, perceive, or think about a vast range of questions that affect the education process. In this class, you will develop one of these questions or problems and administer your self-generated survey to samples of subjects from two different populations in order to make comparisons of responses.

Obviously, not all surveys are done to make such comparisons. You might find it desirable simply to survey all middle school students to learn something about how their dietary preferences fit in with school lunches. You might want to administer a survey or questionnaire to parents to determine their preferences in terms of group meeting times or parent-teacher conferences in general. For action research, surveys constitute a potential gold mine to the teacher who is creative, oriented to problem-solving, and has the basic skills necessary to construct and administer surveys and also knows how to properly analyze results and report them.

Surveys include questionnaires, telephone or personal interviews, checklists, and observational data. For this class, a survey instrument will be developed to address the purpose of your particular research problem and will be a paper-and-pencil instrument administered by you, the researcher, in your presence. We will call our instruments surveys rather than questionnaires because all items will be in the form of declarative statements rather than questions. Mailing out surveys is a legitimate procedure but too time-consuming for our purposes this semester. Because any such mail-out should be accompanied by a stamped, self-addressed envelope, this can get expensive. In addition, the return rate for mailed or emailed surveys is almost always very poor. Certainly there is a sampling problem involved as well when you consider that the people who return surveys are certainly different from those who do not. This would skew your results and make your findings questionable. The external validity, ability to generalize, would be negatively affected because only certain volunteers in your original population took part. Serious sampling problems can also occur with use of commercial Internet survey providers. Survey Monkey (http://www.surveymonkey.com) does a fine job of helping you put together a professional-looking instrument for a fee, but returns are limited to those who, like with the mailed surveys, agree to take part. In addition, they will be somewhat different from the population in general in that those who respond are familiar with the technology involved. In other words, the sample will be biased and your ability to generalize will be questionable.

A survey is not necessarily a questionnaire. For this class, you’ll use a series of declarative statements rather than questions, so yours will be a survey. A questionnaire asks for answers to a series of questions. Avoid the use of the term questionnaire in your writing.

In preparation of the survey, there are several items to consider. Use the following guidelines in preparation and subsequent evaluation of your survey instrument. Some suggestions, many adapted from Patten (1998), include:

1. Avoid using negative statements, all, never, and always. Negative statements are too often confusing and is anything really never or always? Does the sun always rise? Well, what about millions of years later when it has burned out?

2. Make sure that each item addresses a single point. For example, if a subject is asked to respond to “Special education students often exhibit behavior problems and lack parental support”, there’s an obvious difficulty. What if one has problems, but also has parental support? Keep each item simple, limited to one concept.

3. Don’t anticipate knowledge of education esoterica. Define terms that might not be obvious to non-educators and don’t use acronyms such as ADD, ADHD, SACS, etc. Spell these out so that the person not closely tied to education will be informed.

4. Use italics, bold print, different color font, or underlining to emphasize important parts of the question

5. Do not waste the subject’s time and effort by asking questions that have obvious responses. Why ask if math mastery or reading is an important skill? Should anyone doubt that physical activity is important to children? No? Then why ask? Limit your survey items to those that address something directly related to solving the problem.

6. When looking at knowledge, attitudes, and perceptions, use a Likert scale, a declarative statement with choices such as you see in the following example:

I approve of the use of capital punishment in elementary school.

__strongly agree __agree __neutral __disagree __strongly disagree

Responses could be coded 5, 4, 3, 2, and 1 to make calculation of a mean and standard deviation possible and if two groups give information, a t-test could be used for statistical analysis.

7. Modification of the categories suggested above might be necessary in some cases, such as in evaluation of a textbook, a teaching methodology, or a program:

__very satisfied __ satisfied __ neutral __ dissatisfied __very dissatisfied

8. Surveys can also include yes/no and true/false items as well as open-ended questions that will be analyzed by qualitative methods. In this class, to broaden your experience, you’ll include both of these as well as Likert-response items.

9. Only address demographics sparingly to avoid the appearance of invasion of privacy. If the questions are pertinent to the study, that is only proper. If you just want to satisfy your curiosity, forget it. If your question is about income, use ranges rather than specifics to avoid subject irritation with your probing. Be careful when questioning about family and please forget race and ethnicity in this class. There will be times when such research is justified but the complications of simply defining race and ethnicity have become so complex as to be almost meaningless. For this class, the only demographic question should generally be an item that requests information about to which group the subject belongs.

10. Make the instrument attractive, perhaps using borders, colors, graphics, etc. Be absolutely certain that there are no misspelled words, problems with grammar, or errors in the instrument. Your professional prestige is at risk in such cases.

11. Whether your administration of the survey is face to face or via telephone, be sure to use a script in your instructions so that all subjects get precisely the same information and direction.

It is a given that a survey must be reliable and valid. Otherwise, the data obtained is meaningless. One must never commence collecting data prior to establishing the reliability and validity of the instrument, a survey in this case. Reliability refers to the degree to which scores are consistent. There are several ways to calculate reliability. One of the simplest and most logical is to determine the correlation between repeated administrations of the test. For our Likert items, prior to the pilot study testing you could administer the survey twice to a separate group of subjects and then calculate a correlation coefficient between the two sets of scores, item by item. If the correlation for each item is >.70, this particular test item can be said to be reliable. If it is lower, perhaps it requires rewording or other clarification or simply needs to be discarded and replaced. Multiple choice items can be subjected to item analysis, but that is irrelevant to our projects because no such items will be included. Determining the reliability of true-false items and qualitative data is very complex, controversial in nature, and beyond the scope of this course. There are several methods of estimating whole test reliability, but there is simply not time to cover that topic in this class on educational research. If this becomes an issue that you’d like to address someday, you can consult a statistician, take a class on statistics or on tests and measurements, or you can obtain books and use the Internet to master the process on your own.

Like reliability, determination of validity can be carried out by several means. If a test is valid, it measures exactly what it is proposed to measure. One of the most used procedures in educational research is to subject instruments to review by a panel of experts and if problems are detected to adjust or tweak the test item until the panel agrees that it passes muster, that it has content validity. You will subject the instrument that you develop to examination and critique by a panel of experts which will include at least two other students (In a discussion board assignment) and the instructor. In real life rather than a pilot study, your panel of experts might include professionals skilled in test construction, statistics, and the subject matter being investigated. Only after revisions and subsequent approval by the panel can you assume your survey instrument to be valid.

Survey research, done properly, is a powerful, practical, and very useful tool for educators, psychologists, and a wide variety of others. Survey findings often stimulate creativity and lead to more sophisticated study that has the potential to delve into cause and effect. While many seasoned researchers use surveys, this approach is particularly appropriate for the novice.

Additional Reading

Fowler, F. J. (1988). Survey research methods. Newbury Park, CA: Sage

Lavrakas, P. (1987). Telephone survey methods: sampling, selection, and

supervision. Beverly Hills: Sage

INFORMED CONSENT

Objective: Following completion of this course, the student will understand the necessity for use of informed consents in educational research and will be able to construct one for a hypothetical research project.

A basic part of every research project is the informed consent. There are two fundamental parts: (1) information about the research, and (2) consent of the subject to participate.

Information that is the subject’s right to know prior to giving or refusing consent includes basic information about the purpose of the study and the procedures that will be followed. In addition, if there are any risks of any sort, they must be explained. There should be a statement guaranteeing anonymity. Then, potential benefits should be briefly stated and the subject informed that he or she can simply quit or withdraw at any time without penalty of any sort. The order of these is not particularly important and not all research will require all parts. For example, there may be no risk involved. All of this can be succinctly stated in a brief, well-planned and well-written paragraph.

The second part of the document is the subject consent. There should be a statement that the subject has the right to ask questions if any part of the information is unclear or if there are any points unrelated to what has been presented that require clarification. Following this, there will be a statement that the subject agrees to participate and lines are provided for signatures by the subject and a witness.

An informed consent must be understood not to protect the researcher from litigation should negligence or other circumstance result in injury to the subject. This is not a legal document and it gives the researcher no protection. It is simply information for the subject plus his or her agreement to take part. For a survey, simply participating constitutes consent and there is no real need for a statement of consent unless children and potential controversy are involved.

Here is an example that you can modify as necessary for your pilot study. You should always consider the reading capability of your subjects and adjust the reading level accordingly. The example below includes a parent signature line that would be required only if a minor child is serving as a subject. You should feel free to adopt the following example and use it as your own with appropriate modifications. Of course, if you do, you should give credit in your footer (Adapted from Byrd, R. (2103). Educational Research: A Focused Approach. Etext.).

Informed Consent Example

There seems to be a possible difference between the thinking of special education teachers and parents of special education students concerning the benefits and disadvantages of mainstreaming, putting special education students into regular classrooms. The purpose of this study will be to compare thoughts of these two groups. A benefit from pinpointing different ideas is that we will then know how to attack the problem and get both groups working together more effectively.

Completion of a survey will be anonymous and nobody will have access to your responses. Your name will not be on the survey and will not be used in reporting. You have the right to refuse to take part and you also have the right to stop at any point while taking the survey if any part makes you uncomfortable. This is a basic right and refusal to take part or stopping at any point will not be a source of concern to the researcher and there will be no bad feelings on the part of the researcher.

I have had an opportunity to ask questions about the research and all procedures and now give my consent to participate.

___________________________ ________________

(Subject’s signature) (Date)

___________________________ ________________

(Witness’ signature) (Date)

___________________________ ________________

(Parent’s signature for minor) (Date)

Additional Permission

Besides the informed consent, it is imperative that you obtain your principal’s or other supervisor’s support. Never leave yourself open, for example, to the ire of a principal who finds out after the fact that some parent is unhappy with her child being a guinea pig! This is a serious matter and can have grave career consequences. We live in a society where everyone is under the microscope and litigation is an everyday occurrence. Don’t let it happen to you!

An additional point is that you must assure your principal that there will be no intrusion on instructional time. This should go without saying, but it is important that your research does not interfere with education of the children. Remember that your research is secondary to the normal function of the school and must interfere in no way. Be open, keep the authorities informed, use common sense, and your research project should go well.

With respect to research being done by students or faculty at SAU, the Institutional Review Board (IRB) might come into play, depending on the circumstances. Obviously, you’re not particularly interested in faculty requirements, but as a student you should be aware of the following. From the SAU document Guidelines for the Protection of Human Participants in Research, Section I, Part B, item 1 (Available en toto on the SAU Website): “Typically, classroom exercises do not meet the IRB’s definition of research, since they do not contribute to generalizable knowledge. Such classroom research exercises do not require IRB review”. Thus, a pilot study such as you’re involved with in this class certainly does not qualify as requiring IRB approval. The class professor will make sure that the methods you propose to use are such that there will be no risk to subjects and that the possibility of controversy over survey items will be minimized or eliminated.

Suggested reading

Berg, J. W., Appelbaum, P. S., Parker, L. S., & Lidz, C. W. (2001). Informed Consent: Legal theory and clinical practice.

Leedy, P. D., & Ormand, J. E. (2010). Planning your research project. In Practical Research (9th ed., pp. 85-114). Upper Saddle River, NJ: Pearson Education Inc

An alternative scale