for Professor M

Summer 2014

A Mixed Methods CUMULATIVE ACTIVITY:

Grant Writing and Program Evaluation

For this assignment you are going to bring together some of the work that we have done in both quantitative and qualitative research, to set up a mixed methods study. Specifically, you are going to plan how you would write a grant proposal for a new program in a hypothetical school district. You will include ideas about how you would assess the proposed program’s effectiveness (e.g., on students’ learning, teachers’ satisfaction, parents’ involvement or other). To make it somewhat easier you will focus on a plan to write a grant proposal to study cooperative learning in a medium-sized, ethnically-diverse school district.

In the real world, the following process could take weeks, even months of meetings—you have a few hours. Here is the information you will need and the ground rules:

The Lots of Bucks Corporation has invited us at CSUDH in the COE to submit a proposal for a $500,000 Effective Teaching grant. To qualify for the grant we must involve at least 50 teachers in a two-year staff development program to train them to effectively use cooperative learning instructional strategies. Assume that this district has a traditional nine-month academic calendar.

Description of District . The Unified School District (USD) has 53 schools (40 elementary, 8 middle and 5 high schools). The district is 30% Black, 30% Hispanic, 30% Non Hispanic White and 10% Asian. The district is lower middle class and has a history of tension between ethnic groups. Test scores are lower than other districts in the state, even when socioeconomic factors are controlled.

Following are two sample quantitative goals with objectives. Your job for part one of this proposal is to write two of your own quantitative goals that have two objectives each, and to write two qualitative research questions.

SAMPLE QUANTITATIVE GOALS AND OBJECTIVES

Sample Goal 1. The percentage of total class time devoted to cooperative learning and the skill with which teachers implement it will increase during the two years of the grant.

Sample Objective 1. A. From Month 1 to Month 18, Experimental Teachers (n=50) will increase the percentage of class time spent in cooperative-learning activities by at least 15% (based on observations of in-class instruction and lesson plans).

Sample Objective 1. B. From Month 1 to Month 18, Experimental Teachers (n=50) will increase the skill with which they model the critical features of cooperative learning by at least 75% (based on observations of in-class instruction).

Sample Goal 2. Appreciation of diversity will increase during the course of the grant.

Objective 2. A . From Month 1 to Month 18 the number of cross-ethnic positive interactions in the Experimental Schools will increase by 10% (based on observation checklists concerning seating proximity during lunch and assemblies, playmate interactions during recess, etc).

Objective 2. B. From Month 1 to Month 18 the number of ethnically-motivated incidents reported to personnel at the eight Experimental Schools will decline by at least 20%.

SAMPLE QUALITATIVE RESEARCH QUESTIONS

1. How will teachers feel about their working/teaching environment after implementation of Cooperative Learning?

2. How will students view the change in their learning environment?

REVIEW: Go back and reread the sample goals and objectives. What is the independent variable for each?

What is the dependent variable for each?

IMPORTANT NOTE:

YOU MAY NOT WRITE GOALS, OBJECTIVES, OR QUALITATIVE RESEARCH QUESTIONS MEASURING THE SAME DEPENDENT VARIABLES THAT I USED FOR THE EXAMPLES. Be VERY careful; using what I have given as examples has been a common error of students in the past.

YOU MAY NOT HAVE 9 MONTH OBJECTIVES—ALL MUST BE 18 MONTHS.

OK— TIME TO GET STARTED!

Begin by spending about 5-10 minutes working quietly, ALONE. Use the chart on the next page and come up with some ideas about things that might happen if the quality of instruction increased dramatically. You many then share your ideas with your teammates if you choose to run your ideas by them. It is not required that you do so—you may begin on your own.

Next, settle on a total of two quantitative and two qualitative outcomes that will be specific to your plan. At least one of your qualitative outcomes must be in the affective (attitude) domain.

OUTCOMES LIST

TEACHERS STUDENTS OTHERS

After choosing your outcomes, you will have to: 1) construct specific objectives to assess if the goals are achieved, and 2) write two qualitative research questions. The forms to fill in with your goals, objectives and qualitative research questions begin on the next page.

Keep in mind that goals are general and objectives are specific. Objectives must specify WHO, WHAT, WHEN AND HOW MUCH. See the Check Sheet for Cumulative Activity Part 1 for a preview of how you will be “checking” your objectives.

You must use the following form sheets to complete this activity. Do not turn this into a paper. It is an activity—not a research paper. However, do use complete sentences and be careful to reread your writing for clarity and grammar.

Form Sheets for Cumulative Activity Part 1

Lots of Bucks Corporation Teaching Effectiveness Grant Plan

The following goals, objectives and research questions would be the focus for the two-year duration of the proposed cooperative learning staff development program.

GOAL 1

OBJECTIVE 1-A

OBJECTIVE 1-B

GOAL 2

OBJECTIVE 2-A

OBJECTIVE 2-B

QUALITATIVE RESEARCH QUESTION #1

QUALITATIVE RESEARCH QUESTION #2

Check Sheets for Form Sheets for Cumulative Activity Part 1

Use the following form to “check” your objectives. You may consult with your teammates for this checking activity. This is not a critiquing exercise, it is an editing exercise, so please do not ask questions or talk to the other person who is checking your ideas during this exercise. I want you to read the other person’s objectives without any clarification, in order to help him/her find strengths and weaknesses before they are handed in to me for points.

Remember, objectives must be very specific and give the following information: 1) who will be affected—experimental ELL students, all teachers in the district, etc.; 2) what is going to change—SAT scores, monthly attendance, etc.; 3) how much change is going to take place—10%, 20 students, etc.; 4) when—by the end of the grant, from month 1 to month 18, etc. and; 5) how the change (or gain) is to be measured—based on school attendance records, monthly classroom observations by the research team, etc.

Check Sheet for Cumulative Activity Part 1— self check form

OBJECTIVE 1A

WHO: ______________________________________________________________

WHAT: ______________________________________________________________

HOW MUCH: _________________________________________________________

HOW MEASURED: _____________________________________________________

WHEN _________________________________________

OBJECTIVE 1B

WHO: ______________________________________________________________

WHAT: ______________________________________________________________

HOW MUCH: _________________________________________________________

HOW MEASURED: _____________________________________________________

WHEN _________________________________________

OBJECTIVE 2A

WHO: ______________________________________________________________

WHAT: ______________________________________________________________

HOW MUCH: _________________________________________________________

HOW MEASURED: _____________________________________________________

WHEN _________________________________________

OBJECTIVE 2B

WHO: ______________________________________________________________

WHAT: ______________________________________________________________

HOW MUCH: _________________________________________________________

HOW MEASURED: _____________________________________________________

WHEN _________________________________________

Check Sheet —teammate

OBJECTIVE 1A

WHO: ______________________________________________________________

WHAT: ______________________________________________________________

HOW MUCH: _________________________________________________________

HOW MEASURED: ____________________________________________________

WHEN _________________________________________

OBJECTIVE 1B

WHO: ______________________________________________________________

WHAT: ______________________________________________________________

HOW MUCH: _________________________________________________________

HOW MEASURED: ____________________________________________________

WHEN _________________________________________

OBJECTIVE 2A

WHO: ______________________________________________________________

WHAT: ______________________________________________________________

HOW MUCH: _________________________________________________________

HOW MEASURED: _____________________________________________________

WHEN _________________________________________

OBJECTIVE 2B

WHO: ______________________________________________________________

WHAT: ______________________________________________________________

HOW MUCH: _________________________________________________________

HOW MEASURED: _____________________________________________________

WHEN _________________________________________

Cumulative Activity Part 2: Evaluation Planning

Now that you have constructed Quantitative Goals with Objectives and Qualitative Research Questions for your $500,000 grant proposal plan, you need to tell the granting agency exactly how you would evaluate your quantitative objectives and answer your qualitative research questions. Specifically, you need to tell the granting agency how you will pick the schools, the 50 teachers and the students (as appropriate to your proposal). In other words, who will participate in the program? Will it be volunteer schools? Low-achieving schools? Randomly-selected schools? How will you pick the teachers within each of the schools?

If the objectives are written as a gain score from the start of the program to the end (Month 1 to Month 18) a One-group Pretest Posttest Design is suggested. However, you might consider a stronger design—but you will need to make sure you reread your goals and objectives since they “drive” the design.

The quantitative group designs must be the strongest designs possible, bearing in mind the real-world difficulty of doing action research in public education. If you have an objective that calls for 50 teachers who will receive cooperative-learning training to be compared with 50 comparison teachers, in some way, you may use one of the designs that have Control or Comparison Groups, such as the Nonequivalent Control Group Design. AGAIN, check your goals and objectives—you may need to rewrite/modify them at this point.

When you finish this activity you will have constructed evaluation designs for those objectives. You will probably use the same evaluation design for both objectives, such as the One-group Pretest Posttest and that is ok, as long as it is correct. (-:

Next, reread your two Qualitative Research Questions. How will you carry out the needed research to answer that one question? Who will be involved, the teachers, the students, the parents, others? How many participants will be involved? Which qualitative sampling method(s) will you use? How will you analyze the data? How will you know if the intervention (Cooperative Learning) was successful? The answers to these questions will be the basis for the evaluation of the qualitative portion of your proposal.

Evaluation Design INSTRUCTIONS

For all objectives you will explain the following, using the form sheets that follow.

How will you select the schools, teachers, and students?

If you select 10 schools based on low test scores, then get volunteer teachers from each of the 10 schools by offering $2000 stipends for participating in staff development, you have purposive cluster sampling of the schools and convenience sampling of 5 teachers from each of the schools.)

Name the Group Research Design to be used and explain exactly how you will implement the design. (e.g., if you used the sampling method described above you would use a One-group Pretest Posttest Design if you compared the amount of time the teachers chosen for the study used cooperative learning in their classes at the beginning of the study to the time they used cooperative learning at the end of the study.) By the way, you cannot use time spent using cooperative learning as an outcome because I used it as an example.

Name and explain two possible threats to internal or external validity in the design described above. You need to tell why these two threats are most likely to occur and how you could control them.

Next, address the two qualitative research questions you wrote.

Name the qualitative sampling method(s) you would use. Explain why that method is the best to use for this study.

Describe the participants who will be involved (teachers, students, parents, others). Explain approximately how many, and why they will be chosen to participate.

Describe the type(s) of qualitative data you will collect.

Describe how you would analyze the qualitative findings.

Evaluation Design Form Sheet for Cumulative Activity Part 2

1. Goal 1 Objective A

A. Sampling: How will you select the schools, teachers students?

B. Group Research Design to be used and explanation of exactly how it would be implemented.

C. Possible threats to internal or external validity, named and explained.

2. Goal 1 Objective B

A. Sampling.

B. Group Research Design & Implementation

C. Name and explain threats to validity.

1. Goal 2 Objective A

A. Sampling: How will you select the schools, teachers students?

B. Group Research Design to be used and explanation of exactly how it would be implemented.

C. Possible threats to internal or external validity, named and explained.

2. Goal 2 Objective B

A. Sampling.

B. Group Research Design & Implementation

C. Name and explain threats to validity.

3. Qualitative Research Questions

A. Name the qualitative sampling method(s) you will use. Explain why that method is the best to use for this study.

B. Describe the participants who will be involved (teachers, students, parents, others). Explain how many, and why they will be chosen to participate.

C. Describe the type(s) of qualitative data you will collect.

D. Describe how you will analyze the qualitative findings.