A6- Research Plan Overview

1

Global Feedback Press Release for Research Plan Stage Two

I have finished evaluating your Stage 2 of the Research Plan Assignment (I will get to your 4th DB ASAP but I

thought it best to comment on the Research Plan ASAP so you have my feedback as you work on Stage 3). Some

(well most) of you are making progress; I can tell, however, some of you are still really struggling to get your minds

wrapped around qualitative research. (By way of example: some of you proposed doing a phenomenology with 40,

50, or 60+ (150) participants!). Have no fear--that's precisely how you should feel but I promise, if you do the work

and the readings, it will come to you. That’s why the RRO is scaffolded—to hopefully help you learn as you go!

As alluded to, on the plus side, however, it seems that for most of you, the scaffolding approach to this class— along with the extensive peer and instructor review—is working! Corporately, as a class, each stage improves, sometimes exponentially. So congrats for that!

A few (okay more like a lot of ☺) observations regarding the research plans which I would like you to consider as you prepare for Stage 3. FYI—I realize some of these items have been mentioned before, either to some of you individually or to the class in one of my global press releases. And again, for some of you, there will be some redundancy since you already “get it.” Nevertheless, they are all important—especially moving into Stage 3 and are therefore (in my mind) worth reiterating. Here they are:

1. Surprisingly, a few of you did NOT implement the feedback from Stage 1 and this really impacted (negatively) your score for Stage 2. Please pay attention to all feedback!!

2. Participants--particularly the number of them. Qualitative inquiry emphasizes quality of data, not quantity. Thus, having an n in a qual study of 10 is not unusual (our program wants 10-15 for a phenomenology for example); having an n, however, of say 40---not likely--simply b/c of the amount of data that would produce. For example: in my study, I interviewed 8 people for about 90 minutes each. That translated into 150 pages of single spaced text! That's a lot to read, transcribe, and go through in order to come up with solid themes. Remember--the goal is "deep" not "wide."

3. Speaking of participants, we do NOT use “random” or “stratified” sampling in qual; rather we use things like “purposeful,” “criterion” and/or “snow-ball” sampling (look them up in Creswell & Poth or even other sources). We also seek to achieve “maximum variation” among participants (you can look that up too ☺).

4. Data collection: As mentioned in the videos, The primary means of data collection in all qual studies are open ended, semi-structured interviews. Yes, we do conduct participant observations (though less and less all of the time for a variety of ethical and practical reasons) and analyze documents, and we do focus groups, photo narratives, etc. but interviews are the primary source of data. Surveys are used, but primarily either as a means of the participant pool/participant screening, or to get demographic data. Survey research in and of itself is primarily done quantitatively—especially those of the “open-ended” variety. Several of you prioritize surveys; that’s not qual but quant!

5. Also, with regards to interview questions: some of you have (wisely) figured out the benefits of “cherry picking” (adapting) interview questions from the articles you reviewed for the article critiques—and in other qual research studies you have been reading. Good for you! That’s not plagiarism—that’s just what I like to call, “working smarter—not harder!”

6. One more thing RE: interviews (and your interview guide): Begin your interviews with a broad, ice-breaker kind of question to get the interviewee “in the flow” and comfortable (and feel safe).

7. While we do use instruments in qual research, they are not used to collect data per se but rather for descriptive purposes or to help triangulate the data. A good example of how instruments can be used can be found in the study by Jama Davis (Center for Counseling) on Loneliness and Pastors' Wives. She used the UCLA Loneliness scale NOT to collect data and analyze but merely to provide evidence that yes, her participants

2

were in fact lonely (and to do some comparing/contrasting). Here’s a link to her study, FYI: http://digitalcommons.liberty.edu/doctoral/49/

8. Speaking of dissertations, here’s a link to all dissertations in our program (the EdD in the Department of Community Care & Counseling)—just in case you missed it: https://digitalcommons.liberty.edu/dccc/

9. Some of you still seem to be confusing research questions (RQs) w/interview questions. RQs (especially the first or central research question) are basically the purpose of the study in question form. The RQs provide the guiding structure for the study. Interview questions are the questions you will ask participants in order to try and answer the RQs. The give away for me is seeing the word “you” in the question.

10. As mentioned last time, once you figure out which design you will be using (phenomenology, case study, etc.), make sure you include that in your title (A Phenomenology of Teachers’ Perceptions… or a Case Study on Characteristics of….something like that).

11. Make sure your “problem” is grounded in the most recent literature, not in just your own personal situation/context, beef, or interest.

12. Speaking of your “beef”, in the “role of the researcher” section, make sure you articulate your “beef” with the issue/problem; why are you interested in it? What is your relationship to the issue/phenomenon and to the site, setting, participants, etc. in addition to who you are as a researcher?

13. Make sure you add to/build previous sections (lit review and methods for example) in future assignments. In fact, one of the things a few of you lost points for this time is failing to expand your literature (last item in the rubric); each which your understanding of the literature, topic, and method SHOULD be growing; just an FYI. Remember, I have your stage 1 with which I can compare your stage 2 submission,

14. Additionally, it is important that (some of you) understand that looking at the effects of programs or clinical interventions on measurable constructs like child behavior or academic achievement are best suited for quantitative studies. Students often try to turn a topic best suited for a quantitative study into a qualitative study by adding words like "perceptions of..." or "beliefs about...," however, when it comes to this era of standards based reform and educational accountability, researchers and practitioners are most concerned with the measurable outcomes of programs and interventions, and these are best established quantitatively as the purpose is to generalize. In other words, programs that are "perceived" as effective but are not actually effective when looking at achievement data are not (or should not) be adopted. If you are interested in establishing the efficacy of a program or intervention, stick to quantitative research ☺.

15. Some still are stuck in a quant mindset which I get, evidenced by your desire to do “random selection” of participants, some kind of two (or more) group comparison and/or an efficacy study, or something regarding the relationship b/t two variables—none of which are qual. I suggest you review some qual studies to see how they are constructed (Purpose, RQs, etc.).

16. With regards to design: For many of you, there seems to be some confusion between case study and phenomenology; this is understandable. In a way, all qualitative research is phenomenological since it all seeks to describe something. But what makes a study a case study is that it focuses on a unit of analysis (the case) and NOT the topic to be investigated. As Merriam (2009)1 illustrates, “For example, a study of how older adults learn to use computers would probably be a qualitative study but not a case study, the unit of analysis would be the learners’ experiences, and an indefinite number of older adult learners and their experiences using computers could be selected for the study. For it to be a case study, one particular program or one particular classroom of leaners (a bounded system) , or one particular older learner selected on the basis of typicality, uniqueness, success, and so forth, would be the unity of analysis” (p. 41).

1 Merriam, S. B. (2009). Qualitative research: A guide to design and implementation. San Francisco: Jossey-Bass.

3

17. One more thing about case studies and phenomenology: If you are planning a phenomenology, make sure you choose participants from multiple sites and not just one as doing so (sampling from one site) makes the design more in line with case study. I suggest to my students 3 or 4 sites if possible.

18. Okay, one more one more thing: Make sure you look at how data is analyzed AS PER THE DESIGN. Case studies have slightly different ways of analyzing data as compared to phenomenology. You may need to dig a bit into the design (look at some of the examples I have provided) to see the steps in analyzing data.

19. I have found that everything boils down/leads back to the PS (purpose statement). So, make sure you write yours as tight as possible; make sure it expresses precisely what it is you want to study, that it is so clear anyone could read it and get it. See below for what that might look like.

20. With regards to the Site/Setting: For a variety of ethical and logistical reasons, rarely, if ever, can you do research at a site in which you work; this is especially true if you are a leader/administrator or have any position of “power” or authority. Keep that in mind as you move into Stage 3.

21. Speaking of the site: Some of you want to do research at your own “site” (i.e. school, etc.) with participants you may know; be advised that—in spite of your desire to “bracket” your assumptions—this (most likely) would not be possible as it could/would lead to researcher bias and undermine the trustworthiness of your study. There are exceptions of course, but they are rare.

22. One more thing about the site: Normally we allow participants to choose the when and where—time and place—of the interview; remember the focus is on them and they are doing us a favor by sharing with us.

23. If you insist on doing research that includes children under 18 as participants (not something I would do ☺), remember you need assent/consent forms from BOTH the children AND their parents/guardians.

24. I also remind you to avoid any/all “yes/no” questions; qual research is about deep, thick, rich, description which you cannot get with a yes/no (or for that matter, any one-word) question.

25. Avoid “will” statements; stick to words like “may” or “hope to” b/c you do not know what you will (or will not) find; that’s why you are doing the research in the first place!!

26. Also, given that the study is being proposed (and not completed) you should write in future tense. Past tense is reserved for when the study is completed.

27. Make sure you include an APA current edition formatted reference page and make sure ALL citations are listed on the ref page. Also please PASTE your plan into the thread (no worries about formatting); please do NOT include it as an attachment.

28. Don’t forget that Appendix B is for other (in addition to interviews) data collection methods (like observations, focus groups, document analysis, etc). If you are planning on doing observations, then make sure you state that you will develop and utilize an observational protocol/template for consistency of observations (see Creswell & Poth on this). If you are doing a focus group, artifact or document analysis, etc., please provide some detail on these methods as well.

29. One more thing about observations: It is difficult to “observe” in most situations because from an ethical (IRB) perspective, you would have to get approval from every person in the room (and if the learners are minors, from their parents as well). Just one “no” and the observation cannot be conducted.

30. Speaking of Appendices: For Stage 3, you need to include Appendix C and D. Historically, students tend to underestimate the costs and time of completing this program and doing a dissertation. FYI: The quickest I have ever seen one of our students go through the dissertation (qualitatively) is three semesters (once they started the dissertation). Also, for Appendix D—you may want to check out Liberty’s Institutional Review Board (IRB) website as it provides some good assistance in this regard.

4

31. Don’t worry if you don't get it 100% yet, this is a work in progress. Still, please make sure you read carefully Creswell & Poth and other qual books so that you will be sure to use qual terminology and write qualitative research questions. Same is true of participants--they are not SUBJECTS and you do not have a large "n" in a qual study--read Creswell and Poth!!!

32. Speaking of Creswell & Poth, make sure you include at least one qualitative reference; you can even cite Creswell & Poth!

33. Please submit a CLEAN copy with each stage (i.e. w/o comments, track changes, or other editing marks). I have the previous stages and can compare current with previous iterations.

34. Finally, FOLLOW DIRECTIONS (including what is in the rubric) CAREFULLY and most of all, EDIT! EDIT! EDIT!!! You are doctoral students and writing (along with following directions) is fundamental to this little thing that lies ahead of you called a "dissertation" so please, do it!! Some of you (more than I would hope ) lost significant points needlessly simply because of poor editing and/or not following directions with regards to what should—and should not—be completed in Stage 2 so please, please pay attention to the details. This will serve you well as you move to the dissertation!!

Whew!! That was a truckload of information and thankfully, that’s about all for now.

You are doing pretty well overall; remember: this is a scaffolded process, which is why you are doing the plan in stages, and not all at once—so hang on! As always, email me if you have any questions.

Have a great week! And thanks!

Dr. Milacci

PS: As you may know, the open-ended, semi-structured interview is the primary means of data collection in all qualitative research! However, for this assignment I am asking you for at least three (3) types of data collection. Some of you seemed to struggle with this so here are some ideas:

• Focus groups—virtual or face to face; if you are doing phenomenology and have multiple sites to choose participants, you could have one focus group/location/site

• Document analysis—any pertinent documents that may “shed light’ on the case or phenomenon of interest (case studies also have artifact analysis)

• Journaling—have participants journal a few times about the experience or case

• Writing prompt—just like it says! Give them a prompt and have them write about it

• Cognitive representations—Here is an excerpt from a dissertation that focuses on pre-martial counseling

workshops: At the end of each couple’s first interview, the couples were asked to gather white paper and colorful drawing instruments of their choice (i.e., crayons, colored pencils, markers, etc.). The researcher then asked the couples to take a few minutes to reflect upon their thoughts and feelings of having gone through the workshop. Once they had reflected, each partner was asked to draw an image, with or without words, that described their thoughts and feelings of having gone through the workshop.

• Photo narrative--The photo-narrative method invites participants to answer research questions by first taking photographs and then talking to the researcher about them.

• You can search for more if you want!

PPS: For those of you who are struggling to develop a purpose statement, here is a template that we provide students in some of the dissertation classes—see if it can help you tighten things up.

The purpose of this _________ (phenomenological, grounded theory, ethnographic, case) study is to _______________ (understand? describe? develop? discover?) the _____________ (central phenomenon of the study) for _____________ (the participants) at __________ (the site). At this stage in the research,

5

___________ (central phenomenon) will be generally defined as ________________ (a general definition of the central concept). The theory guiding this study is __________________ (identify theory and cite theorist) as it _____________ (explain the relationship between the theory and your focus of inquiry).

Hope it helps! Thanks! Oh and don’t forget to let me know if you cannot see my feedback to you individually (and make sure you implement it in Stage 3)!! Thanks!