EDMG611Wk6

Analysis Now it's time for the heart of the matter. Time to get down to the nitty-gritty, the nuts and bolts, the cause celebre. It's time to analyze. To this point, the project has consisted of learning what is already known of the topic through the literature review; defining and gathering data which will take the knowledge off into new directions; and selecting a theory-based methodology for examining the data in order to develop this new knowledge. What all these have in common is that they're all essentially knowledge that has been developed and organized by others. The analysis will be different. The analysis will be the product of you.

Be sure as you begin the analysis, that you remain aware that there is a difference between research and advocacy. The researcher approaches the data hoping to find a result but without a preconceived idea of what that result will be. The analyst has no opinions, no success markers, and no axe to grind. In contrast, the advocate approaches the data in an effort to make the data say what the advocate wants to hear. In research, that's called bias, and is to be avoided.

What is analysis? Well, in the broadest definition, analysis is the process of applying logic and reason to facts and evidence in order to draw reasoned conclusions. In the quantitative world, the facts and evidence are represented by numbers and the logic and reason are represented by statistics. The reasoned conclusions are statistically significant results that indicate that something outside of the norm has been made to appear from the data. Although the most rigorous and widely accepted type of result, the statistical significant result is also the most limited, since it can only describe itself as content without context. For the most part - multiple arrays of data can be seen to have context, but it tends to be limited and difficult to discern.

Case studies are often presented in arts and health evaluation. They are used to highlight participants’ stories of the impact of arts projects. A well written case study can powerfully convey the impacts of an arts project. However, weak or poorly presented case studies can undermine credibility. Evaluation case studies are not necessarily research, but they can be strengthened by drawing on good research practice. Case study research involves an in depth consideration of issues including the research question, case selection, representation, reporting and ethics. A useful review of case study research is provided in the book by Gomm et al. (2000). Here we discuss these issues as they relate to arts and health evaluation.

Case studies offer a means of presenting a rich and detailed picture and telling a strong story about different aspects of a project. A single case can be any person, object or place that is the focus of evaluation. Case selection should be related to a research or evaluation question. A case study is an umbrella for a range of different forms of inquiry. The content of the case study is generated by a process of collection and analysis. Case studies can utilise different methods including monitoring data, interviews, focus groups and observation. Sometimes quantitative data are included in case studies although the analytic process for case study research tends to be qualitative. Case study data are combined in a narrative format to tell a story.

It is important to remember that the resulting story is particular to the case. It is not a story about the project as a whole, hence it cannot be generalised to others or used to ‘speak’ for the experiences of a group. This is why case studies need to be combined with other methods if they are being used for the purposes of evaluation in order to give a broader picture of the project impacts. However, when used in this way, case studies can be a useful way of highlighting particular experiences and issues that have emerged from the evaluation. Case studies involve a trade-off between detailed understanding and the ability to make further generalisations. This means that it is very important to consider the reasons behind case selection.

Case selection Case selection should be determined by the evaluation

question. There should be a clear reason for choosing a particular case as being of special interest. Perhaps there is a specific need to find out how a person with particular circumstances has responded to an activity. Or perhaps there is a more general desire to explore a particular dimension of a project, such as how it fits with national policy. It is important to clearly identify the reasons for choosing a particular case.

When to use a case study approach? Below is a summary of the pros and cons of the case study:

Pros • Narrative - the beauty and power of a case study is in the

specific details and the particular story it tells. • Rich picture – a case study can provide depth but not

necessarily breadth. • Contextual information can be included, giving a holistic

picture that broadens our understanding. For example, case studies can allow consideration of a broader range of psychological, social, political, economic and geographical influences than other forms of evaluation when examining the impact of an arts intervention.

Cons • To the critics, case study is not a method. There are not

recognized procedures for data analysis in case studies which often rely on narrative means of presenting data.

• Generalisation is not possible from single case studies • It is difficult to combine data from multiple case studies,

although case studies can be compared. • Case studies are time consuming • To make sense of case studies it is necessary to

understand the specific circumstances, time and space in which the activity took place.

What (and who) am I writing for? In any evaluation design, the methods used to collect data affect the findings. A case study can usefully fit into a wider evaluation provided that its structure and narrative fit the evaluation methods and overall objectives. Lengthy case studies are difficult to include in project reports, but if the underlying methods are rigorous (including sampling, data

Creative and Credible is a knowledge exchange project between the University of West of England and Willis Newson, funded by the Economic and Social Research Council.

WRITING AND REPORTING CASE STUDIES Written by Mel McCree and Norma Daykin with the Creative and Credible Project Team.

www.creativeandcredible.co.uk

collection and analysis) then it may be possible to summarise a case study effectively in a few sentences. This is often needed in project reports to provide a rich detail in order to enhance broader reporting of the impacts of the project on participants and staff.

Writing a case study There are four basic steps in case study preparation: sampling and case selection, data collection, data analysis, and finally the writing to draw it together. When you reach the writing stage you may find you need to go back and find out even more information. Like many research processes, case study development can go around in an iterative loop.

Structure Firstly describe the question you have - think of a good detective novel where the crime happens first, and then the audience follow as the case is solved. Case studies are often written chronologically, for good reason, as we need the thread of the story.

From there, a simple structure could be: • Introduction - how the case related to your question. • Background to the place and context. Who is in the

context? • What happened? • How does that answer your question?

The need to anonymise case study data Always anonymise people in a case study. This may be challenging where someone is particularly proud of their story, yet anonymising means that the story can be used for years to come whether or not they change their mind about feeling proud. Different processes that researchers use for this include changing not just the name and location, but other identifying details surrounding the case, the setting and other people in it. You could also combine a few different people in an instrumental study to show a particular aspect of your data, and noting that you have done this.

Conclusion: are case studies important? Within the call for ‘robust evidence’, case studies are often dismissed as anecdotal. Yet case studies can be ‘robust’ when done well, and they offer a credible form when used in qualitative research. A well written case study speaks to its audience more strongly than other forms of data presentation. There is power in a good story and case studies can help to elucidate and evaluate an issue. However, there are good case studies and poor case studies. The quality of the final story depends as much on the methods used to collect and analyse the data as much, if not more, than as the quality of the writing.

References

Gomm, R., Hammersley, M., Foster, P. eds (2000) Case Study Method. London: Sage/The Open University.

Creative and Credible is a knowledge exchange project between the University of West of England and Willis Newson, funded by the Economic and Social Research Council. www.creativeandcredible.co.uk

Qualitative Analysis Qualitative analysis is all about contextual relationships. Where quantitative analysis can provide the answer as to whether something important has happened, qualitative analysis has the potential of explaining why it happened. Qualitative analysis can explain the spike in foreclosures, the drop in birth rates, the melting ice caps, whereas quantitative analysis can only tell you that they exist. That's why qualitative analysis is at the same time more complex and more subject to academic criticism - the human and natural condition is an infinitely complex thing. The quantitative statistic can describe it in a number. Event the simplest qualitative descriptor adds complexity. What color is the sky? Blue. What does 'blue' mean? Which blue? Is it the same blue at 8am as 9am? Why? And so the boundaries around the case study become even more important to define as the complexity increases.

The literature and the websites have many accepted theoretical techniques and processes for accomplishing both types of research. All have merit, and all are constrained by the fact there are questions that each cannot answer. This is the art of research - to choose the question, the data, the method, and the analysis that will help us understand humans and the world around them. In anything you do in this class or in life, don't shirk the analysis. In every transaction in life, it's appropriate to ask: What are we talking about? (boundary); How did we get here? (background); What are the facts? (data); How should we interpret them? (method); What do we conclude? (analysis); What should we do about it (recommendations)? To the scholar, these are the formula to success.