week 8 PPOL 505 DB
Mz Genuine
CHAPTER14OSullivan.docxStrategic managers and entrepreneurial researchers know that unless research findings are reported effectively nothing happens. You may report findings at meetings, in press releases, brochures, project reports, annual reports, and academic papers. In this chapter we argue for clear, focused presentations tailored to the needs of a particular audience. At the end of the chapter we touch on ethical concerns that occur in connection with reporting data.
VARIATIONS IN AUDIENCES AND THEIR NEEDS
Whether you plan to make an oral presentation or write a report, the first steps are to focus on your purpose and the characteristics of the intended audience(s). First, you need to get the audience’s attention. Virtually everyone is overloaded with information. Effective administrators and policy makers may be particularly adept at protecting their time and ignoring information that they do not need or want. Second, you want to prevent having the listeners or readers miss your main point. If you aren’t clear and don’t get their attention they can misunderstand or ignore important findings. Third, you may want to teach your audience members something, influence their thinking, or motivate them to act.
Identifying the audience for an oral presentation is straightforward. Your first question should be “why will people attend the presentation?” Once you have the answer you can tailor the presentation to the audience’s concerns, its level of knowledge, and its motivation to act. Identifying potential readers is more difficult. A report may be passed on to supervisors, staff, agency analysts, interest-group members, professional acquaintances, legislators, or students. Reports may be placed in an agency library or posted on a Web site. To satisfy diverse readers, reports must be clearly written and research procedures should be fully documented. On the other hand, including full details, especially about the methodology, can diminish a report’s readability or an audience’s attention. You can resolve this apparent conflict by putting important information, such as the report’s findings and recommendations, first and placing complicated or technical details in footnotes or appendices. You may also direct an audience to Web sites or other easily accessed sources for additional information.
If you are conducting a study for an organization that you don’t work for, you may want to learn how it normally organizes and presents information. You may attend oral briefings or ask potential audience members to identify presentations that they thought were especially effective. You can use this information to infer what features generate audience interest and involvement. When you read reports, save ones that seem particularly well done.
ORAL PRESENTATIONS OF RESEARCH FINDINGS
No matter whether you see yourself as a strategic manager or an analyst, you need to hone your oral presentation skills. As teachers we have observed talented students who avoid making oral presentations. These students lose valuable opportunities to practice presenting their ideas, listening to others, and phrasing and answering questions. Whether you normally speak to one person, a small group, or a large, formal audience, your ability to explain your work clearly will serve you well. Professionals who feel pressed for time may prefer to hear about a study rather than read through a report. Some people are “oral learners,” that is, they efficiently absorb and understand information they hear. Others value debating information and discussing it with investigators and colleagues.
An oral presentation provides an excellent opportunity to have an impact. Audience members may feel compelled to pay attention. Their interactions may motivate the group or individuals to discuss and follow up on the findings. You can prepare by asking yourself, “Why will people attend this presentation? What do I want them to learn? What action do I want them to take? How can I convince them to take it?” The answers should guide how you organize a presentation.
You should not discount the importance of one-on-one informal discussions of your research. Their informality can be deceptive—don’t overlook the opportunities they provide. They offer an important occasion for others to develop interest in your project. What would you say if your agency head were to ask what you are working on? Wouldn’t you want to generate interest in your current project? Wouldn’t you want to lay the groundwork for a decision based on your findings? A trick that some researchers employ is to prepare an “elevator speech.” That is, they prepare a very short description of what they are working on that is short enough to be said during an elevator ride. You may never be caught in an elevator with a person you want to impress, but if you are prepared you will not waste an unexpected opportunity to sell your project.
An effective presentation requires planning and practice. Select the points you want to emphasize, the evidence you will use to support these points, the order in which the information will be presented, and visual aids. The traditional order for a research presentation—background, methodology, findings, and discussion—usually works well. It develops the material logically. People with training in the sciences, including the social and behavioral sciences, have come to expect it. If audience members are already informed about the program or policy, identifying the study’s purpose may be sufficient. Otherwise, you should describe the program or policy to put the information in context and to help audience members follow the presentation. Usually a brief discussion of the methodology is sufficient. Except for specialized audiences, you can skip the technical details. Remember that while you have learned to pay careful attention to detail and to examine findings from various perspectives, these skills can translate into tedious, unfocused presentations. Avoid trying to cover too much information. Instead concentrate on a few important points and encourage the audience to ask about the details, especially those details that may affect their willingness to accept the findings.
Visual aids may be used throughout a presentation. PowerPoint slides, tables, or graphs focus the presenter and the audience. To select a visual aid, consider whether it communicates the information clearly and effectively, requires special equipment, or slows down the presentation. Too many visuals can bore an audience. Detailed tables and graphics leave people in the back rows squinting or feeling left out. Wordy slides focus the audience on trying to decipher the slide instead of listening to you.
The slide shown in Figure 14.1 is from a presentation on the legislative history of a U.S. health care policy. You might rightly point out the slide is too wordy, but even more confusing is that it lacks coherence. We might start our revision by focusing only on the Social Security Act and Wagner-Murray-Dingell Bill. We would give the year of the Wagner-Murray-Dingell bill. The citations probably could be deleted from the slide. Figure 14.2 shows an improved version. Using PowerPoint you might try your hand at further improving the slide in Figure 14.1.
If your presentation contains a number of tables or slides, or if you expect the audience to take notes from the PowerPoints, you should prepare a handout containing the same information. Alternately you may post the slides on a Web site or e-mail them to participants.
Presentations with lively graphics can be fun to put together, but make sure that they don’t draw attention away from the presentation’s content. Similarly, fumbling around with unfamiliar equipment creates a serious distraction.
Inexperienced presenters may overlook the importance of practice. A researcher who has poured over a study may feel confident in her ability to ad lib the presentation. Unfortunately, she may bog down on the study’s minutiae or move erratically from point to point. Typically, one should practice with an audience of colleagues, team members, or friends. Practice-session observers should make sure that the major points are clearly presented, the statement of key points does not become repetitious or condescending, the transitions are smooth, and the equipment operates correctly. The observers should ask questions about the methodology and the interpretation of the findings. Preparing answers to “hard” questions avoids the embarrassment of stumbling around during the actual presentation. If questions challenging the credibility of the study or its findings go unanswered or are poorly answered, the written report may never be read and its potential impact may be undermined.
|
|
FIGURE 14.1 An Ineffective Slide |
|
|
FIGURE 14.2 A More Effective Slide. |
A written research report should cover the study’s purpose, relevant background articles and reports, its methodology, findings, discussion, and recommendations. An executive summary, a report summary that goes at the beginning of the report, is actually the last part of the report you write. The written report should be a permanent record of what was done, why and how it was done, and what was found. Although the number of people who actually read the report may be small, this written record remains and may be available to all interested parties.
Research findings may be summarized on handouts and Web sites, in brochures or press releases. Typically, to prepare such summaries you extract material from the research report. Readers of summaries miss the details they need to judge the credibility of the findings, to pick up information that may be pertinent to them, or to justify making a decision or taking action. However, the summaries may be the only part of the report that some policy makers and others read. The executive summary provides an overview of the important aspects of the research report. It often is included as the first part of the research report and can also be used as a separate document. We first discuss the structure and content of the research report and then discuss the executive summary.
You should begin the report by identifying the question you are asking and the value of answering it. As appropriate, a report may discuss the program or policy’s origins, implementation history, goals, relevant stakeholders, resources, and activities. The specific information included depends on the report’s audience and its purpose.
To develop the background information, you may cite interviews, documents, and the research literature. You are most likely to include a formal literature review in program evaluations. The literature may justify the study’s design, the variables you chose, the relationships you examined, and how you interpreted the findings. You may present previous research in chronological order or you may organize the discussion around key variables or concepts. You may weave information from the literature into the background presentation, assign it to an appendix, or include it in an annotated bibliography.
The final project report should be comprehensive enough for others to use the report, verify its findings, or replicate the research. The methodology section is key to providing this information. At a minimum it should have enough detail so that readers can decide if the findings are credible and policy makers can use them as evidence. You should discuss how you defined and measured the study’s variables, any intervention you introduced, your sample, when you collected the data, and how often you collected them.
If your study design was an experiment or a quasi-experiment you should describe the intervention, the study population, and how you created the study group(s). Your goal is to provide sufficient information on the design and its implementation so readers can assess the study’s internal validity, and subsequent investigators and policy makers can assess how the findings might apply in other settings. When reporting performance measures or survey results you can limit the methodological discussion to writing about the measures and samples.
In the measurement section you should identify the operational definitions, how you categorized or assigned numerical values, how you grouped values and combined variables to create indicators, and evidence supporting the reliability and operational validity of the measures. Customarily, in quantitative studies researchers report only the findings from mathematical tests of reliability and empirical evidence of operational validity. To illustrate what is included we give a hypothetical example of how to report the operational definition. If an analyst divided the scale into categories, such as high trust, somewhat trustful, and low trust, she could include the information in a footnote.
REPORTING ON A MEASURE
To measure trust we asked respondents to rate the following statements using a 7-point scale where 1 = strongly disagree and 7 = strongly agree; the alpha coefficient was .88.
• The people who represent the funder are trustworthy.
• My organization can count on the funder to meet its obligations to the program.
• My organization feels it worthwhile to continue to work with the funder. ■
In the discussion of the sample you should identify the target population, sampling frame, sampling design, response rate, and when the data were collected. To avoid ambiguity, you should report the initial sample size, how many members of the sample were contacted, how many of those contacted belonged to the target population, how many refused to provide data, and how many supplied incomplete data. If possible you should compare respondents and nonrespondents. Any other sources of nonsampling error should be identified.
Whether you are writing a report or preparing an oral presentation the key considerations of how to present your findings are the same. You need to (1) organize the findings into a coherent presentation, (2) focus on the important findings and avoid overwhelming the audience with unnecessary detail, and (3) decide on how to present the data. Presented with an uninteresting analysis or an overwhelming amount of detail, audience members may stop listening or reading.
Your graphs and tables should complement the verbal presentation and exhibit data efficiently. Attractive graphics and clear explanations allow readers to assess the richness of the data. The location of graphics and explanations and the amount of space devoted to them signal the importance of the information they contain. You should not waste space on graphics that illustrate unimportant or trivial findings; they do not deserve major emphasis.
Tables may show exact numerical values. They are effective when you want to encourage many specific comparisons. Graphs are especially effective for time series and to make simple comparisons. They permit an audience: to pick out long-term trends, cycles, and seasonal fluctuations; to compare different groups or organizations; and to see differences before and after an intervention.
You should take care to avoid ambiguous labels. Spell words out and avoid abbreviations. The following summarizes practices associated with constructing effective tables and graphs.
1. Tables or graphs should have a precise, descriptive title. A title may list by name the dependent variable by independent variable by control variable (if any). Alternatively, a title may summarize a major finding supported by a graphic, for example, “City homicide rates have dropped over the past twenty years.” All variables and their corresponding categories should be clearly labeled and appropriate units (e.g., years) should be indicated.
2. The independent variable normally heads the columns of a table and the dependent variable heads the rows.
3. If percents are used, the percent sign (%) should be entered at the top of each column.
4. The number of cases on which percent figures are based should be indicated. The total number of cases used in the analysis also should be indicated.
5. Statistical measures, if any, should be placed at the bottom of the table.
6. Definitions of key terms should appear as a table or graph footnote.
7. Data source(s) should be identified in the table or graph’s footnote.
8. A good table supplements, not duplicates, the text. The table and its data should be referred to in the text, but you need to discuss only the highlights. As well, tables and graphs should be able to stand alone, that is, readers should be able to grasp the essential information without referring to the text.
9. As you work on preparing tables and graphs, remember to date them; you may even want to note the time. This is because as you analyze the data you may note and correct errors and you may decide on a different, more effective way to group your data. Unless your graphs or tables are dated you may not remember which represents the most recent version.
In the findings section you report objective, verifiable information. In the findings section you organize and present the quantitative and qualitative data; in the discussion section you discuss what you observed about the information and interpret the findings. You may note
■ what seems important;
■ how the findings compare with the literature or stakeholders’ perceptions;
■ findings that were unexpected and your thoughts about why they occurred;
■ implications of the findings for policy making, action, or further research.
Program evaluations, policy analyses, and other studies done for a legislative or administrative body may include recommendations. Recommendations are normative statements about changes that should be made in the program or policy. Although you may feel ill-equipped or uncomfortable in making normative statements or telling clients what they should do, the study’s sponsors may expect recommendations.
You may find that recommendations focus decision makers on what needs to be done and increase the utilization of your findings. Recommendations should naturally follow from the research findings, that is, a reader should be able to figure out from the report’s content why the recommendations were made. In making recommendations you should address only those changes the agency can make; for example, recommending a change in federal program requirements will not be of any value to a local social service agency. In some cases the costs and benefits of adopting a recommendation may be identified and included. Alternatively, you may suggest several options for agencies to consider.
The executive summary highlights a report’s content. The intended audience is the executive who has little time to read complete reports. An executive summary is also useful for many different audiences. Busy administrators and policy makers scan an executive summary to decide if and when to read the entire report or to refer it to an associate. Administrators with a limited interest in the topic skim a summary to keep themselves current. Policy actors may distribute summaries to communicate and endorse the report’s findings. Investigators doing literature reviews can infer if the report is relevant and if they should read it.
Although usually included at the front of the research report, the executive summary is the last part of a report to be written. It includes only information contained in the report, but it can be read and understood independently of the report. In writing an executive summary, you decide what you want a reader to know. For example, you may visualize the impatient administrator who asks “What’s the headline?” You can go through the report and find sentences that concisely describe why the study was done, who the subjects were, how the data were collected, major limitations in the methodology or its implementation, and what the major findings were. You should include any recommendations that were part of the report.
You should use clear direct sentences and visual cues to allow an individual to read the summary quickly. Keep its length and degree of detail consistent with the length and complexity of the report, agency expectations, and the importance of the findings. In preparing an executive summary, you should avoid including too many details; otherwise, its benefits are defeated. The following sample executive summary may serve as a model of how to organize and summarize a report.
Completing a research project, presenting findings, and storing information have ethical dimensions. A joint committee convened by the National Academy of Sciences, the National Academy of Engineering, and the Institute of Medicine identified three sets of ethical issues: research misconduct, questionable research practices, and other misconduct.1 Research misconduct consists of acts of fabrication, falsification, or plagiarism. Questionable practices refer to decisions with regard to data retention and sharing, record quality, authorship, statistical analysis, and release of information. Other misconduct refers to acts that are unacceptable but not unique to researchers—for example, misuse of funds, vandalism, violations of government research regulations, and conflicts of interest. In this section we focus on research misconduct, handling research errors, and record-keeping issues as these are ones that you are most likely to encounter in the course of your work.
Fabrication is defined as making up data or results, and falsification as changing data or results. We assume that you know that fabrication is wrong. Falsification can be a bit more ambiguous. An easy way to falsify results is to drop cases from a dataset. Dropping selected cases can strengthen your statistical evidence or even rescue a weak statistical model. For example, you might eliminate cases that you think have measurement errors, such as when you suspect that incorrect data were reported. If you think that measurement error occurred, you should try to confirm it. However, if you cannot confirm the error you may decide to remove the cases, in which case you must report that you removed the cases, explain why you removed the cases, and indicate how their elimination affected the results. The greater the effect, the more diligent you must be in reporting the decision. A decision that markedly affects the findings should not be buried in fine print.
AN EXECUTIVE SUMMARY
A Program Evaluation of the Vocational Training Programs at Portal
Portal [pseudonym] is a community-based rehabilitation facility whose mission is “to help people with vocational disabilities achieve a sense of self-worth by optimizing their potential to earn their own wages through work.”The facility has two programs to provide persons with disabilities vocational training suited to their needs and abilities.
Study Questions
■ How successful are Portal trainees in obtaining permanent jobs?
■ What characteristics are associated with successful job placement?
■ Is one of the training programs more successful than the other?
Findings
A customer satisfaction survey of trainees of both programs indicated that 22 of the 23 contacted were satisfied with the services they received; and were currently working in permanent positions.
An examination of Portal’s databases found that based on case closure its success rate has declined in the past 3 years. In the first year it had 100 percent successes as compared to the state average of 87.2 percent. By the third year it had 81.3 percent successes as compared to the state average of 82.7 percent. The decrease may be attributed to a change in definition of the term success, which no longer considers probationary employment as a success. Analysis disclosed no differences in race, sex, or disability of clients in either program between those who were successful and those who were unsuccessful.
Inconsistencies in Portal’s databases limited the evaluators’ ability to find information and clarify definitions. A substantial amount of useful information was missing from the databases.
Recommendations
■ Developing a centralized database for the entire agency with clear definitions of database fields, leaving little need for interpretation of information by data entry staff.
■ Collecting more information on processes that could lead to better service for clients and employers.
■ Connecting billing sheets to database to monitor hours of job development and job coaching for each client.
■ Surveying or interviewing clients who have been placed in a permanent job to monitor their long-term success. ■
You need not be overly concerned about avoiding charges of falsification. You simply need to be careful in documenting your decisions and why you made them. The documentation makes your decisions accessible for peer review.
Plagiarism is falsely presenting another’s ideas or words as one’s own. Quoted material should be placed in quotation marks and references cited. Closely following another author’s diction is wrong. You should either use your own words and sentence structure or quote directly from your sources. Relying on the works of others is inevitable in research. No one knows this better than a textbook writer. We have referenced sources that we relied on to write segments of this text or that provided a unique or valuable perspective on the material. We have not referenced sources for ideas and perspectives that we know are part of the common knowledge of social science researchers.
Diligent referencing and use of your own words should be adequate to avoid charges of plagiarism. In our experience, the most common instances of plagiarism are using information from Web sites without citing sources, or editing and presenting another’s work as one’s own. Changing words and dropping sentences also constitutes plagiarism. If a report is to be published, you need to pay attention to copyright laws. You must get permission from the copyright holder to reproduce graphs, tables, long quotes, and other materials, including song lyrics, poetry, and cartoons. However, government documents are not covered by copyright, and their contents can be reproduced without obtaining permission. Nevertheless, you should use standard referencing procedures to cite a government document.
Error is inevitable in research. The joint committee convened by the National Academy of Sciences, the National Academy of Engineering, and the Institute of Medicine identified four potential sources of error: the accuracy and precision of measurements, the generalizability of experiments, the quality of the experimental design, and the interpretation of the practical significance of the findings. To reduce the diffusion of incorrect knowledge, you should fully disclose your research procedures, and acknowledge and correct errors.
Full disclosure allows others to scrutinize the research. They may find errors by examining the research documents or by attempting to replicate the research. Concealing limitations in your methodology amounts to deception. Research reports should clearly identify and evaluate the limitations. In fact, the more troublesome a limitation, the more emphasis it should receive.
Complete information on research procedures can overwhelm readers with details and seriously diminish a report’s effectiveness. The professional standards for program evaluation may serve as a useful guide. To provide useful information, the standards advise evaluators to write clearly, present information that their audiences can understand, and indicate the relative importance of their findings and recommendations. To achieve full disclosure, the standards advise evaluators to state their assumptions, their constraints, and how readers may obtain full information on research procedures, including data analysis.2 The standards relieve evaluators of the burden of providing complete research information in every report, but they must take reasonable actions to ensure the accessibility of the database and documentation.
Data must be saved and be accessible to allow research audits, replication of results, refinement of the analysis, additional analyses, or incorporation of data into other research designs Research data include completed data collection instruments, protocols for collecting and entering data, descriptions of experimental procedures, data files, computer printouts, field notes, videotapes, audio tapes, CDs, and DVDs. With this information the research can be reconstructed or replicated. Audits may be a component of ensuring integrity; auditors can investigate charges of falsification or fabrication.
Misunderstandings can be avoided if you and other involved parties agree on who will retain the data, how long they will be kept, and the conditions governing data sharing. Deciding who can access the data and issues affecting the confidentiality of respondents should also be agreed upon. If such agreements are not made explicit, administrators may find that they cannot access the data for further analysis. Some stakeholders, such as legislative bodies, may be unhappy if they cannot access and verify data collected by a funded program. Managers and researchers need to take into account others who may have legitimate reasons for reviewing the data.
Investigators may have discarded or misplaced data, information on how to retrieve data may not exist, or research documents may be scattered. If any of these situations exist, managers or investigators have not met their research obligations.
The exact amount of time for which data should be retained may depend on its assumed shelf life, agency customs, research agreements, or contracts. Completed questionnaires and records, needed primarily for research audits or to correct data-entry errors, may be kept the shortest time. Data files, data dictionaries, data collection instruments, and other research protocols may be saved indefinitely. Materials relating to a published work may be kept longer than those relating to an unpublished or in-house study. If an agency centralizes data collected on its behalf, it should develop a mechanism to exert quality control so that it does not get overwhelmed with useless data.
Before presenting your findings, identify your audience, and decide how to get its attention and how to motivate its members to follow through on the report’s findings and recommendations. Perusal of agency reports or listening to oral presentations may suggest how to organize information effectively for an agency.
Both oral and written presentations tend to include the same material, although the amount of time or detail will vary according to the audience. The sections of a research report are an overview or executive summary, a statement of the study’s purpose, background information, the description of the methodology and results, along with supporting analyses, discussion, and, as appropriate, recommendations. Important information should be clearly distinguishable and receive major emphasis. Remember that written reports may be passed around. You should not assume that every reader will know about a study’s background or design. Consequently, you need to include necessary elaboration and documentation in the final written report. The methodology and findings sections in particular should be sufficiently documented so that the careful reader can make her own judgment on the adequacy of the report.
Oral presentations involve a captive audience. Still, the presenters are not guaranteed its attention. They should practice a presentation so that it flows logically and listeners can easily follow it. Visual aids should be easily seen and interpreted. If visual aids require special equipment, the presenters should know how to operate the equipment with minimal effort.
Your research reports should conform to ethical practice. Plagiarism, fabrication, and falsification clearly violate research ethics. Although human error is inevitable, you should take reasonable steps to minimize errors in your work. You should verify that that the correct data were entered, any modifications to the data were done correctly, and that the computer outputs were read and interpreted correctly. The descriptions of the study’s methodology should disclose problems encountered, design limitations, and other information needed to facilitate critical review. Where possible, you should let peers review your work. If you later find errors you are expected to acknowledge them.
When initiating a study, investigators and administrators need to agree on who will retain the data, how long data will be kept, and any conditions governing data sharing. During the study you should write up descriptions of research processes and decisions and label all research documents. Keeping a research log, where you record what was done, when, and by whom, will allow you to track and remember the various decisions and events that occurred. As the study ends, you should organize the data and documents and save them so others can use them to verify the research or conduct further analysis.
Bardach, Eugene, A Practical Guide for Policy Analysis: The Eightfold Path to More Effective Problem Solving, Second Edition (Washington, D.C.: CQ Press, 2005). See especially “Tell Your Story,” pp. 53–60.
Grob, George, “Writing for Impact,” in Handbook of Practical Program Evaluation, edited by J. S. Wholey, H. P. Hatry, and K. E. Newcomer (2nd ed.; San Francisco: 2004, pp. 604–627) Grob’s chapter covers the organization, presentation, and content of evaluation reports. The “Handbook” includes chapters on all aspects of program evaluation including research design, data collection, analysis, reporting and other topics.
Harrington, R., and Rakdal, S., How to Wow with PowerPoint (Berkeley, CA: Peachpit Press, 2007).
Publication Manual of the American Psychological Association, Sixth Edition, second printing, is the guide often used in the social sciences to cite sources. It offers guidance on the format for in-text citations, endnotes, references, reporting, and other topics.
Smith, Catherine, Writing Public Policy: A Practical Guide to Communicating in the Policy Process, Second Edition (New York: Oxford University Press, 2010). The book contains sections on the research process and sections on presenting information in a variety of written and oral formats.
Strunk, William, Jr., and White, E. B., The Elements of Style, Fourth Edition (New York: Longman, 2000) is a classic with rules for grammar and composition that have held up well for over 90 years. The book is available online.
The Chicago Manual of Style, Fifteenth Edition (Chicago: University of Chicago Press, 2003) covers referencing and copyright laws.
Walker, T. J., How to Create More Effective Power-Point Presentations (New York: Media Training Worldwide, 2003).
A useful online resource for oral presentations may be found at http://www.the-eggman.com/writings/keystep1.html
CHAPTER 14 EXERCISES
This chapter has two exercises. In each you are asked to review and assess a presentation.
• Exercise 14.1 Learning from Practice: From Written Report to Oral Presentation asks you to read a written report, identify how it organized and presented the information, and outline an oral presentation based on the report.
• Exercise 14.2 Learning from Practice: Critiquing an Oral Presentation asks you to view and critique an oral report found on YouTube or another Web site.
EXERCISE 14.1 A Learning from Practice: From Written Report to Oral Presentation
Policy groups typically conduct research and present their findings in a variety of formats. To begin this exercise go to the Web site of such a group and download a research report. It can be reporting on public opinion, on program performance, or an evaluation. You may want to start your search at one of the following two sites.
The Urban Institute (www.urban.org)
The World Bank (http://econ.worldbank.org)
Section A: Getting Started
1. The Web site I visited was that of [name of organization and URL].
2. The name of the report I accessed was [name of the report].
3. The major divisions, section, or chapters of the report were [name(s) of divisions or chapters].
4. If it had an executive summary note how long it was. List in order its major sections.
5. What methodological information was included in the body of the report and in the appendices?
6. What did you notice about the tables and graphs contained in the body of the report?
a. What types of graphs were used?
b. How were the tables and graphs titled? (Did the report just name the variables or present a statement highlighting the findings?)
c. What statistics were reported?
d. Note the details included in a table or graph and comment on how the table or graph was discussed in the text. (Be sure to report the number and title of this graph or table.)
7. Were other research products produced from the same research project, for example, press releases? For each product found, summarize what parts of the research report were highlighted?
8. Identify a target audience that would be interested in listening to a presentation on the report. Justify your choice.
a. Prepare an outline for a presentation.
b. Create one or more PowerPoint slides to describe the study’s methodology.
c. Develop a list of tables and figures that you would include. Give each table or figure a descriptive title and briefly describe its format, for example, a bar chart.
d. Create one or more PowerPoint slides of the findings that you would present.
Section B: Group Exercises
Group members who have selected similar reports (evaluations, citizen surveys, or academic papers) should work together in groups of two to four. Each group should develop a “Guide for Effective Written Reports” and a “Guide for Effective Oral Presentations.”
1. Introduce each other to each report’s purpose and intended audiences. From this information you may modify the guide’s title to insert the type of report or audience, for example, “Guide for Writing Effective Reports of Citizen Surveys.”
2. Compare each of the following components: organization of the executive summary and the report’s organization and content (report sections; their order; use of visuals amount of detail in the discussion of methodology, tables, and figures; content and presentation of findings and recommendations; content of appendices). From this information develop your writing guide.
3. Compare and discuss each other’s oral presentation outline: its organization, presentation of methodology, and presentation of findings. Based on this review develop your oral presentation guide.
4. Come together as a class and identify
a. common elements in the guides;
b. elements that are unique to one or more guides. Discuss whether these unique elements belong in guides for specific types of reports or if they should be included in all the guides;
c. elements that seem to give contradictory advice. Discuss and try to resolve the differences: are they due to different types of reports, different target audiences, or different perceptions of what is effective?
EXERCISE 14.2 Learning from Practice: Critiquing an Oral Presentation
Go to YouTube. Find and watch a research presentation.
Perform the following assignments:
1. Describe and critique the presentation’s organization.
2. Critique the presentation’s clarity in establishing its purpose, describing its methodology, presenting its findings, discussing its findings.
3. Critique the presentation’s use of visuals.
4. What advice would you give to the presenter(s) if the presentation was to a professional group instead of to a general audience? Would your advice be different if the presentation is to a legislative body?
NOTES
1Panel on Scientific Responsibility and the Conduct of Research, Responsible Science: Ensuring the Integrity of the Research Process, Report by the U.S. Committee on Science, Engineering, and Public Policy (Washington, D.C.: National Academy Press, 1992, Vols. 1 and 2).
2The Joint Committee on Standards for Educational Evaluation, The Program Evaluation Standards: How to Assess Evaluations of Educational Programs, Second Edition (Thousand Oaks, CA: Sage Publications, 1994). Seventeen standards address reporting issues. The standards’ application is not limited to educational studies.
