Final Project

U.S. News, a media organization, release an annual report in which it ranks world countries with respect to adventure, moves, citizenship, open for business, cultural influence, power, entrepreneurship, quality of life, and heritage (Best Countries 2020, 2020). U.S. News ranking of the countries are global, and involves data insights and international news as critical component for constructing the index. There are a Ttotal of 73 countries in the ranking are seventy-three with score for each category ranging between 1st and 73rd. This year number one country in the overall Best Countries is Switzerland, followed by Canada, Japan, Germany, and Australia. United Kingdom ranks sixth, while United States ranks seventh in the global ranking of Best Countries. For example, United States overall ranking is seen with third, fourth, and first position in entrepreneurship, cultural influence, and power. In a similar way it rankings 33rd in adventure, 15th in citizenship, 18th in heritage, 26th in moves, 45th in openness for business, and ranks 15th on quality of life in the ranking of total 73 countries. Best Countries Report is released on annual basis to understand the perception of people about a country, and it provides easy to understand comparison of countries for decision-making in businesses, academia, media, governments, civil society activists and organizations. The usability test allows understanding of the document in terms of usability of the reader, that is, the effectiveness of the documents article in conveying relevant information in an effective and efficient manner. The A method of conducting usability test was conducted with online participants using aadopts snow-ball sampling method that allows the investigators to ascertain the feedback on Skype calls. The results of usability test suggest that the document is not complicated in terms of layout and writing style, and provides easy access for users to the available information. ABC Company is a third-party which conducted the tests on the request U.S. News for improving the document in reflecting key information in more effective manner. Comment by Joe Sample: Most style guides would italicize the title. U.S. News and World Report is the title of a periodical but I am not sure if U.S. News is classified as a “media organization.” Comment by Joe Sample: This is a little confusing. Comment by Joe Sample: This information can be in the report itself or in an appendices. But the point of this introduction is to introduce the report that the reader is reading. Comment by Joe Sample: You already used the word “effective” in this sentence. Comment by Joe Sample: What does this mean? Are you reporting someone else’s results or your own?

Insert a page break. An executive summary typically is on a page by itself.

The ‘Best Countries 2020’ is a brief report of twelve pages, including title page and briefly divides the content of the report in three sections or chapters. First section is the tabular ranking of seventy three countries with associated scores on nine variables, and an average figure of these variables define rank of the countries at index. The second and third sections of the report are methodology and frequently asked questions (FAQ). The text-based and user based testing provides evidence that the ‘Best Countries 2020’ is a brief analysis of the ranking of world’s seventy-three nations on nine variables. As the document is divided into three section with first section in tabular form ranking 73 countries along with their nine variable scores. Participant’s feedback highlights. The next two sections of methods and frequently ask questions are an easy to understand approach. Methods section provide details of the construction of index, followed by simple questions and answers that responds to possible queries of the document. The document testing through user-based and text-based testing also highlights the effectiveness of the document in accordance with Plain Language principles. Comment by Joe Sample: Think of the executive summary as the part of the report that very busy people would read. They read the executive summary to decide if they want to invest the time to read the rest of the report.So the description here of the document is probably not necessary. Comment by Joe Sample: This is not stated correctly. The tests did not provide “evidence” of the physical characteristics of the report. Comment by Joe Sample: This is what the reader expects to encounter in an executive summary.

The testing methodology is based on already designed evaluation plan for testing the document, ‘Best Countries 2020’ report through text-based and user-based testing types. The users of Best Countries 2020 report are diverse in nature, ranging from student and researchers from academia to government officials and civil society organizations. The two text-based readability testing involves following: Comment by Joe Sample: Different style guides treat titles differently. MLA for example uses the big/little rule: Big works (entire books or magazines) are in italics. Little works (chapter in books or articles in magazines) are in double quotes. I am not sure what the rule is for this particular report.

1. Flesch Reading Ease. It ranks the chunks of text from the document with 100-point scale, and an easy to grasp text means closer to 100-points on test. A reasonable score of 60-70 falls into acceptance of the document as easy to read. Comment by Joe Sample: You can devote some attention to describing the tests themselves but remember that the tests are only the means you used to analyze the document. If you think the reader is unfamiliar with the tests then you can direct them to more extensive explanations in the appendices, but here you only need a sentence or two about the nature of the tests in general. The point of the methods section is to tell the reader what you did in testing this particular document.

2. Flesch-Kicaid Grade Level. The test provides score equivalent to the U.S. grade school level, and for example an 8.0 score signifies usability for an eighth grader.

Additionally, evaluation plan for thorough user-based testing adopts concurrent testing through following:

1. Exploratory Testing

2. Assessment Testing

3. Validation Testing

4. Beta Testing

Concurrent testing adopts realistic scenarios to which use responds, and they are formulated with the available seven categories of each rank. For example, when asked about the ease of doing business, many users correctly identified the top five countries with high ranking on the variable, as opposed to top five countries, for business point of view. However, social acceptability and pluralistic culture is more evident in case of North European nations, and Canada. Comment by Joe Sample: You appear to be citing research here. You need to indicate where this came from. Also, again, you are defining test here as opposed to telling the reader what you did in testing the report.

Goals of Text-based Testing:

The goals of text-based testing are two fold, which is, improving comprehensibility and accessibility of the text, and improving effectiveness of the users. Additionally, understanding writing level is also critical for the report because it explains the type of readership comfortable with the content of document; moreover, establishes connection between the reader and the written report (Appendix A). They can generally be classified as below: Comment by Joe Sample: Not “effectiveness of the users” but effectiveness FOR the users. Comment by Joe Sample: “Comfortable with the content” makes it sound as though the content is controversial or risqué. Readability is about how well a person will understand the content. Comment by Joe Sample: ? Comment by Joe Sample: The reference to your evaluation plan is not clear.

· Ascertaining information directly from users (user feedback).

· Utilizing user assumptions, skills, needs, and concepts required for understanding effectiveness of the document.

The five critical principles of usability are learnability, memorability, efficiency, error recovery, and satisfaction which means each of the critical principle reflects on one aspect of the document usability experience. Learnability allows for the learning phenomenon of the document and its ease in communicating with user of the document. Productivity of the document is discussed through efficiency, while ease of remembering lies in the memorability aspect. Usability also highlights on the error recovery whereby people assess the document and shows a quick response in users correcting them. Lastly, satisfaction level of user signifies the user experience after spending time and energy while using the document. Additionally, usability testing involves cognition, perception, affect, and performance and behavior as purposes that is reflective of the user experience during data collection. Perception and cognition are interconnected, but cognition connects more with intellectual capacity of the user while perception is the personal view-point on formatting of document content. Comment by Joe Sample: The information here needs to be cited and is not altogether relevant to the task at hand.

Appendix B suggests that needed testing material for text-based using readability tests include guidelines or checklists for verbal information. It also suggest explains the testing of using a survey of users in user-based testing with testing material of handouts containing list of questions for discussions. LThe list of questions for discussion includes a combination of open-ended and close-ended (Likert scale) questions for ascertaining desired information. Comment by Joe Sample: This does not make sense. Nothing is suggested. Comment by Joe Sample: The list of questions is not included.

Participants

Testing participants for usefulness of the document highlights a suitable selection of sample size, snowball sampling in this case due to limited social mobility avenues available. Participants from diverse backgrounds are approached with peered endorsement, that is, new participants are identified on the way. Test participants were contacted through e-mail address and Skype for sparing out some time, and providing lead on the suitable person around for conducting the usability test. A total of 30 participants responded in the study, with 15 participants having graduate level education, while remaining half pertains to journalists, civil society activist, government officials, and researchers. Comment by Joe Sample: This content is coming from somewhere. It is not altogether relevant. I am on page three of the methods and I am still not sure what you did.

Text-based testing. Demographic variables of the researcher states the characteristics as of male gender, 22 years age, no professional experience, and education level of college graduategraduation.

User-based testing. Appendix A highlights on the characteristics of participants for ‘Best Countries 2020’ report and include mostly high-school graduates, and a relatively low proportion of undergraduates. The reason for overwhelming high-school graduates sample is assessment of the document in terms of basic accessibility, cognition, and perception. Comment by Joe Sample: You can be specific here so the reader does not need to search for the information. Comment by Joe Sample: Does this statement justify the decision.

Sessions

During COVID-10, social distancing protocols allowed utilization use of digital tools for data collection. An e-mail was sent to participants over the course of two weeks, and resulting in hundred percent response rate due to snow-ball sampling technique which means selection of suitable candidate, although taking into account the diversity aspects that can provide feedback reflective of the testing procedures. Comment by Joe Sample: This is more of a pet peeve. “Utilization” is just an uppity version of the word “use.” Comment by Joe Sample: You should include the email in the appendices. Comment by Joe Sample: I am not sure what you mean here.

Evaluation Tasks/Scenarios

A total of thirty 30 participants awere asked for providing information into following two tests.

1. Flesch Reading Ease Test for gaining insight into the user’s perception of the selected text from document, and gained a score of 0-99 from each participant. Comment by Joe Sample: These measures can be attained without users.

2. Flesch-Kicaid Grade Level testing gained information on the document readiness, and a collective response of eighth-grade is received.

3. Concurrent testing through providing a brief that illustrates a paragraph to which participants were expected to respond after utilizing the document. The question asked about the countries with ‘ease to businesses for investors looking to invest overseas.

Key Features and Elements of Testing. The key specific features and elements for text-based testing are diction, sentence length, appropriate use of images, and paragraph length that provides in-depth analysis of the English writing in document. Additionally, key features and elements for user-based testing are navigation, organization, and completeness that provides overview of the user user-experience during the process of assessing the document during testing procedures.

Key Questions and Testing Scenarios for Text- based Testing

The two key questions under investigation through user based and text based usability tools are listed as below:

· Is the design of report effective in placing text and images that increases usability of the reader? Comment by Joe Sample: How were you planning to determine this? The images can be well placed, but that seems to be different from increasing the usability of the document.

· What are the average sentence and paragraph lengths in the report?

Testing Scenario:

· It entails the context in which participant is responding to the text, and the actual information expected to retrieve. For example, an international student studying in United States hopes to find an employment opportunity in a country that is most suited for his or her needs. Comment by Joe Sample: You should describe the scenario here.

Key Questions and Testing Scenarios for User- based Testing

Below are the two critical questions pertaining to testing scenarios

1. Was it easy to understand in terms of technical jargons and English writing style for non-native English speakers? Comment by Joe Sample: What aspect of the document does “it” refer to?

2. How was the formatting of the report? And did it manage to communicate critical information while reading the document of twelve 12 pages. Comment by Joe Sample: This requires a descriptive response. Comment by Joe Sample: This seems to me to be a matter of content and (maybe) unrelated to user-based testing

Tasks Completion Success Rate

The hundred percent response rate is recorded from the thirty 30 participant due to snow-ball sampling that allowed going after participants whose response is reflective of the document usability through text-based and user-based models. Comment by Joe Sample: Do you mean people whose “interests” would be reflective in the content of the document?

Task Ratings

The task ratings for Flesch Reading Ease Test is with a scale between 0-99, while Flesch-Kicaid Grade Level provides readability results at eight-grade because the tabulation section, followed by methods and frequently asked questions are formatted in an easy to understand format. Concurrent testing also explains the easy to understand concept of the report for anyone from eighth grade to post graduate when participants response stating

Time on Task

There were 24 hours available with each participant to respond to the questions, which includes mentioning the time for the user-based testing. Overall, the response rate suggest that user-based testing responded with

Post-Task Overall Questionnaire

After receiving the questionnaires back, and additional e-mail asked comfortability of the index and an open-ended question mentioning the ease at which they managed to provide their responses. Comment by Joe Sample: Again, include the email in the appendices.

Where are the results/analysis?

The documents is aimed at providing media personnel and organization, non-governmental organizations and civil rights activists, and people of business community with sufficient information on the top seventy-three countries with their associated ranking. The readership of document is diverse, for example, and investor might want to look at the Best Countries 2020 for assessment of the investment opportunities abroad. Although, half of the sample is dominated by high-school graduates, the remaining half is reflective of civil society members, businessmen, journalists, and researchers. The document is an easy to access source for users with intend of gaining insight into the countries with aggregate ranking on nine variables. For each variable, data is obtained from other indexes to form a comprehensive ranking in the ‘Best Countries 20202’ report. The document is not difficult to grasp and gives insights that can be read and understand by an eighth grade student, and a high school graduate can utilize the information for effective usage. Data is published in simple possible way; however, improvement with respect to layout can be further enhanced through the use of pictures and images reflective of each of the nine variables in the index. Comment by Joe Sample: Given that the document was only 12 pages you could have included it in the appendices as well.

The findings of the research suggest that the selected document of ‘Best Countries 2020’ is an effective document in terms of usability with respect to text-based and usability-based. However, every written document has margin for improvement and working on more appealing document formatting that includes extensive usage of images and pictures will add value to the document. Plain Language concepts are applied to the document while assessment, and data suggest that users are friendly in terms of learning and using the content, while accommodates diverse readership for their respective usage of information.

Testing Goals

Participants

Methods

Key Features and Elements for Testing

Key Questions and Testing Scenarios

Testing Material Needed

Testing Materials Needed