Reporting the Results of a Website Evaluation
jlel
Assignment1Abdullah.docx
Assignment 1: WEBSITE EVALUATION
IDE 611: TECHNOLOGIES FOR INSTRUCTIONAL SETTING
Site Title: The Khan Academy
Site URL: https://www.khanacademy.org/
Founder & CEO: Sal Khan
Evaluation contact person: Abdullah Albelehy, graduate student
IDD&E, School of Education
Syracuse University
WEBSITE EVALUATION
WEBSITE DESCRIPTION
Overview
Khan Academy an educational organization which is non-profit. It was created in the year 2005 by Salman Khan. The goal of creating Khan Academy was to set an online tool which would be able to help in educating many students around the world. This educational organization normally produces class lessons which are informed of YouTube videos for studies. The website includes supplementary lessons for practices and other materials of the educators. All the resources on the website are available for all users of the website.
Khan Academy is a free website which has videos that are educational with an aim of self-paced instructions. The website has a library app which acts a mirror to the content which is found on the website. The site is organized in such a way that it is easy to access information especially when there is a need for a procedural kind of help needed. My intention was to evaluate whether or not the needs of the user, and most importantly the needs of a user who may be new the online world, appear to have been met or if they were kept in the minds of the people who designed the website layout and its functionality of the course directory.
Since the website was developed, it has become a tool which is significant for many potential users and it has stood the core reason for development which includes the needless abilities in the user’ mind. This is due to the lost complexity of the website, the ease of how it can be used which is down to just mere functionality and its clarity of the display is lost in the middle of the programming features, for example, flashing, frames/ and very clever graphics.
Purpose of the site
The main objective of the website is to provide a personalized learning process which is mainly built on videos hosted on YouTube. The website is a supplement to the videos which it provides. This is because it has other features, for example, progress tracking, practice exercises, and other teaching tools. The materials which are provided in the website, are also purposed to be produced on a mobile application.
Intended audience
The website is intended to be used for the purposes of educational activities and learning to educators, students, and the community around the world. The website is to provide a free, world-class education to any user located at any part of the world. They have personalized learning resources for users of all ages and tools which are enhanced for the use of teacher and parents.
Website content
The website to have a good layout on its Homepage and it is well-structured and one can be able to navigate through the site with ease. The Home page has a logo which is at the center of the page. Through browsing the website it is clear that its domain look and format is very consistent. A user can be able to find any pieces of information relevant, for example, login, signup, and donate toolbars which are available for selection. The courses toolbar is also available which includes a variety of options which a user can select. Through a minimum number of clicks, a user can also be able to take advantage of the detailed but simple orientation of the site.
Through navigating through the site with selected adults, I realized that the user can be able to rate each video and also leave a comment and also ask an answer content which is related to the question. Through the selected users, signing up feature of the website is quite easy since it can be done through Facebook or Google accounts. It was also possible to sign up through email address accounts. After users are logged in, they can be able to monitor progress on a personal dashboard. The videos which are the most vital thing in the website, are short and presented in a very simple manner. The website also is linked to an application which is available for users who have done an installation of it. The videos display recording through an electronic blackboard which has similarities with the style of a teacher giving a lecture. The Khan Academy website is also able to host content from the educational channels and organizations.
You provided a clear overview of the website.
Techniques employed
The website uses text content, videos, flat icon, and different colors. The interactive technique which is available on the website is the case where a learner or an educator can leave a comment after watching a video. The news section is consistent in its updates.
METHODS
Two tools were provided for evaluation of the website. The evaluation was aimed at identifying the weakness and strength of the website and the process which is used in facilitating its services to the users. The objective of the website evaluation is to evaluate the user perception and at the same time check on the enhanced online presence of the leaners in similar situations as physically being in a classroom. This report was conducted by the author of this report with the assistance of other selected two adults.
Through the methods of evaluation, it was found out the site provides a platform where learners can be able to access the services not only from the browser but also a mobile application. The website provides a database which has valuable information and links to various media and the media related content which serves and assist students to learn with online media directories (Thompson, 2011). Another objective of the evaluation was to assess how easy it is to navigate through, access and its general performance. It was also aimed at evaluating the performance and be able to establish comprehensibility of the information which is contained in the website and also test the impression of the site, appeal, and its design. This reports also has a keen interest on the objective which is meant to test the strengths and the weaknesses of the site and make recommendations on the possible improvements which can be done on the site.
Instruments used Comment by Jing Lei: The introduction of the instruments was clear.
Content validity scale
The instrument of the content validity scale uses how accurate a conclusion in the evaluation has been made and measured and much it corresponds to being tested. It defines the extent to which an assessment accurately measures what is intended to be measured. It is generally a statistical tool. A minimum of ten question was used to evaluate the credibility of the website with a scale of 1 to 5. 1 represents strongly disagree and 5 strongly agree.
The professional version of WebMac
The design of the WebMac Professional is made for the assessment of the website which is meant for the educational purposes. For example, it evaluates the use of the website in the classrooms instruction and for the use of the student in specific projects. This instrument encourages the exploration through the website in trying to gain some knowledge about the content and the website is structured. It rates the website on its benefits with either a disagreement or an agreement which are either strong, week or not applicable options.
This instrument has 32 sets of questions which have been grouped together in a very unique way for the measurement of four sections. These sections include Stimulating, Meaning, Organization, and how Easy to use the website. They are scaled from 1 to 3 which represents either strongly agree or strongly disagrees. Some of the questions are considered as being not relevant hence they will fall under the category of “Not Applicable” which is given a zero mark. This tool in its representation form is visual.
Before going into results, please introduce who the evaluators were, how the evaluation results were obtained, i.e., did you take the average score from different ealuators?
RESULTS
Content validity scale
The score of the content validity was seen relatively high with a percentage of more than 85%. This a good status for such a website. This evaluation also indicated that most of the evaluators strongly had an agreement that the website source of information is credible and can be relied upon. Comment by Jing Lei: How did you come up with this percentage? Each item was evaluated from 1-5, so what was the number for each item? A table would be helpful to show the results. Please summarize the evaluation results and include the final results in the report.
WebMac Professional
Through the WebMac Professional, the evaluation indicated that the website was stimulating with an average score and meaningful with a high score. The website is also organized and very easy to use according to the evaluators of the website which marked a high score. Comment by Jing Lei: What were the exact scores? Please report the specific scores and use the figure to indicate how well it scored.
Value and the expectation of the evaluation process
The Khan Academy website has an average high score which is indicated by the two evaluation instruments. This places the site in a category of an excellent website for all its appropriate users. The website has been able to meet the expectations of this evaluation. It is what can be described as a successful website. Comment by Jing Lei: Again, please report the specific scores in the report. Comment by Jing Lei: The results in the attachment showed that it had low score for expectation for success. Can you explain why?
Interpretation and Synthesis of Overall Findings from the two instruments
The selected methodologies which were used in this evaluation include the Hall test and the telephone interviews which were very essential in the meeting of the evaluation objectives. The hall test involved two members of the public who were selected. They were invited to review the Khan Academy website using the computer which was provided with a specific internet link. They also required to complete a questionnaire (Elling et al., 2012).
Recommendations and Improvements
This comprehensive report will be presents the findings and the recommendations which are arising from the evaluation of the website. The report was conducted and evaluated with the help of two adults who are members of the public. The reviewers approximately spent two hours using the website. The evaluation was conducted under a set of accepted standard measures although, they can be improved. The first impression of the website which is gained can be considered to be very positive. It is enhanced through the extensive use of the color blue; which is seen on some parts of the site, which can often be described one that conveys the importance of the site which is the main objective is to be educative. On the other hand, the branding of the website is great with the Khan Academy logo at the center and being recalled on the page of the majority of the users. Through my evaluation, I think that this could be improved through development more personalized dashboard in each candidate page.
The display clarity of the website, I found it to be working well, although the title of the picture bar at the top of the monitor have problems as they seem to be distracting and resulting into overshadowing what I considered to be more important features. To add to the evaluation, the menu on the screen serves its intentions function of assisting the user to try and understand where they are not and what options can be available although I found that it has the capabilities of being improved through a form which enlarges the front. I would suggest that the menu is replaced with a static menu at the top of the screen or on the other hand, it can be a changing menu which is replaced by clear graphics which would be seen as being attractive to users.
REFLECTION Comment by Jing Lei: Please also talk about what you learned from this experience.
The theme of the Khan Academy website services is providing a world-class education for educators and students around the world. According to https://www.khanacademy.org/ it has videos which have been translated into several languages with more than 20,000 subtitle translation available. The website is available in English, Spanish, Portuguese, Turkish, French, and Bangla. It also partially contain other languages. The accessibility of the site can frequently be accessed through the use of Google with a user typing the keywords; Khan Academy. This will be able to provide with Khan Academy link among the links of the matched returns after a search. The two users who used in this evaluation, described the site as an easy one to navigate through although when they were given a task which they were required to find some very specific information from the site, the user would usually attempt severally before they were successful in completing their tasks.
References
Elling, S., Lentz, L., de Jong, M., & Van den Bergh, H. (2012). Measuring the quality of governmental websites in a controlled versus an online setting with the ‘Website Evaluation Questionnaire’. Government information quarterly, 29(3), 383-393.
Thompson, C. (2011). How Khan Academy is changing the rules of education. Wired Magazine, 126, 1-5.
Content Validity Scale© (Arnone & Small, 1999)
Instructions: Rate your agreement with the following statements.
Strongly Disagree 1 2 3 4 5 Strongly Agree
1. The source of information for this Web site is credible.
Score evaluation one is 5
Score evaluation two is 4
2. There is a way to contact the author of this Website, if necessary.
Score evaluation is 2
Score evaluation is 3
3. The factual information or content of the Web site seems accurate.
Score evaluation one is 4
Score evaluation two is 4
4. If the Web site presents concepts or principles in its domain (e.g. science,
Art), they are appropriately presented without confusing or missing information.
Score evaluation one is 2
Score evaluation two is 2
5. There are no typographical or spelling errors that could potentially cause
The information at this Web site to be misunderstood.
Score evaluation one is 2
Score evaluation two is 3
6. The content is appropriate for the intended audience.
Score evaluation one is 4
Score evaluation two is 5
7. The links from this site appear to be credible.
Score evaluation on is 5
Score evaluation two is 5
8. This Web site appears to be free of bias.
Score evaluation one is 3
Score evaluation two is 5
9. The information at this Web site is current enough for the type of
Information it includes.
Score evaluation one is 3
Score evaluation two is 4
10. The links from this Web site appear current and unbiased.
Score evaluation one is 5
Score evaluation two is 4
Website Motivational Analysis Checklist (WebMAC) Professional (v. 2.0)
DIRECTIONS:
WebMAC Professional was designed for professional educators to assess Web sites for use in classroom instruction or for student use in homework assignments or projects. Before using WebMAC Professional, it's a good idea to spend some time exploring the Web site to be evaluated in order to have some familiarity with its content and structure. You may need to go through the Web site at least once more to complete this checklist.
Rate your level of agreement with each of the 32 items on the next two pages by placing the appropriate numerical value on the line in front of each item. If you are not sure about any item, select the best response you can give.
3 = strongly agree
2 = somewhat agree
1 = somewhat disagree
0 = strongly disagree
NA = Not Applicable
Example of completed item:
3 0. This Web site makes me happy.
After you have rated all 32 items, go back and look at each NA item. You should place one of the following scores next to the NA for that item.
0 = the Web site would have benefited if it had included the item (e.g. audio).
1 = the Web site didn't require that item.
2 = the Web site was better off for not including that item.
Copyright 1999 (revised 2004) by Ruth V. Small & Marilyn P. Arnone, Syracuse, New York. All rights reserved. No part of the material protected by this copyright notice may be reproduced or utilized in any form or by any means, electronic or mechanical, including photocopying, recording, or by any information storage and retrieval system (except for non-profit educational use), without the written permission of the copyright owner.
WebMAC Professional (v. 2.0)
Your Name _____________________________________________________________________________
Grade level/subject ______________________________________________________________________
Web Site URL ____https://www.khanacademy.org/______________________________________________________________________
Directions: Place the appropriate number about this Web site on the line preceding each item.
3 = strongly agree
2 = somewhat agree
1 = somewhat disagree
0 = strongly disagree
NA = Not Applicable
_3__ 1. The screen layout of this Web site is attractive.
_2__ 2. There is a menu or site map at the beginning that describes what content is contained within the Web site.
_3 3. Visual (e.g. videos, photographs) or audio information included in this Web site helps to clarify or describe the topic.
_1__ 4. Navigating this Web site does not require any special skills or experience by me or my students.
_2__ 5. There is an eye-catching title and/or visual on the home page of this Web site that will attract students' attention.
_2__ 6. This Web site provides valuable links to other useful Web sites.
_3__ 7. The purpose of this Web site is always clear to me.
_2__ 8. The Web site has a help function that I or my students can use at any time.
_1__ 9. This Web site provides opportunities for interactivity.
_3__ 10. The Web site's information is provided by credible sources.
_2__ 11. The directions for using this Web site are simple and clear.
_2__ 12. This Web site makes it easy to search or query for information.
__1_ 13. The information on my topic included in this Web site is interesting.
__2_ 14. The information contained in this Web site is current and up-to-date.
_2__ 15. There is useful information on my topic at this Web site.
_1__ 16. This Web site's graphics are crisp and clearly visible.
__3_ 17. The variety of formats (e.g. text, images, sound) helps to maintain attention.
1__ 18. The information at this Website is accurate and unbiased.
__3_ 19. All of the information at this Web site is presented using clear and consistent language and style.
__1_ 20. All of this Web site's hyperlinks are active and fully functioning.
WebMAC Professional (v. 2.0) page 2
Name Khan Academy_________________________ URL _https://www.khanacademy.org/_______________________
3 = strongly agree
2 = somewhat agree
1 = somewhat disagree
0 = strongly disagree
NA = Not Applicable
_3__ 21. This Web site has novel or unique features that make it more interesting for students.
_0__ 22. There is little or no unimportant or redundant information at this Web site.
_1__ 23. The text at this Web site is well-written without grammatical, spelling, or other errors.
_3__ 24. At all times, I can control what information at this Web site I wish to see or show my students.
_1__ 25. This Web site stimulates curiosity and exploration.
_3__ 26. This Web site provides opportunities for users to actively participate or contribute ideas.
__1_ 27. This Web site provides the appropriate amount of information on the topic for a lesson or assignment.
__3_ 28. All buttons and other navigation mechanisms for moving around at this Web site work the way they should.
_2__ 29. The colors and/or background patterns used in this Website are pleasing.
1___ 30. This Web site provides opportunities to communicate with its authors.
__2_ 31. No matter where I am in this Web site, I can return to the home page.
_1__ 32. The amount of time it takes for pictures, games, videos, etc. to appear on the screen at this Web site is reasonable for teaching/learning situations.
There is enough at this Web site to warrant a return visit.
I would recommend this Web site to my colleagues.
What is the greatest strength of this Website?
The greatest strength of the website is its capability to effectively deliver on the needs of its user.
What needs improvement at this Website?
The content of the website is very large and improvement on the specialization can be developed for various users.
1
Scoring WebMAC Professional (v. 2.0)
Before transferring scores, go back to each item designated as "NA" and choose a score for that item based on the following criteria:
· 0 points if you feel that the site would have benefited if it had included the item (e.g. audio).
· 1 point if you feel that the Web site didn't require that item.
· 2 points if you believe the site was better off for not including that item.
When all 32 items have a number score, transfer each score into the appropriately numbered space in one of the four columns below. Then add the scores for each column and write the total on the appropriate line.
S M O E
1. _1__ 2. _1__ 3. _2__ 4. _2__
5. _2__ 6. _2__ 7. __2_ 8. _1__
9. _1__ 10. _2__ 11. _1__ 12. _2__
13. _2__ 14. _1__ 15. _2__ 16. _3__
17. 1___ 18. _2__ 19. _1__ 20. _2__
21. __1_ 22. 2___ 23. __2_ 24. _2__
25. _0__ 26. _1__ 27. _2__ 28. _3__
29. _1__ 30. _2__ 31. __1_ 32. __1_
TOTAL S _14__ TOTAL M _13__ TOTAL O _12__ TOTAL E __15_
The S column reflects how STIMULATING this Web site is for you.
The M column reflects how MEANINGFUL this Web site is to you.
The O column reflects how ORGANIZED you think this Web site is.
The E column reflects how EASY-TO-USE this Web site is for you.
Transforming Scores
Now you may transform your scores into visual representations so that you can
Clearly see the results of your evaluation.
1. Take each of the four total scores above and plot them (either with a dot or a bar) on the graph below.
2. If you use dots, connect the dots to make a line.
HIGHLY MOTIVATING
S
M
O
E
30
24
2
16
8
0
NEEDS MUCH IMPROVEMENT
NEEDS SOME IMPROVEMENT
Now take the total scores from the four categories (S, M, O, and E) and add them together in the following way to get your two overall motivational quality scores.
S + M = ______ (V) O + E = ______ (XS)
The V score reflects a summary motivation score on the Value dimension; i.e. how stimulating and meaningful the Web site is. The XS score reflects a summary motivation score on the Expectation for Success dimension; i.e. how organized and easy-to-use the Web site is.
Plotting the Scores
To plot scores on the scoring grid,
1. Place a dot for the V score along the Value continuum and a dot for the XS score along the Expectation for Success continuum on the grid.
2. Draw straight lines to their intersection point, representing the overall motivational quality score of the Website
In the example below the evaluated Web site received a Value score of 40 and an Expectation for Success score of 18. Their intersection point falls within the lower right quadrant of the grid. Now go to the next page to record your scores.
Example Completed Scoring Grid
High Expectation for Success
Low Expectation for Success
Low Value
High Value
X
40V;18XS
To record your actual scores onto the scoring grid.
1. Plot the score for V along the Value continuum and the score for XS along the Expectation for Success continuum.
2. Draw straight lines to the intersection point.
| | | | | | | | | | | | | | | | | | | | | | |
0
48
0
48
32
16
16
32
Average to high for value/Average to high for expectation for success
24
40
40
8
8
Average to high for value/Below average to low for expectation for success
Below average to low for value/Below average to low for expectation for success
Awesome
Website!
High Expectation for Success
Low Expectation for Success
Low Value
High Value
Average to high for expectation for success/Below average to low for value
| | | | | | | | | | | | | | | | | | | | | | |
0
48
0
48
32
16
16
32
24
40
40
8
8
Average to high for value/Below average to
low for expectation for success
Below average to low for value/Below
average to low for expectation for success
High Expectation for Success
Low Expectation for Success
Low ValueHigh Value
| | | | | | | | | | | | | | | | | | | | | | |
0
48
0
48
32
16
16
32
24
40
40
8
8
Average to high for value/Below average to
low for expectation for success
Below average to low for value/Below
average to low for expectation for success
High Expectation for Success
Low Expectation for Success
Low ValueHigh Value
