User interface prototype wireframe part 2
Hunnysid
381Week9Deliverable3.1briefing.pptxUser Study
Deliverable 3 Part 1 Briefing
Creating your app prototype
Deliverable 1
Building the skeleton of the FP
Deliverable 2
Building the interaction flows –
‘click streams;’
Finalising the build & bug testing
on devices
Deliverable 3 Part 1
User testing & tweaking FP
Deliverable 3 Part 2
More user testing & adding visuals: graphics & colour.
Human Computer Interaction Design
Image source: http://mobiledesign.org/wireframes
2
User study - mission
Human Computer Interaction Design
find out what works and what doesn’t for users of your prototype –
develop your live user testing skills.
find out what works and what doesn’t for users of your prototype – develop your live user testing skills.
3
User study - Brief
Design a user study that will explore usability aspects of your prototype
Preferably mobile first
Human Computer Interaction Design
brief
Design a user study that will explore usability aspects of your mobile prototype:
create a user study plan and associated materials
perform a pilot study with at least one person. Modify your plan and prototype as needed as a result of this study. (For internal students, this should be done in class)
user test your functional prototype with different people ( each team member is expected to perform a usability test)
analyse the data (user feedback) in terms of your study goals and then generate insights, to inform revisions to your final prototype.
4
User study - Brief
TEAM TASKS
1 . PLAN & RESOURCES Create a user study plan & associated resources
2. PILOT STUDY Team performs a pilot study with at least one person
3. MODIFY PLAN
Modify plan and prototype as needed as a result of this study
Human Computer Interaction Design
INDIVIDUALLY
1. CONDUCT USER TEST user test your functional prototype with participant (ideally they should not have seen the product before)
2. ANALYSE DATA TO SHARE
analyse the data (user feedback) to inform revisions to your final prototype.
brief
Design a user study that will explore usability aspects of your mobile prototype:
create a user study plan and associated materials
perform a pilot study with at least one person. Modify your plan and prototype as needed as a result of this study. (For internal students, this should be done in class)
user test your functional prototype with different people ( each team member is expected to perform a usability test)
analyse the data (user feedback) in terms of your study goals and then generate insights, to inform revisions to your final prototype.
5
Team + Individual
Human Computer Interaction Design
Plan & resources
Interface alternative
Pilot user test
Adjust based on feedback
Individual user study
Image Source: http://www.uxbooth.com/articles/complete-beginners-guide-to-design-research/
http://www.90percentofeverything.com/wp-content/uploads/2010/10/belen1.png
6
Two parts
Part A
User Test plan
Paper prototype (functional for extra credit)
Pilot study
Update from Report 2
Part B
Individual user studies
Results & changes (Implementation Plan)
Submit Report 3
Human Computer Interaction Design
7
Part A – User Test Plan
What do you want to find out about your prototype?
Develop a user test plan improved from last submission:
What do you want to measure with your testing?
What tasks will the user perform?
How will the user share their feedback
Human Computer Interaction Design
http://news.qooxdoo.org/qooxdoo-mobile-prototype
First, develop a user test plan. Think about what you want to learn about your prototype, and then design your test plan based on that.
Things that you will want to think about:
What do you want to measure with your testing? (Note – how good is my interface is not a specific enough question!)
What tasks will you ask the user to perform so that you will be able to measure participants responses and help answer your questions?
Decide how you want the user to share their feedback. Will they:
think aloud, and/or
fill out a questionnaire, and/or
complete a post-test interviews, and/or
all of the above.
Prepare an outline of:
What is the question you are asking? How can you measure this
How the user test will be run,
Written instructions that you will read to the users, and
Any other materials (e.g. questionnaires, interview questions) that will be used during the session.
Ethical issues – informed consent You will need consent forms [mandatory]for your participants: you can use a template (see workshop materials) to create one if you find it helpful.
Plan template
Use the template provided to create your user plan.
8
Evaluation framework
Determine the goals.
Explore the questions.
Choose the evaluation approach and methods.
Identify the practical issues.
Decide how to deal with the ethical issues.
Evaluate, analyze, interpret and present the data.
Human Computer Interaction Design
DEC
IDE
9
Reference Ch 13 Preece Rogers & Sharp, 2007
The evaluation framework may be useful to think things through
User study - Planning
prepare ....
An outline of how the user study will be run
written introductions that you will all read to each participant, (script)
any other materials that will be used during the test
methods for collecting & analysing your data (video)
consent forms [mandatory]
Human Computer Interaction Design
Prepare :
What is the question you are asking? How can you measure this
Prepare an outline of how the user test will be run,
written instructions that you will read to the users, and
any other materials (e.g. questionnaires, interview questions) that will be used during the session. You will need consent forms for your participants, you can use a template (see workshop materials) to create one if you find it helpful.
Ethical issues – informed consent You will need consent forms [mandatory]for your participants: you can use a template (see workshop materials) to create one if you find it helpful. These MUST be signed.
10
Human Computer Interaction Design
Step 2–Alternative Design
Select one component of your prototype and develop an alternative design
Mock up screens (paper prototype)
Implement these (optional – extra credit)
Keep URL separate
Human Computer Interaction Design
Select ONE component of your prototype and come up with a redesign for that part.
Mock up a couple of screens for your redesign. This is like a mini paper prototype that you can use to test with a user.
EXTRA CREDIT (20points): functionally implement your redesign (code up the screens you mockup). Everyone in the group who participates in this process will each receive 20 points. Remember to keep the URL to this redesign separate from your original implementation (you will need both for testing).
This is NOT meant to be a huge redesign of your prototype. We want you to select something small and manageable in scope. Designers often go through many, many different versions of prototypes before their final design, so we want you to get familiar with the process (given the time constraints for the course).
12
Step 3– Pilot Test
Pilot Study (TEAM)
Pilot - user test original implementation
Video one aspect
At the end – introduce the redesigned component
(paper or functional)
Looking for comparative feedback (IE which works best/etc)
Team roles – facilitator; recording notes, technology problems
FIND WAYS TO IMPROVE THE INTERFACE
Human Computer Interaction Design
Make changes to interface
conducting the user study
Run user tests on your interactive prototype with at least three different people.
You will first test your original implementation according to the plan you drafted up. This is the pilot.
Ensure you video at least one aspect of this test.
At the end of your user testing session, you will introduce your redesigned component and test it out (test the functional screens if you've done them, test with the paper prototype if you haven't).
You are mainly looking for comparative feedback from the user.
Just like when you conducted the Heuristic Evaluations, each team member will have its own role during testing: - One person will be the facilitator;
one will be in charge of taking notes/photos/video/etc.
any team member who is not currently in one of those two roles should observe and take notes.
This time your user will not be writing down the problems they find for you. It's your job to learn what your users are thinking, the feedback they provide you will be invaluable for your next iteration.
Your high-level goal is to find ways to improve your interface. Look for breakdowns and pain points in your interface and try to understand what the problems are and how you might fix them.
If possible, try modifying or updating your prototype before continuing to the next user.
13
Step 3–Pilot Study
Perform your test with a class member from another group
Work as a team
Video the intro and first task
Post in YouTube
Show that all members are present
If needed use your clickstream diagrams
Human Computer Interaction Design
Perform your user test with a live volunteer from another class group
USER TEST: Working as a team perform your first user test on your prototype according to the plan you drafted together. This is the pilot study. Ensure you video at least the intro and first task of this test. Post your video of your pilot test on the unit YouTube site.
The video needs to show all team members doing a role in the process and the live user.
(NOTE- Faking the whole group attendance in this pilot session may result in the team losing all marks for this section of the deliverable.
If someone legitimately can’t be there – acknowledge that in your write up)
Redesigned Element: At the end of your user testing session, introduce your redesigned component and test it out (test the functional screens if you've done them, test with the paper prototype if you haven't). You are mainly looking for comparative feedback from the user.
Analyse & review: Adjust your user test plan after the pilot test, changing any aspect that did not work well. The tasks may be the area which needs closest attention. Any adjusted user test plan needs to be submitted with the group write up. SUBMIT TO THE DISCUSSION BOARD
14
You will submit final User Plan & Prototype
Submit as a team:
User Test Plan
With justifications
Questionnaires/surveys
Resources developed as part of the testing ( scripts etc)
Paper Prototype
Photos of paper prototype OR URL of wireframe
Pilot Study
Brief outline of results and changes that you made as a result of this test: testing process and or prototype
Human Computer Interaction Design
The URL of your project. Note: the URL must work at least until your assignment is graded. If it doesn't work, you'll receive no credit.
In 100 words or less, describe your user and their goals.
A description of the steps required for us to use your interface to accomplish this goal. The level of detail should be sufficient for me to follow the instructions. (They need not work yet; they will for the full functional prototype.)
An implementation plan. What are the "chunks" of development that need to happen? Who is responsible for which chunks? When will they be completed? What research(if any) is required?
Ensure you have rank-ordered your list of functionalities you are implementing indicating c
The URL of your project. Note: the URL must work at least until your assignment is graded. If it doesn't work, you'll receive no credit.
In 100 words or less, describe your user and their goals.
A description of the steps required for us to use your interface to accomplish this goal. The level of detail should be sufficient for me to follow the instructions. (They need not work yet; they will for the full functional prototype.)
An implementation plan. What are the "chunks" of development that need to happen? Who is responsible for which chunks? When will they be completed? What research(if any) is required?
Ensure you have rank-ordered your list of functionalities you are implementing indicating current progress and future plans
urrent progress and future plans
User Test Plan
Group Details
The URL of your project. Note: the URL must work at least until your assignment is graded. If it doesn't work, you'll receive no credit.
In 100 words or less, describe your user and their goals.
A description of the steps required for us to use your interface to accomplish this goal. The level of detail should be sufficient for me to follow the instructions. (They need not work yet; they will for the full functional prototype.)
An implementation plan. What are the "chunks" of development that need to happen? Who is responsible for which chunks? When will they be completed? What research(if any) is required?
Ensure you have rank-ordered your list of functionalities you are implementing indicating c
The URL of your project. Note: the URL must work at least until your assignment is graded. If it doesn't work, you'll receive no credit.
In 100 words or less, describe your user and their goals.
A description of the steps required for us to use your interface to accomplish this goal. The level of detail should be sufficient for me to follow the instructions. (They need not work yet; they will for the full functional prototype.)
An implementation plan. What are the "chunks" of development that need to happen? Who is responsible for which chunks? When will they be completed? What research(if any) is required?
Ensure you have rank-ordered your list of functionalities you are implementing indicating current progress and future plans
urrent progress and future plans
The URL of your project. Note: the URL must work at least until your assignment is graded. If it doesn't work, you'll receive no credit.
In 100 words or less, describe your user and their goals.
A description of the steps required for us to use your interface to accomplish this goal. The level of detail should be sufficient for me to follow the instructions. (They need not work yet; they will for the full functional prototype.)
An implementation plan. What are the "chunks" of development that need to happen? Who is responsible for which chunks? When will they be completed? What research(if any) is required?
Ensure you have rank-ordered your list of functionalities you are implementing indicating c
The URL of your project. Note: the URL must work at least until your assignment is graded. If it doesn't work, you'll receive no credit.
In 100 words or less, describe your user and their goals.
A description of the steps required for us to use your interface to accomplish this goal. The level of detail should be sufficient for me to follow the instructions. (They need not work yet; they will for the full functional prototype.)
An implementation plan. What are the "chunks" of development that need to happen? Who is responsible for which chunks? When will they be completed? What research(if any) is required?
Ensure you have rank-ordered your list of functionalities you are implementing indicating current progress and future plans
urrent progress and future plans
submit
There needs to be a team submission with evidence of each person’s contribution:
The following should be included:
PART A (pilot plan & test Thursday Week 9 @midnight)
Plan: A description of your evaluation plans (length as needed), including justification for your choices based on what you hope to learn. (Indicate who wrote up the plan and use the template provided).
Support materials: Any questionnaires or surveys you wrote as part of your plan (either as text or a scanned image – again indicate who developed what).
Redesign: Your paper prototype for your redesign. You need to include what you currently have and how and why you have changed the design. If you've functionally implemented it as well, also include the URL to those screens. (Indicate who created the alternative design and if this is part of the team or individual work – where there is no indication, it will be assumed that all team members were part of the process.
Pilot test: An outline of the pilot test and the results including any changes that occurred in the process as a result of the test (This must include the Video URL of your pilot test 6 – 10 mins. Videos need to be uploaded to the HIT 381 YouTube site)
15
Conduct your User Tests
Individual User Studies
keep things consistent
between your user tests:
same script
same protocol
answer questions in the same way
use the same questionnaires/
interview questions
Immediately after each study, record your reactions – paper/digital
Human Computer Interaction Design
A. ROLES & RESPONSIBILITIES All team members should have a role during testing.
One person will be the facilitator;
One will be in charge of taking notes/photos/video/etc.
Any team member who is not currently in one of those two roles should observe and take notes.
This time your user will not be writing down the problems they find for you. It's your job to learn what your users are thinking, the feedback they provide you will be invaluable for your next iteration.
Your high-level goal is to find ways to improve your interface. Look for breakdowns and pain points in your interface and try to understand what the problems are and how you might fix them. If possible, try modifying or updating your prototype before continuing to the next user.
B. INDIVIDUAL TESTING
Once you have conducted your pilot test as a team, then each of you need to perform your own user study.
You may do this individually or with other team members is they are available and willing
You may need to video your individual user testing for review as a group. At least 6 – 10 minutes of this footage needs to be uploaded to your Teams portfolio to demonstrate your user testing skills.
As much as possible, keep things consistent between your user tests:
use the same script,
follow the same protocol, and
answer questions in the same way. (You should be working from the same plan)
Immediately after each test, do a quick debrief with your team and write down any reactions or thoughts that came up.
You will most likely forget them, so it's important to write them down right after the user test. Each test needs video evidence, raw data photographed and submitted, user consent forms submitted (scanned or photographed) and any other information you decide is appropriate
Part B –Individual testing & group findings, Tuesday Week 10 @9am
Individual user tests: Individual write up of the tests individual team members conducted including photo evidence of notes used AND consent forms (This must include the Portfolio URL of your individual test 6 – 10 mins. Videos need to be uploaded to the HIT 381 YouTube site)
As much as possible, keep things consistent between your user tests: use the same script, follow the same protocol, answer questions in the same way; use the same questionnaires/interview questions. Immediately after each study, record your reactions down on paper (paper or digital) there is a good chance you will forget things so write them down asap.
Each test needs to have some video evidence.
16
Step 5 Collate Test Results – Findings
After testing….
with your team, review your findings:
Go through all the notes and video/audio recordings.
Try to be objective, don't write problems off.
Human Computer Interaction Design
STEP 4: COLLATE TEST RESULTS
After testing, with your team, review your findings:
Go through all the notes and video/audio recordings.
Try to be objective, don't write problems off.
Discuss as a team and try and define some general patterns in user behaviour.
When you identify some interesting points, talk deeply about them - ask each other questions, recreate the different user tests, analyse user decisions, other paths they could have taken, and so on.
Let your insights guide redesigns for your next prototype iteration.
The end product of your discussion should be:
An analysis of the patterns of behaviour identified
a summary of the major findings from the tests
a list of changes that you plan to implement.
Revisit your implementation plan and add your new tasks.
17
Data analysis
talk deeply about interesting points
ask each other questions
recreate the different user tests, analysing user decisions, other paths they could have taken,
explain user behaviour with theories of your own
Human Computer Interaction Design
Use quotes from
study participants
When you identify some interesting points, talk deeply about them - ask each other questions, recreate the different user tests, analysing user decisions, other paths they could have taken, and so on. This is your chance to explain user behaviour with theories of your own. As often as possible, use quotes for study participants - quotes are a great way to ground your discussion and they add a lot of texture to your results.
18
From the analysis, create a group summary
Include:
An analysis of the patterns of behaviour identified
a summary of the major findings from the tests
a list of changes that you plan to implement.
Human Computer Interaction Design
Where do
breakdowns occur?
general patterns
for user behaviour?
Collate Test Results – Findings
STEP 5: COLLATE TEST RESULTS
Let your insights guide redesigns for your next prototype iteration.
The end product of your discussion should be:
An analysis of the patterns of behaviour identified
a summary of the major findings from the tests
a list of changes that you plan to implement.
19
Improve your prototype by improving the design in light of these results.
These need to be scheduled into the implementation plan with details of who & when
Human Computer Interaction Design
Collate Test Results – Findings
You will use the results of this assignment to inform your final iterations and updates for your group project.
20
workshop
Be prepared to present findings from your evaluation
(about 1 minute per team member)
Human Computer Interaction Design
21
Submit – User Study
Submit as a team:
Overview of user studies (who, what when where why)
Summary of findings
Changes to be made as a result (justify decisions)
Individual studies should be included as an Appendix (video data links )
Implementation plan
Update to reflect changes
Pdf of plan
Human Computer Interaction Design
The URL of your project. Note: the URL must work at least until your assignment is graded. If it doesn't work, you'll receive no credit.
In 100 words or less, describe your user and their goals.
A description of the steps required for us to use your interface to accomplish this goal. The level of detail should be sufficient for me to follow the instructions. (They need not work yet; they will for the full functional prototype.)
An implementation plan. What are the "chunks" of development that need to happen? Who is responsible for which chunks? When will they be completed? What research(if any) is required?
Ensure you have rank-ordered your list of functionalities you are implementing indicating c
The URL of your project. Note: the URL must work at least until your assignment is graded. If it doesn't work, you'll receive no credit.
In 100 words or less, describe your user and their goals.
A description of the steps required for us to use your interface to accomplish this goal. The level of detail should be sufficient for me to follow the instructions. (They need not work yet; they will for the full functional prototype.)
An implementation plan. What are the "chunks" of development that need to happen? Who is responsible for which chunks? When will they be completed? What research(if any) is required?
Ensure you have rank-ordered your list of functionalities you are implementing indicating current progress and future plans
urrent progress and future plans
User Testing
Group Details
The URL of your project. Note: the URL must work at least until your assignment is graded. If it doesn't work, you'll receive no credit.
In 100 words or less, describe your user and their goals.
A description of the steps required for us to use your interface to accomplish this goal. The level of detail should be sufficient for me to follow the instructions. (They need not work yet; they will for the full functional prototype.)
An implementation plan. What are the "chunks" of development that need to happen? Who is responsible for which chunks? When will they be completed? What research(if any) is required?
Ensure you have rank-ordered your list of functionalities you are implementing indicating c
The URL of your project. Note: the URL must work at least until your assignment is graded. If it doesn't work, you'll receive no credit.
In 100 words or less, describe your user and their goals.
A description of the steps required for us to use your interface to accomplish this goal. The level of detail should be sufficient for me to follow the instructions. (They need not work yet; they will for the full functional prototype.)
An implementation plan. What are the "chunks" of development that need to happen? Who is responsible for which chunks? When will they be completed? What research(if any) is required?
Ensure you have rank-ordered your list of functionalities you are implementing indicating current progress and future plans
urrent progress and future plans
The URL of your project. Note: the URL must work at least until your assignment is graded. If it doesn't work, you'll receive no credit.
In 100 words or less, describe your user and their goals.
A description of the steps required for us to use your interface to accomplish this goal. The level of detail should be sufficient for me to follow the instructions. (They need not work yet; they will for the full functional prototype.)
An implementation plan. What are the "chunks" of development that need to happen? Who is responsible for which chunks? When will they be completed? What research(if any) is required?
Ensure you have rank-ordered your list of functionalities you are implementing indicating c
The URL of your project. Note: the URL must work at least until your assignment is graded. If it doesn't work, you'll receive no credit.
In 100 words or less, describe your user and their goals.
A description of the steps required for us to use your interface to accomplish this goal. The level of detail should be sufficient for me to follow the instructions. (They need not work yet; they will for the full functional prototype.)
An implementation plan. What are the "chunks" of development that need to happen? Who is responsible for which chunks? When will they be completed? What research(if any) is required?
Ensure you have rank-ordered your list of functionalities you are implementing indicating current progress and future plans
urrent progress and future plans
Part B –Individual testing & group findings, Tuesday Week 10 @9am
Individual user tests: Individual write up of the tests individual team members conducted including photo evidence of notes used AND consent forms (This must include the Video URL of your individual test 6 – 10 mins. Videos need to be uploaded to the HIT 381 YouTube site)
Findings: A summarised list of your findings from all tests and actions to be taken as a result. As a group summarise;
What were the major interaction patterns identified?
What were the major findings? How do these relate to the goals of your test?
What changes did they translate to?
For each change, include a brief explanation for why you selected it
How did this round of testing compare to the Heuristic Evaluations?
Plan: A PDF copy of this week’s implementation plan and link to online version.
22
criteria
Evaluation plan (group)
Pilot Study(group)
Alternative Redesign
Execution (individual)
Test findings & Implications(group)
Implementation plan
(group)
Human Computer Interaction Design
Evaluation plan. (group)
How well-planned was your user test?
Did you justify your approach to the user testing and what components you chose or chose not to include?
Are you questions and measure clear and cohesive?
Did you submit a copy of all your materials that will be used in the user test?
Are the user testers clearly identified?
Is the write up well structured, cohesive, clear & concise?
(Max 15 points)
Pilot test. (group)
Did pilot testing occur?
Was video evidence provided showing 6- 10 mins (max) of the pilot test? Demonstrating the process to be used?
Were all group members present and engaged
Are any changes to the plan clearly articulated and used in subsequent testing
Are any changes to the prototype required and documented as a results of the pilot test (Max 15 points)
Alternative redesign. (as indicated)
Did you select an interesting component to redesign?
Was a justification provided for you selection?
Is an image of the paper prototype included and is it a quality artefact
Are group/individual roles clear in this task
(Max 10 points)
Was a functional alternative created? Are before and after images included?
(Bonus 10 points)
Execution of user testing. (INDIVIDUAL)
Was your study well executed? Is it evident that the user test plan modified by the team was used (15)
Was video data submitted of at least 6- 10 minutes of your user test?(5)
Is the write up well structured, cohesive and clearly details insights ( ie data is analysed and synthesised) (10)
Were any substantial issues with your interface design identified?(5)
Were consent forms and other testing documentation used and included in write up (5) (Max 40 points)
Test findings & implications(GROUP)
Has the data from the individual tests been analysed and synthesised to summarise findings
Have these been discussed in relation to the goals of your usability test?
Did you come up with multiple changes to make to your application based on user feedback?
Are there justified reasons for making or not making particular changes?
Is the write up well structured, cohesive and clearly details insights ( ie data is analysed and synthesised)
Was a comparison with HE made
(Max1 5 points)
Implementation Plan:
Did you update your implementation plan accordingly?
Does it reflect the changes you need to make to update your prototype?
Is it clear who is doing what?
(Max 5 points)
23
Your turn
Begin planning your user study
Go to Learnline & download the assessment task
As a group develop your user test plan
Decide what aspect of the prototype you will redesign
Human Computer Interaction Design
