I conducted a heuristic evaluation on the 1.1.40 release of Schlage Sense app for Android on September 24, 2018. The Schlage Sense app is used in conjunction with the lock of the same name. I used Nielsen’s 10 heuristics to perform this evaluation, noting potential issues based on expert experience. Due to account restrictions, I was only able to perform the first couple of steps of the adding the wifi adapter portion of the procedue.

Overall, the review found many issues throughout the app, with several of them being of high severity. In the early sections of the app, it’s really doing too many things on one page—whether that’s agreeing to policies you don’t have to look at, or being able to perform actions without having to create accounts or sign in. Once in the installation guide, there are some serious issues with navigation and not being able to see ways to recover from installation errors unless going into the menu to find them from an extremely long list of topics.

Smaller, easily-fixable elements included broken images and buttons, placement of instructions and buttons, and the displaying of notices.

The big issue, however, is that there is no unifying experience throughout the app. Sometimes the menu is on the right side, sometimes it’s on the left. There’s no way to navigate to previous/ earlier parts of the app, and then once the user returns to the app, it’s a different experience from when they started. It feels like the amalgamation of at least three different systems into an experience that is ultimately very difficult for the user.

Outside of UX, there are marketing issues as well re: integration with Alexa or other non-Apple services. If the wifi adapter is required for Alexa or Google assistant, that needs to be mentioned much earlier in the process as well as on the box, and a bundle with the adapter should be offered as well.

See the Detailed Findings by Item section for further details.

Explanation of Findings Table

Heuristic: Nielsen’s 10 heuristics, http://www.useit.com/papers/heuristic/heuristic_list.html

Visibility of system status

The system should always keep users informed about what is going on, through appropriate feedback within reasonable time.

Match between system and the real world

The system should speak the users' language, with words, phrases and concepts familiar to the user, rather than system-oriented terms. Follow real-world conventions, making information appear in a natural and logical order.

User control and freedom

Users often choose system functions by mistake and will need a clearly marked "emergency exit" to leave the unwanted state without having to go through an extended dialogue. Support undo and redo.

Consistency and standards

Users should not have to wonder whether different words, situations, or actions mean the same thing. Follow platform conventions.

Error prevention

Even better than good error messages is a careful design which prevents a problem from occurring in the first place. Either eliminate error-prone conditions or check for them and present users with a confirmation option before they commit to the action.

Recognition rather than recall

Minimize the user's memory load by making objects, actions, and options visible. The user should not have to remember information from one part of the dialogue to another. Instructions for use of the system should be visible or easily retrievable whenever appropriate.

Flexibility and efficiency of use

Accelerators -- unseen by the novice user -- may often speed up the interaction for the expert user such that the system can cater to both inexperienced and experienced users. Allow users to tailor frequent actions.

Aesthetic and minimalist design

Dialogues should not contain information which is irrelevant or rarely needed. Every extra unit of information in a dialogue competes with the relevant units of information and diminishes their relative visibility.

Help users recognize, diagnose, and recover from errors

Error messages should be expressed in plain language (no codes), precisely indicate the problem, and constructively suggest a solution.

Help and documentation

Even though it is better if the system can be used without documentation, it may be necessary to provide help and documentation. Any such information should be easy to search, focused on the user's task, list concrete steps to be carried out, and not be too large.

Feedback/Issue: Explain the reasons why the interface violates or upholds this heuristic. Be sure to be clear about where in the screen you are referencing.

Screen/Scope: Describe the scope of the feedback or the problem; include whether the scope of the issue is throughout the product or within a specific screen or screens. If the problems are specific to a page, include the appropriate page numbers.

Severity (H/M/L): Your assessment as to whether the implication of the feedback is low, medium, or high severity.

Solutions / Trade-offs: Suggestion for the modifications that might be made to the user interface to address the issue or issues in this row. You MUST include trade-offs to be credible. If you can’t think of some bad trade-off, say so.

Detailed Findings

Heuristic(s)	· User control and freedom · Consistency and standards · Error prevention · Aesthetic and minimalist design
Feedback / Issues	· Users can accept the agreement without even looking at it. · Why have the option to disagree when you are required to agree? · Agree to our Terms and Privacy Policy message is difficult to read.
Screen / Scope	Privacy policy screens
Severity	Low
Solutionsand Trade-offs	· Eliminate agree and disagree buttons from this page. · Have agree button disabled until user has scrolled to the bottom of the policy. · “Agree” should be “Accept” · Increase opacity of message
Screen Captures	See below

Finding 1: Privacy Agreement

Finding 2: Sign In

Heuristic(s)	· User control and freedom · Consistency and standards · Error prevention
Feedback / Issues	· What is the purpose of having users log in or create accounts? · If we want users to log in to the system, there should not be a bypass.
Screen / Scope	Sign in screen
Severity	Low
Solutions and Trade-offs	· Do not include other elements on a sign in screen. See below for example. · Allow federated login? · There should be a different treatment for the Learn More button. At the least, it should be presented as a link. It could be presented on both these pages if need be in a different location as to remove emphasis.
Screen Captures	See below

Finding 3: Schlage Sense Instructions

Heuristic(s)	· Consistency and standards · Aesthetic and minimalist design · Visibility of system status
Feedback / Issues	· Begin button is oddly placed · Hamburger menu needs a label · Is this going to a website?
Screen / Scope	Opening screen/ all screens for hamburger and logo
Severity	Low
Solutions and Trade-offs	· Center or right justify begin button · Label hamburger with “menu” either to the left or below icon. · If going to a website, must receive notification that they’re leaving the app and going to a site. · Center the Schlage logo
Screen Captures	See below

Finding 4: Parts Included

Heuristic(s)	· Match between system and the real world · Help and documentation
Feedback / Issues	· What should a user do if they’re missing a part? · Some of the items are small and difficult to see.
Screen / Scope	Parts included page
Severity	Medium
Solutions and Trade-offs	· Include zoom controls · Have itimized list · Label parts · Include link to instructions for what to do if they’re missing a part
Screen Captures	See below

Finding 5: Hold deadbolt against door

Heuristic(s)	· Error prevention
Feedback / Issues	No way back to parts list from this page
Screen / Scope	Hold deadbolt screen
Severity	Low
Solutions and Trade-offs	Include back button
Screen Captures	See below

Finding 6: Install drive-in style plate

Heuristic(s)	· Consistency and standards · Error prevention
Feedback / Issues	· Video shows flathead screwdriver, when Tools Needed said that you only needed a phillips head screwdriver
Screen / Scope	Install drive-in style plate
Severity	Low
Solutions and Trade-offs	Include in Tools Needed message that a flathead screwdriver or knife might be necessary to pry parts.
Screen Captures	See below

Finding 7: Notices

Heuristic(s)	· Consistency and standards · Error prevention
Feedback / Issues	· In some cases, the notice appears on page open, while others it doesn’t. · Notice button, ironcially enough, is not easily noticable. · There’s no solution presented to the notice. E.g. if my door does have a side light, what should I do? · There’s going to be a disconnect in the popup on open situation, because users aren’t going to know the context of the issue since they haven’t read about the step yet.
Screen / Scope	All pages containing notices
Severity	Medium
Solutions and Trade-offs	· Move notices to being included in the direction text rather than a popup that you have to open to read. · Have call to action for solution · Many of these solutions are found in Finding 11.
Screen Captures	See below

Finding 8: Install cable into inside assembly

Heuristic(s)	· Match between the system and the real world · Consistency and standards · Error prevention
Feedback / Issues	The video doesn’t totally match what is happening when trying to plug in the lock.
Screen / Scope	Install cable screen
Severity	Low
Solutions and Trade-offs	Add line to instructions saying that there is a notch on the right of the outlet that needs to line up in order to plug it in.
Screen Captures	See below

Finding 9: Review label on your printed instructions

Heuristic(s)	· Consistency and standards
Feedback / Issues	Instructions are located at the top of the screen, while all others so far have been at the bottom.
Screen / Scope	All screens where text is on the top
Severity	Low
Solutions and Trade-offs	Move instructions to the bottom
Screen Captures	See below

Finding 10: Get the sense app

Heuristic(s)	Error prevention
Feedback / Issues	· This is really late in the process, especially when this is being done from the app. · Probably not best to show the low rating of the app while in the app istelf. · Instructions are at the top, while all others have been at the bottom. · No links to them in their respective stores.
Screen / Scope	Get the Sense App
Severity	Low
Solutions and Trade-offs	Remove page? Is this page even necessary at this point?
Screen Captures	See below

Finding 11: Installation is complete

Heuristic(s)	· Visibility of system status · Error prevention
Feedback / Issues	· The user just sits in a loop of installing hardware with no real way to return to the start of the app.
Screen / Scope	Installation complete page.
Severity	High
Solutions and Trade-offs	· Next button should say “Start over” · There should be a link to adding your lock from the pairing pairing page.
Screen Captures	See below

Finding 11: Menu containing unique items

Heuristic(s)	· User control and freedom · Consistency and standards · Error prevention · Recognition rather than recall
Feedback / Issues	· There are many issues/ topics that are not mentioned during the installation process · The next buttons often are dead and go nowhere when tapped. · If a user never taps on the menu, they would never know that there are solitions to some of the problems they may have.
Screen / Scope	· Measure door · Measure between bolt and latch · Take another measurement as shown · Minimum clearance · Find white dot on connector · Check the thumbturn · Change to square corner plate · Measure backset and select which one you have · Review product labels · Troubleshooting · Forgot my programming code – possible problems · Need to delete an access code
Severity	High
Solutions and Trade-offs	· These solution topics need to be metioned during the installation proccess. Include these as buttons to the particualr sections, with the previous/ next buttons working correctly in relation to where they occur in the flow. · Troubleshooting/ help should be available as a link on every page.
Screen Captures	See below

Finding 11: Broken images and videos

Heuristic(s)	Help users recognize, diagnose, and recover from errors
Feedback / Issues	Many of the extra topics outside of the main installation screens contain broken images and videos.
Screen / Scope	· Measure between bolt and latch · Find white dot on connector · Straighten keypad and plate · Measure backset and select which one you have · Review product labels
Severity	High
Solutions and Trade-offs	Repair these screens so the images and videos work properly.
Screen Captures	See below

Finding 12: Create account

Heuristic(s)	· Error prevention · Help and documentation
Feedback / Issues	There is no check of email address or passwords until after selecting sign up.
Screen / Scope	· Create account screen
Severity	Medium
Solutions and Trade-offs	· Check accounts and password requirements before clicking sign up · Allow for federated login?
Screen Captures	See below

Finding 13: Add/ pair lock

Heuristic(s)	· Consistency and standards · Error prevention
Feedback / Issues	· Hamburger on opposite side from what was presented before this. · The order of elements on this screen is confusing and might not be necessary at all.
Screen / Scope	· Main Schlage sense app pages
Severity	Medium
Solutions and Trade-offs	· Hamburger menu needs to be on the same side throughout and have a label for menu. Either side is fine, although it might make more sense to put it on the left side and put the pair new lock on the right side. · There isn’t much of a need to have the add a lock that someone else has set up direction, and it’s inomplete because they would find that over in the menu. · Both of these directions could be lost if the menu was labeled and there was a label under the lock icon that says “add lock” · If the direction line needs to stay, have it appear white to match the top right icon.
Screen Captures	See below

Finding 14: Lock settings

Heuristic(s)	· Error prevention · Match between the system and the real world · Help and documentation
Feedback / Issues	· Initially set to auto lock at 30 seconds without any warning · What is one-touch locking and keypress beep?
Screen / Scope	· Settings page
Severity	Medium-high
Solutions and Trade-offs	· Include these settings when initially naming the lock and creating the code · Have a tooltip to explain 1-touch locking and keypress beep
Screen Captures	See below

Finding 14: Register device

Heuristic(s)	Visibility of system status
Feedback / Issues	· Users aren’t notified that they are being taken to a site outside of the app.
Screen / Scope	· Register device page
Severity	Low
Solutions and Trade-offs	Need to be notified that you’re leaving the app and loading a web page.
Screen Captures	See below

Finding 15: Adding a new lock

Heuristic(s)	Error prevention
Feedback / Issues	· When adding a new lock, the instructions go straight to entering a code rather than installation
Screen / Scope	· Add a new lock screens
Severity	Low
Solutions and Trade-offs	Include a selection page of choosing between starting with hardware installation or adding the code to the app.
Screen Captures	N/A

Finding 16: Waiting for status update

Heuristic(s)	· Help a user recognize, diagnose, and recover from errors
Feedback / Issues	· There’s no response to things taking too long or suggestions for recovering from that issue. I wound up tapping the ? icon, which reset the system and showed the lock status.
Screen / Scope	· Lock status screen
Severity	Medium
Solutions and Trade-offs	After some period of time, maybe around 5-10 seconds, there needs to be a suggestion about what to do. Hitting the refresh icon near the bottom did nothing, but tapping the ? icon refreshed the page.
Screen Captures	N/A

Finding 17: Add wifi adapter

Heuristic(s)	Error prevention
Feedback / Issues	· The + doesn’t really make the wifi adapter enrollment icon that unique.
Screen / Scope	· Wifi adapter screen
Severity	Low
Solutions and Trade-offs	Have a more unique icon for this action.
Screen Captures	Below

Finding 18: Add wifi adapter error

Heuristic(s)	Help users recognize, diagnose, and recover from errors
Feedback / Issues	· There are no solutions for recovering from the error of the adapter not being found by the app beyond swiping down. What happens if it still can’t be found?
Screen / Scope	· Add wifi adapter screen
Severity	High
Solutions and Trade-offs	· Have the same message that’s there re: swiping down to reset, but have a link saying something to the effect of “still not working?” and then some troubleshooting solutions.
Screen Captures	Below

Heuristic Evaluation for Schlage Sense Page 17 of 26