Case study
SensiWatch Platform Usability Testing
@ Sensitech Inc
PROJECT OVERVIEW
I was in charge of the Usability studies for Sensitech’s SensiWatch Platform.
I conducted usability test sessions on Product Managers and Engineers and created reports covering the overview and goals, interview script, methodology, role play and task list of the usability testings performed. I also implemented metrics gathering system and user paths for quantitative analysis and identified priorities and potential opportunities to pursue.
SUMMARY
The purpose of this study was to test the usability of SensiWatch Platform. Our goal was to identify any usability problems, collect data about ease of access and find-ability, and determine the participants’ satisfaction with SensiWatch Platform, via observing and listening to users as they complete typical tasks using the platform, such as editing a trip.
GOALS
Task completion - Can users follow through a task and complete it?
Find-ability - Does users know where to find a specific page?
Surprises - Were there a limited number of surprises? Were most surprises positive, negative, or somewhere in between?
METHODOLOGY
Duration: 1 hour
Length: 7/6/2020 – 7/23/20 Location: Virtual
Privacy: Not recorded, confidential Mode of Instruction: Verbal
PARTICIPANTS
Eighteen (18) individuals in the following departments and roles participated in the testing sessions held between July 6 and July 23, 2020:
Professional Services SQA Tester/Programmer Software Development Supply Chain/Materials Stockroom/Receiving Quality Compliance Product Management/Product Marketing/Accounts Payable/Client Services Production/HR/Admin/BA
INTRODUCTION
CONFIDENTIALITY
Users were informed that they would be “turned into a number” and that their identity would be kept confidential.
CLARIFICATION
It was emphasised that the system was being tested and not the users. Users were encouraged to be honest and open, and to think out loud.
CONNECTION
To further establish a connection with the user, the session started with role play. Users were told to imagine that they were the shipping managers for a global company and that they have to perform several tasks on the platform daily. After which, they were told that they are the admin team and have to perform several administrative tasks.
TASKS POST-TEST QUESTIONS
While there were some variation in the prompts users were given, all users were asked to complete the following tasks:
1. Find a trip in transit with an acknowledged alarm - Is the filter feature intuitive and useful to users?
2. Edit a trip - Users should be able to edit information on a trip. Would the action be intuitive?
3. Create a trip template - Users are presented with pop up prompting if they want to exit, would they be able to understand it? 4. Export a report - User should be able to find the button that allows them to export a report.
In addition to being prompted to complete the above tasks, users were also asked the following questions at the end of each session:
On a scale of 1 to 10 (1 being not likely and 10 being highly likely) would you recommend this platform to a colleague?
On a scale of 1 to 10 (1 being not likely and 10 being highly likely) would you enjoy this product in its current state on a daily basis?
METRICS
SCALE OF SUCCESS
The user had to be informed of how to complete the task.
The user required guidance to complete the task OR the user spent a significant amount of time to complete it.
The user was able to complete the task after moderate difficulty OR The user did not select the most efficient method.
The user was able to complete the task with few challenges OR The user did not select the most efficient method.
The user knew how to complete the task, or managed to do so without assistance or encountering any challenges.
Sensiwatch platform
Sensiwatch platform
HIGH LEVEL FINDINGS
Appendix
Users were given scores (based on Metrics) and the scores were used to determine if the tasks has usability issues.
Task 01
Find an In-Transit Trip with an Acknowledged Alarm
Goal: Explore the Trips Page and use Filters
Findings
91% of participants did not require guidance to complete the task. (Users were able to locate and use the filter function on the Trips page to find an in-transit trip with an acknowledged alarm.)
9% of participants required guidance to complete the task. (Users were unable to locate and use the filter function on the Trips page to find an in-transit trip with an acknowledged alarm.)
95% of participants enjoyed the overall experience related to the first task.
Users did not have major issues with the filters but many commented that the Search boxes and information on this page were overwhelming.
Users also tried out the Search bar in the Filter function but the results led them back to the main page instead of searching in the Filter page. It was a surprise for the users as they were expecting the Search bar to be searching for keywords in the Filter page only.
+ Quote: "I like the map, it's helpful to see the trips."
- Quote: "There should be some information on the icon when I hover over it."
Task 02
Edit a Trip
Goal: Explore Trips page and use the floating Button/Trip ID Field
Findings
85% of participants did not require guidance to complete the task. (Users were able to locate and use the floating button or the Trip ID field on the Trips page to edit a trip.)
15% of participants required guidance to complete the task. (Users were unable to locate and use the floating button or the Trip ID field on the Trips page to edit a trip.)
27% of participants enjoyed the overall experience related to the second task.
Users find it a hassle to have to scroll all the way to the right to edit a trip. Even though there was the option of double clicking the Trip ID to edit the trip, they would prefer to be able to edit it just by clicking on the whole row.
+ Quote: N/A
- Quote: "Other websites I visit allow me to right click or double click the area to edit something."
Task 03
Create a Trip Template
Goal: Navigate the Trips tab on the Admin Page
Findings
74% of participants did not require guidance to complete this task. (Users were able to navigate to the trips tab from the admin tab to create a trip template.)
26% of participants required guidance to complete this task. (Users were unable to navigate to the trips tab from the admin tab to create a trip template.)
17% of participants enjoyed the overall experience related to the third task.
Users think there should be flexibility when adding and arranging stops/location’s order. A significant number of Users tried to drag the stops to change the sequence only to find that it was not possible to do that.
+ Quote: "The forms on the different pages seems like they have the same layout, I like that it is consistent." “It is nicely laid out, not all crammed together and has organization to it."
- Quote: "I want to be able to save as draft instead of having only the option of saving or cancelling."
Task 04
Export a Report
Goal: Explore the Reports page and locate export button
Findings
34% of participants did not require guidance to complete the task. (Users were able to navigate to the reports tab to export a report.)
66% of participants required guidance to complete the task. (Users were unable to navigate to the reports tab to export a report.)
35% of participants enjoyed the overall experience related to the fourth task.
Users reported feeling inconvenienced at not having an easily-accessible button for exporting. All users asked to complete this task suggested having a prominent button.
+ Quote: “I love how colorful the graphs are. It looks very clear and I think it’s really helpful.”
- Quote: "If there was an export button on the top, it would be a lot easier."
DESIGN RECCOMENDATIONS
TRIPS PAGE
Painpoints
Users had a difficult time locating the icon to view/edit a trip.
Confusing pop-up message prompting the users to save their changes causes Users to make the wrong action.
No flexibility when adding and arranging stops/location’s order. Users tried to drag the stops to change the sequence only to find that it was not possible to do that. The only way to rearrange stops is to delete each stop.
Recommendation
A working (visible) floating button that floats over the row, and/or allowing users to access the edit trip action via right click.
Edit the message in the pop-up window. Asking "Do you want to continue?" often confuses users who assume the message is asking, "Do you want to save?" Changing the message to "If you leave this page, your changes will not be saved" with the option of cancel or ok, makes it less confusing.
The ability to reorder stops by drag and drop action.
Original
Users struggled to locate the icon for editing the trip as they had to scroll horizontally to locate the icon at the end.
Recommended Design
Working (visible) floating button that floats over the row, and/or allowing users to access the edit trip action via right click.
Original
Users were confused and clicked no because they were not sure which was the right button to click.
Recommended Design
A clearer message prompting Users that their changes will not be saved if they choose to leave the page.
Original
Users would have to remove stops individually if they make a mistake in the sequence.
Recommended Design
Drag and drop function allows Users to save time when they want to rearrange the stops or if they made a mistake.
FILTER PAGE
Pain points
Overwhelming information in Filter page.
Search bar in the Filter function brings Users back to the main page instead of searching in the Filter page. Users were expecting the Search bar to be searching for keywords in the Filter page only.
Recommendation
Our recommendations for this page involve redundancy and ambiguity surrounding the various search bars.
One option is to remove several unnecessary search bars and have search bars only for fields that have an indeterminate number of filter options. Another option is to completely remove the top search bar at the all filters page because it serves the same function as the search bar in Trips page, or include the field hint "Search Trips" in the search bar.
The field names should stand out more. Users also indicated a preference for having Type ahead function for all the search bar to help users more efficiently locate trips.
Original
Users commented that the Search boxes and information on this page were overwhelming.
Recommended Design
A higher color contrast so that the information is well segmented.
Remove the Search boxes and make the Search option show up only when Users click on the magnifying glass icon.
REPORTS PAGE
Pain points
No easily-accessible button for exporting.
Design inconsistency in color choices.
Recommendation
We recommend adding an easily-accessible export button at the top of the page, along with checkboxes on each report to give users the option of making one or multiple selections for exporting.
Original
Users would have to click on the little icon to find the button for exporting data.
Recommended Design
Have a prominent button on the top right panel so that Users can see it and export data easily.
Original
The Export button is in yellow. There is no yellow color in the design theme and overall layout of the platform.
Recommended Design
Change the button color to blue color from the design theme.