Arrivy Route Planning
Summary
Arrivy hired me to complete a usability study on a new feature of their platform. With the help of their Head of Product, I created a study to help evaluate this feature based on customer needs. After conducting usability studies with current customers, I was able to make design recommendations.
The above image shows the timeline for this project, I had a total of 7 weeks to deliver results.
Goal
Evaluate the usability and desirability of the "Route Planning" feature on Arrivy's website.
Preparation
I discussed with the Head of Product about what specifically about “Route Planning” Arrivy was most interested in learning more about, and any concerns they had. From this information, I was able to build a study plan. This study plan included these research questions:
1. Is the terminology used in the Route Planning feature easy to understand?
2. Can users complete the main tasks of the feature?
3. Do users think the feature is useful?
Research
In order to prepare for this project, I needed to learn more about Arrivy and their product, as well as their customers. I spent some time getting to know the current product and its many functions on their website. I also was able to spend some time getting to know the new feature, and discussing with the Head of Product what they would be interested in seeing people try to use. I also went to Arrivy’s social media pages, and tried to get a feel for what their user base looked like. From this research, I came up with a list of tasks and questions for the usability test.
Testing
Recruitment was done through the Head of Product, and we were able to get 4 relatively diverse participants for the study. We completed testing on UX Cloud. Each participant went through a pre-test questionnaire (answers shown above), 3 tasks, and a post-test questionnaire.
Analysis
To determine the difficulty of tasks I recorded time on each task and if they were completed. I recorded all answers to the pre and post task questionnaires, noting their reasoning for the “rating” questions. In my report I examine the results of each questionnaire, and I go through each task before moving on to my recommendations.
Design Recommendations
These recommendations came from common errors or themes seen throughout the testing. I have 5 recommendations total detailed below:
Time Changes
Currently, Arrivy shows time changes adjustments the user made in this format:
Changes:
Start Time:
6:00 AM to 9:00 AM
End Time:
10:00 AM to 1:00 PM
One participant was particularly confused by this format, thinking that the start time was from 6am to 9am, not that it had been changed from 6am to 9am. That same participant commented that it may be easier to understand if changed to this format:
Original Time:
06:00 AM - 10:00 AM
Changed Time:
09:00 AM - 01:00 PM
I recommend moving forward with this format instead.
Routes Automatically Opening
Multiple participants struggled moving tasks around, with that activity taking the longest out of any other part of the sessions. Part of this was because if they grabbed a task and dragged it over to a “closed” route, the route wouldn’t open up to show the tasks. Also, when you click off a route, it automatically closes, meaning that each time you move a task from one route to the next, you had to drop it into a closed route.
I recommend a few changes here.
Make it so when someone hovers a task over a closed route, it opens up
Don’t automatically close route when clicked off of them, or provide an option to keep all routes open.
It is likely that this type of activity will be used a lot, so it’s important that the user is able to easily move tasks around from route to route. By making these small changes, your users will have a much smoother experience.
Route Visibility on Map
This design recommendation ties in with the last one. Right now, when you click off of a route, regardless of if you have checked the box saying “Show on map,” it automatically takes it off the map. This also made things difficult for participants during the first activity.
Similarly to the last recommendation, I recommend either having an option to keep all routes on the map, or simply make it so that they don’t disappear when clicking off of them. This may make it difficult for users to see which route is selected, so when they click on a different route, I recommend providing some sort of feedback on the map to show the route you just clicked on. This could be done by highlighting the current route of shading the unselected route.
Show the Actual Duration of the Routes
This was a direct request from P2. P2 said that he would really like if this feature could show the exact duration of each route, but currently the durations are rounded
In order for P2 to use this feature, he said it is a requirement for it not to be rounded, but show the exact time. I do not know what goes into calculating this rounded time, but if possible I recommend showing the actual expected duration.
More Feedback from Compact
There were several participants that had difficulties understanding if their routes had been compacted. It was not clear to them that they selected the right section (route or task), and had compacted it. Part of the reason this was difficult for them was because if the route was closed, then they could not see the times being adjusted.
There are a couple ways this could be alleviated. Next to the route name, you could say “Not Compacted” or something similar until the user has compacted it. Or, if a route has already been compacted, you could shade over the compact button so that users cannot click on it, thus knowing it doesn’t need to be compacted. There are likely more solutions to this problem, regardless it just needs to be clearer to users that they’ve compacted their routes.
Conclusion
To conclude this study, I went back to my initial research questions:
Is the terminology used in the Route Planning feature easy to understand?
While we decided not to add any specific tasks to address this questions, I can confidently say that there were no issues with the terminology used in the Route planning feature.
Can users complete the main tasks of the feature?
There was one instance where a task was not completed, which may have been due to some issues with the version the first participant was using. Otherwise, all the other tasks were completed successfully by all other participants in a reasonable amount of time.
Do users think the feature is useful?
While participants varied on how useful they thought this feature was, all of them said they would use it. Nearly all participants gave the feature a high rating.
Reflection
Overall, I feel that this study went very well. Each participant provided great feedback which I’ve been able to translate into design recommendations. Regardless of the issues each participant had, they said they would still use the feature. In its current form, this feature seems to work very well. By making some of the changes I’ve listed in design recommendations, this feature can work even better for customers. If I were to do this study again, I would additionally analyze task success based on error rate. I might also add a few more questions to each questionnaire addressing terminology used in the feature, and some specific questions about what the participant’s company needs from Arrivy. Interviewing the participants, I realized that each participant used Arrivy in a unique way, and thus had different needs. This would be interesting to look into for a future study.
