Gap Analysis
Case Study
Purpose of the Project
The purpose of the project is to utilize a framework to identify the knowledge, motivation, and organizational issues that may be contributing to the low conversion rate of users adopting and using the transit feature.
This was the final assignment, simulating a fictional gap analysis, for my Measuring and Performance ITLS 4160 class at USU
The Performance Gap
EpiAssist, an epilepsy management app developed by Brainware, has been in beta for two months and is gaining significant traction. The team recently completed work on a new transit feature that allows users to order ride-sharing and find public transportation routes within the app. However, a few weeks into user testing, they noticed that fewer users are adopting the transit feature than expected.
​
The goal is for at least 35% of the 20 million beta testers to regularly use the transit feature by its official launch in six months. Currently, only 12% of regular users are using the feature, leaving a 23% gap. To be considered a regular user of the transit feature, a user must use it at least three times a week.
Highlights
Skills Used
-
UX Principles
-
Graphic Design
-
Website Design
-
Learning Theory
-
Technical Writing
-
Copy Writing
Tools
-
Sketch Software
-
WIX
-
Photoshop
-
Illustrator
-
Apple Pages
Insights
From a design perspective, my biggest takeaway from this project is understanding that components only look good if they complement the core idea and enhance the user's understanding of it. Good design makes technology easier to use and, in this case, easier to learn from. Different design elements work for different types of content, but if applied incorrectly, they can distract or overwhelm the viewer. Through this project, I feel confident that my ability to work with design elements that emphasize key ideas has improved. However, I acknowledge that my work is not perfect, and there is always room for improvement.
Sketch
Sketch is a design software used for building UX concepts and prototypes before implementation. It is well-known and widely used among industry professionals. As a designer, this project was my first experience using Sketch to develop a user interface.
Learning the Software
Like any new software, Sketch comes with a learning curve. Fortunately, Sketch offers two free courses on their website that new users can take to learn both the fundamentals and advanced features of the software. While not mandatory, each lesson includes project files that learners can use to follow along with the presenter. As a hands-on learner, I took full advantage of this opportunity, knowing it would lead to the best outcome.
Putting it Together
The design of this project went through multiple iterations, with most changes occurring during the wireframing of the low-fidelity version of the site. These changes primarily involved breaking the content into well-designed attributes relevant to the content being discussed. I asked questions like, "What is a container that fits the style of the site yet adds value to content exploration?" and "What design increases accessibility while remaining dynamic?"
The color palette of the site was refined over time, ultimately landing on honey yellow and baby blue for the primary and secondary colors. I felt it was important to also incorporate the national epilepsy color, purple, as an accent for the site. Near the end of the high-fidelity wireframe process, I realized there was a lack of media, such as pictures, to drive interest and motivation, so I added some to close that gap.
Converting a Document Into a Webpage
One of the most challenging parts of the project began even before the more colorful tasks, like the actual design concept. This involved condensing the case study report into the necessary content groupings for the website. It was challenging because much of the original report had to be cut or rewritten, and finding the right balance required strong analytical and critical thinking skills.
Stakeholders
The stakeholders crucial to the success of EpiAssist's new transit feature include the executives at Brainware, the project manager, and the design team.
The design team is responsible for completing the necessary work to ensure the project manager meets the project deadlines.
The executives are responsible for expanding and growing the business while capturing a larger share of the health app market.
The project manager is responsible for ensuring the success of the feature.
Perspectives on the Gap
The team fears that a poor launch could lead to layoffs and downsizing, affecting their ability to support their families, while the project manager seems unconcerned.
The executives at Brainware are committed to meeting the deadline and believe it is achievable. They have invested over $8 million in developing the app and need 35% of users to become regular users to break even and start profiting within a year.
Misison
Successfully release a feature the provides users with a new and exciting way to get around without relying on friends and family for transportation, creating a more complete all-in-one solution in seizure management
Stakeholder Goal
Improve the percentage gap of regular users on the new feature by at least 23% by release day. Additionally, achieve a 80% satisfaction rating from its current 72%.
Global Goal
EpiAssist’s is to have at least 35% of its 20 million subscribers who are enrolled in beta testing adopt and be regularly using the new transit feature by its launch in 6 months.
Goal
Context
1
2
3
4
5
The EpiAssist design team recently had some of its talent reassigned to other Brainware product teams.
Since the reassignment, the team has been working overtime for weeks, leading to burnout and low energy. They are struggling to meet the project's demands by the release deadline.
The team has been diligently investigating why beta users are not utilizing the transit feature. They understand that many people with epilepsy cannot drive and depend on friends and family for transportation. Moreover, most users interviewed and surveyed expressed a desire for more transportation options and welcomed this feature in a seizure app. Despite having created a solution, the team is puzzled by the lack of user engagement with the feature.
The team assigned two UX designers to investigate the problem, leaving just one designer to handle the rest of the work. This designer complained to the project manager about the lack of support to meet the project deadline. The project manager said he would look into it but never did.
The executives at Brainware learned about the issue at the most recent meeting and assigned a senior UX researcher and UX designer to help the team increase regular users to 35% by the feature's release deadline.
Investigation into Performance Problems
The senior UX researcher led the investigation into performance gaps that might be causing the new transit feature's poor performance. They focused on identifying user issues that could be hindering the feature's adoption.
the goal was to determine if users were struggling to understand, find, or even be aware of the transit feature in the app. The researcher ultimately identified the factors contributing to the performance gap for the app's transit feature:
External Contributing Factors
Internal Contributing Factors
Some employees need additional knowledge and assistance on some tasks.
The current tutorial doesn't teach users how to navigate the transit feature, leaving them unsure how to use it.
The team needs a proper instructional designer to build a tutorial.
The size of the design team is currently too small for the amount of work they need finished.
The design team is overworked, resulting in low moral and energy to complete work.
No support and organization due to the lack of project management.
Users are giving up on ordering rides because the process takes too much time and effort.
The app's ride-sharing services are unreliable due to an error that doesn't notify users of the arrival time.
Knowledge Causes and Validation
Ineffective Onboarding Tutorial for Transit Feature
Our user research indicates a critical gap in the current tutorial for the new feature. This lack of clear instruction is causing confusion and hindering user adoption. As a result, users are abandoning the feature and seeking alternative solutions. By creating a more comprehensive and user-friendly tutorial, we can empower users to leverage this valuable feature, potentially leading to a significant increase in user engagement and retention.
Employees Need the Necessary Knowledge
A junior designer on their first major assignment needs help simplifying the new feature's tutorial and would like guidance from a senior designer. Training and guidance from senior employees would provide junior employees the necessary skills and knowledge, as well as a sense of confidence, when tackling new and challenging problems.
Motivational Causes and Validation
The Design Team is Overworked
Our design team is facing a stressful environment, leading to burnout and extended hours for the small staff. This has unfortunately fostered a culture of disengagement, impacting on-time project completion. By addressing these concerns, we can create a more positive work environment, boosting team morale and ensuring projects meet deadlines.
Lack of Accountability from Project Manager
Employee interviews revealed a lack of engagement from the project manager. The team feels unsupported due to infrequent communication and remote work arrangements. This has led to confusion about project direction and some designers taking on additional project management tasks.
Organizational Causes and Validation
Minimal Support for User Notifications
Our ride-sharing feature lacks a crucial function: notifying users when their driver arrives. Interviewed users reported frustration due to the lack of clear arrival alerts. They mentioned missing driver calls and being unable to locate their ride within the app's transit view. Implementing a clear arrival notification system will improve user experience and prevent them from switching to competing services.
Solutions
Knowledge Solutions
Declarative:
-
Educate users on app jargon inside the app tutorial.
-
Provide struggling employees with necessary training, promoting team mentoring.
Procedural:
-
Organize the user tutorial into coherent and reasonable steps for easy navigation.
Metacognitive:
-
Utilize interactive systems to encourage user learning and retention.
Motivation Solutions
-
Hire an instructional designer to assist the design team in the tutorial overhaul.
-
Restructure the design team to address disorganization within the organization.
-
Hire more designers to prevent overwork and boost moral.
-
Discipline project manager.
Organizational Solutions
-
There is a critical user pain point in the app’s ride sharing feature, one that does not notify users when their driver has arrived.
Integration Plan
Hire an instructional designer to overhaul the tutorial system. This needs to happen immediately, because the process of designing and developing a new tutorial can take time intensive.
Restore the design team to its original size to streamline workflow and efficiency. this will relieve the burnout that many are feeling.
The executives need to conduct a performance review of the project manager and evaluate his poor performance during the past six months to determine what kind of discipline is necessary.
Brainware needs to invest in Learning & Development to support individuals training in the company.
The development team will fix the notification issue so that users will receive the toast notification and audible sound on their mobile device when driver has arrived. The update will be released in two months.
Evaluation Plan
Reactions
To measure the effectiveness of the redesigned tutorial, we'll conduct user surveys before and after the beta launch. This will help us understand if the new tutorial improves user experience and encourages them to use the transit feature more.
Learning and Performance
The instructional designer will evaluate user performance and learning during the beta phase of the transit feature, before and after implementing the new tutorial. The designer will investigate whether users successfully use the feature after the tutorial compared to before the update, which may indicate a difference in performance.
Transfer of Behavior
We'll analyze user surveys and app usage data after two months to see if the new tutorial improves navigation, increases user activity, and reduces the need to retake sections. User feedback will also help us identify areas for further improvement in the next version.
Impact
The new tutorial is expected to be a game-changer for our app's transit feature. We anticipate a significant increase in usage and traffic, along with user satisfaction exceeding our initial target of 80%. This will be a major win for our stakeholders!
Conclusion
The success of EpiAssist’s new transit feature hinged on its aim to convert 35% of users into regular users upon launch. With the assistance of an internal researcher and later an instructional designer, they anticipate to exceeded this goal of increasing regular users from 12% to 40% within six months.
This analysis examined all factors, both known and unknown, that could affect product performance, internal team dynamics, and the organization's ability to adapt. Our goal is to develop a comprehensive app solution that not only helps people with epilepsy manage their condition but also improves their quality of life. The app's features aim to restore independence for those who often feel confined at home, offering them greater mobility beyond the limitations of the app itself.