🎯 Reason
With the development of TikTok’s Local Service Business, a tool was needed to monitor merchants’ experience and guide product iteration. Through research, I found that "Inaccessible data” and “No direct results” hindered users' ability to locate problems.
🛠️ Action
I designed a data dashboard that summarized and visualized UX data. It allowed Product Managers and Executives to monitor merchant experiences and follow-up problems. Facing the challenges above, I implemented 4 design strategies such as differentiated user flows and layered information display.
📉 Impact
1. The redesign achieved an effectiveness rate of 75%, meeting the launching standards.
2. The redesigned workflow improved the efficiency of users’ locating and analyzing data by 33%.
Methods
Competitive Analysis, User Interviews, Workflow Organization of Various Personas, Information Architecture, Usability Test
*For best viewing exprience on mobile, please switch your phone to landscape mode
1 minute design summary!
User experience data can be messy and hard to grasp. Product Managers at TikTok struggled to locate product problems from the sea of UX data.
Condensed information architecture
Layered information display
Data visualization templates
1. Monitor and compare real-time data for 3 key UX metrics
2. Locate product problems with short paths using pre-analyzed results
3. Understand the problems with well-organized information
4. Easily track and follow up the problem
Design Process:
Jump to any part of the project by clicking the link
Why I designed it?
Background: Product managers and executives need a dashboard to summarize, visualize, and monitor the user experience data of merchants to improve merchants' business experience.
This UX data dashboard is called UX Measurement Platform. With only out-dated design files from 2 years ago, I began this project basically from scratch, except using some visual guidance.
I started with defining user needs ...
User research: “Scattered data across platforms” and “Hard to follow up on problems” hindered executives and product managers to locate problems.
To gain a deeper understanding of user needs, I conducted 5 interviews with 2 user groups, Product Managers and Executives.
Interview notes
“Locating product problems” is the primary goal of users, but ...
😢 Pain Point No.1
Scattered data across platforms
Users need to find different data one by one
😢 Pain Point No.2
Hard to follow up on problems
Excels and meetings are hard to synchronize
When users cannot fully articulate their needs, I turned to ...
Competitive Analysis: “Inaccessible data” and “No direct results” are common problems in mainstream products that should be avoided.
I researched 7 mainstream products and summarized some takeaways and 2 common problems.
Competitor analysis form
😢 Pain Point No.3
Inaccessible data
Excessive data buries important info deep in pages
😢 Pain Point No.4
No direct results
Users still need to calculate some essential metrics
After summarizing all problems ...
Design challenge: How might we design a UX measurement platform that provides easy access to important data and quick understanding of product problems?
Painpoints
Scattered data across platforms
Users need to find different data one by one
New features/designs
Data unification
Unify different types of UX data on one platform
Hard to follow up on problems
Excels and meetings are hard to synchronize
Follow-up workflow
Design problem follow-up user flows
No direct results
Excessive data buries important info deep in pages
Differentiated workflows
To give 2 user groups the results they need
Inaccessible data
Users still need to calculate some essential metrics
Condensed information architecture
To identify shortest user paths to locate data
Layered information display
To identify shortest user paths to locate data
Data visualization templates
To better visualize different sources of data
How I designed it?
Design strategy NO.1 : I designed differentiated user flows to address different needs.
Executives read the overview on the homepage
“I want to see the overall UX situation, and the progress of top-ranking problems on Q2, 2023.”
PMs drills down into pages to identify problems
“I want to see the problems of the Merchants’ Settling module this month.”
Click to enlarge
Design Strategy NO.2 : Condensed information architecture to identify shortest user paths to locate data.
3 Layers of information provides easy access within 2~3 clicks.
Design Strategy NO.3 : Summarized the key information of UX problems and designed layered information display for 2 scenarios.
Scenario 1: Identify problems
Display problem cards to show 5 important information.
Scenario 2: Analyze problems:
Display problem details to show 5 categories of related information.
Design strategy NO.4 : Designed 2 kinds of data visualization templates for different sources of UX problems.
Type 1 : Subjective user opinion problems
Use quantity visualization.
Type 2 : Objective behavioral problems
Use journey visualization.
But these strategies didn’t just pop up all at once.
Prototyping and iterations: Finalized the prototype after 3 iterations after exploring alternatives and tradeoffs.
Iteration example
1. Expectation 💭
The 4 cards that represent 4 categories of UX problems can help users find out what problem is worth improving
2. Problem 🚩
Users were confused about:
1. The way these categories were formed
2. How they could interact with it
3. Thinking 🤔
3 reasons behind this confusion:
1. Users' classification logic differs from designers
2. 4 cards are not salient and look similar
3. Users are not familiar with this interaction
4. Exploring Alternatives 🆎
I tried these but did not pick them:
1. Display all 4 cards - classification logic still unclear
2. Display data without categorizing - info overload
3. Contain data in each card - unneccessary layers
5. Solution ✍🏻
1. Split this section into 2 tabs
2. Replaced with a more familiar info structure
3. Reduce the visual weight of the lower part
6. Tradeoffs ⚖️
Lack of a sense that "the platform has it all summarized for you", but I think user understanding is more important
Iteration process
What is the result?
Project impact: The design gain 75% effectiveness and increased 33% problem locating efficiency. The platform is currently being developed.
Results from the usability test
My contribution of this project
As this is a North Star project initiated by the design department, I completed this project rather independently, including research, design, and visual. But in other 5 design requirements I intake, I closely collaborating with product, engineering, and UI teams.
Click here to see moreAt the end ...
Reflection
What I learned:
User needs can be ambiguous and unstable, and we need to find ways to investigate or complement it.
I also learned other UX skills such as principle/methods for toB UX design, UX metrics, data visualization, industry-standard visuals ...
What I could have done better:
I should discuss the expectation clearer before any iteration starts.
I should bring more independent ideas before any discussion and learn from critiques.
I should be braver to own my design.
Future plans for the project:
I will keep iterating, encourage more business units to use the this platform, and track usage rates and the number of issues resolved on a bi-monthly basis to validate the design.
------ Here starts the additional content for this project, including UX Metrics Research and Requirement Intakes ------
Background: The UX measurement requires an automated analysis process for to turn UX data into conclusions.
Evaluation methods: Assessed merchants’ experience through 4 types of UX data, found root causes by VOC analysis using experience dictionary.
Highlight 1：Discovered that satisfaction data was too higher. Upgraded to effort score and raised the confidence rate.
- Distorted Satisfaction data
With several modules consistently scoring above 9.5, the satisfaction score is too high to provide meaningful results for comparison.
The reason behind this was the survey trigger mechanism, which mainly collected ratings from "successful operations" by merchants.
- Switch from satisfaction score to effort score
With several modules consistently scoring above 9.5, the satisfaction score is too high to provide meaningful results for comparisons.
Following industry-standard practices. I changed the question format from "Are you satisfied with..." to "It is easy to ..." and applied this change to all online surveysnThe reason behind this was the survey trigger mechanism, which mainly collected ratings from "successful operations" by merchants.
- The transition increased data reliability
Through a week-to-week comparison, a 16% decrease in data scores is observed, indicating the success of the metrics upgrade.
Highlight 2: Optimized the analysis process by splitting the full-quantity analysis into sampling and supplementary analysis, improved accuracy and efficiency.
Highlight 3: Drafted the Q2 Merchant UX evaluation Report to present the analysis results, and shared 100+ UX issues with business colleagues
The content of the UX evaluation reports:
Section 1: UX Performance
Section 2: Modular UX
Section 3: Industrial UX
Section 4: UX issues summary
Requirement 1: Online consultation activation
IM online consultation allows consumers to directly ask questions to merchants by clicking on the online consultation button on the merchant's page.
Current online consultation activation process is conducted offline, which is cumbersome, time-consuming, and involves high labor costs.
There is a need to transit this process online. In a sentence, we need to enable merchants to independently activate and deactivate online consultation features online.
Highlight 01: Streamlined activation process to meet the expectation of merchants.
Merchants expect low activation costs 👉🏻 Streamlined activation
Onboarding
Activation
Highlight 02: Tailored guidance for diverse merchant needs.
Diverse merchant needs 👉🏻 Tailored guidance
The onboarding only needs to cover functions needed by merchants because most of them have a basic understanding of this feature. In addition, as we were unable to categorize who needs what, providing a permanent entrance of the onboarding feature is necessary.
Function introduction
Function guide
Final design file
Research
Design
Collaboration, conflict, communication
💥 Conflict
PM: "I don't want this onboarding become to heavy. We can have step-by-step intros of only 2 most important features."
Me: "In-depth introduction of 2 features before users know what are all the features doesn't make sense. And important features not necessarily mean universal features. There is only 1 universal feature, and the rest really depends on what is the need of the users".
💬 Communication
1st discussion with the PM: I proposed that we need to have a permanent entrance of some important features, but PM insisted that this is not the scope of this iteration.
2nd discussion with mentor & leadership: They all agree that a permanent entrance will be better and what proposed by the PM will not be able to pass the design critique.
3rd discussion with the PM: I convinced her that it is not extra workload to make it a permanent entrance, we were just building a structure that will make the onboarding process more flexible, which is good for any future functions that need onboarding. She finally agreed with my proposal.
📉 Impact
This function has been launched successfully after I discussed and iterated the new UI and components of the page with the UI designer.
Requirement 2: Customer satisfaction survey
A customer satisfaction survey will show up when a customer finishes his/her conversation with the customer service of a merchant, but the filling rate of this survey was low.
In order to raise the filling rate, I had a discussion with the PM and UXR, who just conducted a research on this feature. After discussion, my task was to update the user flow and UI of the survey card and change the submission mode from hitting a submit button to real-time auto-submit.
Collaboration, conflict, tradeoffs
💥 Conflict
Me: "I want the survey card to be a popup on the bottom of the page when it initially appear. After users finish the survey, it becomes part of the message flow."
Engineer: " This is hard to realize. You need to make it either a popup or a message as part of the message flow."
Popup example
Message example
💬 Thinking
Popup
Message
Pros
Attract users attention, a higher chance of filling
Smoother interaction in most situations; An easier approach for engineers
Cons
Still need to make it a message if users do not answer;
No submission button now, when this popup should disappear?
The keyboard might cover the survey if user scrolls
📉 Impact
I chose to make it a message, and I added pages explaining how the survey card will self-adapt in the message flow. This feature had been launched, and pm told me that there is an 8% increase of filling rate within a week.
Final design file
