UX Metrics: How to Know If Your User Experience is Working

Before we get to learning about UX metrics, let’s understand what user experience means. According to Don Norman, who first coined the term, “User Experience (UX) encompasses all aspects of the end user’s interaction with the company, its services, and its products.” True, user experience goes far beyond giving customers what they say they want or providing checklist features. In order to achieve a high-quality user experience in a company’s offerings, there must be a seamless merging of the services of multiple disciplines including user research, engineering, marketing, graphical and industrial design, and interface design.

Although UX is a universal concept, applicable to all physical and digital products and services, it is most commonly studied in its applicability towards digital products and services, like software, apps, and websites.

Let me paint a picture for you- Imagine that you built a website, which gets a lot of traffic. However, the conversion rate is disappointing at its best. Or a lot of people signed up for your software, but have not returned after the first few days. Or else, consider that though an initial number of downloads for your app were high, thanks to the marketing and promotions, but more than half of the users never bother to use it, or even worse, have uninstalled the app within a week or two.

Driving traffic to a site, software or app is easy; deriving initial conversions can also come easily if you are ready to splurge. However, making your users stay engaged and loyal to your product goes beyond marketing. Usability and user experience become crucial factors here.

Steve Krug (author of Don’t Make Me Think – highly recommended for internet dwellers) defines Usability as making sure something works well, and that a person of average ability or experience can use it for its intended purpose without getting hopelessly frustrated.

However, usability is just the tip of the iceberg of User Experience.

With the rising competition in the digital market, product managers and entrepreneurs are steadily investing more time and resources in honing User Experience for the products and services they offer.

For instance – the automobile industry stays centered around physical, tangible products fair and square. With minor digital integrations in a smoother drive around curves, terrains, and roundabouts; better music systems, and safety measurements, automobile giants like Audi, BMW, Ford, and Volkswagen have been trying to one-up each other for generations. However, with its disruptive technology of self-driven electric cars, Tesla Motors has not only ousted its competition but also created a niche market for itself. An intuitive UX and product design give Tesla an edge over other automobile players, proving that if done right, UX can find its applicability in all segments.

To make the case for UX applicability, one needs to be able to track the difference it made. This is where the user experience design metrics come into play.

What are UX Metrics?

Spuriously, UX is considered a subjective aspect.

User interface design metrics are quantifiable, measurable, and reliable data points that evaluate the effectiveness of your UX strategy. They can be aligned to your marketing metrics as well as compared against an ROI to calculate financial feasibility and scale profits accordingly.

UI UX metrics, also known as UX measurement tools lie under three broad categories-

1. Usability Metrics or Descriptive Metrics

As I pointed out earlier, usability is just the tip of the UX iceberg.

Usability metrics focus on the ease of task or event completion by users. It includes commonly tracked metrics like time on task, navigation vs. search, task success rate, ease-of-use rating, etc. Some advanced metrics may also include tracking of interaction patterns.

2. Engagement Metrics or Perception Metrics

Engagement metrics focus on the way users interact with a site, app, or software. Apart from the time per interaction, it also focuses on event streams, page views, scrolling, and most importantly, the attitude of the interaction.

3. Conversion Metrics or Outcome Metrics

Conversion metrics are just as important for the UX team as they are for the marketing team. It focuses on a relatively small set of users who are ready to commit to your product via prolonged engagement, usage, and retention. Insights provided by conversion metrics including conversion rate and Net Promoter Score (NPS) can also help design better solutions that help a business meet its goals.

You may also like: How to Conduct User Research That Matters

Common UX Measurement Tools

Various companies and UX agencies deploy various methods and data points to track UX metrics. Before I dive into which metrics are right for your product, let’s take a look at the prevalent methods and key metrics:

Method 1: Net Promoter Score (NPS)

Introduced by Bain & Company founder, Fred Reichheld, Net Promoter Score (NPS) is a customer loyalty metric that gauges how willing a customer is to recommend a product or service.

At its core, NPS tracks how customers represent an organization to their friends, families, and associates. The same is true for virtually any product or service.

Net Promoter Score categorizes users into one of three groups: Promoters, Passives, and Detractors through their response to a simple question: “How likely is it that you would recommend [Organization X] to a friend or colleague?”

Respondents are given a standard scale that ranges from 0 (not at all likely) to 10 (extremely likely). Based on the responses, users are categorized as Promoters (9-10), Passives (7-8), and Detractors (0-6).

Promoters are the users who are loyal and likely to refer others, thus helping you to lower your cost per acquisition. Passives are generally satisfied users who can sway both ways, depending on competitive offers and incentives. However, Detractors are highly unsatisfied users, who may have serious issues with your product.

Net Promoter Score is calculated by subtracting the percentage of Detractors from the percentage of Promoters. The score is generally displayed as a whole number by dropping the percent sign for the final number and scores range from anywhere between –100 (all Detractors) to +100 (all Promoters). It is imperative that in order to be successful, you would want to convert your Detractors to Promoters.

One of the main reasons why so many companies (physical as well as digital) choose Net Promoter Score is its simplicity. Although NPS lacks deep insights and specific reasons for a product’s success or failure, it can help you gain an insight into the overall scenario and the level of urgency required to handle a given situation. This is why NPS is commonly used as an alternative or supplement metrics to measure UX.

Method 2: System Usability Scale (SUS)

Another standardized UX measurement methodology is a usability testing tool developed by John Brooke in 1986, known as the System Usability Scale (SUS).

Comprised of a ten-item questionnaire, with five response options for respondents (ranging from Strongly Agree to Strongly Disagree), SUS helps UX professionals gather subjective information which NPS cannot provide.

Interpreting SUS scores is a bit complex. The participant’s scores for each question are converted to a new number, added together, and then multiplied by 2.5 to convert the original scores of 0-40 to 0-100. This is then used to produce a percentile ranking. Studies show that a SUS score above 68 would be considered above average and anything below 68 is below average.

Although SUS is a very easy method to administer and can be used on small user/test-case samples; the scoring system is complex and is not diagnostic.

Method 3: PULSE Metrics

Slightly more insightful methodology employed by UX professionals and agencies. This method includes tracking of the following aspects-

• Page views

  The number of pages viewed by a single user.

• Uptime

  Percentage of time the website, app, or software is accessible to the users.

• Latency

  Amount of time it takes to transfer data from one location to another.

• Seven-day Active Users

  Number of users who interacted with the website, app or software in the last seven days.

• Earnings

  Revenue generated by the website, app or software.

Also read User Research on a Budget

While the PULSE metrics still remain more insightful than NPS or SUS methodology, it should be noted that it lacks context sometimes, thus becoming redundant for UX measurement. For example, the number of Seven Day Active Users reflects at the sticky quotient of the site, app, or software, but it doesn’t differentiate between new and returning users and doesn’t provide any insights on the users’ level of commitment.

Method 4: HEART Framework & Goals-Signals-Metrics Process

Developed by Kerry Roden, Hillary Hutchinson, and Xin Fu of the Google Research team, the HEART framework fulfilled the shortcomings of PULSE metrics.

Based on large-scale behavioural data, HEART framework stands for Happiness, Engagement, Adoption, Retention and Task Success.

The categories can be further demarcated as –

• Happiness 

  Tracks attitudinal metrics about user’s feelings towards the product. It includes subjective aspects of user experience, like satisfaction, visual appeal, likelihood to recommend, and perceived ease of use. With a general, well-designed survey, you can track the progress over time, with each modification incurred in the product.

• Engagement

  Measures the user’s level of involvement with the product. It includes metrics like the frequency, intensity, or depth of interaction over a time period. It is generally more useful to report Engagement metrics as an average per user, rather than as a total count – because an increase in the total could be a result of more users, not more usage.

• Adoption

  Indicates the number of new users who decided to use the product within a given period of time. For instance, the number of sign-ups, downloads, and purchases made.

• Retention

  Refers to the number of users who ‘adopted’ at an earlier time period are still ‘engaged’ with the product. This includes sticky quotient and retention rate measured over a time period, subscription renewal, and repurchases.

• Task Success

  Encompasses several traditional behavioral metrics of user experience, such as efficiency (e.g. time to complete a task), effectiveness (e.g. percentage of tasks completed), and error rate (e.g. percentage of failed attempts to log in).

Each of these categories has specific metrics that can be applied for an individual feature/ section or the entire product/ project. The HEART framework can be modified as per the project requirement; by adding key metrics per category, by removing irrelevant categories or by focusing on a single category.

The Goals-Signals-Metrics Process
Each metric from the HEART framework is scaled against a Goals-Signals-Metrics process using a simple dashboard. It is a simple method that keeps the entire UX team in sync through articulating the Goals of a product or feature, then identifying signals that indicate success, and finally, building-specific Metrics to track on a dashboard. Further details on the Goals-Signals-Metrics process are as below –

1. Goals

The first step is assigning goals of the product or the feature, in terms of user experience. It may be related to task success or increasing engagement, or to simply attract more traffic/users.

2. Signals

Once the goals have been determined, choose measurable signals that are sensitive and specific to the goal, as well as the method to collect and map those signals.

3. Metrics

To capture metrics, transform the signals into measurable data points, suitable for tracking overtime on a dashboard. Mean, fraction, and percentage rates are preferred over raw data for normalizing. It is also advised to filter out automated data and drawing comparisons with other competing features or products.

The combination of the HEART framework with the Goals-Signals-Metrics process provides you with precise information and analytical solutions to improvise User Experience.

Key Usability Metrics for User Experience

Some of the frequently measured User Experience KPIs are –

• Task Success Rate

  Task success rate is the percentage of correctly completed tasks by users. It reflects how effectively users are able to complete certain tasks.

• Time on Task

  Time spent on each task provides us insights into how much time is being spent on the successful completion of a pre-defined task. Prolonged processes affect UX negatively.

• User Errors

  During a given task, the number of error opportunities provides an error rate. Depending on the frequency and type of errors, we can evaluate the UX and usability of a website, app or software.

• Search vs. Navigation

  Analytic tools provide much-required information about users’ interaction with a site, software or app. Navigation vs. Search Ratio can provide information about the pages/features most frequently used.

• Form Usage

  Every website, app, or software uses a form to capture users’ information. However, from a UX point of view, it is advised to keep the forms restricted to essential fields only. Measuring the time taken to fill up a form or the number of visitors who abandon the process can provide an idea about the situation.

• Back Button Usage

  Again analytic tools can tell you how frequently the ‘back’ button has been used while navigating through your website, app, or software. A higher number indicates a broken or faulty information architecture.

• Random Visitors vs. Conversions

  While high traffic is a good thing, the conversion rate is the most important metric for any website, app, or software.

• Sticky Quotient

  Apart from conversions, the sticky quotient of a website, software, or app also marks the effectiveness of UX and usability.

• Offline vs. Online Visits

  In case you own an online as well as an offline entity, the difference between the number of visitors as well as conversions you get on both mediums should be a fair indicator of your UX and usability on digital platforms.

Implementation based on UX Metrics

Now that we are well acquainted with the commonly tracked metrics and methodology to evaluate the success of a UX strategy, let’s figure out a plan for implementation.

For a perfect evaluation of UX metrics, it is important to develop a deep understanding of define your organisation’s Key Performance Indicators (KPIs) as well as your target audience / users.

Once you have determined KPIs, link them to UX metrics to create direct relativity between business goals and customer satisfaction. This can offer some simple solutions to improve the user experience.

After you start tracking particular UX metrics in correlation to your business KPIs, make sure that you repeat calculations over a period of time to observe trends and get a better idea of what’s working and what is erroneous.

Moving further ahead, you would like to work on benchmarking the key aspects, features, and tasks to improve the overall user experience. With a fair idea of important tasks, features, and aspects of your product, you can begin an improvisation plan. Frequent iterations, A/B, and multivariate testing should reflect the context of each modification implemented in terms of UX as well as its corresponding KPI. The entire process keeps your UX strategy on track and in alignment with business goals.

It should be also noted that measuring UX is an ongoing process. Periodic UX audits should be performed to evaluate the right methods and metrics. With a solid plan of action and fair evaluation of UX strategy using the UX metrics, your products can keep the users happy and scale higher profits.

More Resources:

UX CONSISTENCY: A Universal Challenge For Enterprise Products

UX For Business Intelligence

5 Important Pain Points that Push Your SaaS App Back to the Retro Era