How good is my Product?

Subscribe now

Das ganze Video auf Youtube

Software and digital products require constant quality control. On the one hand, you want to compare how the product performs over several iterations. On the other hand, you want to determine whether the product is understood and accepted by users.

There are some metrics to collect statistics and measure product quality according to scientific standards. We present the most important ones and show advantages and disadvantages. Of course, there are also some examples and how we could determine that our product is good. How we define this "good" and which products we might not find so great, you can hear in the current episode of "Software for People".

What are the methods for measuring software quality?

There are three different categories of metrics to capture software quality. There are experience-based, behavior-based, and value-based metrics.

Experience-based methods

Experience-based metrics and KPIs are qualitative statements from users. For example: a survey on recommending a product is such an experience-based metric. Another possibility is a questionnaire about the user experience with the product. This can already give quite a good indication of satisfaction (Customer Satisfaction) with the product, but is often a subjective product assumption.

Users can be dissatisfied with a product but still use it because there is no alternative or they are already used to the "quirks". In our view, experience-based methods should not be chosen as the sole method, but should be combined with other methods.

Behavior-based methods

Classical statistics on product quality and user behavior can be collected in usability tests. Classically, there are metrics like the Task Success Rate, the Time on Task Rate and the Error Rate.

The task success rate is determined as follows: Users are given a task that they have to solve using the software. Now you can measure how many users complete the task. This test can be done every few interations, for example.

Among the successfully completed tasks you can also measure the time it takes to complete them. But be careful: a fast time on task rate should not always be the goal. If we assume that the UX of an online store leads to very fast and ill-considered purchases for the users, this can cause problems. Shoppers buy something they don't need and there are an excessive number of returns. Social media also manipulates users to stay on their platform as long as possible. This is a common problem because it may not add value to the user to use that platform for hours on end.

Despite the fact that the task (keeping users scrolling on a platform for a long time) is fulfilled, this does not speak for good quality in our view.

Value-based methods

The problem just described can be avoided by adding value-based methods to evaluate software quality. Here it is a question of: have I created a concrete value with my product? This value can be quite different, but there are several commonalities. The value is visible to users, the product improves their lives a bit, be it in a private or business context. For example: our new control panel makes workflows 3 times faster and reduces the error rate by 50%. That is a concrete value.

We think: a really good product has to create this value and unfortunately value-based ones are often neglected today. So: best check what value your product offers and try to work more on the concrete value for users. Then your product will be great and solve real problems!

Published by Lisa Gorelik on 2021-02-12

Sparked your interest? Discover our services!