Quality of Experience

Quality of Experience (QoE) indicates how satisfied the user is for using the application/service. QoE is always an application-specific measure, and the actual result can vary from person to person. This section introduces how QoE can be estimated automatically and in real-time for a connected application without consulting the user.

1. QoE Basics #

QoE is typically related to applications used over network connections. This is one main difference to User Experience (UX), which is much associated with, for example, how well the user interface matches user’s expectations. Network connections bring impairments to the application traffic. However, QoE does not indicate only the quality of connection but includes the entire service. For example, in a video application, the used codec, picture resolution, compression rate, etc., are all aspects that affect how delighted the user is with the video quality. Although application configuration and settings affect QoE, when they are known, clearly, the most variation to user satisfaction comes from the network QoS, from delay, jitter, and packet loss.

2. The MOS Scale #

The most widely used measure for QoE is MOSMean Opinion Score
The most widely used measure for QoE., 1-5. The value of 5 means excellent, non-impaired satisfaction, and the value of 1 that the application is not usable at all. We use the MOS scale also in Qosium due to its widespread utilization.

3. QoE Models #

Qosium comes with QoE models that estimate user satisfaction. A scientifically proven technique, PSQAPseudo-Subjective Quality Assessment
A neural network based model for estimating QoE., can accurately estimate the quality as perceived by the application users. PSQA was originally developed for assessing multimedia applications. The model is based on a neural network technology trained with data acquired from actual user tests. Users are asked to assess a set of application samples, for example, video samples over different, pre-defined, network conditions. As PSQA uses real-time QoS statistics (e.g., delay, jitter, and packet loss) in the assessment process, careful preparation of the test sample material is essential.

Currently, Qosium has integrated models for a few VoIP codecs and an H.264 encoded video (SD, HD, and Full-HD resolutions). However, Kaitotek has experience in developing models for new applications and media codecs. Overall, the model development process has three steps:

• Create samples with different kind of impairments
• Collect a group of people, end-users, for the user test
• Create a neural network model

PSQA is not the optimal model for applications for which the end-user is not a human but, for example, a robot. Most robots are still deterministic what comes to their perception of network connection quality. The same robots perceive the application usability the same in similar conditions. Qosium also has a QoE model, which you can parameterize yourself. When you know the tolerated delay, jitter, packet loss, and/or connection break durations for your application(s), the generic model calculates you a single real-time value in the MOS scale. However, the estimation of the user experience is based purely on the settings values of yours. The generic model is known as GQoSMGeneric QoS Measure Algorithm
A parameter based QoS mapping algorithm allowing to map a single quality indicator from several parameters. When tuned with real user tests, GQoSM allows also QoE estimations..

You can argue that is this generic model QoE. By the original QoE definition, it’s definitely not. However, if you have many different applications, developing PSQA models for each of them may be too heavy. Moreover, suppose you want to measure the quality of multiple applications simultaneously, but you would rather like to know the quality of the connection than the absolute user experience without following numerous QoS statistics separately. In that case, the generic model works better as you also likely already know the QoS demands of your application(s). Although the generic model is not based on user tests, the difference in the result value to user-test-based models is observed to be only fractions in many cases.

In summary, it depends much on the use case that do you need a very accurate estimation of how satisfied users truly are, for example, for voice and video applications. Or, could a single model that suits all your applications with different parameter settings but does not provide you with that accurate estimation be still sufficient? Whatever your need is, both of these methods are supported by Qosium.