QoE Modelling, Measurement and Prediction: A Review
---
paper_title: Quality of experience in distributed interactive multimedia environments: toward a theoretical framework

paper_content:

The past decades have witnessed a rapid growth of Distributed Interactive Multimedia Environments (DIMEs). Despite their intensity of user-involved interaction, the existing evaluation frameworks remain very much system-centric. As a step toward the human-centric paradigm, we present a conceptual framework of Quality of Experience (QoE) in DIMEs, to model, measure, and understand user experience and its relationship with the traditional Quality of Service (QoS) metrics. A multi-displinary approach is taken to build up the framework based on the theoretical results from various fields including psychology, cognitive sciences, sociology, and information technology. We introduce a mapping methodology to quantify the correlations between QoS and QoE, and describe our controlled and uncontrolled studies as illustrating examples. The results present the first deep study to model the multi-facet QoE construct, map the QoS-QoE relationship, and capture the human-centric quality modalities in the context of DIMEs.
---
paper_title: Does context matter in quality evaluation of mobile television

paper_content:

Subjective quality evaluation is used to optimize the produced audiovisual quality from fundamental signal processing algorithms to consumer services. These studies typically follow the basic principles of controlled psychoperceptual experiments. However, when compromising compression and transmission parameters for consumer services, the ecological validity of conventional quality evaluation methods can be questioned. To tackle this, we firstly present a novel user-oriented quality evaluation method for mobile television in its usage contexts. Secondly, we present the results of an experiment conducted with 30 participants comparing acceptability and satisfaction of quality as well as goals of viewing in three mobile contexts and under four different residual transmission error rates, when the participants also performed simultaneous assessment tasks. Finally, we compare the results with a previous laboratory experiment. The studied error rates impacted negatively on all measured tasks with some contextual differences. Moreover, the evaluations were more favorable and less discriminate in the mobile contexts compared to the laboratory.
---
paper_title: The QoE Evaluation Method through the QoS-QoE Correlation Model

paper_content:

Currently, the satisfaction of user is becoming one of the most important topics concerned by the service providers. So we propose the approach method for the objective QoE measurement through the QoS parameters. The measurements and provisioning of the quality of service (QoS) are generally defined in terms of network delivery capacity and resource availability, not in terms of satisfaction to the end-user. The fundamental assumption behind such traditional provisioning is that the measured quality of service is closely related to the quality of experience (QoE) for the end-user. In the paper, we describe the QoS and QoE correlation model, and the QoE evaluation method using QoS parameter in the converged network environment is studied.
---
paper_title: A taxonomy of quality of service and Quality of Experience of multimodal human-machine interaction

paper_content:

Quality of Service (QoS) and Quality of Experience (QoE) are not only important for services transmitting multimedia data, but also for services involving multimodal human-machine interaction. In order to guide the assessment and evaluation of such services, we developed a taxonomy of the most relevant QoS and QoE aspects which result from multimodal human-machine interactions. It consists of three layers: (1) The QoS-influencing factors related to the user, the system, and the context of use; (2) the QoS interaction performance aspects describing user and system behavior and performance; and (3) the QoE aspects related to the quality perception and judgment processes taking place inside the user. For each of these layers, we provide metrics which make system evaluation more systematic and comparable.
---
paper_title: QoE: Quality of Experience: A Conceptual Essay

paper_content:

The ICT industry and services are on the verge of substituting quality of service and utility functions with quality of experience, QoE. While intuitive, QoE, has eluded the research community. QoE takes into account what the user is experiencing with new products and services. Once this information is collected, it becomes an input in the development of the new product or new service. But how does one measure directly quality of experience? In this conceptual paper, I discuss from an ICT perspective, the utility function and quality of service, QoS. I also present how active interested the industry is in QoE. I concluded with some key points that may serve as a guideline in the process of constructing the QoE function.
---
paper_title: A decision-theoretic approach for quality-of-experience measurement and prediction

paper_content:

This paper presents a pioneering context-aware approach for quality of experience (QoE) measurement and prediction. The proposed approach incorporates an intuitive context-aware framework and decision theory. It is capable of incorporating several QoE related classes and context information to correctly measure and predict the overall QoE on a single scale. Our approach can be used in measuring and predicting QoE in both lab and living-lab settings based on user, device and network related context parameters. The predicted QoE can be beneficial for network operators to minimize network churn and can help application developers to build smart user-centric applications. We perform extensive experimentation and the results validate our approach.
---
paper_title: Quality of Experience in Communications Ecosystem

paper_content:

Communications ecosystem covers a huge area from technical issues to business models and human behaviour. Due to this extreme diversity various societies need to discuss with each other, each of them using their own language. Engineers talk about network performance and quality of service, business people talk about average revenue per user and customer churn while behavioural scientists talk about happiness and experiences. Thus, everyone who wants to understand, or even analyze, the whole ecosystem, has to deal with all these diverse issues. In addition to the apparent communication problems, the main challenges of ecosystem analysis are to realistically model human behaviour, and to efficiently combine the models developed for different domains. A central concept when solving these problems is quality of experience (QoE). This paper sheds light on the role of QoE by means of a common framework that covers the whole communications ecosystem. Additionally, a research agenda for a holistic ecosystem analysis is outlined.
---
paper_title: User measures of quality of experience: why being objective and quantitative is important

paper_content:

This article develops the concept of quality of user experience by focusing on requirements for its measurement and communication for use in industry. Whereas more typical work involves either modeling media quality based on technical measures or user subjective assessment of media quality, it is argued that QoE measurement should include objective measures of the process and outcomes of usage. Also, subjective measures should be collected as quantitative data rather than qualitative data as is currently common, because quantitative data enables combined analyses of objective and subjective measures for deriving global QoE ratings. Global ratings are important for optimizing the communicability of results to target audiences. Examples of measures and scales are provided. A structured approach to defining and measuring QoE in relation to QoS is explained and illustrated.
---
paper_title: A probabilistic context-aware approach for quality of experience measurement in pervasive systems

paper_content:

In this paper, we develop a novel context-aware approach for quality of experience (QoE) modeling, reasoning and inferencing in mobile and pervasive computing environments. The proposed model is based upon a state-space approach and Bayesian networks for QoE modeling and reasoning. We further extend this context model to incorporate influence diagrams for efficient QoE inferencing. Our approach accommodates user, device and quality of service (QoS) related context parameters to determine the overall QoE of the user. This helps in user-related media, network and device adaptation. We perform experimentation to validate the proposed approach and the results verify its modeling and inferencing capabilities.
---
paper_title: Football on mobile phones : algorithms, architectures and quality of experience in streaming video

paper_content:

In this thesis we study algorithms and architectures that can provide a better Quality of Experience (QoE) for streaming video systems and services. With cases and examples taken from the applicati ...
---
paper_title: Context-Aware QoE Modelling, Measurement, and Prediction in Mobile Computing Systems

paper_content:

Quality of Experience (QoE) as an aggregate of Quality of Service (QoS) and human user-related metrics will be the key success factor for current and future mobile computing systems. QoE measurement and prediction are complex tasks as they may involve a large parameter space such as location, delay, jitter, packet loss, and user satisfaction just to name a few. These tasks necessitate the development of practical context-aware QoE models that efficiently determine relationships between user context and QoE parameters. In this paper, we propose, develop, and validate a novel decision-theoretic approach called CaQoEM for QoE modelling, measurement, and prediction. We address the challenge of QoE measurement and prediction where each QoE parameter can be measured on a different scale and may involve different units of measurement. CaQoEM is context-aware and uses Bayesian networks and utility theory to measure and predict users’ QoE under uncertainty. We validate CaQoEM using extensive experimentation, user studies and simulations. The results soundly demonstrate that CaQoEM correctly measures range-defined QoE using a bipolar scale. For QoE prediction, an overall accuracy of 98.93% was achieved using 10-fold cross validation in multiple diverse network conditions such as vertical handoffs, wireless signal fading and wireless network congestion.
---
paper_title: Quality of experience in distributed interactive multimedia environments: toward a theoretical framework

paper_content:

The past decades have witnessed a rapid growth of Distributed Interactive Multimedia Environments (DIMEs). Despite their intensity of user-involved interaction, the existing evaluation frameworks remain very much system-centric. As a step toward the human-centric paradigm, we present a conceptual framework of Quality of Experience (QoE) in DIMEs, to model, measure, and understand user experience and its relationship with the traditional Quality of Service (QoS) metrics. A multi-displinary approach is taken to build up the framework based on the theoretical results from various fields including psychology, cognitive sciences, sociology, and information technology. We introduce a mapping methodology to quantify the correlations between QoS and QoE, and describe our controlled and uncontrolled studies as illustrating examples. The results present the first deep study to model the multi-facet QoE construct, map the QoS-QoE relationship, and capture the human-centric quality modalities in the context of DIMEs.
---
paper_title: Speech Quality While Roaming in Next Generation Networks

paper_content:

In NGNs, handovers between different wireless access technologies provide seamless roaming during voice calls. The resulting speech quality depends on the audio bandwidth of the speech codecs used in the respective networks, as well as on degradations resulting from the handover, coding, and packet loss. We present the results of four listening experiments where speech quality is quantified as a function of network and codec characteristics, and compare them to estimations obtained from instrumental models. The results show when and under which circumstances a network handover and/or codec changeover should be scheduled in order to obtain better speech quality. This is important for the development of high-quality roaming strategies.
---
paper_title: Proposed framework for evaluating quality of experience in a mobile, testbed-oriented living lab setting

paper_content:

The framework presented in this paper enables the evaluation of Quality of Experience (QoE) in a mobile, testbed-oriented Living Lab setting. As a result, it fits within the shift towards more user-centric approaches in innovation research and aims to bridge the gap between technical parameters and human experience factors. In view of this, Quality of Experience is seen as a multi-dimensional concept, which should be considered from an interdisciplinary perspective. Although several approaches for evaluating perceived QoE have been proposed in the past, they tend to focus on a limited number of objective dimensions and fail to grasp the subjective counterparts of users' experiences. We therefore propose a distributed architecture for monitoring network Quality of Service (QoS), context information and subjective user experience based on the functional requirements related to real-time experience measurements in real-life settings. This approach allows us to evaluate all relevant QoE-dimensions in a mobile context.
---
paper_title: A decision-theoretic approach for quality-of-experience measurement and prediction

paper_content:

This paper presents a pioneering context-aware approach for quality of experience (QoE) measurement and prediction. The proposed approach incorporates an intuitive context-aware framework and decision theory. It is capable of incorporating several QoE related classes and context information to correctly measure and predict the overall QoE on a single scale. Our approach can be used in measuring and predicting QoE in both lab and living-lab settings based on user, device and network related context parameters. The predicted QoE can be beneficial for network operators to minimize network churn and can help application developers to build smart user-centric applications. We perform extensive experimentation and the results validate our approach.
---
paper_title: Modeling subjective tests of quality of experience with a Generalized Linear Model

paper_content:

Subjective tests of telecommunication services engaging a panel of subjects aim to analyze how service users experience their quality (QoE) with varying parameters of both a network and/or application itself. Subjects express their opinion on a few-grade categorical scale during tests performed in a controlled environment. The proposed Generalized Linear Model (GLZ)1 relates probabilities of particular QoE levels to independent network and application explanatory variables in a statistically credible way. The GLZ model is executed in several steps including maximum likelihood estimation of model parameters, selection of the most effective model with the Bayesian information criterion, model validation with a chi-squared test, and last but not least confidence interval estimation. Monitoring a QoE level with a GLZ model provides to operators an effective tool towards a mid- and long-term adaptation of both network and/or application parameters in order to follow users' experience and expectations.
---
paper_title: Utility curves: mean opinion scores considered biased

paper_content:

In the Coqos project task performance measures and a corresponding framework are suggested and pursued as a novel and more suitable means for determining utility curves. TPM are intended to avoid limits inherent in traditional measures like mean opinion scores. MOS rely merely on subjective ratings rather than on more objective performance in relation to a particular task or application of interest. Informational relevance and its impact on subjects can be measured more effectively by TPM. Inhibiting psychological and cognitive effects like consciousness or nonconsciousness of degradations or individual focusing and perspectives of subjects can be more appropriately evaluated and dealt with by means of TPM. The increasing importance of adaptation, in particular with the advance of MPEG4, as a means for QoS provisioning, both in wireless and wired environments, requires sensible techniques to effectively determine utility curves.
---
paper_title: Perceptual QoS assessment technologies for VoIP

paper_content:

Since quality is not generally guaranteed in an IP network, the proper design and management of networks and/or terminals for high-quality voice over IP services and maintenance of service levels is important. In terms of quality design and management, methodologies for appropriately and effectively evaluating the perceptual QoS of VoIP are indispensable. This article gives an overview of the state of the art of quality assessment technologies for VoIP, including recent work on improving their accuracy.
---
paper_title: Multiple Attribute Decision Making: Methods and Applications

paper_content:

Decision makers are often faced with several conflicting alternatives [1]. How do they evaluate trade-offs when there are more than three criteria? To help people make optimal decisions, scholars in the discipline of multiple criteria decision making (MCDM) continue to develop new methods for structuring preferences and determining the correct relative weights for criteria. A compilation of modern decision-making techniques, Multiple Attribute Decision Making: Methods and Applications focuses on the fuzzy set approach to multiple attribute decision making (MADM). Drawing on their experience, the authors bring together current methods and real-life applications of MADM techniques for decision analysis. They also propose a novel hybrid MADM model that combines DEMATEL and analytic network process (ANP) with VIKOR procedures.
---
paper_title: User measures of quality of experience: why being objective and quantitative is important

paper_content:

This article develops the concept of quality of user experience by focusing on requirements for its measurement and communication for use in industry. Whereas more typical work involves either modeling media quality based on technical measures or user subjective assessment of media quality, it is argued that QoE measurement should include objective measures of the process and outcomes of usage. Also, subjective measures should be collected as quantitative data rather than qualitative data as is currently common, because quantitative data enables combined analyses of objective and subjective measures for deriving global QoE ratings. Global ratings are important for optimizing the communicability of results to target audiences. Examples of measures and scales are provided. A structured approach to defining and measuring QoE in relation to QoS is explained and illustrated.
---
paper_title: Quantifying Skype user satisfaction

paper_content:

The success of Skype has inspired a generation of peer-to-peer-based solutions for satisfactory real-time multimedia services over the Internet. However, fundamental questions, such as whether VoIP services like Skype are good enough in terms of user satisfaction,have not been formally addressed. One of the major challenges lies in the lack of an easily accessible and objective index to quantify the degree of user satisfaction.In this work, we propose a model, geared to Skype, but generalizable to other VoIP services, to quantify VoIP user satisfaction based on a rigorous analysis of the call duration from actual Skype traces. The User Satisfaction Index (USI) derived from the model is unique in that 1) it is composed by objective source-and network-level metrics, such as the bit rate, bit rate jitter, and round-trip time, 2) unlike speech quality measures based on voice signals, such as the PESQ model standardized by ITU-T, the metrics are easily accessible and computable for real-time adaptation, and 3) the model development only requires network measurements, i.e., no user surveys or voice signals are necessary. Our model is validated by an independent set of metrics that quantifies the degree of user interaction from the actual traces.
---
paper_title: Evaluating Users ’ Satisfaction in Packet Networks Using Random Neural Networks

paper_content:

Quantifying the quality of a video or audio transmission over the Internet is usually a hard task, as based on the statistical processing of the evaluations made by a panel of humans (the corresponding and standardized area is called subjective testing). In this paper we describe a methodology called Pseudo-Subjective Quality Assessment (PSQA), based on Random Neural Networks, which is able to perform this task automatically, accurately and efficiently. RNN had been chosen here because of their good performances over other possibilities; this is discussed in the paper. Some new insights on PSQA’s use and performance are also given. In particular we discuss new results concerning PSQA–based dynamic quality control, and conversational quality assessment.
---
paper_title: A generic quantitative relationship between quality of experience and quality of service

paper_content:

Quality of experience ties together user perception, experience, and expectations to application and network performance, typically expressed by quality of service parameters. Quantitative relationships between QoE and QoS are required in order to be able to build effective QoE control mechanisms onto measurable QoS parameters. Against this background, this article proposes a generic formula in which QoE and QoS parameters are connected through an exponential relationship, called IQX hypothesis. The formula relates changes of QoE with respect to QoS to the current level of QoE, is simple to match, and its limit behaviors are straightforward to interpret. It validates the IQX hypothesis for streaming services, where QoE in terms of Mean Opinion Scores is expressed as functions of loss and reordering ratio, the latter of which is caused by jitter. For web surfing as the second application area, matchings provided by the IQX hypothesis are shown to outperform previously published logarithmic functions. We conclude that the IQX hypothesis is a strong candidate to be taken into account when deriving relationships between QoE and QoS parameters.
---
paper_title: Perceptual QoS assessment technologies for VoIP

paper_content:

Since quality is not generally guaranteed in an IP network, the proper design and management of networks and/or terminals for high-quality voice over IP services and maintenance of service levels is important. In terms of quality design and management, methodologies for appropriately and effectively evaluating the perceptual QoS of VoIP are indispensable. This article gives an overview of the state of the art of quality assessment technologies for VoIP, including recent work on improving their accuracy.
---
paper_title: User measures of quality of experience: why being objective and quantitative is important

paper_content:

This article develops the concept of quality of user experience by focusing on requirements for its measurement and communication for use in industry. Whereas more typical work involves either modeling media quality based on technical measures or user subjective assessment of media quality, it is argued that QoE measurement should include objective measures of the process and outcomes of usage. Also, subjective measures should be collected as quantitative data rather than qualitative data as is currently common, because quantitative data enables combined analyses of objective and subjective measures for deriving global QoE ratings. Global ratings are important for optimizing the communicability of results to target audiences. Examples of measures and scales are provided. A structured approach to defining and measuring QoE in relation to QoS is explained and illustrated.
---
paper_title: Does context matter in quality evaluation of mobile television

paper_content:

Subjective quality evaluation is used to optimize the produced audiovisual quality from fundamental signal processing algorithms to consumer services. These studies typically follow the basic principles of controlled psychoperceptual experiments. However, when compromising compression and transmission parameters for consumer services, the ecological validity of conventional quality evaluation methods can be questioned. To tackle this, we firstly present a novel user-oriented quality evaluation method for mobile television in its usage contexts. Secondly, we present the results of an experiment conducted with 30 participants comparing acceptability and satisfaction of quality as well as goals of viewing in three mobile contexts and under four different residual transmission error rates, when the participants also performed simultaneous assessment tasks. Finally, we compare the results with a previous laboratory experiment. The studied error rates impacted negatively on all measured tasks with some contextual differences. Moreover, the evaluations were more favorable and less discriminate in the mobile contexts compared to the laboratory.
---
paper_title: Proposed framework for evaluating quality of experience in a mobile, testbed-oriented living lab setting

paper_content:

The framework presented in this paper enables the evaluation of Quality of Experience (QoE) in a mobile, testbed-oriented Living Lab setting. As a result, it fits within the shift towards more user-centric approaches in innovation research and aims to bridge the gap between technical parameters and human experience factors. In view of this, Quality of Experience is seen as a multi-dimensional concept, which should be considered from an interdisciplinary perspective. Although several approaches for evaluating perceived QoE have been proposed in the past, they tend to focus on a limited number of objective dimensions and fail to grasp the subjective counterparts of users' experiences. We therefore propose a distributed architecture for monitoring network Quality of Service (QoS), context information and subjective user experience based on the functional requirements related to real-time experience measurements in real-life settings. This approach allows us to evaluate all relevant QoE-dimensions in a mobile context.
---
paper_title: Quantifying Skype user satisfaction

paper_content:

The success of Skype has inspired a generation of peer-to-peer-based solutions for satisfactory real-time multimedia services over the Internet. However, fundamental questions, such as whether VoIP services like Skype are good enough in terms of user satisfaction,have not been formally addressed. One of the major challenges lies in the lack of an easily accessible and objective index to quantify the degree of user satisfaction.In this work, we propose a model, geared to Skype, but generalizable to other VoIP services, to quantify VoIP user satisfaction based on a rigorous analysis of the call duration from actual Skype traces. The User Satisfaction Index (USI) derived from the model is unique in that 1) it is composed by objective source-and network-level metrics, such as the bit rate, bit rate jitter, and round-trip time, 2) unlike speech quality measures based on voice signals, such as the PESQ model standardized by ITU-T, the metrics are easily accessible and computable for real-time adaptation, and 3) the model development only requires network measurements, i.e., no user surveys or voice signals are necessary. Our model is validated by an independent set of metrics that quantifies the degree of user interaction from the actual traces.
---
paper_title: Dynamic Bayesian Networks for Sequential Quality of Experience Modelling and Measurement

paper_content:

This paper presents a novel context-aware methodology for modelling and measuring user-perceived quality of experience (QoE) over time. In particular, we create a context-aware model for QoE modelling and measurement using dynamic Bayesian networks (DBN) and a context-aware state-space approach. The proposed model is then used to infer and determine users' QoE in a sequential manner. We performed experimentation to validate the proposed model. The results prove that it can efficiently model, reason and measure QoE of the users'.
---
paper_title: A decision-theoretic approach for quality-of-experience measurement and prediction

paper_content:

This paper presents a pioneering context-aware approach for quality of experience (QoE) measurement and prediction. The proposed approach incorporates an intuitive context-aware framework and decision theory. It is capable of incorporating several QoE related classes and context information to correctly measure and predict the overall QoE on a single scale. Our approach can be used in measuring and predicting QoE in both lab and living-lab settings based on user, device and network related context parameters. The predicted QoE can be beneficial for network operators to minimize network churn and can help application developers to build smart user-centric applications. We perform extensive experimentation and the results validate our approach.
---
paper_title: OneClick: A Framework for Measuring Network Quality of Experience

paper_content:

As the service requirements of network applications shift from high throughput to high media quality, interactivity, and responsiveness, the definition of QoE (Quality of Experience) has become multidimensional. Although it may not be difficult to measure individual dimensions of the QoE, how to capture users' overall perceptions when they are using network applications remains an open question. In this paper, we propose a framework called OneClick to capture users' perceptions when they are using network applications. The framework only requires a subject to click a dedicated key whenever he/she feels dissatisfied with the quality of the application in use. OneClick is particularly effective because it is intuitive, lightweight, efficient, time-aware, and application-independent. We use two objective quality assessment methods, PESQ and VQM, to validate OneClick's ability to evaluate the quality of audio and video clips. To demonstrate the proposed framework's efficiency and effectiveness in assessing user experiences, we implement it on two applications, one for instant messaging applications, and the other for first- person shooter games. A Flash implementation of the proposed framework is also presented.
---
paper_title: Evaluating Users ’ Satisfaction in Packet Networks Using Random Neural Networks

paper_content:

Quantifying the quality of a video or audio transmission over the Internet is usually a hard task, as based on the statistical processing of the evaluations made by a panel of humans (the corresponding and standardized area is called subjective testing). In this paper we describe a methodology called Pseudo-Subjective Quality Assessment (PSQA), based on Random Neural Networks, which is able to perform this task automatically, accurately and efficiently. RNN had been chosen here because of their good performances over other possibilities; this is discussed in the paper. Some new insights on PSQA’s use and performance are also given. In particular we discuss new results concerning PSQA–based dynamic quality control, and conversational quality assessment.
---
paper_title: A generic quantitative relationship between quality of experience and quality of service

paper_content:

Quality of experience ties together user perception, experience, and expectations to application and network performance, typically expressed by quality of service parameters. Quantitative relationships between QoE and QoS are required in order to be able to build effective QoE control mechanisms onto measurable QoS parameters. Against this background, this article proposes a generic formula in which QoE and QoS parameters are connected through an exponential relationship, called IQX hypothesis. The formula relates changes of QoE with respect to QoS to the current level of QoE, is simple to match, and its limit behaviors are straightforward to interpret. It validates the IQX hypothesis for streaming services, where QoE in terms of Mean Opinion Scores is expressed as functions of loss and reordering ratio, the latter of which is caused by jitter. For web surfing as the second application area, matchings provided by the IQX hypothesis are shown to outperform previously published logarithmic functions. We conclude that the IQX hypothesis is a strong candidate to be taken into account when deriving relationships between QoE and QoS parameters.
---
paper_title: User measures of quality of experience: why being objective and quantitative is important

paper_content:

This article develops the concept of quality of user experience by focusing on requirements for its measurement and communication for use in industry. Whereas more typical work involves either modeling media quality based on technical measures or user subjective assessment of media quality, it is argued that QoE measurement should include objective measures of the process and outcomes of usage. Also, subjective measures should be collected as quantitative data rather than qualitative data as is currently common, because quantitative data enables combined analyses of objective and subjective measures for deriving global QoE ratings. Global ratings are important for optimizing the communicability of results to target audiences. Examples of measures and scales are provided. A structured approach to defining and measuring QoE in relation to QoS is explained and illustrated.
---
paper_title: A probabilistic context-aware approach for quality of experience measurement in pervasive systems

paper_content:

In this paper, we develop a novel context-aware approach for quality of experience (QoE) modeling, reasoning and inferencing in mobile and pervasive computing environments. The proposed model is based upon a state-space approach and Bayesian networks for QoE modeling and reasoning. We further extend this context model to incorporate influence diagrams for efficient QoE inferencing. Our approach accommodates user, device and quality of service (QoS) related context parameters to determine the overall QoE of the user. This helps in user-related media, network and device adaptation. We perform experimentation to validate the proposed approach and the results verify its modeling and inferencing capabilities.
---
paper_title: A taxonomy of quality of service and Quality of Experience of multimodal human-machine interaction

paper_content:

Quality of Service (QoS) and Quality of Experience (QoE) are not only important for services transmitting multimedia data, but also for services involving multimodal human-machine interaction. In order to guide the assessment and evaluation of such services, we developed a taxonomy of the most relevant QoS and QoE aspects which result from multimodal human-machine interactions. It consists of three layers: (1) The QoS-influencing factors related to the user, the system, and the context of use; (2) the QoS interaction performance aspects describing user and system behavior and performance; and (3) the QoE aspects related to the quality perception and judgment processes taking place inside the user. For each of these layers, we provide metrics which make system evaluation more systematic and comparable.
---
paper_title: Dynamic Bayesian Networks for Sequential Quality of Experience Modelling and Measurement

paper_content:

This paper presents a novel context-aware methodology for modelling and measuring user-perceived quality of experience (QoE) over time. In particular, we create a context-aware model for QoE modelling and measurement using dynamic Bayesian networks (DBN) and a context-aware state-space approach. The proposed model is then used to infer and determine users' QoE in a sequential manner. We performed experimentation to validate the proposed model. The results prove that it can efficiently model, reason and measure QoE of the users'.
---
paper_title: A decision-theoretic approach for quality-of-experience measurement and prediction

paper_content:

This paper presents a pioneering context-aware approach for quality of experience (QoE) measurement and prediction. The proposed approach incorporates an intuitive context-aware framework and decision theory. It is capable of incorporating several QoE related classes and context information to correctly measure and predict the overall QoE on a single scale. Our approach can be used in measuring and predicting QoE in both lab and living-lab settings based on user, device and network related context parameters. The predicted QoE can be beneficial for network operators to minimize network churn and can help application developers to build smart user-centric applications. We perform extensive experimentation and the results validate our approach.
---
paper_title: User measures of quality of experience: why being objective and quantitative is important

paper_content:

This article develops the concept of quality of user experience by focusing on requirements for its measurement and communication for use in industry. Whereas more typical work involves either modeling media quality based on technical measures or user subjective assessment of media quality, it is argued that QoE measurement should include objective measures of the process and outcomes of usage. Also, subjective measures should be collected as quantitative data rather than qualitative data as is currently common, because quantitative data enables combined analyses of objective and subjective measures for deriving global QoE ratings. Global ratings are important for optimizing the communicability of results to target audiences. Examples of measures and scales are provided. A structured approach to defining and measuring QoE in relation to QoS is explained and illustrated.
---
paper_title: A model for measuring Quality of Experience

paper_content:

In this paper we present a model for measuring the Quality of Experience (QoE) of multimedia services. Our model consists of measurable and non-measurable parameters and describes a framework for quantifying the model parameters by transformation into an XML document with discreet values. The model has been validated by running a field trial over 4 weeks using 3G mobile phones for a video on demand service and mobile TV.
---
paper_title: A probabilistic context-aware approach for quality of experience measurement in pervasive systems

paper_content:

In this paper, we develop a novel context-aware approach for quality of experience (QoE) modeling, reasoning and inferencing in mobile and pervasive computing environments. The proposed model is based upon a state-space approach and Bayesian networks for QoE modeling and reasoning. We further extend this context model to incorporate influence diagrams for efficient QoE inferencing. Our approach accommodates user, device and quality of service (QoS) related context parameters to determine the overall QoE of the user. This helps in user-related media, network and device adaptation. We perform experimentation to validate the proposed approach and the results verify its modeling and inferencing capabilities.
---
paper_title: Quality of experience in distributed interactive multimedia environments: toward a theoretical framework

paper_content:

The past decades have witnessed a rapid growth of Distributed Interactive Multimedia Environments (DIMEs). Despite their intensity of user-involved interaction, the existing evaluation frameworks remain very much system-centric. As a step toward the human-centric paradigm, we present a conceptual framework of Quality of Experience (QoE) in DIMEs, to model, measure, and understand user experience and its relationship with the traditional Quality of Service (QoS) metrics. A multi-displinary approach is taken to build up the framework based on the theoretical results from various fields including psychology, cognitive sciences, sociology, and information technology. We introduce a mapping methodology to quantify the correlations between QoS and QoE, and describe our controlled and uncontrolled studies as illustrating examples. The results present the first deep study to model the multi-facet QoE construct, map the QoS-QoE relationship, and capture the human-centric quality modalities in the context of DIMEs.
---
paper_title: Towards a theory of context spaces

paper_content:

We propose initial work on a conceptual framework for context-aware systems, towards a general context model to aid thinking, describing, manipulating and utilizing context. Much work on context-aware computing have utilized context in specialized applications, with application-specific software (and perhaps hardware) and specialized context representation and manipulation. Context-aware systems are becoming increasingly important, and emerging research has begun to look at context-aware systems more generally, independently of specific applications, including context middleware and toolkits and ontologies for describing context.
---
paper_title: Model-Based Approach to Measuring Quality of Experience

paper_content:

Quantifiable quality measures are necessary for VoIP service providers in order to offer services and be sure to keep users for their services. Existing methods used for evaluating the Quality of Experience (QoE) mostly rely on user survey and scores from the user, which are too subjective and need much processing time and cost. This paper proposes a pentagram model for measuring QoE. The model consists of five factors, including integrality, retainability, availability, usability, and instantaneousness. The experiments show that the proposed model offers better performance in terms of quality measurement than the conventional method.
---
paper_title: Toward total quality of experience: A QoE model in a communication ecosystem

paper_content:

In recent years, the quality of experience notion has become a major research theme within the telecommunications community. QoE is an assessment of the human experience when interacting with technology and business entities in a particular context. A communication ecosystem encompasses various domains such as technical aspects, business models, human behavior, and context. For each aspect of a communication ecosystem, various models have been developed. However, few models have been designed to integrate all aspects of a communication ecosystem to understand human behavioral needs in a detailed and structured way. While existing models have produced the basic sketch of QoE modeling, more concepts and interdomain mapping are to be incorporated in order to have a clear picture of QoE in communication ecosystems. The aim of the current work is to build on the existing research being conducted in disparate disciplines about human behavior in order to provide a high-level model that can be adapted to many specific contexts and to encourage future research which examines these crossdomain relationships.
---
paper_title: A decision-theoretic approach for quality-of-experience measurement and prediction

paper_content:

This paper presents a pioneering context-aware approach for quality of experience (QoE) measurement and prediction. The proposed approach incorporates an intuitive context-aware framework and decision theory. It is capable of incorporating several QoE related classes and context information to correctly measure and predict the overall QoE on a single scale. Our approach can be used in measuring and predicting QoE in both lab and living-lab settings based on user, device and network related context parameters. The predicted QoE can be beneficial for network operators to minimize network churn and can help application developers to build smart user-centric applications. We perform extensive experimentation and the results validate our approach.
---
paper_title: Analysing user experience of personal mobile products through contextual factors

paper_content:

Context plays a critical role in the user experience (UX) of mobile products. In UX studies, context is often seen as a single entity such as mobile context or car context. More fine-grained categorization of contextual factors is needed when we want to understand how context affects user experience. The purpose of our study was to adapt existing context and experience categorizations, and try out in practice how well these are suited for analyzing the user experience of mobile products. We conducted a ten-day field study where 21 participants reported their experiences of personal mobile products by writing experience reports. The reports were analyzed by using eight context categories to identify the contextual factors. Among these factors we determined the triggering context, which has the major influence on the most meaningful user experience described in a report. Our results show that having a detailed categorization for analyzing contextual factors is valuable for understanding how context affects user experience. This becomes practical in identifying the most meaningful user experiences among the reported ones. We propose that triggering context analysis should become a part of user experience studies of mobile products.
---
paper_title: A model for measuring Quality of Experience

paper_content:

In this paper we present a model for measuring the Quality of Experience (QoE) of multimedia services. Our model consists of measurable and non-measurable parameters and describes a framework for quantifying the model parameters by transformation into an XML document with discreet values. The model has been validated by running a field trial over 4 weeks using 3G mobile phones for a video on demand service and mobile TV.
---
paper_title: A probabilistic context-aware approach for quality of experience measurement in pervasive systems

paper_content:

In this paper, we develop a novel context-aware approach for quality of experience (QoE) modeling, reasoning and inferencing in mobile and pervasive computing environments. The proposed model is based upon a state-space approach and Bayesian networks for QoE modeling and reasoning. We further extend this context model to incorporate influence diagrams for efficient QoE inferencing. Our approach accommodates user, device and quality of service (QoS) related context parameters to determine the overall QoE of the user. This helps in user-related media, network and device adaptation. We perform experimentation to validate the proposed approach and the results verify its modeling and inferencing capabilities.
---
paper_title: Football on mobile phones : algorithms, architectures and quality of experience in streaming video

paper_content:

In this thesis we study algorithms and architectures that can provide a better Quality of Experience (QoE) for streaming video systems and services. With cases and examples taken from the applicati ...
---
paper_title: Context-Aware QoE Modelling, Measurement, and Prediction in Mobile Computing Systems

paper_content:

Quality of Experience (QoE) as an aggregate of Quality of Service (QoS) and human user-related metrics will be the key success factor for current and future mobile computing systems. QoE measurement and prediction are complex tasks as they may involve a large parameter space such as location, delay, jitter, packet loss, and user satisfaction just to name a few. These tasks necessitate the development of practical context-aware QoE models that efficiently determine relationships between user context and QoE parameters. In this paper, we propose, develop, and validate a novel decision-theoretic approach called CaQoEM for QoE modelling, measurement, and prediction. We address the challenge of QoE measurement and prediction where each QoE parameter can be measured on a different scale and may involve different units of measurement. CaQoEM is context-aware and uses Bayesian networks and utility theory to measure and predict users’ QoE under uncertainty. We validate CaQoEM using extensive experimentation, user studies and simulations. The results soundly demonstrate that CaQoEM correctly measures range-defined QoE using a bipolar scale. For QoE prediction, an overall accuracy of 98.93% was achieved using 10-fold cross validation in multiple diverse network conditions such as vertical handoffs, wireless signal fading and wireless network congestion.
---
paper_title: Quality of experience in distributed interactive multimedia environments: toward a theoretical framework

paper_content:

The past decades have witnessed a rapid growth of Distributed Interactive Multimedia Environments (DIMEs). Despite their intensity of user-involved interaction, the existing evaluation frameworks remain very much system-centric. As a step toward the human-centric paradigm, we present a conceptual framework of Quality of Experience (QoE) in DIMEs, to model, measure, and understand user experience and its relationship with the traditional Quality of Service (QoS) metrics. A multi-displinary approach is taken to build up the framework based on the theoretical results from various fields including psychology, cognitive sciences, sociology, and information technology. We introduce a mapping methodology to quantify the correlations between QoS and QoE, and describe our controlled and uncontrolled studies as illustrating examples. The results present the first deep study to model the multi-facet QoE construct, map the QoS-QoE relationship, and capture the human-centric quality modalities in the context of DIMEs.
---
paper_title: Speech Quality While Roaming in Next Generation Networks

paper_content:

In NGNs, handovers between different wireless access technologies provide seamless roaming during voice calls. The resulting speech quality depends on the audio bandwidth of the speech codecs used in the respective networks, as well as on degradations resulting from the handover, coding, and packet loss. We present the results of four listening experiments where speech quality is quantified as a function of network and codec characteristics, and compare them to estimations obtained from instrumental models. The results show when and under which circumstances a network handover and/or codec changeover should be scheduled in order to obtain better speech quality. This is important for the development of high-quality roaming strategies.
---
paper_title: The QoE Evaluation Method through the QoS-QoE Correlation Model

paper_content:

Currently, the satisfaction of user is becoming one of the most important topics concerned by the service providers. So we propose the approach method for the objective QoE measurement through the QoS parameters. The measurements and provisioning of the quality of service (QoS) are generally defined in terms of network delivery capacity and resource availability, not in terms of satisfaction to the end-user. The fundamental assumption behind such traditional provisioning is that the measured quality of service is closely related to the quality of experience (QoE) for the end-user. In the paper, we describe the QoS and QoE correlation model, and the QoE evaluation method using QoS parameter in the converged network environment is studied.
---
paper_title: Quantifying Skype user satisfaction

paper_content:

The success of Skype has inspired a generation of peer-to-peer-based solutions for satisfactory real-time multimedia services over the Internet. However, fundamental questions, such as whether VoIP services like Skype are good enough in terms of user satisfaction,have not been formally addressed. One of the major challenges lies in the lack of an easily accessible and objective index to quantify the degree of user satisfaction.In this work, we propose a model, geared to Skype, but generalizable to other VoIP services, to quantify VoIP user satisfaction based on a rigorous analysis of the call duration from actual Skype traces. The User Satisfaction Index (USI) derived from the model is unique in that 1) it is composed by objective source-and network-level metrics, such as the bit rate, bit rate jitter, and round-trip time, 2) unlike speech quality measures based on voice signals, such as the PESQ model standardized by ITU-T, the metrics are easily accessible and computable for real-time adaptation, and 3) the model development only requires network measurements, i.e., no user surveys or voice signals are necessary. Our model is validated by an independent set of metrics that quantifies the degree of user interaction from the actual traces.
---
paper_title: Dynamic Bayesian Networks for Sequential Quality of Experience Modelling and Measurement

paper_content:

This paper presents a novel context-aware methodology for modelling and measuring user-perceived quality of experience (QoE) over time. In particular, we create a context-aware model for QoE modelling and measurement using dynamic Bayesian networks (DBN) and a context-aware state-space approach. The proposed model is then used to infer and determine users' QoE in a sequential manner. We performed experimentation to validate the proposed model. The results prove that it can efficiently model, reason and measure QoE of the users'.
---
paper_title: Performance evaluation of a decision-theoretic approach for quality of experience measurement in mobile and pervasive computing scenarios

paper_content:

Measuring and predicting users quality of experience (QoE) in dynamic network conditions is a challenging task. This paper presents results related to a decision-theoretic methodology incorporating Bayesian networks (BNs) and utility theory for quality of experience (QoE) measurement and prediction in mobile computing scenarios. In particular, we show how both generative and discriminative BNs can be used to measure and predict users QoE accurately for voice applications under several wireless network conditions such as wireless signal fading, vertical handoffs, wireless network congestion and normal hotspot traffic. Through extensive simulation studies and results analysis, we show that our proposed methodology can achieve an average accuracy of 98.70% using three different types of Bayesian network.
---
paper_title: A decision-theoretic approach for quality-of-experience measurement and prediction

paper_content:

This paper presents a pioneering context-aware approach for quality of experience (QoE) measurement and prediction. The proposed approach incorporates an intuitive context-aware framework and decision theory. It is capable of incorporating several QoE related classes and context information to correctly measure and predict the overall QoE on a single scale. Our approach can be used in measuring and predicting QoE in both lab and living-lab settings based on user, device and network related context parameters. The predicted QoE can be beneficial for network operators to minimize network churn and can help application developers to build smart user-centric applications. We perform extensive experimentation and the results validate our approach.
---
paper_title: OneClick: A Framework for Measuring Network Quality of Experience

paper_content:

As the service requirements of network applications shift from high throughput to high media quality, interactivity, and responsiveness, the definition of QoE (Quality of Experience) has become multidimensional. Although it may not be difficult to measure individual dimensions of the QoE, how to capture users' overall perceptions when they are using network applications remains an open question. In this paper, we propose a framework called OneClick to capture users' perceptions when they are using network applications. The framework only requires a subject to click a dedicated key whenever he/she feels dissatisfied with the quality of the application in use. OneClick is particularly effective because it is intuitive, lightweight, efficient, time-aware, and application-independent. We use two objective quality assessment methods, PESQ and VQM, to validate OneClick's ability to evaluate the quality of audio and video clips. To demonstrate the proposed framework's efficiency and effectiveness in assessing user experiences, we implement it on two applications, one for instant messaging applications, and the other for first- person shooter games. A Flash implementation of the proposed framework is also presented.
---
paper_title: Modeling subjective tests of quality of experience with a Generalized Linear Model

paper_content:

Subjective tests of telecommunication services engaging a panel of subjects aim to analyze how service users experience their quality (QoE) with varying parameters of both a network and/or application itself. Subjects express their opinion on a few-grade categorical scale during tests performed in a controlled environment. The proposed Generalized Linear Model (GLZ)1 relates probabilities of particular QoE levels to independent network and application explanatory variables in a statistically credible way. The GLZ model is executed in several steps including maximum likelihood estimation of model parameters, selection of the most effective model with the Bayesian information criterion, model validation with a chi-squared test, and last but not least confidence interval estimation. Monitoring a QoE level with a GLZ model provides to operators an effective tool towards a mid- and long-term adaptation of both network and/or application parameters in order to follow users' experience and expectations.
---
paper_title: Evaluating Users ’ Satisfaction in Packet Networks Using Random Neural Networks

paper_content:

Quantifying the quality of a video or audio transmission over the Internet is usually a hard task, as based on the statistical processing of the evaluations made by a panel of humans (the corresponding and standardized area is called subjective testing). In this paper we describe a methodology called Pseudo-Subjective Quality Assessment (PSQA), based on Random Neural Networks, which is able to perform this task automatically, accurately and efficiently. RNN had been chosen here because of their good performances over other possibilities; this is discussed in the paper. Some new insights on PSQA’s use and performance are also given. In particular we discuss new results concerning PSQA–based dynamic quality control, and conversational quality assessment.
---
paper_title: A generic quantitative relationship between quality of experience and quality of service

paper_content:

Quality of experience ties together user perception, experience, and expectations to application and network performance, typically expressed by quality of service parameters. Quantitative relationships between QoE and QoS are required in order to be able to build effective QoE control mechanisms onto measurable QoS parameters. Against this background, this article proposes a generic formula in which QoE and QoS parameters are connected through an exponential relationship, called IQX hypothesis. The formula relates changes of QoE with respect to QoS to the current level of QoE, is simple to match, and its limit behaviors are straightforward to interpret. It validates the IQX hypothesis for streaming services, where QoE in terms of Mean Opinion Scores is expressed as functions of loss and reordering ratio, the latter of which is caused by jitter. For web surfing as the second application area, matchings provided by the IQX hypothesis are shown to outperform previously published logarithmic functions. We conclude that the IQX hypothesis is a strong candidate to be taken into account when deriving relationships between QoE and QoS parameters.
---
paper_title: A probabilistic context-aware approach for quality of experience measurement in pervasive systems

paper_content:

In this paper, we develop a novel context-aware approach for quality of experience (QoE) modeling, reasoning and inferencing in mobile and pervasive computing environments. The proposed model is based upon a state-space approach and Bayesian networks for QoE modeling and reasoning. We further extend this context model to incorporate influence diagrams for efficient QoE inferencing. Our approach accommodates user, device and quality of service (QoS) related context parameters to determine the overall QoE of the user. This helps in user-related media, network and device adaptation. We perform experimentation to validate the proposed approach and the results verify its modeling and inferencing capabilities.
---
paper_title: Online QoE prediction

paper_content:

The Quality of Experience (QoE) is an irreplaceable metric for evaluating the perceived quality of consumers of multimedia content. Due to the subjectiveness of QoE the most suitable way to measure it is by executing subjective studies. However, executing subjective studies is a complex and expensive process. Careful recreation of the viewing conditions is necessary, and a strict selection of the test subjects is required based on many criteria. This is why solutions are often found in various objective methodologies for measuring the QoE of multimedia. These solutions even though more practical are less accurate and cannot reflect the user expectations. In this work we present a method for building QoE prediction models using machine learning techniques from continuous real-time customer feedback, i.e., during the service execution. This online learning approach builds and adapts prediction models that estimate the QoE based on given Quality of Service metrics from real-time user feedback and does not require apriori execution of subjective studies.
---
paper_title: Quality of experience in distributed interactive multimedia environments: toward a theoretical framework

paper_content:

The past decades have witnessed a rapid growth of Distributed Interactive Multimedia Environments (DIMEs). Despite their intensity of user-involved interaction, the existing evaluation frameworks remain very much system-centric. As a step toward the human-centric paradigm, we present a conceptual framework of Quality of Experience (QoE) in DIMEs, to model, measure, and understand user experience and its relationship with the traditional Quality of Service (QoS) metrics. A multi-displinary approach is taken to build up the framework based on the theoretical results from various fields including psychology, cognitive sciences, sociology, and information technology. We introduce a mapping methodology to quantify the correlations between QoS and QoE, and describe our controlled and uncontrolled studies as illustrating examples. The results present the first deep study to model the multi-facet QoE construct, map the QoS-QoE relationship, and capture the human-centric quality modalities in the context of DIMEs.
---
paper_title: The QoE Evaluation Method through the QoS-QoE Correlation Model

paper_content:

Currently, the satisfaction of user is becoming one of the most important topics concerned by the service providers. So we propose the approach method for the objective QoE measurement through the QoS parameters. The measurements and provisioning of the quality of service (QoS) are generally defined in terms of network delivery capacity and resource availability, not in terms of satisfaction to the end-user. The fundamental assumption behind such traditional provisioning is that the measured quality of service is closely related to the quality of experience (QoE) for the end-user. In the paper, we describe the QoS and QoE correlation model, and the QoE evaluation method using QoS parameter in the converged network environment is studied.
---
paper_title: Quantifying Skype user satisfaction

paper_content:

The success of Skype has inspired a generation of peer-to-peer-based solutions for satisfactory real-time multimedia services over the Internet. However, fundamental questions, such as whether VoIP services like Skype are good enough in terms of user satisfaction,have not been formally addressed. One of the major challenges lies in the lack of an easily accessible and objective index to quantify the degree of user satisfaction.In this work, we propose a model, geared to Skype, but generalizable to other VoIP services, to quantify VoIP user satisfaction based on a rigorous analysis of the call duration from actual Skype traces. The User Satisfaction Index (USI) derived from the model is unique in that 1) it is composed by objective source-and network-level metrics, such as the bit rate, bit rate jitter, and round-trip time, 2) unlike speech quality measures based on voice signals, such as the PESQ model standardized by ITU-T, the metrics are easily accessible and computable for real-time adaptation, and 3) the model development only requires network measurements, i.e., no user surveys or voice signals are necessary. Our model is validated by an independent set of metrics that quantifies the degree of user interaction from the actual traces.
---
paper_title: Statistical analysis of ordinal user opinion scores

paper_content:

Data-sets derived from subjective experiments are often exploited to construct objective quality models using parametric statistics such as MOS and multiple regression. In this paper, using data type and normality tests, we verify that non-normally distributed user opinion scores in nominal or ordinal responses should not be analysed using parametric statistics. The paper introduces a number of non-parametric statistics for valid model building and parameter estimation based on user opinion scores. A set of modelling results are also presented to demonstrate the effectiveness of non-parametric statistics.
---
paper_title: A decision-theoretic approach for quality-of-experience measurement and prediction

paper_content:

This paper presents a pioneering context-aware approach for quality of experience (QoE) measurement and prediction. The proposed approach incorporates an intuitive context-aware framework and decision theory. It is capable of incorporating several QoE related classes and context information to correctly measure and predict the overall QoE on a single scale. Our approach can be used in measuring and predicting QoE in both lab and living-lab settings based on user, device and network related context parameters. The predicted QoE can be beneficial for network operators to minimize network churn and can help application developers to build smart user-centric applications. We perform extensive experimentation and the results validate our approach.
---
paper_title: OneClick: A Framework for Measuring Network Quality of Experience

paper_content:

As the service requirements of network applications shift from high throughput to high media quality, interactivity, and responsiveness, the definition of QoE (Quality of Experience) has become multidimensional. Although it may not be difficult to measure individual dimensions of the QoE, how to capture users' overall perceptions when they are using network applications remains an open question. In this paper, we propose a framework called OneClick to capture users' perceptions when they are using network applications. The framework only requires a subject to click a dedicated key whenever he/she feels dissatisfied with the quality of the application in use. OneClick is particularly effective because it is intuitive, lightweight, efficient, time-aware, and application-independent. We use two objective quality assessment methods, PESQ and VQM, to validate OneClick's ability to evaluate the quality of audio and video clips. To demonstrate the proposed framework's efficiency and effectiveness in assessing user experiences, we implement it on two applications, one for instant messaging applications, and the other for first- person shooter games. A Flash implementation of the proposed framework is also presented.
---
paper_title: Modeling subjective tests of quality of experience with a Generalized Linear Model

paper_content:

Subjective tests of telecommunication services engaging a panel of subjects aim to analyze how service users experience their quality (QoE) with varying parameters of both a network and/or application itself. Subjects express their opinion on a few-grade categorical scale during tests performed in a controlled environment. The proposed Generalized Linear Model (GLZ)1 relates probabilities of particular QoE levels to independent network and application explanatory variables in a statistically credible way. The GLZ model is executed in several steps including maximum likelihood estimation of model parameters, selection of the most effective model with the Bayesian information criterion, model validation with a chi-squared test, and last but not least confidence interval estimation. Monitoring a QoE level with a GLZ model provides to operators an effective tool towards a mid- and long-term adaptation of both network and/or application parameters in order to follow users' experience and expectations.
---
paper_title: A generic quantitative relationship between quality of experience and quality of service

paper_content:

Quality of experience ties together user perception, experience, and expectations to application and network performance, typically expressed by quality of service parameters. Quantitative relationships between QoE and QoS are required in order to be able to build effective QoE control mechanisms onto measurable QoS parameters. Against this background, this article proposes a generic formula in which QoE and QoS parameters are connected through an exponential relationship, called IQX hypothesis. The formula relates changes of QoE with respect to QoS to the current level of QoE, is simple to match, and its limit behaviors are straightforward to interpret. It validates the IQX hypothesis for streaming services, where QoE in terms of Mean Opinion Scores is expressed as functions of loss and reordering ratio, the latter of which is caused by jitter. For web surfing as the second application area, matchings provided by the IQX hypothesis are shown to outperform previously published logarithmic functions. We conclude that the IQX hypothesis is a strong candidate to be taken into account when deriving relationships between QoE and QoS parameters.
---
paper_title: Multiple Attribute Decision Making: Methods and Applications

paper_content:

Decision makers are often faced with several conflicting alternatives [1]. How do they evaluate trade-offs when there are more than three criteria? To help people make optimal decisions, scholars in the discipline of multiple criteria decision making (MCDM) continue to develop new methods for structuring preferences and determining the correct relative weights for criteria. A compilation of modern decision-making techniques, Multiple Attribute Decision Making: Methods and Applications focuses on the fuzzy set approach to multiple attribute decision making (MADM). Drawing on their experience, the authors bring together current methods and real-life applications of MADM techniques for decision analysis. They also propose a novel hybrid MADM model that combines DEMATEL and analytic network process (ANP) with VIKOR procedures.
---
paper_title: Context-Aware QoE Modelling, Measurement, and Prediction in Mobile Computing Systems

paper_content:

Quality of Experience (QoE) as an aggregate of Quality of Service (QoS) and human user-related metrics will be the key success factor for current and future mobile computing systems. QoE measurement and prediction are complex tasks as they may involve a large parameter space such as location, delay, jitter, packet loss, and user satisfaction just to name a few. These tasks necessitate the development of practical context-aware QoE models that efficiently determine relationships between user context and QoE parameters. In this paper, we propose, develop, and validate a novel decision-theoretic approach called CaQoEM for QoE modelling, measurement, and prediction. We address the challenge of QoE measurement and prediction where each QoE parameter can be measured on a different scale and may involve different units of measurement. CaQoEM is context-aware and uses Bayesian networks and utility theory to measure and predict users’ QoE under uncertainty. We validate CaQoEM using extensive experimentation, user studies and simulations. The results soundly demonstrate that CaQoEM correctly measures range-defined QoE using a bipolar scale. For QoE prediction, an overall accuracy of 98.93% was achieved using 10-fold cross validation in multiple diverse network conditions such as vertical handoffs, wireless signal fading and wireless network congestion.
---
paper_title: Quantifying Skype user satisfaction

paper_content:

The success of Skype has inspired a generation of peer-to-peer-based solutions for satisfactory real-time multimedia services over the Internet. However, fundamental questions, such as whether VoIP services like Skype are good enough in terms of user satisfaction,have not been formally addressed. One of the major challenges lies in the lack of an easily accessible and objective index to quantify the degree of user satisfaction.In this work, we propose a model, geared to Skype, but generalizable to other VoIP services, to quantify VoIP user satisfaction based on a rigorous analysis of the call duration from actual Skype traces. The User Satisfaction Index (USI) derived from the model is unique in that 1) it is composed by objective source-and network-level metrics, such as the bit rate, bit rate jitter, and round-trip time, 2) unlike speech quality measures based on voice signals, such as the PESQ model standardized by ITU-T, the metrics are easily accessible and computable for real-time adaptation, and 3) the model development only requires network measurements, i.e., no user surveys or voice signals are necessary. Our model is validated by an independent set of metrics that quantifies the degree of user interaction from the actual traces.
---
paper_title: Performance evaluation of a decision-theoretic approach for quality of experience measurement in mobile and pervasive computing scenarios

paper_content:

Measuring and predicting users quality of experience (QoE) in dynamic network conditions is a challenging task. This paper presents results related to a decision-theoretic methodology incorporating Bayesian networks (BNs) and utility theory for quality of experience (QoE) measurement and prediction in mobile computing scenarios. In particular, we show how both generative and discriminative BNs can be used to measure and predict users QoE accurately for voice applications under several wireless network conditions such as wireless signal fading, vertical handoffs, wireless network congestion and normal hotspot traffic. Through extensive simulation studies and results analysis, we show that our proposed methodology can achieve an average accuracy of 98.70% using three different types of Bayesian network.
---
paper_title: A decision-theoretic approach for quality-of-experience measurement and prediction

paper_content:

This paper presents a pioneering context-aware approach for quality of experience (QoE) measurement and prediction. The proposed approach incorporates an intuitive context-aware framework and decision theory. It is capable of incorporating several QoE related classes and context information to correctly measure and predict the overall QoE on a single scale. Our approach can be used in measuring and predicting QoE in both lab and living-lab settings based on user, device and network related context parameters. The predicted QoE can be beneficial for network operators to minimize network churn and can help application developers to build smart user-centric applications. We perform extensive experimentation and the results validate our approach.
---
paper_title: OneClick: A Framework for Measuring Network Quality of Experience

paper_content:

As the service requirements of network applications shift from high throughput to high media quality, interactivity, and responsiveness, the definition of QoE (Quality of Experience) has become multidimensional. Although it may not be difficult to measure individual dimensions of the QoE, how to capture users' overall perceptions when they are using network applications remains an open question. In this paper, we propose a framework called OneClick to capture users' perceptions when they are using network applications. The framework only requires a subject to click a dedicated key whenever he/she feels dissatisfied with the quality of the application in use. OneClick is particularly effective because it is intuitive, lightweight, efficient, time-aware, and application-independent. We use two objective quality assessment methods, PESQ and VQM, to validate OneClick's ability to evaluate the quality of audio and video clips. To demonstrate the proposed framework's efficiency and effectiveness in assessing user experiences, we implement it on two applications, one for instant messaging applications, and the other for first- person shooter games. A Flash implementation of the proposed framework is also presented.
---
paper_title: Modeling subjective tests of quality of experience with a Generalized Linear Model

paper_content:

Subjective tests of telecommunication services engaging a panel of subjects aim to analyze how service users experience their quality (QoE) with varying parameters of both a network and/or application itself. Subjects express their opinion on a few-grade categorical scale during tests performed in a controlled environment. The proposed Generalized Linear Model (GLZ)1 relates probabilities of particular QoE levels to independent network and application explanatory variables in a statistically credible way. The GLZ model is executed in several steps including maximum likelihood estimation of model parameters, selection of the most effective model with the Bayesian information criterion, model validation with a chi-squared test, and last but not least confidence interval estimation. Monitoring a QoE level with a GLZ model provides to operators an effective tool towards a mid- and long-term adaptation of both network and/or application parameters in order to follow users' experience and expectations.
---
paper_title: Predicting quality of experience in multimedia streaming

paper_content:

Measuring and predicting the user's Quality of Experience (QoE) of a multimedia stream is the first step towards improving and optimizing the provision of mobile streaming services. This enables us to better understand how Quality of Service (QoS) parameters affect service quality, as it is actually perceived by the end user. Over the last years this goal has been pursued by means of subjective tests and through the analysis of the user's feedback. Existing statistical techniques have lead to poor accuracy (order of 70%) and inability to evolve prediction models with the system's dynamics. In this paper, we propose a novel approach for building accurate and adaptive QoE prediction models using Machine Learning classification algorithms, trained on subjective test data. These models can be used for real-time prediction of QoE and can be efficiently integrated into online learning systems that can adapt the models according to changes in the environment. Providing high accuracy of above 90%, the classification algorithms become an indispensible component of a mobile multimedia QoE management system.
---
paper_title: Evaluating Users ’ Satisfaction in Packet Networks Using Random Neural Networks

paper_content:

Quantifying the quality of a video or audio transmission over the Internet is usually a hard task, as based on the statistical processing of the evaluations made by a panel of humans (the corresponding and standardized area is called subjective testing). In this paper we describe a methodology called Pseudo-Subjective Quality Assessment (PSQA), based on Random Neural Networks, which is able to perform this task automatically, accurately and efficiently. RNN had been chosen here because of their good performances over other possibilities; this is discussed in the paper. Some new insights on PSQA’s use and performance are also given. In particular we discuss new results concerning PSQA–based dynamic quality control, and conversational quality assessment.
---
paper_title: A probabilistic context-aware approach for quality of experience measurement in pervasive systems

paper_content:

In this paper, we develop a novel context-aware approach for quality of experience (QoE) modeling, reasoning and inferencing in mobile and pervasive computing environments. The proposed model is based upon a state-space approach and Bayesian networks for QoE modeling and reasoning. We further extend this context model to incorporate influence diagrams for efficient QoE inferencing. Our approach accommodates user, device and quality of service (QoS) related context parameters to determine the overall QoE of the user. This helps in user-related media, network and device adaptation. We perform experimentation to validate the proposed approach and the results verify its modeling and inferencing capabilities.
---
paper_title: Online QoE prediction

paper_content:

The Quality of Experience (QoE) is an irreplaceable metric for evaluating the perceived quality of consumers of multimedia content. Due to the subjectiveness of QoE the most suitable way to measure it is by executing subjective studies. However, executing subjective studies is a complex and expensive process. Careful recreation of the viewing conditions is necessary, and a strict selection of the test subjects is required based on many criteria. This is why solutions are often found in various objective methodologies for measuring the QoE of multimedia. These solutions even though more practical are less accurate and cannot reflect the user expectations. In this work we present a method for building QoE prediction models using machine learning techniques from continuous real-time customer feedback, i.e., during the service execution. This online learning approach builds and adapts prediction models that estimate the QoE based on given Quality of Service metrics from real-time user feedback and does not require apriori execution of subjective studies.
---
paper_title: The memory effect and its implications on Web QoE modeling

paper_content:

Quality of Experience (QoE) has gained enormous attention during the recent years. So far, most of the existing QoE research has focused on audio and video streaming applications, although HTTP traffic carries the majority of traffic in the residential broadband Internet. However, existing QoE models for this domain do not consider temporal dynamics or historical experiences of the user's satisfaction while consuming a certain service. This psychological influence factor of past experience is referred to as the memory effect. The first contribution of this paper is the identification of the memory effect as a key influence factor for Web QoE modeling based on subjective user studies. As second contribution, three different QoE models are proposed which consider the implications of the memory effect and imply the required extensions of the basic models. The proposed Web QoE models are described with a) support vector machines, b) iterative exponential regressions, and c) two-dimensional hidden Markov models.
---
paper_title: Dynamic Bayesian Networks for Sequential Quality of Experience Modelling and Measurement

paper_content:

This paper presents a novel context-aware methodology for modelling and measuring user-perceived quality of experience (QoE) over time. In particular, we create a context-aware model for QoE modelling and measurement using dynamic Bayesian networks (DBN) and a context-aware state-space approach. The proposed model is then used to infer and determine users' QoE in a sequential manner. We performed experimentation to validate the proposed model. The results prove that it can efficiently model, reason and measure QoE of the users'.
---
paper_title: QoE estimation and prediction using hidden Markov models in heterogeneous access networks

paper_content:

Quality of Experience (QoE) based handoffs in heterogeneous access networks (HAN) necessitates accurate QoE estimation and prediction. The current approaches to QoE-aware handoffs are limited. These approaches either lack the availability of underlying probing mechanism or lack the availability of QoE estimation and prediction mechanism. In this paper, we propose, develop and validate a novel method for QoE estimation and prediction using passive probing mechanisms. Our method is based on hidden Markov models and multi-homed mobility management protocol. Using extensive simulations and experimental studies, we show that our method achieves QoE estimation accuracy of 100% and prediction accuracy of 97% in HAN without using additional probe packets for QoE estimation and prediction.
---
paper_title: A tutorial on hidden Markov models and selected applications in speech recognition

paper_content:

This tutorial provides an overview of the basic theory of hidden Markov models (HMMs) as originated by L.E. Baum and T. Petrie (1966) and gives practical details on methods of implementation of the theory along with a description of selected applications of the theory to distinct problems in speech recognition. Results from a number of original sources are combined to provide a single source of acquiring the background required to pursue further this area of research. The author first reviews the theory of discrete Markov chains and shows how the concept of hidden states, where the observation is a probabilistic function of the state, can be used effectively. The theory is illustrated with two simple examples, namely coin-tossing, and the classic balls-in-urns system. Three fundamental problems of HMMs are noted and several practical techniques for solving these problems are given. The various types of HMMs that have been studied, including ergodic as well as left-right models, are described. >
---
paper_title: A model for measuring Quality of Experience

paper_content:

In this paper we present a model for measuring the Quality of Experience (QoE) of multimedia services. Our model consists of measurable and non-measurable parameters and describes a framework for quantifying the model parameters by transformation into an XML document with discreet values. The model has been validated by running a field trial over 4 weeks using 3G mobile phones for a video on demand service and mobile TV.
---
paper_title: Toward total quality of experience: A QoE model in a communication ecosystem

paper_content:

In recent years, the quality of experience notion has become a major research theme within the telecommunications community. QoE is an assessment of the human experience when interacting with technology and business entities in a particular context. A communication ecosystem encompasses various domains such as technical aspects, business models, human behavior, and context. For each aspect of a communication ecosystem, various models have been developed. However, few models have been designed to integrate all aspects of a communication ecosystem to understand human behavioral needs in a detailed and structured way. While existing models have produced the basic sketch of QoE modeling, more concepts and interdomain mapping are to be incorporated in order to have a clear picture of QoE in communication ecosystems. The aim of the current work is to build on the existing research being conducted in disparate disciplines about human behavior in order to provide a high-level model that can be adapted to many specific contexts and to encourage future research which examines these crossdomain relationships.
---
paper_title: A decision-theoretic approach for quality-of-experience measurement and prediction

paper_content:

This paper presents a pioneering context-aware approach for quality of experience (QoE) measurement and prediction. The proposed approach incorporates an intuitive context-aware framework and decision theory. It is capable of incorporating several QoE related classes and context information to correctly measure and predict the overall QoE on a single scale. Our approach can be used in measuring and predicting QoE in both lab and living-lab settings based on user, device and network related context parameters. The predicted QoE can be beneficial for network operators to minimize network churn and can help application developers to build smart user-centric applications. We perform extensive experimentation and the results validate our approach.
---
paper_title: Quality of Experience in Communications Ecosystem

paper_content:

Communications ecosystem covers a huge area from technical issues to business models and human behaviour. Due to this extreme diversity various societies need to discuss with each other, each of them using their own language. Engineers talk about network performance and quality of service, business people talk about average revenue per user and customer churn while behavioural scientists talk about happiness and experiences. Thus, everyone who wants to understand, or even analyze, the whole ecosystem, has to deal with all these diverse issues. In addition to the apparent communication problems, the main challenges of ecosystem analysis are to realistically model human behaviour, and to efficiently combine the models developed for different domains. A central concept when solving these problems is quality of experience (QoE). This paper sheds light on the role of QoE by means of a common framework that covers the whole communications ecosystem. Additionally, a research agenda for a holistic ecosystem analysis is outlined.
---
paper_title: User measures of quality of experience: why being objective and quantitative is important

paper_content:

This article develops the concept of quality of user experience by focusing on requirements for its measurement and communication for use in industry. Whereas more typical work involves either modeling media quality based on technical measures or user subjective assessment of media quality, it is argued that QoE measurement should include objective measures of the process and outcomes of usage. Also, subjective measures should be collected as quantitative data rather than qualitative data as is currently common, because quantitative data enables combined analyses of objective and subjective measures for deriving global QoE ratings. Global ratings are important for optimizing the communicability of results to target audiences. Examples of measures and scales are provided. A structured approach to defining and measuring QoE in relation to QoS is explained and illustrated.
---
paper_title: A model for measuring Quality of Experience

paper_content:

In this paper we present a model for measuring the Quality of Experience (QoE) of multimedia services. Our model consists of measurable and non-measurable parameters and describes a framework for quantifying the model parameters by transformation into an XML document with discreet values. The model has been validated by running a field trial over 4 weeks using 3G mobile phones for a video on demand service and mobile TV.
---
paper_title: A probabilistic context-aware approach for quality of experience measurement in pervasive systems

paper_content:

In this paper, we develop a novel context-aware approach for quality of experience (QoE) modeling, reasoning and inferencing in mobile and pervasive computing environments. The proposed model is based upon a state-space approach and Bayesian networks for QoE modeling and reasoning. We further extend this context model to incorporate influence diagrams for efficient QoE inferencing. Our approach accommodates user, device and quality of service (QoS) related context parameters to determine the overall QoE of the user. This helps in user-related media, network and device adaptation. We perform experimentation to validate the proposed approach and the results verify its modeling and inferencing capabilities.
---
paper_title: Model-Based Approach to Measuring Quality of Experience

paper_content:

Quantifiable quality measures are necessary for VoIP service providers in order to offer services and be sure to keep users for their services. Existing methods used for evaluating the Quality of Experience (QoE) mostly rely on user survey and scores from the user, which are too subjective and need much processing time and cost. This paper proposes a pentagram model for measuring QoE. The model consists of five factors, including integrality, retainability, availability, usability, and instantaneousness. The experiments show that the proposed model offers better performance in terms of quality measurement than the conventional method.
---
paper_title: The memory effect and its implications on Web QoE modeling

paper_content:

Quality of Experience (QoE) has gained enormous attention during the recent years. So far, most of the existing QoE research has focused on audio and video streaming applications, although HTTP traffic carries the majority of traffic in the residential broadband Internet. However, existing QoE models for this domain do not consider temporal dynamics or historical experiences of the user's satisfaction while consuming a certain service. This psychological influence factor of past experience is referred to as the memory effect. The first contribution of this paper is the identification of the memory effect as a key influence factor for Web QoE modeling based on subjective user studies. As second contribution, three different QoE models are proposed which consider the implications of the memory effect and imply the required extensions of the basic models. The proposed Web QoE models are described with a) support vector machines, b) iterative exponential regressions, and c) two-dimensional hidden Markov models.
---
paper_title: Dynamic Bayesian Networks for Sequential Quality of Experience Modelling and Measurement

paper_content:

This paper presents a novel context-aware methodology for modelling and measuring user-perceived quality of experience (QoE) over time. In particular, we create a context-aware model for QoE modelling and measurement using dynamic Bayesian networks (DBN) and a context-aware state-space approach. The proposed model is then used to infer and determine users' QoE in a sequential manner. We performed experimentation to validate the proposed model. The results prove that it can efficiently model, reason and measure QoE of the users'.
---
paper_title: Statistical analysis of ordinal user opinion scores

paper_content:

Data-sets derived from subjective experiments are often exploited to construct objective quality models using parametric statistics such as MOS and multiple regression. In this paper, using data type and normality tests, we verify that non-normally distributed user opinion scores in nominal or ordinal responses should not be analysed using parametric statistics. The paper introduces a number of non-parametric statistics for valid model building and parameter estimation based on user opinion scores. A set of modelling results are also presented to demonstrate the effectiveness of non-parametric statistics.
---
paper_title: A decision-theoretic approach for quality-of-experience measurement and prediction

paper_content:

This paper presents a pioneering context-aware approach for quality of experience (QoE) measurement and prediction. The proposed approach incorporates an intuitive context-aware framework and decision theory. It is capable of incorporating several QoE related classes and context information to correctly measure and predict the overall QoE on a single scale. Our approach can be used in measuring and predicting QoE in both lab and living-lab settings based on user, device and network related context parameters. The predicted QoE can be beneficial for network operators to minimize network churn and can help application developers to build smart user-centric applications. We perform extensive experimentation and the results validate our approach.
---
paper_title: Modeling subjective tests of quality of experience with a Generalized Linear Model

paper_content:

Subjective tests of telecommunication services engaging a panel of subjects aim to analyze how service users experience their quality (QoE) with varying parameters of both a network and/or application itself. Subjects express their opinion on a few-grade categorical scale during tests performed in a controlled environment. The proposed Generalized Linear Model (GLZ)1 relates probabilities of particular QoE levels to independent network and application explanatory variables in a statistically credible way. The GLZ model is executed in several steps including maximum likelihood estimation of model parameters, selection of the most effective model with the Bayesian information criterion, model validation with a chi-squared test, and last but not least confidence interval estimation. Monitoring a QoE level with a GLZ model provides to operators an effective tool towards a mid- and long-term adaptation of both network and/or application parameters in order to follow users' experience and expectations.
---
paper_title: A generic quantitative relationship between quality of experience and quality of service

paper_content:

Quality of experience ties together user perception, experience, and expectations to application and network performance, typically expressed by quality of service parameters. Quantitative relationships between QoE and QoS are required in order to be able to build effective QoE control mechanisms onto measurable QoS parameters. Against this background, this article proposes a generic formula in which QoE and QoS parameters are connected through an exponential relationship, called IQX hypothesis. The formula relates changes of QoE with respect to QoS to the current level of QoE, is simple to match, and its limit behaviors are straightforward to interpret. It validates the IQX hypothesis for streaming services, where QoE in terms of Mean Opinion Scores is expressed as functions of loss and reordering ratio, the latter of which is caused by jitter. For web surfing as the second application area, matchings provided by the IQX hypothesis are shown to outperform previously published logarithmic functions. We conclude that the IQX hypothesis is a strong candidate to be taken into account when deriving relationships between QoE and QoS parameters.
---
paper_title: Multiple Attribute Decision Making: Methods and Applications

paper_content:

Decision makers are often faced with several conflicting alternatives [1]. How do they evaluate trade-offs when there are more than three criteria? To help people make optimal decisions, scholars in the discipline of multiple criteria decision making (MCDM) continue to develop new methods for structuring preferences and determining the correct relative weights for criteria. A compilation of modern decision-making techniques, Multiple Attribute Decision Making: Methods and Applications focuses on the fuzzy set approach to multiple attribute decision making (MADM). Drawing on their experience, the authors bring together current methods and real-life applications of MADM techniques for decision analysis. They also propose a novel hybrid MADM model that combines DEMATEL and analytic network process (ANP) with VIKOR procedures.
---
paper_title: User measures of quality of experience: why being objective and quantitative is important

paper_content:

This article develops the concept of quality of user experience by focusing on requirements for its measurement and communication for use in industry. Whereas more typical work involves either modeling media quality based on technical measures or user subjective assessment of media quality, it is argued that QoE measurement should include objective measures of the process and outcomes of usage. Also, subjective measures should be collected as quantitative data rather than qualitative data as is currently common, because quantitative data enables combined analyses of objective and subjective measures for deriving global QoE ratings. Global ratings are important for optimizing the communicability of results to target audiences. Examples of measures and scales are provided. A structured approach to defining and measuring QoE in relation to QoS is explained and illustrated.
---
paper_title: Context-Aware QoE Modelling, Measurement, and Prediction in Mobile Computing Systems

paper_content:

Quality of Experience (QoE) as an aggregate of Quality of Service (QoS) and human user-related metrics will be the key success factor for current and future mobile computing systems. QoE measurement and prediction are complex tasks as they may involve a large parameter space such as location, delay, jitter, packet loss, and user satisfaction just to name a few. These tasks necessitate the development of practical context-aware QoE models that efficiently determine relationships between user context and QoE parameters. In this paper, we propose, develop, and validate a novel decision-theoretic approach called CaQoEM for QoE modelling, measurement, and prediction. We address the challenge of QoE measurement and prediction where each QoE parameter can be measured on a different scale and may involve different units of measurement. CaQoEM is context-aware and uses Bayesian networks and utility theory to measure and predict users’ QoE under uncertainty. We validate CaQoEM using extensive experimentation, user studies and simulations. The results soundly demonstrate that CaQoEM correctly measures range-defined QoE using a bipolar scale. For QoE prediction, an overall accuracy of 98.93% was achieved using 10-fold cross validation in multiple diverse network conditions such as vertical handoffs, wireless signal fading and wireless network congestion.
---
paper_title: Online QoE prediction

paper_content:

The Quality of Experience (QoE) is an irreplaceable metric for evaluating the perceived quality of consumers of multimedia content. Due to the subjectiveness of QoE the most suitable way to measure it is by executing subjective studies. However, executing subjective studies is a complex and expensive process. Careful recreation of the viewing conditions is necessary, and a strict selection of the test subjects is required based on many criteria. This is why solutions are often found in various objective methodologies for measuring the QoE of multimedia. These solutions even though more practical are less accurate and cannot reflect the user expectations. In this work we present a method for building QoE prediction models using machine learning techniques from continuous real-time customer feedback, i.e., during the service execution. This online learning approach builds and adapts prediction models that estimate the QoE based on given Quality of Service metrics from real-time user feedback and does not require apriori execution of subjective studies.
---

Title: QoE Modelling, Measurement and Prediction: A Review

Section 1: Introduction
Description 1: Introduce the importance of understanding Quality of Experience (QoE), user expectations, and how these perceptions impact user retention and satisfaction, including various definitions and perspectives on QoE.

Section 2: Subjective Methods
Description 2: Discuss methods of direct data collection from users to assess QoE, such as the mean opinion score (MOS) and the challenges associated with conducting subjective tests.

Section 3: Objective Methods
Description 3: Explore various objective methods for estimating QoE without requiring subjective tests, and the complexities involved in developing, modeling, and deploying these methods.

Section 4: The role of context
Description 4: Examine the importance of context in QoE measurement and prediction, highlighting how dynamic and static context parameters affect user experience in real-life environments.

Section 5: Research Challenges
Description 5: Identify and elaborate on the challenges faced by researchers in QoE modeling, measurement, and prediction, including the complexity of parameter space and the need for models that can handle multiple QoE parameters.

Section 6: QoE Modelling
Description 6: Review different approaches and models for QoE measurement, including their strengths and limitations, and discuss conceptual and practical models for QoE.

Section 7: QoE Measurement and Prediction
Description 7: Detail various methods for QoE prediction, including regression-based, correlation analysis-based, and AI/ML-based methods, and discuss their effectiveness in different applications.

Section 8: Methods for QoE Measurement and Prediction Over Time
Description 8: Discuss longitudinal approaches and techniques for QoE measurement and prediction, emphasizing the importance of tracking and predicting QoE over extended periods.

Section 9: From Conceptual to Practical Models for QoE Measurement and Prediction
Description 9: Discuss the progression from conceptual to practical models, emphasizing the need for realistic implementations that can be applied across various systems and applications.

Section 10: Conclusion
Description 10: Summarize the key points discussed in the paper, highlight the challenges and the state-of-the-art in QoE research, and outline future research directions for QoE modeling, measurement, and prediction.