QoE-Centric Multimedia Communications
Background and Challenges
High-definition video streaming such as YouTube and Netflix and highly real-time online cloud gaming have become deeply integrated into our daily lives. However, conventional network controls have focused predominantly on optimizing physical Quality of Service (QoS) parameters at the network layer, such as maximizing throughput or minimizing packet latency.
Such physical metrics do not directly correlate with the subjective Quality of Experience (QoE) of users. For example, sudden resolution drops (bitrate switching) during video playback or unexpected lag during critical moments in games create significant visual and operational stress. Innovative control mechanisms are required to efficiently allocate limited wireless frequencies and mobile edge computing (MEC) resources to maximize overall user satisfaction.
Our Approach
Our laboratory investigates next-generation, QoE-centric communication control techniques that feed back subjective user evaluations, content features, and gameplay contexts into network resource management.
1. Dynamic Resource Allocation Based on User Interest and Engagement
We research resource allocation schemes that consider the user's interest level (Attention) or engagement towards specific video scenes and incorporate them into wireless resource scheduling. By leveraging MEC and Software-Defined Mobile Networks, we develop resource allocation algorithms such as CLQDBA that prioritize high-quality bitrates for critical scenes while suppressing resources elsewhere. We also propose dynamic bandwidth control based on mobile user engagement to efficiently utilize wireless link resources.
2. Personalized QoE Estimation using Biological Information and Federated Learning
Rather than relying on one-size-fits-all evaluations, we develop personalized models tailored to individual user profiles. Our work includes estimating QoE drops and stress levels during ad insertions using biological signals such as brainwaves and eye-tracking, and personalizing QoE estimation via clustered federated learning to optimize models locally on user terminals without leaking raw data. We also evaluate and analyze how different ad insertion timings including pre-roll and mid-roll and advertisement memorability impact the overall subjective user comfort.
3. Context-Aware Dynamic Bandwidth Control in Cloud Gaming
For multiplayer online games like First-Person Shooters (FPS), we propose control systems that adjust bandwidth allocation based on game-state metadata, such as the spatial distance between players and the probability of combat events in the virtual space. By prioritizing low-latency connections and high bitrates for active combat zones, we prevent playability drops caused by network congestion and enhance competitive fairness and immersion.
4. Packet Loss Concealment (PLC) for High-Quality Audio Transmission
In real-time audio transmission over best-effort networks, packet loss is inevitable, causing audio drops and disturbing noise. To tackle this, we study Packet Loss Concealment (PLC) technologies combining deep learning and time-series prediction (such as linear predictive coding) specifically targeting ensemble music sources. By reconstructing missing audio segments in both frequency and time domains via neural network prediction, we achieve seamless and high-quality audio reproduction even under severe packet loss environments.