HEAT

HEAT Project

CORDIS Reference	Start date	End date	Coordinator
https://cordis.europa.eu/project/id/101135637	01/06/2024	31/05/2027	UNIVERSITA DEGLI STUDI DI CAGLIARI / Italy

Project description

Hybrid Extended reAliTy (HEAT) is born to pave the way for the next-generation distributed experiences by addressing major challenges to make those experiences that up to now could only be in our imagination: being realistically immersed (holo-ported) within real captured omnidirectional and navigable hyper-realistic 3D spaces, feeling their atmosphere, and sharing these experiences with others, regardless of their location. The aim of this proposal is to integrate immersive media technologies such as point cloud/holographic imaging, multi-sensorial media, Social VR in a multi-user, feedback-enabled communication system to provide the construction of compelling context-aware and embodied experiences for innovative hybrid XR applications, where remote users can experience a real captured environment through immersive VR, while in presence users can visualise and interact with the holograms of remote users integrated in the real environment through holographic rendering. The system will aim at facilitating the exploitation of agile (multi-sensory) 3D data acquisition techniques, enhancing performance while reducing technology costs. It will create a scalable communication pipeline embodying either encapsulation of different media from classical audio-video to multisensorial to holographic video or a combination of them, providing means for efficient and scalable encoding, processing, storage, (real-time) streaming and rendering. In such a way, the system will allow users to provide/exploit the features according to the acquisition/rendering system available. The project will provide well designed and fully tested scenarios in real-world environments for enhanced XR experiences: a blended learning, a modern theatre act, a music festival and an opera show. All pilot actions will ensure that GDPR and ethics are addressed for end-users (privacy and ethics by design methodology).

Project outputs

Publications

Domain	Type of output	Title	DOI URL
AI, Machine Learning & Data Science	Conference proceedings	Forecasting Ethereum Prices with Machine Learning, Deep Learning, and Explainable Artificial Intelligence Using Multi-source Market Articles and Hybrid Sentiment Analysis	https://doi.org/10.1007/978-3-032-04339-9_12
AI, Machine Learning & Data Science	Conference proceedings	Pixelating to the Edge: Generative AI Art on Edge Devices	https://doi.org/10.1109/BMSB65076.2025.11165718
Computer Vision, 3D Modeling & Rendering	Peer reviewed articles	Low-Complexity Patch-Based No-Reference Point Cloud Quality Metric Exploiting Weighted Structure and Texture Features	https://doi.org/10.48550/ARXIV.2503.15001
Computer Vision, 3D Modeling & Rendering	Peer reviewed articles	CNN-Based 360° Scene Recognition for Automatic Generation of Omnidirectional Scent Effects	https://doi.org/10.1109/TMM.2025.3618574
Ethics, Society, Arts & Culture	Conference proceedings	Proxemic Behavior in a Multi-User Virtual Reality Experience for Multilingual Cultural Heritage Education	https://doi.org/10.1109/ISPA66905.2025.11259464
Extended Reality (VR/AR/MR) & HCI	Conference proceedings	Immersive Live Concert: A Multi-Sensory Experience Based on Real-Time Lyrics Detection from Spatial Audio Data	https://doi.org/10.5753/IMXW.2025.1139
Extended Reality (VR/AR/MR) & HCI	Conference proceedings	Social eXtended Reality (XR) and Virtual Production: Toward New Engaging Immersive Experiences	https://doi.org/10.1145/3706370.3731642
Extended Reality (VR/AR/MR) & HCI	Conference proceedings	Immersive Application for Real-Time Interactive Music Performances Using Spatial Audio	https://doi.org/10.1109/BMSB65076.2025.11165553
Extended Reality (VR/AR/MR) & HCI	Conference proceedings	Live Feedback for Immersive Music Performances - A Case Study	https://doi.org/10.5753/IMXW.2025.8520
Extended Reality (VR/AR/MR) & HCI	Conference proceedings	aDapT-XR: Adaptive Data Allocation and Prioritization for Synchronizing Real and Virtual Worlds in XR Digital Twins	https://doi.org/10.1109/BMSB65076.2025.11165558
Extended Reality (VR/AR/MR) & HCI	Conference proceedings	EmoLoop: A Bi-Directional System for Emotion-Driven Interaction Between Remote Audiences and Live Performers	https://doi.org/10.1109/IS264627.2025.11284604
Extended Reality (VR/AR/MR) & HCI	Conference proceedings	A Fast Volumetric Capture and Reconstruction Pipeline for Dynamic Point Clouds and Gaussian Splats	https://doi.org/10.1145/3756863.3769713
Extended Reality (VR/AR/MR) & HCI	Conference proceedings	An Architecture for Real-Time Interactive Music Performances	https://doi.org/10.1109/MEDCOMNET65822.2025.11103534
Extended Reality (VR/AR/MR) & HCI	Conference proceedings	Enhancing XR Theatre with Remote Scent Delivery Using Audio Speech Recognition and Convolutional Neural Network-Based Scene Detection	https://doi.org/10.1109/IS264627.2025.11284558
Extended Reality (VR/AR/MR) & HCI	Conference proceedings	Social eXtended Reality (XR) and Virtual Production: Crafting New Frontiers in Immersive Storytelling	https://doi.org/10.5753/IMXW.2025.1505
Extended Reality (VR/AR/MR) & HCI	Conference proceedings	Toward Interactive and Distributed Hybrid Extended Reality Experiences	https://doi.org/10.1145/3672406.3672424
Extended Reality (VR/AR/MR) & HCI	Conference proceedings	Social VR with multi-user holo-portation: toward a new medium for rich interactive shared media consumption	https://doi.org/10.1109/BMSB65076.2025.11165698
Extended Reality (VR/AR/MR) & HCI	Peer reviewed articles	Social VR With Holographic Comms: Enablers for New Engaging Experiences Within the TV / Video Consumption Landscape	https://doi.org/10.1109/TBC.2025.3570869
Networks, Cloud & Telecommunications (5G/6G)	Conference proceedings	Collaborative 360-Degree Video Streaming: A Multi-User Synchronization Solution with Socket.IO	https://doi.org/10.1109/BMSB65076.2025.11165535
Networks, Cloud & Telecommunications (5G/6G)	Conference proceedings	Estimating Quality of Experience in Multicast Point Cloud Streaming over 5G Networks	https://doi.org/10.1109/BMSB65076.2025.11165660
Networks, Cloud & Telecommunications (5G/6G)	Conference proceedings	Mass Live and Interactive Multi-Cam VR360 Experiences as an Immersive Parallel Window to TV Broadcast Shows	https://doi.org/10.1109/BMSB65076.2025.11165554
Networks, Cloud & Telecommunications (5G/6G)	Conference proceedings	QoE-aware ML models based on network parameters for video streaming over 5G O-RAN architecture	https://doi.org/10.1109/BMSB65076.2025.11165626
Networks, Cloud & Telecommunications (5G/6G)	Peer reviewed articles	Utility-Aware Adaptive Streaming of Segmented Holographic Video Over Wireless Networks: A Knapsack-Theoretic Approach	https://doi.org/10.1109/JIOT.2025.3605830
Networks, Cloud & Telecommunications (5G/6G)	Peer reviewed articles	Quality and Energy-Aware Cooperative Distributed Solution for Mobile Rich Media Delivery in Wireless Heterogeneous Environments	https://doi.org/10.1109/TBC.2025.3609056
Networks, Cloud & Telecommunications (5G/6G)	Peer reviewed articles	A UAV-Centric Improved Soft Actor-Critic Algorithm for QoE-Focused Aerial Video Streaming	https://doi.org/10.1109/TVT.2024.3396349
Networks, Cloud & Telecommunications (5G/6G)	Peer reviewed articles	A Live Adaptive Streaming Solution for Enhancing Quality of Experience in Co-Created Opera	https://doi.org/10.1109/TBC.2025.3541875

Technological assets

Title	Type of Asset	Link / DOI	Description
EmoLoop	System / Software	https://doi.org/10.1109/IS264627.2025.11284604	A bi-directional system designed for emotion-driven interaction between remote audiences and performers.