DIDYMOS-XR

DIDYMOS-XR Project

CORDIS Reference	Start date	End date	Coordinator	Project website
https://cordis.europa.eu/project/id/101092875	01/01/2023	31/12/2025	JOANNEUM / Austria	https://didymos-xr.eu/

Project description

The digital transformation and the availability of more diversified and cost-effective means for 3D capture have led to the creation of digital twins also for physical environments. Based on such digital twins, various applications could be built using real-time data from real-world environments, serving as a blueprint for smart cities and for improving performance and efficiency across industries. Currently, creating high-fidelity digital twins is costly, and their update requires manual intervention. Furthermore, data integration from heterogeneous sensors is challenging. The EU-funded DIDYMOS-XR project will implement technology to create improved large-scale digital twins, synchronised with the real world. DIDYMOS-XR will investigate and develop methods for data reconstruction and mapping from heterogeneous inputs, including static and mobile sensors, AI-based data fusion, scene understanding and rendering.

Project outputs

Publications

Domain	Type of output	Title	DOI URL
AI, Machine Learning & Data Science	Conference proceedings	Perception for Connected Autonomous Vehicles under Adverse Weather Conditions	https://doi.org/10.1109/IROS58592.2024.10801295
AI, Machine Learning & Data Science	Conference proceedings	User-Centric Evaluation Methods for Digital Twin Applications in Extended Reality	https://doi.org/10.1109/AIXVR63409.2025.00028
AI, Machine Learning & Data Science	Conference proceedings	MGSO: Monocular Real-Time Photometric SLAM with Efficient 3D Gaussian Splatting	https://doi.org/10.48550/ARXIV.2409.13055
Computer Vision, 3D Modeling & Rendering	Conference proceedings	HINT-3D: Human-in-the-Loop Interactive Test-Time Adaptation for 3D Segmentation	https://doi.org/10.5281/ZENODO.18491843
Computer Vision, 3D Modeling & Rendering	Conference proceedings	Deep 3D Geometric Saliency Estimation from Light Field Images	https://doi.org/10.1109/DSP58604.2023.10167953
Computer Vision, 3D Modeling & Rendering	Conference proceedings	HAME-NeRF: High Accuracy Mesh Extraction Leveraging Neural Radiance Fields	https://doi.org/10.1007/978-3-032-04968-1_28
Computer Vision, 3D Modeling & Rendering	Conference proceedings	Visual Localization in Complex Environments: Merging Traditional Geometry with Learning-Based Techniques	https://doi.org/10.1109/AIXVR63409.2025.00022
Computer Vision, 3D Modeling & Rendering	Conference proceedings	Volumetric Video Reconstruction and Communications: Toward a New Era of Interactive and Immersive Social Virtual Reality (VR) Experiences	https://doi.org/10.1145/3672406.3672421
Computer Vision, 3D Modeling & Rendering	Conference proceedings	Training a Segmentation-Based Visual Anonymization Service for Street Scenes	https://doi.org/10.1007/978-981-96-2074-6_26
Computer Vision, 3D Modeling & Rendering	Conference proceedings	GDNeRF: Generalizable Depth-based NeRF for sparse view synthesis	https://doi.org/10.1109/ICME59968.2025.11209482
Computer Vision, 3D Modeling & Rendering	Conference proceedings	HAL-NeRF: High Accuracy Localization Leveraging Neural Radiance Fields	https://doi.org/10.1109/AIXVR63409.2025.00024
Computer Vision, 3D Modeling & Rendering	Conference proceedings	Leveraging Anisotropic Error for Robust Point Cloud Registration	https://doi.org/10.1109/DSP65409.2025.11074865
Computer Vision, 3D Modeling & Rendering	Peer reviewed articles	Urban scene removal and completion	https://doi.org/10.3233/FAIA250603
Computer Vision, 3D Modeling & Rendering	Peer reviewed articles	Visual localization using implicit representations and particle filtering-based pose refinement	https://doi.org/10.5281/ZENODO.18468790
Computer Vision, 3D Modeling & Rendering	Peer reviewed articles	ExpPoint-MAE: Better Interpretability and Performance for Self-Supervised Point Cloud Transformers	https://doi.org/10.1109/ACCESS.2024.3388155
Computer Vision, 3D Modeling & Rendering	Peer reviewed articles	Interactive digital twins enabling responsible extended reality applications	https://doi.org/10.1038/S41598-025-17855-9
Computer Vision, 3D Modeling & Rendering	Peer reviewed articles	MaskUno: Switch-Split Block For Enhancing Instance Segmentation	https://doi.org/10.48550/ARXIV.2407.21498
Ethics, Society, Arts & Culture	Conference proceedings	Digital Twins for Extended Reality Tourism: User Experience Evaluation Across User Groups	https://doi.org/10.1007/978-3-031-97769-5_3
Ethics, Society, Arts & Culture	Peer reviewed articles	Reimagining Historical Exploration: Multi-User Mixed Reality Systems for Cultural Heritage Sites	https://doi.org/10.3390/APP15052854
Extended Reality (VR/AR/MR) & HCI	Conference proceedings	Cooperative Perception for Digital Twin Reconstruction*	https://doi.org/10.1109/AIXVR63409.2025.00023

Technological assets

Title	Type of Asset	Link / DOI	Description
Dataset for Learning Scene Semantics from Vehicle-centric Data for City-scale Digital Twins	Dataset	https://ieeexplore.ieee.org/document/10859207	Data utilized for learning scene semantics intended for city-scale digital twins.
ADAPT JR-Sim2Real dataset	Dataset	https://zenodo.org/records/12805642	Dataset associated with domain transfer for instance segmentations for AR scenes.
MGSO	Software / Algorithm	https://doi.org/10.48550/ARXIV.2409.13055	A monocular real-time photometric SLAM algorithm utilizing efficient 3D Gaussian Splatting.
HINT-3D	Software / Framework	https://doi.org/10.5281/ZENODO.18491843	A human-in-the-loop interactive test-time adaptation framework for 3D segmentation.
Visual localization using implicit representations	Software / Model	https://doi.org/10.5281/ZENODO.18468790	Software for visual localization utilizing implicit representations and particle filtering-based pose refinement.