View source for XReco

=== XReco Project === 
{| class='wikitable' style='margin:auto' 
|-
! CORDIS Reference !! Start date !! End date !! Coordinator
|- 
| https://cordis.europa.eu/project/id/101070250 || 01/09/2022 || 31/08/2025 || DEUTSCHE WELLE / Germany
|} 
=== Project description ===
While media organisations increasingly support non-linear experiences for the consumer, those are still limited to single channels and media domains. Although several media organisations have recently succeeded in breaking data silos, data sharing is mostly limited to the organisation. Thus, there are challenges for producing content feeding multiple channels with different granularities and structures, mainly related to data discovery, management and (re-)use. XRECO will create a new data-driven ecosystem for the media industry, focusing on facilitating data sharing, search and discovery and supporting creation of news and entertainment content, in particular, the creation and (re-)use of location-related 2D and 3D assets and the creation of XR experiences. The ecosystem core, represented by a Neural Media Repository (NMR), will foster inter-organisation content sharing and provide increased access to content for media creators, considering novel data monetization and rights management policies. A set of AI-based media transformation services are built around the NMR to produce novel media- and XR experiences, including 3D neural reconstructions, neural based device localisation, image stitching, de-/re-lighting and holoportation. The developed technology will be validated in use case scenarios for (i) the news media for XR-based broadcasting and automatic and customized multitarget news publishing, and for (ii) location-based information and entertainment content, with applications in tourism and the automotive industry.

=== Project outputs ===
==== Publications ====
{| class="wikitable sortable"
! Domain !! Type of output !! Title !! DOI URL
|-
| AI, Machine Learning & Data Science || Conference proceedings || Efficient Few-Shot Incremental Training for Landmark Recognition || https://doi.org/10.1145/3672406.3672414
|-
| Computer Vision, 3D Modeling & Rendering || Book chapters || Image Valuation in NeRF-Based 3D Reconstruction || https://doi.org/10.1007/978-3-032-04968-1_32
|-
| Computer Vision, 3D Modeling & Rendering || Book chapters || Complete Convolutional Neural Networks Environment for Computer Vision Problems With Nvidia Drive AGX Xavier || https://doi.org/10.1007/978-3-031-70248-8_7
|-
| Computer Vision, 3D Modeling & Rendering || Book chapters || Synthetic Football Sprite Animations Learned Across the Pitch || https://doi.org/10.1007/978-3-031-41774-0_48
|-
| Computer Vision, 3D Modeling & Rendering || Conference proceedings || Descriptor Impact on Multimodal 3D Retrieval || https://doi.org/10.5281/ZENODO.11942398
|-
| Computer Vision, 3D Modeling & Rendering || Conference proceedings || A Dataset and Metric for Textual Video Content Description || https://doi.org/10.1145/3746027.3758224
|-
| Computer Vision, 3D Modeling & Rendering || Conference proceedings || Urban Scene Removal and Completion || https://doi.org/10.3233/FAIA250603
|-
| Computer Vision, 3D Modeling & Rendering || Conference proceedings || Analysis of Objective 3D Mesh Quality Metrics for Cultural Heritage || https://doi.org/10.1109/QOMEX65720.2025.11219903
|-
| Computer Vision, 3D Modeling & Rendering || Conference proceedings || Untethered Real-Time Immersive Free Viewpoint Video || https://doi.org/10.1145/3652212.3652214
|-
| Computer Vision, 3D Modeling & Rendering || Conference proceedings || Multimodal Understanding: Investigating the Capabilities of Large Multimodal Models for Object Detection in XR Applications || https://doi.org/10.1145/3688866.3689126
|-
| Computer Vision, 3D Modeling & Rendering || Conference proceedings || Volumetric Video Reconstruction and Communications: Toward a New Era of Interactive and Immersive Social Virtual Reality (VR) Experiences || https://doi.org/10.1145/3672406.3672421
|-
| Computer Vision, 3D Modeling & Rendering || Conference proceedings || Semi -Automated Digital Human Production for Enhanced Media Broadcasting || https://doi.org/10.1109/GEM61861.2024.10585601
|-
| Computer Vision, 3D Modeling & Rendering || Conference proceedings || Multimedia Information Retrieval in XR || https://doi.org/10.1145/3664647.3689176
|-
| Computer Vision, 3D Modeling & Rendering || Conference proceedings || 3DMSE: An Interactive 3D Media Search Engine || https://doi.org/10.1145/3652583.3657593
|-
| Computer Vision, 3D Modeling & Rendering || Conference proceedings || Is Real-time Deep Learning-based Monocular Depth Estimation accurate for Multi-Camera Setups? || https://doi.org/10.1109/ICCT-EUROPE63283.2025.11157669
|-
| Computer Vision, 3D Modeling & Rendering || Conference proceedings || Analysis and Development of Deep Learning Depth Estimation Techniques for Volumetric Capture and Free Viewpoint Video || https://doi.org/10.1145/3625468.3652913
|-
| Computer Vision, 3D Modeling & Rendering || Conference proceedings || Real-Time Free Viewpoint Video for Immersive Videoconferencing || https://doi.org/10.1109/QOMEX61742.2024.10598259
|-
| Computer Vision, 3D Modeling & Rendering || Conference proceedings || Multimodality in Media Retrieval || https://doi.org/10.1145/3652583.3657583
|-
| Computer Vision, 3D Modeling & Rendering || Conference proceedings || Exploring Image Search on Quantum Computing Systems || https://doi.org/10.5220/0013562200004525
|-
| Computer Vision, 3D Modeling & Rendering || Conference proceedings || Towards a Universal Query Representation for Multimodal Information Retreival || https://doi.org/10.1145/3746027.3758155
|-
| Computer Vision, 3D Modeling & Rendering || Conference proceedings || Multimedia Retrieval in and for XR || https://doi.org/10.1145/3652583.3658421
|-
| Computer Vision, 3D Modeling & Rendering || Conference proceedings || Enabling Domain Experts to Train Efficient Few-Shot Incremental Landmark Recognition || https://doi.org/10.1109/CBMI62980.2024.10859238
|-
| Computer Vision, 3D Modeling & Rendering || Conference proceedings || XReco Platform and RAI News Media Demonstrator || https://doi.org/10.1145/3746027.3761840
|-
| Computer Vision, 3D Modeling & Rendering || Conference proceedings || Open-Source Multimedia Retrieval with vitrivr-engine || https://doi.org/10.1145/3746027.3756874
|-
| Computer Vision, 3D Modeling & Rendering || Conference proceedings || Ubervvald: Advanced Object Detection Library for Optimizing Complex Convolutional Neural Networks (CNNs) || https://doi.org/10.1007/978-981-96-5887-9_14
|-
| Computer Vision, 3D Modeling & Rendering || Conference proceedings || Nostrils and Mouth Detection for Drivers Using Convolutional Neural Networks with Automatically Generated Ground Truth Data || https://doi.org/10.1109/CSCI58124.2022.00265
|-
| Computer Vision, 3D Modeling & Rendering || Conference proceedings || Ground Truth Data Generator in Automotive Infrared Sensor Vision Problems Using a Minimum Set of Operations || https://doi.org/10.1109/SYNASC61333.2023.00039
|-
| Computer Vision, 3D Modeling & Rendering || Conference proceedings || A new Retrieval Engine for vitrivr || https://doi.org/10.1007/978-3-031-53302-0_28
|-
| Computer Vision, 3D Modeling & Rendering || Conference proceedings || Improving Query and Assessment Quality in Text-Based Video Retrieval Evaluation || https://doi.org/10.1145/3591106.3592281
|-
| Computer Vision, 3D Modeling & Rendering || Conference proceedings || Real-Time Layered View Synthesis for Free-Viewpoint Video from Unreliable Depth Information || https://doi.org/10.1145/3592834.3592881
|-
| Computer Vision, 3D Modeling & Rendering || Conference proceedings || Free-form Multi-Modal Multimedia Retrieval (4MR) || https://doi.org/10.1007/978-3-031-27077-2_58
|-
| Computer Vision, 3D Modeling & Rendering || Conference proceedings || Multimedia Retrieval in Mixed Reality: Leveraging Live Queries for Immersive Experiences || https://doi.org/10.1109/AIxVR59861.2024.00048
|-
| Computer Vision, 3D Modeling & Rendering || Conference proceedings || Mining Landmark Images for Scene Reconstruction from Weakly Annotated Video Collections || https://doi.org/10.1007/978-3-031-53302-0_12
|-
| Computer Vision, 3D Modeling & Rendering || Conference proceedings || Synthetic Football Sprite Animations Learned Across the Pitch. || https://doi.org/10.1007/978-3-031-41774-0_48
|-
| Computer Vision, 3D Modeling & Rendering || Conference proceedings || Subjective Evaluation of Dynamic Point Clouds: Impact of Compression and Exploration Behavior || https://doi.org/10.23919/EUSIPCO58844.2023.10290086
|-
| Computer Vision, 3D Modeling & Rendering || Conference proceedings || Multimodal 3D Object Retrieval || https://doi.org/10.5281/zenodo.10226588
|-
| Computer Vision, 3D Modeling & Rendering || Peer reviewed articles || Interactive Multimodal Video Search: An Extended Post-Evaluation for the VBS 2022 Competition || https://doi.org/10.1007/s13735-024-00325-9
|-
| Computer Vision, 3D Modeling & Rendering || Peer reviewed articles || Multimedia Systems || https://doi.org/10.5167/uzh-236035
|-
| Computer Vision, 3D Modeling & Rendering || Peer reviewed articles || An Assessment of the Stereo and Near-Infrared Camera Calibration Technique Using a Novel Real-Time Approach in the Context of Resource Efficiency || https://doi.org/10.3390/PR13041198
|-
| Cybersecurity, Privacy & Blockchain || Conference proceedings || Data as Remuneration in Digital Copyright Licensing: Some Reflections on the Concept of ‘Appropriate and Proportionate Remuneration’ Under Art. 18 EU Directive 2019/790 in the Data Era || https://doi.org/10.5281/ZENODO.18482948
|-
| Cybersecurity, Privacy & Blockchain || Conference proceedings || Secure, Dynamic and Uncomplicated Licensing of Movies on a Blockchain Infrastructure || https://doi.org/10.1109/ICOIN56518.2023.10049017
|-
| Extended Reality (VR/AR/MR) & HCI || Peer reviewed articles || IEEE Access || https://doi.org/10.5167/UZH-261490
|-
| Extended Reality (VR/AR/MR) & HCI || Peer reviewed articles || Delay Threshold for Social Interaction in Volumetric eXtended Reality Communication || https://doi.org/10.1145/3651164
|-
| Networks, Cloud & Telecommunications (5G/6G) || Conference proceedings || VERGE in CBMI2024 || https://doi.org/10.5281/ZENODO.10652893
|-
| Networks, Cloud & Telecommunications (5G/6G) || Conference proceedings || VERGE in VBS 2024 || https://doi.org/10.5281/zenodo.10652893
|}

==== Technological assets ====
{| class="wikitable sortable"
! Title !! Type of Asset !! Link / DOI !! Description
|-
| XR and Media Transformation APIs and Authoring Tools || APIs / Tools || https://xreco.eu/deliverables/#toc_D41_XR_and_Media_Transformation_Services_API_and || APIs integrating vertical technologies for XR media transformation and content creation.
|-
| Textual Video Content Dataset || Dataset / Metric || https://doi.org/10.1145/3746027.3758224 || A dataset and corresponding metric created specifically for textual video content description.
|-
| vitrivr-engine || Open-Source Engine || https://doi.org/10.1145/3746027.3756874 || An open-source multimedia retrieval engine for content and similarity searches.
|}