Skip to main navigation Skip to search Skip to main content

Easy3D-Labels: Supervising Semantic Occupancy Estimation With 3D Pseudo-Labels for Automotive Perception

  • University of Limerick

Research output: Contribution to journal › Article › peer-review

Abstract

In perception for automated vehicles, safety is critical not only for the driver but also for other agents in the scene, particularly vulnerable road users such as pedestrians and cyclists. Previous representation methods, such as Bird’s Eye View, collapse vertical information, leading to ambiguity in 3D object localisation and limiting accurate understanding of the environment for downstream tasks such as motion planning and scene forecasting. In contrast, semantic occupancy provides a full 3D representation of the surroundings, addressing these limitations. Furthermore, self-supervised semantic occupancy has seen increased attention in the automated vehicle domain. Unlike supervised methods that rely on manually annotated data, these approaches use 2D pseudo-labels, improving scalability by reducing the need for labour-intensive annotation. Consequently, such models employ techniques such as novel view synthesis, cross-view rendering, and depth estimation to allow for model supervision against the 2D labels. However, such approaches often incur high computational and memory costs during training, especially for novel view synthesis. To address these issues, we propose Easy3D-Labels, which are 3D pseudo-ground-truth labels generated using Grounded-SAM and Metric3Dv2, with temporal aggregation for densification, permitting supervision directly in 3D space. Easy3D-Labels can be readily integrated into existing models to provide model supervision, yielding substantial performance gains, with mIoU increasing by 45% and RayIoU by 49% when applied to OccNeRF on the Occ3D-nuScenes dataset. Additionally, we introduce EasyOcc, a streamlined model trained solely on these 3D pseudo-labels, avoiding the need for complex rendering strategies and achieving 15.7 mIoU on Occ3D-nuScenes. Easy3D-Labels improve scene understanding by reducing object duplication and enhancing depth estimation accuracy, as reflected by improvements in the RayIoU metric. These findings highlight the importance of foundation models, temporal information, and 3D loss formulation in self-supervised learning for comprehensive scene understanding.

Original languageEnglish
Pages (from-to)1964-1978
Number of pages15
JournalIEEE Open Journal of Vehicular Technology
Volume7
DOIs
Publication statusPublished - 2026

UN SDGs

This output contributes to the following UN Sustainable Development Goals (SDGs)

  1. SDG 3 - Good Health and Well-being
    SDG 3 Good Health and Well-being

Keywords

  • 3D labelling
  • Automated vehicles
  • self-supervision
  • semantic occupancy

Fingerprint

Dive into the research topics of 'Easy3D-Labels: Supervising Semantic Occupancy Estimation With 3D Pseudo-Labels for Automotive Perception'. Together they form a unique fingerprint.

Cite this