Memory-Augmented Re-Completion for 3D Semantic Scene Completion

Abstract

3D semantic scene completion (SSC) aims to predict both the occupancy and semantic labels of a 3D scene from a single depth image. While existing methods have shown promising results, they often struggle with complex scenes and incomplete observations. In this paper, we propose MARE (Memory-Augmented RE-completion), a novel framework that leverages historical information to enhance completion accuracy. Our key insight is that previously completed scenes can provide valuable context for current predictions, especially in scenarios with similar structural patterns. MARE introduces a memory bank that stores and retrieves relevant scene completions, enabling the model to learn from past experiences and refine its predictions through an iterative re-completion process. Experimental results demonstrate that MARE achieves state-of-the-art performance on benchmark datasets, with significant improvements in both geometric accuracy and semantic consistency.

Key Contributions

• Introduced MARE, a novel memory-augmented framework for 3D semantic scene completion that effectively utilizes historical scene information.
• Developed a dynamic memory bank mechanism that stores and retrieves relevant scene completions to guide current predictions.
• Designed an iterative re-completion process that progressively refines scene predictions by incorporating memory-based context.
• Demonstrated superior performance on benchmark datasets, achieving state-of-the-art results in both geometric and semantic metrics.

Related Links

• Paper PDF
• Code
• Project Page

Citation

If you find this work helpful, please consider citing:

@inproceedings{tseng2025memory,
  title={Memory-Augmented Re-Completion for 3D Semantic Scene Completion},
  author={Tseng, Yu-Wen and Liu, Hou-I and Chang, Kai-Cheng and Wang, Pin-Jyun and Shuai, Hong-Han and Cheng, Wen-Huang},
  booktitle={Proceedings of the AAAI Conference on Artificial Intelligence},
  year={2025}
}

Recommended citation: Yu-Wen Tseng, Hou-I Liu, Kai-Cheng Chang, Pin-Jyun Wang, Hong-Han Shuai, and Wen-Huang Cheng. "Memory-Augmented Re-Completion for 3D Semantic Scene Completion." In Proceedings of the AAAI Conference on Artificial Intelligence (AAAI), 2025.
Download Paper