How To Not Train Your Dragon: Training-free Embodied Object Goal Navigation with Semantic Frontiers

Junting Chen*

Guohao Li*

Suryansh Kumar

Bernard Ghanem

Fisher Yu

[Paper]

[Video]

[GitHub (coming soon)]

Published on RSS 2023

With semantic reasoning on the partial scene reconstruction, our agent can leverage the general scene layout priors and predict the possible target locations. As a result, our agent can navigate to the target object with shorter paths in unseen environments.

Abstract

Summary: This work presnets a modular and training-free system to tackle the object goal navigation problem. The system builds a structured scene representation during active exploration, propagates semantics in the scene graphs to infer the target location, and introduces those semantics to the geometric frontiers. With semantic frontiers, the agent navigate to the most promising areas to search for the goal object and avoid detours in unseen environments.

Intuition of this work. ObjectNav can be decomposed into 1) semantic questiosn: inferring the potential position of the target object in the scene and 2) geometric question: point-to-point navigation. These two questions can be solved by semantic reasoning and classic planning separately on different representations.

Video

Method

The proposed method builds a structured scene representation on the fly, which consists of a semantic point cloud, a 2D occupancy map, and a spatial scene graph. Then the scene graph is used to propagate semantics to the geometric frontiers. With semantic frontiers, the agent can navigate to the most promising areas to search for the goal object and avoid detours in unseen environments.

[Code (coming soon)]

Paper and Bibtex

Junting Chen, Guohao Li, Suryansh Kumar, Bernard Ghanem, Fisher Yu.
How To Not Train Your Dragon: Training-free Embodied
Object Goal Navigation with Semantic Frontiers.
In Conference Robotics: Science and Systems, 2023.
(hosted on ArXiv)

				
	@misc{chen2023train,
		title={How To Not Train Your Dragon: Training-free Embodied Object Goal Navigation with Semantic Frontiers}, 
		author={Junting Chen and Guohao Li and Suryansh Kumar and Bernard Ghanem and Fisher Yu},
		year={2023},
		eprint={2305.16925},
		archivePrefix={arXiv},
		primaryClass={cs.CV}
	}

Acknowledgements

This template was originally made by Phillip Isola and Richard Zhang for a colorful ECCV project; the code can be found here.