Skip to main content

Towards embodied agents for navigating and recognizing human activity

Resource type
Thesis type
(Thesis) Ph.D.
Date created
2024-04-24
Authors/Contributors
Author: Hu, Sha
Abstract
In this dissertation, we aim to make progress toward building embodied agents that coexist with humans and assist humans to perform various tasks physically. We focus on developing visual perception and navigational capabilities for embodied agents in human environments, and we explore three aspects of visual perception and navigation - human activity recognition, crowd navigation and agent morphology design. We investigate three research questions to approach the above three aspects of perception and navigation: (1) Can we improve an embodied agent's visual recognition of an ongoing human activity by utilizing its mobility? (2) Can we develop safe and efficient navigation behaviors of embodied agents by anticipating human future trajectories and modelling relations? (3) Can we efficiently design an agent's morphology automatically for optimal task performance? To address these research questions, we present three works that provide possible solutions. First, we introduce the embodied human activity recognition problem, where an agent moves in a 3D environment to classify ongoing human activities. We propose a reinforcement learning approach that learns a policy controlling the agent's movements over time, with the goal of acquiring new views that lead to accurate human activity classification. Second, we present a relational graph learning approach for robotic crowd navigation using model-based deep reinforcement learning that plans actions by looking into the future. Third, we present a continuous multi-fidelity Bayesian Optimization framework that efficiently utilizes computational resources via low-fidelity evaluations for automatic morphology design.
Document
Extent
105 pages.
Identifier
etd23006
Copyright statement
Copyright is held by the author(s).
Permissions
This thesis may be printed or downloaded for non-commercial research and scholarly purposes.
Supervisor or Senior Supervisor
Thesis advisor: Mori, Greg
Language
English
Member of collection
Download file Size
etd23006.pdf 6.1 MB

Views & downloads - as of June 2023

Views: 17
Downloads: 1