Acquiring stylized motor skills for physics-based characters

Character animation is a core research topic in computer graphics, which studies synthesizing realistic movements for virtual characters. Physics-based character animation utilizes physics simulation to generate physically plausible character motions. One long-standing goal of physics-based character animation is to equip simulated characters with vast and agile motor skills. Most recent physics-based animation methods learn these agile and impressive motion skills by imitating motion capture data or human demonstrations. However, most of these methods mainly focus on motion tracking, and cannot discover novel skills that are visually fundamentally different from reference motions. Therefore, these imitation-based methods cannot be applied to motor skill learning tasks where high-quality motion capture data is not available, such as challenging sports movements. In this thesis, we present several computational methods that enable simulated characters to learn diverse and stylized motor skills in the absence of task-specific motion capture data. First, for motion tasks where a limited number of reference motions are available, we present a deep reinforcement learning framework to help simulated characters explore and develop stylized motor skills from reference motions. This system can be used to enrich the variations of the motor skills performed by physics-based characters. Second, for challenging motion tasks where no reference motions are available such as athletic jumping, we design a deep reinforcement learning framework to discover diverse high jumping strategies for simulated characters. Our framework can discover many well-known jumping strategies, like Fosbury flop and Scissor kick, without using task-specific mocap data. Third, we further extend and apply our method developed for full-body high jumping tasks to hand tool manipulation tasks, where we present a learning and control system to enable simulated hands to use chopsticks for object grasping. We demonstrate dexterous object relocation skills with chopsticks in different styles, holding positions and for various hand morphologies. Finally, motivated by the observation that many skill discovery problems can be formulated as hyperparameter optimization problems, we propose a novel multi-fidelity Bayesian Optimization algorithm to optimize hyperparameters of deep reinforcement learning-based animation systems. Our algorithm significantly outperforms state-of-the-art hyperparameter optimization methods applicable for physics-based character animation.