Learning Person Trajectory Features for Sports Video Analysis

We propose a generic deep model to learn features describing person trajectories. This network uses layers of 1D temporal convolutions over person location inputs. The network can model the patterns of motion exhibited by people when performing different activities. These trajectory features are used in a two-stream deep model that takes as input both visual data and person trajectories for sports video analysis. Our model utilizes one stream to learn the visual temporal dynamics from video clips and the other stream to learn the space-time dependencies from trajectories. We evaluate our trajectory feature learning model on data from NBA basketball games. We also utilize a dataset from NHL hockey games, which contains broadcast videos and uses state of the art automatic camera calibration, human detection, and tracking algorithms to estimate player positions in world coordinates. Experiments show that person trajectories can provide strong spatio-temporal cues, which improve performance over baselines that do not incorporate trajectory data.