Learning Person Trajectory Representations for Team Activity Analysis

Activity analysis in which multiple people interact across a large space is challenging due to the interplay of individual actions and collective group dynamics. We propose an end-to-end generic approach for learning person trajectory representations for group activity analysis. The learned representations encode rich spatio-temporal dependencies and capture useful motion patterns for recognizing individual events, as well as characteristic group dynamics that can be used to identify groups from their trajectories alone. We develop our deep learning approach in the context of team sports, which provide well-defined sets of events (e.g. pass, shot) and groups of people (teams). We evaluate our model on NBA basketball and NHL hockey games datasets. Analysis of events and team formations using these two sports datasets demonstrate the generality of our approach. Experiments show that our model is capable of (1) capturing strong spatio-temporal cues for recognizing events in hockey dataset (2) capturing distinctive group dynamics for identifying group identity.