A Hierarchical Deep Temporal Model for Group Activity Recognition

In group activity recognition, the temporal dynamics of the whole activity can be inferredbased on the dynamics of the individual people representing the activity. We build a deepmodel to capture these dynamics based on LSTM (long-short term memory) models. Tomake use of these observations, we present a 2-stage deep temporal model for the groupactivity recognition problem. In our model, a LSTM model is designed to represent actiondynamics of individual people in a sequence and another LSTM model is designed to aggregatehuman-level information for whole activity understanding. We evaluate our modelover two datasets: the collective activity dataset and a new volleyball dataset. Experimentalresults demonstrate that our proposed model improves group activity recognitionperformance as compared to baseline methods.