Weakly Supervised Models For Recognizing And Clustering High-level Complex Events In Video

In the last decade, we have witnessed exponential growth of visual content in internet social media such as YouTube or Facebook. Developing automatic analysis tools is becoming essential to summarize and retrieve desired videos from a large collection on the internet. In this thesis, we propose frameworks for recognizing and clustering high-level events in unconstrained internet videos. The events considered here are complex events such as ``wedding ceremony'' or ``getting a vehicle unstuck'' that can take place in various ways in different scenes. The main challenges in analysis of these events are complex temporal and spatial structure, high intra-class variation, camera motion, and background clutter.First, we present a compositional model for video event recognition. A video is modeled using a collection of both global and segment-level features and kernel functions are employed for similarity comparisons. The locations of salient, discriminative video segments are treated as a latent variable, allowing the model to explicitly ignore portions of the video that are unimportant for classification. A novel multiple kernel learning (MKL) latent support vector machine (SVM) is defined, that is used to combine and re-weight multiple feature types in a principled fashion while simultaneously operating within the latent variable framework.Second, we propose Flip Support Vector Machine and Flip Max Margin Clustering frameworks for classifying and clustering videos using noisy structured tag labels. The main idea of these frameworks is that annotated tag labels in internet visual data are noisy. These proposed models gain robustness against tag noise by explicitly considering the possibility of label change while training max-margin based classification or clustering models.