Understanding RNN States with predictive semantic encodings and adaptive representations

Recurrent Neural Networks are an effective and prevalent tool used to model sequential data such as natural language text. However, their deep nature and massive number of parameters pose a challenge for those intending to study precisely how they work. This is especially the case for researchers with the expertise to understand the mathematics behind these models at a macroscopic level, but who often lack the tools to expose the microscopic details of what information they internally represent. We present a combination of visualization and analysis techniques to show some of the inner workings of Recurrent Neural Networks and facilitate their study at a fine level of detail. Specifically, we use an auxiliary model to interpret the meaning of hidden states with respect to the task level outputs. A visual encoding is designed for this model that is quickly interpreted and relates to other elements of the visual design. We also introduce a consistent visual representation for vector data that is adaptive with respect to the available visual space. When combined, these techniques provide a unique insight into RNN behaviours, allowing for both architectural and detail views to be visualized in concert. These techniques are leveraged in a fully interactive visualization tool which is demonstrated to improve our understanding of common Natural Language Processing tasks.