Tagging with Movement: Somatic Strategies for Digital Image Classification

This dissertation presents an exploratory investigation of ways to incorporate somatic, or movement, experience into interaction with computers. The research centers on the concept design of a hypothetical application that uses movement instead of text to generate tags for digital content. These kinesthetic tags provide an alternate approach to interaction with digital images, one that prioritizes somatic perception over visual perception. Imagery has a long history of use in movement-based disciplines for teaching, conditioning, and heightening awareness of somatic experience. Kinesthetic tagging provided a focus for investigating this connection by providing insight into process through which people enact their relationship with visual media, exploring contents, concepts, and meanings. The research study addressed a gap in the literature pertaining to the integration of functional and experiential movement. Although a kinesthetic tagging application was not developed as part of this research, the concept served to facilitate the exploration of movement experience and its potential use for interaction. This exploration took place in a two-day movement-based workshop in which participants focused on the investigation of movement qualities derived from the concept of Effort as defined in the Laban Movement Analysis (LMA) framework. LMA Effort factors describe the experiential content of movements through the expressive qualities they exhibit. This feature provides a systematic method for linking observable movements with peoples’ somatic states, making the Effort factors useful tools for investigating movement experience. The research workshop incorporated various methods from design, performance, and Somatics, and utilized a modified version of grounded theory for data analysis. The outcome of the analysis is a conceptual framework explicating how users’ approach the task of enacting visual content using expressive movement. This framework identifies three modes of connection and seven mechanisms of interaction that inform a user’s process. A set of hypotheses relating to the process of enactment are generated, as well as a set of design considerations for a kinesthetic tagging system. The dissertation concludes with the articulation of five areas that would benefit from the integration of functional and experiential movement.