Testing the usability of Microsoft Photosynth, an application with a complex and unfamiliar conceptual model

How do we test the usability of software whose design features an unprecedented and complex conceptual model? Current usability practices ask users for objective and subjective responses, which do not explicitly test how well a conceptual model is understood. An example of software with a complex conceptual model is Microsoft Photosynth. This is a Web-based tool that stitches together multiple photographs of a particular object into a three dimensional view. I tested its usability in two phases; firstly I asked users to explore a synth and answer questions that would stimulate thought about the conceptual model, and secondly I asked them to perform a complex task that would require a thorough understanding of this model. This two-phase approach allowed me to distinguish usability bugs that were due to misunderstandings of the conceptual model from usability bugs due to the interface’s design.