SIAT Faculty Publications

Receive updates for this collection

Ratava's line: Emergent learning and design using collaborative virtual worlds

Date created: 

Ratava's Line is an online, 3D virtual world fashion and interactive
narrative project created collaboratively by students at both the
Fashion Institute of Technology (FIT) in New York City and at
Interactive Arts at Simon Fraser University (SFU) in Vancouver,
Canada, using emergent, collaborative 2D and 3D systems. This
distance learning project, developed over two months and
culminating in an online event in multiple, remote locations,
integrated three key design elements: the translation of original 2D
fashion designs from FIT students into 3D avatar space; exhibits of
artwork of student and professional artists from New York City and
Vancouver in virtual galleries; and creation of an interactive
narrative "fashion cyber-mystery" for online users to participate in
and solve in a culminating, cyber-physical event. The overall
project goal was to explore how online collaboration systems and
virtual environments can be used practically for distance learning,
fashion and virtual worlds design, development of new marketing
tools including virtual portfolios, and creation of cross cultural
online/physical events. The result of this process was an
interdisciplinary, cross-institutional, international effort in
collaborative design in virtual environments, and a successful
exercise in emergent, collaborative distance learning. © ACM, 2004. This is the author's version of the work. It is posted here by permission of ACM for your personal use. Not for redistribution. The definitive version was published in International Conference on Computer Graphics and Interactive Techniques, page 25. (2004).

Document type: 
Conference presentation

A social metaphor-based 3D virtual environment

Date created: 

Our design goal for OnLive Traveler was to develop a virtual
community system that emulates natural social paradigms,
allowing the participants to sense a tele-presence, the subjective
sensation that remote users are actually co-located within a
virtual space. Once this level of immersive "sense of presence"
and engagement is achieved, we believe an enhanced level of
socialization, learning, and communication are achievable.
OnLive Traveler is a client-server application allowing realtime
synchronous communication between individuals over the
Internet. The Traveler client interface presents the user with a
shared virtual 3D world, in which participants are represented by
avatars. The primary mode of communication is through multipoint,
full duplex voice, managed by the server.
We examine a number of very specific design and
implementation decisions that were made to achieve this goal
within platform constraints. We also will detail some observed
results gleaned from the virtual community and virtual learning
user-base, which has been using Traveler for several years.
© ACM, 2003. This is the author's version of the work. It is posted here by permission of ACM for your personal use. Not for redistribution. The definitive
version was published in International Conference on Computer Graphics and Interactive Techniques. Pages 1-2. (2003)

Document type: 
Conference presentation

Rembrandt's textural agency: A shared perspective in visual art and science

Date created: 

This interdisciplinary paper hypothesizes that Rembrandt developed new painterly techniques —
novel to the early modern period — in order to engage and direct the gaze of the observer.
Though these methods were not based on scientific evidence at the time, we show that they
nonetheless are consistent with a contemporary understanding of human vision. Here we propose
that artists in the late ‘early modern’ period developed the technique of textural agency —
involving selective variation in image detail — to guide the observer’s eye and thereby influence
the viewing experience. The paper begins by establishing the well-known use of textural agency
among modern portrait artists, before considering the possibility that Rembrandt developed these
techniques in his late portraits in reaction to his Italian contemporaries. A final section brings the
argument full circle, with the presentation of laboratory evidence that Rembrandt’s techniques
indeed guide the modern viewer’s eye in the way we propose.

Document type: 

Simulating face to face collaboration for interactive learning systems

Date created: 

The use of Problem-Based Learning (PBL) in medical education and other educational settings has escalated. PBL's strength in learning is mostly due to its collaborative and open-ended problem solving approach. Traditional PBL was designed to be used in live team environments rather than in an online setting. We describe research that allows for web-based PBL via geographically distributed physical locations that emphasize PBL's collaboration and open brainstorming approach using interactive web, gaming and simulation techniques. We describe Interactive Face Animation - Comprehensive Environment (iFACE) which allows for expressive voice based character agents along with Collaborative Online Multimedia Problem-based Simulation Software (COMPS) which integrates iFace within a customizable web-based collaboration system. COMPS creates an XML-based multimedia communication medium that is effective for group based case presentations, discussions and other PBL activities.

Document type: 
Conference presentation

Facial actions as visual cues for personality

Date created: 

What visual cues do human viewers use to assign personality characteristics to animated characters?
While most facial animation systems associate facial actions to limited emotional states or speech content,
the present paper explores the above question by relating the perception of personality to a wide variety of
facial actions (e.g., head tilting/turning, and eyebrow raising) and emotional expressions (e.g., smiles and
frowns). Animated characters exhibiting these actions and expressions were presented to human viewers in
brief videos. Human viewers rated the personalities of these characters using a well-standardized adjective
rating system borrowed from the psychological literature. These personality descriptors are organized in a
multidimensional space that is based on the orthogonal dimensions of Desire for Affiliation and Displays of
Social Dominance. The main result of the personality rating data was that human viewers associated
individual facial actions and emotional expressions with specific personality characteristics very reliably. In
particular, dynamic facial actions such as head tilting and gaze aversion tended to spread ratings along the
Dominance dimension, whereas facial expressions of contempt and smiling tended to spread ratings along
the Affiliation dimension. Furthermore, increasing the frequency and intensity of the head actions increased
the perceived Social Dominance of the characters. We interpret these results as pointing to a reliable link
between animated facial actions/expressions and the personality attributions they evoke in human viewers.
The paper shows how these findings are used in our facial animation system to create perceptually valid
personality profiles based on Dominance and Affiliation as two parameters that control the facial actions of
autonomous animated characters.

Document type: 

Painterly rendered portraits from photographs using a knowledge-based approach

Date created: 

Portrait artists using oils, acrylics or pastels use a specific but open human vision methodology to create a painterly portrait
of a live sitter. When they must use a photograph as source, artists augment their process, since photographs have: different
focusing - everything is in focus or focused in vertical planes; value clumping - the camera darkens the shadows and lightens
the bright areas; as well as color and perspective distortion. In general, artistic methodology attempts the following: from the
photograph, the painting must 'simplify, compose and leave out what?s irrelevant, emphasizing what?s important'. While
seemingly a qualitative goal, artists use known techniques such as relying on source tone over color to indirect into a
semantic color temperature model, use brush and tonal "sharpness" to create a center of interest, lost and found edges to
move the viewers gaze through the image towards the center of interest as well as other techniques to filter and emphasize.
Our work attempts to create a knowledge domain of the portrait painter process and incorporate this knowledge into a multispace
parameterized system that can create an array of NPR painterly rendering output by analyzing the photographic-based
input which informs the semantic knowledge rules.

Document type: 

Knowledge based approach to modeling portrait painting methodology

Date created: 

Traditional portrait artists use a specific but open human vision methodology to create a
painterly portrait of a live or photographed sitter. Portrait artists attempt to simplify,
compose and leave out what is irrelevant, emphasizing what is important. While seemingly
a qualitative pursuit, artists use known but open techniques such as relying on source tone
over colour to indirect into a colour temperature model, using "sharpness" to create a
centre of interest, using edges to move the viewers gaze, and other techniques to filter and
emphasize. Our interdisciplinary work attempts to compile and incorporate this portrait
painter knowledge into a multi-space parameterized system that can create an array of
painterly rendering output.

Document type: 
Conference presentation

Blending Science Knowledge and AI Gaming Techniques for Experiential Learning

Date created: 

This paper addresses the scientific, design and experiential learning issues in creating an extremely realistic 3D interactive of a wild beluga whale pod for a major aquarium that is situated next to a group of real beluga whales in an integrated marine mammal exhibit. The Virtual Beluga Interactive was conceived to better immerse and engage visitors in complicated educational concepts about the life of wild belugas compared to what is typically possible via wall signage or a video display, thereby allowing them to interactively experience wild whale behavior and hopefully have deeper insights into the life of beluga whales. The gaming simulation is specifically informed by research data from live belugas, (e.g. voice recordings
tied to mother/calf behavior) and from interviews with marine mammal scientists and education staff at the Vancouver Aquarium. The collaborative user interface allows visitors to engage in educational "what-if" scenarios of wild beluga emergent behavior using techniques from advanced gaming systems, such as physically based animation, real-time photo-realistic rendering, and artificial intelligence algorithms.

Document type: 
Conference presentation

Face modeling and animation language for MPEG-4 XMT framework

Date created: 

This paper proposes FML, an XML-based face modeling and animation language. FML provides a structured content description method for multimedia presentations based on face animation. The language can be used as direct input to compatible players, or be compiled within MPEG-4 XMT framework to create MPEG-4 presentations. The language allows parallel and sequential action description, decision-making and dynamic event-based scenarios, model configuration, and behavioral template definition. Facial actions include talking, expressions, head movements, and low-level MPEG-4 FAPs. The ShowFace and iFACE animation frameworks are also reviewed as example FML-based animation systems.

Document type: 

Face, portrait, mask: The virtuality of the synthetic face

Date created: 

With new technological artistic tools that allow us to author 3D computer generated faces that can be as real or as abstract or as iconified as we choose, what aesthetic and cultural communication language do we elicit? Is it the historically rich language of the fine art portrait ? the abstracted artifact of the human face? What happens when this portrait animates, conveying lifelike human facial emotion ? does it cease to be a portrait and instead moves into the realm of embodied face ? when it begins to emote and feel and possibly react to the viewer? Is it then more in the language of the animated character, or as we make it photo-realistic, the language of the video actor with deep dramatic back-story or simply as real as a person on the other side of the screen? A viewer can not be rude to a portrait but can feel that they are being rude to an interactive character in an art installation. When does it become not an embodied face nor portrait but a mask ? the icon that speaks of face but is never embodied? Masks also have a deep cultural, historic and ethnic language far different than that of human faces or art portraits. More eastern compared to the western portrait. Iconized faces (i.e. the smiley face or the emoticon face) takes the mask full through to the western modern world of technology.

Document type: