Digital Gesture Library, an archive that displays Chinese traditional puppet shows as linear multi-perspective videos (puppetry gestures, puppet movement, and the whole performance) and multi-dimensional videos (gestures, genres, information about theatre and puppeteers). This prototype gestural archive supports audiences with varied cultural backgrounds to cross cultural barriers (e.g. language, dialect, local background) to understanding and appreciating the performance. It also allows participants to appreciate the gestural repertoire of Chinese puppetry and engages participants to interact using a multimedia system. The application is composed of two parts: (1) the multimedia system and (2) a tangible interface.
Exploring How Interactive Technology Enhances Gesture-Based Expression and Engagement
I designed and developed prototype processing equipment that was based on the oscP5 and SimpleOpenNI open-source libraries. This software was connected to a Microsoft Kinect to digitally capture interactive gestures. Two Kinect devices were utilised to capture gestures from two participants separately and accurately; these gestures would then be presented on a screen in real-time. Meanwhile, these gestures formed interactive calligraphy artwork through collaboration between the participants.
Enhancing the Appreciation of Traditional Chinese Painting Using Interactive Technology
With consideration of affordances for the operability and portability of gestures, I chose to develop a tablet application based on the iOS and Android system. A key design feature was the use of colours and subjects as two key representative elements which were extracted from the data from the cultural appreciation study with tangible interaction to support culturally diverse participants. To address the research questions, the tablet-based application needed: (1) to integrate the colours and subjects to build an element-based archive able to provide multiple colours, themes, and styles; (2) to offer users diverse elements from the archive to encourage them to create their own artwork; and (3) to serve as a platform that would motivate users to make collaborative artwork and share their artistic thinking of traditional Chinese painting.