Controlling sound and animation with gestures

Digital Futures grad student and visual artist Afaq Ahmed Karadia, has designed a machine learning system which uses cognitive data and gesture technologies to recognize and interpret movements of the human body to create sound and animation.

Afaq’s performance uses a “virtual instrument” controlled by gesture-based movements. The instrument creates sound and these sounds create visualizations in real-time. These real-time visualizations replaces pre-made animations using shadow and light.

The prototype is part of his larger research project that examines the non-functional characteristics of gesture, such as expressivity, which remains a challenge for computers.

Template: 
Inline Image Template
Embed Video: 

QUIPUCAMAYOC

Quipucamayoc is an interactive art and technology project that merges a range of contemporary art forms – including 3D gaming, electroacoustical music, dance, experimental theatre, and wearable design – to construct a prototype communication network. This network is not based in text or language but is instead embodied, performative, and sensorial.

The network joins two Andean communities: one in Cusco, Peru, the former capital of the Incan empire, and another in Buenos Aires, Argentina, which has a large expatriate community of Peruvian Andean migrant workers. The network is activated through wearable technology inspired by quipucamayoc, the data keepers of the Incan court who recorded information about life in the empire using a complex string and knot notation system which they wore around their bodies. Body sensor arrays, which serve simultaneously as game controllers and musical instruments, are worn by movement artists to co-create live interactive generative narratives, imagery and music. The result is a public performance which reconnects two communities through interactive gameplay based in their common histories.

This project was inspired by the Huarochiri Manuscript, a 16th century written compilation of pre-colombian Andean religious rites, which offers a complex and fragmented narrative structure as well as rich visual and sound passages. Quipucamayoc was developed and presented by a collective of over 35 artists, historians, cultural theorists and technical experts in Peru, Argentina and Canada.

The audiovisual materials listed below are the result of a series of workshops held in Cusco and Buenos Aires between 2012 and 2016, and the final performance which was streamed live online in December 2016. The Quipucamayoc documentary can be viewed here.

Quipucamayoc also produced a series of 2D digital prints on canvas, adapted from 3D virtual environments. These prints were exhibited in a group show of the Faculty of Art Summer Institute from September 18 to 22nd, 2017 in the Ada Slaight Galleries at 100 McCaul. 

AUDIO AND VIDEO:
QUIPUCAMAYOC SOUNDCLOUD (audio)
QUIPUCAMAYOC WORKSHOP IN CUSCO (video)
QUIPUCAMAYOC WORKSHOP IN BUENOS AIRES (video)

ARTICLES OF INTEREST:
David McIntosh Receives SSHRC Funding for QUIPUCAMAYOC
Site-Specific Interview with David McIntosh

For more information please visit: http://quipucamayoc.com/

Quipucamayoc es una investigación de arte y tecnología que combina varios formatos artísticos contemporaneous – videojuego 3D, música electroacústica, danza, teatro experimental, diseño textil, sensores vestibles – con el objetivo de construir una red de comunicación que no es textual o linguística, sino corporal, performativa, sensorial. Esta red reune Cusco, Perú, y Buenos Aires, Argentina a través de artístas de movimiento usando trajes como controladores de juego e instrumentos musicales, por lo que los bailarines serán los cocreadores de música, de imágenes 3D y de una narrativa generativa/interactiva en vivo.
La inspiración de este proyecto es el manuscrito del Huarochirí, texto del siglo XVI que recopila ritos y mitología andina prehispánica, reconocido por su narrativa fragmentada, compleja, por sus pasajes riquísimos en imágenes visuales y sonoras. Quipucamayoc fue desarrollado y presentado por el colectivo de más de 35 artístas, teóricos y técnicos peruanos, argentinos y canadienses.
Los materiales audiovisuales presentados han sido producidos durante una serie de talleres en Cusco y Buenos Aires entre 2012 and 2016, y el performance final que fue transmitido en línea en vivo en Diciembre 2016.

 

This research was supported by the Social Sciences and Humanities Research Council of Canada.

 

SSHRC Logo

 

Creator: 
Digital illustration of a distorted face
Photograph collage of dancers performing Quipucamayoc
Digital illustration, a screen grab from Quipucamayoc
Split screen photo of dancers performing while their in-game characters move
Digital image from Quipucamayoc - two figures dancing over clouds
Photograph of Quipucamayoc team in Cusco
Digital image from Quipucamayoc - Incan figures dancing
Friday, September 29, 2017 - 2:15pm
Lab Member: 
David Mcintosh
Judith Doyle
Emma Westecott

Michael Trommer

Michael Trommer is a Toronto-based producer and sound artist; his experimental work has been focused primarily on psychogeographical and acoustemological explorations via the use of field recordings, infra- and ultrasound, as well as multi-channel installation and expanded video techniques.

David Griffin

Dr. David Griffin is a multidisciplinary artist working with painting and drawing, graphic music notations, and performance. He has exhibited his artwork both nationally and internationally, is an instructor at OCAD University in Toronto, and holds a PhD from the Glasgow School of Art.

The artist's creative practice and theoretical writing focus on notational drawing as a hybrid document and connotative system, through which we may construct innovative performance practices.