VVV: Volumetric Video in Videogames

VVV: Volumetric Video in Videogames is a practice based research inquiry that uses full motion video (FMV) videogame design patterns to scaffold the design of new games using volumetric (spatial 3D) video. It aims to advance critical discourse and design knowledge surrounding volumetric video and other emerging forms of hybrid captured media, within videogames and related immersive experiences. VVV is a multi-institution collaboration between experimental game designer Dr. Cindy Poremba (OCAD University), game historian Dr. Carl Therrien (Université de Montréal), and Prof. Nicolas Hesler (Sheridan College); in partnership with Simile, the engineers of the volumetric video platform DepthKit. VVV leverages an extensive historical archive of cinematic videogames housed at Université de Montréal, the advanced game development skills of Sheridan's students and faculty, and Dr. Poremba's research-creation work exploring captured media in game worlds (Poremba 2011), as well as her prior exploration into volumetric capture (Eyebeam 2013, Anderson Ranch 2016). It will result in opportunities for student training at both the undergraduate and graduate level, presentations to both academic and public audiences, the publication of two journal papers, the delivery of an advanced workshop, contributions towards a scholarly manuscript, and a creative work that will be showcased at high profile events locally and internationally.

Volumetric video is a computational fusion of digital video recording and depth sensor data, resulting in a spatialized, and potentially navigable, 3D captured moving image. Volumetric images are becoming increasingly prevalent and sophisticated, driven by interest in augmented reality (AR) and virtual reality (VR) production. Although there is an emerging maker community surrounding this technology, creators have yet to explore its full expressive potential, particularly in procedurally intense, interactive forms such as videogames. Canada's videogame industry contributes $3 billion annually to our GDP, and 54% of Canadians identify as videogame players (ESA 2015). But games struggle to diversify their content, creator, and player base. Volumetric video content, appearing snatched from the world and evoking feelings of presence and actuality, could be a powerful addition to a game development toolkit, but it has properties that differ from what audiences and creators expect to encounter in games. A poor understanding of the use of this material, in both design and critical contexts, can damage our perception of the value and potential of emerging works before they have the opportunity to develop. How can we take an informed, critical approach to the integration of volumetric video in games? What theoretical and design resources can we draw upon to scaffold experimentation, critical understanding, and ethical critique in this area?

VVV: Volumetric Video in Videogames will investigate how design patterns (formal solutions and responses to common design challenges) from underexamined game genres such as fullmotionvideo (FMV) videogames can suggest future paths for design using captured content such as volumetric video. Taking a new materialist approach to hybrid capture technologies, this research aims to make visible the increasingly complex entanglements that make up digital media forms we casually refer to as "real." In doing so, it will support critical dialogue around hybrid captured media, particularly involving real people and places, and further reinforce Canada’s creative leadership in this emerging area. VVV will also contribute to a growing body of game design research on the application of design patterns to new work, and mobilize design knowledge from an underutilized body of research into early videogame design. It has the potential to expand and enrich artistic, documentary and commercial game development and audiences, and to broaden the expressive palette for other forms of media using volumetric content, including nongame AR and VR applications.

 

This research was supported by the Social Sciences and Humanities Research Council of Canada.

SSHRC Logo

Creator: 
A screenshot of volumetric capture of a figure
Sunday, July 1, 2018 - 4:00am
Lab Member: 
Cindy Poremba

Cindy Poremba

Cindy Poremba is a digital media researcher, gamemaker and curator. She is an Assistant Professor (Digital Entertainment), at OCAD University.

Nick Sweetman: Digital Adaptations 

blue and red artwork
installation view of artwork Object Pop1
Friday, January 10, 2014 - 5:00am to Sunday, January 19, 2014 - 5:00am

Nick Sweetman's Digital Adapatations

January 2014

What are the qualities of a successful representation and how does it lead the viewer to interpret the image as plausible? The work in "Digital Adaptations" can be seen as an attempt to picture this process - as a pictorial metaphor for the role of individual visualization in seeing, interpreting and making representative images.

 

Each artwork in Digital Adaptations begins from an image - a photograph taken of the surface of a found object. Through various material translations, the picture is continued onto a new surface and extended beyond the boundaries of the photo into an imagined larger composition that incorporates the image seamlessly. How is plausibility achieved in representing beyond the limited field of the image?

 

One side of the gallery was arranged to display the objects I found, collected, examined, photographed, and elaborated upon during my thesis work. They were presented with didactic panels describing the object, as well as when, where and how each was found. Directly across from each object, on the opposite wall, was hung the mixed media artwork that it inspired, each one containing a photographic sample of the corresponding object’s surface.

 

The work drew on my interdisciplinary research completed in the IAMD program, which merged my painting practice and new material explorations with text-based research into aesthetic philosophy, art history, theories on cognition and the neurobiology of vision.

blue and red sculptural object