Creating a visual language of marks

Above image: a page from the Register of Liberated Africans, c. 1837

Creating a visual language of marks: approaching African identities through data visualization

Co-investigators: Martha Ladly, Ph.D. (OCAD University) and Katrina Keefer, Ph.D. (Adjunct Professor of History, Trent University, Katrinakeefer@trentu.ca

Project Manager and Research Assistant: Kartikay Chadha (kchadha@faculty.ocadu.ca)

Collaborators: Paul Lovejoy (York University), Dean Rehberger (Michigan State University), Mohammed Salau (University of Mississippi), and Abubakar Babajo Sani (Umaru Musa Yar'adua University, Katsina).
Research Assistants: Eric Lehman, Michael McGill, Maria Yala, Georgina Yeboah
Past Research Assistants: Ma Qianyi

The trans-Atlantic slave trade was a centuries-long trauma that saw approximately 12.5 million Africans forcibly taken from their homes and transported to work in the emerging plantation societies of the Americas. The trauma of enslavement and sustained repression of language, culture and beliefs blurred memories of origins and birthplaces. Previous attempts at analyzing large datasets of names recorded in manumission records to unearth individuals and personal histories have been challenged by practices of slave renaming. Drs. Ladly and Keefer will work with their collaborators to develop a searchable visual database using the entries from the 19th century Registers of Liberated Africans to reveal individual identities and origins. Their research includes appropriate methods for collection, analysis and presentation of the sensitive personal information within these datasets. They will design and train an AI model to work in conjunction with ethno-linguistic and visual models, so that researchers and members of the public may extract meaningful information from the data. Working in the Visual Analytics Lab, the OCAD U design team will construct computational architectures for the visual/linguistic database, develop a mathematical model for data analysis, and design dynamic 2D and 3D visual models and user interfaces.

For more information, please visit  http://decodingorigins.org 

 

 

This research is supported by the Social Sciences and Humanities Research Council of Canada.

SSHRC Logo

Photograph of a page from the Register of Liberated Africans, circa 1837
Monday, July 9, 2018 - 9:30am
Lab Member: 
Martha Ladly
Kartikay Chadha

Elders, Memory & Data Visualization: Changing Aging Cues with News Media

This project explores memory, embodiment, and the social interactions of elders and others using digital media experiences and simulated virtual reality, which targets memory. Through testing with personal images, and news video from the CBC Newsworld archives, with our current and prospective prototypes, we demonstrate the generation of new conceptual lenses, technological forms and experiences that may enrich the lives of elders and have potential benefits for their families and caregivers, and other communities of participants.

This project investigates how digital technology can stimulate the memories of older adults and improve their cognitive function by using an audio-visual dataset of historical CBC news.

Key Considerations:

  • How might a multimodal, personalized digital media application with embodied, social interaction create a form of “virtualized” reality?
  • Could this experience evoke memories and improve memory recall?
  • How might this experience enhance social connection?
  • How can we build user interfaces that optimally support seniors’ comprehension, use, browsing, and interactions with digital/mobile devices?
  • How can effective 2D/3D aesthetic visualizations be created to give non-expert users a sense of a personal relationship with the data they are observing?Can such new media and VR/AR applications assist elders by enhancing memory, and other cognitive abilities?
  • Can we produce feelings of pleasure and delight in our elder audiences, and enhance this joyful interaction by including caregivers, family and friends?

 

Premise and Research Question:

With an aging population, there is an increasing need for facilities, activities and technologies that support and enrich the experiences of seniors. Environments that foster mental acuity, cognition, and physical health are in increasing demand.

How might we create applications and digital/virtual experiences that can enhance self-actualisation, and support elder’s abilities, cognition, memory, and social interactions?

Experimental Prototypes:

Our researchers have created and demonstrated 2D and real and simulated 3D aesthetic visualizations to give non-expert users a sense of relationship with the data we are displaying. We have created new media applications that simulate “virtualized” realities, by stimulating memory and recall. These applications may assist elders by acquiring new technical skills, and enhancing memory, mental acuity, personal interaction and communication, and cognitive abilities.

This project is highly multidisciplinary (involving interaction design, mobile technologies, natural language processing, video search, big data, quantified self-actualisation, and data visualisation, gerontology, psychology) and employs multiple prior developments: Postcard Memories, a networked, tablet application that facilitates creative, interactive sharing of family memories, and the CBC Newsworld Holodeck, a gesture-based, immersive installation which  simulates a virtual experience using large digital audio-video multi-screen environments, a gestural interface, and content from 24 years of CBC Newsworld programming.

Participant Studies:

Through user studies and participant observations we have found that when an internalized virtual reality is created in participants, it can have cognitive and affective benefits, which are also replicated in the elderly and those with some memory loss. The analysis of results from a user study with the Postcard Memories application suggests that the application improved connection and communication between elders and their family members, friends, and caregivers. Elder participants agreed that both applications helped them to recall personal stories, which enabled them to communicate more openly with others. Participant observation studies of the CBC Newsworld Holodeck installation have similarly found an improvement in users’ social interactivity and memory recall. Participants have also reported feelings of increased self-actualization, while others have displayed increased conversational opportunities, and interests in technology, and the news content they viewed.

New Developments:

We are now developing a new geographical interface and a touch-screen environment, enabling multi-user data interactions, in collaboration with industry partner GestureTek. Our aim is develop data visualizations methods that will enhance user interaction with specific ‘drilling down’ to keyword sensitive data, to aid participants’ interaction with this multi-year CBC corpus. Our research points to innovation in novel search-and-display techniques which we will continue to refine and develop through iterations of additional ‘days’ in the data corpus, chosen by users for personal significance.

If valid evidence of positive health effects is derived, this could be a commercialisable method and product, delivering health effects to significant numbers. There is further potential to continue development of tools related to video browsing, search-and-display technologies and associated user interface-design, with committed industry partners GestureTek.

Conclusion:

Our efforts have yielded delightful results and some valuable insights. We have and will build user interfaces that support seniors’ comprehension and adoption, assisting them to build their social interactions, technological competencies, and feelings of self-hood.

Click here to see Visual Analytics Lab's video on the CBC Holodeck

Click here to see a demonstration of the Postcard Memories Application

See more from our contributors:
Dr. Martha Ladly
Dr. Gerald Penn
Dr. Frank Rudzicz
Kartikay Chadha

 

Photograph of a  person's hand, using a tablet to selecting images from the CBC News Holodeck
Photograph of two screens and computers displaying the Holodeck setup
Close-up of hand making selections in the Postcard Memories app
Photograph of an individual using the postcard memories app. Their selection appears on a tablet and a large wall mounte screen
Photograph of CBC Newsworld Holodeck - several news clips from different years displayed concurrently
GRAND NCE Funding Logo
Gesturetek corporate logo: blue text on a white background
Friday, September 29, 2017 - 12:00pm
Lab Member: 
Martha Ladly
Embed Video: