This project explores memory, embodiment, and the social interactions of elders and others using digital media experiences and simulated virtual reality, which targets memory. Through testing with personal images, and news video from the CBC Newsworld archives, with our current and prospective prototypes, we demonstrate the generation of new conceptual lenses, technological forms and experiences that may enrich the lives of elders and have potential benefits for their families and caregivers, and other communities of participants.
This project investigates how digital technology can stimulate the memories of older adults and improve their cognitive function by using an audio-visual dataset of historical CBC news.
- How might a multimodal, personalized digital media application with embodied, social interaction create a form of “virtualized” reality?
- Could this experience evoke memories and improve memory recall?
- How might this experience enhance social connection?
- How can we build user interfaces that optimally support seniors’ comprehension, use, browsing, and interactions with digital/mobile devices?
- How can effective 2D/3D aesthetic visualizations be created to give non-expert users a sense of a personal relationship with the data they are observing?Can such new media and VR/AR applications assist elders by enhancing memory, and other cognitive abilities?
- Can we produce feelings of pleasure and delight in our elder audiences, and enhance this joyful interaction by including caregivers, family and friends?
Premise and Research Question:
With an aging population, there is an increasing need for facilities, activities and technologies that support and enrich the experiences of seniors. Environments that foster mental acuity, cognition, and physical health are in increasing demand.
How might we create applications and digital/virtual experiences that can enhance self-actualisation, and support elder’s abilities, cognition, memory, and social interactions?
Our researchers have created and demonstrated 2D and real and simulated 3D aesthetic visualizations to give non-expert users a sense of relationship with the data we are displaying. We have created new media applications that simulate “virtualized” realities, by stimulating memory and recall. These applications may assist elders by acquiring new technical skills, and enhancing memory, mental acuity, personal interaction and communication, and cognitive abilities.
This project is highly multidisciplinary (involving interaction design, mobile technologies, natural language processing, video search, big data, quantified self-actualisation, and data visualisation, gerontology, psychology) and employs multiple prior developments: Postcard Memories, a networked, tablet application that facilitates creative, interactive sharing of family memories, and the CBC Newsworld Holodeck, a gesture-based, immersive installation which simulates a virtual experience using large digital audio-video multi-screen environments, a gestural interface, and content from 24 years of CBC Newsworld programming.
Through user studies and participant observations we have found that when an internalized virtual reality is created in participants, it can have cognitive and affective benefits, which are also replicated in the elderly and those with some memory loss. The analysis of results from a user study with the Postcard Memories application suggests that the application improved connection and communication between elders and their family members, friends, and caregivers. Elder participants agreed that both applications helped them to recall personal stories, which enabled them to communicate more openly with others. Participant observation studies of the CBC Newsworld Holodeck installation have similarly found an improvement in users’ social interactivity and memory recall. Participants have also reported feelings of increased self-actualization, while others have displayed increased conversational opportunities, and interests in technology, and the news content they viewed.
We are now developing a new geographical interface and a touch-screen environment, enabling multi-user data interactions, in collaboration with industry partner GestureTek. Our aim is develop data visualizations methods that will enhance user interaction with specific ‘drilling down’ to keyword sensitive data, to aid participants’ interaction with this multi-year CBC corpus. Our research points to innovation in novel search-and-display techniques which we will continue to refine and develop through iterations of additional ‘days’ in the data corpus, chosen by users for personal significance.
If valid evidence of positive health effects is derived, this could be a commercialisable method and product, delivering health effects to significant numbers. There is further potential to continue development of tools related to video browsing, search-and-display technologies and associated user interface-design, with committed industry partners GestureTek.
Our efforts have yielded delightful results and some valuable insights. We have and will build user interfaces that support seniors’ comprehension and adoption, assisting them to build their social interactions, technological competencies, and feelings of self-hood.
Click here to see Visual Analytics Lab's video on the CBC Holodeck
Click here to see a demonstration of the Postcard Memories Application
See more from our contributors:
Dr. Martha Ladly
Dr. Gerald Penn
Dr. Frank Rudzicz