This project explores memory, embodiment, and the social interactions of elders and others using digital media experiences and simulated virtual reality, which targets memory. Through testing with personal images, and news video from the CBC Newsworld archives, with our current and prospective prototypes, we demonstrate the generation of new conceptual lenses, technological forms and experiences that may enrich the lives of elders and have potential benefits for their families and caregivers, and other communities of participants.

This project investigates how digital technology can stimulate the memories of older adults and improve their cognitive function by using an audio-visual dataset of historical CBC news.

Key Considerations:

  • How might a multimodal, personalized digital media application with embodied, social interaction create a form of “virtualized” reality?
  • Could this experience evoke memories and improve memory recall?
  • How might this experience enhance social connection?
  • How can we build user interfaces that optimally support seniors’ comprehension, use, browsing, and interactions with digital/mobile devices?
  • How can effective 2D/3D aesthetic visualizations be created to give non-expert users a sense of a personal relationship with the data they are observing?Can such new media and VR/AR applications assist elders by enhancing memory, and other cognitive abilities?
  • Can we produce feelings of pleasure and delight in our elder audiences, and enhance this joyful interaction by including caregivers, family and friends?

 

Premise and Research Question:

With an aging population, there is an increasing need for facilities, activities and technologies that support and enrich the experiences of seniors. Environments that foster mental acuity, cognition, and physical health are in increasing demand.

How might we create applications and digital/virtual experiences that can enhance self-actualisation, and support elder’s abilities, cognition, memory, and social interactions?

Experimental Prototypes:

Our researchers have created and demonstrated 2D and real and simulated 3D aesthetic visualizations to give non-expert users a sense of relationship with the data we are displaying. We have created new media applications that simulate “virtualized” realities, by stimulating memory and recall. These applications may assist elders by acquiring new technical skills, and enhancing memory, mental acuity, personal interaction and communication, and cognitive abilities.

This project is highly multidisciplinary (involving interaction design, mobile technologies, natural language processing, video search, big data, quantified self-actualisation, and data visualisation, gerontology, psychology) and employs multiple prior developments: Postcard Memories, a networked, tablet application that facilitates creative, interactive sharing of family memories, and the CBC Newsworld Holodeck, a gesture-based, immersive installation which  simulates a virtual experience using large digital audio-video multi-screen environments, a gestural interface, and content from 24 years of CBC Newsworld programming.

Participant Studies:

Through user studies and participant observations we have found that when an internalized virtual reality is created in participants, it can have cognitive and affective benefits, which are also replicated in the elderly and those with some memory loss. The analysis of results from a user study with the Postcard Memories application suggests that the application improved connection and communication between elders and their family members, friends, and caregivers. Elder participants agreed that both applications helped them to recall personal stories, which enabled them to communicate more openly with others. Participant observation studies of the CBC Newsworld Holodeck installation have similarly found an improvement in users’ social interactivity and memory recall. Participants have also reported feelings of increased self-actualization, while others have displayed increased conversational opportunities, and interests in technology, and the news content they viewed.

New Developments:

We are now developing a new geographical interface and a touch-screen environment, enabling multi-user data interactions, in collaboration with industry partner GestureTek. Our aim is develop data visualizations methods that will enhance user interaction with specific ‘drilling down’ to keyword sensitive data, to aid participants’ interaction with this multi-year CBC corpus. Our research points to innovation in novel search-and-display techniques which we will continue to refine and develop through iterations of additional ‘days’ in the data corpus, chosen by users for personal significance.

If valid evidence of positive health effects is derived, this could be a commercialisable method and product, delivering health effects to significant numbers. There is further potential to continue development of tools related to video browsing, search-and-display technologies and associated user interface-design, with committed industry partners GestureTek.

Conclusion:

Our efforts have yielded delightful results and some valuable insights. We have and will build user interfaces that support seniors’ comprehension and adoption, assisting them to build their social interactions, technological competencies, and feelings of self-hood.

Click here to see Visual Analytics Lab's video on the CBC Holodeck

Click here to see a demonstration of the Postcard Memories Application

See more from our contributors:
Dr. Martha Ladly
Dr. Gerald Penn
Dr. Frank Rudzicz
Kartikay Chadha

 

Photograph of a  person's hand, using a tablet to selecting images from the CBC News Holodeck
Photograph of two screens and computers displaying the Holodeck setup
Close-up of hand making selections in the Postcard Memories app
Photograph of an individual using the postcard memories app. Their selection appears on a tablet and a large wall mounte screen
Photograph of CBC Newsworld Holodeck - several news clips from different years displayed concurrently
GRAND NCE Funding Logo
Gesturetek corporate logo: blue text on a white background
Friday, September 29, 2017 - 12:00pm
Lab Member: 
Martha Ladly
Embed Video: 

Elders, Memory & Data Visualization: Changing Aging Cues with News Media

Overview

Friday September 29th, 2017
Photograph of a  person's hand, using a tablet to selecting images from the CBC News Holodeck

This project explores memory, embodiment, and the social interactions of elders and others using digital media experiences and simulated virtual reality, which targets memory. Through testing with personal images, and news video from the CBC Newsworld archives, with our current and prospective prototypes, we demonstrate the generation of new conceptual lenses, technological forms and experiences that may enrich the lives of elders and have potential benefits for their families and caregivers, and other communities of participants.

This project investigates how digital technology can stimulate the memories of older adults and improve their cognitive function by using an audio-visual dataset of historical CBC news.

Key Considerations:

  • How might a multimodal, personalized digital media application with embodied, social interaction create a form of “virtualized” reality?
  • Could this experience evoke memories and improve memory recall?
  • How might this experience enhance social connection?
  • How can we build user interfaces that optimally support seniors’ comprehension, use, browsing, and interactions with digital/mobile devices?
  • How can effective 2D/3D aesthetic visualizations be created to give non-expert users a sense of a personal relationship with the data they are observing?Can such new media and VR/AR applications assist elders by enhancing memory, and other cognitive abilities?
  • Can we produce feelings of pleasure and delight in our elder audiences, and enhance this joyful interaction by including caregivers, family and friends?

 

Premise and Research Question:

With an aging population, there is an increasing need for facilities, activities and technologies that support and enrich the experiences of seniors. Environments that foster mental acuity, cognition, and physical health are in increasing demand.

How might we create applications and digital/virtual experiences that can enhance self-actualisation, and support elder’s abilities, cognition, memory, and social interactions?

Experimental Prototypes:

Our researchers have created and demonstrated 2D and real and simulated 3D aesthetic visualizations to give non-expert users a sense of relationship with the data we are displaying. We have created new media applications that simulate “virtualized” realities, by stimulating memory and recall. These applications may assist elders by acquiring new technical skills, and enhancing memory, mental acuity, personal interaction and communication, and cognitive abilities.

This project is highly multidisciplinary (involving interaction design, mobile technologies, natural language processing, video search, big data, quantified self-actualisation, and data visualisation, gerontology, psychology) and employs multiple prior developments: Postcard Memories, a networked, tablet application that facilitates creative, interactive sharing of family memories, and the CBC Newsworld Holodeck, a gesture-based, immersive installation which  simulates a virtual experience using large digital audio-video multi-screen environments, a gestural interface, and content from 24 years of CBC Newsworld programming.

Participant Studies:

Through user studies and participant observations we have found that when an internalized virtual reality is created in participants, it can have cognitive and affective benefits, which are also replicated in the elderly and those with some memory loss. The analysis of results from a user study with the Postcard Memories application suggests that the application improved connection and communication between elders and their family members, friends, and caregivers. Elder participants agreed that both applications helped them to recall personal stories, which enabled them to communicate more openly with others. Participant observation studies of the CBC Newsworld Holodeck installation have similarly found an improvement in users’ social interactivity and memory recall. Participants have also reported feelings of increased self-actualization, while others have displayed increased conversational opportunities, and interests in technology, and the news content they viewed.

New Developments:

We are now developing a new geographical interface and a touch-screen environment, enabling multi-user data interactions, in collaboration with industry partner GestureTek. Our aim is develop data visualizations methods that will enhance user interaction with specific ‘drilling down’ to keyword sensitive data, to aid participants’ interaction with this multi-year CBC corpus. Our research points to innovation in novel search-and-display techniques which we will continue to refine and develop through iterations of additional ‘days’ in the data corpus, chosen by users for personal significance.

If valid evidence of positive health effects is derived, this could be a commercialisable method and product, delivering health effects to significant numbers. There is further potential to continue development of tools related to video browsing, search-and-display technologies and associated user interface-design, with committed industry partners GestureTek.

Conclusion:

Our efforts have yielded delightful results and some valuable insights. We have and will build user interfaces that support seniors’ comprehension and adoption, assisting them to build their social interactions, technological competencies, and feelings of self-hood.

Click here to see Visual Analytics Lab's video on the CBC Holodeck

Click here to see a demonstration of the Postcard Memories Application

See more from our contributors:
Dr. Martha Ladly
Dr. Gerald Penn
Dr. Frank Rudzicz
Kartikay Chadha

 

Photograph of two screens and computers displaying the Holodeck setupClose-up of hand making selections in the Postcard Memories appPhotograph of an individual using the postcard memories app. Their selection appears on a tablet and a large wall mounte screen Photograph of CBC Newsworld Holodeck - several news clips from different years displayed concurrentlyGRAND NCE Funding LogoGesturetek corporate logo: blue text on a white background

Documents:

PDF icon 'Postcard Memories' Presentation
PDF icon Postcard Memories: ISEA2015 Paper
PDF icon Reality Recalled: Elders, Memory, and VR Paper

Contributors
  • Dr. Gerald Penn
  • Dr. Frank Rudzicz
  • Dr. Paul Ritvo
  • Kartikay Chadha
  • Glen Farrelly
  • Katie Micak
  • Thoreau Bakker
  • Natasha Mody
  • April Xie