Creating a visual language of marks

Above image: a page from the Register of Liberated Africans, c. 1837

Creating a visual language of marks: approaching African identities through data visualization

Co-investigators: Martha Ladly, Ph.D. (OCAD University) and Katrina Keefer, Ph.D. (Adjunct Professor of History, Trent University,

Project Manager and Research Assistant: Kartikay Chadha (

Collaborators: Paul Lovejoy (York University), Dean Rehberger (Michigan State University), Mohammed Salau (University of Mississippi), and Abubakar Babajo Sani (Umaru Musa Yar'adua University, Katsina).
Research Assistants: Eric Lehman, Michael McGill, Maria Yala, Georgina Yeboah
Past Research Assistants: Ma Qianyi

The trans-Atlantic slave trade was a centuries-long trauma that saw approximately 12.5 million Africans forcibly taken from their homes and transported to work in the emerging plantation societies of the Americas. The trauma of enslavement and sustained repression of language, culture and beliefs blurred memories of origins and birthplaces. Previous attempts at analyzing large datasets of names recorded in manumission records to unearth individuals and personal histories have been challenged by practices of slave renaming. Drs. Ladly and Keefer will work with their collaborators to develop a searchable visual database using the entries from the 19th century Registers of Liberated Africans to reveal individual identities and origins. Their research includes appropriate methods for collection, analysis and presentation of the sensitive personal information within these datasets. They will design and train an AI model to work in conjunction with ethno-linguistic and visual models, so that researchers and members of the public may extract meaningful information from the data. Working in the Visual Analytics Lab, the OCAD U design team will construct computational architectures for the visual/linguistic database, develop a mathematical model for data analysis, and design dynamic 2D and 3D visual models and user interfaces.

For more information, please visit 



This research is supported by the Social Sciences and Humanities Research Council of Canada.


Photograph of a page from the Register of Liberated Africans, circa 1837
Monday, July 9, 2018 - 9:30am
Lab Member: 
Martha Ladly
Kartikay Chadha

Canada Council Visualization: Training for visual analysis of complicated data

The Canada Council for the Arts engaged researchers from OCAD’s Visual Analytics Lab to meet a need for the visual analysis of complicated data from various industries, including the Canada Council’s Open Data set, the CADAC data set, and Parliamentary appropriation tables.

Researchers met with Canada Council staff to identify key challenges and gain an understanding of the user group that the visual analytics approach would first serve. They defined co-dependencies of this user group and, working closely with Canada Council employees, developed an appropriate approach to visualizing the various data sets.

A key component of this collaboration was the training of Canada Council employees in the methodologies and applications of data visualization.

This project culminated in the creation of a web-based interface and methods to guide users in exploring and gaining valuable information from the visual representations of grant allocation data.

See the data visualization tool in action on the Canada Council's website.


Canada Council for the Arts dashboard
Canada Council for the Arts dashboard
Canada Council for the Arts Logo
Wednesday, May 23, 2018 - 12:15pm
Lab Member: 
Dr. Sara Diamond
Dr. Ana Jofre
Afrooz Samaei
Marcus A. Gordon

Taste Graph: A narrative visualization tool for massive media data

Traditional newspapers are moving dramatically to digital publishing and data analytics in order to better understand their users’ behaviors, build their subscriber base, maintain their online readers, determine advertising placements. The goal is to deepen and diversify their revenue streams. For this purpose, VAL's Taste Graph research analyzes and synthesis the Globe and Mail data about their subscribers and their web-browsing habits, indicating connections between tastes.

Discovering affinities across different categories is a promising method of segmenting the audience within the context of media planning and potential advertising campaigns. For example, if someone shops for organic vegetables they might be more likely to shop to organic tea. Also, if data indicates close affinities between the categories of organic clothes and organic foods, it is reasonable that a purchaser of both organic vegetable and tea packs will be interested in shopping for an organic cotton shirt if they are provided with a choice to pick a shirt. As well, the people within the taste category  ‘organic’ could also be interested in certain types of drinks, types of shoes, and a sports travel lifestyle.

To establish similar taste correlations within Globe and Mail data, we follow a “narrative approach” that helps tell stories with the data by providing a smooth transition from raw data to communicating through data visualization. The tool we are developing supports the Globe and Mail marketing teams. Firstly, it provides an easy way to filter multiple sources of data and find relationships. Secondly, it shows patterns regarding Globe and Mail audience tastes in customized narrative visualizations. From these, the marketing teams could gain holistic knowledge about their audience tastes and see the impact of certain taste correlations or become aware of some hidden insights of interest regarding relationships between tastes. Thirdly, it allows Globe and Mail people to remain continuously knowledgeable about their business performance measures.

Our organization of the design space involves two types of narrative tactics: visual and structural. For visual tactics, we deploy several visual mechanisms that assist and facilitate the narrative. We chose a bubble chart, and grouped bar chart to illustrate, evaluate, and compare tastes, scores and engagement levels. Colour is then applied to different categories of advertisements to indicate degrees of divergence in tastes. We depend on navigation strategies as a structural tactic to assist the narrative. For example, we arrange the paths the viewer might take through the visualizations, and we make the visualizations interactive by including filtering, selecting, searching, and navigating of advertisement data. These strategies are tested and refined with Globe and Mail design and marketing teams and will then be audience tested.

Our visualization tool, in general, respects secure web application standards. Our goal is to provide the Globe and Mail with intuitive reports about the overall and manifold correlations of tastes of their readership and advertising audiences. Our proposed tool handles the complexity of massive and heterogeneous marketing data records and translates it into a communicative interface.

See a preview of TasteGraph from the IEEE Computer Society Visualization and Graphics Technical Committee VIS2018 here.

Click here to see Ahmad Karawash presenting this research at the Ontario OAR Conference, May 16 2018.
Click here to view this project on SOSCIP's page.

This research was presented by Dr. Ahmad Karawash, Postdoctoral Fellow and Team Lead, and Sana Shepko at the Ontario Centres of Excellence annual Discovery conference on May 1st, 2018 (see below).

Image of interactive tool, showing a graphical comparison of universe vs. specific category's engagement with taste groups
Photograph of Ahmad Karawash and Sana Shepko attending Discovery 2018 conference to present research.
Friday, May 18, 2018 - 10:45am
Lab Member: 
Dr. Sara Diamond
Dr. Ahmad Karawash
Marcus A. Gordon
Jad Rabbaa
Sana Shepko
Afrooz Samaei
Lan-Xi Dong
Dr. Greice C. Mariano
Hugh Ritchie
Ana Jofre
Stephen Keller
Dr. Steve Szigeti

Martha Ladly

Dr. Martha Ladly is a Professor of Design, and Graduate Program Director of the Digital Futures program. Martha specializes in teaching and practice-based research in design, art, media and technology, which she teaches in the Digital Futures Graduate and Undergraduate Programs and the Interdisciplinary Master's Program at OCAD University. She is the former Associate Dean, Graduate Studies, former Graduate Program Director of the Interdisciplinary Master's Program, and founding Chair of the Research Ethics Board at OCAD U in Toronto, Canada.

By Design - OCAD University/Report on Business series continues

Data visualization depicting comments related to sections of the Globe and Mail newspaper
Monday, August 22, 2016 - 4:00am

Sara Diamond, Patricio Davila and Steve Szigeti have collaborated on an op-ed titled There's no Big Data without intelligent interface . The piece is part of the OCAD U/Globe and Mail summer series “By Design" highlighting design thinking, issues and innovation.


Three related factors appear to be relevant in allowing an understanding of online behaviors: attention (the time that individuals and groups expend); influence (the relationships between ideas, products and behaviors) and affect (the emotions and sentiments that are expressed in relation to ideas and products). The extraction of accurate data, then the analysis of these factors in online behavior, and the charting and representation of relationships between these factors poses a significant challenge. For one thing, these factors need to be related to specific content. Data analytics and visualization tools are needed to represent each factor and to chart these relationships. There is very little research to date that works across these fields. This large-scale project seeks to shed light on each element of online social media practice and to then draw relationships between these elements.

Research Description:

People perform topic-based content exploration on large-scale social media systems. Such sites continue to expand rapidly. For example, Twitter continues to grow around the globe at a record pace. Just a year ago, they delivered 65 million Tweets a day. Today, they generate over 200 million Tweets per day. One year ago, there were approximately 150,000 registered Twitter apps. Now, there are more than one million. Facebook has more than 800 million active users of which more than 50% log on to Facebook in any given day where the average user has 130 friends. Seventy-seven percent of active Internet users read blogs. At the same time specialized media companies, brand development agencies and brands have developed social media applications that allow their users to communicate and at the same time, allow them to track the resulting data.

Editorial and business leaders see value in understanding the emotional tone, influences, attention span and diversity of their various sections and offerings, contributors and readers. Attention and influence, for example, currently directly impact advertising dollar interest in an article. In going digital, media publications have added commentary in the form of opinion blogs by its core of writers as well as ample opportunity for readers to vote and comment. Currently a majority of online media allow readers to express their thoughts and opinions on content through social media commentary. This information can impact advertising sales, decisions on style and relatedness of writers and design and even the kind of influence that different sections, authors or columns may have. Editorial leadership is eager to better manage the means for reader commentary. At the same time it is valuable to understand any underlying patterns that suggest reasons for specific emotional tone. Discovering sentiments, patterns and relationships embedded in articles as well as comments is important for tracking the newspaper’s role in shaping public opinion on contemporary issues and the ways that readers interact with these opinions. It can help media analysts better understand the impact of sentiments on news events. What is more, new tools, on multiple platforms can be developed for media users that allow them to shape their emotional content and respond to others, and chart the influence of their ideas, media patterns and behaviors.

For almost a decade contemporary brands have relied on a growing direct dialogue with their consumer base through social media, and gamification (direct play as a means of polling). These relationships engender loyalty and provide a rich source of data to understand and predict consumer behaviors. Consumer opinion that is expressed in response to new offerings, system breakdowns, or customer service is of critical importance in a world where viral trends erupt quickly with significant impact. Events and opinion outside of an immediate enterprise can have a direct impact in a social media era. Marketing and advertising companies analyze consumer attitudes and relationships to brands for trend analysis and product development. The technology of “predictive analytics” is being fine-tuned by digital media and ICT companies with new offerings such as inferSYTEMS. While the technology of monitoring is becoming more sophisticated, the underlying assumptions of analysis have not changed dramatically for many years, continuing to rely on twentieth century psychology structures. Brands and media analysis companies seek to bring together social media data with data that tracks consumer behaviors – in specific their attention to media, to products and services and their consumption patterns.

In some areas, e.g., healthcare, free-form texts are the most common form of valuable data. These data range from doctor’s notes, descriptions of patient histories, to healthcare-related messages posted by patients on social media such as blogs, bulletin boards, and discussion forums. Such narrative text data contain the most valuable information for physicians to use in their practice and for public and government agencies to make their healthcare-related decisions. Recently, the New York Times reported on a study by MIT researchers, which showed that companies included in their study that adopted data-driven decision-making achieved 5-6% higher productivity than those that did not.

Since data are continuously generated every day in large volumes, the sheer amount of data is too overwhelming for humans to read and analyze manually. Automatic text analysis tools are in great need to discover the hidden information trapped inside the free-form texts. For example, a tool that identifies and analyzes the healthcare-related posts in social media can detect public opinions, activities and preferences in healthcare-related issues.

Understanding consumer opinion of reliability and service quality across an industry like banking can have an impact on a specific company’s quality of service as well as enabling an entire industry to improve. Natural language analysis, data mining and information retrieval are key techniques that can be used to build such text analysis tools.

It is difficult to discern meaning by extracting information piece by piece. We hypothesize that taking a data-driven design approach to visualizing content would make the aggregate meanings more apparent. The advantage of working with this partially processed data is that issues of confidentiality do not arise since any confidential or client information has been abstracted from the media. A second advantage is that research can also focus on visualization and design issues rather than duplicate commercially available linguistic parsing capabilities.

Colourful lines from design piece
Colourful lines from design piece
Colourful lines from design piece
Monday, September 14, 2015 - 3:45pm
Lab Member: 
Dr. Sara Diamond



This project intends to develop tools to support media companies in transition from print to digital through addressing the following research questions:

1) How do subscribers and non-subscribers consume print and/or online media – what is similar and what different?

2) How can social media data be leveraged to build and retain readers, and to inform a sophisticated next generation personalized recommendation system?  


The Globe & Mail


Dr Sara Diamond and Dr Steve Szigeti


Diamond, S. & Szigeti, S. (2013). Social Media Data Visualization Case Study: Globe and Mail. Workshop: CIVDDD Collaborative Research in Big Data Analytics and Visualization. At CASCON 2013, November 18-20, 2013. Toronto, ON.conference Extended Abstracts on Human Factors in Computing Systems (CHI’12), pp. 1493--1498.

Oliver, S., Gali, G., Chevalier, F. & Diamond, S. (2012). Discursive Navigation  of Online  News,  in ACM  Proceedings  of the Designing Interactive Systems Conference (DIS’12), pp. 82—85.

Research Paper image
Monday, August 11, 2014 - 3:45pm
Lab Member: 
Dr. Sara Diamond
Dr. Fanny Chevalier
Symon Olivier


Current Team

OCAD announces innovative research partnership

Thursday, March 4, 2010 - 5:00am

(Toronto — March 4, 2010) The Ontario College of Art & Design (OCAD) announced an $11.5 million interdisciplinary research project designed to develop the next generation of data analysis and visualization tools. The collaborative project includes computer scientists, vision scientists, designers, artists and social scientists at York, OCAD and U of T, with 14 industry partners.

How do you look at millions of genomic patterns and see the diagnostic implications? How do you assimilate satellite data to better predict and visualize the effects of global warming, pollution, and weather patterns? How can you chart the global migration of millions of people under slavery? How do you assess the impact millions of blog entries have had on the print media economy? How do you sift global intelligence reports to identify the real threats? Each day, humanity poses thousands of similar questions as we produce massive data sets in every field − but as the data grows, the challenge becomes translating this data for the human senses and delivering the best analysis to solve real-world problems.

The Centre for Innovation in Information Visualization and Data-Driven Design (CIV-DDD), led by York University in partnership with the Ontario College of Art & Design (OCAD), promises to develop the next generation of data discovery, design, and visualization techniques by developing new computational tools, representational strategies, and interfaces.

The $11.5 million five-year project brings together a unique multidisciplinary team of over 50 researchers from York, OCAD, the University of Toronto, 14 industry partners, and significant international collaborators. John Tsotsos, Canada Research Chair in Computational Vision and professor of vision science at York, is the project’s principal investigator. Nick Cercone, professor of artificial intelligence and data mining, will co-lead York’s team of 14 researchers, who include Amir Asif, professor and chair of the Department of Computer Science, and Caitlin Fisher, Canada Research Chair in Digital Culture in the Department of Film. Sara Diamond, visualization design researcher and president of the Ontario College of Art & Design, will lead OCAD’s team of 12 researchers, who include Vladimir Spicanovic, dean of the Faculty of Art, and Greg van Alstyne, professor in the Faculty of Design and director of Research, Strategic Innovation Lab.

Ontario's Ministry of Research and Innovation has provided over $3.8 million in funding through the Ontario Research Fund-Research Excellence Program; industry partners will contribute over $3.6 million, with the remaining $4.1 million coming from the project’s institutional partners.

“Humans’ capacity to interpret sensory stimuli is limited, which is why the human processing strategy is to attend to the relevant and ignore the irrelevant,” says Tsotsos. “Determining what’s relevant is a key task. Equally important is discovering how best to present such data in a form that is quickly and effectively understood. By combining our team’s expertise in computer science, design, digital media, visual perceptual science, fine arts, and the social sciences, CIV-DDD will discover and provide new visualizations for underlying patterns, processes, and relationships within data sets. These in turn will suggest new research directions and applications, laying the foundation for a new wave of knowledge creation and technological products.”

CIV-DDD formalizes many existing research collaborations, both within York University’s faculties and departments and among researchers at York, OCAD, and U of T. “In the new paradigm of data-driven discovery, art and design researchers have a profound role to play in partnership with scientists, making the invisible visible, heralding a new era of knowledge, cultural creation, and technological products,” said Diamond. “This preeminent research hub for the development of next-generation data visualization techniques is unique in its level of interdisciplinary fire power, strong collaboration with end-users and international links. It aggregates and extends much successful collaboration amongst the researchers, and between OCAD and York University.”

Many techniques and technologies developed by research groups associated with CIV-DDD will become resources for the entire team as they focus on new data-driven design and information visualization solutions in four thematic areas:

  • bioinformatics and medicine
  • fine and performing arts
  • engineering and physical sciences
  • humanities and social sciences

Collaboration between industry and academia is crucial to the project’s success. “Platform Computing is honoured to support the CIV-DDD project and provide the students and researchers at York and the OCAD with cutting-edge technologies to explore and create the next generation of visualization solutions and services and help them tackle scientific, social and human challenges,” said Jingwen Wang, vice-president, products, at Platform Computing. “Collaboration and information sharing are fundamental to academic research and Platform products enable researchers and students to easily collaborate and manage data and resources to capture, simulate and analyze their results.”

CIV-DDD’s industry partners highlight its wide applicability across sectors, including:

“CIV-DDD responds both to a dramatic paradigm shift in the health, social and economic challenges facing Canadians and the need for more research capacity and state-of-the-art infrastructure in this region,” said Stan Shapson, York’s vice-president Research & Innovation. “It also builds on the existing strengths of York’s Centre for Vision Research in the Faculty of Science and Engineering, the Institute for Research on Learning Technologies in the Faculty of Education, and Future Cinema Lab in the Faculty of Fine Arts, among others.

“By leading the Consortium on New Media, Creative, and Entertainment R&D in the Toronto Region (CONCERT), York and its partner universities and industries identified the preconditions necessary to develop a high-end visualization industry in Ontario. This unique collaboration will help fulfill one of CONCERT’s long-term objectives, which was to grow the region’s entertainment, screen-based and other innovation-driven creative industries into a globally competitive cluster.”

CIV-DDD will also support Ontario’s economy by producing innovative technology for commercialization, such as new:

  • data-mining algorithms
  • 3D-vision and 3D-modeling technologies
  • data-display tools and protocols
  • visualization-design methods and techniques
  • data-inquiry paradigms
  • user-friendly interfaces that can be scaled to a variety of screen displays
  • new aesthetics and research practices

“We’re proud of the cutting-edge work that our researchers do at York University and the wealth and jobs they create in York West and across Ontario,” said Mario Sergio, MPP for York West. “New discoveries will continue to be made and we want those people, those ideas and those jobs right here in our community.”

CIV-DDD is one of 21 projects funded by the Ontario Research Fund–Research Excellence program, which has provided almost $69.5 million to support more than 214 researchers in seven cities across Ontario. The Ontario Research Fund is a key part of the province’s Innovation Agenda, a $3.2 million strategy designed to move world-class research from the lab to the global marketplace in key areas such as life sciences, digital media, and green energy to ensure Ontario will be among the 21st Century’s winning economies. The Research Excellence program helps develop Ontario’s research talent while ensuring Ontario researchers have the operating funding they need to undertake world-leading projects.

For a full list of funded projects, visit MRI’s Web site.

About York University
York University is the leading interdisciplinary research and teaching university in Canada. York offers a modern, academic experience at the undergraduate and graduate level in Toronto, Canada’s most international city. The third largest university in the country, York is host to a dynamic academic community of 50,000 students and 7,000 faculty and staff, as well as 200,000 alumni worldwide. York’s 10 Faculties and 28 research centres conduct ambitious, groundbreaking research that is interdisciplinary, cutting across traditional academic boundaries. This distinctive and collaborative approach is preparing students for the future and bringing fresh insights and solutions to real-world challenges. York University is an autonomous, not-for-profit corporation.

About the Ontario College of Art & Design (OCAD)
The Ontario College of Art & Design ( is Canada’s “university of the imagination.” OCAD is dedicated to art and design education, practice and research and to knowledge and invention across a wide range of disciplines. The university is building on its traditional, studio-based strengths, adding new approaches to learning that champion cross-disciplinarity, collaboration and the integration of emerging technologies. In the Age of Imagination, OCAD community members will be uniquely qualified to act as catalysts for the next advances in culture, technology and quality of life for all Canadians.

- 30 -

For more information and images please contact:

Sarah Mulholland, Media & Communications Officer, OCAD
416.977.6000 Ext. 327 (mobile Ext. 1327)

Elizabeth Monier-Williams, Research Communications, York University,
416 736 2100 x21069

Sean Billingsley, Faculty of Science and Engineering, York University,
416 736 2100 x22814