Abstract image with blue, purple, and orange tiles moving upward
October 26, 2022

Metaphors for Human-AI Interaction Workshop

10:00–15:30 BST

Location: Virtual | Cambridge, UK

This is an invite-only workshop. Please do not forward.

Design for human-AI interaction has drawn on various metaphors, including the collaborating partner, the helpful assistant and the co-pilot. These metaphors tend to focus on explicit interactions between humans and AI. However, interactions between humans and intelligent systems are also implicit (opens in new tab), making it difficult for users to build mental models of what the system is doing or how it does it. In this workshop, we will explore an extended set of metaphors, with the aim of facilitating (i) design and (ii) user understanding of how people work both with and through AI systems, as they create content and data, both intentionally and through traces of activity. 

  • For instance, Viva Topics (opens in new tab) is an intelligent system that builds an organisational knowledge base from content generated by organisation members, and then disseminates this across the organisation. Interactions between AI and organisation members in this case are largely implicit, and the algorithms that build the knowledge base and highlight its content to other organisation members might be understood as mediators (opens in new tab), in that they mediate interactions between people and the knowledge base, and also between people and other people by connecting them through content recommendations. Another relevant metaphor is that of infrastructure (opens in new tab). The pervasive and background qualities of these systems resonate with other technological infrastructures that the HCI community has considered.

    Despite the infrastructural quality of Viva Topics, the output of the ML that underpins it can be foregrounded and directly edited by people. For instance, human-readable schema, produced by probabilistic programming (opens in new tab) techniques, can be curated by organisation members and are then stored as stable values in the knowledge base. These representations of knowledge fold into organisational work, by forming the basis of AI-enabled recommendations (e.g., of other organisation members who are knowledgeable about a topic, or of relevant resources). In contrast, ML outputs produced by neural embedding based ML models are fluid, being produced in response to user queries in the moment. Deep neural ML is often associated with partnership experiences such as GitHub Copilot (opens in new tab). While this interaction is, in many ways, explicit, it also has implicit qualities, in that human input informs the ML in ways that are not visible to its users. 

    Thus, different ML technologies have different implications for how metaphors can support users, designers and developers in understanding and creating intelligent systems. These metaphors may speak to both implicit and explicit qualities of interactions between people and the same ML technology.

    In this workshop, we will explore the idea that expanding the repertoire of metaphors employed when developing ML systems and communicating their properties to users could:

    • Support the users of intelligent systems in understanding how, through their activity and interactions, they are impacting and being impacted by algorithms, thus allowing opportunities for agency and repair; 
    • Support designers who create user experiences that incorporate human-AI interactions in (i) understanding and articulating the nature of those interactions and (ii) making systems more transparent and explainable to their users; 
    • Support decision-makers who design and develop ML systems in understanding the implications of building software that incorporates different ML technologies, especially in terms of their potential to make ML outputs human-readable, curatable, stable, or otherwise potentially capable of serving as a ‘boundary object (opens in new tab)’ between humans and AI. 

Speakers

headshot of Susanne Bødker (opens in new tab)
Susanne Bødker

Professor
Aarhus University

headshot of Ewa Luger (opens in new tab)
Ewa Luger

Professor of Human-Data Interaction
University of Edinburgh

headshot of Andrew Rice (opens in new tab)
Andrew Rice

Principal Researcher
GitHub

headshot of Yvonne Roger (opens in new tab)
Yvonne Rogers

Professor and Director of UCLIC
UCL

headshot of Nur Yildirim (opens in new tab)
Nur Yildirim

PhD student
Carnegie Mellon University

headshot of Yordan Zaykov
Yordan Zaykov

Principal Research Engineering Manager
Microsoft Research

Agenda

Time (BST)Session
10:00Opening remarks and framing
10:15Session 1: Interacting with intelligent systems
We will consider the scope of interactions between end-users and AI and identify metaphors that help articulate and explain these. We will explore both existing metaphors and spaces where new metaphors are needed, and consider associated values and challenges. 

The discussion will be seeded by three 10-minute lightening talks that cover different ways of thinking about human-AI interaction: 
• Metaphors for machine learning: partners, tools, or companions? (Yvonne Rogers (opens in new tab)) | video (opens in new tab)
• Technologies as tools/mediators (Susanne Bødker (opens in new tab)) | video (opens in new tab)
• AI as an educator (Ewa Luger (opens in new tab)

This will be followed by breakout sessions (~30m) in which attendees will discuss different metaphors in more depth (after self-selecting metaphors of interest) and consider their utility in supporting user experience and understanding of intelligent systems design, alongside the challenges they raise.
11:15Short break
11:30Report back and discussion
12:15Lunch break
13:15Session 2: Interacting with ML-mined data
We will consider how metaphors could play a role in supporting the designers, developers and decision-makers that create intelligent systems in understanding how the ML technologies they use have implications for how people can interact with ML outputs, due to the ways in which it is generated, represented, and can be made visible to or editable by humans.  

The discussion will be seeded by three 10-minute lightening talks that highlight the complexities of designing for AI systems, and some of the differences between different ML technologies: 
• Designing Human-AI Interaction (Nur Yildirim (opens in new tab)) | video (opens in new tab)
• Enterprise foundation model of knowledge (Yordan Zaykov (opens in new tab)) | video (opens in new tab)
• Explicit and Implicit User-Interaction with Github Copilot (Andrew Rice (opens in new tab)

This will be followed by breakout sessions (~30m) in which attendees will consider how designers, developers and decision-makers can be supported in understanding how the deployment of different ML technologies has different implications for supporting user understanding of what those models are doing and how people can interact with them.
14:20Short break
14:35Report back and discussion
15:20Closing remarks
15:30End

Workshop organizers

Siân Lindley, Microsoft Research Cambridge
Yordan Zaykov, Microsoft Research Cambridge
Ida Larsen-Ledet, Microsoft Research Cambridge
Britta Burlin, Microsoft Research Cambridge

Microsoft’s Event Code of Conduct

Microsoft’s mission is to empower every person and every organization on the planet to achieve more. This includes events Microsoft hosts and participates in, where we seek to create a respectful, friendly, and inclusive experience for all participants. As such, we do not tolerate harassing or disrespectful behavior, messages, images, or interactions by any event participant, in any form, at any aspect of the program including business and social activities, regardless of location.

We do not tolerate any behavior that is degrading to any gender, race, sexual orientation or disability, or any behavior that would violate Microsoft’s Anti-Harassment and Anti-Discrimination Policy, Equal Employment Opportunity Policy, or Standards of Business Conduct (opens in new tab). In short, the entire experience at the venue must meet our culture standards. We encourage everyone to assist in creating a welcoming and safe environment. Please report (opens in new tab) any concerns, harassing behavior, or suspicious or disruptive activity to venue staff, the event host or owner, or event staff. Microsoft reserves the right to refuse admittance to or remove any person from company-sponsored events at any time in its sole discretion.