Problem

The user needs an explanation for why the system did what it did.

Solution

Provide users with a set of relevant system-output characteristics so they can understand what properties influence the system’s behaviors.

Use when

• The output is complex or extensive.
• The system’s output characteristics can be summarized in a human-interpretable way.
• The user doesn’t understand why the system took a specific action.
• The user wants to understand the system’s logic.
• Policy or regulations require the system to make an explanation available.

How

Collaborate with an AI/ML practitioner to:

• Understand which system-output properties are relevant to the user.
• Translate output properties into characteristics meaningful to the user.

Show system outputs and their respective characteristics.

Output characteristics may be independent of each other (e.g., retail prices) or presented relative to one another (e.g. an item’s price is lower than another).

User benefits

• Enables the user to evaluate the tradeoffs that the AI system makes while deciding.
• Enables the user to pick among system outputs.
• Gives the user insights into the system’s reasoning through output characteristics.
• Enhances user trust because of system transparency.

Common pitfalls

• It’s difficult for the user to process the output characteristics because they are too many, too vague, or irrelevant.
• When presented relative to each other, the relationship between output characteristics is unclear to the user.