Problem
The system adapts as it learns over time and it needs to update its outputs without disrupting or disorienting the user.
Solution
The system makes a controlled and deliberate comprehensive update in response to user behaviors or other additional data.
Use when
- The user would benefit from an updated set of outputs.
- The model has sufficient additional information for updating.
- If updating in response to user behavior: The model is capable of accepting additional information from user interactions, interpreting user intent, and adapting over time.
How
Update system outputs in a way that permeates the whole user experience, not just an isolated part.
Make the update in a controlled and deliberate manner by matching the scale and the rate of the change to user expectations.
Scale: Balance the proportion of new, updated outputs to previous ones.
Rate: Control how quickly the update is reflected in the user experience. For example, the updated outputs can be released gradually or all at once, depending on user expectations and context.
Collaborate with an AI/ML practitioner to decide how to infer user intent from user behaviors (if applicable) and to adjust the nature, scale, and rate of the update.
User benefits
- Provides a better user experience through improved or new functionality, and/or personalization.
- Provides value to users by improving relevance of content.
- Enables the user to discover new relevant content.
Common pitfalls
- The system triggers the update based on insufficient or misinterpreted information and mischaracterizes user intent.
- The scale and/or rate of change is mismatched with user expectations.
- For example, delivering a very high proportion of updated to previous outputs quickly can be disrupting and disorienting if it floods the user with new outputs and makes it difficult to maintain the previous context.
- Conversely, if the user expects the system to have learned something (e.g., a name), a slow and gradual change can be disorienting because the user would be unsure whether the system learned.
References
- Bansal, G., Nushi, B., Kamar, E., Weld, D.S., Lasecki, W.S. and Horvitz, E. 2019. Updates in Human-AI Teams: Understanding and Addressing the Performance/Compatibility Tradeoff. Proceedings of the AAAI Conference on Artificial Intelligence. 33, 01 (Jul. 2019), 2429-2437. DOI: https://doi.org/10.1609/aaai.v33i01.33012429.
