AI Chat Logs Research
AI Chat Log Research is a v-team collaboration among E+D Office of Applied Research, Microsoft Research, Bing Metrics and Analytics, and Turing to address practical challenges arising from the analysis of AI Chat logs.
Large Language Model-based AI is transforming how users interact with assistive systems. Logs of user interactions from new chat and copilot AI systems provide more extensive signals of user satisfaction, success, and enjoyment than conventional search and recommendation logs due to the rich use of natural language in the interactions. Moreover, the LLMs themselves can be leveraged to extract the signals effectively from the natural language in the logs. This combination provides a unique opportunity to utilize the chat logs to quickly understand user intent, assess satisfaction, and characterize usage to drive continuous improvements in the chat and copilot systems over time.