Generation Probabilities Are Not Enough: Exploring the Effectiveness of Uncertainty Highlighting in AI-Powered Code Completions
Helena Vasconcelos, Gagan Bansal, Adam Fourney, Q. Vera Liao, Jennifer Wortman Vaughan
ToCHI | August 2024
Q. Vera Liao
Principal Researcher
Publications
Publications by Year
-
UX Matters: The Critical Role of UX in Responsible AI
Q. Vera Liao, Mihaela Vorvoreanu, Hariharan Subramonyam, Lauren Wilcox
ACM Interactions | July 2024, Vol 31: pp. 22-27
“I’m Not Sure, But…”: Examining the Impact of Large Language Models’ Uncertainty Expression on User Reliance and Trust
Sunnie S. Y. Kim, Q. Vera Liao, Mihaela Vorvoreanu, Stephanie Ballard, Jennifer Wortman Vaughan
7th ACM Conference on Fairness, Accountability, and Transparency (FAccT 2024) | June 2024
Generative Echo Chamber? Effects of LLM-Powered Search Systems on Diverse Information Seeking
Nikhil Sharma, Q. Vera Liao, Ziang Xiao
CHI 2024 | May 2024
Best Paper, CHI 2024
The Who in XAI: How AI Background Shapes Perceptions of AI Explanations
Upol Ehsan, Samir Passi, Q. Vera Liao, Larry Chan, I-Hsiang Lee, Michael Muller, Mark O Riedl
CHI 2024 | May 2024
AI Transparency in the Age of LLMs: A Human-Centered Research Roadmap
Q. Vera Liao, Jennifer Wortman Vaughan
Harvard Data Science Review | February 2024
-
Understanding the Role of Human Intuition on Reliance in Human-AI Decision-Making with Explanations
Valerie Chen, Q. Vera Liao, Jennifer Wortman Vaughan, Gagan Bansal
Proc. ACM Hum.-Comput. Interact. | October 2023, Vol 7
Article 370
Designerly Understanding: Information Needs for Model Transparency to Support Design Ideation for AI-Powered User Experience
Q. Vera Liao, Hariharan Subramonyam, Jennifer Wang, Jennifer Wortman Vaughan
2023 ACM CHI Conference on Human Factors in Computing Systems (CHI 2023) | April 2023
fAIlureNotes: Supporting Designers in Understanding the Limits of AI Models for Computer Vision Tasks
Steven Moore, Q. Vera Liao, Hariharan Subramonyam
CHI 2023 | April 2023
Why is AI not a Panacea for Data Workers? An Interview Study on Human-AI Collaboration in Data Storytelling
Haotian Li, Yun Wang, Q. Vera Liao, Huamin Qu
MSR-TR-2023-39 | April 2023
Published by Microsoft
Powering an AI Chatbot with Expert Sourcing to Support Credible Information Access
Ziang Xiao, Q. Vera Liao, Michelle X. Zhou, Tyrone Grandison, Yunyao Li
IUI 2023 | March 2023
-
Generation Probabilities are Not Enough: Improving Error Highlighting for AI Code Suggestions
Helena Vasconcelos, Gagan Bansal, Adam Fourney, Q. Vera Liao, Jennifer Wortman Vaughan
NeurIPS Workshop on Human-Centered AI | October 2022
Connecting Algorithmic Research and Usage Contexts: A Perspective of Contextualized Evaluation for Explainable AI
Q. Vera Liao, Yunfeng Zhang, Ronny Luss, Finale Doshi-Velez, Amit Dhurandhar
AAAI Conference on Human Computation and Crowdsourcing | October 2022
Best Paper Award, Honorable Mention
Human-AI Collaboration via Conditional Delegation: A Case Study of Content Moderation
Vivian Lai, Samuel Carton, Rajat Bhatnagar, Q. Vera Liao, Yunfeng Zhang, Chenhao Tan
CHI 2022 | May 2022
Designing for Responsible Trust in AI Systems: A Communication Perspective
Q. Vera Liao, S. Shyam Sundar
FAccT 2022 | April 2022
Preprint
Responsible Language Technologies: Foreseeing and Mitigating Harms
Su Lin Blodgett, Q. Vera Liao, Alexandra Olteanu, Rada Mihalcea, Michael Muller, Morgan Klaus Scheuerman, Chenhao Tan, Qian Yang
CHI 2022 | April 2022
Publications by Research Area
-
Generation Probabilities Are Not Enough: Exploring the Effectiveness of Uncertainty Highlighting in AI-Powered Code Completions
Helena Vasconcelos, Gagan Bansal, Adam Fourney, Q. Vera Liao, Jennifer Wortman Vaughan
ToCHI | August 2024
UX Matters: The Critical Role of UX in Responsible AI
Q. Vera Liao, Mihaela Vorvoreanu, Hariharan Subramonyam, Lauren Wilcox
ACM Interactions | July 2024, Vol 31: pp. 22-27
“I’m Not Sure, But…”: Examining the Impact of Large Language Models’ Uncertainty Expression on User Reliance and Trust
Sunnie S. Y. Kim, Q. Vera Liao, Mihaela Vorvoreanu, Stephanie Ballard, Jennifer Wortman Vaughan
7th ACM Conference on Fairness, Accountability, and Transparency (FAccT 2024) | June 2024
AI Transparency in the Age of LLMs: A Human-Centered Research Roadmap
Q. Vera Liao, Jennifer Wortman Vaughan
Harvard Data Science Review | February 2024
Understanding the Role of Human Intuition on Reliance in Human-AI Decision-Making with Explanations
Valerie Chen, Q. Vera Liao, Jennifer Wortman Vaughan, Gagan Bansal
Proc. ACM Hum.-Comput. Interact. | October 2023, Vol 7
Article 370
Designerly Understanding: Information Needs for Model Transparency to Support Design Ideation for AI-Powered User Experience
Q. Vera Liao, Hariharan Subramonyam, Jennifer Wang, Jennifer Wortman Vaughan
2023 ACM CHI Conference on Human Factors in Computing Systems (CHI 2023) | April 2023
Generation Probabilities are Not Enough: Improving Error Highlighting for AI Code Suggestions
Helena Vasconcelos, Gagan Bansal, Adam Fourney, Q. Vera Liao, Jennifer Wortman Vaughan
NeurIPS Workshop on Human-Centered AI | October 2022
Connecting Algorithmic Research and Usage Contexts: A Perspective of Contextualized Evaluation for Explainable AI
Q. Vera Liao, Yunfeng Zhang, Ronny Luss, Finale Doshi-Velez, Amit Dhurandhar
AAAI Conference on Human Computation and Crowdsourcing | October 2022
Best Paper Award, Honorable Mention
Designing for Responsible Trust in AI Systems: A Communication Perspective
Q. Vera Liao, S. Shyam Sundar
FAccT 2022 | April 2022
Preprint
-
Generation Probabilities Are Not Enough: Exploring the Effectiveness of Uncertainty Highlighting in AI-Powered Code Completions
Helena Vasconcelos, Gagan Bansal, Adam Fourney, Q. Vera Liao, Jennifer Wortman Vaughan
ToCHI | August 2024
The Who in XAI: How AI Background Shapes Perceptions of AI Explanations
Upol Ehsan, Samir Passi, Q. Vera Liao, Larry Chan, I-Hsiang Lee, Michael Muller, Mark O Riedl
CHI 2024 | May 2024
Generative Echo Chamber? Effects of LLM-Powered Search Systems on Diverse Information Seeking
Nikhil Sharma, Q. Vera Liao, Ziang Xiao
CHI 2024 | May 2024
Best Paper, CHI 2024
AI Transparency in the Age of LLMs: A Human-Centered Research Roadmap
Q. Vera Liao, Jennifer Wortman Vaughan
Harvard Data Science Review | February 2024
Understanding the Role of Human Intuition on Reliance in Human-AI Decision-Making with Explanations
Valerie Chen, Q. Vera Liao, Jennifer Wortman Vaughan, Gagan Bansal
Proc. ACM Hum.-Comput. Interact. | October 2023, Vol 7
Article 370
Why is AI not a Panacea for Data Workers? An Interview Study on Human-AI Collaboration in Data Storytelling
Haotian Li, Yun Wang, Q. Vera Liao, Huamin Qu
MSR-TR-2023-39 | April 2023
Published by Microsoft
fAIlureNotes: Supporting Designers in Understanding the Limits of AI Models for Computer Vision Tasks
Steven Moore, Q. Vera Liao, Hariharan Subramonyam
CHI 2023 | April 2023
Designerly Understanding: Information Needs for Model Transparency to Support Design Ideation for AI-Powered User Experience
Q. Vera Liao, Hariharan Subramonyam, Jennifer Wang, Jennifer Wortman Vaughan
2023 ACM CHI Conference on Human Factors in Computing Systems (CHI 2023) | April 2023
Powering an AI Chatbot with Expert Sourcing to Support Credible Information Access
Ziang Xiao, Q. Vera Liao, Michelle X. Zhou, Tyrone Grandison, Yunyao Li
IUI 2023 | March 2023
Generation Probabilities are Not Enough: Improving Error Highlighting for AI Code Suggestions
Helena Vasconcelos, Gagan Bansal, Adam Fourney, Q. Vera Liao, Jennifer Wortman Vaughan
NeurIPS Workshop on Human-Centered AI | October 2022
Connecting Algorithmic Research and Usage Contexts: A Perspective of Contextualized Evaluation for Explainable AI
Q. Vera Liao, Yunfeng Zhang, Ronny Luss, Finale Doshi-Velez, Amit Dhurandhar
AAAI Conference on Human Computation and Crowdsourcing | October 2022
Best Paper Award, Honorable Mention
Human-AI Collaboration via Conditional Delegation: A Case Study of Content Moderation
Vivian Lai, Samuel Carton, Rajat Bhatnagar, Q. Vera Liao, Yunfeng Zhang, Chenhao Tan
CHI 2022 | May 2022
Designing for Responsible Trust in AI Systems: A Communication Perspective
Q. Vera Liao, S. Shyam Sundar
FAccT 2022 | April 2022
Preprint
Responsible Language Technologies: Foreseeing and Mitigating Harms
Su Lin Blodgett, Q. Vera Liao, Alexandra Olteanu, Rada Mihalcea, Michael Muller, Morgan Klaus Scheuerman, Chenhao Tan, Qian Yang
CHI 2022 | April 2022
Publications by Type
-
Generation Probabilities Are Not Enough: Exploring the Effectiveness of Uncertainty Highlighting in AI-Powered Code Completions
Helena Vasconcelos, Gagan Bansal, Adam Fourney, Q. Vera Liao, Jennifer Wortman Vaughan
ToCHI | August 2024
UX Matters: The Critical Role of UX in Responsible AI
Q. Vera Liao, Mihaela Vorvoreanu, Hariharan Subramonyam, Lauren Wilcox
ACM Interactions | July 2024, Vol 31: pp. 22-27
AI Transparency in the Age of LLMs: A Human-Centered Research Roadmap
Q. Vera Liao, Jennifer Wortman Vaughan
Harvard Data Science Review | February 2024
Understanding the Role of Human Intuition on Reliance in Human-AI Decision-Making with Explanations
Valerie Chen, Q. Vera Liao, Jennifer Wortman Vaughan, Gagan Bansal
Proc. ACM Hum.-Comput. Interact. | October 2023, Vol 7
Article 370
-
“I’m Not Sure, But…”: Examining the Impact of Large Language Models’ Uncertainty Expression on User Reliance and Trust
Sunnie S. Y. Kim, Q. Vera Liao, Mihaela Vorvoreanu, Stephanie Ballard, Jennifer Wortman Vaughan
7th ACM Conference on Fairness, Accountability, and Transparency (FAccT 2024) | June 2024
The Who in XAI: How AI Background Shapes Perceptions of AI Explanations
Upol Ehsan, Samir Passi, Q. Vera Liao, Larry Chan, I-Hsiang Lee, Michael Muller, Mark O Riedl
CHI 2024 | May 2024
Generative Echo Chamber? Effects of LLM-Powered Search Systems on Diverse Information Seeking
Nikhil Sharma, Q. Vera Liao, Ziang Xiao
CHI 2024 | May 2024
Best Paper, CHI 2024
fAIlureNotes: Supporting Designers in Understanding the Limits of AI Models for Computer Vision Tasks
Steven Moore, Q. Vera Liao, Hariharan Subramonyam
CHI 2023 | April 2023
Designerly Understanding: Information Needs for Model Transparency to Support Design Ideation for AI-Powered User Experience
Q. Vera Liao, Hariharan Subramonyam, Jennifer Wang, Jennifer Wortman Vaughan
2023 ACM CHI Conference on Human Factors in Computing Systems (CHI 2023) | April 2023
Powering an AI Chatbot with Expert Sourcing to Support Credible Information Access
Ziang Xiao, Q. Vera Liao, Michelle X. Zhou, Tyrone Grandison, Yunyao Li
IUI 2023 | March 2023
Generation Probabilities are Not Enough: Improving Error Highlighting for AI Code Suggestions
Helena Vasconcelos, Gagan Bansal, Adam Fourney, Q. Vera Liao, Jennifer Wortman Vaughan
NeurIPS Workshop on Human-Centered AI | October 2022
Connecting Algorithmic Research and Usage Contexts: A Perspective of Contextualized Evaluation for Explainable AI
Q. Vera Liao, Yunfeng Zhang, Ronny Luss, Finale Doshi-Velez, Amit Dhurandhar
AAAI Conference on Human Computation and Crowdsourcing | October 2022
Best Paper Award, Honorable Mention
Human-AI Collaboration via Conditional Delegation: A Case Study of Content Moderation
Vivian Lai, Samuel Carton, Rajat Bhatnagar, Q. Vera Liao, Yunfeng Zhang, Chenhao Tan
CHI 2022 | May 2022
Designing for Responsible Trust in AI Systems: A Communication Perspective
Q. Vera Liao, S. Shyam Sundar
FAccT 2022 | April 2022
Preprint
Responsible Language Technologies: Foreseeing and Mitigating Harms
Su Lin Blodgett, Q. Vera Liao, Alexandra Olteanu, Rada Mihalcea, Michael Muller, Morgan Klaus Scheuerman, Chenhao Tan, Qian Yang
CHI 2022 | April 2022
-
Why is AI not a Panacea for Data Workers? An Interview Study on Human-AI Collaboration in Data Storytelling
Haotian Li, Yun Wang, Q. Vera Liao, Huamin Qu
MSR-TR-2023-39 | April 2023
Published by Microsoft