Emily Sheng

Senior Applied Scientist

Publications

View by:

- ASSERT: A Measurement Pipeline for GenAI Audits
  
  Riccardo Fogliato, Abhinav Palia, Xiawei Wang, Emily Sheng, Chad Atalla, Jean Garcia-Gathright, Nicholas Pangakis, Sharman Tan, Dan Vann, Hannah Washington, P. A. Dow, Heba Elfardy, Hanna Wallach, Sandeep Atluri
  
  August 2026
  
  arXiv | August 2026
  
  Publication
- AI-Assisted Systematization for Evaluating GenAI Systems
  
  Dhruv Agarwal, Emily Sheng, Chad Atalla, Jean Garcia-Gathright, Hussein Mozannar, Hannah Washington, Alexandra Chouldechova, Solon Barocas, Hanna Wallach
  
  May 2026
  
  arXiv | May 2026
  
  Publication
- Understanding and Meeting Practitioner Needs When Measuring Representational Harms Caused by LLM-Based Systems
  
  Emma Harvey, Emily Sheng, Su Lin Blodgett, Alex Chouldechova, Jean Garcia-Gathright, Alexandra Olteanu, Hanna Wallach
  
  Findings of the Association for Computational Linguistics: ACL 2025 | July 2025
  
  Publication
- Taxonomizing Representational Harms using Speech Act Theory
  
  Emily Corvi, Hannah Washington, Stefanie Reed, Chad Atalla, Alex Chouldechova, Alex Dow, Jean Garcia-Gathright, Nick Pangakis, Emily Sheng, Dan Vann, Matthew Vogel, Hanna Wallach
  
  March 2025
  
  March 2025
  
  Publication
- Position: Evaluating Generative AI Systems is a Social Science Measurement Challenge
  
  Hanna Wallach, Meera Desai, A. Feder Cooper, Angelina Wang, Chad Atalla, Solon Barocas, Su Lin Blodgett, Alex Chouldechova, Emily Corvi, Alex Dow, Jean Garcia-Gathright, Alexandra Olteanu, Nick Pangakis, Stefanie Reed, Emily Sheng, Dan Vann, Jennifer Wortman Vaughan, Matthew Vogel, Hannah Washington, Abigail Z. Jacobs
  
  ICML 2025 | January 2025
  
  Publication
- A Shared Standard for Valid Measurement of Generative AI Systems’ Capabilities, Risks, and Impacts
  
  Alex Chouldechova, Chad Atalla, Solon Barocas, A. Feder Cooper, Emily Corvi, Alex Dow, Jean Garcia-Gathright, Nick Pangakis, Stefanie Reed, Emily Sheng, Dan Vann, Matthew Vogel, Hannah Washington, Hanna Wallach
  
  December 2024
  
  December 2024
  
  Publication
- Gaps Between Research and Practice When Measuring Representational Harms Caused by LLM-Based Systems
  
  Emma Harvey, Emily Sheng, Su Lin Blodgett, Alex Chouldechova, Jean Garcia-Gathright, Alexandra Olteanu, Hanna Wallach
  
  November 2024
  
  November 2024
  
  Publication
- Evaluating Generative AI Systems is a Social Science Measurement Challenge
  
  Hanna Wallach, Meera Desai, Nick Pangakis, A. Feder Cooper, Angelina Wang, Solon Barocas, Alex Chouldechova, Chad Atalla, Su Lin Blodgett, Emily Corvi, Alex Dow, Jean Garcia-Gathright, Alexandra Olteanu, Stefanie Reed, Emily Sheng, Dan Vann, Jennifer Wortman Vaughan, Matthew Vogel, Hannah Washington, Abigail Z. Jacobs
  
  November 2024
  
  November 2024
  
  Publication
- A Framework for Automated Measurement of Responsible AI Harms in Generative AI Applications
  
  Ahmed Magooda, Alec Helyar, Kyle Jackson, David Sullivan, Chad Atalla, Emily Sheng, Dan Vann, Richard Edgar, Hamid Palangi, Roman Lutz, Hongliang Kong, Vincent Yun, Eslam Kamal, Federico Zarfati, Hanna Wallach, Sarah Bird, Mei Chen
  
  ArXiv | October 2023
  
  DOI Publication Publication

- ASSERT: A Measurement Pipeline for GenAI Audits
  
  Riccardo Fogliato, Abhinav Palia, Xiawei Wang, Emily Sheng, Chad Atalla, Jean Garcia-Gathright, Nicholas Pangakis, Sharman Tan, Dan Vann, Hannah Washington, P. A. Dow, Heba Elfardy, Hanna Wallach, Sandeep Atluri
  
  August 2026
  
  arXiv | August 2026
  
  Publication
- AI-Assisted Systematization for Evaluating GenAI Systems
  
  Dhruv Agarwal, Emily Sheng, Chad Atalla, Jean Garcia-Gathright, Hussein Mozannar, Hannah Washington, Alexandra Chouldechova, Solon Barocas, Hanna Wallach
  
  May 2026
  
  arXiv | May 2026
  
  Publication
- Taxonomizing Representational Harms using Speech Act Theory
  
  Emily Corvi, Hannah Washington, Stefanie Reed, Chad Atalla, Alex Chouldechova, Alex Dow, Jean Garcia-Gathright, Nick Pangakis, Emily Sheng, Dan Vann, Matthew Vogel, Hanna Wallach
  
  March 2025
  
  March 2025
  
  Publication
- Position: Evaluating Generative AI Systems is a Social Science Measurement Challenge
  
  Hanna Wallach, Meera Desai, A. Feder Cooper, Angelina Wang, Chad Atalla, Solon Barocas, Su Lin Blodgett, Alex Chouldechova, Emily Corvi, Alex Dow, Jean Garcia-Gathright, Alexandra Olteanu, Nick Pangakis, Stefanie Reed, Emily Sheng, Dan Vann, Jennifer Wortman Vaughan, Matthew Vogel, Hannah Washington, Abigail Z. Jacobs
  
  ICML 2025 | January 2025
  
  Publication
- A Shared Standard for Valid Measurement of Generative AI Systems’ Capabilities, Risks, and Impacts
  
  Alex Chouldechova, Chad Atalla, Solon Barocas, A. Feder Cooper, Emily Corvi, Alex Dow, Jean Garcia-Gathright, Nick Pangakis, Stefanie Reed, Emily Sheng, Dan Vann, Matthew Vogel, Hannah Washington, Hanna Wallach
  
  December 2024
  
  December 2024
  
  Publication
- Gaps Between Research and Practice When Measuring Representational Harms Caused by LLM-Based Systems
  
  Emma Harvey, Emily Sheng, Su Lin Blodgett, Alex Chouldechova, Jean Garcia-Gathright, Alexandra Olteanu, Hanna Wallach
  
  November 2024
  
  November 2024
  
  Publication
- Evaluating Generative AI Systems is a Social Science Measurement Challenge
  
  Hanna Wallach, Meera Desai, Nick Pangakis, A. Feder Cooper, Angelina Wang, Solon Barocas, Alex Chouldechova, Chad Atalla, Su Lin Blodgett, Emily Corvi, Alex Dow, Jean Garcia-Gathright, Alexandra Olteanu, Stefanie Reed, Emily Sheng, Dan Vann, Jennifer Wortman Vaughan, Matthew Vogel, Hannah Washington, Abigail Z. Jacobs
  
  November 2024
  
  November 2024
  
  Publication
- A Framework for Automated Measurement of Responsible AI Harms in Generative AI Applications
  
  Ahmed Magooda, Alec Helyar, Kyle Jackson, David Sullivan, Chad Atalla, Emily Sheng, Dan Vann, Richard Edgar, Hamid Palangi, Roman Lutz, Hongliang Kong, Vincent Yun, Eslam Kamal, Federico Zarfati, Hanna Wallach, Sarah Bird, Mei Chen
  
  ArXiv | October 2023
  
  DOI Publication Publication
- AI-Assisted Systematization for Evaluating GenAI Systems
  
  Dhruv Agarwal, Emily Sheng, Chad Atalla, Jean Garcia-Gathright, Hussein Mozannar, Hannah Washington, Alexandra Chouldechova, Solon Barocas, Hanna Wallach
  
  May 2026
  
  arXiv | May 2026
  
  Publication
- Understanding and Meeting Practitioner Needs When Measuring Representational Harms Caused by LLM-Based Systems
  
  Emma Harvey, Emily Sheng, Su Lin Blodgett, Alex Chouldechova, Jean Garcia-Gathright, Alexandra Olteanu, Hanna Wallach
  
  Findings of the Association for Computational Linguistics: ACL 2025 | July 2025
  
  Publication
- Understanding and Meeting Practitioner Needs When Measuring Representational Harms Caused by LLM-Based Systems
  
  Emma Harvey, Emily Sheng, Su Lin Blodgett, Alex Chouldechova, Jean Garcia-Gathright, Alexandra Olteanu, Hanna Wallach
  
  Findings of the Association for Computational Linguistics: ACL 2025 | July 2025
  
  Publication
- Taxonomizing Representational Harms using Speech Act Theory
  
  Emily Corvi, Hannah Washington, Stefanie Reed, Chad Atalla, Alex Chouldechova, Alex Dow, Jean Garcia-Gathright, Nick Pangakis, Emily Sheng, Dan Vann, Matthew Vogel, Hanna Wallach
  
  March 2025
  
  March 2025
  
  Publication
- Position: Evaluating Generative AI Systems is a Social Science Measurement Challenge
  
  Hanna Wallach, Meera Desai, A. Feder Cooper, Angelina Wang, Chad Atalla, Solon Barocas, Su Lin Blodgett, Alex Chouldechova, Emily Corvi, Alex Dow, Jean Garcia-Gathright, Alexandra Olteanu, Nick Pangakis, Stefanie Reed, Emily Sheng, Dan Vann, Jennifer Wortman Vaughan, Matthew Vogel, Hannah Washington, Abigail Z. Jacobs
  
  ICML 2025 | January 2025
  
  Publication
- A Shared Standard for Valid Measurement of Generative AI Systems’ Capabilities, Risks, and Impacts
  
  Alex Chouldechova, Chad Atalla, Solon Barocas, A. Feder Cooper, Emily Corvi, Alex Dow, Jean Garcia-Gathright, Nick Pangakis, Stefanie Reed, Emily Sheng, Dan Vann, Matthew Vogel, Hannah Washington, Hanna Wallach
  
  December 2024
  
  December 2024
  
  Publication
- Gaps Between Research and Practice When Measuring Representational Harms Caused by LLM-Based Systems
  
  Emma Harvey, Emily Sheng, Su Lin Blodgett, Alex Chouldechova, Jean Garcia-Gathright, Alexandra Olteanu, Hanna Wallach
  
  November 2024
  
  November 2024
  
  Publication
- Evaluating Generative AI Systems is a Social Science Measurement Challenge
  
  Hanna Wallach, Meera Desai, Nick Pangakis, A. Feder Cooper, Angelina Wang, Solon Barocas, Alex Chouldechova, Chad Atalla, Su Lin Blodgett, Emily Corvi, Alex Dow, Jean Garcia-Gathright, Alexandra Olteanu, Stefanie Reed, Emily Sheng, Dan Vann, Jennifer Wortman Vaughan, Matthew Vogel, Hannah Washington, Abigail Z. Jacobs
  
  November 2024
  
  November 2024
  
  Publication
- A Framework for Automated Measurement of Responsible AI Harms in Generative AI Applications
  
  Ahmed Magooda, Alec Helyar, Kyle Jackson, David Sullivan, Chad Atalla, Emily Sheng, Dan Vann, Richard Edgar, Hamid Palangi, Roman Lutz, Hongliang Kong, Vincent Yun, Eslam Kamal, Federico Zarfati, Hanna Wallach, Sarah Bird, Mei Chen
  
  ArXiv | October 2023
  
  DOI Publication Publication

- ASSERT: A Measurement Pipeline for GenAI Audits
  
  Riccardo Fogliato, Abhinav Palia, Xiawei Wang, Emily Sheng, Chad Atalla, Jean Garcia-Gathright, Nicholas Pangakis, Sharman Tan, Dan Vann, Hannah Washington, P. A. Dow, Heba Elfardy, Hanna Wallach, Sandeep Atluri
  
  August 2026
  
  arXiv | August 2026
  
  Publication
- AI-Assisted Systematization for Evaluating GenAI Systems
  
  Dhruv Agarwal, Emily Sheng, Chad Atalla, Jean Garcia-Gathright, Hussein Mozannar, Hannah Washington, Alexandra Chouldechova, Solon Barocas, Hanna Wallach
  
  May 2026
  
  arXiv | May 2026
  
  Publication
- Taxonomizing Representational Harms using Speech Act Theory
  
  Emily Corvi, Hannah Washington, Stefanie Reed, Chad Atalla, Alex Chouldechova, Alex Dow, Jean Garcia-Gathright, Nick Pangakis, Emily Sheng, Dan Vann, Matthew Vogel, Hanna Wallach
  
  March 2025
  
  March 2025
  
  Publication
- A Shared Standard for Valid Measurement of Generative AI Systems’ Capabilities, Risks, and Impacts
  
  Alex Chouldechova, Chad Atalla, Solon Barocas, A. Feder Cooper, Emily Corvi, Alex Dow, Jean Garcia-Gathright, Nick Pangakis, Stefanie Reed, Emily Sheng, Dan Vann, Matthew Vogel, Hannah Washington, Hanna Wallach
  
  December 2024
  
  December 2024
  
  Publication
- Gaps Between Research and Practice When Measuring Representational Harms Caused by LLM-Based Systems
  
  Emma Harvey, Emily Sheng, Su Lin Blodgett, Alex Chouldechova, Jean Garcia-Gathright, Alexandra Olteanu, Hanna Wallach
  
  November 2024
  
  November 2024
  
  Publication
- Evaluating Generative AI Systems is a Social Science Measurement Challenge
  
  Hanna Wallach, Meera Desai, Nick Pangakis, A. Feder Cooper, Angelina Wang, Solon Barocas, Alex Chouldechova, Chad Atalla, Su Lin Blodgett, Emily Corvi, Alex Dow, Jean Garcia-Gathright, Alexandra Olteanu, Stefanie Reed, Emily Sheng, Dan Vann, Jennifer Wortman Vaughan, Matthew Vogel, Hannah Washington, Abigail Z. Jacobs
  
  November 2024
  
  November 2024
  
  Publication
- Understanding and Meeting Practitioner Needs When Measuring Representational Harms Caused by LLM-Based Systems
  
  Emma Harvey, Emily Sheng, Su Lin Blodgett, Alex Chouldechova, Jean Garcia-Gathright, Alexandra Olteanu, Hanna Wallach
  
  Findings of the Association for Computational Linguistics: ACL 2025 | July 2025
  
  Publication
- Position: Evaluating Generative AI Systems is a Social Science Measurement Challenge
  
  Hanna Wallach, Meera Desai, A. Feder Cooper, Angelina Wang, Chad Atalla, Solon Barocas, Su Lin Blodgett, Alex Chouldechova, Emily Corvi, Alex Dow, Jean Garcia-Gathright, Alexandra Olteanu, Nick Pangakis, Stefanie Reed, Emily Sheng, Dan Vann, Jennifer Wortman Vaughan, Matthew Vogel, Hannah Washington, Abigail Z. Jacobs
  
  ICML 2025 | January 2025
  
  Publication
- A Framework for Automated Measurement of Responsible AI Harms in Generative AI Applications
  
  Ahmed Magooda, Alec Helyar, Kyle Jackson, David Sullivan, Chad Atalla, Emily Sheng, Dan Vann, Richard Edgar, Hamid Palangi, Roman Lutz, Hongliang Kong, Vincent Yun, Eslam Kamal, Federico Zarfati, Hanna Wallach, Sarah Bird, Mei Chen
  
  ArXiv | October 2023
  
  DOI Publication Publication

Emily Sheng

Publications

ASSERT: A Measurement Pipeline for GenAI Audits

AI-Assisted Systematization for Evaluating GenAI Systems

Understanding and Meeting Practitioner Needs When Measuring Representational Harms Caused by LLM-Based Systems

Taxonomizing Representational Harms using Speech Act Theory

Position: Evaluating Generative AI Systems is a Social Science Measurement Challenge

A Shared Standard for Valid Measurement of Generative AI Systems’ Capabilities, Risks, and Impacts

Gaps Between Research and Practice When Measuring Representational Harms Caused by LLM-Based Systems

Evaluating Generative AI Systems is a Social Science Measurement Challenge

A Framework for Automated Measurement of Responsible AI Harms in Generative AI Applications

ASSERT: A Measurement Pipeline for GenAI Audits

AI-Assisted Systematization for Evaluating GenAI Systems

Taxonomizing Representational Harms using Speech Act Theory

Position: Evaluating Generative AI Systems is a Social Science Measurement Challenge

A Shared Standard for Valid Measurement of Generative AI Systems’ Capabilities, Risks, and Impacts

Gaps Between Research and Practice When Measuring Representational Harms Caused by LLM-Based Systems

Evaluating Generative AI Systems is a Social Science Measurement Challenge

A Framework for Automated Measurement of Responsible AI Harms in Generative AI Applications

AI-Assisted Systematization for Evaluating GenAI Systems

Understanding and Meeting Practitioner Needs When Measuring Representational Harms Caused by LLM-Based Systems

ASSERT: A Measurement Pipeline for GenAI Audits

AI-Assisted Systematization for Evaluating GenAI Systems

Taxonomizing Representational Harms using Speech Act Theory

A Shared Standard for Valid Measurement of Generative AI Systems’ Capabilities, Risks, and Impacts

Gaps Between Research and Practice When Measuring Representational Harms Caused by LLM-Based Systems

Evaluating Generative AI Systems is a Social Science Measurement Challenge

Understanding and Meeting Practitioner Needs When Measuring Representational Harms Caused by LLM-Based Systems

Position: Evaluating Generative AI Systems is a Social Science Measurement Challenge

A Framework for Automated Measurement of Responsible AI Harms in Generative AI Applications

Contact Emily Sheng

Microsoft Research Lab – New York City