Chad Atalla

Position: Evaluating Generative AI Systems is a Social Science Measurement Challenge

Hanna Wallach, Meera Desai, A. Feder Cooper, Angelina Wang, Chad Atalla, Solon Barocas, Su Lin Blodgett, Alex Chouldechova, Emily Corvi, P. A. Dow, Jean Garcia-Gathright, Alexandra Olteanu, Nick Pangakis, Stefanie Reed, Emily Sheng, Dan Vann, Jennifer Wortman Vaughan, Matthew Vogel, Hannah Washington, Abigail Z. Jacobs

January 2025

Publication

A Shared Standard for Valid Measurement of Generative AI Systems’ Capabilities, Risks, and Impacts

Alex Chouldechova, Chad Atalla, Solon Barocas, A. Feder Cooper, Emily Corvi, P. A. Dow, Jean Garcia-Gathright, Nick Pangakis, Stefanie Reed, Emily Sheng, Dan Vann, Matthew Vogel, Hannah Washington, Hanna Wallach

December 2024

Publication

Dimensions of Generative AI Evaluation Design

P. A. Dow, Jennifer Wortman Vaughan, Solon Barocas, Chad Atalla, Alex Chouldechova, Hanna Wallach

November 2024

Publication

Evaluating Generative AI Systems is a Social Science Measurement Challenge

Hanna Wallach, Meera Desai, Nick Pangakis, A. Feder Cooper, Angelina Wang, Solon Barocas, Alex Chouldechova, Chad Atalla, Su Lin Blodgett, Emily Corvi, P. A. Dow, Jean Garcia-Gathright, Alexandra Olteanu, Stefanie Reed, Emily Sheng, Dan Vann, Jennifer Wortman Vaughan, Matthew Vogel, Hannah Washington, Abigail Z. Jacobs

November 2024

Publication

A Framework for Automated Measurement of Responsible AI Harms in Generative AI Applications

Ahmed Magooda, Alec Helyar, Kyle Jackson, David Sullivan, Chad Atalla, Emily Sheng, Dan Vann, Richard Edgar, Hamid Palangi, Roman Lutz, Hongliang Kong, Vincent Yun, Eslam Kamal, Federico Zarfati, Hanna Wallach, Sarah Bird, Mei Chen

ArXiv | October 2023, Vol abs/2310.17750

DOI Publication Publication

A Framework for Automated Measurement of Responsible AI Harms in Generative AI Applications

Ahmed Magooda, Alec Helyar, Kyle Jackson, David Sullivan, Chad Atalla, Emily Sheng, Dan Vann, Richard Edgar, Hamid Palangi, Roman Lutz, Hongliang Kong, Vincent Yun, Eslam Kamal, Federico Zarfati, Hanna Wallach, Sarah Bird, Mei Chen

October 2023

Publication

FairPrism: Evaluating Fairness-Related Harms in Text Generation

Eve Fleisig, Aubrie Amstutz, Chad Atalla, Su Lin Blodgett, Hal Daumé III, Alexandra Olteanu, Emily Sheng, Dan Vann, Hanna Wallach

ACL 2023 | July 2023

Publication

When does text prediction benefit from additional context? An exploration of contextual signals for chat and email messages

Stojan Trajanovski, Chad Atalla, Kunho Kim, Vipul Agarwal, Milad Shokouhi, Chris Quirk

NAACL 2021 | June 2021

PDF Publication

Position: Evaluating Generative AI Systems is a Social Science Measurement Challenge

Hanna Wallach, Meera Desai, A. Feder Cooper, Angelina Wang, Chad Atalla, Solon Barocas, Su Lin Blodgett, Alex Chouldechova, Emily Corvi, P. A. Dow, Jean Garcia-Gathright, Alexandra Olteanu, Nick Pangakis, Stefanie Reed, Emily Sheng, Dan Vann, Jennifer Wortman Vaughan, Matthew Vogel, Hannah Washington, Abigail Z. Jacobs

January 2025

Publication

A Shared Standard for Valid Measurement of Generative AI Systems’ Capabilities, Risks, and Impacts

Alex Chouldechova, Chad Atalla, Solon Barocas, A. Feder Cooper, Emily Corvi, P. A. Dow, Jean Garcia-Gathright, Nick Pangakis, Stefanie Reed, Emily Sheng, Dan Vann, Matthew Vogel, Hannah Washington, Hanna Wallach

December 2024

Publication

Dimensions of Generative AI Evaluation Design

P. A. Dow, Jennifer Wortman Vaughan, Solon Barocas, Chad Atalla, Alex Chouldechova, Hanna Wallach

November 2024

Publication

Evaluating Generative AI Systems is a Social Science Measurement Challenge

Hanna Wallach, Meera Desai, Nick Pangakis, A. Feder Cooper, Angelina Wang, Solon Barocas, Alex Chouldechova, Chad Atalla, Su Lin Blodgett, Emily Corvi, P. A. Dow, Jean Garcia-Gathright, Alexandra Olteanu, Stefanie Reed, Emily Sheng, Dan Vann, Jennifer Wortman Vaughan, Matthew Vogel, Hannah Washington, Abigail Z. Jacobs

November 2024

Publication

A Framework for Automated Measurement of Responsible AI Harms in Generative AI Applications

Ahmed Magooda, Alec Helyar, Kyle Jackson, David Sullivan, Chad Atalla, Emily Sheng, Dan Vann, Richard Edgar, Hamid Palangi, Roman Lutz, Hongliang Kong, Vincent Yun, Eslam Kamal, Federico Zarfati, Hanna Wallach, Sarah Bird, Mei Chen

ArXiv | October 2023, Vol abs/2310.17750

DOI Publication Publication

A Framework for Automated Measurement of Responsible AI Harms in Generative AI Applications

Ahmed Magooda, Alec Helyar, Kyle Jackson, David Sullivan, Chad Atalla, Emily Sheng, Dan Vann, Richard Edgar, Hamid Palangi, Roman Lutz, Hongliang Kong, Vincent Yun, Eslam Kamal, Federico Zarfati, Hanna Wallach, Sarah Bird, Mei Chen

October 2023

Publication

FairPrism: Evaluating Fairness-Related Harms in Text Generation

Eve Fleisig, Aubrie Amstutz, Chad Atalla, Su Lin Blodgett, Hal Daumé III, Alexandra Olteanu, Emily Sheng, Dan Vann, Hanna Wallach

ACL 2023 | July 2023

Publication

When does text prediction benefit from additional context? An exploration of contextual signals for chat and email messages

Stojan Trajanovski, Chad Atalla, Kunho Kim, Vipul Agarwal, Milad Shokouhi, Chris Quirk

NAACL 2021 | June 2021

PDF Publication

Position: Evaluating Generative AI Systems is a Social Science Measurement Challenge

Hanna Wallach, Meera Desai, A. Feder Cooper, Angelina Wang, Chad Atalla, Solon Barocas, Su Lin Blodgett, Alex Chouldechova, Emily Corvi, P. A. Dow, Jean Garcia-Gathright, Alexandra Olteanu, Nick Pangakis, Stefanie Reed, Emily Sheng, Dan Vann, Jennifer Wortman Vaughan, Matthew Vogel, Hannah Washington, Abigail Z. Jacobs

January 2025

Publication

A Shared Standard for Valid Measurement of Generative AI Systems’ Capabilities, Risks, and Impacts

Alex Chouldechova, Chad Atalla, Solon Barocas, A. Feder Cooper, Emily Corvi, P. A. Dow, Jean Garcia-Gathright, Nick Pangakis, Stefanie Reed, Emily Sheng, Dan Vann, Matthew Vogel, Hannah Washington, Hanna Wallach

December 2024

Publication

Dimensions of Generative AI Evaluation Design

P. A. Dow, Jennifer Wortman Vaughan, Solon Barocas, Chad Atalla, Alex Chouldechova, Hanna Wallach

November 2024

Publication

Evaluating Generative AI Systems is a Social Science Measurement Challenge

Hanna Wallach, Meera Desai, Nick Pangakis, A. Feder Cooper, Angelina Wang, Solon Barocas, Alex Chouldechova, Chad Atalla, Su Lin Blodgett, Emily Corvi, P. A. Dow, Jean Garcia-Gathright, Alexandra Olteanu, Stefanie Reed, Emily Sheng, Dan Vann, Jennifer Wortman Vaughan, Matthew Vogel, Hannah Washington, Abigail Z. Jacobs

November 2024

Publication

A Framework for Automated Measurement of Responsible AI Harms in Generative AI Applications

Ahmed Magooda, Alec Helyar, Kyle Jackson, David Sullivan, Chad Atalla, Emily Sheng, Dan Vann, Richard Edgar, Hamid Palangi, Roman Lutz, Hongliang Kong, Vincent Yun, Eslam Kamal, Federico Zarfati, Hanna Wallach, Sarah Bird, Mei Chen

ArXiv | October 2023, Vol abs/2310.17750

DOI Publication Publication

When does text prediction benefit from additional context? An exploration of contextual signals for chat and email messages

Stojan Trajanovski, Chad Atalla, Kunho Kim, Vipul Agarwal, Milad Shokouhi, Chris Quirk

NAACL 2021 | June 2021

PDF Publication

Position: Evaluating Generative AI Systems is a Social Science Measurement Challenge

Hanna Wallach, Meera Desai, A. Feder Cooper, Angelina Wang, Chad Atalla, Solon Barocas, Su Lin Blodgett, Alex Chouldechova, Emily Corvi, P. A. Dow, Jean Garcia-Gathright, Alexandra Olteanu, Nick Pangakis, Stefanie Reed, Emily Sheng, Dan Vann, Jennifer Wortman Vaughan, Matthew Vogel, Hannah Washington, Abigail Z. Jacobs

January 2025

Publication

A Shared Standard for Valid Measurement of Generative AI Systems’ Capabilities, Risks, and Impacts

Alex Chouldechova, Chad Atalla, Solon Barocas, A. Feder Cooper, Emily Corvi, P. A. Dow, Jean Garcia-Gathright, Nick Pangakis, Stefanie Reed, Emily Sheng, Dan Vann, Matthew Vogel, Hannah Washington, Hanna Wallach

December 2024

Publication

Dimensions of Generative AI Evaluation Design

P. A. Dow, Jennifer Wortman Vaughan, Solon Barocas, Chad Atalla, Alex Chouldechova, Hanna Wallach

November 2024

Publication

Evaluating Generative AI Systems is a Social Science Measurement Challenge

Hanna Wallach, Meera Desai, Nick Pangakis, A. Feder Cooper, Angelina Wang, Solon Barocas, Alex Chouldechova, Chad Atalla, Su Lin Blodgett, Emily Corvi, P. A. Dow, Jean Garcia-Gathright, Alexandra Olteanu, Stefanie Reed, Emily Sheng, Dan Vann, Jennifer Wortman Vaughan, Matthew Vogel, Hannah Washington, Abigail Z. Jacobs

November 2024

Publication

A Framework for Automated Measurement of Responsible AI Harms in Generative AI Applications

Ahmed Magooda, Alec Helyar, Kyle Jackson, David Sullivan, Chad Atalla, Emily Sheng, Dan Vann, Richard Edgar, Hamid Palangi, Roman Lutz, Hongliang Kong, Vincent Yun, Eslam Kamal, Federico Zarfati, Hanna Wallach, Sarah Bird, Mei Chen

ArXiv | October 2023, Vol abs/2310.17750

DOI Publication Publication

A Framework for Automated Measurement of Responsible AI Harms in Generative AI Applications

Ahmed Magooda, Alec Helyar, Kyle Jackson, David Sullivan, Chad Atalla, Emily Sheng, Dan Vann, Richard Edgar, Hamid Palangi, Roman Lutz, Hongliang Kong, Vincent Yun, Eslam Kamal, Federico Zarfati, Hanna Wallach, Sarah Bird, Mei Chen

October 2023

Publication

FairPrism: Evaluating Fairness-Related Harms in Text Generation

Eve Fleisig, Aubrie Amstutz, Chad Atalla, Su Lin Blodgett, Hal Daumé III, Alexandra Olteanu, Emily Sheng, Dan Vann, Hanna Wallach

ACL 2023 | July 2023

Publication

When does text prediction benefit from additional context? An exploration of contextual signals for chat and email messages

Stojan Trajanovski, Chad Atalla, Kunho Kim, Vipul Agarwal, Milad Shokouhi, Chris Quirk

NAACL 2021 | June 2021

PDF Publication