{"id":954237,"date":"2023-07-07T10:03:29","date_gmt":"2023-07-07T17:03:29","guid":{"rendered":"https:\/\/www.microsoft.com\/en-us\/research\/?post_type=msr-research-item&#038;p=954237"},"modified":"2023-10-13T03:38:57","modified_gmt":"2023-10-13T10:38:57","slug":"coldeco-an-end-user-spreadsheet-inspection-tool-for-ai-generated-code","status":"publish","type":"msr-research-item","link":"https:\/\/www.microsoft.com\/en-us\/research\/publication\/coldeco-an-end-user-spreadsheet-inspection-tool-for-ai-generated-code\/","title":{"rendered":"ColDeco: An End User Spreadsheet Inspection Tool for AI-Generated Code"},"content":{"rendered":"<div>\n<div>Code-generating large language models (LLMs) are transforming programming. Their capability to generate multi-step solutions provides even non-programmers a mechanism to harness the power of programming. Non-programmers typically use spreadsheets to manage tabular data, as it offers an intuitive understanding of data manipulation and formula outcomes. Considering that LLMs can generate complex, potentially incorrect code, our focus is on enabling user trust in the accuracy of LLM-generated code.<\/div>\n<div><\/div>\n<div>We present ColDeco, the first end user inspection tool for comprehending code produced by large language models for tabular data tasks. ColDeco integrates two new features for inspection with a grid-based interface. First, users can decompose a generated solution into intermediate helper columns to understand how the problem is solved. Second, users can interact with a filtered table of summary rows, which highlight interesting cases in the program. We evaluate our tool using a within-subjects user study (n=24) where participants are asked to verify the correctness of programs generated by a language model.<\/div>\n<div><\/div>\n<div>We found that while all features are independently useful and intuitive, participants preferred them in combination. Users especially noted the usefulness of helper columns, but wanted more transparency in how summary rows are generated to assist with understanding and trusting them. Users also highlighted the application of ColDeco in collaborative settings for explaining and understanding existing formulas.<\/div>\n<\/div>\n","protected":false},"excerpt":{"rendered":"<p>Code-generating large language models (LLMs) are transforming programming. Their capability to generate multi-step solutions provides even non-programmers a mechanism to harness the power of programming. Non-programmers typically use spreadsheets to manage tabular data, as it offers an intuitive understanding of data manipulation and formula outcomes. Considering that LLMs can generate complex, potentially incorrect code, our [&hellip;]<\/p>\n","protected":false},"featured_media":0,"template":"","meta":{"msr-url-field":"","msr-podcast-episode":"","msrModifiedDate":"","msrModifiedDateEnabled":false,"ep_exclude_from_search":false,"_classifai_error":"","footnotes":""},"msr-content-type":[3],"msr-research-highlight":[],"research-area":[13556,13554,13560],"msr-publication-type":[193716],"msr-product-type":[],"msr-focus-area":[],"msr-platform":[],"msr-download-source":[],"msr-locale":[268875],"msr-post-option":[],"msr-field-of-study":[246694,248485],"msr-conference":[],"msr-journal":[],"msr-impact-theme":[],"msr-pillar":[],"class_list":["post-954237","msr-research-item","type-msr-research-item","status-publish","hentry","msr-research-area-artificial-intelligence","msr-research-area-human-computer-interaction","msr-research-area-programming-languages-software-engineering","msr-locale-en_us","msr-field-of-study-artificial-intelligence","msr-field-of-study-human-computer-interaction"],"msr_publishername":"IEEE","msr_edition":"","msr_affiliation":"","msr_published_date":"2023-10-1","msr_host":"","msr_duration":"","msr_version":"","msr_speaker":"","msr_other_contributors":"","msr_booktitle":"","msr_pages_string":"","msr_chapter":"","msr_isbn":"","msr_journal":"","msr_volume":"","msr_number":"","msr_editors":"","msr_series":"","msr_issue":"","msr_organization":"IEEE","msr_how_published":"","msr_notes":"","msr_highlight_text":"","msr_release_tracker_id":"","msr_original_fields_of_study":"","msr_download_urls":"","msr_external_url":"","msr_secondary_video_url":"","msr_longbiography":"","msr_microsoftintellectualproperty":1,"msr_main_download":"","msr_publicationurl":"","msr_doi":"","msr_publication_uploader":[{"type":"file","viewUrl":"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2023\/07\/Inspection_Tools_for_End_User_Programming_in_Spreadsheets-3.pdf","id":"973506","title":"__inspection_tools_for_end_user_programming_in_spreadsheets-3","label_id":"243109","label":0}],"msr_related_uploader":"","msr_attachments":[{"id":973506,"url":"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2023\/10\/Inspection_Tools_for_End_User_Programming_in_Spreadsheets-3.pdf"}],"msr-author-ordering":[{"type":"text","value":"Kasra Ferdowsi","user_id":0,"rest_url":false},{"type":"user_nicename","value":"Jack Williams","user_id":40156,"rest_url":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/microsoft-research\/v1\/researchers?person=Jack Williams"},{"type":"user_nicename","value":"Ian Drosos","user_id":41922,"rest_url":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/microsoft-research\/v1\/researchers?person=Ian Drosos"},{"type":"user_nicename","value":"Andy Gordon","user_id":30825,"rest_url":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/microsoft-research\/v1\/researchers?person=Andy Gordon"},{"type":"user_nicename","value":"Carina Negreanu","user_id":40924,"rest_url":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/microsoft-research\/v1\/researchers?person=Carina Negreanu"},{"type":"text","value":"Nadia Polikarpova","user_id":0,"rest_url":false},{"type":"user_nicename","value":"Advait Sarkar","user_id":37146,"rest_url":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/microsoft-research\/v1\/researchers?person=Advait Sarkar"},{"type":"user_nicename","value":"Ben Zorn","user_id":35154,"rest_url":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/microsoft-research\/v1\/researchers?person=Ben Zorn"}],"msr_impact_theme":[],"msr_research_lab":[199561,199565],"msr_event":[],"msr_group":[144812],"msr_project":[1053711,717493,511097],"publication":[],"video":[],"download":[],"msr_publication_type":"inproceedings","related_content":{"projects":[{"ID":1053711,"post_title":"Tools for Thought","post_name":"t4t","post_type":"msr-project","post_date":"2024-07-03 05:21:39","post_modified":"2025-04-03 18:41:43","post_status":"publish","permalink":"https:\/\/www.microsoft.com\/en-us\/research\/project\/t4t\/","post_excerpt":"Better thinking through AI Much AI research focuses on solving specific tasks for people - generating content or automating processes. While such systems may be powerful, there are risks that this approach impact the way people think and therefore learn, build skills, and deploy expertise. The Tools for Thought (T4T) team aims to help researchers and developers imagine&nbsp;how AI might help people to think better, so that:&nbsp; Outputs of T4T include&nbsp;principles and guidelines&nbsp;for supporting cognition&hellip;","_links":{"self":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-project\/1053711"}]}},{"ID":717493,"post_title":"The New Future of Work","post_name":"the-new-future-of-work","post_type":"msr-project","post_date":"2021-01-25 07:40:51","post_modified":"2025-01-27 11:49:54","post_status":"publish","permalink":"https:\/\/www.microsoft.com\/en-us\/research\/project\/the-new-future-of-work\/","post_excerpt":"This cross-company initiative is dedicated to creating solutions for a future of work that is meaningful, productive, and equitable. The focus has shifted from remote to hybrid to the role of artificial intelligence in changing work practices.","_links":{"self":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-project\/717493"}]}},{"ID":511097,"post_title":"Calc Intelligence","post_name":"calc-intelligence","post_type":"msr-project","post_date":"2020-02-17 06:40:29","post_modified":"2023-11-27 06:33:39","post_status":"publish","permalink":"https:\/\/www.microsoft.com\/en-us\/research\/project\/calc-intelligence\/","post_excerpt":"By Calc Intelligence, we mean the research goal of bringing intelligence to end-user programming, and in particular to spreadsheets.","_links":{"self":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-project\/511097"}]}}]},"_links":{"self":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-research-item\/954237","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-research-item"}],"about":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/types\/msr-research-item"}],"version-history":[{"count":2,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-research-item\/954237\/revisions"}],"predecessor-version":[{"id":954243,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-research-item\/954237\/revisions\/954243"}],"wp:attachment":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/media?parent=954237"}],"wp:term":[{"taxonomy":"msr-content-type","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-content-type?post=954237"},{"taxonomy":"msr-research-highlight","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-research-highlight?post=954237"},{"taxonomy":"msr-research-area","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/research-area?post=954237"},{"taxonomy":"msr-publication-type","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-publication-type?post=954237"},{"taxonomy":"msr-product-type","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-product-type?post=954237"},{"taxonomy":"msr-focus-area","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-focus-area?post=954237"},{"taxonomy":"msr-platform","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-platform?post=954237"},{"taxonomy":"msr-download-source","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-download-source?post=954237"},{"taxonomy":"msr-locale","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-locale?post=954237"},{"taxonomy":"msr-post-option","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-post-option?post=954237"},{"taxonomy":"msr-field-of-study","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-field-of-study?post=954237"},{"taxonomy":"msr-conference","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-conference?post=954237"},{"taxonomy":"msr-journal","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-journal?post=954237"},{"taxonomy":"msr-impact-theme","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-impact-theme?post=954237"},{"taxonomy":"msr-pillar","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-pillar?post=954237"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}