METAREFLECTION: Learning Instructions for Language Agents using Past Reflections
Priyanshu Gupta, Shashank Kirtania, Ananya Singha, Sumit Gulwani, Arjun Radhakrishna, Sherry Shi, Gustavo Soares
EMNLP'24 | October 2024
Gustavo Soares
Principal Research Manager
Publications
Publications by Year
-
STACKFEED: Structured Textual Actor-Critic Knowledge Base Editing with Feedback
Naman Gupta, Shashank Kirtania, Priyanshu Gupta, Krishna Kariya, Sumit Gulwani, Arun Iyer, Suresh Parthasarathy, Arjun Radhakrishna, Sriram Rajamani, Gustavo Soares
October 2024
Preprint
Let’s Fix this Together: Conversational Debugging with GitHub Copilot
Yasharth Bajpai, Bhavya Chopra, Param Biyani, Cagri Aslan, Sumit Gulwani, Dustin Coleman, Chris Parnin, Arjun Radhakrishna, Gustavo Soares
IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC) | August 2024
Best Research Paper Award
RUBICON: Rubric-based Evaluation of Domain Specific Human-AI Conversations
Param Biyani, Yasharth Bajpai, Arjun Radhakrishna, Gustavo Soares, Sumit Gulwani
AIware: Proceedings of the 1st ACM International Conference on AI-Powered Software | July 2024
Best Paper Presentation Award
Automating Human Tutor-Style Programming Feedback: Leveraging GPT-4 Tutor Model for Hint Generation and GPT-3.5 Student Model for Hint Validation
Tung Phung, Victor-Alexandru Pădurean, Anjali Singh, Christopher Brooks, José Cambronero, Sumit Gulwani, Adish Singla, Gustavo Soares
International Learning Analytics and Knowledge Conference | March 2024
Exploring Interaction Patterns for Debugging: Enhancing Conversational Capabilities of AI-assistants
Bhavya Chopra, Yasharth Bajpai, Param Biyani, Gustavo Soares, Arjun Radhakrishna, Chris Parnin, Sumit Gulwani
NAACL: Proceedings of the Third Workshop on Bridging Human-Computer Interaction and Natural Language Processing | February 2024
Investigating Student Mistakes in Introductory Data Science Programming
Anjali Singh, Anna Fariha, Christopher Brooks, Gustavo Soares, Austin Z. Henley, Ashish Tiwari, Chethan M, Heeryung Choi, Sumit Gulwani
Technical Symposium on Computer Science Education | January 2024
PyDex: Repairing Bugs in Introductory Python Assignments using LLMs
Jialu Zhang, José Cambronero, Sumit Gulwani, Vu Le, Ruzica Piskac, Gustavo Soares, Gust Verbruggen
Object-oriented Programming, Systems, Languages, and Applications (OOPSLA) | January 2024
-
Grace: Language Models Meet Code Edits
Priyanshu Gupta, Avishree Khare, Yasharth Bajpai, Saikat Chakraborty, Sumit Gulwani, Aditya Kanade, Arjun Radhakrishna, Gustavo Soares, Ashish Tiwari
ESEC/FSE 2023: Proceedings of the 31st ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering | November 2023
Generative AI for Programming Education: Benchmarking ChatGPT, GPT-4, and Human Tutors
Tung Phung, Victor-Alexandru Padurean, José Cambronero, Sumit Gulwani, Tobias Kohn, Rupak Majumdar, Adish Singla, Gustavo Soares
Conference on International Computing Education Research | September 2023
Generating High-Precision Feedback for Programming Syntax Errors using Large Language Models
Tung Phung, José Cambronero, Sumit Gulwani, Tobias Kohn, Rupak Majumdar, Adisha Singla, Gustavo Soares
International Conference on Educational Data Mining | July 2023
Towards More Effective AI-assisted Programming: A Systematic Design Exploration to Improve Visual Studio IntelliCode’s User Experience
Priyan Vaithilingam, Elena L. Glassman, Peter Groenwegen, Sumit Gulwani, Austin Z. Henley, Rohan Malpani, David Pugh, Arjun Radhakrishna, Gustavo Soares, Joey Wang
ICSE SEIP | May 2023
-
NL2Viz: Natural Language to Visualization via Constrained Syntax-Guided Synthesis
Zhengkai Wu, Vu Le, Ashish Tiwari, Sumit Gulwani, Arjun Radhakrishna, Ivan Radicek, Gustavo Soares, Xinyu Wang, Zhenwen Li, Tao Xie
FSE | November 2022
Overwatch: Learning Patterns in Code Edit Sequences
Yuhao Zhang, Yasharth Bajpai, Priyanshu Gupta, Ameya Ketkar, Miltos Allamanis, Titus Barik, Sumit Gulwani, Arjun Radhakrishna, Mohammad Raza, Gustavo Soares, Ashish Tiwari
OOPSLA: Proceedings of the ACM on Programming Languages | October 2022
Repairing Bugs in Python Assignments using Large Language Models
Jialu Zhang, José Cambronero, Sumit Gulwani, Vu Le, Ruzica Piskac, Gustavo Soares, Gust Verbruggen
September 2022
Synchromesh: Reliable code generation from pre-trained language models
Gabriel Poesia, Alex Polozov, Vu Le, Ashish Tiwari, Gustavo Soares, Chris Meek, Sumit Gulwani
ICLR 2022 | April 2022
-
Learning Quick Fixes from Code Repositories
Reudismam Rolim, Gustavo Soares, Rohit Gheyi, Titus Barik, Loris D’Antoni
Brazilian Symposium on Software Engineering | October 2021
Multi-modal Program Inference: a Marriage of Pre-trained Language Models and Component-based Synthesis
Kia Rahmani, Mohammad Raza (moraza), Sumit Gulwani (sumitg), Vu Le (levu), Dan Morris (damorri), Arjun Radhakrishna (arradha), Gustavo Soares (gsoares), Ashish Tiwari (astiwar)
OOPSLA | October 2021
APIfix: Output-Oriented Program Synthesis for Combating Breaking Changes in Libraries
Xiang Gao, Arjun Radhakrishna, Gustavo Soares, Ridwan Shariffdeen, Sumit Gulwani, Abhik Roychoudhury
OOPSLA | September 2021
-
Feedback-Driven Semi-Supervised Synthesis of Program Transformations
Xiang Gao, Shraddha Barke, Arjun Radhakrishna, Gustavo Soares, Sumit Gulwani, Alan Leung, Nachi Nagappan, Ashish Tiwari
OOPSLA | October 2020
-
On the fly synthesis of edit suggestions
Anders Miltner, Sumit Gulwani, Vu Le, Alan Leung, Arjun Radhakrishna, Gustavo Soares, Ashish Tiwari, Abhishek Udupa
Object-Oriented Programming, Systems, Languages & Applications (OOPSLA) | October 2019
Evaluating Feedback Tools in Introductory Programming Classes
Ruan Reis, Gustavo Soares, Melina Mongiovi, Wilkerson L. Andrade
Frontiers in Education Conference (FIE) | August 2019
Revisiting the refactoring mechanics
Jonhnanthan Oliveira, Rohit Gheyi, Melina Mongiovi, Gustavo Soares, Márcio Ribeiro, Alessandro Garcia
Information and Software Technology | August 2019, Vol 110: pp. 136-138
-
Detecting overly strong preconditions in refactoring engines
Melina Mongiovi, Rohit Gheyi, Gustavo Soares, Márcio Ribeiro, Paulo Borba, Leopoldo Teixeira
IEEE Transactions on Software Engineering | August 2018, Vol 44(5): pp. 429-452
-
TraceDiff: Debugging unexpected code behavior using trace divergences
Ryo Suzuki, Gustavo Soares, Andrew Head, Elena Glassman, Ruan Reis, Melina Mongiovi, Loris D'Antoni, Björn Hartmann
VL/HCC | August 2017
Writing Reusable Code Feedback at Scale with Mixed-Initiative Program Synthesis
Andrew Head, Elena Glassman, Gustavo Soares, Ryo Suzuki, Lucas Figueredo, Loris D'Antoni, Bjoern Hartmann
Learning at Scale (L@S) | August 2017
Learning Syntactic Program Transformations from Examples
Reudismam Rolim, Gustavo Soares (gsoares), Loris D'Antoni, Alex Polozov, Sumit Gulwani, Rohit Gheyi, Bjoern Hartmann
39th International Conference on Software Engineering (ICSE) | May 2017
-
User Interaction Models for Disambiguation in Programming by Example
Mikaël Mayer, Gustavo Soares (gsoares), Maxim Grechkin, Vu Le, Mark Marron, Alex Polozov, Rishabh Singh, Ben Zorn, Sumit Gulwani
ACM Symposium on User Interface Software and Technology (UIST) 2015 | November 2015
-
Automated Behavioral Testing of Refactoring Engines
Gustavo Soares, Rohit Gheyi, Tiago Massoni
IEEE Transactions on Software Engineering | August 2013, Vol 39(2): pp. 147-162
-
Making Program Refactoring Safer
Gustavo Soares, Rohit Gheyi, Dalton Serey, Tiago Massoni
IEEE Software | August 2010, Vol 27(4): pp. 52-57
Publications by Research Area
-
STACKFEED: Structured Textual Actor-Critic Knowledge Base Editing with Feedback
Naman Gupta, Shashank Kirtania, Priyanshu Gupta, Krishna Kariya, Sumit Gulwani, Arun Iyer, Suresh Parthasarathy, Arjun Radhakrishna, Sriram Rajamani, Gustavo Soares
October 2024
Preprint
METAREFLECTION: Learning Instructions for Language Agents using Past Reflections
Priyanshu Gupta, Shashank Kirtania, Ananya Singha, Sumit Gulwani, Arjun Radhakrishna, Sherry Shi, Gustavo Soares
EMNLP'24 | October 2024
Let’s Fix this Together: Conversational Debugging with GitHub Copilot
Yasharth Bajpai, Bhavya Chopra, Param Biyani, Cagri Aslan, Sumit Gulwani, Dustin Coleman, Chris Parnin, Arjun Radhakrishna, Gustavo Soares
IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC) | August 2024
Best Research Paper Award
RUBICON: Rubric-based Evaluation of Domain Specific Human-AI Conversations
Param Biyani, Yasharth Bajpai, Arjun Radhakrishna, Gustavo Soares, Sumit Gulwani
AIware: Proceedings of the 1st ACM International Conference on AI-Powered Software | July 2024
Best Paper Presentation Award
Automating Human Tutor-Style Programming Feedback: Leveraging GPT-4 Tutor Model for Hint Generation and GPT-3.5 Student Model for Hint Validation
Tung Phung, Victor-Alexandru Pădurean, Anjali Singh, Christopher Brooks, José Cambronero, Sumit Gulwani, Adish Singla, Gustavo Soares
International Learning Analytics and Knowledge Conference | March 2024
Exploring Interaction Patterns for Debugging: Enhancing Conversational Capabilities of AI-assistants
Bhavya Chopra, Yasharth Bajpai, Param Biyani, Gustavo Soares, Arjun Radhakrishna, Chris Parnin, Sumit Gulwani
NAACL: Proceedings of the Third Workshop on Bridging Human-Computer Interaction and Natural Language Processing | February 2024
PyDex: Repairing Bugs in Introductory Python Assignments using LLMs
Jialu Zhang, José Cambronero, Sumit Gulwani, Vu Le, Ruzica Piskac, Gustavo Soares, Gust Verbruggen
Object-oriented Programming, Systems, Languages, and Applications (OOPSLA) | January 2024
Grace: Language Models Meet Code Edits
Priyanshu Gupta, Avishree Khare, Yasharth Bajpai, Saikat Chakraborty, Sumit Gulwani, Aditya Kanade, Arjun Radhakrishna, Gustavo Soares, Ashish Tiwari
ESEC/FSE 2023: Proceedings of the 31st ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering | November 2023
Generative AI for Programming Education: Benchmarking ChatGPT, GPT-4, and Human Tutors
Tung Phung, Victor-Alexandru Padurean, José Cambronero, Sumit Gulwani, Tobias Kohn, Rupak Majumdar, Adish Singla, Gustavo Soares
Conference on International Computing Education Research | September 2023
Generating High-Precision Feedback for Programming Syntax Errors using Large Language Models
Tung Phung, José Cambronero, Sumit Gulwani, Tobias Kohn, Rupak Majumdar, Adisha Singla, Gustavo Soares
International Conference on Educational Data Mining | July 2023
Towards More Effective AI-assisted Programming: A Systematic Design Exploration to Improve Visual Studio IntelliCode’s User Experience
Priyan Vaithilingam, Elena L. Glassman, Peter Groenwegen, Sumit Gulwani, Austin Z. Henley, Rohan Malpani, David Pugh, Arjun Radhakrishna, Gustavo Soares, Joey Wang
ICSE SEIP | May 2023
Overwatch: Learning Patterns in Code Edit Sequences
Yuhao Zhang, Yasharth Bajpai, Priyanshu Gupta, Ameya Ketkar, Miltos Allamanis, Titus Barik, Sumit Gulwani, Arjun Radhakrishna, Mohammad Raza, Gustavo Soares, Ashish Tiwari
OOPSLA: Proceedings of the ACM on Programming Languages | October 2022
Repairing Bugs in Python Assignments using Large Language Models
Jialu Zhang, José Cambronero, Sumit Gulwani, Vu Le, Ruzica Piskac, Gustavo Soares, Gust Verbruggen
September 2022
Synchromesh: Reliable code generation from pre-trained language models
Gabriel Poesia, Alex Polozov, Vu Le, Ashish Tiwari, Gustavo Soares, Chris Meek, Sumit Gulwani
ICLR 2022 | April 2022
Multi-modal Program Inference: a Marriage of Pre-trained Language Models and Component-based Synthesis
Kia Rahmani, Mohammad Raza (moraza), Sumit Gulwani (sumitg), Vu Le (levu), Dan Morris (damorri), Arjun Radhakrishna (arradha), Gustavo Soares (gsoares), Ashish Tiwari (astiwar)
OOPSLA | October 2021
Feedback-Driven Semi-Supervised Synthesis of Program Transformations
Xiang Gao, Shraddha Barke, Arjun Radhakrishna, Gustavo Soares, Sumit Gulwani, Alan Leung, Nachi Nagappan, Ashish Tiwari
OOPSLA | October 2020
On the fly synthesis of edit suggestions
Anders Miltner, Sumit Gulwani, Vu Le, Alan Leung, Arjun Radhakrishna, Gustavo Soares, Ashish Tiwari, Abhishek Udupa
Object-Oriented Programming, Systems, Languages & Applications (OOPSLA) | October 2019
-
Let’s Fix this Together: Conversational Debugging with GitHub Copilot
Yasharth Bajpai, Bhavya Chopra, Param Biyani, Cagri Aslan, Sumit Gulwani, Dustin Coleman, Chris Parnin, Arjun Radhakrishna, Gustavo Soares
IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC) | August 2024
Best Research Paper Award
RUBICON: Rubric-based Evaluation of Domain Specific Human-AI Conversations
Param Biyani, Yasharth Bajpai, Arjun Radhakrishna, Gustavo Soares, Sumit Gulwani
AIware: Proceedings of the 1st ACM International Conference on AI-Powered Software | July 2024
Best Paper Presentation Award
Exploring Interaction Patterns for Debugging: Enhancing Conversational Capabilities of AI-assistants
Bhavya Chopra, Yasharth Bajpai, Param Biyani, Gustavo Soares, Arjun Radhakrishna, Chris Parnin, Sumit Gulwani
NAACL: Proceedings of the Third Workshop on Bridging Human-Computer Interaction and Natural Language Processing | February 2024
Towards More Effective AI-assisted Programming: A Systematic Design Exploration to Improve Visual Studio IntelliCode’s User Experience
Priyan Vaithilingam, Elena L. Glassman, Peter Groenwegen, Sumit Gulwani, Austin Z. Henley, Rohan Malpani, David Pugh, Arjun Radhakrishna, Gustavo Soares, Joey Wang
ICSE SEIP | May 2023
Overwatch: Learning Patterns in Code Edit Sequences
Yuhao Zhang, Yasharth Bajpai, Priyanshu Gupta, Ameya Ketkar, Miltos Allamanis, Titus Barik, Sumit Gulwani, Arjun Radhakrishna, Mohammad Raza, Gustavo Soares, Ashish Tiwari
OOPSLA: Proceedings of the ACM on Programming Languages | October 2022
On the fly synthesis of edit suggestions
Anders Miltner, Sumit Gulwani, Vu Le, Alan Leung, Arjun Radhakrishna, Gustavo Soares, Ashish Tiwari, Abhishek Udupa
Object-Oriented Programming, Systems, Languages & Applications (OOPSLA) | October 2019
User Interaction Models for Disambiguation in Programming by Example
Mikaël Mayer, Gustavo Soares (gsoares), Maxim Grechkin, Vu Le, Mark Marron, Alex Polozov, Rishabh Singh, Ben Zorn, Sumit Gulwani
ACM Symposium on User Interface Software and Technology (UIST) 2015 | November 2015
-
Let’s Fix this Together: Conversational Debugging with GitHub Copilot
Yasharth Bajpai, Bhavya Chopra, Param Biyani, Cagri Aslan, Sumit Gulwani, Dustin Coleman, Chris Parnin, Arjun Radhakrishna, Gustavo Soares
IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC) | August 2024
Best Research Paper Award
RUBICON: Rubric-based Evaluation of Domain Specific Human-AI Conversations
Param Biyani, Yasharth Bajpai, Arjun Radhakrishna, Gustavo Soares, Sumit Gulwani
AIware: Proceedings of the 1st ACM International Conference on AI-Powered Software | July 2024
Best Paper Presentation Award
Automating Human Tutor-Style Programming Feedback: Leveraging GPT-4 Tutor Model for Hint Generation and GPT-3.5 Student Model for Hint Validation
Tung Phung, Victor-Alexandru Pădurean, Anjali Singh, Christopher Brooks, José Cambronero, Sumit Gulwani, Adish Singla, Gustavo Soares
International Learning Analytics and Knowledge Conference | March 2024
Exploring Interaction Patterns for Debugging: Enhancing Conversational Capabilities of AI-assistants
Bhavya Chopra, Yasharth Bajpai, Param Biyani, Gustavo Soares, Arjun Radhakrishna, Chris Parnin, Sumit Gulwani
NAACL: Proceedings of the Third Workshop on Bridging Human-Computer Interaction and Natural Language Processing | February 2024
PyDex: Repairing Bugs in Introductory Python Assignments using LLMs
Jialu Zhang, José Cambronero, Sumit Gulwani, Vu Le, Ruzica Piskac, Gustavo Soares, Gust Verbruggen
Object-oriented Programming, Systems, Languages, and Applications (OOPSLA) | January 2024
Investigating Student Mistakes in Introductory Data Science Programming
Anjali Singh, Anna Fariha, Christopher Brooks, Gustavo Soares, Austin Z. Henley, Ashish Tiwari, Chethan M, Heeryung Choi, Sumit Gulwani
Technical Symposium on Computer Science Education | January 2024
Grace: Language Models Meet Code Edits
Priyanshu Gupta, Avishree Khare, Yasharth Bajpai, Saikat Chakraborty, Sumit Gulwani, Aditya Kanade, Arjun Radhakrishna, Gustavo Soares, Ashish Tiwari
ESEC/FSE 2023: Proceedings of the 31st ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering | November 2023
NL2Viz: Natural Language to Visualization via Constrained Syntax-Guided Synthesis
Zhengkai Wu, Vu Le, Ashish Tiwari, Sumit Gulwani, Arjun Radhakrishna, Ivan Radicek, Gustavo Soares, Xinyu Wang, Zhenwen Li, Tao Xie
FSE | November 2022
Overwatch: Learning Patterns in Code Edit Sequences
Yuhao Zhang, Yasharth Bajpai, Priyanshu Gupta, Ameya Ketkar, Miltos Allamanis, Titus Barik, Sumit Gulwani, Arjun Radhakrishna, Mohammad Raza, Gustavo Soares, Ashish Tiwari
OOPSLA: Proceedings of the ACM on Programming Languages | October 2022
Repairing Bugs in Python Assignments using Large Language Models
Jialu Zhang, José Cambronero, Sumit Gulwani, Vu Le, Ruzica Piskac, Gustavo Soares, Gust Verbruggen
September 2022
Learning Quick Fixes from Code Repositories
Reudismam Rolim, Gustavo Soares, Rohit Gheyi, Titus Barik, Loris D’Antoni
Brazilian Symposium on Software Engineering | October 2021
Multi-modal Program Inference: a Marriage of Pre-trained Language Models and Component-based Synthesis
Kia Rahmani, Mohammad Raza (moraza), Sumit Gulwani (sumitg), Vu Le (levu), Dan Morris (damorri), Arjun Radhakrishna (arradha), Gustavo Soares (gsoares), Ashish Tiwari (astiwar)
OOPSLA | October 2021
APIfix: Output-Oriented Program Synthesis for Combating Breaking Changes in Libraries
Xiang Gao, Arjun Radhakrishna, Gustavo Soares, Ridwan Shariffdeen, Sumit Gulwani, Abhik Roychoudhury
OOPSLA | September 2021
Feedback-Driven Semi-Supervised Synthesis of Program Transformations
Xiang Gao, Shraddha Barke, Arjun Radhakrishna, Gustavo Soares, Sumit Gulwani, Alan Leung, Nachi Nagappan, Ashish Tiwari
OOPSLA | October 2020
On the fly synthesis of edit suggestions
Anders Miltner, Sumit Gulwani, Vu Le, Alan Leung, Arjun Radhakrishna, Gustavo Soares, Ashish Tiwari, Abhishek Udupa
Object-Oriented Programming, Systems, Languages & Applications (OOPSLA) | October 2019
Revisiting the refactoring mechanics
Jonhnanthan Oliveira, Rohit Gheyi, Melina Mongiovi, Gustavo Soares, Márcio Ribeiro, Alessandro Garcia
Information and Software Technology | August 2019, Vol 110: pp. 136-138
Evaluating Feedback Tools in Introductory Programming Classes
Ruan Reis, Gustavo Soares, Melina Mongiovi, Wilkerson L. Andrade
Frontiers in Education Conference (FIE) | August 2019
Detecting overly strong preconditions in refactoring engines
Melina Mongiovi, Rohit Gheyi, Gustavo Soares, Márcio Ribeiro, Paulo Borba, Leopoldo Teixeira
IEEE Transactions on Software Engineering | August 2018, Vol 44(5): pp. 429-452
TraceDiff: Debugging unexpected code behavior using trace divergences
Ryo Suzuki, Gustavo Soares, Andrew Head, Elena Glassman, Ruan Reis, Melina Mongiovi, Loris D'Antoni, Björn Hartmann
VL/HCC | August 2017
Writing Reusable Code Feedback at Scale with Mixed-Initiative Program Synthesis
Andrew Head, Elena Glassman, Gustavo Soares, Ryo Suzuki, Lucas Figueredo, Loris D'Antoni, Bjoern Hartmann
Learning at Scale (L@S) | August 2017
Learning Syntactic Program Transformations from Examples
Reudismam Rolim, Gustavo Soares (gsoares), Loris D'Antoni, Alex Polozov, Sumit Gulwani, Rohit Gheyi, Bjoern Hartmann
39th International Conference on Software Engineering (ICSE) | May 2017
User Interaction Models for Disambiguation in Programming by Example
Mikaël Mayer, Gustavo Soares (gsoares), Maxim Grechkin, Vu Le, Mark Marron, Alex Polozov, Rishabh Singh, Ben Zorn, Sumit Gulwani
ACM Symposium on User Interface Software and Technology (UIST) 2015 | November 2015
Automated Behavioral Testing of Refactoring Engines
Gustavo Soares, Rohit Gheyi, Tiago Massoni
IEEE Transactions on Software Engineering | August 2013, Vol 39(2): pp. 147-162
Making Program Refactoring Safer
Gustavo Soares, Rohit Gheyi, Dalton Serey, Tiago Massoni
IEEE Software | August 2010, Vol 27(4): pp. 52-57
Publications by Type
-
STACKFEED: Structured Textual Actor-Critic Knowledge Base Editing with Feedback
Naman Gupta, Shashank Kirtania, Priyanshu Gupta, Krishna Kariya, Sumit Gulwani, Arun Iyer, Suresh Parthasarathy, Arjun Radhakrishna, Sriram Rajamani, Gustavo Soares
October 2024
Preprint
Repairing Bugs in Python Assignments using Large Language Models
Jialu Zhang, José Cambronero, Sumit Gulwani, Vu Le, Ruzica Piskac, Gustavo Soares, Gust Verbruggen
September 2022
-
METAREFLECTION: Learning Instructions for Language Agents using Past Reflections
Priyanshu Gupta, Shashank Kirtania, Ananya Singha, Sumit Gulwani, Arjun Radhakrishna, Sherry Shi, Gustavo Soares
EMNLP'24 | October 2024
Let’s Fix this Together: Conversational Debugging with GitHub Copilot
Yasharth Bajpai, Bhavya Chopra, Param Biyani, Cagri Aslan, Sumit Gulwani, Dustin Coleman, Chris Parnin, Arjun Radhakrishna, Gustavo Soares
IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC) | August 2024
Best Research Paper Award
RUBICON: Rubric-based Evaluation of Domain Specific Human-AI Conversations
Param Biyani, Yasharth Bajpai, Arjun Radhakrishna, Gustavo Soares, Sumit Gulwani
AIware: Proceedings of the 1st ACM International Conference on AI-Powered Software | July 2024
Best Paper Presentation Award
Automating Human Tutor-Style Programming Feedback: Leveraging GPT-4 Tutor Model for Hint Generation and GPT-3.5 Student Model for Hint Validation
Tung Phung, Victor-Alexandru Pădurean, Anjali Singh, Christopher Brooks, José Cambronero, Sumit Gulwani, Adish Singla, Gustavo Soares
International Learning Analytics and Knowledge Conference | March 2024
Exploring Interaction Patterns for Debugging: Enhancing Conversational Capabilities of AI-assistants
Bhavya Chopra, Yasharth Bajpai, Param Biyani, Gustavo Soares, Arjun Radhakrishna, Chris Parnin, Sumit Gulwani
NAACL: Proceedings of the Third Workshop on Bridging Human-Computer Interaction and Natural Language Processing | February 2024
PyDex: Repairing Bugs in Introductory Python Assignments using LLMs
Jialu Zhang, José Cambronero, Sumit Gulwani, Vu Le, Ruzica Piskac, Gustavo Soares, Gust Verbruggen
Object-oriented Programming, Systems, Languages, and Applications (OOPSLA) | January 2024
Investigating Student Mistakes in Introductory Data Science Programming
Anjali Singh, Anna Fariha, Christopher Brooks, Gustavo Soares, Austin Z. Henley, Ashish Tiwari, Chethan M, Heeryung Choi, Sumit Gulwani
Technical Symposium on Computer Science Education | January 2024
Grace: Language Models Meet Code Edits
Priyanshu Gupta, Avishree Khare, Yasharth Bajpai, Saikat Chakraborty, Sumit Gulwani, Aditya Kanade, Arjun Radhakrishna, Gustavo Soares, Ashish Tiwari
ESEC/FSE 2023: Proceedings of the 31st ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering | November 2023
Generative AI for Programming Education: Benchmarking ChatGPT, GPT-4, and Human Tutors
Tung Phung, Victor-Alexandru Padurean, José Cambronero, Sumit Gulwani, Tobias Kohn, Rupak Majumdar, Adish Singla, Gustavo Soares
Conference on International Computing Education Research | September 2023
Generating High-Precision Feedback for Programming Syntax Errors using Large Language Models
Tung Phung, José Cambronero, Sumit Gulwani, Tobias Kohn, Rupak Majumdar, Adisha Singla, Gustavo Soares
International Conference on Educational Data Mining | July 2023
Towards More Effective AI-assisted Programming: A Systematic Design Exploration to Improve Visual Studio IntelliCode’s User Experience
Priyan Vaithilingam, Elena L. Glassman, Peter Groenwegen, Sumit Gulwani, Austin Z. Henley, Rohan Malpani, David Pugh, Arjun Radhakrishna, Gustavo Soares, Joey Wang
ICSE SEIP | May 2023
NL2Viz: Natural Language to Visualization via Constrained Syntax-Guided Synthesis
Zhengkai Wu, Vu Le, Ashish Tiwari, Sumit Gulwani, Arjun Radhakrishna, Ivan Radicek, Gustavo Soares, Xinyu Wang, Zhenwen Li, Tao Xie
FSE | November 2022
Overwatch: Learning Patterns in Code Edit Sequences
Yuhao Zhang, Yasharth Bajpai, Priyanshu Gupta, Ameya Ketkar, Miltos Allamanis, Titus Barik, Sumit Gulwani, Arjun Radhakrishna, Mohammad Raza, Gustavo Soares, Ashish Tiwari
OOPSLA: Proceedings of the ACM on Programming Languages | October 2022
Synchromesh: Reliable code generation from pre-trained language models
Gabriel Poesia, Alex Polozov, Vu Le, Ashish Tiwari, Gustavo Soares, Chris Meek, Sumit Gulwani
ICLR 2022 | April 2022
Learning Quick Fixes from Code Repositories
Reudismam Rolim, Gustavo Soares, Rohit Gheyi, Titus Barik, Loris D’Antoni
Brazilian Symposium on Software Engineering | October 2021
Multi-modal Program Inference: a Marriage of Pre-trained Language Models and Component-based Synthesis
Kia Rahmani, Mohammad Raza (moraza), Sumit Gulwani (sumitg), Vu Le (levu), Dan Morris (damorri), Arjun Radhakrishna (arradha), Gustavo Soares (gsoares), Ashish Tiwari (astiwar)
OOPSLA | October 2021
APIfix: Output-Oriented Program Synthesis for Combating Breaking Changes in Libraries
Xiang Gao, Arjun Radhakrishna, Gustavo Soares, Ridwan Shariffdeen, Sumit Gulwani, Abhik Roychoudhury
OOPSLA | September 2021
Feedback-Driven Semi-Supervised Synthesis of Program Transformations
Xiang Gao, Shraddha Barke, Arjun Radhakrishna, Gustavo Soares, Sumit Gulwani, Alan Leung, Nachi Nagappan, Ashish Tiwari
OOPSLA | October 2020
On the fly synthesis of edit suggestions
Anders Miltner, Sumit Gulwani, Vu Le, Alan Leung, Arjun Radhakrishna, Gustavo Soares, Ashish Tiwari, Abhishek Udupa
Object-Oriented Programming, Systems, Languages & Applications (OOPSLA) | October 2019
Evaluating Feedback Tools in Introductory Programming Classes
Ruan Reis, Gustavo Soares, Melina Mongiovi, Wilkerson L. Andrade
Frontiers in Education Conference (FIE) | August 2019
TraceDiff: Debugging unexpected code behavior using trace divergences
Ryo Suzuki, Gustavo Soares, Andrew Head, Elena Glassman, Ruan Reis, Melina Mongiovi, Loris D'Antoni, Björn Hartmann
VL/HCC | August 2017
Writing Reusable Code Feedback at Scale with Mixed-Initiative Program Synthesis
Andrew Head, Elena Glassman, Gustavo Soares, Ryo Suzuki, Lucas Figueredo, Loris D'Antoni, Bjoern Hartmann
Learning at Scale (L@S) | August 2017
Learning Syntactic Program Transformations from Examples
Reudismam Rolim, Gustavo Soares (gsoares), Loris D'Antoni, Alex Polozov, Sumit Gulwani, Rohit Gheyi, Bjoern Hartmann
39th International Conference on Software Engineering (ICSE) | May 2017
User Interaction Models for Disambiguation in Programming by Example
Mikaël Mayer, Gustavo Soares (gsoares), Maxim Grechkin, Vu Le, Mark Marron, Alex Polozov, Rishabh Singh, Ben Zorn, Sumit Gulwani
ACM Symposium on User Interface Software and Technology (UIST) 2015 | November 2015
-
Revisiting the refactoring mechanics
Jonhnanthan Oliveira, Rohit Gheyi, Melina Mongiovi, Gustavo Soares, Márcio Ribeiro, Alessandro Garcia
Information and Software Technology | August 2019, Vol 110: pp. 136-138
Detecting overly strong preconditions in refactoring engines
Melina Mongiovi, Rohit Gheyi, Gustavo Soares, Márcio Ribeiro, Paulo Borba, Leopoldo Teixeira
IEEE Transactions on Software Engineering | August 2018, Vol 44(5): pp. 429-452
Automated Behavioral Testing of Refactoring Engines
Gustavo Soares, Rohit Gheyi, Tiago Massoni
IEEE Transactions on Software Engineering | August 2013, Vol 39(2): pp. 147-162
Making Program Refactoring Safer
Gustavo Soares, Rohit Gheyi, Dalton Serey, Tiago Massoni
IEEE Software | August 2010, Vol 27(4): pp. 52-57