Improving Productivity by Gradient-based Prompt Optimization for LLMs<\/strong><\/p>\n\n\n\nThis proposal aims to develop a novel gradient-based prompt optimization technique to unlock the prowess of LLMs on complex and challenging tasks to elevate productivity. The novelty of our method lies in our design of a white-box prompt optimization algorithm over open-source language models by leveraging both the gradient of task accuracy and language model decoding probability. This leads us to automatically and efficiently create effective, interpretable, and transferable prompts on a variety of small and large language models on complex and diverse tasks. Our research advances prompt optimization by integrating gradients and LLM probabilities, creating long-term impacts on white-box optimization research to close the gap between small and large LMs.<\/p>\n\n\n\n
\n\n\n\n2023 Recipients<\/h2>\n\n\n\nEytan Adar<\/h4>\n\n\n\n
University of Michigan, Ann Arbor<\/p>\n\n\n\n
Codes of Generative Conduct: Collaboratively Authoring and Testing Policies to Govern LLMs<\/strong><\/p>\n\n\n\nWhen we write, we are guided not only by our intended audience but also by the constraints set by our communities\u2014the standards of our workplace, legal frameworks, and industry norms. Even sub-communities naturally develop their variants of these rules. For large language models to better support generative writing, they should adhere to the same set of community-set standards and rules. In our proposed work, we tackle how groups can collaboratively construct, test, and use writing standards to guide generative applications.<\/p>\n\n\n\n
Co-author(s): Eric Gilbert, University of Michigan, Ann Arbor<\/p>\n\n\n\n
\n\n\n\nVarol Akman<\/h4>\n\n\n\n
\u0130HSAN DO\u011eRAMACI B\u0130LKENT \u00dcN\u0130VERS\u0130TES\u0130<\/p>\n\n\n\n
ChatGPT as a Stage Manager in Large-Scale Institutional Practices<\/strong><\/p>\n\n\n\nLarge-scale institutional structures are characterized by the collective action of numerous employees at different levels working towards common goals. Decision-making in these structures is complex due to the absence of egalitarianism and the presence of an authority hierarchy. Group agency emerges within these structures, where multiple agents work together based on joint commitments to advocate for shared goals. The project at hand aims to investigate decision-making and its consequences in such institutional structures using ChatGPT.<\/p>\n\n\n\n
\n\n\n\nDanqi Chen<\/h4>\n\n\n\n
Princeton University<\/p>\n\n\n\n
Grounding Large Language Models with Internal Retrieval Corpora<\/strong><\/p>\n\n\n\nThis proposal tackles the research problem of augmenting large language models (LLMs) with internal retrieval corpora, so we can enable LLMs to generate text that conforms to up-to-date and domain-specific information without additional re-training. The proposal aims to address two technical challenges: a) how to incorporate a large number of (and potentially noisy) retrieved tokens in a limited context window; b) How to train high-quality retrievers for specific domains for better integration with black-box LLMs.<\/p>\n\n\n\n
\n\n\n\nNoshir Contractor<\/h4>\n\n\n\n
Northwestern University<\/p>\n\n\n\n
Deciphering the Structural Signatures of High-Performing Human-AI Teams<\/strong><\/p>\n\n\n\nThis research project explores the dynamics of human-AI collaboration in problem-solving and creative thinking tasks. We will extend prior Human-Autonomy Teaming research that studies human-AI teams with the Wizard-of-Oz methodology\u2019s help by replacing a human confederate with an LLM-powered AI teammate. The experiment involves manipulating task types and AI teammate functions to examine how people orient themselves toward intelligent machine teammates and how technology can be designed to be a collaborator, not a chatbot. Participants will communicate in a group chat environment while conducting tasks, and their experiences, performance, and social interactions will be surveyed, analyzed, and compared to identify the mechanisms of high-performing Human-AI teams.<\/p>\n\n\n\n
\n\n\n\nPhilip Guo<\/h4>\n\n\n\n
UC San Diego<\/p>\n\n\n\n
Beyond just programming: using an LLM\u2019s knowledge about the human world to improve human-AI collaboration for data science<\/strong><\/p>\n\n\n\nWe plan to build an LLM-based data science assistant that reasons about both the content of the code that it generates and the real-world domain that the requested analysis is about. Combining knowledge about both code and the human world can hopefully enable our LLM to perform more rigorous data analyses than current coding assistants like GitHub Copilot. The goal of this project is to advance the new future of work by empowering everyone across the workplace \u2014 from field technicians to customer support representatives to executives \u2014 to directly analyze data that matters most for their jobs. More broadly, we want to use AI to enable everyone to analyze the data that is most meaningful and relevant to them, even if they are not data science or programming experts.<\/p>\n\n\n\n
\n\n\n\nJohn Horton<\/h4>\n\n\n\n
Massachusetts Institute of Technology<\/p>\n\n\n\n
Understanding the Effects of Github Copilot on Worker Productivity and Skill Evolution in the Online Labor Market<\/strong><\/p>\n\n\n\nOur project aims to understand the impact of Github Copilot on software engineers and other technical professionals\u2019 productivity, skill adoption, and labor market outcomes in a real labor market. LLMs have been shown to affect worker productivity on researcher-assigned tasks, but less is known about how workers use such tools in their real-life work, or what happens in equilibrium as more workers adopt them. We plan to use data from a large online labor market merged with survey data where workers report when they began using Copilot to understand how this tool impacted workers productivity, wages, and skill upgrading.<\/p>\n\n\n\n
Co-author(s): Apostolos Filipas, Fordham University; Emma van Inwegen, MIT<\/p>\n\n\n\n
\n\n\n\nAnastasios (Tasos) Kyrillidis<\/h4>\n\n\n\n
Rice University<\/p>\n\n\n\n
Efficient new-task adaptation in the era of Transformers and Federated Learning<\/strong><\/p>\n\n\n\nStarting from traditional Transformer models, one of the goals of this project is to introduce efficient ways to i) adapt to new tasks without re-training the models from scratch; ii) combine existing trained models in a meaningful way, extending MoE modules beyond basic feed-forward layer splitting; and iii) consider novel federated learning scenarios based on Transformer-based models, where computational-\/communication- bottlenecks require novel transformer decompositions, beyond sparse MoEs. Prior advances from this collaboration (results on asynchronous federated learning scenarios, new uses of a mixture of experts for zero-shot personalization in federated learning, and novel transformer-model decomposition for efficient distributed computing, respectively) will be extended based on the above directions. <\/p>\n\n\n\n
\n\n\n\nMatthew Lease<\/h4>\n\n\n\n
University of Texas at Austin<\/p>\n\n\n\n
Providing Natural Language Decision-Support via Large Language Models<\/strong><\/p>\n\n\n\nLarge language models (LLMs) are transforming the way people seek, explore, and assess information for decision making. By attending to how people naturally interact with each other, LLM-based decision-support tooling can strengthen both the quality of support offered and the accompanying user experience. Because LLMs are fallible and can hallucinate incorrect information, LLM interactions should also be designed to handle such fallibility in stride, akin to how we interact with one another around normal human fallibility. This project will explore such opportunities and challenges in designing ways to provide LLM-based decision support that is innovative, intuitive, and effective.<\/p>\n\n\n\n
\n\n\n\nMor Naaman<\/h4>\n\n\n\n
Cornell Tech<\/p>\n\n\n\n
Impact of Language Models on Writing and its Outcomes in the Workplace<\/strong><\/p>\n\n\n\nUse of autocomplete features and other language and text suggestions powered by large language models can shift people\u2019s writing, influence their thinking, and may lead to different outcomes for the person exposed to them. Using large-scale online experiments, this project aims to understand the potential of such AI-based products used in workplace settings to result in these different outcomes. In particular, we aim to understand the disparate effect these products may have on people from different backgrounds who may have different language styles. We will further investigate strategies to mitigate any negative outcomes.<\/p>\n\n\n\n
\n\n\n\nHamed Zamani<\/h4>\n\n\n\n
University of Massachusetts Amherst<\/p>\n\n\n\n
Improving Productivity by Personalizing Large Language Models<\/strong><\/p>\n\n\n\nLarge language models (LLMs) have recently revolutionized natural language processing, including a number of user-facing applications that largely impact users\u2019 productivity, such as QA, writing assistants, and task management. In user-facing applications, it is widely accepted that users have different needs and behave differently. This suggests the importance of personalization in these applications. The goal of this project is to study different approaches for personalizing LLMs and their potential impact on user\u2019s satisfaction and productivity.<\/p>\n\n\n\n
\n\n\n\nAmy Zhang<\/h4>\n\n\n\n
University of Washington<\/p>\n\n\n\n
Interactive Personalized Information Artifacts to Summarize Team Communication<\/strong><\/p>\n\n\n\nWe examine how to design team-AI systems embedded into teams\u2019 existing communication spaces that can use LLMs to leverage rich prior communication data to support team goals and activities. Information artifacts that convey information from prior communication logs should not only summarize content but also support additional team goals such as sharing out different information to different audiences and serving as a springboard for follow-on discussion. We aim to use LLMs in a novel system for generating interactive summary artifacts that live within group chat and that summarize recorded video meetings. These artifacts will use LLMs to 1) personalize the content of the summary to different team members based on their conversation history, such as by highlighting specific points for follow-on discussion, 2) enable users to interactively expand the summary and dive into the context of the original conversation, and 3) allow users to customize an artifact in order to share it with different audiences.<\/p>\n\n\n","protected":false},"featured_media":1030995,"template":"","meta":{"msr-url-field":"","msr-podcast-episode":"","msrModifiedDate":"","msrModifiedDateEnabled":false,"ep_exclude_from_search":false,"_classifai_error":"","msr_hide_image_in_river":0,"footnotes":""},"msr-opportunity-type":[155533],"msr-region":[256048],"msr-locale":[268875],"msr-program-audience":[],"msr-post-option":[],"msr-impact-theme":[],"class_list":["post-1169836","msr-academic-program","type-msr-academic-program","status-publish","has-post-thumbnail","hentry","msr-opportunity-type-grants-and-fellowships","msr-region-global","msr-locale-en_us"],"msr_description":"","msr_social_media":[],"related-researchers":[],"tab-content":[],"msr_impact_theme":[],"_links":{"self":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-academic-program\/1169836","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-academic-program"}],"about":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/types\/msr-academic-program"}],"version-history":[{"count":12,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-academic-program\/1169836\/revisions"}],"predecessor-version":[{"id":1170079,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-academic-program\/1169836\/revisions\/1170079"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/media\/1030995"}],"wp:attachment":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/media?parent=1169836"}],"wp:term":[{"taxonomy":"msr-opportunity-type","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-opportunity-type?post=1169836"},{"taxonomy":"msr-region","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-region?post=1169836"},{"taxonomy":"msr-locale","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-locale?post=1169836"},{"taxonomy":"msr-program-audience","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-program-audience?post=1169836"},{"taxonomy":"msr-post-option","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-post-option?post=1169836"},{"taxonomy":"msr-impact-theme","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-impact-theme?post=1169836"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}