{"id":731851,"date":"2021-03-09T01:40:04","date_gmt":"2021-03-09T09:40:04","guid":{"rendered":"https:\/\/www.microsoft.com\/en-us\/research\/?post_type=msr-project&#038;p=731851"},"modified":"2021-10-29T07:51:36","modified_gmt":"2021-10-29T14:51:36","slug":"project-litmus","status":"publish","type":"msr-project","link":"https:\/\/www.microsoft.com\/en-us\/research\/project\/project-litmus\/","title":{"rendered":"Project LITMUS"},"content":{"rendered":"<section class=\"mb-3 moray-highlight\">\n\t<div class=\"card-img-overlay mx-lg-0\">\n\t\t<div class=\"card-background  has-background-racing-green card-background--full-bleed\">\n\t\t\t\t\t<\/div>\n\t\t<!-- Foreground -->\n\t\t<div class=\"card-foreground d-flex mt-md-n5 my-lg-5 px-g px-lg-0\">\n\t\t\t<!-- Container -->\n\t\t\t<div class=\"container d-flex mt-md-n5 my-lg-5 align-self-center\">\n\t\t\t\t<!-- Card wrapper -->\n\t\t\t\t<div class=\"w-100 w-lg-col-5\">\n\t\t\t\t\t<!-- Card -->\n\t\t\t\t\t<div class=\"card material-md-card py-5 px-md-5\">\n\t\t\t\t\t\t<div class=\"card-body \">\n\t\t\t\t\t\t\t\n\t\t\t\t\t\t\t\n\n<h1 id=\"project-litmus\" class=\"h2\">Project LITMUS<\/h1>\n\n\t\t\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t<\/div>\n\t\t<\/div>\n\t<\/div>\n<\/section>\n\n\n\n\n\n<p>LITMUS: <strong>L<\/strong>inguistically <strong>I<\/strong>nspired <strong>T<\/strong>raining and testing of <strong>MU<\/strong>ltilingual <strong>S<\/strong>ystems<\/p>\n\n\n\n<p>Transformer-based Language Models have revolutionized the field of NLP and have shown great improvements in various benchmarks and are being used to power various NLP applications today. Multilingual versions of these models can also potentially serve many low-resource languages for which labeled data is not available by making use of the zer0-shot paradigm. However, there remain challenges associated with deploying such models, including the question of evaluating the models across a wide variety of languages which may not be present in standard evaluation benchmarks.<\/p>\n\n\n\n<p>The goal of Project LITMUS is to discover strategies to evaluate massive multilingual models and also suggest data collection and training strategies to improve the performance of these models.<\/p>\n\n\n\n<figure class=\"wp-block-embed is-type-video is-provider-youtube wp-block-embed-youtube wp-embed-aspect-16-9 wp-has-aspect-ratio\"><div class=\"wp-block-embed__wrapper\">\n<iframe loading=\"lazy\" title=\"Litmus Predictor\" width=\"500\" height=\"281\" src=\"https:\/\/www.youtube-nocookie.com\/embed\/20dR8oKc9j0?feature=oembed&rel=0\" frameborder=\"0\" allow=\"accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share\" referrerpolicy=\"strict-origin-when-cross-origin\" allowfullscreen><\/iframe>\n<\/div><\/figure>\n\n\n\n<div class=\"wp-block-buttons is-content-justification-center is-layout-flex wp-block-buttons-is-layout-flex\">\n<div class=\"wp-block-button is-style-fill-github\"><a data-bi-type=\"button\" class=\"wp-block-button__link\" href=\"https:\/\/github.com\/microsoft\/Litmus\" target=\"_blank\" rel=\"noreferrer noopener\">GitHub<\/a><\/div>\n<\/div>\n\n\n\n<p>You may also be interested in the other projects our group works on: Project <a href=\"https:\/\/www.microsoft.com\/en-us\/research\/project\/ellora\/\">ELLORA<\/a>&nbsp;(Enabling Low Resource Languages) and <a href=\"https:\/\/www.microsoft.com\/en-us\/research\/project\/melange\/\">Project M\u00e9lange<\/a> (Understanding Mixed Language).<\/p>\n\n\n","protected":false},"excerpt":{"rendered":"<p>The goal of Project LITMUS is to discover strategies to evaluate massive multilingual models and also suggest data collection and training strategies to improve the performance of these models.<\/p>\n","protected":false},"featured_media":0,"template":"","meta":{"msr-url-field":"","msr-podcast-episode":"","msrModifiedDate":"","msrModifiedDateEnabled":false,"ep_exclude_from_search":false,"_classifai_error":"","footnotes":""},"research-area":[13545],"msr-locale":[268875],"msr-impact-theme":[],"msr-pillar":[],"class_list":["post-731851","msr-project","type-msr-project","status-publish","hentry","msr-research-area-human-language-technologies","msr-locale-en_us","msr-archive-status-active"],"msr_project_start":"2020-07-01","related-publications":[732127,812008,816697],"related-downloads":[786961],"related-videos":[],"related-groups":[],"related-events":[],"related-opportunities":[],"related-posts":[],"related-articles":[],"tab-content":[],"slides":[],"related-researchers":[{"type":"user_nicename","display_name":"Kalika Bali","user_id":32477,"people_section":"Section name 0","alias":"kalikab"},{"type":"user_nicename","display_name":"Tanuja Ganu","user_id":38883,"people_section":"Section name 0","alias":"taganu"},{"type":"user_nicename","display_name":"Sunayana Sitaram","user_id":37287,"people_section":"Section name 0","alias":"susitara"}],"msr_research_lab":[199562],"msr_impact_theme":[],"_links":{"self":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-project\/731851"}],"collection":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-project"}],"about":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/types\/msr-project"}],"version-history":[{"count":11,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-project\/731851\/revisions"}],"predecessor-version":[{"id":790556,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-project\/731851\/revisions\/790556"}],"wp:attachment":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/media?parent=731851"}],"wp:term":[{"taxonomy":"msr-research-area","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/research-area?post=731851"},{"taxonomy":"msr-locale","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-locale?post=731851"},{"taxonomy":"msr-impact-theme","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-impact-theme?post=731851"},{"taxonomy":"msr-pillar","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-pillar?post=731851"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}