{"id":171065,"date":"2012-11-23T11:45:31","date_gmt":"2012-11-23T11:45:31","guid":{"rendered":"https:\/\/www.microsoft.com\/en-us\/research\/project\/recurrent-neural-networks-for-language-processing\/"},"modified":"2019-08-19T14:55:37","modified_gmt":"2019-08-19T21:55:37","slug":"recurrent-neural-networks-for-language-processing","status":"publish","type":"msr-project","link":"https:\/\/www.microsoft.com\/en-us\/research\/project\/recurrent-neural-networks-for-language-processing\/","title":{"rendered":"Recurrent Neural Networks for Language Processing"},"content":{"rendered":"<p class=\"asset-content\">This project focuses on advancing the state-of-the-art in language processing with recurrent neural networks. We are currently applying these to language modeling, machine translation, speech recognition, language understanding and meaning representation. A special interest in is adding side-channels of information as input, to model phenomena which are not easily handled in other frameworks.<\/p>\n<p>A toolkit for doing RNN language modeling with side-information is in the associated <a href=\"https:\/\/www.microsoft.com\/en-us\/download\/details.aspx?id=52319\">download<\/a>. Sample word vectors for use with this toolkit can be found in the <strong>sample_vectors<\/strong> directory (be sure to unzip), along with training and test scripts. These are for Penn Treebank words, and\u00a0achieve a perplexity of 128; removing the context dependence results in a perplexity of 144.<\/p>\n<p>As described in the NAACL-2013 paper &#8220;Linguistic Regularities in Continuous Space Word Representations,&#8221; we have found that the word representations capture many linguistic regularities. A data set for quantifying the degree to which syntactic regularities are modeled can be found in the <strong>test_set<\/strong> directory of the <a href=\"https:\/\/www.microsoft.com\/en-us\/download\/details.aspx?id=52319\">download<\/a>.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>This project focuses on advancing the state-of-the-art in language processing with recurrent neural networks. We are currently applying these to language modeling, machine translation, speech recognition, language understanding and meaning representation. A special interest in is adding side-channels of information as input, to model phenomena which are not easily handled in other frameworks. A toolkit [&hellip;]<\/p>\n","protected":false},"featured_media":0,"template":"","meta":{"msr-url-field":"","msr-podcast-episode":"","msrModifiedDate":"","msrModifiedDateEnabled":false,"ep_exclude_from_search":false,"footnotes":""},"research-area":[13561,13556,13545,13554],"msr-locale":[268875],"msr-impact-theme":[],"msr-pillar":[],"class_list":["post-171065","msr-project","type-msr-project","status-publish","hentry","msr-research-area-algorithms","msr-research-area-artificial-intelligence","msr-research-area-human-language-technologies","msr-research-area-human-computer-interaction","msr-locale-en_us","msr-archive-status-active"],"msr_project_start":"2012-11-23","related-publications":[163439,165510,163799,166355,163800,166356,163801,168091,163802,168509,163803,215131,163804,215419,163909,238042,162643,164422,244496,162762,164529,363011,162888,165428,363038],"related-downloads":[234727],"related-videos":[],"related-groups":[],"related-events":[],"related-opportunities":[],"related-posts":[],"related-articles":[],"tab-content":[],"slides":[],"related-researchers":[{"type":"user_nicename","display_name":"Chris Quirk","user_id":31430,"people_section":"Group 1","alias":"chrisq"},{"type":"user_nicename","display_name":"Michel Galley","user_id":32887,"people_section":"Group 1","alias":"mgalley"}],"msr_research_lab":[],"msr_impact_theme":[],"_links":{"self":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-project\/171065"}],"collection":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-project"}],"about":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/types\/msr-project"}],"version-history":[{"count":2,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-project\/171065\/revisions"}],"predecessor-version":[{"id":390509,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-project\/171065\/revisions\/390509"}],"wp:attachment":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/media?parent=171065"}],"wp:term":[{"taxonomy":"msr-research-area","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/research-area?post=171065"},{"taxonomy":"msr-locale","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-locale?post=171065"},{"taxonomy":"msr-impact-theme","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-impact-theme?post=171065"},{"taxonomy":"msr-pillar","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-pillar?post=171065"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}