Challenges<\/h3>\n
A significant proportion of questions in NewsQA cannot be solved without reasoning. The reasoning types we have identified in our analysis are as follows:<\/p>\n
- \n
- Synthesis: Some answers can only be inferred by synthesizing information distributed across multiple sentences.<\/li>\n
- Paraphrasing: A single sentence in the article might entail or paraphrase the question. Paraphrase recognition may require synonymy and word knowledge.<\/li>\n
- Inference: Some answers must be inferred from incomplete information in the article or by recognizing conceptual overlap. This typically draws on general knowledge.<\/li>\n
- Additionally, some questions have no answer or no unique answer in the corresponding story, so a system must learn to recognize when given information is not sufficient.<\/li>\n<\/ul>\n
See other datasets from Microsoft Montreal:
\nFrames<\/a> | FigureQA<\/a><\/b><\/p>\n","protected":false},"excerpt":{"rendered":"With massive volumes of written text being produced every second, how do we make sure that we have the most recent and relevant information available to us? Microsoft research Montreal is tackling this problem by building AI systems that can read and comprehend large volumes of complex text in real-time. The purpose of the NewsQA […]<\/p>\n","protected":false},"featured_media":489527,"template":"","meta":{"msr-url-field":"","msr-podcast-episode":"","msrModifiedDate":"","msrModifiedDateEnabled":false,"ep_exclude_from_search":false,"_classifai_error":"","footnotes":""},"research-area":[13556],"msr-locale":[268875],"msr-impact-theme":[],"msr-pillar":[],"class_list":["post-486827","msr-project","type-msr-project","status-publish","has-post-thumbnail","hentry","msr-research-area-artificial-intelligence","msr-locale-en_us","msr-archive-status-active"],"msr_project_start":"","related-publications":[],"related-downloads":[471963],"related-videos":[],"related-groups":[],"related-events":[],"related-opportunities":[],"related-posts":[],"related-articles":[],"tab-content":[{"id":0,"name":"Stats","content":"
Summary<\/h3>\r\n[row]\r\n[column class=\"l-col-4-24\"]\r\n
\r\n12,744<\/div>\r\n \r\nStories<\/div>\r\n<\/div>\r\n[\/column]\r\n\r\n[column class=\"l-col-4-24\"]\r\n\r\n119,633<\/div>\r\n \r\nQuestion-Answer Pairs<\/div>\r\n<\/div>\r\n[\/column]\r\n\r\n[column class=\"l-col-4-24\"]\r\n\r\n616<\/div>\r\n \r\nAverage Words per Article<\/div>\r\n<\/div>\r\n[\/column]\r\n\r\n[column class=\"l-col-4-24\"]\r\n\r\n4.13<\/div>\r\n \r\nAverage Words per Answer<\/div>\r\n<\/div>\r\n[\/column]\r\n\r\n[column class=\"l-col-4-24\"]\r\n\r\n74.9%<\/div>\r\n \r\nHuman Performance (F1)<\/div>\r\n<\/div>\r\n[\/column]\r\n\r\n[\/row]\r\nReasoning Statistics<\/h3>\r\nReasoning mechanisms needed to answer questions in NewsQA based on 500 examples. For each type, we show an example question with the text snippet that contains the answer span, with words relevant to the reasoning type in bold.\r\n
![\"Question](\"https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2018\/05\/plot_question_types.svg\")
\r\n