Answering questions about a given image is a difficult task, requiring both an understanding of the image and the accompanying query. Microsoft research Montreal’s FigureQA dataset introduces a new visual reasoning task for research, specific to graphical plots and figures. The task comes with an additional twist: all of the questions are relational, requiring the comparison of several or all elements of the underlying plot.
Images are comprised on five types of figures commonly found in analytical documents. Fifteen question types were selected for the dataset concerning quantitative attributes in relational global and one-vs-one contexts. These include properties like minimum and maximum, greater and less than, medians, curve roughness, and area under the curve (AUC). All questions in the training and validation sets have either a yes or no answer.
For more details concerning the task, dataset, and our experiments, please read our paper: FigureQA: An Annotated Figure Dataset for Visual Reasoning (opens in new tab).
Click on a figure below to enlarge it and see some of its questions, answers, and bounding boxes.
People
Samira Ebrahimi Kahou
Postdoctoral Researcher
McGill University, Mila
Vincent Michalski
Research Intern
MILA
Adam Atkinson
Software Developer
Akos Kadar
Research Intern
Yoshua Bengio
Founder and Scientific Director
Mila – Quebec AI Institute
Mahmoud Adada
Principal Engineering Manager
Rahul Mehrotra
Senior Program Manager