FigureQA Dataset

If you would like to evaluate your results on the FigureQA test sets, please compile your predicted answers in the format documented below and email them to the authors.

Please send your results to Samira and Adam A. at figureqa@microsoft.com (opens in new tab).

We provide the same statistics on each of the test1 and test2 sets as those in the FigureQA paper, namely:

• Overall accuracy.
• Accuracy by question type.
• Accuracy by figure type.

Predictions Format

Results must be provided in a CSV format, having the first line as a header and each row like so:

question_index,image_index,question_id,question_string,answer

Here question_index refers to the sample’s index in the qa_pairs array within qa_pairs.json. All other fields are the same as in qa_pairs.json, documented here.

Here are example results files for test1 (opens in new tab) and test2 (opens in new tab). Simply replace the final column field < 0/1 > with your prediction as a  0  or  1  in each row.

To evaluate your results faster, please ensure that your results file is UTF-8 encoded and can be read using the read_csv (opens in new tab) function from the Python Pandas library.