An extensible point-based method for data chart value detection
Carlos Soto Shinjae Yoo
We present an extensible method for identifying semantic points to reverse
engineer (i.e. extract the values of) data charts, particularly those in
scientific articles. Our method uses a point proposal network (akin to region
proposal networks for object detection) to directly predict the position of
points of interest in a chart, and it is readily extensible to multiple chart
types and chart elements. We focus on complex bar charts in the scientific
literature, on which our model is able to detect salient points with an
accuracy of 0.8705 F1 (@1.5-cell max deviation); it achieves 0.9810 F1 on
synthetically-generated charts similar to those used in prior works. We also
explore training exclusively on synthetic data with novel augmentations,
reaching surprisingly competent performance in this way (0.6621 F1) on real
charts with widely varying appearance, and we further demonstrate our unchanged
method applied directly to synthetic pie charts (0.8343 F1). Datasets, trained
models, and evaluation code are available at
https://github.com/BNLNLP/PPN_model.