MDA-Network: Mask and Dual Attention Network for Handwritten Mathematical Expression Recognition

Building a system for automatic handwritten mathematical expressions recognition (HMER) has received considerable attention for its extensive applications. However, HMER remains challenging due to its own many characteristics such as the ambiguity of handwritten symbol, the two-dimensional characteristics of expression structure, and a large amount of context information. Inspired by research on machine translation and image caption, we proposed an Encoder-Decoder structure to recognize the handwritten mathematical expression. Encoder based dual attention is used to extract the features of the expression image and attention-decoder achieves symbol recognition and structural analysis. The mask information is added to the input data allows the model to better focus on the region of interest. In order to verify the effectiveness of our method, we train the model on the CROHME-2016 train set and use the CROHME-2014 test set as the validation set, the CROHME-2016 test set as the test set. The experimental results show that our method is greatly improved compared with other recognition methods, achieved respectively 47.49% and 45.10% ExpRate in the two test sets.