ICFHR 2020 Competition on Offline Recognition and Spotting of Handwritten Mathematical Expressions

Acronym: OffRaSHME (Offline Recognition and Spotting of Handwritten Mathematical Expressions)

Evaluation Protocol

Offline Formula Recognition Metrics

Consist with previous CROHME competitions, formula recognition results are evaluated in the format of symbol label graph. Symbol label graph metrics include both the math symbols and the structural relationships. The evaluation on validation dataset are available to participants offline with the CROHMELib and LgEval libraries. Participating systems that recognize formula images into LaTeX codes or other format representations are required to convert the results into Symbol Label graphs. LaTeX format results can be converted into Symbol Label graphs with the LgEval libraries.

Handwritten Formula Spotting Metrics

For the Handwritten Formula Spotting task, participating systems are required to retrieve all the formula images in the test set that are consistent with the content of each image (or each online stroke sequence for subtask 2b) in the query set. The retrieval results are measured with F1-Score.