Consist with previous CROHME competitions, formula recognition results are evaluated in the format of symbol label graph. Symbol label graph metrics include both the math symbols and the structural relationships. The evaluation on validation dataset are available to participants offline with the CROHMELib and LgEval libraries. Participating systems that recognize formula images into LaTeX codes or other format representations are required to convert the results into Symbol Label graphs. LaTeX format results can be converted into Symbol Label graphs with the LgEval libraries.
For the Handwritten Formula Spotting task, participating systems are required to retrieve all the formula images in the test set that are consistent with the content of each image (or each online stroke sequence for subtask 2b) in the query set. The retrieval results are measured with F1-Score.