ICFHR 2020 Competition on Offline Recognition and Spotting of Handwritten Mathematical Expressions

Acronym: OffRaSHME (Offline Recognition and Spotting of Handwritten Mathematical Expressions)

Tasks

Different from CROHME, we prohibit participants from using additional training data or mathematical corpus, and encourage contestants to innovate some new algorithms. In addition, when submitting test set results, multiple submissions will only take the highest results of the first three submissions for final ranking.

Task 1

Offline Handwritten Formula Recognition This is the base task in the OffRaSHME. Participants need to design recognition systems to automatically convert real scene formula images into Symbol Layout Tree or LaTeX. Participating systems are ranked according to the number of correctly recognized formulas (expression recognition rate).

In this task, training set contains handwritten formula images obtained by scanning or taking pictures. Besides, all symbol positions in the images are also provided. Since symbol-level markup requires a lot of manual work, participants need to declare whether strong marks are used. Ranked by expression recognition rate.

Task 2

Handwritten Formula Spotting This task aims to explore the prospects of formula recognition in computer-assisted scoring and formula retrieval. Participating system in this task can either recognize the formula and compare the results or perform formula-image matching and clustering. This task can be used for document retrieval and computer-assisted scoring to retrieve results consistent with the correct answer. In addition, retrieval or matching with a two-dimensional structure is also enlightening for other computer vision tasks. Specifically, this task contains two subtasks.

Task 2a (Offline Handwritten Formula Spotting): In the education system, the teacher gives a correct answer, the auxiliary mark system can determine which students' homework is correct. Given an offline formula image, participating systems find out all the formula images in the test set that match its content. Ranked by the F1-Score.

Task 2b (Cross-Modal Handwritten Formula Spotting): It is convenient to use handwriting input on some handheld devices like mobile phone. When looking through the web or databases for information, the user may select online handwriting input. In this task, online handwritten formulas are provided. Participating systems find out all the formula images in the test set that contain the content of the online handwritten formula to be queried. Ranked by the F1-Score.

Important dates

The following table presents important dates for the participants of the competition.

Table 1. Important dates for participants

Date Activity
1 Jan. - 29 Feb. Manifestation of interest by participants
Asking/Answering questions about the details of the competition
Initial website available
1 Feb. Competition formal announcement, publicity
Sample training image set available
Evaluation protocol, file formats etc. available
29 Feb. Evaluation tools available
Website fully ready
1 Mar. - 1 May. Train set available - training period - competition in progress
Participants evaluate their methods on the training/sample set
Prepare for submission
1 May. Participation in the competition closes
1 May. - 1 Jun. Test set available
1 Jun. Deadline for submission of results by participants
1 Oct. The public release of the full dataset