Evaluation notes on MIR

Reported hits are evaluated on precision, recall, results ranking and (if present) score. Hits without subformula identifiers (i.e. where the URI reference only identifies the document will be penalized by losing half of the points they would contribute to the results. Hits with dangling references are eliminated from the result. Moreover, reported hits will be de-duplicated by removing any reported hits that point to the same formula (the id identifiers reference subexpressions of the same element in the document) with non-maximal rank. Hits that do not reference mathematical subformulae but document-fragments will be de-duplicated accordingly. Additional de-duplication may be applied subject to the discussion of the organizers.