... decisions, synonymy, andhypernymy. These phenomena are well-known tobe important to language understanding and RTE(Mirkin et al., 2009; Roth and Sammons, 2007).Many synonymy and hypernymy databases ... κ handles more than two annotators, unlike themore familiar Cohen’s κ.330tions and all categories. Separately, we also computeκ once for each category C, by treating all annota-tions for ... which of theseforms is intended, but for annotators quickly read-ing through the proofs, the two kinds of knowledgecan look superficially very similar, and the annota-tors can become confused.The...