... pages 89–92,Prague, June 2007.c2007 Association for Computational LinguisticsTest Collection Selection and Gold Standard Generation for a Multiply-Annotated Opinion Corpus Lun-Wei Ku, ... gold standards are proposed. Under the gold standards, an opinion extraction sys-tem is evaluated. The experiment results show some interesting phenomena. 1 Introduction Opinion information ... information retrieval issue. For opin-ions, because their strength varies, sometimes it is hard for annotators to tell if a sentence is opinion-ated. However, for the opinion polarities, the...