... Proceedings of the 2nd International Conference on Language Resources and Evaluation, Athens, Greece. NIST. 2002. AutomaticEvaluationofMachine Translation Qualityusing N-gram Co- Occurrence Statistics. ... Quantitative Quality Measure for Machine Translation System. In Proceedings of COLING-92, Nantes, France. Thompson, H. S. 1991. AutomaticEvaluationof Translation Quality: Outline of Methodology ... skip-bigrams: 3 Combination: C(4,2) = 4!/(2!*2!) = 6. AutomaticEvaluationofMachineTranslationQualityUsing Longest Com-mon Subsequence and Skip-Bigram Statistics Chin-Yew Lin...
... checks.These translations are all correct. Actually, thecorpus of Takezawa et al. (2002) contains ten dif-ferent translations of this source sentence. Whenwe construct MT knowledge from corpora thatcontain ... showthat most context/situation-dependent translationsare non-literal.MT knowledge constructed fromcontext/situation-dependent translations causeincorrect target sentences, which may containomissions ... effect of literalness in MTknowledge construction, we constructed knowl-edge by using the methods described in Section5 and evaluated the MT qualityof the resultingEnglish-to-Japanese translation. 6.1...
... Corpus BLEU Score Evaluation Corpus BLEU ScoreNumber of RulesFigure 5: Relationship between Number of Itera-tions and BLEU Scores/Number of Rules2. Subjective Quality A total of 510 sentences ... test corpus BLEU score and the subjective2In this experiment, it took about 80 hours until convergence using a Pentium 4 2-GHz computer. Feedback Cleaning ofMachineTranslation Rules Using Automatic ... requires very large corpora in order to obtain asufficient number of rules that are statistically confi-dent.Another current topic ofmachinetranslation is automatic evaluationof MT quality (Papineni...
... Doddington, 2002. AutomaticEvaluationof Machine TranslationQualityUsing N-gram Co- occurrence Statistics. Proceedings of the Second International Conference on Human Language Technology ... for MachineTranslation Performance. Proceedings of the Third Workshop on Statistical Machine Translation, pages 224-232, Columbus, Ohio, USA. George Doddington, 2002. AutomaticEvaluationof ... Fordyce, Philipp Koehn, Christof Monz and Josh Schroeder, 2007. (Meta-) EvaluationofMachine Translation. Proceedings of the Second Workshop on Statistical Machine Translation, pages 136-158,...
... sentence ordering. In Proceedings of ACL, pages 545–552.C.Y. Lin and E. Hovy. 2003. Automaticevaluation of summaries using n-gram co- occurrence statistics. InProceedings of NAACL/HLT, page 78.C.Y. ... more predictive of the linguistic qualityof these summaries.7 ConclusionWe have presented an analysis of a wide variety of features for the linguistic qualityof summaries.Continuity between ... EACL,pages 148–156.J.M. Conroy and H.T. Dang. 2008. Mind the gap: dan-gers of divorcing evaluations of summary contentfrom linguistic quality. In Proceedings of COLING,pages 145–152.S....
... Summary and Conclusions Evaluation of the qualityofmachine translations by means of a test of reading comprehension and by judg- ments of sentence clarity, was investigated. Human translations ... necessarily accurate, translations, EVALUATION OFMACHINE TRANSLATIONS 7 [Mechanical Translation, vol. 8, No. 2, February 1965] Evaluation ofMachine Translations by Reading Comprehension Tests ... both types of translations, while loss of context in a paragraph affects only machine trans- lations, is hardly surprising. The range of values of a set of stimuli along a judged continuum...
... characterising an aspect of text quality thatis not related to its content source but to the use of language itself. It is a goal of this paper to providean automaticevaluation method for fluency ... the Role of Bleu in Machine Translation Research. In Proceedings of EACL, pages 249–256.Jos´e Coch. 1996. Evaluating and comparing three text-production strategies. In Proceedings of the 16th ... Evaluationof Ma-chine Translation. Technical Report RC22176, IBM.Joseph Turian, Luke Shen, and I. Dan Melamed. 2003. Evalua-tion ofMachineTranslation and its evaluation. In Proceedings of MT...
... ),()()()(ferencesRnRSngramferencesRnRSngramclipngramCountngramCountnR where, as before, Count (ngram) is the number of n-gram counts, and Countclip (ngram) is the maximum number of co- occurrences ofngram ... framework for automaticevaluationof NLP applications using N-gram co- occurrence statistics. The automatic evaluation metrics proposed to date for MachineTranslation and Automatic Summarization ... ∑∑∑∑∈∈∈∈=}{),(}{),()()()(CandidatesCnCSngramCandidatesCnCSngramclipngramCountngramCountnP where Count (ngram) is the number of n-gram counts, and Countclip (ngram) is the maximum number of co- occurrences ofngram in the candidate answer and its...
... your|$ coffee|coffee or|or $|teareorderingI|$ would|would you|you like|like have|$ some|$ coffee|coffee $|or tea|tea$ would you like $ $ coffee or teaconfusion$ would you have $ $ coffee ... teaoriginal2. would you have tea or coffeehypotheses3. would you like your coffee or4. I have some coffee tea would you likealignment would|would you|you have|like coffee|coffee or|or tea|teaandwould|would ... from the union of confu-35 Table 7: Improvements in translationquality onthe BTEC Italian-English task through comput-ing consensus translations from the output of twospeech translation systems...
... human and machine translations.ã Distinguish uent machine translations frompoor machine translations.ã Distinguish the better (in terms of uency) translation among two translations of thesame ... done in thecontext ofmachinetranslation for preliminary in-vestigations of fluency. Nevertheless, our findingsare also potentially beneficial for sentence-level evaluation ofmachine translation. 2 ... and C. Brockett. 2001.A machine learning approach to the automatic eval-uation ofmachine translation. In Proceedings of 39th Annual Meeting of the Association for Compu-tational Linguistics,...
... 2.1 MachineTranslationQuality Prediction Machine translationevaluation aims to assess the correctness and qualityof the translation. Usu-ally, the human reference translation is provided, ... by making use of both the machinetranslationquality score and the informativeness score; 4) The extracted Eng-lish summary is translated into Chinese summary by usingmachinetranslation ... Section 5. 4 MachineTranslationQuality Predic-tion 4.1 Methodology In this study, machinetranslation (MT) quality reflects both the translation accuracy and the flu-ency of the translated...
... Principles of context-based machinetranslation evaluation. Machine Translation, 17(1):43–75.Masaki Murata, Kiyotaka Uchimoto, Qing Ma, ToshiyukiKanamaru, and Hitoshi Isahara. 2005. Analysis of machine ... ref-erence translation. 1 Introduction Machine translationevaluation is a difficult task,since there is not only one correct translationof asentence, but many equally good translation options.Often, ... developers.2 MT Evaluation and Error AnalysisHovy et al. (2002) discussed the complexity of MT evaluation, and stressed the importance of adjusting evaluation to the purpose and context of the trans-lation....
... extraction of I(Tn), whereas the main problem ofmachinetranslation is the algorithmiza- tion of this process of extraction. The introduction into the theory oftranslationof this logical construct—the ... basic conclusion: the process of human translation (and therefore of analysis in the course oftranslation from one natural language into another, for such translation could be treated as a combination ... examination of the process of understanding and, conse- quently, on an examination of the process of generation in language communication (e.g., multiple synthesis) and of translation, that is, of...
... relational concepts (mainly reflected by verbs): we are trying to get a set of core verbs as proto- types of classes based on certain statistics and genus verb sense nets (the latter is being con- ... layer. 3. Dynamic creation of Cases. Initial rep- resentations in the context layer may be ad- justed and new Cases be created according to a set of contextual conditions (mainly when mismatches ... An Artificial Intelligence Approach to Machine Translation, in Computer Models o] Thoaght 6nd Language, R.Schaak and K.Colby (eds.), W.H.Freemaa Co. , 1973. 338 ...