Automatic evaluation of machine translation, paraphrase generation, and summarization a linear programming based analysis

... weights Variants of TESLA are devised for many different evaluation tasks: TESLAM, TESLA-B, and TESLA-F for the machine translation evaluation of European languages, TESLA-CELAB for the machine translation ... chapter reviews the current state of the art in the various natural language automatic evaluation tasks, including machine translation evaluation and its applications in tuning machine translation ... systems, paraphrase evaluation, and summarization evaluation They can all be viewed as variations of the same underlying task, that of measuring the semantic similarity between segments of text Among...

Tài liệu Báo cáo khoa học: "Automatic Evaluation of Machine Translation Quality Using Longest Common Subsequence and Skip-Bigram Statistics" doc

... automatic evaluation methods for machine translation, ROUGE-L based on longest common subsequence (LCS) statistics between a candidate translation and a set of reference translations Longest common subsequence ... two sequences X and Y, the longest common subsequence (LCS) of X and Y is a common subsequence with maximum length We can find the LCS of two sequences of length m and n using standard dynamic ... maximum number of n-grams co-occurring in a candidate translation and a reference translation, and Count(ngram) is the number of n-grams in the candidate translation To prevent very short translations...

Báo cáo khoa học: "a Method for Automatic Evaluation of Machine Translation" pot

... output of ChineseEnglish MT systems References E.H Hovy 1999 Toward finely differentiated evaluation metrics for machine translation In Proceedings of the Eagles Workshop on Standards and Evaluation, ... Human Evaluation Figure shows a linear regression of the monolingual group scores as a function of the BLEU score over two reference translations for the systems The high correlation coefficient of ... Evaluation We had two groups of human judges The first group, called the monolingual group, consisted of 10 native speakers of English The second group, called the bilingual group, consisted of...

Báo cáo khoa học: "A machine learning approach to the automatic evaluation of machine translation" potx

... using the human reference translations and the output of the MT system at a given point in time Using the same set of features, we expect the accuracy of the classifiers to go down over time as the ... sampling the initial datasets Thanks also to John Platt for helpful discussion on parameter setting for the SVM tools, and to the members of the MSR NLP group for feedback on the uses of the methodology ... template, i.e the labels of the immediate children of the root node of a sentence, was also used, as was subject-verb disagreement The final five features are: number of OOV words the presence of a word...

Báo cáo khoa học: "Automatic Construction of Machine Translation Knowledge Using Translation Literalness" docx

... (B) Literal Translation Figure 3: Examples of Generated Rules for Japanese-to-English Translation (A) from Non-literal Translation by Split Construction (B) from Literal Translation translations ... effect of literalness in MT knowledge construction, we constructed knowledge by using the methods described in Section and evaluated the MT quality of the resulting English-to-Japanese translation ... for knowledge construction, so the coverage of MT knowledge can be maintained However, some context/situation-dependent translations remain in the extracted corpus when only one non-literal translation...

Báo cáo khoa học: "Automatic Evaluation of Chinese Translation Output: Word-Level or Character-Level" doc

... reference translation segmented into words: Translation: 多少_钱_的_伞_吗_？ Reference: 这些_雨伞_多少_钱_？ The word “伞” is a synonym for the word “雨伞”, and both words are translations of the English word “umbrella” ... character-level metrics more suitable for the automatic evaluation of Chinese translation output Conclusion In this paper, we conducted a detailed study of the relative merits of word-level versus character-level ... word “京都” is the Chinese translation of the English word “Kyoto” However, it is segmented into two words, “京” and “都”, in the reference translation by the same CWS tool When this happens, a word-level...

Báo cáo khoa học: "An Empirical Evaluation of Data-Driven Paraphrase Generation Techniques" pptx

... goal of our experimental evaluation is to analyze the effectiveness of a variety of paraphrase generation techniques, ranging from simple to sophisticated Our evaluation focuses on generating paraphrases ... Our evaluation helps develop a better understanding of the strengths and weaknesses of each type of approach The evaluation also brings to light additional properties, including the number of ... lose” within the top 10 paraphrases All of these are simple variants that contain different forms of the verb “lose” Under the lenient evaluation criterion almost all of these paraphrases would be...

Báo cáo khoa học: "Automatic Evaluation of Linguistic Quality in Multi-Document Summarization" pptx

... ORGANIZATIONs, and LOCATIONs Indicators of linguistic quality Multiple factors in uence the linguistic quality of text in general, including: word choice, the reference form of entities, and local coherence ... rankings of content The development of automatic linguistic quality measurements will allow researchers to optimize both content and linguistic quality 5.2 Evaluation of rankings We examine the ... that in the input Since the previous sentence in the input text often contains the antecedent of pronouns in the current sentence, if the previous sentence from the input is also included in the...

Báo cáo khoa học: "Feedback Cleaning of Machine Translation Rules Using Automatic Evaluation" pot

... portion of correct MT results Focusing on the number of the rules, the rule set of the simple feedback cleaning is clearly a locally optimal solution, since the number of rules is more than that of ... procedure of cross -cleaning is as follows Feature English Japanese # of Sentences 149,882 # of Words 868,087 984,197 # of Sentences 10,145 # of Words 59,533 67,554 # of Sentences 10,150 # of Words ... Masuzo Yanagida 2001 An automatic evaluation method of translation quality using translation answer candidates queried from a parallel corpus In Proceedings of Machine Translation Summit VIII,...

Tài liệu Báo cáo khoa học: "Evaluation of Machine Translations by Reading Comprehension Tests and Subjective Judgments" doc

... would appear likely to be of some use Summary and Conclusions Evaluation of the quality of machine translations by means of a test of reading comprehension and by judgments of sentence clarity, was ... translated by humans While tests of reading comprehension provide a more direct measure of the usefulness of translations than judgments of sentence clarity, the latter approach is simpler, and may ... DISCUSSION OF THE READING COMPREHENSION TESTS In considering the results of the reading comprehension tests, perhaps the most striking feature is the relatively small difference in the number of correct...

Tài liệu Báo cáo khoa học: "Predicting the ﬂuency of text with shallow structural features: case studies of machine translation and human-written text" doc

... understand which factors are predictive of good fluency The distribution of fluency scores in the dataset is rather skewed, with the majority of the sentences rated as being of average fluency as ... assess the content of the sentence compared to a human gold-standard Yet, the assessments of the two aspects were often the same—readability /fluency of the sentence is important for understanding the ... from machine translations • Distinguish human and machine translations • Distinguish fluent machine translations from poor machine translations • Distinguish the better (in terms of fluency) translation...

Báo cáo khoa học: "Correlating Human and Automatic Evaluation of a German Surface Realiser" doc

... 2006 Comparing automatic and human evaluation of NLG systems In Proceedings of EACL 2006, pages 313–320, Trento, Italy Amanda Stent, Matthew Marge, and Mohit Singhai 2005 Evaluating evaluation methods ... of NAACL-03, pages 61–63, NJ, USA Karolina Owczarzak, Josef van Genabith, and Andy Way 2008 Evaluating machine translation with LFG dependencies Machine Translation, 21:95– 119 Karolina Owczarzak ... generation in the presense of variation In Proceedings of CICLING, pages 341–351 Aoife Cahill and Martin Forst 2009 Human Evaluation of a German Surface Realisation Ranker In Proceedings of EACL...

Báo cáo khoa học: "Combining Source and Target Language Information for Name Tagging of Machine Translation Output" ppt

... constituent, such as a name, which could lead to boundary errors in tagging English names We have therefore used an alternative method to fetch the source language information for information extraction, ... from using source language information In the nist05 data, we find 1893 named entities in the English NER output (target language part) and 1968 named entities in the ET output (source language ... to extract the named entities from machine translated text, using name entity information from both source and target language Our experiments show that with a combination of ET and English NER,...

Báo cáo khoa học: "DICTIONARY ORGANIZATION FOR MACHINE TRANSLATION: THE EXPERIENCE AND IMPLICATIONS OF THEUMIST JAPANESE PROJECT" ppt

... mapped For mnemonic reasons these pointers normally take the form of the lexical stem of the translation equivalent or gloss, but this is purely a convenience for the user, and should not obscure their ... reference to translation: the optimum grain-size for lexical entries, the division of information about separate languages, and the level of abstraction appropriate to the task of translation ... dictionary information available both in a form perspicuous to the human user and, independently, in a form optimally adapted to the design of the parser CREATION The dictionaries a n d the s y s...

Tài liệu Báo cáo khoa học: "Automatic Evaluation of Sentence-Level Fluency Andrew Mutton∗" pdf

... grammar, rhythm and flow, appropriateness of tone, and several other specific characteristics of good text In terms of automatic evaluation, we are not aware of any technique that measures only fluency ... PoStag In the first of these, we constructed a rough approximation of typical sentence grammar structure by taking bigrams over part -of- speech tags.6 Then, given a string of PoS tags of length n, t1 ... Evaluation of Machine Translation Technical Report RC22176, IBM Joseph Turian, Luke Shen, and I Dan Melamed 2003 Evaluation of Machine Translation and its evaluation In Proceedings of MT Summit...

Xem thêm

Từ khóa: automatic evaluation of machine translation quality using ngram automatic evaluation of machine translation quality using ngram co occurrence statistics automatic evaluation of machine translation quality using ngram cooccurrence statistics automatic evaluation of chinese translation output an empirical evaluation of data driven paraphrase generation techniques automatic evaluation of chinese translation output wordlevel or characterlevel automatic evaluation of text coherence models and representations machine translation the experience and implications case studies of machine translation feedback cleaning of machine translation rules automatic evaluation of sentencelevel statistical machine translation with word and sentence aligned parallel corpora evaluation of machine learning methods for natural language processing tasks 11 resources evaluation of innovation projects between lean and slack cfd for simulation of acid mine drainage generation and subsequent pollutants transportation through groundwater flow systems and rivers Báo cáo thực tập tại nhà thuốc tại Thành phố Hồ Chí Minh năm 2018 Nghiên cứu sự biến đổi một số cytokin ở bệnh nhân xơ cứng bì hệ thống chuyên đề điện xoay chiều theo dạng Nghiên cứu tổ chức pha chế, đánh giá chất lượng thuốc tiêm truyền trong điều kiện dã ngoại Biện pháp quản lý hoạt động dạy hát xoan trong trường trung học cơ sở huyện lâm thao, phú thọ Phát hiện xâm nhập dựa trên thuật toán k means Nghiên cứu, xây dựng phần mềm smartscan và ứng dụng trong bảo vệ mạng máy tính chuyên dùng Nghiên cứu về mô hình thống kê học sâu và ứng dụng trong nhận dạng chữ viết tay hạn chế Nghiên cứu tổng hợp các oxit hỗn hợp kích thƣớc nanomet ce 0 75 zr0 25o2 , ce 0 5 zr0 5o2 và khảo sát hoạt tính quang xúc tác của chúng Tìm hiểu công cụ đánh giá hệ thống đảm bảo an toàn hệ thống thông tin Kiểm sát việc giải quyết tố giác, tin báo về tội phạm và kiến nghị khởi tố theo pháp luật tố tụng hình sự Việt Nam từ thực tiễn tỉnh Bình Định (Luận văn thạc sĩ)Quản lý nợ xấu tại Agribank chi nhánh huyện Phù Yên, tỉnh Sơn La (Luận văn thạc sĩ)BT Tieng anh 6 UNIT 2 Tranh tụng tại phiên tòa hình sự sơ thẩm theo pháp luật tố tụng hình sự Việt Nam từ thực tiễn xét xử của các Tòa án quân sự Quân khu (Luận văn thạc sĩ)Nguyên tắc phân hóa trách nhiệm hình sự đối với người dưới 18 tuổi phạm tội trong pháp luật hình sự Việt Nam (Luận văn thạc sĩ)Giáo án Sinh học 11 bài 14: Thực hành phát hiện hô hấp ở thực vật Chiến lược marketing tại ngân hàng Agribank chi nhánh Sài Gòn từ 2013-2015 Đổi mới quản lý tài chính trong hoạt động khoa học xã hội trường hợp viện hàn lâm khoa học xã hội việt nam MÔN TRUYỀN THÔNG MARKETING TÍCH HỢP TÁI CHẾ NHỰA VÀ QUẢN LÝ CHẤT THẢI Ở HOA KỲ