automatic evaluation of chinese translation output wordlevel or characterlevel
Báo cáo khoa học: "Automatic Evaluation of Chinese Translation Output: Word-Level or Character-Level" doc
Ngày tải lên :
17/03/2014, 00:20
...
for automatic evaluation of Chinese translation
output. Our analysis suggests several key reasons
behind this finding.
2 Chinese Translation Evaluation
Automatic MT evaluation aims at formulating ... Association for Computational Linguistics
Automatic Evaluation of Chinese Translation Output:
Word-Level or Character-Level?
Maoxi Li Chengqing Zong Hwee Tou Ng
National Laboratory of Pattern ... character-level metrics
more suitable for the automatic evaluation of
Chinese translation output.
6 Conclusion
In this paper, we conducted a detailed study of the
relative merits of word-level versus...
- 6
- 344
- 1
Tài liệu Báo cáo khoa học: "Automatic Evaluation of Machine Translation Quality Using Longest Common Subsequence and Skip-Bigram Statistics" doc
Ngày tải lên :
20/02/2014, 16:20
... human evalua-
tors. For the input of automatic evaluation meth-
ods, we created three evaluation sets from the MT
outputs:
1. Case set: The original system outputs with
case information.
2. ... section, we present the evaluations of
ROUGE-L, ROUGE-S, and compare their per-
formance with other automatic evaluation meas-
ures.
5 Evaluations
One of the goals of developing automatic evalua-
tion ... construction of N-best translation
lexicons from parallel text. Melamed (1995) used
the ratio (LCSR) between the length of the LCS of
two words and the length of the longer word of the
two words...
- 8
- 442
- 0
Báo cáo khoa học: "a Method for Automatic Evaluation of Machine Translation" pot
Ngày tải lên :
23/03/2014, 20:20
... Proceedings of the First Conference of
the Association for Machine Translation in the Ameri-
cas, pages 193–205, Columbia, Maryland.
BLEU: a Method for Automatic Evaluation of Machine Translation
Kishore ... judges
which substitutes for them when there is
need for quick or frequent evaluations.
1
1 Introduction
1.1 Rationale
Human evaluations of machine translation (MT)
weigh many aspects of translation, including ... “perfect” translations of a
given source sentence. These translations may vary
in word choice or in word order even when they use
the same words. And yet humans can clearly dis-
tinguish a good translation...
- 8
- 336
- 0
Tài liệu Báo cáo khoa học: "Automatic Evaluation of Sentence-Level Fluency Andrew Mutton∗" pdf
Ngày tải lên :
20/02/2014, 12:20
... Method for Automatic Evaluation of Ma-
chine Translation. Technical Report RC22176, IBM.
Joseph Turian, Luke Shen, and I. Dan Melamed. 2003. Evalua-
tion of Machine Translation and its evaluation. ... types of language model for
the generated text. Then in Section 6 we conclude.
2 Related Work
In terms of human evaluation, there is no uniform
view on what constitutes the notion of fluency, or ... letters for
correct spelling, good grammar, rhythm and flow,
appropriateness of tone, and several other specific
characteristics of good text.
In terms of automatic evaluation, we are not aware
of any...
- 8
- 507
- 0
Báo cáo khoa học: " A Tool for Error Analysis of Machine Translation Output" doc
Ngày tải lên :
07/03/2014, 22:20
... Morpho-syntactic information for au-
tomatic error analysis of statistical machine translation
output. In Proceedings of WMT, pages 1–6, New York
City, New York, USA.
Matthew Snover, Bonnie Dorr, ... though the main purpose of BLAST is for
error annotation of machine translation output, the
freedom in the use of error typologies and support
annotations also makes it suitable for other tasks
where ... colored text, where the color depends on
the main type of the error. For each new annotation
the user selects the word or words that are wrong,
and selects an error type. In figure 1, the words...
- 6
- 479
- 0
Báo cáo khoa học: "Automatic Construction of Machine Translation Knowledge Using Translation Literalness" docx
Ngày tải lên :
08/03/2014, 21:20
... Language
Mean # of Translation Words per Source Word
Mean Context-freeness (# of Word Link = 4)
28,300 words (49.5%)
3,107 words
1.51 trans./word
4.45
20,722 words (34.0%)
3,601 words
1.94 trans./word
4.21
Rewritten ... Agreement:
The word order of
a source sentence agrees substantially with
that of a target sentence. This measure en-
sures that the cost of word order adjustment
is small.
ã
Word Translation Stability: ... trans./word
4.21
Rewritten Translations
Original Translations
Table 1: Comparison of TDMT Training Translations and Original Translations
the rule writers rewrote translations to make tar-
get words as simple...
- 8
- 345
- 0
Báo cáo khoa học: "Automatic Evaluation of Linguistic Quality in Multi-Document Summarization" pptx
Ngày tải lên :
16/03/2014, 23:20
... of systems,
ignoring the linguistic quality of the system out-
put. Part of the reason for this imbalance is the
existence of ROUGE (Lin and Hovy, 2003; Lin,
2004), the system for automatic evaluation ... pair-
wise comparisons of competing systems
over a test set of several inputs and 70%
for ranking summaries of a specific input.
1 Introduction
Efforts for the development of automatic text sum-
marizers ... are the total number of word types from
both sentences s
i
and s
i+1
. Stop words were re-
tained. The value of each dimension for a sentence
is the number of tokens of that word type in that
sentence....
- 11
- 407
- 0
Báo cáo khoa học: "Correlating Human and Automatic Evaluation of a German Surface Realiser" doc
Ngày tải lên :
23/03/2014, 17:20
... measures of fluency, i.e. syntactic cor-
rectness.
In this paper, we carry out experiments to corre-
late automatic evaluation of the output of a surface
realisation ranking system for German ... 0.3333
Table 2: Correlation between human judgements
for experiment A (rank 1–3) and automatic metrics
for attempting to account for different but equiva-
lent translations of a given source word, typically
by ... between the automatic eval-
uation of system output and human (expert and
non-expert) evaluation of the same data (English
weather forecasts). Their findings show that the
NIST metric correlates...
- 4
- 285
- 0
Báo cáo khoa học: "Feedback Cleaning of Machine Translation Rules Using Automatic Evaluation" pot
Ngày tải lên :
08/03/2014, 04:22
... size of the evaluation corpus. Furthermore, in
order to find the nearest neighbor solution, we have
to calculate all BLEU scores of the neighborhood.
If the number of rules is R and the neighborhood
is ... domain, bilingual corpora of the new do-
main are necessary. However, the sizes of the new
corpora are generally smaller than that of the orig-
inal corpus because the collection of bilingual sen-
tences ... 149,882
Corpus
# of Words 868,087 984,197
Evaluation # of Sentences 10,145
Corpus
# of Words 59,533 67,554
Test # of Sentences 10,150
Corpus
# of Words 59,232 67,193
Table 1: Corpus Size
the rule subsets cannot...
- 8
- 313
- 0
Tài liệu Báo cáo khoa học: "Collecting a Why-question corpus for development and evaluation of an automatic QA-system" pdf
Ngày tải lên :
20/02/2014, 09:20
... development of sys-
tems. Automatic evaluation requires a cor-
pus of questions and answers, a definition of
what is a correct answer, and a way to com-
pare the correct answers to automatic answers
produced ... selection of natural questions. The
articles varied in topic, degree of formality and the
amount of details; from ”Horror film” and ”Christ-
mas worldwide” to ”G-Man (Half-Life)” and ”His-
tory of London”. ... and
Tsunenori Mori. 2007. An Overview of the 4th Ques-
tion Answering Challenge (QAC-4) at NTCIR work-
shop 6. In Proceedings of the Sixth NTCIR Workshop
Meeting, pages 433–440.
Chiori Hori, Takaaki Hori,...
- 9
- 610
- 1
Báo cáo khoa học: "A Formula Finder for the Automatic Synthesis of Translation Algorithms" docx
Ngày tải lên :
16/03/2014, 19:20
... ing, etc., from English word forms, thereby re-
ducing the inflected word forms to more or less stand-
ard stem forms.
Machinable rules for the automatic splitting of word
affixes, a process ... order according to rule (1),
added new English correspondents according to rule
(2), deleted the translations of homographic Russian
words according to rule (3), inserted short words ac-
cording ...
English correspondent, the inflection of a cor-
respondent into the plural, etc.
2
(2) A determiner formula D
r
for the desired algor-
ithm. This is the portion of the algorithm
known beforehand;...
- 14
- 432
- 0
Báo cáo khoa học: "Automatic Adaptation of Annotation Standards: Chinese Word Segmentation and POS Tagging – A Case Study" potx
Ngày tải lên :
17/03/2014, 01:20
... point for word
segmentation and 1 point for Joint S&T, with cor-
responding error reductions of 30.2% and 14%.
The final result outperforms the latest work on the
same corpus which uses more ... representation
of Ng and Low (2004). For word segmentation
only, there are four boundary tags:
ã b: the begin of the word
ã m: the middle of the word
ã e: the end of the word
ã s: a single-character word
while ... Processing
norms of modern chinese corpus. Technical report.
Yue Zhang and Stephen Clark. 2007. Chinese seg-
mentation with a word-based perceptron algorithm.
In Proceedings of the 45th Annual Meeting of...
- 9
- 404
- 0
Báo cáo khoa học: "Automatic Evaluation Method for Machine Translation using Noun-Phrase Chunking" pptx
Ngày tải lên :
23/03/2014, 16:20
... Utsuro and Noriko Kando.
2009. Meta -Evaluation of Automatic Evaluation
Methods for Machine Translation using Patent
Translation Data in NTCIR-7. In Proc. of the
3rd Workshop on Patent Translation, ... Na-
tional Institute of Informatics (NII) provided
corpora used in this work. The author grate-
fully acknowledges JAPIO and NII for their
support. Moreover, this work was partially
supported by Grants ... a parameter for the
weight of score
np
: it is between 0.0 and 1.0.
The ratio of score
wd
to score
np
is 1:1 when δ is
1.0. Moreover, score
wd multi
and score
np multi
are used for Eq. (13) in...
- 10
- 415
- 0
Báo cáo khoa học: "Towards Web-Based Evaluation of Automatic Natural Language Phrase Generation" potx
Ngày tải lên :
23/03/2014, 18:20
... the length, i.e., number of words, among the
analyzed phrases differs and our algorithm does
not use semantical information, in order to avoid
the distortion of the original phrase, in our system
the ... generator for open do-
main dialogue systems, or chatbots.
In this paper, we propose the automatic gene-
ration of trivial dialogue phrases through the ap-
plication of a transfer-like genetic algorithm ... which one of the genera-
ted expressions is correct. Our method differs in
the application of a GA-like transfer process in
order to automatically insert new features on the
selected original...
- 4
- 260
- 0
Báo cáo khoa học: "Comparing Automatic and Human Evaluation of NLG Systems" potx
Ngày tải lên :
24/03/2014, 03:20
... score for a set of generator forecasts
is the average of scores for individual forecasts.
Human evaluations: We recruited 9 experts
(people with experience reading forecasts for off-
shore oil ... types of NLG systems for which
the correlation of automatic and human evaluation
was particularly good or bad.
Data: We extracted from each forecast in the
SUMTIME corpus the first description of ... top of Table 1).
We randomly selected a subset of 21 forecast
dates for use in human evaluations. For these 21
forecast dates, we also asked two meteorologists
who had not contributed to the original...
- 8
- 376
- 0
