For instance, if a text is only needed for gisting purposes, typos will not be fixed. What’s the level of quality required? Will the text be published, circulated internally or does it only serve for “gisting” purposes? What is the quality of the source text? Depending on the answer to these and many other questions, post-editing can involve more or fewer edits. No localization project is the same as the previous one, as we all know. TER does not account for the actual time spent in the process. Similarly, leaving a sentence unchanged (which is becoming more and more frequent) does not mean that the post-editor won’t have to read the target sentence and compare it to the source text. One single edit in a sentence might take more to spot than the time needed to do the actual change and, again, requires much cognitive effort. However, the high level of fluency of texts translated with neural MT can be misleading and hide more serious mistakes (for instance, mistranslations, missing words or wrong terminology). Depending on different items (source quality, language combination, vertical, to mention a few) quality of NMT can be very high, with just a few edits (if any) needed. With this technology, many of the typical issues of MT are solved, including lack of fluency, grammar mistakes, wrong concordance. Machine Translation has improved a lot in the last 4 years following the introduction of Neural Machine Translation by Google. The quality of the edits also needs to be taken into account.įor this reason, evaluating the quality of a machine translation raw output by the number of edits it requires fails to take into account the actual effort involved in the process. Similarly, not all edits require much cognitive effort.Īlso, the quantity of edits is just part of the issue, as one single edit might be more critical than multiple minor edits. This is because cognitive decisions do not necessarily involve any edits (for instance, when the MT output is left unchanged). reduced) accordingly.Īsk any post-editor, seasoned or not, you’ll be told post-editing requires a lot of effort.īut does TER (or better, does the number of edits) really reflect the cognitive effort by the post-editor? Simply said, cognitive effort is the effort related to the mental processes that take place during post-editing, which are not necessarily proportional to technical effort. As a consequence, the post-editing rate can be adapted (i.e. To help those buyers, some CAT tools have embedded post-editing analysis with the ultimate objective to make post-editing pricing easier and more democratic.īasically, the assumption is that the fewer the edits made, the less effort is needed from the post-editor. However, many big MT players (including big post-editing buyers) have adopted the same approach to calculate the number of changes made during post-editing, therefore retrospectively, rather than as a predictive method. HOW TER IS USEDĪs explained, TER evaluation method has been created to predict post-editing effort. A high TER score implies that a translation will require more changes during post-editing. TER score can be a value between 0 and 1, but more frequently it is calculated as a percentage. It’s important to reiterate that the effort is measured in terms of quantity of changes. When this analysis is repeated on different sample translations, it’s possible to estimate the post-editing effort required. TER basically measures the amount of editing that a linguist would need to perform in order for a translation to match a reference (human) translation. One of the methods developed to help in predicting post-editing effort is Translation Error Rate (TER).
#Ter definition how to#
As a consequence, how to price machine translation post-editing is a hot topic since ever. For this reason, it’s difficult to predict how long it would take to post-edit an automated translation in order to make it publishable (or suitable for the desired use, whatever that is). The quality of raw output varies a lot, and the expectation of quality from post-editors is also different from project to project. Post-editing machine-translated texts is a challenging task that requires intense effort.
0 kommentar(er)
