I've read a few million words of French—nothing special, but enough that if a text gives me trouble, then it's also likely to break Google Translate. Over the years, I've collected a few example texts which give Google Translate an especially hard time, and I use those for testing other machine learning systems.
DeepL performs significantly better on the most difficult texts I've given it. It's substantially better with colloquial language, and—oddly—nautical language. It also seems to be much better at tracking relationships between words in longer sentences. Good work!
It's amazing how much nautical language and references contribute to colloquial phrases. It may be just the company I keep but I find it consistently comes up in French.
Impressive! David vs Goliath!
Google thought Angela Merkel was a "He". And DeepL not just translated correctly, it also gave a really concise text. Following text from "Der Spiegel".
Original: Angela Merkel hat sich gegen Vorwürfe gewehrt, dass es dem Bundestagswahlkampf an Spannung fehle.
Google translate: Angela Merkel has reproached himself against allegations that the Bundestag election campaign is lacking in tension.
DeepL: Angela Merkel resisted accusations that the Bundestag election campaign lacked tension.
DeepL back to German: Angela Merkel widersetzte sich dem Vorwurf, der Bundestagswahlkampf sei spannungsarm.
English again: Angela Merkel opposed the accusation that the election campaign in the Bundestag was not tense.
German: Angela Merkel wandte sich gegen den Vorwurf, der Wahlkampf im Bundestag sei nicht gespannt.
English: Angela Merkel objected to the accusation that the election campaign in the Bundestag was not tense.
German: Angela Merkel wandte sich gegen den Vorwurf ein, der Wahlkampf im Bundestag sei nicht angespannt.
This is a fixed point (the translations no longer change).
The quality and stability of the translations is impressive, but the final German is a bit off, it seems to be confused between "objected to [something objectionable]" and "objected that [some counterargument]", mixing both in the same sentence.
EDIT: I tried going through all languages, German->English->French->Spanish->Italian->Dutch->Polish->..., after a few iterations, it settled on "Merkel wendet sich gegen die Vorwuerfe, der Bundestagswahlkampf sei nicht gespannt." (Merkel is opposed to accusations that the Bundestag election campaign is not tense.)
Things that got lost in translation: Merkel's first name and the past tense (EDIT: and the subtle distinction between "not lacking tension" and "being tense"). The pluralization of accusation(s) seems to change based on the language. Really quite impressive to maintain the meaning over so many steps.
In my mind, the translation is an improvement over the original. For one, summaries are to be written using the present tense. Secondly, there is no technical difference between Spannung and Anspunnung that maatered, and Anspannung is the more technical term whereas Spannung suggests government elections were some form of entertainment.
And by the way. It's impressive^W^W Its impressive handling of the abomination that is "dass es" and similar constructions is pretty impressive (see what I did there?). In that sense, I'm surprised the reflexive form was recovered in the German.
Some background: DeepL is the new name of Linguee, a great online dictionary by German founder Gereon Frahling, who was an ex-Google employee. Here's an early interview with him that you can run through DeepL to translate: https://www.gruenderszene.de/allgemein/linguee-gereon-frahli...
It probably does well when translating general human speech, but I was rather interested in its performance on a paragraph of a scientific text (it being a much more challenging task, obviously).
* "The impact of the solar wind protons on the surface of Mercury" became "Der Einfluss der solaren Windprotonen auf die Oberfläche von Quecksilber".
Note that 'solar wind protons' should have been translated as 'Sonnenwindprotonen' instead, i.e. the word 'solar' was to be a part of the noun's modifier, but it was pushed out.
* The lack of domain-specific training is especially obvious with the case of the planet's name being translated as "Quecksilber" instead of "Merkur" (Quecksilber being the name of the metal).
* "pure northward interplanetary magnetic field (IMF)" became "reines interplanetares interplanetares Magnetfeld nach Norden (IWF)". Aside from this being a poor translation, it's worth noting that DeepL didn't properly process the introduction of an abbreviation (IWF being the abbrev. for the International Monetary Fund in German).
It's remarkly better than Google's, and possibly the best out there I've tried so far.
Even so, the correct translation for the second sentence would be one of "Moscas de la fruta como plátanos" or most probably "A las moscas de la fruta les gustan los platanos" instead, the ambiguity is due to like being either a verb or adverb.
"I saw some weird fruit flies."
"What were they like?"
"They were like bananas."
"Fruit flies like bananas?"
"Yeah, they were implausibly yellow and banana-shaped."
The tricky part is the zero relative pronoun in your construction that could be confused for a highly irregular-verb-like use of like. You wouldn't say either of "he like(s) (a) banana" in standard english, unless taking Influence from creole perhaps. Another source of confusion to me is the difference of uncountability, generic nouns etc. The whole thing would work without the interjection in "fruit fly bananas" or vice versa, from "banana-like fruit fly".
TIL about zero relative pronouns, despite using them all the time and wondering about why we can elide them in English but not reliably in many other languages, e.g. "The car (optional that) I bought" vs. "El coche (required que) compré".
How is fruit uncountable? Is human, too? Is fruits or the fruits not disambiguous enough? This is idiosyncratic at best. I wouldn't blame any learner misinterpreting the example and would avoid to teach this as correct.
I tried it and I was fully expecting DeepL to fare much worse than Google Translate. I was wrong. It's great to see that a small company can go head to head with Google.
By the way, most machine translation models are trained on news data. Try some out of domain data like tweets or other social media comments, if you want to put it to the test.
> Better than Google and Microsoft - the German company DeepL is committed to translation services. DeepL uses a novel architecture of neural networks and uses a supercomputer with 5.1 petaflops. By the same company, the service comes Linguee who has already made with translations of individual words or phrases a name. Texts translated by humans are used for translations in order to provide better results.
DeepL:
> Better than Google and Microsoft - this is the goal that DeepL, a German company, has set itself, at least for translation services. DeepL uses a novel architecture of neural networks and relies on a supercomputer with 5.1 petaflops. The Linguee service, which has already made a name for itself with translations of individual words or groups of words, comes from the same company. In doing so, human-translated texts are used for translations in order to deliver better results.
I think the DeepL version makes more sense here, but I am not a German reader so I might be off.
That's also first thing I tired, ... Looks like people are more predicable than we think. If I was them i would make sure that I do a good job on that.
Besides translation quality, I think the ability to change words in the target language is neat - could be a useful way to do assisted translation. These guys are interesting.
> Specific details of our network architecture will not be published at this time. DeepL Translator is based on a single, non-ensemble model.
Kinda sad to hear, but completely understandable. I'm curious whether the difference in performance is due to their model specifics or just better training data.
They have the perfect training data as this is a Linguee venture (https://www.linguee.com/). They have millions of translations of paragraphs from one language to another.
I have no information on the model, unfortunately.
Surprisingly good on Jabberwocky, though my French is too weak to really judge:
"Twas Brillig, et les fentes fendues tournoyaient et gimblaient dans l'épée. Tous les mimsy étaient des borogoves, et les mome raths dépassaient les ragots.
Méfie-toi du Jabberwock, mon fils! Les mâchoires qui mordent, les griffes qui attrapent. Et'ware l'oiseau Jubjub, et fuyez le bandersnatch frumieux.
It's curious that it didn't understand "'twas" for the French translation, but apparently did for the German one. (My German is almost nonexistent, though.)
Yes there are a number of models, but since the quality of the results depends a lot on the training data as well, which I wouldn't know how to find or evaluate, and possibly might require tweaking algorithms for different languages, which I wouldn't know how to do, it's not really 'usable' (for me).
I figured someone would have gone to the trouble of combining models with a maintained collection of datasets to produce an open source alternative to Google Translate by now. I've been wondering that for years and it never seems to happen. Not saying anyone should feel obligated - I'm just curious why we don't see this, when we see so many other open source software projects that are competetive with their commercial alternatives.
Is it difficult/expensive to acquire these datasets? Is it a lot of effort to actually fine-tune the algorithms to reach passable results?
It seems (without knowing the details myself) that the state of the art in actually usable machine translation tools is always locked up in commercial IP, even though it feels (at least to me) like something that should be a free public service and therefore an ideal candidate for the 'open source' treatment.
I should have mentioned, it's state-of-the-art on open datasets. It's not comparable with DeepL or Google Translate which have their own proprietary datasets. Also, Translation models are very big (gigabytes).
"The translations are really accurate in context and sound very natural, noticeably more than Google Translate or Microsoft translator..." gets the "sound" wrongly translated to French (et son) but correct in Spanish (y suenan ...)
I probably should have checked the drop down of supported languages before typing Japanese. :)
I was a bit shocked at first when it was classified as German and then French.
"DeepL also intends to release an API in the coming months, allowing its superior translations to enhance other products such as digital assistants, dictionaries, language learning apps, and professional translation programs." from https://www.deepl.com/press.html
"Ruh" is the German name for Roo from the Winnie-the-Pooh stories (Kanga & Roo → Kängu & Ruh), so I guess someone gave the network some A. A. Milne to read.
I wonder if you could find more artifacts like this to find the training data they used.
DeepL performs significantly better on the most difficult texts I've given it. It's substantially better with colloquial language, and—oddly—nautical language. It also seems to be much better at tracking relationships between words in longer sentences. Good work!