Wrong behavior with TranslateTextOptions::TAG_HANDLING xml and notranslate tag

[VincentLanglet]

Hi @daniel-jones-deepl,

I discover another bug with the TranslateTextOptions::TAG_HANDLING => xml (But different from #43 (comment))

Translating FR to ES

En seulement 4 mois, l’établissement a généré <strong wg-1="">30k€ de <notranslate>ingresos</notranslate></strong> et a enregistré <strong wg-2="">plus de 80 nouvelles réservations</strong> suite à l’envoi d’e-mails marketing personnalisés

En sólo 4 meses, el establecimiento generó <strong wg-1="">30k€ en <notranslate>ingresos</notranslate></strong> y registró <strong wg-2="">más de 80 nuevas reservas</strong> tras el envío de emails de marketing personalizados.

Translating FR to ES with TranslateTextOptions::TAG_HANDLING => xml

En seulement 4 mois, l’établissement a généré <strong wg-1="">30k€ de <notranslate>ingresos</notranslate></strong> et a enregistré <strong wg-2="">plus de 80 nouvelles réservations</strong> suite à l’envoi d’e-mails marketing personnalisés

<strong wg-1=""> <notranslate>ingresos</notranslate>En sólo 4 meses, el establecimiento generó</strong> <strong wg-1="">30.000 euros</strong> y registró <strong wg-1=""></strong> <strong wg-2="">más de 80 nuevas reservas</strong> tras el envío de correos electrónicos de marketing personalizados.

I would expect a result similar to the first one.

Could you confirm and forward the issue to your backend team ? Thanks.

[daniel-jones-dev]

HI @VincentLanglet, thanks for the report.
I was able to reproduce the problem by also setting ignore_tags to include notranslate tag. This is indeed a strange problem, some target languages (eg en-US) seem to handle it correctly, others have a similar problem. I've forwarded it to our backend team.

[myfbone]

Same problem with German to Polish language pair

[ricardoaraujo330]

Same here, please fix it asap 🙏

[osiris-mm]

Same thing happens in html using "don't translate" tags as per the documentation: <p translate="no">My first paragraph.</p>.

When these tags are present, translation becomes inconsistent. It removes blank spaces between words and doesn't seem to understand the context around the tagged words.

[remyberda]

Hi team, do you have some new on the tag handling problems?

[JanEbbing]

We released tag handling v2 a while ago, which fixes a lot of the issues with tag handling. The original example from this thread is fixed for me AFAICT:

$ curl -X POST https://api.deepl.com/v2/translate \
  -H "Content-Type: application/json" \
  -H "Authorization: DeepL-Auth-Key KEY" \
  -d '{"text": ["En seulement 4 mois, l'\''établissement a généré <strong wg-1=\"\">30k€ de <notranslate>ingresos</notranslate></strong> et a enregistré <strong wg-2=\"\">plus de 80 nouvelles réservations</strong> suite à l'\''envoi d'\''e-mails marketing personnalisés"],"target_lang": "ES",
  "tag_handling":"xml", "tag_handling_version":"v2", "source_lang":"FR"}'

{"translations":[{"text":"En solo cuatro meses, el establecimiento ha generado <strong wg-1=\"\">30 000 € en <notranslate>ingresos</notranslate></strong> y ha registrado <strong wg-2=\"\">más de 80 nuevas reservas</strong> tras el envío de correos electrónicos de marketing personalizados","detected_source_language":"FR","tag_handling_version":"v2"}]}

New tag handling users automatically get it, existing users need to opt in with the tag_handling_version flag. However, tag handling is a much more difficult problem than mere translation, so please send us any cases where it still fails :)

Hope this helps!

[VincentLanglet]

New tag handling users automatically get it, existing users need to opt in with the tag_handling_version flag. However, tag handling is a much more difficult problem than mere translation, so please send us any cases where it still fails :)

Hi @JanEbbing, thanks for your answer.
I just tried the version 2 with the following option

[
  TranslateTextOptions::TAG_HANDLING => 'xml',
  TranslateTextOptions::TAG_HANDLING_VERSION => 'v2',
  TranslateTextOptions::FORMALITY => 'default',
  TranslateTextOptions::MODEL_TYPE => 'prefer_quality_optimized',
]

and we're getting some unexpected translations changes.

1. Translation of one<br wg-1=""/>two changed

+ <br wg-10=""/>un-deux
- un<br wg-1=""/>deux

2. Something like

Hello <notranslate value="welcome &lt;notranslate value=&quot;welcome&quot;&gt;welcome&lt;/notranslate&gt; welcome">welcome <notranslate value="welcome">welcome</notranslate> welcome</notranslate>

is now translated as

+ Bonjour<notranslate value="welcome &lt;notranslate value=&quot;welcome&quot;&gt;welcome&lt;/notranslate&gt; welcome">, bienvenue<notranslate value="welcome">, bienvenue,</notranslate> bienvenue</notranslate>
- Bonjour, <notranslate value="welcome &lt;notranslate value=&quot;welcome&quot;&gt;welcome&lt;/notranslate&gt; welcome"> <notranslate value="welcome">bienvenue, bienvenue</notranslate>, bienvenue</notranslate>

which means that once notranslate tag are replaced by their original value, it becomes

+ Bonjourwelcome welcome welcome
- Bonjour, welcome welcome welcome

3. Kinda anecdotic but something like

<a wg-1=\"\">Portal<span wg-2=\"\"></span></a><a wg-3=\"\">So funktioniert&#x27;s<span wg-4=\"\"></span></a>

is not translated anymore on v2 while it is on v1.

NB: I also tried TAG_HANDLING 'html' + the v2 and I have

• The issue 1 with one<br wg-1="">two<br wg-2=""> three four having changed translation

+ un<br wg-1=""/>deux<br wg-2=""/> trois quatre
- un<br wg-1=""/>, deux<br wg-2=""/>, trois, quatre

and it would make no sens having a , just after a br in some html.

• The issue 2