Bug #17554
lint_po bails out when fed locales with an "@" e.g. ru@petr1708
100%
Description
Syncing PO/POT files from Transifex for Whisperback leads to simply two updates:
kibi@armor:~/work/clients/tails/whisperback.git$ git diff --stat
po/fr.po | 4 ++--
po/hu.po | 72 ++++++++++++++++++++++++++++++++++++++++++------------------------------
2 files changed, 44 insertions(+), 32 deletions(-)
but lint_po
is quite unhappy about the language suffix:
kibi@armor:~/work/clients/tails/whisperback.git$ "${RELEASE_CHECKOUT:?}"/submodules/jenkins-tools/slaves/lint_po
multiprocessing.pool.RemoteTraceback:
"""
Traceback (most recent call last):
File "/usr/lib/python3.5/multiprocessing/pool.py", line 119, in worker
result = (True, func(*args, **kwds))
File "/usr/lib/python3.5/multiprocessing/pool.py", line 44, in mapstar
return list(map(*args))
File "/home/kibi/work/clients/tails/release/release-checkout/submodules/jenkins-tools/slaves/lint_po", line 231, in check_po_file
issues = poFile.i18nspector(my_env)
File "/home/kibi/work/clients/tails/release/release-checkout/submodules/jenkins-tools/slaves/lint_po", line 186, in i18nspector
self.lang_without_script(), self.fname]
File "/home/kibi/work/clients/tails/release/release-checkout/submodules/jenkins-tools/slaves/lint_po", line 118, in lang_without_script
lang = self.lang()
File "/home/kibi/work/clients/tails/release/release-checkout/submodules/jenkins-tools/slaves/lint_po", line 113, in lang
raise NoLanguageError(self.fname)
NoLanguageError: Can't detect expect file suffix .XX.po for 'po/ru@petr1708.po'.
"""
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/home/kibi/work/clients/tails/release/release-checkout/submodules/jenkins-tools/slaves/lint_po", line 351, in <module>
main(logging.getLogger())
File "/home/kibi/work/clients/tails/release/release-checkout/submodules/jenkins-tools/slaves/lint_po", line 334, in main
files, 10):
File "/usr/lib/python3.5/multiprocessing/pool.py", line 314, in <genexpr>
return (item for chunk in result for item in chunk)
File "/usr/lib/python3.5/multiprocessing/pool.py", line 695, in next
raise value
__main__.NoLanguageError: Can't detect expect file suffix .XX.po for 'po/ru@petr1708.po'.
That doesn’t seem to be quite new, the only recent fixes (going back to June 2019) seem to be about Python versions instead.
(BTW: s/expect/expected/)
So I’ve decided to proceed with the rest of the release process for that component, but I suppose we should fix the jenkins-tools
submodule at some point to avoid further loss of time in the future?
Subtasks
History
#1 Updated by intrigeri 2020-03-26 08:39:41
- blocks Feature #16209: Core work: Foundations Team added
#2 Updated by intrigeri 2020-03-26 08:40:39
- Subject changed from lint_po vs. whisperback vs. ru@petr1708 to lint_po bails out when fed locales with an "@" e.g. ru@petr1708
- Category set to Internationalization
#3 Updated by intrigeri 2020-03-26 08:41:00
- Assignee set to hefee
Hi @hefee, could you please take a look?
For the curious, “petr1708” refers to:
- https://en.wikipedia.org/wiki/Reforms_of_Russian_orthography#18th-century_changes
- https://en.wikipedia.org/wiki/Civil_Script
This locale is unlikely to ever be made available via our Welcome Screen to Tails users, but I see a bunch of other locales that have an at-sign in their name, so perhaps it’s worth supporting this case?
Alternatively, we could filter those locales out when we import files from Transifex (import-translations
script).
#4 Updated by hefee 2020-03-26 17:05:19
- Status changed from Confirmed to Needs Validation
- Assignee changed from hefee to intrigeri
intrigeri wrote:
> This locale is unlikely to ever be made available via our Welcome Screen to Tails users, but I see a bunch of other locales that have an at-sign in their name, so perhaps it’s worth supporting this case?
As the “normal” po files in Tails had no `@` sign, I ignored this case, when writing the script.
The manpage of i18nspector tells us, that is only likes only `language_territory`. I scanned through the code of `lint_po` and there are only two places to add the support this. `lang()` - should return the language with all modifiers. `lang_without_script()` - shoul return the language code like `i18nspector` likes it.
I post a diff, as creating a branch etc. for a submodule takes more time and needs interaction of you anyways.
<code class="diff">
diff --git a/slaves/lint_po b/slaves/lint_po
index 9cc6432..c7db193 100755
--- a/slaves/lint_po
+++ b/slaves/lint_po
@@ -108,7 +108,7 @@ class PoFile:
"""@returns: language of filename, possibly ending with an underscore
followed by the territory or script"""
name = os.path.basename(self.fname)
- m = re.match(r"^(?:[^.].*\.)?(?P<lang>[A-Za-z_]+)\.po$", name)
+ m = re.match(r"^(?:[^.].*\.)?(?P<lang>[A-Za-z0-9_@]+)\.po$", name)
if not m:
raise NoLanguageError(self.fname)
return m.group("lang")
@@ -119,7 +119,7 @@ class PoFile:
# a script suffix (e.g. "_Latn") starts with an underscore,
# followed by the script name in title case, which we
# approximate as an upper case letter followed by a lower case one
- m = re.match(r"(?P<lang_without_script>.*)_[A-Z][a-z][A-Za-z]*$", lang)
+ m = re.match(r"^(?P<lang_without_script>.*?)(_[A-Z][a-z][A-Za-z]*)?(@[A-Za-z0-9]+)?$", lang)
if m:
return m.group("lang_without_script")
else:
diff --git a/slaves/test/testLintPo.py b/slaves/test/testLintPo.py
index 3ed3ead..bba5455 100644
--- a/slaves/test/testLintPo.py
+++ b/slaves/test/testLintPo.py
@@ -97,6 +97,7 @@ class TestCheckPo(unittest.TestCase):
self.assertEqual(lint_po.PoFile("index.de.po").lang(), "de")
self.assertEqual(lint_po.PoFile("x/a/a.fb.xx.po").lang(), "xx")
self.assertEqual(lint_po.PoFile("a.po").lang(), "a")
+ self.assertEqual(lint_po.PoFile("ru@petr1708.po").lang(), "ru@petr1708")
_p = lint_po.PoFile(".de.po")
with self.assertRaises(lint_po.NoLanguageError, msg=_p.fname) as e:
@@ -117,6 +118,8 @@ class TestCheckPo(unittest.TestCase):
self.assertEqual(lint_po.PoFile("index.de_DE.po").lang_without_script(), "de_DE")
self.assertEqual(lint_po.PoFile("index.sr_Latn.po").lang_without_script(), "sr")
self.assertEqual(lint_po.PoFile("index.sr_LAtn.po").lang_without_script(), "sr_LAtn")
+ self.assertEqual(lint_po.PoFile("ru@petr1708.po").lang_without_script(), "ru")
+ self.assertEqual(lint_po.PoFile("ru_RU@petr1708.po").lang_without_script(), "ru_RU")
def test_needs_rewrap(self):
with lint_po.pofile_readonly(os.path.join(DIRNAME, "checkPo/length")) as poFile:
</code>
#5 Updated by intrigeri 2020-03-27 10:25:37
- Target version set to Tails_4.6
#6 Updated by intrigeri 2020-04-12 12:48:20
- Status changed from Needs Validation to Resolved
- % Done changed from 0 to 100
Applied in changeset commit:tails|20ab48eecaa45bdb36757ce078c40f2633c455be.