Tails

#1 Updated by BitingBird 2014-03-23 17:04:35

• Status changed from New to Confirmed

#2 Updated by intrigeri 2014-03-23 18:41:58

• Assignee set to intrigeri
• Type of work changed from Website to Code

That’s a bug in the ikiwiki PO plugin. I’ll take care of it, but will treat it as low-priority personally. Don’t hesitate being faster than me :)

#3 Updated by intrigeri 2014-04-05 10:59:01

• Subject changed from Problem with the "news" titles on Frensh version, and partly in :de and :pt to Recurring problems with the "news" titles on non-English homepage

#4 Updated by intrigeri 2014-04-21 07:44:47

• has duplicate Bug #7113: News titles are truncated on https://tails.boum.org/index.fr.html added

#5 Updated by sajolida 2014-10-24 15:16:36

• Subject changed from Recurring problems with the "news" titles on non-English homepage to Truncated news titles on non-English homepage

#6 Updated by intrigeri 2015-10-03 02:55:06

• related to Bug #9671: rss and atom feed of l10n news show po file source added

#7 Updated by intrigeri 2015-10-03 02:57:46

• has duplicate Bug #9564: translated file doesn't get parsed in ikiwiki added

#8 Updated by intrigeri 2015-10-03 03:04:27

• Subject changed from Truncated news titles on non-English homepage to ikiwiki po plugin does not play well with inline directives
• Description updated
• Assignee deleted (intrigeri)

Making this ticket (which is the oldest one we have about that ikiwiki bug) track the root cause of this problem. We have other tickets that track what looks like various symptoms of that bug (not sure what to do with them but people won’t stop opening new ones anyway).

This bug is an old one, we’ve been seeing it from time to sime since years. And since years, I’ve tried to find a reliable reproducers for this bug a few times, but never managed to find one. One should instrument ikiwiki to debug it properly.

I doubt I’ll have time to get to it any time soon. I think this ticket should have higher priority, though.

#9 Updated by intrigeri 2015-10-03 03:12:43

• Description updated

#10 Updated by sajolida 2015-10-06 09:49:38

• related to Bug #9279: ISO file size unit is not translatable on the download page added

#11 Updated by sajolida 2015-10-06 09:49:49

• related to Bug #10124: po_translatable_pages is unstable on boum.org added

#15 Updated by intrigeri 2015-11-01 10:19:44

• Target version set to Hole in the Roof

sajolida wrote:
> Maybe Hole in the Roof?

Totally qualifies, good catch.

#17 Updated by sajolida 2015-11-02 13:28:21

• Assignee set to sajolida

This is not assigned to anybody, so I’ll take it and report it upstream with instructions to reproduce it. But my Perl skills will end up here :)

#20 Updated by sajolida 2016-03-30 06:51:44

• Assignee deleted (sajolida)

I’m aware of this and that’s why I’m removing wiki/src/.ikiwiki each time (for me that’s the “ikiwiki state directory”) and I really think I did that last time as well… I’ll stay on the lookout for more occurrence of this but with no promise.

#21 Updated by anonym 2017-02-03 20:52:19

• has duplicate Bug #12212: French version of https://tails.boum.org/news/ is broken added

#22 Updated by xin 2017-02-17 11:29:39

• related to Bug #12246: Untranslatable parts of documentation due to inline errors added

#23 Updated by intrigeri 2017-12-24 08:58:02

• has duplicate Bug #15032: Buggy “News” entries for non-English languages added

#24 Updated by sajolida 2018-04-11 13:32:34

• File broken-inline.png added
• Description updated

#26 Updated by sajolida 2018-04-11 13:36:32

• related to deleted (Bug #12246: Untranslatable parts of documentation due to inline errors)

#27 Updated by sajolida 2018-04-11 13:36:39

• has duplicate Bug #12246: Untranslatable parts of documentation due to inline errors added

#30 Updated by sajolida 2018-04-20 09:27:04

• blocks Feature #15411: Core work 2018Q2 → 2018Q3: Technical writing added

#31 Updated by sajolida 2018-04-20 09:28:25

• File ikiwiki-inline.zip added

Thanks a lot xin and Emma, I think this is a major finding!!!

https://tails.boum.org/install/mac/usb/index.es.html and https://tails.boum.org/install/mac/usb/index.fr.html are always broken on step 6/7. I tried rebuilding the website completetly and the problem occurs reliably (rm wiki/src/.ikiwiki and rm -r my destination folder).

The structure of the inlines on this page is as follow:

• /install/mac/dvd
  • /install/inc/steps/restart_first_time.inline
    • /install/inc/steps/mac_startup_disks.inline
    • /install/inc/steps/not_at_all.inline
  • /install/inc/steps/install_final.inline
  • /install/inc/steps/restart_second_time.inline
    • /install/inc/steps/mac_startup_disks.inline
    • /install/inc/steps/not_at_all.inline
  • /install/inc/steps/create_persistence.inline

The inline that is broken on step 6/7 is mac_startup_disks.inline which is otherwise inlined correctly in step 4/7 (in English “Immediately press-and-hold”, in Spanish “Inmediatamente aprieta-y-mantiene”).

To go further I wrote a minimal ikiwiki repo that reproduces the problem every time. I’m putting the archive in attachment.

intrigeri: Can you confirm that we have a reproducer now and tell us what would be the next steps?

It might not be the same bug as always (as sometimes rebuilding the wiki fixes the issue) but at least now we have something clear to work on.

#32 Updated by sajolida 2018-04-20 09:28:45

• Assignee set to intrigeri
• QA Check set to Info Needed

#34 Updated by intrigeri 2018-04-27 09:08:30

• QA Check changed from Info Needed to Dev Needed

#35 Updated by intrigeri 2018-04-27 09:08:42

• blocks Feature #15139: Core work 2018Q2: Foundations Team added

#38 Updated by lamby 2018-05-28 14:30:55

• Assignee changed from intrigeri to lamby
• Estimated time set to 4 h

Self-assigning ticket during meeting on tails-meeting with nod from intrigeri. Adding 4 hours to estimate/max-time as discussed.

#39 Updated by intrigeri 2018-05-28 15:33:35

• Target version changed from Hole in the Roof to Tails_3.8

#40 Updated by intrigeri 2018-06-26 16:27:38

• Target version changed from Tails_3.8 to Tails_3.9

#42 Updated by intrigeri 2018-06-28 14:30:32

• blocked by deleted (Feature #15139: Core work 2018Q2: Foundations Team)

#43 Updated by intrigeri 2018-06-28 14:30:41

• blocks Feature #15334: Core work 2018Q3: Foundations Team added

#44 Updated by lamby 2018-06-28 18:56:40

• Assignee changed from lamby to intrigeri
• % Done changed from 0 to 100
• QA Check changed from Dev Needed to Ready for QA

Fixed, patch sent upstream: https://ikiwiki.info/todo/Re-use_translated_content_instead_of_skipping_if_previously_translated/

#45 Updated by intrigeri 2018-07-06 17:00:34

• Status changed from Confirmed to In Progress
• % Done changed from 100 to 80
• QA Check deleted (Ready for QA)
• Type of work changed from Code to Communicate

lamby wrote:
> Fixed, patch sent upstream: https://ikiwiki.info/todo/Re-use_translated_content_instead_of_skipping_if_previously_translated/

Excellent! Reviewed and approved there (I created the bug in the first place, mind you :)

I’ll handle the next steps i.e. ping upstream if needed and ensure this gets deployed on our production website (possibly without waiting for the upstream release and package update, no idea which version boum.org runs in production).

#46 Updated by intrigeri 2018-08-04 01:14:08

• Target version changed from Tails_3.9 to Tails_3.10.1

I’ll ping upstream in ~1 month, early in the 3.10 cycle.

#48 Updated by intrigeri 2018-10-08 13:58:51

• blocked by deleted (Feature #15334: Core work 2018Q3: Foundations Team)

#49 Updated by intrigeri 2018-10-08 13:58:54

• blocks Feature #15506: Core work 2018Q4: Foundations Team added

#50 Updated by intrigeri 2018-10-12 20:38:11

• Assignee changed from intrigeri to lamby
• Target version changed from Tails_3.10.1 to Tails_3.11

Actually, if you don’t mind, I’d rather see you track this and keep pinging upstream: you’re doing it just as well, if not better, than I would :) Feel free to batch this with similar tasks that are now mostly blocked on pinging upstream, this will make it slightly easier to clock non-ridiculous amounts of time.

Last ping was on Aug 24 so postponing to 3.11.

#51 Updated by intrigeri 2018-10-12 20:38:22

• related to Feature #14588: Self-host our website added

#54 Updated by lamby 2018-10-19 19:20:44

• Assignee changed from lamby to intrigeri

In https://bugs.debian.org/911356#17:

If you can put together a regression test for this bug that renders a
translated page and inspects the HTML output (t/img.t, t/meta.t, or a
combination of t/po.t and t/render.t might be a good basis) that would
make me a lot more confident about accepting patches. At the moment I
don't think we have any "full stack" test coverage that actually
renders HTML from a translated page.

I believe would need a few hours assigned to run with this all the way through so assigning over to you for approval.

#57 Updated by intrigeri 2018-11-01 11:06:50

• Assignee changed from intrigeri to lamby
• Estimated time changed from 4 h to 6 h

#58 Updated by intrigeri 2018-11-01 11:07:02

• Type of work changed from Communicate to Code

#60 Updated by sajolida 2018-11-18 17:25:41

• blocked by deleted (Feature #15411: Core work 2018Q2 → 2018Q3: Technical writing)

#62 Updated by lamby 2018-11-22 14:16:10

• Assignee changed from lamby to intrigeri

I just had another run at this but, alas, I did not manage to get a test harness working as mentioned above. I fear I have reached my “approved” hours here, so I this is a soft-request for additional time to dedicate to this. I wonder if we should follow the testing methodology in po.t instead instead calling perl directly, but I am unsure. Anyway, you thoughts and opiniion are always welcome of course.

#63 Updated by intrigeri 2018-11-30 14:00:00

• Assignee changed from intrigeri to lamby
• Estimated time changed from 6 h to 8 h

Sorry for the delay!

lamby wrote:
> There is something fundamentally at issue with the way we are doing the test though in that it one cannot translate properly when we call via $perl$ with a getcwd for some reason. It is basically generating “real” ikiwiki pages for the PO files instead of translating. If I use the system-installed ikiwiki it works just fine, even with the same ikiwiki.setup.
>
> Any ideas?

I’ve tried hefee’s test with git clean -fdx && perl Makefile.PL && prove t/po-integration.t and it worked just fine. Then I’ve passed CLEANUP => 0 to tempdir and confirmed that the generated PO (in the in directory) and HTML (in the out directory) have the correct content. So it seems that this can possibly work. To understand better what’s going on on your system I would suggest two things:

• Try the test procedure I’ve used from hefee’s branch. If you still see weird behaviour, unset PERL_MM_OPT (in the past I’ve had issues with it when doing this sort of ikiwiki work) and retry. If that still fails, retry in a clean chroot or VM with a clean environment, to eliminate any possibility that your local system is tainted in a way that breaks this.
• Share the current state of your work so I can try to reproduce the issue.

Adding 2 hours to the budget for this debugging and whatever next steps we’ll come up with after that.

#64 Updated by intrigeri 2018-11-30 14:06:10

• Estimated time changed from 8 h to 4 h

(Removing what’s already been accounted for in Q2.)

#65 Updated by intrigeri 2018-12-03 15:22:30

• Target version changed from Tails_3.11 to Tails_3.12

#69 Updated by lamby 2018-12-24 15:16:55

• Assignee changed from lamby to intrigeri

> Can you please take a look at Simon’s questions and reassign to me if you feel I specifically am needed?

I think addressing smcv’s concerns etc regarding the alreadyfiltered method is the first step before we try and reapproach the problem we are seeing and I not only think you will be more efficient at answering his questions as you wrote that code I share some of Simon’s stated not-understandings of it.

#71 Updated by intrigeri 2019-01-04 10:19:18

• blocks Feature #15507: Core work 2019Q1: Foundations Team added

#72 Updated by intrigeri 2019-01-04 10:19:21

• blocked by deleted (Feature #15506: Core work 2018Q4: Foundations Team)

#73 Updated by intrigeri 2019-01-04 15:12:23

• Target version changed from Tails_3.12 to Tails_3.13

#74 Updated by Anonymous 2019-01-17 10:55:39

• related to Feature #16366: Teach po4a to ignore the [[!meta directives that shall not be translated added

#75 Updated by Anonymous 2019-01-17 10:55:43

• related to deleted (Feature #16366: Teach po4a to ignore the [[!meta directives that shall not be translated)

#76 Updated by intrigeri 2019-01-19 13:35:37

• Estimated time changed from 4 h to 2 h

Removing what’s been accounted for in 2018Q4.

#77 Updated by intrigeri 2019-02-06 14:34:51

• related to Bug #16185: Please deploy ikiwiki po plugin fix from ticket #6907 added

#78 Updated by intrigeri 2019-02-06 14:35:03

• Priority changed from Normal to Elevated

#82 Updated by intrigeri 2019-02-18 17:19:39

• Target version changed from Tails_3.13 to Tails_3.14
• % Done changed from 100 to 70
• Type of work changed from Code to Wait

intrigeri wrote:
> Replied on the Debian bug, deployed smcv’s proposed fix on our production website. Let’s see how it goes.

Worked fine so far, including 3.12.1 release (releases often trigger this bug) ⇒ asked upstream to merge their proposed patch.

#85 Updated by intrigeri 2019-03-01 07:34:53

• Category set to Infrastructure
• Target version changed from Tails_3.14 to Tails_3.13
• Type of work changed from Wait to Sysadmin

And the fix made it into 3.20190228-1. I’ll ensure we install it.

#86 Updated by intrigeri 2019-03-05 11:01:20

• Due date set to 2019-03-12

The 3.20170111.1 security update superseded our custom package on our webserver. Given the fixed version will migrate to Debian testing in a week, I won’t bother rebasing our custom package; instead, once the package has migrated, I’ll pin it to Buster.

#87 Updated by intrigeri 2019-03-07 08:32:25

• Assignee changed from intrigeri to groente
• % Done changed from 70 to 80
• QA Check set to Ready for QA

The tails::website::builder Puppet class now installs ikiwiki from Buster. I’ve pushed this, run Puppet and then apt upgrade ⇒ the correct version is installed on www.lizard. Refreshed the relevant pages of our website and they don’t expose the bug.

@groente, please review the Puppet bits and if happy, close as resolved :)

#88 Updated by groente 2019-03-19 10:54:48

• Status changed from In Progress to Resolved
• % Done changed from 80 to 100
• QA Check changed from Ready for QA to Pass