Feature #12185

Update Debian package to run Dogtail using Python 3

Added by anonym 2017-01-28 10:38:45 . Updated 2019-08-13 07:26:08 .

Status:
Resolved
Priority:
Normal
Assignee:
Category:
Test suite
Target version:
Tails_3.16
Start date:
2017-01-28
Due date:
% Done:

0%

Feature Branch:
Type of work:
Debian
Blueprint:

Starter:
Affected tool:
Deliverable for:

Description

Currently we install python-dogtail, which uses Python 2 and makes it impossible to deal with objects containing unicode character (e.g. “Save As…” (note the triple dots)).

The Debian packaging repo for dogtail has switched to Python 3 since almost a year back, but there has been no release since then. However, I tried building a package and installing it, and it worked fine vs unicode strings at least.

Part of the “Make test suite survive tech changes” roadmap item.

Subtasks

Related issues

Blocked by Tails - Feature #14585: Investigate Dogtail's long-term viability Resolved 2017-09-02
Blocks Tails - Bug #15953: Make our test suite survive changes in the surrounding environment Resolved 2018-09-14
Blocks Tails - Bug #15396: Core work: Debian Confirmed 2019-02-08
Blocks Tails - Bug #16976: Run Dogtail using Python 3 Resolved

History

#1 Updated by intrigeri 2017-01-29 11:12:42

  • Priority changed from Normal to Low

(Indicating that this is not a blocker for 3.0. If it doesn’t make it into 3.0, then it should go back to priority Normal.)

#2 Updated by intrigeri 2017-05-20 07:15:49

  • Priority changed from Low to Normal
  • Target version changed from Tails_3.0 to Tails_3.2

I think we’ll be plenty busy with other matters during this cycle.

#3 Updated by anonym 2017-05-30 10:40:24

  • Subject changed from Generate Python 3 code for Dogtail to Run Dogtail using Python 3
  • Description updated

Please see the updated description. With that as background, I guess we might want to consider building our own python3-dogtail from version 0.9.9-1, until we can make Debian release it for stretch backports.

#4 Updated by spriver 2017-05-30 11:04:04

  • blocks Feature #8788: Automatically test the "Tails documentation" link on the Desktop added

#5 Updated by anonym 2017-06-29 13:33:13

  • blocks Feature #13239: Core work 2017Q3: Test suite maintenance added

#6 Updated by intrigeri 2017-09-02 10:42:54

Looking at your 3.2 plate, I suggest you postpone this to another major release.

#7 Updated by intrigeri 2017-09-07 06:35:27

  • Target version changed from Tails_3.2 to Tails_3.3

#8 Updated by intrigeri 2017-09-14 17:24:56

  • blocked by Feature #14585: Investigate Dogtail's long-term viability added

#9 Updated by intrigeri 2017-09-14 17:25:28

Please check Feature #14585 before investing lots more time into dogtail.

#10 Updated by intrigeri 2017-10-01 09:57:22

  • blocks Feature #13240: Core work 2017Q4: Test suite maintenance added

#11 Updated by intrigeri 2017-10-01 09:57:26

  • blocked by deleted (Feature #13239: Core work 2017Q3: Test suite maintenance)

#12 Updated by intrigeri 2017-11-06 15:43:12

  • Target version changed from Tails_3.3 to Tails_3.5

#13 Updated by intrigeri 2017-12-07 12:44:17

  • Target version deleted (Tails_3.5)

#14 Updated by intrigeri 2018-01-01 16:58:31

  • blocked by deleted (Feature #13240: Core work 2017Q4: Test suite maintenance)

#15 Updated by Anonymous 2018-01-16 12:01:26

If this is still needed, I suggest instead to ask Debian packagers to provide such a package. Outsource!

#16 Updated by anonym 2018-01-16 12:21:05

  • Assignee deleted (anonym)
  • QA Check set to Info Needed

u wrote:
> If this is still needed

It is! The limitations with unicode has already required a couple ugly workarounds.

> I suggest instead to ask Debian packagers to provide such a package. Outsource!

Absolutely! Can I outsource this outsourcing to you? :P Correct me if I am wrong, but my current understanding is that your role includes interfacing with Debian for things like this (?).

#17 Updated by Anonymous 2018-01-16 13:38:01

  • QA Check deleted (Info Needed)

Sure. I’ll take care of it.

#18 Updated by Anonymous 2018-01-16 13:38:12

  • Target version set to Tails_3.6

#19 Updated by bertagaz 2018-03-14 11:32:13

  • Target version changed from Tails_3.6 to Tails_3.7

#20 Updated by bertagaz 2018-05-10 11:09:18

  • Target version changed from Tails_3.7 to Tails_3.8

#21 Updated by Anonymous 2018-05-28 12:54:16

@anonym, are you aware that Dogtail is missing a maintainer in Debian? See https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=777382

This not only means that we cannot ask any Debian maintainer to put out a new version, but we could theoretically commit to updating this package ourselves. As it is not actively developed, this could be a one-timer.

The tracker page currently shows no VCS information: https://tracker.debian.org/pkg/dogtail.

Upstream code seems to be living here nowadays: https://gitlab.com/dogtail/dogtail I could still ask the question there..

#22 Updated by intrigeri 2018-05-28 15:18:11

u wrote:
> Upstream code seems to be living here nowadays: https://gitlab.com/dogtail/dogtail I could still ask the question there..

I don’t think it’s worth spending any more time on this before Feature #14585 is done so feel free to reassign to anonym. It’s not urgent anyway.

#23 Updated by intrigeri 2018-06-26 16:27:55

  • Target version changed from Tails_3.8 to Tails_3.9

#24 Updated by Anonymous 2018-09-03 17:46:02

  • Assignee set to anonym
  • Target version changed from Tails_3.9 to Tails_3.11

#26 Updated by intrigeri 2018-09-03 18:11:00

  • Assignee changed from anonym to intrigeri

#27 Updated by intrigeri 2018-09-14 07:55:44

  • Description updated
  • Target version changed from Tails_3.11 to 2019

#28 Updated by intrigeri 2018-09-14 08:00:19

  • blocks Bug #15953: Make our test suite survive changes in the surrounding environment added

#29 Updated by intrigeri 2018-09-18 16:11:12

  • 0.9.9 was uploaded to Debian but 0.9.10 was not.
  • There’s still no python3-dogtail in Debian.

#30 Updated by intrigeri 2018-09-18 16:16:21

If we want to make this happen, at this point I think we need to start maintaining the package in Debian ourselves. At this point, that’s a useful data point for Feature #14585, not an action item.

#31 Updated by intrigeri 2018-09-18 16:29:35

  • Assignee deleted (intrigeri)
  • Target version changed from 2019 to Tails_3.10.1
  • QA Check set to Info Needed

I’ve reached a conclusion on Feature #14585 and I think we should start contributing to the maintenance of Dogtail in Debian.

First action items would be:

  1. Ensure the RC bug is forwarded upstream and its patch becomes an upstream MR.
  2. Add a python3-dogtail binary package and verify it fixes https://bugs.debian.org/728658. Once we have a WIP package I’m happy to test this in our test suite.
  3. Package upstream 0.9.10.
  4. If needed, clean up and modernize the packaging, but don’t aim for perfection here and leave the package orphaned for now (it’s unclear whether we’ll keep using Dogtail on the long-term).

My questions for today are:

  • Would this fit in your Debian maintenance budget?
  • Any chance this is done in time for the Buster freeze?

#32 Updated by intrigeri 2018-10-24 17:03:38

  • Target version changed from Tails_3.10.1 to Tails_3.11

#33 Updated by CyrilBrulebois 2018-12-16 14:10:50

  • Target version changed from Tails_3.11 to Tails_3.12

#34 Updated by intrigeri 2019-01-28 15:15:24

intrigeri wrote:
> My questions for today are:
>
> * Would this fit in your Debian maintenance budget?

I’m still interested in knowing the answer, even if this can’t happen in time for the Buster freeze: if the answer is no, then someone else should do it on the test suite maintenance budget.

> * Any chance this is done in time for the Buster freeze?

FTR this would require the package to be uploaded and accepted by ftp-masters (it has to through NEW since it adds a new binary package) by Feb 5 so it migrates to testing by Feb 11 (no new packages accepted once the soft freeze starts).
I’ll be on holiday myself until Feb 4 so it’s too late for me to take over this task. So if you can’t do it within this timeframe, feel free to postpone as you see fit.

#35 Updated by Anonymous 2019-01-28 15:49:32

Interestingly it seems I did not see updates of this ticket since I had reassigned it to anonym four months ago (even though it was reassigned to me the updates did go unnoticed for me). So sorry, I did also not see your question. Yes, this can fit in the budget, and I can try to make it happen before Feb 5. But if it does not work out, I’ll postpone it.

Please do not hesitate to send me a personal email if you have a specific question on some kind of urgent matter next time.

#36 Updated by Anonymous 2019-01-28 15:49:51

  • QA Check deleted (Info Needed)

#37 Updated by Anonymous 2019-01-28 15:53:41

intrigeri wrote:
> Might be fixed by https://gitlab.com/dogtail/dogtail/merge_requests/10

this looks like it was merged into master upstream.

#38 Updated by Anonymous 2019-01-28 16:04:12

intrigeri wrote:
> I’ve reached a conclusion on Feature #14585 and I think we should start contributing to the maintenance of Dogtail in Debian.
>
> First action items would be:
>
> # Ensure the RC bug is forwarded upstream and its patch becomes an upstream MR.

I’m not sure what RC bug you are talking about.
Also, are you referring to this merge request https://gitlab.com/dogtail/dogtail/merge_requests/10? that was merged in the meantime?

#39 Updated by intrigeri 2019-01-29 14:59:00

> Interestingly it seems I did not see updates of this ticket since I had reassigned it to anonym four months ago. So sorry, I did also not see your question.

Ouch, I did this mistake again! I try to always first add someone as a watcher before reassigning to them (as IIRC the comment that goes with the reassignment operation is not sent to the person the ticket is reassigned to, crappy UX, yeah) and this time I forgot.

> Yes, this can fit in the budget, and I can try to make it happen before Feb 5. But if it does not work out, I’ll postpone it.

OK, thanks!

> Please do not hesitate to send me a personal email if you have a specific question on some kind of urgent matter next time.

It’s not that I would hesitate, it’s rather that from my perspective, I had already put this on your radar, by assigning to you with a “Target version” not far in the future, regardless of the missed comment issue; then this was not on my own radar anymore and I simply forgot about it (I’m not managing this work and I have nothing set up to track it). But I’ve noticed recently that this way of putting things on someone’s radar does not work well with many of our co-workers, simply because not everyone uses Redmine the same way. Point taken, I’ll try to fix my assumptions.

#40 Updated by intrigeri 2019-01-29 15:08:46

>> Ensure the RC bug is forwarded upstream and its patch becomes an upstream MR.

> I’m not sure what RC bug you are talking about.

https://bugs.debian.org/860523 (found by looking at the bugs closed by the only upload done since I wrote that comment). It’s fixed in Debian but I don’t know if the fix made it upstream.

> Also, are you referring to this merge request https://gitlab.com/dogtail/dogtail/merge_requests/10? that was merged in the meantime?

I have definitely referred to this MR in the context of the char encoding support problem that made us file this ticket (similar to https://bugs.debian.org/728658).
This MR may be part of the solution. It’s not clear to me which one(s) of these steps is needed or sufficient to fix the problem:

  • switching to Python 3
  • applying that upstream merge request (be it as a Debian patch or by packaging an upstream release that includes it, if any)

#41 Updated by anonym 2019-01-30 11:59:15

  • Target version changed from Tails_3.12 to Tails_3.13

#42 Updated by Anonymous 2019-02-08 10:02:42

  • Subject changed from Run Dogtail using Python 3 to Update Debian package to run Dogtail using Python 3
  • Parent task set to Bug #15396
  • Type of work changed from Code to Debian

#43 Updated by Anonymous 2019-02-08 18:31:29

I’m still on it, hopefully next week.

#44 Updated by intrigeri 2019-02-08 19:11:52

> I’m still on it, hopefully next week.

tl;dr: don’t worry about it, better focus on things that can still make it into Buster.

Unless I’m mistaken, it’s too late to fix this in Buster: it’s not possible anymore that a package goes through NEW and then migrates to testing by Feb 12 (see Feature #12185#note-34 for details). Therefore, there’s no rush anymore: as I wrote there, “if you can’t do it within this timeframe, feel free to postpone as you see fit”.

So I think the target is now buster-backports. This would help us get rid of some dirty hacks in our test suite. If we can have it there mid-2019, it would be awesome: we could install it in feature/buster and benefit from it in Tails 4.0 :) And if this has to wait a bit more, fine: this problem has been around for 2 years, no reason to pretend it’s an emergency now that the Buster deadline is behind us.

#45 Updated by intrigeri 2019-02-15 07:59:33

#46 Updated by intrigeri 2019-02-15 07:59:42

#47 Updated by Anonymous 2019-03-18 16:04:17

  • Target version changed from Tails_3.13 to Tails_3.14

As it won’t make it into Buster, I’d rater focus on more urgent things now.

#48 Updated by CyrilBrulebois 2019-05-23 21:23:21

  • Target version changed from Tails_3.14 to Tails_3.15

#49 Updated by CyrilBrulebois 2019-07-10 10:33:58

  • Target version changed from Tails_3.15 to Tails_3.16

#50 Updated by intrigeri 2019-08-13 07:24:25

  • blocks Bug #16976: Run Dogtail using Python 3 added

#51 Updated by intrigeri 2019-08-13 07:26:08

  • Status changed from Confirmed to Resolved

intrigeri wrote:
> So I think the target is now buster-backports.

sid now has 0.9.11-2 with a python3-dogtail binary package and I see nothing that prevents us from installing it on Stretch or Buster, so I’m calling this done. Let’s not bother about backports until we actually need them :)

I’ve filed Bug #16976 to track the next steps (i.e. what this ticket was originally about).

#52 Updated by intrigeri 2019-08-13 07:28:35

  • blocked by deleted (Feature #8788: Automatically test the "Tails documentation" link on the Desktop)