Feature #17615

Make the Upgrader work its way around broken mirrors

Added by sajolida 2020-04-14 15:26:48 . Updated 2020-04-19 15:50:19 .

Status:
Confirmed
Priority:
Normal
Assignee:
Category:
Target version:
Start date:
Due date:
% Done:

0%

Feature Branch:
Type of work:
Code
Blueprint:

Starter:
Affected tool:
Upgrader
Deliverable for:

Description

Some implementation options from Feature #15875, such as “C + fallback DNS mirrors pool” would do the job wrt. a mirror that’s entirely broken (which was the initial motivation behind this issue) and possibly even wrt. a mirror that’s flaky.

Original idea

What happens currently when trying to download an IUK from a broken mirror? Could we do anything more to prevent broken downloads by testing the mirror first?

My Perl skills are very limited but, at first sight, I couldn’t spot anything in the code that does this :)

Could we try downloading https://mirrors.edge.kernel.org/tails/project/trace from the candidate mirror before deciding to use it? This relates to Feature #15875 but wouldn’t imply resuming the download of the IUK after it started.


Subtasks


Related issues

Related to Tails - Feature #15875: Make it possible to resume an IUK download from within Tails In Progress 2018-08-30

History

#1 Updated by intrigeri 2020-04-15 14:00:11

  • related to Feature #15875: Make it possible to resume an IUK download from within Tails added

#2 Updated by intrigeri 2020-04-15 14:09:07

> What happens currently when trying to download an IUK from a broken mirror?

I did not test this but I believe it falls into the broad category of “upgrade download failure”, that we’ve been working on via Feature #15875.

> Could we do anything more to prevent broken downloads by testing the mirror first?

> Could we try downloading https://mirrors.edge.kernel.org/tails/project/trace from the candidate mirror before deciding to use it? This relates to Feature #15875 but wouldn’t imply resuming the download of the IUK after it started.

What you’re suggesting is technically feasible. It also adds code complexity, which would be good to avoid, if possible.

At first glance, from an implementation perspective, I would prefer treating the “failed at byte 0” as a special case of “failed at byte N”, rather than implementing an additional mechanism to handle the “failed at byte 0” case.

That is, I would prefer to optimistically try the download, and when it fails, ensure one of the mechanisms proposed on Feature #15875 to resume from a different mirror handles it. For example, I believe “C + fallback DNS mirrors pool” would do the job, and at the same time take better care of flaky mirrors (i.e. those who would pass the pre-download test you’re proposing, but fail the download later). How I prioritize my work on Feature #15875 (among the various things I can add on top of what I implemented already) would be informed by this very issue.

How does this sound?

#3 Updated by sajolida 2020-04-16 21:20:32

Sure! From Feature #15875#note-13, I understand that the 1st increment is “©”:
retry from the same mirror.

I create this one as a separate ticket because it could be worked on in
parallel or before this 1st increment. I thought it might be simpler and
maybe even worth it if the first code goes to the trash once we have
“(C + new mirror)”.

This said, I’ll let the cost analysis and software engineering decisions
up to you :)

#4 Updated by intrigeri 2020-04-19 15:45:56

> From Feature #15875#note-13, I understand that the 1st increment is “©”: retry from the same mirror.

Correct.

> I create this one as a separate ticket because it could be worked on in parallel or before this 1st increment.

I see. tl;dr: not going to happen unless someone else than me suddenly feels like writing Modern Perl.

(That 1st iteration has been implemented months ago.
All that’s currently blocking me from shipping it is: time to do a full-blown integration test.
I’m trying to decrease my amount of WIP so I’d rather not start another project in the same area of that code base before, or in parallel to, completing this.)

#5 Updated by intrigeri 2020-04-19 15:50:19

  • Subject changed from Test the download mirror before using it on automatic upgrades to Make the Upgrader work its way around broken mirrors
  • Description updated
  • Status changed from New to Confirmed