Bug #11735

Tails-Upgrader "cannot choose a download server"

Added by goupille 2016-08-26 14:12:24 . Updated 2017-04-19 17:25:15 .

Status:
Resolved
Priority:
Elevated
Assignee:
Category:
Target version:
Start date:
2016-08-26
Due date:
% Done:

100%

Feature Branch:
bugfix/11735-mirror-pool-size, mirror-pool-dispatcher:bugfix/11735-mirror-pool-size
Type of work:
Code
Blueprint:

Starter:
Affected tool:
Deliverable for:

Description

there were multiple reports about tails-upgrader crashing with this error :

Error message stating "cannot chose a download server" "This should not happen. Please report a bug"

The following is copied from terminal:

Prototype mismatch: sub Tails::MirrorPool::assert ($;$) vs none at /usr/share/perl5/Tails/MirrorPool.pm line 28.
Prototype mismatch: sub Tails::IUK::Frontend::assert ($;$) vs none at /usr/share/perl5/Tails/IUK/Frontend.pm line 42.
This version of Tails is outdated, and may have security issues.
Upgrade available
<b>You should upgrade to Tails 2.5.</b>

For more information about this new version, go to https://tails.boum.org/news/version_2.5/

It is recommended to close all the open applications during the upgrade.
Downloading the upgrade might take a long time, from several minutes to a few hours.
The networking will be disabled after downloading the upgrade.

Download size: 324.66MiB

Do you want to upgrade now?: Upgrade now / Upgrade later

** (zenity:5402): WARNING **: Couldn't register with accessibility bus: Did not receive a reply. Possible causes include: the remote application did not send a reply, the message bus security policy blocked the reply, the reply timeout expired, or the network connection was broken.
Gtk-Message: GtkDialog mapped without a transient parent. This is discouraged.
Downloading the upgrade to Tails 2.5...

undefined:183
    },
    ^
SyntaxError: Unexpected token }
    at Object.parse (native)
    at Object.<anonymous> (/usr/local/bin/tails-transform-mirror-url:22:8)
    at Module._compile (module.js:456:26)
    at Object.Module._extensions..js (module.js:474:10)
    at Module.load (module.js:356:32)
    at Function.Module._load (module.js:312:12)
    at Function.Module.runMain (module.js:497:10)
    at startup (node.js:119:16)
    at node.js:906:3
Error while choosing a download server
<b>Could not choose a download server.</b>

This should not happen. Please report a bug.

** (zenity:5409): WARNING **: Couldn't register with accessibility bus: Did not receive a reply. Possible causes include: the remote application did not send a reply, the message bus security policy blocked the reply, the reply timeout expired, or the network connection was broken.
Gtk-Message: GtkDialog mapped without a transient parent. This is discouraged.
Error while choosing a download server
<b>Could not choose a download server.</b>

This should not happen. Please report a bug.
"tails-transform-mirror-url" unexpectedly returned exit value 8 at /usr/share/perl5/Tails/MirrorPool.pm line 82.
 at /usr/share/perl5/Tails/IUK/Frontend.pm line 593.

Subtasks


Related issues

Blocks Tails - Bug #12405: DAVE: bump the mirrors.json size limit Resolved 2017-03-28

History

#1 Updated by goupille 2016-08-26 14:18:49

  • Assignee set to intrigeri

#2 Updated by goupille 2016-08-26 14:21:56

the reports were from the 23, between midniht and 4

#3 Updated by goupille 2016-08-26 14:26:05

more accuracy about the report : they were received the 23 around midnight, 04:00, and 15:00

#4 Updated by intrigeri 2016-08-27 09:34:56

  • Status changed from New to Resolved
  • Assignee deleted (intrigeri)
  • % Done changed from 0 to 100

Both the timing and error message match the fact that commit 56e643dcb11eb83e3c807be07be1f9bd0254b1c1 in mirror-pool.git broke the JSON syntax on Mon Aug 22 18:27:11 2016 +0000, which was repaired with commit 003c3b7fa267368605ba58250a84d5d388f2abcd on Tue Aug 23 15:18:00 2016 +0000.

I’m going to suggest my team-mates to add a pre-commit hook that runs ./bin/validate-config.

#5 Updated by aerete 2016-09-18 18:50:14

DOWNLOAD MANAGERS TO GET MULTITHREAD AND FAST TAILS ISO FROM dl.amnesia.boum.org ?

This is not only problem with Tails-Upgrader, so please recommend officially download managers to get Tails .iso from your website on pc with linux distributions.

So e.g. downloading problems with URL http://dl.amnesia.boum.org/tails/stable/tails-i386-2.5/tails-i386-2.5.iso

wget is downloading Tails only multithread, and when connection is breaking, it lose iso downloaded file and user must start to download at the begining of. For size 1,1gb it really causes downloading problems.

Multiget www.multiget.sourceforge.net has multithread downloading and correctly extract this URL but download not start.

Do you block multiget or other download managers to get iso file from your website?

Please reccomend multithread and lightweight (with minimum dependecies in Linux) download manager.

Thank you

#6 Updated by aerete 2016-09-18 18:55:07

wget has only one thread of downloading (I am correcting last post)

#7 Updated by Sandokhan 2016-09-19 16:53:47

To build completely script based on wget command line tool which in fact will transform your usual wget into full multithreads, reasuming, masking download manager, you need not only concentrate on:

1. MULTITHREADS downloading like ON website http://blog.netflowdevelopments.com/2011/01/24/multi-threaded-downloading-with-wget or:
http://stackoverflow.com/questions/3430810/wget-download-with-multiple-simultaneous-connections
http://stackoverflow.com/questions/22114610/downloading-a-file-with-wget-using-multiple-connections

where http://example.org/ is a sample path to downloaded file from server
—————————————-
THREADS=10
WGETFLAGS=’-r -np -N’
URL=”http://example.org/”
for i in {1..$THREADS}; do
wget $WGETFLAGS “$URL”
done
—————————————-
or
—————————————-
THREADS=10
WGETFLAGS=’-r -np -N’
URL=”http://example.org/”
for i in `seq 1 $((THREADS-1))`
do
Command=”$Command””wget $WGETFLAGS $URL”‘ & ‘
done
Command=”$Command””wget $WGETFLAGS $URL”
eval $Command
——————————————

but you need to complete wget script with:

2. LOCATION of downloading file e.g. on hdd like on website: http://stackoverflow.com/questions/1078524/how-to-specify-the-location-with-wget

-O is the option to specify the path of the file you want to download to.

wget <file.ext> -O /path/to/folder/file.ext

-P is prefix where it will download the file in the directory

wget <file.ext> -P /path/to/folder

This options will protect e.g. live cd users when wget by default download to RAM , so user can change into hdd or usb and download larger files.

3. REASUMING OPTION like on: http://askubuntu.com/questions/610761/how-do-i-restart-a-wget-download e.g.:
wget -c http://to-downloading-file-from-internet
but this will only work if the server supports Range headers.

4. Masking wget before blocking direct download, which some servers do, like described on website: http://askubuntu.com/questions/24935/wget-downloads-corrupt-jpeg-file

using wget with the -U flag, adding the user agent string of your browser, e.g.:

wget -U “Mozilla/5.0 (X11; U; Linux i686; en-US) AppleWebKit/534.17 (KHTML, like Gecko) Ubuntu/11.04 Chromium/11.0.654.0 Chrome/11.0.654.0 Safari/534.17” http://to-downloading-file-from-internet

But finally even if you will tired and complete your script, if owner of the server doesn’t want free open linux (no tracking) software, the only way to download from this server will be forced by this owner of the server’s software.

#8 Updated by Sandokhan 2016-09-19 17:22:55

Tails is one of the last free non tracking software based on linux, so it’s needed easier and quicker ways to download their iso, so my question to you Tails managers is:

Could you confirm that it’s possible to download iso from your website with 10 times quicker by wget with option:
————————————————-
THREADS=10
WGETFLAGS=’-r -np -N’
URL=”http://dl.amnesia.boum.org/tails/stable/tails-i386-2.5/tails-i386-2.5.iso”
for i in {1..$THREADS}; do
wget $WGETFLAGS “$URL”
————————————————-
and could you please don’t block direct downloads and please support wget -c options?

And answer to aerete, your multiget download manager is ok, but till time when managers will let on direct downloads you need to mask your multiget as e.g. firefox: symilar like wget:

wget -U “Mozilla/5.0 (X11; U; Linux i686; en-US) AppleWebKit/534.17 (KHTML, like Gecko) Ubuntu/11.04 Chromium/11.0.654.0 Chrome/11.0.654.0 Safari/534.17” http://dl.amnesia.boum.org/tails/stable/tails-i386-2.5/tails-i386-2.5.iso

Thanks and good luck

#9 Updated by intrigeri 2016-09-20 02:54:40

> Tails is one of the last free non tracking software based on linux, so it’s needed easier and quicker ways to download their iso, so my question to you Tails managers is:

First of all, this is off-topic on this ticket, so please raise your concerns and ideas to a new, dedicated one. Thanks in advance!
But perhaps you don’t need to do that: downloading over multiple mirrors at the same time might make things faster for the person who does that, but it likely will makes things slower for everybody else, so IMO we should not advertise such methods.

#10 Updated by sajolida 2016-09-21 09:53:34

And if HTTP downloads are not fast enough for you, then I recommend instead using the recommended way of doing “multithreaded downloads” which is BitTorrent, and is always proposed as an option when downloading Tails.

#11 Updated by mercedes508 2017-03-27 14:06:24

  • Status changed from Resolved to Confirmed
  • Assignee set to intrigeri
  • % Done changed from 100 to 0

We received several bug reports for that error since tonight (1am)

#12 Updated by intrigeri 2017-03-27 19:02:37

  • Assignee changed from intrigeri to mercedes508
  • QA Check set to Info Needed

> We received several bug reports for that error since tonight (1am)

I can’t reproduce this and the mirror pool config validation script is happy, so I have no idea what is “that error”. May you please provide the exact error message?

(Meta: in general, let’s avoid re-opening bugs when the problem they’re about has been fixed already, even when the new error looks the same :)

#13 Updated by mercedes508 2017-03-27 21:13:50

  • Assignee changed from mercedes508 to intrigeri

This the error message I get:

Do you want to upgrade now?: Upgrade now / Upgrade later

(zenity:4554): WARNING : Couldn’t register with accessibility bus: Did not receive a reply. Possible causes include: the remote application did not send a reply, the message bus security policy blocked the reply, the reply timeout expired, or the network connection was broken.
Gtk-Message: GtkDialog mapped without a transient parent. This is discouraged.
Downloading the upgrade to Tails 2.11…
Error while choosing a download server
Could not choose a download server.

This should not happen. Please report a bug.

(zenity:4559): WARNING : Couldn’t register with accessibility bus: Did not receive a reply. Possible causes include: the remote application did not send a reply, the message bus security policy blocked the reply, the reply timeout expired, or the network connection was broken.
Gtk-Message: GtkDialog mapped without a transient parent. This is discouraged.
Error while choosing a download server
Could not choose a download server.

This should not happen. Please report a bug.
Could not download ’https://tails.boum.org/mirrors.json', request failed (Maximum file size exceeded): Maximum file size exceeded
at /usr/share/perl5/Tails/MirrorPool.pm line 78.
at /usr/share/perl5/Tails/IUK/Frontend.pm line 593.
localuser:tails-upgrade-frontend being removed from access control list

#14 Updated by intrigeri 2017-03-28 08:31:53

  • Status changed from Confirmed to In Progress
  • Priority changed from Normal to Elevated
  • Target version set to Tails_2.12
  • % Done changed from 0 to 10
  • QA Check deleted (Info Needed)
  • Feature Branch set to bugfix/11735-mirror-pool-size
  • Type of work changed from Research to Code

Thanks! So, that’s a totally different problem than the one this ticket was originally about: mirrors.json is valid, but it has grown above the limit we expected (8 KiB).

mercedes508: I’ve workaround’ed the problem by cleaning the file up a bit => can you please confirm the problem has gone away?

Now I’m working on a branch that bumps the limit.

#15 Updated by mercedes508 2017-03-28 09:22:03

I just tried and it’s fixed! Thanks.

#16 Updated by intrigeri 2017-03-28 10:00:29

> I just tried and it’s fixed! Thanks.

Thank you!

#17 Updated by intrigeri 2017-03-28 13:06:20

  • Assignee changed from intrigeri to anonym
  • % Done changed from 10 to 50
  • QA Check set to Ready for QA
  • Feature Branch changed from bugfix/11735-mirror-pool-size to bugfix/11735-mirror-pool-size, mirror-pool-dispatcher:bugfix/11735-mirror-pool-size

Please review’n’merge into stable, and m-p-d:master.

#18 Updated by intrigeri 2017-03-28 13:09:27

  • blocks Bug #12405: DAVE: bump the mirrors.json size limit added

#19 Updated by anonym 2017-04-03 18:17:28

  • Status changed from In Progress to Fix committed
  • Assignee deleted (anonym)
  • % Done changed from 50 to 100
  • QA Check changed from Ready for QA to Pass

Looks good!

#20 Updated by anonym 2017-04-19 17:25:15

  • Status changed from Fix committed to Resolved