Bug #17203

Automatic install of additional software fails because Tails has not yet connected to Tor

Added by cbrownstein 2019-11-01 01:19:42 . Updated 2019-11-13 19:36:58 .

Status:
Resolved
Priority:
Normal
Assignee:
Category:
Persistence
Target version:
Start date:
Due date:
% Done:

100%

Feature Branch:
bugfix/17203-upgrade-additional-software-after-tor-has-bootstrapped
Type of work:
Code
Blueprint:

Starter:
Affected tool:
Additional Software Packages
Deliverable for:

Description

Sometimes Tails (v4.0) fails to automatically install the additional software I have selected for automatic installation.

I believe the install fails because Tails has not yet connected to Tor when the additional software is attempted to be installed. (I am displayed the “Tor is ready” notification only after the install fails.)

N.B. I encounter no problems installing the additional software with apt after Tails has connected to Tor.

Here is the log I am given after the install fails:

[INFO] Starting to install additional software...
[INFO] Found additional packages list.
[WARNING] Found a copy of old APT lists, restoring it.
[INFO] Will install the following packages: fonts-powerline po4a vim-nox powerline ikiwiki python3-powerline perlmagick libyaml-syck-perl vim-addon-manager
[INFO] Reading package lists...
[INFO] Building dependency tree...
[INFO] Reading state information...
[INFO] The following package was automatically installed and is no longer required:
[INFO]   libpcre2-posix0
[INFO] Use 'apt autoremove' to remove it.
[INFO] The following additional packages will be installed:
[INFO]   libhtml-scrubber-perl libhtml-template-perl libimage-magick-perl
[INFO]   libimage-magick-q16-perl libjson-perl libmarkdown2 libosp5 libruby2.5
[INFO]   libsgmls-perl libtcl8.6 libtext-markdown-discount-perl libyaml-tiny-perl
[INFO]   opensp rake ruby ruby-did-you-mean ruby-minitest ruby-net-telnet
[INFO]   ruby-power-assert ruby-test-unit ruby-xmlrpc ruby2.5 rubygems-integration
[INFO]   sgml-base vim-runtime
[INFO] Suggested packages:
[INFO]   dvipng graphviz libfile-mimeinfo-perl libhighlight-perl libmagickcore-extra
[INFO]   libmailtools-perl libnet-amazon-s3-perl libnet-inet6glue-perl
[INFO]   libsearch-xapian-perl libsort-naturally-perl libsparkline-php
[INFO]   libtext-csv-perl libtext-multimarkdown-perl libtext-textile-perl
[INFO]   libtext-typography-perl libtext-wikicreole-perl libtext-wikiformat-perl
[INFO]   libxml-feed-perl libxml-writer-perl polygen python3-docutils texlive tidy
[INFO]   viewvc | gitweb | viewcvs xapian-omega libipc-sharedcache-perl
[INFO]   imagemagick-doc sgmlspl sgmls-doc tcl8.6 doc-base sgml-data powerline-doc ri
[INFO]   ruby-dev bundler sgml-base-doc cscope vim-doc
[INFO] Recommended packages:
[INFO]   gcc | c-compiler libauthen-passphrase-perl libc6-dev | libc-dev
[INFO]   libcgi-formbuilder-perl libcgi-session-perl libcrypt-ssleay-perl
[INFO]   libgravatar-url-perl liblwpx-paranoidagent-perl libmail-sendmail-perl
[INFO]   libnet-openid-consumer-perl librpc-xml-perl libterm-readline-gnu-perl
[INFO]   libxml-simple-perl libjson-xs-perl libterm-readkey-perl zip fonts-lato
[INFO] The following NEW packages will be installed:
[INFO]   fonts-powerline ikiwiki libhtml-scrubber-perl libhtml-template-perl
[INFO]   libimage-magick-perl libimage-magick-q16-perl libjson-perl libmarkdown2
[INFO]   libosp5 libruby2.5 libsgmls-perl libtcl8.6 libtext-markdown-discount-perl
[INFO]   libyaml-syck-perl libyaml-tiny-perl opensp perlmagick po4a powerline
[INFO]   python3-powerline rake ruby ruby-did-you-mean ruby-minitest ruby-net-telnet
[INFO]   ruby-power-assert ruby-test-unit ruby-xmlrpc ruby2.5 rubygems-integration
[INFO]   sgml-base vim-addon-manager vim-nox vim-runtime
[INFO] 0 upgraded, 34 newly installed, 0 to remove and 20 not upgraded.
[INFO] Need to get 9156 kB/17.6 MB of archives.
[INFO] After this operation, 70.5 MB of additional disk space will be used.
[INFO] Err:1 tor+http://vwakviie2ienjx6t.onion/debian buster/main amd64 libimage-magick-q16-perl amd64 8:6.9.10.23+dfsg-2.1
[INFO]   Timed out while waiting to read 'first part of response' from proxy socks5h://127.0.0.1:9050 [IP: 127.0.0.1 9050]
[INFO] Get:2 tor+http://vwakviie2ienjx6t.onion/debian buster/main amd64 po4a all 0.55-1 [1542 kB]
[INFO] Get:3 tor+http://vwakviie2ienjx6t.onion/debian buster/main amd64 python3-powerline all 2.7-2 [126 kB]
[INFO] Get:4 tor+http://vwakviie2ienjx6t.onion/debian buster/main amd64 powerline amd64 2.7-2 [51.6 kB]
[INFO] Get:5 tor+http://vwakviie2ienjx6t.onion/debian buster/main amd64 vim-addon-manager all 0.5.10 [19.9 kB]
[INFO] Get:6 tor+http://vwakviie2ienjx6t.onion/debian buster/main amd64 vim-runtime all 2:8.1.0875-5 [5775 kB]
[INFO] Get:7 tor+http://vwakviie2ienjx6t.onion/debian buster/main amd64 vim-nox amd64 2:8.1.0875-5 [1401 kB]
[INFO] Fetched 8866 kB in 1min 14s (120 kB/s)
[INFO] E: Failed to fetch tor+http://vwakviie2ienjx6t.onion/debian/pool/main/i/imagemagick/libimage-magick-q16-perl_6.9.10.23+dfsg-2.1_amd64.deb  Timed out while waiting to read 'first part of response' from proxy socks5h://127.0.0.1:9050 [IP: 127.0.0.1 9050]
[INFO] E: Unable to fetch some archives, maybe run apt-get update or try with --fix-missing?
[WARNING] apt-get exited with returncode 100
[WARNING] Warning: installation of fonts-powerline po4a vim-nox powerline ikiwiki python3-powerline perlmagick libyaml-syck-perl vim-addon-manager failed

Subtasks


Related issues

Related to Tails - Bug #17278: Additional Software is often broken after initial persistence setup and when recovering old APT lists Needs Validation
Related to Tails - Bug #17392: apt sometimes fails to fetch additional software packages after upgrading Tails Confirmed
Blocks Tails - Feature #16209: Core work: Foundations Team Confirmed

History

#1 Updated by cbrownstein 2019-11-01 01:56:06

I’m not sure if this matters: I upgraded from Tails 3.16 and therefore had old apt lists and old cached packages.

#2 Updated by intrigeri 2019-11-01 09:55:21

Hi Cody,

> Sometimes Tails (v4.0) fails to automatically install the additional software I have selected for automatic installation.

Ouch!

> I believe the install fails because Tails has not yet connected to Tor when the additional software is attempted to be installed.

The log shows that:

  • the first attempt to download a package fails
  • then the next 6 packages are downloaded successfully

So yeah, it might be that we’re starting too early (e.g. the time sync’ process may mess with ASP).

To investigate, we would need a WhisperBack report.

#3 Updated by op_mb 2019-11-01 21:00:36

if you’re getting this after Tails boots and tries to install aditional software, it means your /var/lib/apt/lists wasn’t synchronized on exit (usually if you pull the usb out).

on Tails start, it will look to “download” from your persistent source=apt/lists.
run: sudo apt-update // after you install additional software for the first time.

if that doesnt help, copy and paste manually from /var/lib/apt/lists to source=apt/lists

#4 Updated by op_mb 2019-11-01 21:04:03

correction: sudo apt-get update

#5 Updated by intrigeri 2019-11-02 06:00:42

> if you’re getting this after Tails boots and tries to install aditional software, it means your /var/lib/apt/lists wasn’t synchronized on exit (usually if you pull the usb out).

The log shared by cbrownstein suggests that the APT lists were just fine. But indeed, one relevant question is: why wasn’t libimage-magick-q16-perl in the cache already?

> I’m not sure if this matters: I upgraded from Tails 3.16 and therefore had old apt lists and old cached packages.

Did this problem happen on first boot on 4.0 after upgrading?
Did this problem happen on any subsequent 4.0 boot?

#6 Updated by cbrownstein 2019-11-02 18:27:42

> […] But indeed, one relevant question is: why wasn’t libimage-magick-q16-perl in the cache already?

Wouldn’t the version in the cache be from stretch rather than from buster?

> Did this problem happen on first boot on 4.0 after upgrading?

Yes, I encountered this problem on my first boot after upgrading from 3.16 to 4.0.

> Did this problem happen on any subsequent 4.0 boot?

Yes, I encountered the problem on every subsequent boot until I ran:

sudo apt install fonts-powerline po4a vim-nox powerline ikiwiki python3-powerline perlmagick libyaml-syck-perl vim-addon-manager

I have not encountered the problem since running that command.

#7 Updated by intrigeri 2019-11-10 17:07:33

  • Status changed from New to Confirmed
  • Assignee set to intrigeri
  • Target version set to Tails_4.1
  • Affected tool set to Additional Software Packages

#8 Updated by intrigeri 2019-11-10 17:11:24

  • Status changed from Confirmed to In Progress
  • Feature Branch set to bugfix/17203-upgrade-additional-software-after-tor-has-bootstrapped

#9 Updated by intrigeri 2019-11-10 17:13:43

Hi,

>> […] But indeed, one relevant question is: why wasn’t libimage-magick-q16-perl in the cache already?

> Wouldn’t the version in the cache be from stretch rather than from buster?

OK, I’m now convinced that you’re right, thanks to this clarification:

>> Did this problem happen on first boot on 4.0 after upgrading?
> Yes, I encountered this problem on my first boot after upgrading from 3.16 to 4.0.

My current conclusion on this front is:

  • APT could fetch the lists successfully, then Tails did something that disturbed Tor, which caused the download of libimage-magick-q16-perl to fail, then Tor came back up and the other packages could be downloaded without trouble.
  • Regarding the “something that disturbed Tor”: to be investigated; potential offenders:
    • ordering of tails-additional-software-upgrade.service vs. Tor bootstrap and time sync’ing
    • the cases where we restart the tor service
    • time sync’ing

A first, obvious bug is that tails-additional-software-upgrade.service has After=tor-has-bootstrapped.service, while we have no such service: it should instead be After=tails-wait-until-tor-has-bootstrapped.service. I don’t know if that’s the only reason for this bug. I’m tempted to be optimistic, fix this one, and see if more reports about such problems flow in.

>> Did this problem happen on any subsequent 4.0 boot?

> Yes, I encountered the problem on every subsequent boot until I ran:
>

> sudo apt install fonts-powerline po4a vim-nox powerline ikiwiki python3-powerline perlmagick libyaml-syck-perl vim-addon-manager
> 

It’s a weird coincidence, unless you’re always using this Tails on computers and networks with similar performance to each other, in which case, indeed there’s a good chance that you hit the exact same race condition every time.

> I have not encountered the problem since running that command.

OK. My understanding is that this ensured all the packages are in the cache, so next time, they’ll be installed via “Software installation at startup” instead of “Software upgrade on Internet connection” (https://tails.boum.org/contribute/design/additional_software_packages/).

#10 Updated by intrigeri 2019-11-10 17:40:25

#11 Updated by cbrownstein 2019-11-11 01:42:41

> >> Did this problem happen on any subsequent 4.0 boot?
>
> > Yes, I encountered the problem on every subsequent boot until I ran:
> > […]
>
> It’s a weird coincidence, unless you’re always using this Tails on computers and networks with similar performance to each other, in which case, indeed there’s a good chance that you hit the exact same race condition every time.

FWIW, I was using the exact same computer and network when I was experiencing the problem.

#12 Updated by intrigeri 2019-11-11 06:32:15

  • Status changed from In Progress to Needs Validation
  • Assignee deleted (intrigeri)

#13 Updated by segfault 2019-11-13 19:36:58

  • Status changed from Needs Validation to Resolved
  • % Done changed from 0 to 100

Applied in changeset commit:tails|f5d5e04d77c35cdf612e431ae7379370eab5cda9.

#14 Updated by intrigeri 2019-12-01 07:18:27

  • related to Bug #17278: Additional Software is often broken after initial persistence setup and when recovering old APT lists added

#15 Updated by cbrownstein 2020-01-04 04:05:54

  • related to Bug #17392: apt sometimes fails to fetch additional software packages after upgrading Tails added