Feature #17296

Stop testing DVDs manually

Added by sajolida 2019-12-03 12:46:20 . Updated 2019-12-12 08:09:47 .

Status:
Resolved
Priority:
Normal
Assignee:
Category:
Target version:
Start date:
Due date:
% Done:

100%

Feature Branch:
Type of work:
Contributors documentation
Blueprint:

Starter:
Affected tool:
Deliverable for:

Description

We have in our manual test suite:

DVD

    Boot on bare-metal from DVD, measuring the boot time (from the syslinux menu until the GNOME desktop is ready — quickly press ENTER in the Greeter).
    Compare with the boot time of the previous Tails version. The new one should not be significantly slower to start.

kibi reported that RMs don’t test it on bare metal anymore (maybe the description should be adjusted) but still burn a DVD and run it through a VM. It still takes around 15 minutes each time according to him.

DVDs are used by less than 5% of our users these days. The 2 data points that we have about this are:

  • Click on the download pages. Putting aside /install/download, /install/download-iso, and upgrades, these are the number of clicks on the download buttons:
    Image   Torrent     
Win     24482   6211    30693   67%
Linux    6374   1172     7546   16%
macOS    2922    490     3412    7%
VM       2224    547     2771    6%
DVD      1222    257     1479    3%
        37224   8677    45901

(Off-topic: Windows users are downloading much more than that are using Tails according to https://tails.boum.org/blueprint/veracrypt/, 67% vs 52%, and we probably have a retention problem here. Or maybe Linux users are more old timers and don’t go through the installation scenario so much anymore compared to Windows users).

  • WhisperBack reports in 2019Q1-3:
USB     472  92%
DVD      21   4%
VM       19   4%
        512

Another question is whether or not this small minority is important enough for us to keep doing these manual tests. My guess is that it is not, for example Secure Drop people are using USB sticks, see Feature #17178. Also, none of our personas would prefer DVDs over USB sticks because they all need Persistence (Kim maybe a bit less than Riou and Cris).

Subtasks

History

#1 Updated by sajolida 2019-12-03 12:50:22

  • Description updated

#2 Updated by intrigeri 2019-12-05 19:58:14

> Another questions is whether or not this small minority is important enough for us to keep doing these manual tests. My guess is that it is not, […]

I’m fine with dropping this manual test but I’m wondering if I misunderstood something: you wrote “Another questions”, and I don’t see what’s the other question this “another” is refering to.

#3 Updated by sajolida 2019-12-07 12:43:51

  • Description updated

Fixed “Another questions” → “Another question”.

Having only a small minority rely on something shouldn’t be an argument in itself to deprioritize it. For example, even a small minority could be really important to us. That’s why I didn’t want to only mention the numbers and leave this (single) question open for more comments. I personally don’t think that this DVD minority is really important to us.

#4 Updated by intrigeri 2019-12-12 08:09:48

  • Status changed from New to Resolved
  • % Done changed from 0 to 100

Applied in changeset commit:tails|ef2f8911cd357a4173fe65a7528f49a132883ca4.