Tails

#2 Updated by intrigeri 2018-03-07 05:43:11

> Three questions that will help evaluating how much work this will require:

And another one: do these tests have noticeable side effects? (I wonder if running each of them invalidates the test environment; if running in the context of the Tails automated test suite, that would imply restoring a fresh snapshot between each individual test run, which is fine but we need to know if we have to do that :)

#3 Updated by segfault 2018-03-14 20:50:24

intrigeri wrote:
> Three questions that will help evaluating how much work this will require:
>
> * In which environment are these tests meant to be run? I’ve tried running config/chroot_local-includes/usr/local/bin/tails-get-bootinfo doctest on my Debian sid and it fails. Do these tests have to run inside Tails?

Yes, they are meant to be run inside Tails.

> If yes, can they run outside of the desktop session?

No, some of tests start graphical applications, so they cannot be run outside a desktop session. Those tests also require that the started application is being closed before the test can continue.

> * Is there a good way to automatically list scripts that have such doctests, or do we need to start maintaining a list in a file?

I don’t know of a good way to list scripts with doctests automatically, I think we need to maintain a list.

> * Do these tests work reliably for you?

No, but it shouldn’t require a lot of work to make them work reliably (the tails-get-bootinfo test seems to be broken, and the tails-upgrade-frontend-wrapper test assumes that there is enough free memory for an upgrade, else the test fails).

> And another one: do these tests have noticeable side effects? (I wonder if running each of them invalidates the test environment; if running in the context of the Tails automated test suite, that would imply restoring a fresh snapshot between each individual test run, which is fine but we need to know if we have to do that :)

None of the 4 scripts we shipped in Tails 3.6 have noticeable side effects, but I didn’t check the other scripts that were already ported and not shipped yet.