Feature #8988
What to do with run_test_suite's dependency checking
100%
Description
Currently it’s out-of-date — one must follow the setup instructions to get a working environment that’s able to run the automated test suite.
Personally I think we should just remove it so it will be up to the user of the test suite to follow the setup instructions. The only benefit I can see in keeping it (+ enforcing that we keep it up-to-date) is that it would be handy to notify the user of new dependencies in a new branch, for instance. But still, its dependency checking is crude and cannot specify that “we need at least this version of $pkg” and similar.
Subtasks
Related issues
Related to Tails - |
Resolved | 2015-04-28 |
History
#1 Updated by kytv 2015-03-02 03:26:19
I automatically go to the set-up docs when/if things aren’t working, so I’ve no strong preference here.
#2 Updated by intrigeri 2015-03-03 11:53:43
- Assignee set to anonym
On the one hand, we have a tails::tester
Puppet module already, that I keep up-to-date since that’s what sets up isotesterN.lizard. Maybe maintaing these bits in two different places is enough, and those who want to have their testing setup always up-to-date can use the Puppet module.
On the other hand, I’ve seen e.g. Alan struggle with keeping his testing environment up-to-date, and with keeping up with the unwritten requirements our automated test suite has (I’ve filed at least one ticket about it recently). Removing these checks would lead to more and more obscure errors and discouragement for such people who don’t run the test suite often => even if they are crude and not entirely up-to-date, they probably help.
So I’m unsure. Maybe ask Alan and others on -dev@ how they would feel about it?
#3 Updated by anonym 2015-03-27 15:12:19
- Status changed from Confirmed to In Progress
Applied in changeset commit:904cde1266f84b06df12c39c1f4839490d893239.
#4 Updated by anonym 2015-03-27 15:13:36
- Assignee changed from anonym to kytv
- % Done changed from 0 to 50
- QA Check set to Ready for QA
- Feature Branch set to test/8988-up-to-date-dep-checks
I ended up updating the deps. I’ll be harder checking that updates to the setup instructions are reflected in the script when reviewing stuff, suggest that you are too, kytv and intrigeri. :)
#5 Updated by kytv 2015-04-06 15:21:15
- Assignee changed from kytv to anonym
- Target version changed from Tails_1.3.2 to Tails_1.4
- QA Check changed from Ready for QA to Dev Needed
The dependencies look good but in it fails for me with
# ./run_test_suite --view --iso tails-i386-1.3.2.iso features/unsafe_browser.feature
run_test_suite: error: 'libvirt-daemon-system' is missing, please install it and run again. Aborting...
(I just upgraded my test VM to Jessie)
#6 Updated by intrigeri 2015-04-06 15:25:50
> The dependencies look good but in it fails for me with
>
> # ./run_test_suite --view --iso tails-i386-1.3.2.iso features/unsafe_browser.feature
> run_test_suite: error: 'libvirt-daemon-system' is missing, please install it and run again. Aborting...
>
That’s Bug #9156.
#7 Updated by kytv 2015-04-06 15:33:27
- Assignee changed from anonym to intrigeri
- QA Check changed from Dev Needed to Ready for QA
intrigeri wrote:
> > The dependencies look good but in it fails for me with
>
> > […]
>
> That’s Bug #9156.
Ah..never mind then. I worked around it locally by commenting out the pipefail line but
package_installed() {
+ set +o pipefail
dpkg -s "${1}" 2>/dev/null | grep -q "^Status:.*installed"
+ set -o pipefail
looks like it’ll be a proper fix.
Re-assigning to intrigeri for his perusal.
#8 Updated by intrigeri 2015-04-06 15:46:03
- Assignee changed from intrigeri to anonym
- % Done changed from 50 to 60
- QA Check changed from Ready for QA to Dev Needed
I see this branch introduces the full list of dependencies, that has changed since we’ve merged Feature #8165, so it can’t really be merged as-is => please merge current stable into this branch, resolve any conflict, perhaps merge my branch for Bug #9156 into it as well, and then we should be good to go.
#9 Updated by anonym 2015-04-29 01:48:18
- Assignee changed from anonym to intrigeri
- QA Check changed from Dev Needed to Ready for QA
intrigeri wrote:
> I see this branch introduces the full list of dependencies, that has changed since we’ve merged Feature #8165, so it can’t really be merged as-is => please merge current stable into this branch, resolve any conflict, perhaps merge my branch for Bug #9156 into it as well, and then we should be good to go.
I’ve merged stable, and beware: the conflict resolution was a quite hairy.
Also noticed that something was off with package_installed()
again… => Bug #9288.
#10 Updated by intrigeri 2015-04-29 02:55:54
- Assignee changed from intrigeri to kytv
- % Done changed from 60 to 70
- Type of work changed from Discuss to Code
anonym wrote:
> I’ve merged stable,
Thanks!
> and beware: the conflict resolution was a quite hairy.
Then reassigning to kytv for another round of review and test before I look at it.
#11 Updated by kytv 2015-05-01 11:55:57
- Assignee changed from kytv to intrigeri
Review and tests pass, but the branch in Bug #9288 is needed for the dependency checking to work (as noted above)
#12 Updated by intrigeri 2015-05-01 12:13:04
Going to merge that, with commit:b389b93 and commit:1f466c5 on top.
#13 Updated by intrigeri 2015-05-01 12:13:58
- Status changed from In Progress to Fix committed
- % Done changed from 70 to 100
Applied in changeset commit:c6ab155fece99c1d9a1044fb631a2dcdef14f2b0.
#14 Updated by BitingBird 2015-05-01 12:15:12
- related to
Bug #9288: Dependency checking in run_test_suite is still broken added
#15 Updated by BitingBird 2015-05-01 12:15:33
- Assignee deleted (
intrigeri) - QA Check changed from Ready for QA to Pass
#16 Updated by BitingBird 2015-05-12 18:43:22
- Status changed from Fix committed to Resolved