Tails

#6 Updated by intrigeri 2019-05-19 05:48:03

intrigeri wrote:
> I suspect these optimizations gain us very little so I’ll try reverting them, will measure the impact

On https://jenkins.tails.boum.org/job/build_Tails_ISO_bugfix-16730-have-reproducible-build-attempt-use-same-commit-as-first-build/1/consoleFull, the full git fetch --tags took about 10 seconds. So even if the optimization brings this down to zero, it’s definitely not worth it to try to fix this bug while keeping the optimization around, if dropping it is enough to fix the bug.

#8 Updated by intrigeri 2019-05-19 07:56:39

The problem is more complicated than just dropping the optimization. The complex interaction between the Git refs tweaks we do on the Jenkins job side and the other refs tweaks we do in build-tails makes the whole things a mess to understand and causes this bug. I’m making this interaction more straightforward (based on passing environment variables to rake build instead of tweaking Git state), which should fix this bug, allow us to drop hard to understand code from the Jenkins job, make it easier to work on this part of our code, and as a bonus, allow us to test locally something closer to what is done on Jenkins. In general I think Jenkins jobs should do as little clever things as possible, and should stick as close as they can to what developers would run.

#9 Updated by intrigeri 2019-05-19 08:13:34

And in the end, I’m not even 100% sure that the aforementioned optimizations have anything to do with this: on the one hand, I think the timing of when these jobs started failing for no reason matches; OTOH, reverting the optimization is not enough to fix the bug, so I’m confused. Anyway, it saves only 10s, it makes it harder than needed to reason on issues such as this one, and anonym was ready to drop it himself, so I’ll keep the revert on my branch, even if it’s not strictly needed (or at least not sufficient).

#10 Updated by intrigeri 2019-05-19 08:14:49

Corresponding changes I’ve applied to these Jenkins jobs: https://git.tails.boum.org/jenkins-jobs/commit/?id=bc890ef102a5ffc49e79f32cea10d95bf72e7932 and https://git.tails.boum.org/jenkins-jobs/commit/?id=c579fc97df8ea41dda67e24047cae131b2425250.

#12 Updated by intrigeri 2019-05-19 16:38:03

https://jenkins.tails.boum.org/job/reproducibly_build_Tails_ISO_bugfix-16730-have-reproducible-build-attempt-use-same-commit-as-first-build/3/ has successfully reproduced https://jenkins.tails.boum.org/job/build_Tails_ISO_bugfix-16730-have-reproducible-build-attempt-use-same-commit-as-first-build/4/ and was built from the same commit (211f15af9ade3381fb5ccc8ea61331ac78cea331), even though I had pushed additional commits to the topic branch in the meantime. This seems to prove that this branch fixes the bug this ticket is about.

Also, https://jenkins.tails.boum.org/job/reproducibly_build_Tails_ISO_bugfix-16730-have-reproducible-build-attempt-use-same-commit-as-first-build/3/consoleFull shows that:

• Jenkins correctly passed GIT_COMMIT=211f15af9ade3381fb5ccc8ea61331ac78cea331 BASE_BRANCH_GIT_COMMIT=66b67bd21baed5087cae0530773370da46c83b97 to rake build, i.e. the values it got from https://jenkins.tails.boum.org/job/build_Tails_ISO_bugfix-16730-have-reproducible-build-attempt-use-same-commit-as-first-build/4/artifact/build-artifacts/tails-build-env.list/*view*/
• The base branch merge was done as expected with git merge --no-edit 66b67bd21baed5087cae0530773370da46c83b97

I’ll now wait to see if the bunch of unrelated I’ve pushed on top of the commits relevant to this problem break anything.

Dear reviewer, once this branch is merged into stable, devel, and feature/buster, please reassign to me and I’ll remove the set_origin_base_branch_head step from the Jenkins job: this branch does the equivalent itself via the $BASE_BRANCH_GIT_COMMIT it receives on the rake build command line, so we can drop this hack (git update-ref) from the Jenkins job.

#17 Updated by intrigeri 2019-05-29 16:12:24

@anonym, I see you’ve merged this only in stable; I assume it’s a mistake and I’ve thus merged stable → devel → feature/buster so I can do the next step of the work here.

#18 Updated by intrigeri 2019-05-29 16:25:34

intrigeri wrote:
> Dear reviewer, once this branch is merged into stable, devel, and feature/buster, please reassign to me and I’ll remove the set_origin_base_branch_head step from the Jenkins job: this branch does the equivalent itself via the $BASE_BRANCH_GIT_COMMIT it receives on the rake build command line, so we can drop this hack (git update-ref) from the Jenkins job.

Done and merged $(cat config/base_branch) into all active development branches, apart those in namespaces I am told not to touch. Let’s see how it goes.

#20 Updated by anonym 2019-05-29 16:44:13

intrigeri wrote:
> anonym, I see you’ve merged this only in stable; I assume it’s a mistake and I’ve thus merged stable → devel → feature/buster so I can do the next step of the work here.

Gah, yes I forgot to push my devel as usual, sorry about that!

(So far I have assumed that feature/buster is just a normal feature branch with devel as base branch so I didn’t even consider treating it specially — am I missing something? Asking to prevent future mistakes.)

#21 Updated by intrigeri 2019-05-29 17:09:14

> (So far I have assumed that feature/buster is just a normal feature branch with devel as base branch so I didn’t even consider treating it specially — am I missing something? Asking to prevent future mistakes.)

Indeed, you’re right :)