Bug #14946

Topic branches that lag behind their base branch don't build reproducibly with mergebasebranch build option

Added by intrigeri 2017-11-10 15:35:38 . Updated 2017-11-15 11:37:43 .

Status:
Resolved
Priority:
Normal
Assignee:
Category:
Build system
Target version:
Tails_3.3
Start date:
2017-11-10
Due date:
% Done:

100%

Feature Branch:
bugfix/14946-mergebasebranch-breaks-reproducibility
Type of work:
Code
Blueprint:

Starter:
Affected tool:
Deliverable for:
301

Description

intrigeri wrote on Feature #12633#note-36:
> I think there’s a bug in the handling of the base branch merge or something related: https://jenkins.tails.boum.org/view/Tails_ISO/job/reproducibly_build_Tails_ISO_bugfix-14796-topicons-position/8/artifact/build-artifacts/diffoscope.html shows differing (merge) commit IDs, while the build logs pretend we’re merging the exact same commit (“at commit …”). Neither of these two commit IDs are in the official tails.git, hence my guess that a different merge commit ID is created for some reason unknown to me. I don’t know what’s going on here but it seems fishy. Please investigate and if this is Someone Else’s Problem™ file a ticket for them, otherwise fix it. I suspect it would help debugging if we passed --verbose to this git merge, and/or displayed the ID of the resulting merge commit, or similar.

I think I’ve got it: two otherwise identical merge commits done at different times get different IDs. So if I got it right:

  • base branches are not affected by this bug, which is nice
  • any topic branch is affected by this bug only if it is lagging behind its base branch; that’s a rather common case so it feels wrong to report such confusing info to the reviewer; it’s easy to workaround (merge base branch,& push) but then you have to wait about 2h20min for the next results, IMO our already painful (some would say “bureaucratic”) reviewe’n’merge process really does not need to be made more cumbersome.

I’ve played locally with using faketime to workaround this problem and it seems to work fine: faketime -f "YYYY-MM-DD hh:mm:ss" git merge --no-edit $whatever always creates the same merge commit ID. Note that -f is important otherwise the clock keeps running and if for some reason things get slow, then the commit can be created after the specified timestamp.

In passing, too bad git merge has no --date option, while git commit does.

Subtasks

Related issues

Related to Tails - Feature #12633: Lower the workload caused by reproducible builds Jenkins jobs Resolved 2017-10-22
Blocks Tails - Bug #14924: reproducibly_build_Tails_ISO_stable Jenkins job always fails Resolved 2017-11-06

History

#1 Updated by intrigeri 2017-11-10 15:36:51

  • related to Feature #12633: Lower the workload caused by reproducible builds Jenkins jobs added

#2 Updated by intrigeri 2017-11-10 15:40:12

  • Subject changed from Our CI can't reproduce builds of topic branches that lag behind their base branch to Topic branches that lag behind their base branch don't build reproducibly with mergebasebranch build option
  • Category changed from Continuous Integration to Build system

(This is not specific to our CI.)

#3 Updated by intrigeri 2017-11-10 15:42:49

  • blocked by Bug #14933: stable branch is not reproducible: differences in some .fa.html website files added

#4 Updated by intrigeri 2017-11-10 15:43:22

(I’ll base my work on the branch for Bug #14933 so there’s a chance the resulting ISO builds reproducibly.)

#5 Updated by intrigeri 2017-11-10 15:45:53

#6 Updated by intrigeri 2017-11-10 15:53:54

  • Status changed from Confirmed to In Progress
  • % Done changed from 0 to 10
  • QA Check set to Ready for QA
  • Feature Branch set to bugfix/14946-mergebasebranch-breaks-reproducibility

#8 Updated by intrigeri 2017-11-11 09:29:46

Please merge into master → stable → devel, and I’ll merge into the active not-yet-reviewed topic branches so Jenkins stops showing them as non-reproducible and stops storing tons of ISOs.

#9 Updated by bertagaz 2017-11-11 13:12:52

  • Assignee changed from anonym to bertagaz

Taking over the reviewing as part of the follow up of Feature #12633.

#10 Updated by intrigeri 2017-11-12 10:30:34

  • blocks Bug #14924: reproducibly_build_Tails_ISO_stable Jenkins job always fails added

#11 Updated by bertagaz 2017-11-13 11:12:09

  • Status changed from In Progress to Fix committed
  • Assignee deleted (bertagaz)
  • % Done changed from 50 to 100
  • QA Check changed from Ready for QA to Pass

bertagaz wrote:
> Taking over the reviewing as part of the follow up of Feature #12633.

Good catch, as said on IRC, I would have spent time trying to hack the merge in the build process, rather than think about the timedate affecting the merge. So I’ve merged it just in time for 3.3.

#12 Updated by anonym 2017-11-15 11:37:20

  • blocks deleted (Bug #14933: stable branch is not reproducible: differences in some .fa.html website files)

#13 Updated by anonym 2017-11-15 11:37:43

  • Status changed from Fix committed to Resolved