Some branches fail to build an ISO because we merge their base branch too late
Our ISO build system errors out when building from a branch based on master:
> + apt-snapshots-serials prepare-build > + apt-mirror debian > Not building from a tag, but last version in changelog was released > + DEBIAN_MIRROR= > + apt-mirror debian-security > Not building from a tag, but last version in changelog was released > + DEBIAN_SECURITY_MIRROR= > + apt-mirror torproject > Not building from a tag, but last version in changelog was released > + TORPROJECT_MIRROR= > + [ -n ] > + exit 1
I believe this is caused by
apt-snapshots-serials prepare-build before
auto/build had a chance to merge the base branch (generally:
stable) into the topic branch that’s being built: a branch based on master generally satisfies the “Not building from a tag, but last version in changelog was released” error condition until their base branch is merged into it.
The simplest, ad-hoc fix would probably be to move a chunk of code from
auto/build. I’ll handle this.
But thinking about the root cause, a more fundamental issue becomes obvious: merging the base branch happens too late, and by running a fair amount of (build) code before we do that, we create a frankenstein ISO whose build system partly comes from the current status of the topic branch, and partly comes from the current status of it base branch. IMO we should do the base branch merge first thing in the build process, in a manner that’s isolated from other build steps, so that all build code (including
Rakefile) except the tiny script that performs the merge will be. anonym, bertagaz: if you agree we have a fundamental problem here, please file a ticket about it.
|Related to Tails - Bug #12557: The mergebasebranch option does the base branch merge too late||Confirmed||2017-05-17|
Related to Tails -
|Blocks Tails - Feature #11355: Re-enable Jenkins notifications on ISO build/test failure||In Progress||2017-08-28|
Blocks Tails -
Blocks Tails -
#8 Updated by intrigeri 2017-09-02 11:39:02
- Status changed from Confirmed to In Progress
- Assignee changed from intrigeri to anonym
- % Done changed from 0 to 50
- QA Check set to Ready for QA
- Feature Branch set to bugfix/14459-merge-base-branch-earlier
I’ve tested my fix this way a branch forked off current master, whose base branch is set to
stable, fails to build as expected (it has commits on top of the last entry in
debian/changelog); cherry-picking my commit on top of that branch fixes the problem.
I wanted to also test a topic branch whose APT snapshots have expired, but this will still fail even with my fix unless the corresponding Vagrant box already exists: we’re still building it before we merge the base branch. So something more involved (Bug #12557) is needed to address this problem.
FWIW I’ve also verified that (locally) after merging my branch into stable and devel doesn’t break the build; I didn’t wait for the build to finish, I’ve only waited for
lb build to have been run by
The topic branch is based on master on purpose, as it should be merged into master (to fix the original problem this ticket was about) in addition to the usual stable & devel. (Once this is done we may have to also merge its base branch into every active topic branch; we’ll see.)
#9 Updated by intrigeri 2017-09-04 20:13:19
- Assignee changed from anonym to intrigeri
- QA Check changed from Ready for QA to Dev Needed
Hold on! This branch breaks reproducibility on Jenkins (not for actual releases though, I think) because it encodes
BUILD_BASENAME, that contains a timestamp, in
#14 Updated by anonym 2017-09-12 18:54:12
- Assignee set to intrigeri
- % Done changed from 100 to 90
- QA Check changed from Pass to Ready for QA
So while I verified that your branch could successfully build an ISO image, I didn’t look closely at their file names. I’ve already pushed a fixup with commit:1bef9097d16d94752e6a0657a2230fc09c544691. Please have a look!