Feature #8555

Adapt packaging process and release doc for Tails Installer DFSG-free tarball

Added by intrigeri 2015-01-06 13:57:52 . Updated 2015-10-31 08:12:50 .

Status:
Resolved
Priority:
Normal
Assignee:
Category:
Installation
Target version:
Tails_1.7
Start date:
2015-01-06
Due date:
% Done:

100%

Feature Branch:
doc/8555-update-installer-release-doc
Type of work:
Code
Blueprint:

Starter:
Affected tool:
Installer
Deliverable for:

Description

These docs live in contribute/release_process/liveusb-creator and contribute/release_process/liveusb-creator/topic_branch.

Subtasks

Related issues

Related to Tails - Feature #10425: Clarify needed branches in Tails Installer release documentation and update doc to rename Rejected 2015-10-26
Blocked by Tails - Feature #8554: Repack a DFSG-free Tails Installer tarball Resolved 2015-01-06
Blocks Tails - Feature #8561: Adapt Tails to the Installer rename Resolved 2015-01-06

History

#1 Updated by intrigeri 2015-01-06 13:58:05

  • blocked by Feature #8554: Repack a DFSG-free Tails Installer tarball added

#2 Updated by intrigeri 2015-01-06 14:01:10

  • blocks Feature #8557: Have Tails Installer uploaded and accepted into Debian added

#3 Updated by intrigeri 2015-01-06 14:06:31

  • blocks Feature #8561: Adapt Tails to the Installer rename added

#4 Updated by intrigeri 2015-01-06 14:47:25

  • blocks #8538 added

#5 Updated by intrigeri 2015-01-11 10:04:17

  • Description updated

#6 Updated by Anonymous 2015-02-26 15:05:45

  • Target version changed from Hardening_M1 to Tails_1.4

#7 Updated by intrigeri 2015-05-08 15:32:45

  • Status changed from Confirmed to In Progress
  • % Done changed from 0 to 10

There’s been some progress, as seen on Feature #8554#note-6 and follow-ups.

#8 Updated by Anonymous 2015-05-11 12:57:17

  • Assignee set to intrigeri
  • QA Check set to Ready for QA
  • Feature Branch set to u451f:doc/installerpkg

#9 Updated by intrigeri 2015-05-11 15:22:18

  • Feature Branch changed from u451f:doc/installerpkg to 451f:doc/installerpkg

#10 Updated by intrigeri 2015-05-11 15:36:54

  • Assignee deleted (intrigeri)
  • QA Check changed from Ready for QA to Dev Needed

That’s a pretty good start. Here are a few initial comments:

  • This branch introduces spurious trailing whitespace.
  • I’m not sure which exact version of gbp introduces the gbp command, and which one deprecates the old git-buildpackage one. Currently this piece of doc uses both, which doesn’t work e.g. on current sid. Would be nice to find a set of commands that work both on Jessie (potentially with some backport) and on current sid.
  • The gbp import-orig part should live in the “Update the Debian package” section (after switching to the packaging branch), rather than in the “Generate a new upstream tarball” one, because it has nothing to do with upstream stuff (in theory, someone could put out a new upstream version, and someone different could package it).
  • In the “Update the Debian package” section, gbp import-orig, once it has moved to the right place as requested above, replaces part of git merge master (and actually, merging the master branch at that point may have unintended consequences). It would even replace 100% of it if:
    • we tagged the master branch $NEW_UPSTREAM_VERSION right before or after creating the new upstream tarball (which would be nice anyway);
    • we passed something like --upstream-vcs-tag=$NEW_UPSTREAM_VERSION to gbp import-orig — pretty nifty, uh? :) See the tails-iuk release process for a real-world example.
  • Meta: I’d rather not call this done before the packaging uses pristine-tar (Feature #8551).

#11 Updated by Anonymous 2015-05-12 06:52:24

  • % Done changed from 10 to 20

intrigeri wrote:
> That’s a pretty good start. Here are a few initial comments:
>
> * This branch introduces spurious trailing whitespace.

fixed

> * I’m not sure which exact version of gbp introduces the gbp command, and which one deprecates the old git-buildpackage one. Currently this piece of doc uses both, which doesn’t work e.g. on current sid. Would be nice to find a set of commands that work both on Jessie (potentially with some backport) and on current sid.

The Jessie commands are the same as in Sid, but not in Wheezy from what I understand. So i modified this accordingly.

> * The gbp import-orig part should live in the “Update the Debian package” section (after switching to the packaging branch), rather than in the “Generate a new upstream tarball” one, because it has nothing to do with upstream stuff (in theory, someone could put out a new upstream version, and someone different could package it).

ok

> * In the “Update the Debian package” section, gbp import-orig, once it has moved to the right place as requested above, replaces part of git merge master (and actually, merging the master branch at that point may have unintended consequences). It would even replace 100% of it if:
> we tagged the master branch $NEW_UPSTREAM_VERSION right before or after creating the new upstream tarball (which would be nice anyway);

mk-origtargz creates such a tag i think - but that is using the dfsg-free version.
So should we have 2 tags or just this one?

> we passed something like --upstream-vcs-tag=$NEW_UPSTREAM_VERSION to gbp import-orig — pretty nifty, uh? :) See the tails-iuk release process for a real-world example.

sounds good, i did not yet implement it in the doc before testing it.

> * Meta: I’d rather not call this done before the packaging uses pristine-tar (Feature #8551).

#12 Updated by Anonymous 2015-05-12 10:14:35

  • % Done changed from 20 to 30

#13 Updated by Anonymous 2015-05-13 09:22:23

  • Target version changed from Tails_1.4 to Tails_1.4.1

#14 Updated by intrigeri 2015-05-29 15:43:01

Ooops, I had missed that one because it had not been assigned to me for review.

u wrote:
> intrigeri wrote:
> > * In the “Update the Debian package” section, gbp import-orig, once it has moved to the right place as requested above, replaces part of git merge master (and actually, merging the master branch at that point may have unintended consequences). It would even replace 100% of it if:
> > we tagged the master branch $NEW_UPSTREAM_VERSION right before or after creating the new upstream tarball (which would be nice anyway);
>
> mk-origtargz creates such a tag i think

I doubt it, given the purpose/scope of that tool. And the manpage doesn’t talk about Git at all. I suspect you’re confusing it with gbp import-orig — no surprise you may be a bit lost, so many tools and moving pieces :)

> - but that is using the dfsg-free version.
> So should we have 2 tags or just this one?

We definitely need two tags, since we’re doing different things under two different hats here:

  • one for the upstream release; I see you’re trying to create it in the doc (git tag -s upstream/$NEW_UPSTREAM_VERSION) but that’s confusing, since the upstream/* namespace is the one used by default precisely for the other tag (see below); I suggest to instead: git tag -s tails-installer_$NEW_UPSTREAM_VERSION. This tag is needed for anyone looking at the upstream side of things to understand what branch/commit a given upstream release, aka. tarball, was built from;
  • another one for the imported repacked upstream tarball into the branch used as “upstream” in the Debian packaging; gbp import-orig creates it automatically I think. This tag is needed for gbp to do its work properly, and for anyone looking at the Debian side of things to understand what branch/commit a given package was built from.

In general, when working on this specific project, I think you should try to clearly separate in your mind what’s about upstream, and what’s about Debian packaging. I’m sure it’ll help avoiding confusion — perhaps get two hats / rubber ducks / mice or something, one for “I’m upstream” and another one for “I’m the Debian packager”, if this helps you :)

> > we passed something like --upstream-vcs-tag=$NEW_UPSTREAM_VERSION to gbp import-orig — pretty nifty, uh? :) See the tails-iuk release process for a real-world example.
>
> sounds good, i did not yet implement it in the doc before testing it.

I see you’ve now implemented it in the doc. Thanks!

Leaving this assigned to you for now, until the pristine-tar thing is documented here and my comments above are addressed.

#15 Updated by Anonymous 2015-06-28 10:03:49

  • Target version changed from Tails_1.4.1 to Tails_1.5

#16 Updated by Anonymous 2015-07-07 03:12:49

  • % Done changed from 30 to 60
  • QA Check changed from Dev Needed to Ready for QA

#17 Updated by Anonymous 2015-07-14 04:25:08

  • Assignee set to intrigeri

#18 Updated by intrigeri 2015-07-18 04:51:58

  • Assignee deleted (intrigeri)
  • QA Check changed from Ready for QA to Dev Needed

The last change I see on that branch is from June 1, so likely you forgot to push it.

#19 Updated by Anonymous 2015-07-18 07:50:52

  • Assignee set to intrigeri
  • QA Check changed from Dev Needed to Ready for QA

sorry about that, probably due to the broken git repo.. pushed it again.

#20 Updated by intrigeri 2015-07-18 09:13:31

  • Assignee deleted (intrigeri)
  • QA Check changed from Ready for QA to Dev Needed
  • Feature Branch changed from 451f:doc/installerpkg to doc/8555-update-installer-release-doc

> sorry about that,

No problem.

> probably due to the broken git repo..

Right.

> pushed it again.

Thanks! I can see that you put a lot of work into it, and the result looks good.

I’ve merged current master into that branch (they had diverged quite a bit), added a few minor changes on top, and pushed it => please have a look to my changes, and please base the next steps of your work on it.

Remaining comments:

  • It feels redundant and error-prone to manually set UPSTREAM_PKG_BRANCH and PACKAGING_BRANCH, and then manually check that gbp.conf is correct. I suggest to instead: 1. check gbp.conf, and then, if correct: 2. set the environment variables we need from gbp.conf, e.g. with something like PACKAGING=BRANCH=$(gbp config buildpackage.debian-branch | sed -r -e 's,.*=,,') etc.
  • At least one example in the doc is misleading: “The new upstream version should be something like 4.x” — currently, it should instead be 3.x. I suggest checking for other Tails/Jessie -specific examples (I see at least PACKAGING_BRANCH=tails/jessie somewhere).
  • In the same sentence, I don’t understand what “based on the upstream branch” means => please clarify.
  • Please use && between all commands that are meant to be copy’n’pasted and run in batch. There are a few examples thereof in that file :)
  • I’m unsure about the If gbp buildpackage complains about a missing upstream/$VERSION [...] paragraph: now that you’ve bootstrapped usage of pristine-tar, when can this happen?
  • wiki/src/contribute/release_process/liveusb-creator/topic_branch.mdwn was not updated yet; once done, please fix the remaining XXX (hint: it’s generally a good idea to re-read the ticket description before submitting it for QA ;)
  • Instructions for preparing a package for Debian (sid, backports) are missing. I guess that bits of Feature #8549#note-3 can be used as a draft.

#21 Updated by Anonymous 2015-07-25 04:02:16

Left to do:

wiki/src/contribute/release_process/liveusb-creator/topic_branch.mdwn was not updated yet; once done, please fix the remaining XXX (hint: it’s generally a good idea to re-read the ticket description before submitting it for QA ;)

#22 Updated by Anonymous 2015-08-04 07:51:46

  • Target version changed from Tails_1.5 to Tails_1.6

#23 Updated by Anonymous 2015-09-22 15:24:51

  • Target version changed from Tails_1.6 to Tails_1.7

#24 Updated by intrigeri 2015-10-02 15:23:44

  • Assignee set to intrigeri
  • % Done changed from 60 to 70
  • QA Check changed from Dev Needed to Ready for QA

We’re close to completion on this one!

U.: note that our up-to-date work lives on the doc/8555-update-installer-release-doc branch, in the main Tails Git repository. Please merge from there if you work on it again.

Left to do:

  • review the topic branch packaging updated doc;
  • review the entire update doc from the PoV of a Tails release manager.

I’m going to do that.

And then, anonym will test and validate the new thing during the 1.7~rc1 release process.

#25 Updated by intrigeri 2015-10-02 17:02:13

Applied in changeset commit:565eaae0685ff475cb80a7ebe6ddc6037899ae58.

#26 Updated by intrigeri 2015-10-02 17:04:30

  • Assignee changed from intrigeri to anonym
  • % Done changed from 70 to 80

Merged into master so that anonym cannot possibly forget to follow this updated doc during the 1.7~rc1 release process.

anonym: if you’re happy with it when preparing 1.7~rc1, please close this ticket. Otherwise, feel free to ask U and/or myself for help, clarification, or doc fixes. Cheers!

#27 Updated by intrigeri 2015-10-02 17:38:48

  • blocked by deleted (Feature #8557: Have Tails Installer uploaded and accepted into Debian)

#28 Updated by anonym 2015-10-06 02:48:25

intrigeri wrote:
> Merged into master so that anonym cannot possibly forget to follow this updated doc during the 1.7~rc1 release process.
>
> anonym: if you’re happy with it when preparing 1.7~rc1, please close this ticket. Otherwise, feel free to ask U and/or myself for help, clarification, or doc fixes. Cheers!

I’ll do this for the freeze/RC (or, if I can remember, a few days before) so that if any clarifications are needed we have plenty of time to get that sorted out for the final 1.7 release.

#29 Updated by anonym 2015-10-26 10:53:34

  • Status changed from In Progress to Resolved
  • Assignee deleted (anonym)
  • % Done changed from 80 to 100
  • QA Check changed from Ready for QA to Pass

Done. I could follow the instructions just fine, but had two issues:

gbp import-orig \
       --upstream-vcs-tag="tails-installer_$NEW_UPSTREAM_VERSION" \
       ../tarballs/liveusb-creator_$NEW_UPSTREAM_VERSION+dfsg.orig.tar.gz


When I ran that I got:

gbp:error: 
Repository does not have branch 'upstream/3.x+dfsg' for upstream sources. If there is none see                                                                    
file:///usr/share/doc/git-buildpackage/manual-html/gbp.import.html#GBP.IMPORT.CONVERT                                                                             
on howto create it otherwise use --upstream-branch to specify it.


so I had to git checkout upstream/3.x+dfsg && git checkout master, and then it worked. Tracked as Feature #10425.

Also,

$ gbp dch 
gbp:error: Version 3.11.6+tails1-20 not found


So I had to manually create a tails/3.11.6+tails1-20 of the same commit as the old debian/3.11.6+tails1-20, but following the instructions then created the correct tails/... tag so this was a one time workaround.

#30 Updated by intrigeri 2015-10-26 11:12:56

> Done. I could follow the instructions just fine, but had two issues:

I’m delighted to read this :)

U.: congrats!

#31 Updated by intrigeri 2015-10-31 08:12:29

  • related to Feature #10425: Clarify needed branches in Tails Installer release documentation and update doc to rename added

#32 Updated by intrigeri 2015-10-31 08:12:50

  • Subject changed from Adapt packaging process and release doc for Tails Installer rename and DFSG-free tarball to Adapt packaging process and release doc for Tails Installer DFSG-free tarball