Feature #8554: Repack a DFSG-free Tails Installer tarball

#1 Updated by intrigeri 2015-01-06 13:56:47

blocked by ~~Feature #8553~~: Adapt Tails Installer packaging to its rename added

#2 Updated by intrigeri 2015-01-06 13:58:05

blocks ~~Feature #8555~~: Adapt packaging process and release doc for Tails Installer DFSG-free tarball added

#3 Updated by intrigeri 2015-01-06 14:47:46

blocks #8538 added

#4 Updated by Anonymous 2015-02-26 15:08:38

Target version changed from Hardening_M1 to Tails_1.4

#5 Updated by intrigeri 2015-05-06 08:56:42

Status changed from Confirmed to In Progress
% Done changed from 0 to 10

I see there’s progress on this one in u’s installer/debian_feature-jessie_u branch. Assuming that’ll all that needs to be done in the packaging itself, next step is probably to create a branch that updates the corresponding section of wiki/src/contribute/release_process/liveusb-creator.mdwn.

#6 Updated by Anonymous 2015-05-07 21:34:20

Here are my notes. I’d be glad to get some feedback before modifying the documentation (i believe that this is ~~Feature #8555~~?)

First of all, I’ve based my branch on installer/debian_feature-jessie:

git checkout installer/debian_feature-jessie

I’ve added the files which need to be excluded in debian/copyright, using the “Files-Excluded” directive.
This then results in:

git diff --stat feature/jessie..installer/debian_feature-jessie_clean  | grep -Ev '^\s+debian/'
# => 11 files changed, 785 insertions(+)

As documented on https://tails.boum.org/contribute/release_process/liveusb-creator/#index4h1, i then create a new upstream tarball:

export NEW_UPSTREAM_VERSION=3.11.6+tails1+dfsg-16+jessie4
git archive --output=../liveusb-creator_$NEW_UPSTREAM_VERSION.orig.tar.gz feature/jessie

And then run mk-origtargz. (If we had a watch file, uscan could do the job, as a sidenote.)

mk-origtargz --copy -S "-16+jessie" ../liveusb-creator_$NEW_UPSTREAM_VERSION.orig.tar.gz
#Leaving ../liveusb-creator_3.11.6+tails1+dfsg-16+jessie.orig.tar.gz where it is, deleting 6 files from it.

Then I edited debian/gbp.conf (debian branch) and committed the changes.

Then import the tarball and automatically create a tag:

gbp import-orig ../liveusb-creator_$NEW_UPSTREAM_VERSION.orig.tar.gz
# What is the upstream version?
#gbp:info: Importing '../liveusb-creator_3.11.6+tails1+dfsg.orig.tar.gz' to branch 'feature/jessie'...
#gbp:info: Source package is liveusb-creator
#gbp:info: Upstream version is 3.11.6+tails1+dfsg
#gbp:info: Merging to 'installer/debian_feature-jessie_clean'
#gbp:info: Successfully imported version 3.11.6+tails1+dfsg of ../liveusb-creator_3.11.6+tails1+dfsg.orig.tar.gz

And from there i can build:

DIST=unstable pdebuild

I needed to clean the branches too, when something failed, as well as delete the tag created by gbp import-orig

git reset origin/feature/jessie
git checkout -- tools/7-Zip-License.txt  tools/7z.dll tools/7z.exe tools/dd.exe tools/syslinux.exe

#7 Updated by Anonymous 2015-05-07 21:35:23

Assignee set to intrigeri
Feature Branch set to installer/debian_feature-jessie_clean

#8 Updated by intrigeri 2015-05-07 23:11:00

QA Check set to Ready for QA

> Here are my notes.

Thanks, that’ll help me review the rest of your work.

> I’d be glad to get some feedback before modifying the documentation (i believe that this is ~~Feature #8555~~?)

Right, that’s ~~Feature #8555~~.

#9 Updated by intrigeri 2015-05-08 12:33:14

Feature Branch changed from installer/debian_feature-jessie_clean to 451f:installer/debian_feature-jessie_clean

(Setting non-ambiguous feature branch.)

#10 Updated by intrigeri 2015-05-08 13:16:06

Assignee deleted (~~intrigeri~~)
QA Check changed from Ready for QA to Dev Needed

I had a look, and here are a few initial comments.

First of all, great that you mastered mk-origtargz and friends. I myself never used them before, so I’m learning too. Thanks :)

> export NEW_UPSTREAM_VERSION=3.11.6+tails1+dfsg-16+jessie4

Well, no, that’s not a new upstream version number, as it embeds the packaging version (-16+jessie4). Also, I don’t think that the resulting archive should have +dfsg in its name, because… it’s not DFSG-compliant yet. So basically, in this respect the currently existing documentation is correct => just follow it :)

> git archive --output=../liveusb-creator_$NEW_UPSTREAM_VERSION.orig.tar.gz feature/jessie

The .orig filename component lets one believe that this tarball is meant to be used for packaging or uploading to Debian, which is not the case given we’re going to repack it => I suggest dropping that component from the non-repacked tarball’s name.

> mk-origtargz --copy -S "-16+jessie" ../liveusb-creator_$NEW_UPSTREAM_VERSION.orig.tar.gz

Here too, -S "-16+jessie" feels wrong: I guess you added it because mk-origtargz@ rightfully drops the Debian packaging version suffix that was erroneously added, and then we should not force it to re-add it (or rather part of it, since “4” disappeared for some reason — anyway).

Perhaps you did all this because of the above mistake (NEW_UPSTREAM_VERSION), so once that first problem is fixed, you should retry this step without the -S option, or more likely with -S '+dfsg' if that’s not the default. Or, perhaps even better, -S '+dfsg.orig', so that in the end the “upstream” tarball that’ll be used for packaging is called the way it should be, that is liveusb-creator_3.11.6+tails1+dfsg.orig.tar.gz.

> DIST=unstable pdebuild

Please use gbp buildpackage (aka. git-buildpackage on older Debian) instead, to match the workflow we’re using in all our Debian packages.

To end with, once these issues are resolved, please switch pristine-tar to True in debian/gbp.conf, so that we can everything that’s needed to re-generate upstream tarballs via Git. Otherwise it’ll be a pain for you to deliver the results of your work, and in the future it’ll be painful every time we have to share tarballs.

> I needed to clean the branches too, when something failed, as well as delete the tag created by gbp import-orig

I’m not sure what the following commands are about. I guess they won’t be needed once the previous steps are fixed and polished, so I’ll ignore them for now :)

#11 Updated by Anonymous 2015-05-11 10:02:43

thanks a lot for the review, I’ve updated my notes and will update the wiki too.

# Edit .git/gbp.conf
export NEW_UPSTREAM_VERSION=3.11.6+tails1
git archive --output=../liveusb-creator_$NEW_UPSTREAM_VERSION.tar.gz feature/jessie
mk-origtargz --copy ../liveusb-creator_$NEW_UPSTREAM_VERSION.tar.gz #orig and dfsg are added automatically
git-buildpackage

#12 Updated by Anonymous 2015-05-11 18:38:46

As a sidenote, I also still needed to configure `gbp` correctly so that the package would be built in a pbuilder chroot.

#13 Updated by Anonymous 2015-05-11 19:43:34

Assignee set to intrigeri
% Done changed from 10 to 20
QA Check changed from Dev Needed to Ready for QA

I’ve pushed a signed git tag as mentioned in the documentation. Do I need to upload this tarball somewhere?
I’ll let you check if there are still missing bits?

#14 Updated by Anonymous 2015-05-12 10:03:49

Assignee deleted (~~intrigeri~~)
QA Check deleted (~~Ready for QA~~)

#15 Updated by Anonymous 2015-05-12 20:33:54

Target version changed from Tails_1.4 to Tails_1.4.1

#16 Updated by intrigeri 2015-05-29 16:05:36

Assignee set to intrigeri
QA Check set to Dev Needed

First, the upstream branch (the one used to work on upstream code and release new upstream release) is your feature/jessie, right? (If not, great parts of what I’m writing below is probably wrong.)

> I’ve pushed a signed git tag as mentioned in the documentation.

I see upstream/3.11.6+tails1+dfsg (which is apparently tagging the result of importing the DFSG-freed tarball with gbp import-orig), but there are a few problems:

We’ve already released 3.11.6+tails1, so you can’t re-release something different (that includes e.g. the port to udisks2, and your recent upstream changes) with the same version name a year later. Hold on, I’ll be discussing versioning/branching below.
I can see no tag for the new upstream (not-DFSG-free) release; see my review of the doc bits on ~~Feature #8555~~, where I just clarified this topic.
That tag lives on the branch used for upstream work, which is wrong; it should live on the branch used as “upstream” by the Debian packaging process (the one gbp import-orig should import DFSG-freed tarballs into); where is that branch?

And then, the commit that tag points to was reverted; this is actually correct, since the DFSG-freed tarball was not imported into the right branch. But then, the tag itself needs to be fixed as well.

> Do I need to upload this tarball somewhere?

I’m not sure which one of the two tarballs the entire process deals with you’re talking of. So I’ll answer for both:

Regarding the DFSG-freed tarball: no need, it’ll be handled by the move to using pristine-tar once it’s been implemented (but hold on, let’s come up with a branching/versioning scheme that works, first).
Regarding the upstream one: iirc we’ve discussed this topic over email a month or three ago, and decided that tagging releases would be enough (which is consistent with the removal of debian/watch). Please look up that past discussion and correct me if I’m wrong.

> I’ll let you check if there are still missing bits?

It’s now clear that to repack, we need both:

a suitable upstream tarball; that’s the call to git archive, that seems fine on your doc branch;
a suitable version number; that part hasn’t been thought through yet; I’ll spare us some lenghty guess/review/correct loop, and will propose something later today on this ticket. Then you can review my proposal for flaws.

And to import the repacked tarball, we also need a suitable branching setup, which quite clearly we’re missing so far, and hence the confusion I’ve highlighted above. I’ll also propose something.

#17 Updated by intrigeri 2015-05-30 19:58:28

Assignee deleted (~~intrigeri~~)

See ~~Feature #8549#note-3~~.

#18 Updated by Anonymous 2015-06-28 10:04:01

Target version changed from Tails_1.4.1 to Tails_1.5

#19 Updated by Anonymous 2015-07-14 04:27:58

Assignee set to intrigeri
% Done changed from 20 to 50
QA Check changed from Dev Needed to Ready for QA
Feature Branch changed from 451f:installer/debian_feature-jessie_clean to 451f:upstream/4.1+dfsg

also see branch tails/jessie

#20 Updated by intrigeri 2015-07-18 04:52:46

blocks deleted (~~~~Feature #8553~~: Adapt Tails Installer packaging to its rename~~)

#21 Updated by intrigeri 2015-07-18 04:54:08

Assignee deleted (~~intrigeri~~)
QA Check changed from Ready for QA to Dev Needed

There’s no upstream/4.1+dfsg branch in your repo, please push it :)

#22 Updated by Anonymous 2015-07-18 07:54:37

Assignee set to intrigeri
QA Check changed from Dev Needed to Ready for QA

sorry again. pushed.

#23 Updated by intrigeri 2015-07-18 08:36:39

Assignee deleted (~~intrigeri~~)
QA Check changed from Ready for QA to Dev Needed
Feature Branch changed from 451f:upstream/4.1+dfsg to 451f:upstream/4.x+dfsg

Cool, I see you’ve pushed the upstream/4.1+dfsg tag, which was the right thing to do given I mistakenly asked you to push that branch, which was incorrect.

However, I don’t see the upstream/4.x+dfsg branch, that we point gbp to.

And also, I see no tags for:

the two new upstream (not-DFSG-free) releases you’ve put out (sorry, I should have noticed this last time I commented wrt. stuff that hadn’t been pushed yet). According to the doc you’ve updated, they should be called tails-installer_4.0 and tails-installer_4.1, right?
upstream/4.0+dfsg: importing the DFSG-freed tarball with gbp import-orig should have created it.

Please push these refs, and by the way, I suggest making sure that the updated documentation instructs the upstream/Debian maintainers to push them all, so that nobody forgets in the future — thanks :)

#24 Updated by Anonymous 2015-07-22 13:37:31

Assignee set to intrigeri
QA Check changed from Dev Needed to Ready for QA

intrigeri wrote:
> Cool, I see you’ve pushed the upstream/4.1+dfsg tag, which was the right thing to do given I mistakenly asked you to push that branch, which was incorrect.
>
> However, I don’t see the upstream/4.x+dfsg branch, that we point gbp to.
>
> And also, I see no tags for:
>
> * the two new upstream (not-DFSG-free) releases you’ve put out (sorry, I should have noticed this last time I commented wrt. stuff that hadn’t been pushed yet). According to the doc you’ve updated, they should be called tails-installer_4.0 and tails-installer_4.1, right?
> * upstream/4.0+dfsg: importing the DFSG-freed tarball with gbp import-orig should have created it.

pushed the refs again.

#25 Updated by intrigeri 2015-07-23 02:37:56

Assignee deleted (~~intrigeri~~)
% Done changed from 50 to 60
QA Check changed from Ready for QA to Dev Needed

Thanks!

It seems that --upstream-vcs-tag was not passed to gbp import-orig, contrary to what the doc (rightfully) says: git log upstream/4.1+dfsg..tails-installer_4.1 shows some commits that would be part of the upstream/4.1+dfsg tag if the correct option had been passed.
I believe Lintian should warn you about too long lines in debian/changelog. But maybe it needs to be run with --pedantic? In any case, those lines should be fixed.
debian/changelog has a merge conflict marker: >>>>>>> debian_feature-jessie
I see a dependency on python-qt4: I think this is the result of a mistake when resolving a merge conflict.
debian/copyright: it’s a bit unclear if data/usb-creator-gtk.svg is under GPL v3 or v3+.

(Sorry, general packaging comments are mixed up with the rest, and so off-topic here, but oh well.)

#26 Updated by Anonymous 2015-07-25 01:51:21

intrigeri wrote:
> Thanks!
>
> * It seems that --upstream-vcs-tag was not passed to gbp import-orig, contrary to what the doc (rightfully) says: git log upstream/4.1+dfsg..tails-installer_4.1 shows some commits that would be part of the upstream/4.1+dfsg tag if the correct option had been passed.

correct. I’ll delete and recreate the tag.

> * I see a dependency on python-qt4: I think this is the result of a mistake when resolving a merge conflict.

correct, removed.

> * debian/copyright: it’s a bit unclear if data/usb-creator-gtk.svg is under GPL v3 or v3+.

README.txt states GPLv3, not v3+.

> (Sorry, general packaging comments are mixed up with the rest, and so off-topic here, but oh well.)

no problem.

#27 Updated by intrigeri 2015-08-03 05:34:57

>> * debian/copyright: it’s a bit unclear if data/usb-creator-gtk.svg is under GPL v3 or v3+.

> README.txt states GPLv3, not v3+.

Yes. That’s part of the problem, though:

The GPL-3 snippet that’s included in debian/copyright says “either version 3 of the License, or (at your option) any later version”. In my book, this means GPL-3+, not GPL-3.
README.txt indeed says GPLv3, which is not very precise: IIRC, the default GPLv3 text has the “any later version” text (please check that, I’m now unsure).

So first step is to determine whether upstream license is “any later version” or not. And then, have debian/copyright reflect that in a consistent way. Yay!

#28 Updated by Anonymous 2015-08-04 08:35:51

Target version changed from Tails_1.5 to Tails_1.6

#29 Updated by Anonymous 2015-08-26 10:49:16

intrigeri wrote:
> >> * debian/copyright: it’s a bit unclear if data/usb-creator-gtk.svg is under GPL v3 or v3+.
>
> > README.txt states GPLv3, not v3+.
>
> Yes. That’s part of the problem, though:
>
> * The GPL-3 snippet that’s included in debian/copyright says “either version 3 of the License, or (at your option) any later version”. In my book, this means GPL-3+, not GPL-3.
> * README.txt indeed says GPLv3, which is not very precise: IIRC, the default GPLv3 text has the “any later version” text (please check that, I’m now unsure).
>
> So first step is to determine whether upstream license is “any later version” or not. And then, have debian/copyright reflect that in a consistent way. Yay!

Upstream says “GPL-3” : https://bazaar.launchpad.net/~usb-creator-hackers/usb-creator/trunk/view/head:/debian/copyright

So I leave this as is.

#30 Updated by Anonymous 2015-08-26 11:05:49

intrigeri wrote:
> * It seems that --upstream-vcs-tag was not passed to gbp import-orig, contrary to what the doc (rightfully) says: git log upstream/4.1+dfsg..tails-installer_4.1 shows some commits that would be part of the upstream/4.1+dfsg tag if the correct option had been passed.

to be fixed

> * I believe Lintian should warn you about too long lines in debian/changelog. But maybe it needs to be run with --pedantic? In any case, those lines should be fixed.