Feature #9788

Take benefit from squashfs-tools 4.3 for faster test builds

Added by romeopapa 2015-07-20 11:58:19 . Updated 2019-01-29 14:40:35 .

Status:
Resolved
Priority:
Low
Assignee:
Category:
Build system
Target version:
Tails_3.0
Start date:
2015-07-20
Due date:
% Done:

100%

Feature Branch:
Type of work:
Code
Blueprint:

Starter:
Affected tool:
Deliverable for:

Description

Initial discussion and benchmarks: https://mailman.boum.org/pipermail/tails-dev/2015-July/009241.html

squashfs-tools 4.3 allows setting the gzip compression level. This means faster test builds.

Once our Vagrant basebox is based on Stretch, we can modify the gzipcomp build option to use a lower gzip compression level.

Subtasks

Related issues

Related to Tails - Bug #16177: Building ISO in RAM often fails: mksquashfs reaped by OOM killer Resolved 2018-12-02

History

#1 Updated by romeopapa 2015-07-21 10:19:00

Ok I’ve submitted a big debdiff for squashfs-tools that updates to the latest source code, updates the manual pages and a patch that fixes a couple of opened CVEs. Hopefully it should be quick to get into sid!

#2 Updated by romeopapa 2015-07-24 05:19:31

Turns out I didn’t send the bug report correctly, so nothing happened: Don’t send an email to the bug submission with a 300K file attached, it’ll just be ignored.

Here are the 3 bugs listed at the debian’s bug tracker:

CVE-2015-4645:
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=793467

CVE-2015-4646:
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=793468

Update to 4.3+20140919 and man pages update:
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=793468

It’s a little bit messy, but now they have everything they need to update the package!

#3 Updated by romeopapa 2015-07-24 05:22:09

> Update to 4.3+20140919 and man pages update:
> -https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=793468-

Update to 4.3+20140919 and man pages update:
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=793480

Little typo fix.

#4 Updated by intrigeri 2015-08-03 05:15:11

  • Status changed from New to In Progress
  • Assignee set to romeopapa
  • % Done changed from 0 to 10

#5 Updated by intrigeri 2015-08-03 05:15:53

  • Subject changed from squashfs-tool 4.3 (faster test builds) to Use squashfs-tool 4.3 for faster test builds

#6 Updated by intrigeri 2015-09-14 03:05:18

4.3 is now in testing/sid, and could in turn be backported for Jessie.

#7 Updated by intrigeri 2016-02-29 23:38:37

  • Subject changed from Use squashfs-tool 4.3 for faster test builds to Use squashfs-tools 4.3 for faster test builds

Any progress on this front? Personally, I’m tempted to just wait until we switch our build system to Stretch and get this newer mksquashfs’ features for free, but if someone feels like doing the work to get it earlier, well, great :)

#8 Updated by romeopapa 2016-04-09 03:51:56

Hey I’m back I’m gonna check out the status of the upstream patches i had submitted.

#9 Updated by intrigeri 2016-04-10 03:19:48

> Hey I’m back I’m gonna check out the status of the upstream patches i had submitted.

Great!

#10 Updated by intrigeri 2016-06-07 15:10:36

  • Description updated

#11 Updated by intrigeri 2016-06-07 15:14:34

So, 4.3 is now in testing, great! So we’ll have this for free when we migrate to Stretch (and it would be nice if someone tried this, e.g. when building our feature/stretch branch, on more powerful hardware), so this can become an easy goal for Tails 3.0. Now, if someone is excited about it and wants it earlier, then they need to have the package into jessie-backports. romeopapa, what are your plans?

#12 Updated by intrigeri 2016-07-31 07:24:01

  • Subject changed from Use squashfs-tools 4.3 for faster test builds to Take benefit from squashfs-tools 4.3 for faster test builds
  • Description updated
  • Assignee deleted (romeopapa)
  • Priority changed from Normal to Low
  • Target version set to Tails_3.0
  • Type of work changed from Debian to Code

intrigeri wrote:
> So, 4.3 is now in testing, great! So we’ll have this for free when we migrate to Stretch (and it would be nice if someone tried this, e.g. when building our feature/stretch branch, on more powerful hardware), so this can become an easy goal for Tails 3.0. Now, if someone is excited about it and wants it earlier, then they need to have the package into jessie-backports. romeopapa, what are your plans?

Time has passed so I’m adjusting this ticket’s description to make it clear it’s about Stretch.

#13 Updated by intrigeri 2016-08-25 04:21:04

  • Status changed from In Progress to Resolved
  • % Done changed from 10 to 100

Applied in changeset commit:729235f61ed9e489e7935a99b3fcd8000337dad4.

#14 Updated by intrigeri 2019-01-29 14:39:40

  • related to Bug #16177: Building ISO in RAM often fails: mksquashfs reaped by OOM killer added

#15 Updated by intrigeri 2019-01-29 14:40:35

FTR this upgrade causes Bug #16177, because mksquashfs will now use by default 25% of the physical memory, which is way too much when most of that memory is used by the tmpfs that backs the build directory.

Lesson learnt: next time, read the changelog :)