Feature #5630

Reproducible builds

Added by Tails 2013-07-18 07:43:45 . Updated 2019-01-09 17:47:32 .

Status:
Resolved
Priority:
Normal
Assignee:
Category:
Build system
Target version:
Start date:
2015-09-23
Due date:
% Done:

100%

Feature Branch:
Type of work:
Code
Blueprint:

https://tails.boum.org/blueprint/reproducible_builds/

Starter:
0
Affected tool:
Deliverable for:
301

Description

To ensure integrity against build machine or developer compromise, we should be able to produce identical binaries when building the same source on two different (but possibly identically configured) machines.

Team: anonym, lamby, bertagaz, u, kibi and intrigeri

Subtasks

Bug #10232: Investigate vagrant-lxc for our build system Rejected

100
Feature #11966: Reproducible website build Resolved

100
Feature #11967: refresh-translations: don't update PO files unless something other POT-Creation-Date has changed Resolved

100
Bug #11970: sys-subsystem-net-devices-multi-user.device/start times out Resolved

100
Feature #11971: Consider migrating some of /lib/live/config/* to systemd unit files Resolved

100
Feature #11972: Switch our Jenkins ISO build system to vagrant-libvirt Resolved

100
Feature #11974: Reproducible IUK builds Resolved

100
Feature #11976: Migrate to mksquashfs that honors $SOURCE_DATE_EPOCH Resolved

100
Feature #11979: Move Vagrant's apt-cacher-ng data to a dedicated disk Resolved

100
Feature #11980: Create and provision a new Vagrant VM for every ISO build Resolved

100
Feature #11981: Delete the Vagrant VM used for an ISO build once it is finished Resolved

100
Feature #11982: Set up processes to update the Vagrant ISO build basebox Resolved

100
Feature #11983: Check if the test suite has more failures on the reproducible ISO Resolved

100
Bug #11986: Generated APT auto-removal config file encodes the build system's kernel version Resolved

100
Bug #11987: ikiwiki includes "Posted" timestamps in some generated web pages Resolved

100
Feature #11988: Update Vagrant boxes management design doc with meeting notes Resolved

100
Feature #12002: Estimate hardware cost of reproducible builds in Jenkins Resolved

100
Bug #12032: The SquashFS creation is not deterministic Resolved

100
Feature #12051: Use ikiwiki 3.20161219+ instead of our patched one Resolved

100
Bug #12329: /usr/share/locale/*/LC_MESSAGES/tails.mo's POT-Creation-Date depends on the build time Resolved

100
Bug #12330: initrd.img is not generated reproducibly Rejected

100
Feature #12338: Test ISO build reproducibility with varying number of CPU cores Resolved

100
Feature #12339: Have the ISO build reproducibility regardless of the current time Resolved

100
Feature #12345: Test ISO build reproducibility with varying CPU type Resolved

100
Feature #12348: Review'n'merge the reproducible builds branch into feature/stretch Resolved

100
Feature #12351: Test building an ISO with a fake system time set in 2018 Resolved

100
Feature #12352: Error out if trying to build an ISO with a system time before SOURCE_DATE_EPOCH Resolved

100
Feature #12409: Reconsider the need for publishing Vagrant baseboxes Resolved

100
Bug #12453: Invalid MBR ID passed to isohybrid Resolved

100
Bug #12527: Duplicate gpg-agent killing code introduced in the build-tails script Resolved

100
Bug #12529: Vagrant box creation needlessly downloads Linux from debian-security Resolved

100
Bug #12530: Vagrant box creation fails: can't unmount chroot that's busy Resolved

100
Bug #12531: ISO builds on Jenkins are fragile since the migration to vagrant-libvirt Resolved

100
Bug #12541: isobuilders memory check keeps switching between OK and WARNING since the switch to Vagrant Resolved

100
Bug #12565: Test failures on Jenkins due to lack of disk space Resolved

100
Bug #12566: ikiwiki image size specification makes the ISO build unreproducible Resolved

100
Bug #12567: fontconfig cache is not generated reproducibly even with patch from Debian#857892 Resolved

100
Bug #12574: isobuilders system_disks check keeps switching between OK and WARNING since the switch to Vagrant Resolved

100
Bug #12575: Fix basebox:clean_old Resolved

100
Feature #12576: Have Jenkins use basebox:clean_old instead of basebox:clean_all Resolved

100
Bug #12579: reproducibly_build_Tails_ISO_* Jenkins job are broken Resolved

100
Bug #12595: Not enough space in /var/lib/jenkins on isobuilders Resolved

100
Bug #12606: Better balance our isobuilders' I/O load over all available SSDs Resolved

100
Feature #12608: Analyze what's still not reproducible on current testing branch Resolved

100
Feature #12616: Document our vagrant based build setup in Jenkins Resolved

100
Bug #12619: /usr/share/doc/tails/website/torrents/rss.html is not reproducible Resolved

100
Bug #12620: /usr/local/lib/tor-browser/omni.ja embeds build timestamp Resolved

100
Feature #12625: Make Ikiwiki resize images deterministically Resolved intrigeri

100
Feature #12626: Report back to the reproducible builds community about how we did it Resolved intrigeri

100
Feature #12628: Draft a "user" (aka. RM) story for the reproducible release process Duplicate

0
Feature #12630: Document how users can verify a reproducibly built ISO/IUK Resolved

100
Feature #12633: Lower the workload caused by reproducible builds Jenkins jobs Resolved bertagaz

100
Bug #12637: Deploy rake libvirt volumes clean up task on all Jenkins build jobs Resolved

100
Bug #12641: Comment changes in POT files make ISO builds non-reproducible Resolved

100
Feature #12654: Introduce more variations in our reproducibility CI tests Resolved intrigeri

100
Bug #12681: reproducibly_build_Tails_ISO_* Jenkins jobs are buggy when building from a tag Resolved

100
Feature #12715: Decide what builds we will try to reproduce in Jenkins Resolved

100
Bug #12725: Sort out the apt-snapshots-disk partition situation on apt.lizard Resolved

100
Bug #12726: There should be a date on the notes in the News section of the website Resolved

100
Bug #12735: live/initrd.img not reproducible in some environments Resolved

100
Bug #12736: live/vmlinuz not reproducible in some environments Duplicate lamby

100
Bug #12738: Remove gconf Resolved

100
Bug #12737: utils/ not reproducible in some environments Duplicate

0
Bug #12739: Metadata for directories inside the squashfs not reproducible in some environments Rejected

0
Bug #12740: Various .cache files not reproducible in some environments Resolved

100
Bug #12741: /lib/modules/*/modules.* not reproducible in some environments Resolved

100
Feature #13436: Have Jenkins jobs that reproduce ISOs when a branch ticket is Ready for QA Resolved

100
Bug #13504: Rebase our custom squashfs-tools package on 1:4.3-3+deb9u1 Rejected

100
Bug #13531: Use ikiwiki 3.20161219+, again Duplicate

50
Bug #13623: Executable bits of /etc/hostname not set deterministically Resolved

100
Feature #13624: Analyze results of Tails 3.1 call for reproduction Resolved

0
Feature #14512: Send second email call to test reproducibility of Tails 3.2alpha1 Resolved

100
Feature #14520: Prepare & publish a blog post about testing Tails ISO reproducibility Resolved intrigeri

100
Feature #14607: Analyze results of Tails 3.2~alpha1 call for reproduction Resolved

0
Bug #14729: Fix gdk-pixbuf vulnerability (CVE-2017-2862) Resolved

100
Feature #14756: Drop update-ca-certificates.service Resolved

100
Feature #14757: Final report for SponsorT 2016 Resolved

90
Bug #14767: ikiwiki does not order news items deterministically Resolved

100
Bug #14924: reproducibly_build_Tails_ISO_stable Jenkins job always fails Resolved intrigeri

100
Bug #14933: stable branch is not reproducible: differences in some .fa.html website files Resolved

100
Bug #14944: jenkins-data-disk is running out of diskspace Resolved groente

100
Bug #14946: Topic branches that lag behind their base branch don't build reproducibly with mergebasebranch build option Resolved

100
Bug #15141: reproducibly_build_Tails_ISO_stable job erroneously fails when build reproduction succeeded Resolved

100

Related issues

Related to Tails - Feature #8511: Have all Debian packages we use build in a deterministic way Confirmed 2015-01-01
Related to Tails - Feature #7100: Decide what to do with machine-id Confirmed 2014-04-16
Blocked by Tails - Bug #8125: Self-host the Tor Browser tarballs we need Resolved 2014-10-15
Blocked by Tails - Bug #9416: Stop shipping ssl-cert-snakeoil in the ISO Resolved 2015-05-17
Blocked by Tails - Bug #9419: eatmydata is not being used in the build chroot Resolved 2015-05-17
Blocked by Tails - Bug #11273: clean up libdvd-pkg build files Resolved 2016-03-21

History

#1 Updated by intrigeri 2013-07-19 01:47:39

  • Type of work set to Code

Type of work: Code

#2 Updated by BitingBird 2014-06-09 12:15:55

  • Description updated
  • Category set to Build system
  • Starter set to No

#3 Updated by intrigeri 2014-11-01 20:34:43

  • blocked by Bug #8125: Self-host the Tor Browser tarballs we need added

#4 Updated by intrigeri 2015-01-01 22:58:41

  • related to Feature #8511: Have all Debian packages we use build in a deterministic way added

#5 Updated by intrigeri 2015-05-17 09:52:35

  • Feature Branch set to feature/5630-deterministic-builds

#6 Updated by intrigeri 2015-05-17 09:55:59

Requires our tails/debian-old-2.0+faketime branch of live-build, otherwise faketime has no effect on the commands run in the chroot set up by live-build.

#7 Updated by intrigeri 2015-05-17 10:05:21

  • blocked by Bug #9416: Stop shipping ssl-cert-snakeoil in the ISO added

#8 Updated by intrigeri 2015-05-17 10:43:00

  • blocked by Bug #9419: eatmydata is not being used in the build chroot added

#9 Updated by sajolida 2015-08-14 12:14:18

  • Description updated
  • Assignee set to intrigeri
  • Target version set to 2017

#10 Updated by intrigeri 2015-09-27 10:04:52

  • Priority changed from Low to Normal

(It’s on our roadmap now.)

#11 Updated by intrigeri 2015-09-27 10:09:33

  • Description updated

#12 Updated by intrigeri 2016-02-16 13:17:36

  • Subject changed from Deterministic builds to Reproducible builds
  • Blueprint set to https://tails.boum.org/blueprint/reproducible_builds/

#13 Updated by intrigeri 2016-02-16 13:22:11

  • Description updated

#14 Updated by BitingBird 2016-06-27 02:12:02

  • Status changed from Confirmed to In Progress

#15 Updated by intrigeri 2016-06-27 04:49:31

  • Status changed from In Progress to Confirmed
  • Assignee changed from intrigeri to anonym

#16 Updated by intrigeri 2016-11-19 15:59:35

  • Deliverable for set to 289

#17 Updated by intrigeri 2016-11-19 16:21:13

  • Description updated

#18 Updated by intrigeri 2016-11-20 14:06:48

  • related to Feature #7100: Decide what to do with machine-id added

#19 Updated by intrigeri 2016-11-21 14:53:06

  • Description updated

#20 Updated by intrigeri 2016-11-22 14:01:08

  • blocks Feature #11990: In 2019, try reproducing an ISO that was released in 2018 added

#21 Updated by intrigeri 2017-03-13 15:10:21

  • blocked by Bug #11273: clean up libdvd-pkg build files added

#22 Updated by lamby 2017-03-16 07:49:23

fontconfig issues should be resolved with: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=857892

#23 Updated by intrigeri 2017-05-20 08:53:55

  • Status changed from Confirmed to In Progress

Applied in changeset commit:ababfdf960ebed3e648d3ed7f05e9e96dde28922.

#24 Updated by intrigeri 2017-05-24 07:03:57

#25 Updated by intrigeri 2017-06-05 09:38:41

  • Target version changed from 2017 to Tails_3.2
  • Feature Branch deleted (feature/5630-deterministic-builds)

(

  • This branch has nothing interesting now that we generate the fontconfig cache in a reproducible manner. I’ve renamed it to wip/feature/5630-deterministic-builds so it doesn’t eat precious cycles on our CI infra. And anyway the only non-merge commit it has on top of testing is a trivial revert.
  • Setting a target version that’s before the sponsor deadline.

)

#26 Updated by anonym 2017-09-26 12:37:26

  • Target version changed from Tails_3.2 to Tails_3.3

It seems Tails 3.2 is reproducible! Woo!

But we have some testing, documentation and communication tasks remaining so => postponed.

#27 Updated by intrigeri 2017-10-29 08:24:06

  • Deliverable for changed from 289 to 301

(Some subtasks are meant to be done after the end of the contract.)

#28 Updated by anonym 2017-11-15 11:30:46

  • Target version changed from Tails_3.3 to Tails_3.5

#29 Updated by intrigeri 2018-01-02 09:12:43

  • blocked by deleted (Feature #11990: In 2019, try reproducing an ISO that was released in 2018)

#30 Updated by anonym 2018-01-23 19:52:26

  • Target version changed from Tails_3.5 to Tails_3.6

#31 Updated by bertagaz 2018-03-14 11:32:05

  • Target version changed from Tails_3.6 to Tails_3.7

#32 Updated by bertagaz 2018-05-10 11:09:00

  • Target version changed from Tails_3.7 to Tails_3.8

#33 Updated by intrigeri 2018-05-25 13:26:51

  • Assignee changed from anonym to intrigeri

#34 Updated by intrigeri 2018-05-26 09:51:44

  • Assignee changed from intrigeri to bertagaz
  • Target version deleted (Tails_3.8)

(Most of the work that remains to do is on bertagaz’ plate so it does not make sense that this ticket is assigned to me.)

#35 Updated by intrigeri 2019-01-09 17:46:06

  • Status changed from In Progress to Resolved
  • Assignee deleted (bertagaz)