Feature #16285

feature/buster branch is not reproducible

Added by lamby 2019-01-05 10:59:49 . Updated 2019-01-08 14:51:24 .

Status:
Resolved
Priority:
Normal
Assignee:
lamby
Category:
Build system
Target version:
Start date:
2019-01-05
Due date:
% Done:

0%

Feature Branch:
buster
Type of work:
Code
Blueprint:

Starter:
Affected tool:
Deliverable for:

Description


Files


Subtasks


Related issues

Related to Tails - Bug #16310: Buster IUKs are not built reproducibly Resolved 2019-01-06
Related to Tails - Bug #15187: fontconfig cache is not reproducible in Buster Resolved 2018-01-17 2018-09-30
Blocked by Tails - Bug #16294: SquashFS metadata is not reproducible in buster Resolved 2019-01-05
Blocked by Tails - Bug #16295: Various base directories/files have varying mtimes Duplicate 2019-01-05
Blocked by Tails - Bug #16296: .mo files have varying POT-Creation-Date in buster Resolved 2019-01-05
Blocks Tails - Feature #15507: Core work 2019Q1: Foundations Team Resolved 2018-04-08

History

#1 Updated by lamby 2019-01-05 16:25:46

Initial reproducibility check at 6aee6394e64f60f7cf57e2091322b7fc1b86cc98:

SquashFS metadata

├── live/filesystem.squashfs
│ ├── unsquashfs -s {}
│ │ @@ -1,10 +1,10 @@
│ │  Found a valid SQUASHFS 4:0 superblock
│ │ -Creation or last append time Sat Jan  5 12:15:29 2019
│ │ -Filesystem size 1710322.73 Kbytes (1670.24 Mbytes)
│ │ +Creation or last append time Sat Jan  5 14:54:01 2019
│ │ +Filesystem size 1710322.79 Kbytes (1670.24 Mbytes)

Timing should be fixed; filesystem size difference very likely symptom of other issues.

Various base directories/files have varying mtimes (eg. /dev/pts, /etc)

│ ├── unsquashfs -d '' -lls {}
│ │ @@ -176,24 +176,24 @@
[..]
│ │  crw-rw-rw- root/root             1,  9 2018-09-05 16:51 /dev/urandom
│ │  crw-rw-rw- root/root             1,  5 2018-09-05 16:51 /dev/zero
│ │ -drwxr-xr-x root/root              4825 2019-01-05 12:13 /etc
│ │ +drwxr-xr-x root/root              4825 2019-01-05 14:52 /etc
[..]
│ │ --rw-r--r-- root/root                 0 2019-01-05 12:13 /etc/hosts
│ │ +-rw-r--r-- root/root                 0 2019-01-05 14:52 /etc/hosts

Different serial

│ ├── /etc/apt/sources.list.d/torproject.list
│ │ @@ -1 +1 @@
│ │ -deb http://time-based.snapshots.deb.tails.boum.org/torproject/2019010501 buster main
│ │ +deb http://time-based.snapshots.deb.tails.boum.org/torproject/2019010502 buster main

This is generated in ./auto/scripts/apt-snapshots-serials. I think this is a false-positive; ie. someone uploaded something to Debian / snapshots?

fontconfig .uuid files

│ ├── /usr/local/share/fonts/.uuid
│ │ @@ -1 +1 @@
│ │ -7fba86c5-74fb-4236-8145-98b25e7e3c0b
│ │ +70b11e09-6dc1-4266-ad58-e480cb8d7c00

This is Bug #15187. Note that this also happens for files outside of /usr/share/fonts (the above is under /usr/local and there are others varying under, say, /usr/share/poppler/cMap/.

.mo files have varying POT-Creation-Date

│ ├── /usr/share/locale/ar/LC_MESSAGES/tails.mo
│ │ ├── msgunfmt {}
│ │ │ @@ -1,12 +1,12 @@
│ │ │  msgid ""
│ │ │  msgstr ""
│ │ │  "Project-Id-Version: The Tor Project\n"
│ │ │  "Report-Msgid-Bugs-To: \n"
│ │ │ -"POT-Creation-Date: 2019-01-05 11:41+0000\n"
│ │ │ +"POT-Creation-Date: 2019-01-05 14:35+0000\n"
│ │ │  "PO-Revision-Date: 2014-12-05 17:21+0000\n"
│ │ │  "Last-Translator: Osama M. Mneina <o.mneina@gmail.com>\n"
│ │ │  "Language-Team: Arabic (http://www.transifex.com/projects/p/torproject/"
│ │ │  "language/ar/)\n"
│ │ │  "Language: ar\n"
│ │ │  "MIME-Version: 1.0\n"
│ │ │  "Content-Type: text/plain; charset=UTF-8\n"

#2 Updated by lamby 2019-01-05 16:26:06

  • blocked by Bug #15187: fontconfig cache is not reproducible in Buster added

#3 Updated by lamby 2019-01-05 16:28:22

  • blocked by Bug #16294: SquashFS metadata is not reproducible in buster added

#4 Updated by lamby 2019-01-05 16:29:19

  • blocked by Bug #16295: Various base directories/files have varying mtimes added

#5 Updated by lamby 2019-01-05 16:30:27

  • blocks Bug #16296: .mo files have varying POT-Creation-Date in buster added

#6 Updated by lamby 2019-01-05 16:33:44

  • blocks Bug #16297: Different DEBIAN_SERIAL value in buster added

#7 Updated by lamby 2019-01-05 16:35:18

  • QA Check set to Dev Needed

#8 Updated by intrigeri 2019-01-05 16:57:57

  • blocked by deleted (Bug #16297: Different DEBIAN_SERIAL value in buster)

#9 Updated by intrigeri 2019-01-05 16:58:25

  • blocked by deleted (Bug #16296: .mo files have varying POT-Creation-Date in buster)

#10 Updated by intrigeri 2019-01-05 16:58:31

  • blocked by Bug #16296: .mo files have varying POT-Creation-Date in buster added

#11 Updated by lamby 2019-01-05 18:45:41

Attaching another output run. Archive serial still changes so I probably need to be quicker with the 2nd build…!

#12 Updated by lamby 2019-01-05 20:51:20

… and another set of results, this time with the same serial. :)

#13 Updated by intrigeri 2019-01-06 07:10:47

> Archive serial still changes so I probably need to be quicker with the 2nd build…!

… or merge Bug #15107 locally so you can benefit from it :)

#14 Updated by intrigeri 2019-01-06 07:23:03

#15 Updated by intrigeri 2019-01-06 11:25:51

  • related to Bug #16310: Buster IUKs are not built reproducibly added

#16 Updated by lamby 2019-01-07 19:08:14

Blocked by Bug #16294

#17 Updated by lamby 2019-01-08 13:12:59

  • blocks deleted (Bug #15187: fontconfig cache is not reproducible in Buster)

#18 Updated by lamby 2019-01-08 13:13:29

  • related to Bug #15187: fontconfig cache is not reproducible in Buster added

#19 Updated by lamby 2019-01-08 13:18:22

  • Status changed from In Progress to Resolved
  • QA Check changed from Dev Needed to Pass

Done, as of https://jenkins.tails.boum.org/job/reproducibly_build_Tails_ISO_feature-buster/80/

#20 Updated by intrigeri 2019-01-08 14:51:24

Woohoo! \o/