Feature #6277

Rewrite the tails-live.git history

Added by intrigeri 2013-09-11 13:28:00 . Updated 2015-08-26 05:33:59 .

Status:
Resolved
Priority:
High
Assignee:
intrigeri
Category:
Infrastructure
Target version:
Tails_1.3.1
Start date:
2015-01-09
Due date:
2015-02-27
% Done:

100%

Feature Branch:
Type of work:
Code
Blueprint:

https://tails.boum.org/blueprint/rewrite_Git_history/

Starter:
0
Affected tool:
Deliverable for:
265

Description

Without:

  • the Debian packages: config/chroot_local-packages/*.deb config/chroot_local-packages/*.gz config/chroot_local-packages/*.bz2 config/chroot_local-packages/*.dsc
  • the forum: wiki/src/forum*
  • promotional material that will be moved somewhere else (Feature #8827): wiki/src/promote/*
  • some large files from the logo contest: wiki/src/blueprint/logo/FINISHED_WORK.7z wiki/src/blueprint/logo/tails_hiding_cat.zip wiki/src/blueprint/logo/tails_walking_cat.zip
  • TrueCrypt tarballs: config/chroot_local-includes/usr/share/amnesia/truecrypt*.gz

Once this is done, we can revert commit 4f5588 in the tails Puppet module (filtering out spurious jenkins-{build,fetch}* tags.

First try, with only one branch:

git filter-branch -d /var/tmp/git-rewrite --tree-filter 'rm -rf config/chroot_local-packages/*.deb config/chroot_local-packages/*.gz config/chroot_local-packages/*.bz2 config/chroot_local-packages/*.dsc wiki/src/forum*' feature/jessie
# git tag | xargs git tag -d # why?!
git remote  rm origin                                   
git for-each-ref --format="%(refname)" refs/original/ | xargs -n 1 git update-ref -d
git reflog expire --expire=now --all                                                
git gc --aggressive --prune=now

=> .git goes from 316MB to 91MB. With all tags and current branches, it’ll probably be a tiny bit more.

To preserve tags, one needs to also pass --tag-name-filter cat to git filter-branch.

Other options:

  • git filter-branch --index-filter is supposedly much faster than --tree-filter
  • using bfg: 
    java -jar /path/to/bfg.jar \
   --delete-folders '{forum,deb-src}' \
   -D '{*.deb,*.diff.gz,*.debian.tar.*,*.orig.tar.*,forum.mdwn}' \
tails-test.git

Subtasks

Feature #8618: Deal with the remaining branches' Git history and interdependencies Resolved intrigeri

100
Feature #8619: Do a trial rewrite of all branches and tags Resolved Dr_Whax

100
Feature #8620: Adjust the Git repository rewrite process as needed Resolved Dr_Whax

100
Feature #8624: Lock the old Git repository Resolved intrigeri

100
Feature #8633: Let contributors using Git know what they should do after the history rewrite Resolved intrigeri

100
Feature #8630: Clean up the rewritten Git repository Resolved intrigeri

100
Feature #8632: Make sure various clones of our Git repository are up-to-date wrt. the rewrite Resolved intrigeri

100
Feature #8631: Force push the rewritten Git repository Resolved intrigeri

100
Feature #8629: Replace rewritten (unsigned) Git tags with signed ones Resolved intrigeri

100
Feature #8628: Do the final rewrite of all branches and tags Resolved intrigeri

100
Feature #8627: Communicate with contributors about the old Git repository being locked Resolved intrigeri

100
Feature #8813: Reduce in size /blueprint/logo Resolved

100
Feature #8939: Check what still takes room in the rewritten Git repo Resolved intrigeri

100
Feature #8946: Reintroduce the contribute/how/promote directory Resolved intrigeri

100

History

#1 Updated by intrigeri 2014-04-07 08:42:27

  • Category set to Infrastructure
  • Assignee set to intrigeri
  • Target version set to Sustainability_M1

#2 Updated by intrigeri 2014-05-12 01:14:44

  • Description updated

#3 Updated by intrigeri 2014-05-12 01:15:14

  • related to Bug #7220: Ignore jenkins-{build,fetch}-* tags when generating reprepro configuration added

#4 Updated by intrigeri 2014-08-02 16:57:25

  • blocks deleted (Feature #6276: Migrate the website to its own Git repository)

#5 Updated by intrigeri 2014-08-02 16:58:27

(We’ve rejected Feature #6276, so the parent ticket doesn’t make any sense anymore. “Orphaning” this one, then.)

#6 Updated by intrigeri 2014-08-02 17:00:17

  • Description updated

#7 Updated by intrigeri 2014-08-02 17:30:13

#8 Updated by intrigeri 2014-08-02 17:30:22

#9 Updated by intrigeri 2014-08-02 17:30:34

#10 Updated by intrigeri 2014-08-16 16:24:19

  • related to Feature #7221: Write a script that deletes old merged Git branches added

#11 Updated by intrigeri 2014-08-16 16:24:54

  • related to deleted (Feature #7221: Write a script that deletes old merged Git branches)

#12 Updated by intrigeri 2014-08-16 16:25:40

  • blocked by Feature #7221: Write a script that deletes old merged Git branches added

#13 Updated by intrigeri 2014-08-17 00:06:30

  • Description updated

#14 Updated by intrigeri 2014-08-17 00:43:12

  • Description updated

#15 Updated by intrigeri 2015-01-09 14:24:43

  • blocks deleted (Feature #7221: Write a script that deletes old merged Git branches)

#16 Updated by intrigeri 2015-01-09 14:25:26

  • related to deleted (Bug #7220: Ignore jenkins-{build,fetch}-* tags when generating reprepro configuration)

#17 Updated by intrigeri 2015-01-09 14:25:30

#18 Updated by intrigeri 2015-01-09 14:26:21

  • Target version changed from Sustainability_M1 to Tails_1.3.2

#19 Updated by intrigeri 2015-01-09 14:33:43

  • blocked by Feature #8617: Regularly delete obsolete Git branches and corresponding data in our APT repository added

#20 Updated by intrigeri 2015-02-21 14:10:50

  • Description updated

#21 Updated by intrigeri 2015-02-21 14:16:31

  • Description updated
  • Status changed from Confirmed to In Progress

#22 Updated by intrigeri 2015-02-21 16:02:36

  • Description updated

#23 Updated by intrigeri 2015-02-21 16:49:40

  • Description updated

#24 Updated by intrigeri 2015-02-22 13:38:06

  • Description updated

#25 Updated by intrigeri 2015-02-22 18:42:20

  • Blueprint set to https://tails.boum.org/blueprint/rewrite_Git_history/

#26 Updated by intrigeri 2015-02-24 23:14:50

  • blocks deleted (Feature #8617: Regularly delete obsolete Git branches and corresponding data in our APT repository)

#27 Updated by intrigeri 2015-02-25 18:02:08

  • Status changed from In Progress to Resolved

#28 Updated by BitingBird 2015-03-22 12:13:00

  • Target version changed from Tails_1.3.2 to Tails_1.3.1

#29 Updated by intrigeri 2015-05-28 15:22:26

  • blocks #8668 added