Tails

#1 Updated by intrigeri 2013-09-26 05:44:26

• Starter changed from No to Yes

#2 Updated by intrigeri 2013-12-29 03:20:47

• Category set to Infrastructure

#3 Updated by intrigeri 2015-05-28 15:53:34

• blocks deleted (Feature #6295: Evaluate consequences of importing large amounts of packages into reprepro)

#4 Updated by intrigeri 2015-05-28 15:53:41

• blocked by deleted (Feature #6299: Regularly pull packages from foreign APT repositories)

#5 Updated by intrigeri 2015-05-28 15:53:46

• blocked by deleted (Feature #6310: Publish source Debian packages used by Tails ISO images)

#6 Updated by intrigeri 2015-05-28 15:54:07

• Assignee set to intrigeri
• Target version changed from Sustainability_M1 to Tails_2.2
• Parent task changed from Feature #5926 to Feature #9489

#7 Updated by intrigeri 2015-05-28 15:54:19

• blocks #8668 added

#8 Updated by intrigeri 2015-05-28 15:58:45

• Subject changed from Adapt our APT repository infrastructure to be able to handle tons of packages to Adapt our infrastructure to be able to handle tons of packages
• Description updated

#9 Updated by intrigeri 2015-05-30 13:42:39

• related to Feature #9508: Evaluate freezable APT repo's storage needs added

#10 Updated by intrigeri 2015-08-26 06:00:00

• Deliverable for set to 269

#11 Updated by intrigeri 2015-12-13 12:11:04

• blocked by Feature #6295: Evaluate consequences of importing large amounts of packages into reprepro added

#12 Updated by intrigeri 2015-12-13 12:19:21

• blocks Feature #6296: Configure reprepro to pull from foreign APT repositories added

#13 Updated by intrigeri 2015-12-13 12:20:00

• Priority changed from Normal to Elevated
• Target version changed from Tails_2.2 to Tails_2.0
• Starter deleted (Yes)

This will be blocking the actual deployment so I’d like to get it done early.

#14 Updated by intrigeri 2016-01-11 17:12:44

• Status changed from Confirmed to In Progress
• Target version changed from Tails_2.0 to Tails_2.2

The urgent part is being handled in the Feature #10851 subtask, and what’s left (backups, and the fail-over system we don’t have yet) can wait.

#15 Updated by intrigeri 2016-02-05 14:33:24

• Assignee changed from intrigeri to bertagaz
• QA Check set to Ready for QA

(The only remaining open subtask is on bertagaz’ plate.)

#16 Updated by intrigeri 2016-02-05 14:35:11

• Assignee changed from bertagaz to intrigeri
• Target version changed from Tails_2.2 to Tails_2.3
• QA Check deleted (Ready for QA)

Oops. Actually there’s also the backup and fail-over bits that I need to tackle at some point.

#17 Updated by intrigeri 2016-03-25 22:40:05

• blocked by deleted (#8668)

#18 Updated by intrigeri 2016-03-25 22:40:10

• blocked by deleted (Feature #6296: Configure reprepro to pull from foreign APT repositories)

#19 Updated by intrigeri 2016-03-25 22:53:21

• Assignee changed from intrigeri to bertagaz
• Target version changed from Tails_2.3 to Tails_2.4

bertagaz: I’m reassigning to you because the only small bit left to do here is on your plate. But feel free to ignore the details of my reasoning, and skip directly to your name. Now, if you want to review my reasoning, I certainly don’t mind :)

Both the failover and the backups will only have to carry tagged snapshots, not time-based snapshots (that we can afford losing). One initial data point is that Feature #6295#note-29 says it will take 90GB after a year, but this was based on assumptions (“we import them in a single, persistent reprepro instance, even though the filterlist etc. used for importing are volatile”) that might not hold anymore. After running reprepro update in each of the reprepro config trees created by a run of tails-prepare-tagged-apt-snapshot-import, I see 6.4 GB used (including i386+source + most amd64 packages). So worst case, it’s 6.4 GB * 15 releases each year = 96 GB / year + some room for expected growth.

So I say that:

• short/mid-term:
  • let’s get ourselves 200GB for each set of backups; the one I’m managing has this already, so I’m reassigning this ticket to bertagaz, so that he does the same (I could give a hand but I bet the logistics overhead won’t be worth it)
  • I’ve adjusted our backup system so that it skips time-based snapshots, but backs up tagged snapshots
  • for the failover: https://tails.boum.org/blueprint/lizard_failover/#system-specs already plans enough storage so we’re good
• long-term, i.e. whenever the occupied space grows enough to become problematic: it should be fairly easy to de-duplicate the stored files, e.g. using the hardlink tool or a filesystem that can de-duplicate itself at the block or file level.

#20 Updated by intrigeri 2016-04-06 14:00:14

• Target version changed from Tails_2.4 to Tails_2.3

#22 Updated by bertagaz 2016-04-22 11:19:22

• Assignee changed from bertagaz to intrigeri

intrigeri wrote:
> So I say that:
>
> * short/mid-term:
> let’s get ourselves 200GB for each set of backups; the one I’m managing has this already, so I’m reassigning this ticket to bertagaz, so that he does the same (I could give a hand but I bet the logistics overhead won’t be worth it)

I have that amount of storage dedicated to this backups.

> * long-term, i.e. whenever the occupied space grows enough to become problematic: it should be fairly easy to de-duplicate the stored files, e.g. using the hardlink tool or a filesystem that can de-duplicate itself at the block or file level.

I would have said ticket to track this space issue, but nowadays our monitoring system should prevent us to do that :). Congrats for this plan, I’m all for it. Fell free to close unless something else is needed from me.

#23 Updated by intrigeri 2016-04-25 13:01:08

• Target version changed from Tails_2.3 to Tails_2.4

#24 Updated by intrigeri 2016-04-26 02:35:07

• Status changed from In Progress to Resolved
• Target version changed from Tails_2.4 to Tails_2.3

#25 Updated by intrigeri 2016-04-26 02:37:08

• Assignee deleted (intrigeri)