Bug #11482

Add a random sleep time to the rsync cronjob we suggest mirror operators use

Added by intrigeri 2016-05-24 15:29:03 . Updated 2016-06-02 13:33:32 .

Status:
Resolved
Priority:
Normal
Assignee:
Category:
Infrastructure
Target version:
Tails_2.4
Start date:
2016-05-24
Due date:
% Done:

100%

Feature Branch:
doc/11482-random-mirrors-sync-time
Type of work:
Contributors documentation
Blueprint:

Starter:
Affected tool:
Deliverable for:
269

Description

Now that we have lots of mirrors, having them all sync’ at the same time is just a recipe to give them slow transfers, and cause scaling issues on the rsync server’s side. Let’s try to insert a 0-30 minutes random delay. This has at least 3 aspects:

  • contribute/how/mirror
  • the corresponding Puppet class
  • the release process doc: in order to avoid blocking on all mirrors to have sync’ed, we’ll want to point (internal) testers to a specific one that has synced already; if it’s not trivial to document how to do that, I’ll add an option to check-mirrors to simply return the first up-to-date mirror it sees while trying them in random order, or something.

Subtasks

History

#1 Updated by intrigeri 2016-06-01 11:02:15

  • Status changed from Confirmed to In Progress
  • % Done changed from 0 to 20
  • Feature Branch set to doc/11482-random-mirrors-sync-time

intrigeri wrote:
> * contribute/how/mirror

Done in the topic branch.

> * the corresponding Puppet class

TBD

> * the release process doc […]

Done in the topic branch.

#2 Updated by intrigeri 2016-06-01 11:05:11

intrigeri wrote:
> > * the corresponding Puppet class

Done, pushed straight to the puppet-tails module.

#3 Updated by intrigeri 2016-06-01 11:05:49

  • Assignee changed from intrigeri to anonym
  • Target version changed from Tails_2.5 to Tails_2.4
  • % Done changed from 20 to 50
  • QA Check set to Ready for QA

#4 Updated by anonym 2016-06-02 11:03:19

  • Assignee changed from anonym to intrigeri
  • % Done changed from 50 to 80

It looks good, except the sleep ... stuff involving a bit shift part, so I pushed commit:91cb966. What do you think?

#5 Updated by anonym 2016-06-02 11:24:40

anonym wrote:
> It looks good, except the sleep ... stuff involving a bit shift part, so I pushed commit:91cb966. What do you think?

Actually, I’d rather avoid the od + /dev/urandom stuff as well, for readability. We don’t need cryptographically strong random values so perhaps we should do something a bit simpler:

sleep $(echo "$(date +%N) % (40*60 + 1)" | bc)


(Note: %N is for nanoseconds)

I’m just trying to make it easier for our mirror operators here. All that crazy stuff in that cron line did throw me off.

Alternatively: I guess we always can rely on perl being available, and I’m sure you can come up with a neater and more readable perl one-liner.

#6 Updated by intrigeri 2016-06-02 13:01:59

> It looks good, except the sleep ... stuff involving a bit shift part, so I pushed commit:91cb966. What do you think?

ACK.

#7 Updated by intrigeri 2016-06-02 13:08:05

  • Assignee changed from intrigeri to anonym

> Alternatively: I guess we always can rely on perl being available, and I’m sure you can come up with a neater and more readable perl one-liner.

Sure! Done on the branch + in the Puppet module.

#8 Updated by anonym 2016-06-02 13:33:32

  • Status changed from In Progress to Resolved
  • Assignee deleted (anonym)
  • % Done changed from 80 to 100
  • QA Check changed from Ready for QA to Pass

> print int(rand(2400))

Technically I think that should be 2400 + 1 but whatever. :) Merged!