Tails

#1 Updated by emmapeel 2018-01-17 19:35:09

• blocks Feature #15079: Integrate the platform with our Git and ikiwiki infrastructure added

#2 Updated by emmapeel 2018-02-10 11:40:20

• % Done changed from 0 to 50
• QA Check changed from Dev Needed to Info Needed

I have installed the cronjobs:

1. cronjobs for weblate, see:
2. https://docs.weblate.org/en/weblate-2.10.1/admin/install.html#production-cron
3. Fulltext index updates
   /5 * * * cd /usr/share/weblate/; ./manage.py update_index
   /6 * * * cd /usr/share/weblate/; ./manage.py updatechecks —all
   /7 * * * cd /usr/share/weblate/; ./manage.py loadpo —all
4. Cleanup stale objects
   daily cd /usr/share/weblate/; ./manage.py cleanuptrans # Commit pending changes after 96 hours hourly cd /usr/share/weblate/; ./manage.py commit_pending —all —age=96 —verbosity=0

with crontab -e as user weblate.

I wonder is there is any puppetinzaing thing to be done though (or we can close this ticket)

#3 Updated by Anonymous 2018-02-11 09:20:45

• Assignee set to emmapeel

Hi emmapeel!

thanks for your work! May you please add to our documentation

• where you added these cronjobs
• for each of them describe what they do
• for each of them what happens if they fail.

Thanks!

#4 Updated by emmapeel 2018-02-12 19:30:05

• Assignee deleted (emmapeel)

They are in the documentation of course. Only problem is people will not read it.

#5 Updated by Anonymous 2018-02-12 23:46:27

• Assignee set to emmapeel
• Feature Branch set to translate-server: master

emmapeel wrote:
> They are in the documentation of course. Only problem is people will not read it.

Uuh, there is absolutely nothing detailed in the documentation about these cronjobs.

The latest commit I see on this repo is dde4906b22c2be851458cac5e20a493da4ba20d9.
In which file is the detailed doc of the cronjobs supposed to be found? grep at least did not even find the word cron so either I miss some details here (is it on the master branch?) or you forgot to push some changes you made.

Basically, what is written here in the ticket description is missing from the Git repo.

#6 Updated by Anonymous 2018-02-13 09:22:36

• QA Check deleted (Info Needed)

I’ll add them myself for now.

#9 Updated by emmapeel 2018-02-13 11:09:37

• Assignee deleted (emmapeel)

I still want to know if there is more to be done regarding puppetization of this cronjobs, or we can close this ticket.

#10 Updated by Anonymous 2018-02-13 11:40:22

• Assignee set to emmapeel

emmapeel wrote:
> u wrote:
> >
> > Uuh, there is absolutely nothing detailed in the documentation about these cronjobs.

> I think you edited it adding the word ‘cronjobs’. I would not call that absolutely nothing.

Yes, the two commands you noted in the documentation were not specified to be cronjobs and it was unclear if they have to be run manually. I also described better what these cronjobs do, and how often they are run. -> All of this information was missing. Furthermore, I added an entire section at the bottom of the file about the other two cronjobs which were mentioned nowhere else.

#11 Updated by Anonymous 2018-02-13 11:42:46

• Assignee changed from emmapeel to groente
• QA Check set to Info Needed

As for your remaining question, we should ask sysadmins if there is anything left to do, rather than merely unassigning this ticket I believe. @groente: I’m unsure about puppetizing cronjobs, is this something we should consider doing?

#13 Updated by Anonymous 2018-03-13 14:24:01

• Target version set to Tails_3.7

Hi groente, may you please check why the cronjobs are not running? Thanks!

#14 Updated by Anonymous 2018-03-13 14:24:28

• Deliverable for set to Sponsor_L

#15 Updated by Anonymous 2018-03-13 14:24:50

• blocked by deleted (Feature #15079: Integrate the platform with our Git and ikiwiki infrastructure)

#16 Updated by Anonymous 2018-03-13 14:25:02

• Parent task set to Feature #15079

#17 Updated by Anonymous 2018-03-13 15:50:28

• Subject changed from Some cronjobs for translate.lizard to Cronjobs for translate.lizard are not running

#18 Updated by groente 2018-03-26 11:40:28

• Assignee changed from groente to emmapeel
• QA Check changed from Info Needed to Dev Needed

I added some checks to the cronjobs (writing to a tmp-file) and they are definately running. Perhaps the commands are missing some environment variables to function properly? Either way, please keep them in regular crontab and don’t move them to cron.daily.

As for puppetising, that would be preferrable, I can do that as soon as the commands actually produce the desired output (please reassign this ticket to me once this is the case).

#19 Updated by emmapeel 2018-04-03 11:19:13

• Status changed from Confirmed to In Progress
• Assignee changed from emmapeel to groente

Ok, please puppetize them then!

#20 Updated by Anonymous 2018-04-03 21:11:54

• QA Check deleted (Dev Needed)

emmapeel wrote:
> Ok, please puppetize them then!

@emmapeel: If you reassign this to groente, this means these commands work correctly? He mentioned there might be missing variables. Why did you think they were not running in first place? Can you please make this clear on the ticket? Thanks.

And then, yes, let’s puppetize it.

#21 Updated by emmapeel 2018-04-06 08:19:59

• QA Check set to Dev Needed

Please puppetize the commands that are running in the weblate user contrab. Maybe they were blocked by ther processes before and that is why they were not running.

There have been some merges on the repo but it is hard to know if they were triggered by this crontab or not.

#22 Updated by groente 2018-04-13 10:03:12

• Assignee changed from groente to emmapeel
• QA Check changed from Dev Needed to Ready for QA

the cronjobs are now puppetised, can you confirm they are working properly?

#23 Updated by bertagaz 2018-05-10 11:09:10

• Target version changed from Tails_3.7 to Tails_3.8

#24 Updated by emmapeel 2018-06-04 09:21:44

• Status changed from In Progress to Resolved
• QA Check changed from Ready for QA to Pass

Yes, they look good!

#25 Updated by emmapeel 2018-06-14 15:25:00

• Status changed from Resolved to In Progress
• Assignee changed from emmapeel to groente
• QA Check changed from Pass to Ready for QA
• Feature Branch changed from translate-server: master to emmapeel/puppet-tails:better_dir

As this tasks are also trigered by other processes, it took me a while to realize the cronjobs have to point to another directory.

Please review this branch:

git.tails.boum.org:emmapeel/puppet-tails.git
* [new branch] better_dir -> better_dir

With the correct directory

#26 Updated by groente 2018-06-15 08:05:26

• Status changed from In Progress to Resolved
• Assignee changed from groente to emmapeel
• QA Check changed from Ready for QA to Pass

I didn’t have read access to your repo, but changed the directory in the cronjobs to what I believe is now the right one. Please test and reopen this ticket if there’s still something wrong with the cronjobs.

#27 Updated by emmapeel 2018-06-20 07:46:37

• File 7629e4f46d42df02f90f28107524542098370482..755fa48e9b978569c4bc215b144647e7515cfc45.patch added
• Status changed from Resolved to In Progress
• Assignee changed from emmapeel to groente
• QA Check changed from Pass to Ready for QA
• Feature Branch changed from emmapeel/puppet-tails:better_dir to emmapeel:puppet-tails/less_cronjobs

Yes, it worked! Unfortunately, the cronjobs apparently are too much for the resources of the server and we are running out of memory quite often now.

I would not mind you having access to my repo, but i think it is public:

https://git-tails.immerda.ch/emmapeel/puppet-tails/

The cronjobs are working, but maybe they are too much, the server is being overloaded often.

I changed a little the frequency of them with 755fa48e9b978569c4bc215b144647e7515cfc45

I attach a patch here for you to review, otherwise is the branch less_cronjobs in said repository.

Please review

#28 Updated by emmapeel 2018-06-21 07:41:24

• related to Bug #15679: update puppet-tails to have scripts in weblate vm added

#29 Updated by intrigeri 2018-06-26 16:27:46

• Target version changed from Tails_3.8 to Tails_3.9

#30 Updated by groente 2018-06-27 11:19:18

• % Done changed from 50 to 80
• QA Check changed from Ready for QA to Pass

the job that was causing problems was the updatechecks, i’ve now set that to hourly instead of every six minutes. let’s see how the machine holds now.

#32 Updated by groente 2018-06-29 08:44:00

• Assignee changed from groente to emmapeel
• QA Check changed from Pass to Ready for QA

I changed the cronjob, is everything working okay now?

#33 Updated by emmapeel 2018-06-29 09:28:23

• Assignee deleted (emmapeel)

The changes are OK, now the cronjob is working correctly.

On the other hand, I assign this ticket to u to

- see if it is really necessary to update git with this cronjob, or we should drop it in exchange for hefee’s new scripts ( Bug #15185 )
- if we drop it, when? (it seems to me it will be replaced when Bug #15185 is solved)

#34 Updated by emmapeel 2018-06-29 10:00:28

• related to Bug #15185: Automatically update Weblate components on mdwn file add/rename/delete in tails.bo git added

#35 Updated by Anonymous 2018-08-16 12:29:49

• Target version changed from Tails_3.9 to Tails_3.10.1

emmapeel wrote:
> The changes are OK, now the cronjob is working correctly.
>
> On the other hand, I assign this ticket to u to
>
> - see if it is really necessary to update git with this cronjob, or we should drop it in exchange for hefee’s new scripts ( Bug #15185 )
> - if we drop it, when? (it seems to me it will be replaced when Bug #15185 is solved)

Let’s talk about it at the summit.

#36 Updated by Anonymous 2018-09-03 14:33:40

• QA Check deleted (Ready for QA)

#37 Updated by Anonymous 2018-09-03 14:34:57

• Status changed from In Progress to Resolved

We said that we want to schedule some work time together with a sysadmin to look into this.
Now, it seems to me that the cronjobs are running now. And yes, we will replace them with hefee’s script and hence I think #1⁵190 is obsolete → closing.

#38 Updated by bertagaz 2018-09-12 09:20:03

• Status changed from Resolved to In Progress
• Assignee set to emmapeel
• QA Check set to Info Needed

Hi,

I’m loosely repopening this ticket to notice you that I’ve fixed some cronjob that were badly configured (see Bug #15906#note-4 for more). Just to be sure I did not mess up anything, would you mind having a look at https://git-tails.immerda.ch/puppet-tails/tree/manifests/weblate.pp#n178 and the translate system/website? If it’s ok feel free to set this ticket back in its previous state (status: resolved, assignee: u). I know you’ll quit the cronjob implementation, but meanwhile I just want to be sure I did not break something. Thanks!

#39 Updated by intrigeri 2018-10-24 17:03:33

• Target version changed from Tails_3.10.1 to Tails_3.11

#40 Updated by hefee 2018-11-05 17:26:59

• Status changed from In Progress to Resolved

the cron jobs are now done via one file, so they can’t lock themselves.

#41 Updated by Anonymous 2019-02-07 15:34:48

• Parent task changed from Feature #15079 to Feature #11759

#42 Updated by intrigeri 2019-06-27 17:16:31

• Assignee deleted (emmapeel)