Feature #15082: Have the Weblate Git communicate with our main Git repository

Feature #15082

Have the Weblate Git communicate with our main Git repository

Added by Anonymous 2017-12-19 16:40:57 . Updated 2019-07-06 20:43:33 .

Status:

Resolved

Priority:

Normal

Assignee:

Category:

Target version:

Start date:

2018-03-01

Due date:

% Done:

100%

Feature Branch:

Type of work:

Sysadmin

Blueprint:

Starter:

Affected tool:

Translation Platform

Deliverable for:

309

Description

(plan created by intrigeri and hefee - Jan 2018)

Translation Server

Have a (rough) review of used scripts and interaction of components (~~Feature #16326~~).

Weblate

We have to make sure that Weblate git only contains reviewed translations.
maybe have pre-commit hooks that run check_po on the commit.
Weblate should push by commit. Or if that to much noise once per hour. If once per hour we need a cronjob for this on the weblate VM.

MAIN GIT

get pre-receive hook reviewed (~~Bug #15402~~).
On merge, our main git has to verify that weblate tries to push only po files, modified or new ones, and nothing else. Email on error to weblate admins and tails-l10n.

Subtasks

Bug #15185: Automatically update Weblate components on mdwn file add/rename/delete in tails.bo git

Resolved

100

Bug #15356: Automate adding new components to Weblate

Duplicate

Bug #15402: update hook for Main git when handling push from weblate

Resolved

Feature #15673: Import non active languages from Weblate's Git repository into our main tails.git

Resolved

Bug #15675: Weblate's git repo contains manual modifications which are not translations

Resolved

Feature #16842: update_weblate_git.py: don't create merge commits when we could fast-forward

Resolved

Related issues

Related to Tails - ~~Feature #15355~~: Make the ikiwiki PO plugin able to update PO files for languages that are disabled on the website	Rejected	2018-03-01
Blocked by Tails - ~~Feature #16328~~: Merge a stricter version of check_po whose expectations are realistic into our master branch	Resolved	2019-01-08

History

#2 Updated by Anonymous 2017-12-19 17:43:15

blocked by ~~Feature #15084~~: Review & Rubberducking added

#3 Updated by Anonymous 2018-03-01 10:30:00

Subject changed from Have the interface communicate with our main Git repository to Have the Weblate Git communicate with our main Git repository

Only reviewed and i18inspector checked translations should make it into our main repository. We need to add hooks to Weblate and Git for this.

#4 Updated by Anonymous 2018-03-02 15:11:55

Description updated

#5 Updated by Anonymous 2018-03-30 14:41:44

Current pre-receive hook for Main Git (works) - but instead of comparing the committer email we might also want to add an env variable to detect when weblate is pushing.

#!/usr/bin/env python3
# Pre-receive or update hook for main git
# Weblate is only allowed to push po files.

import re, subprocess, sys

expected = "tails-l10n@b.o" # fixme
regex = re.compile(".*\.po$")

# read references from stdin
oldref, newref, refname = sys.stdin.read().split()

# get list of modified revisions
rev_list = subprocess.run("git rev-list %s..%s"%(oldref,newref), check=True, shell=True, stdout=subprocess.PIPE)
revlist = rev_list.stdout.decode().split("\n")
for rev in revlist:
    # Email address of committer, not author
    committer = subprocess.run("git cat-file commit %s | grep 'committer' | grep -o -P '(?<=\<).*(?=\>)'"%rev, shell=True, stdout=subprocess.PIPE)
    # Which files have been modified
    files = subprocess.run("git diff-tree --no-commit-id --name-only -r %s"%rev, shell=True, stdout=subprocess.PIPE)
    if expected == committer.stdout.decode().strip() and not regex.search(files.stdout.decode()):
        print ("[POLICY] Weblate is only allowed to modify po files.")
        sys.exit(-1)
~