Tails

#5 Updated by intrigeri 2020-04-13 07:58:56

I’ve reviewed the branch at 86d03365ec90d25e02aa97c5dda9ad1e87383f72 and it LGTM.

I’ve merged current master into the topic branch, so I can use it in a dedicated Puppet environment in order to test this update on my local Jenkins infra.

#6 Updated by intrigeri 2020-04-13 08:29:29

I saw an error as part of the Puppet run so I pushed ed2803f2a3e5dcdf537773d2226906fc9665b895 to puppet-tails to fix it, but that was not enough:

Notice: /Stage[main]/Jenkins::Cli/Exec[jenkins-cli]/returns: mv: cannot stat 'WEB-INF/jenkins-cli.jar': No such file or directory
Error: /Stage[main]/Jenkins::Cli/Exec[jenkins-cli]: Failed to call refresh: jar -xf /usr/share/jenkins/jenkins.war WEB-INF/jenkins-cli.jar && mv WEB-INF/jenkins-cli.jar /usr/share/jenkins/jenkins-cli.jar && rm -rf WEB-INF returned 1 instead of one of [0]
Error: /Stage[main]/Jenkins::Cli/Exec[jenkins-cli]: jar -xf /usr/share/jenkins/jenkins.war WEB-INF/jenkins-cli.jar && mv WEB-INF/jenkins-cli.jar /usr/share/jenkins/jenkins-cli.jar && rm -rf WEB-INF returned 1 instead of one of [0]

Indeed, jenkins.war does not contain WEB-INF/jenkins-cli.jar. Now:

• on my test system, I have -rw-r--r-- 1 root root 0 Sep 19 2019 jenkins-cli.jar
• AFAIK we don’t use jenkins-cli

⇒ I’ll ignore this one-time error.

Then I had to update one more plugin (19702cb991b2b2fcf2c8bfd95d7bfbb60a76cffb).

I’ll now use this upgraded local Jenkins environment and we’ll see how it fares :)

#7 Updated by intrigeri 2020-04-13 09:07:16

First problems found:

• jenkins-slave.service cannot put the node back online. The API request (done by jenkins-enable-node) yields a 403 error.
• Similarly, the jenkins-jobs --flush-cache update --delete-old /etc/jenkins_jobs/ part of our Jenkins jobs update process fails with a 403 error.

I’ve followed the “Disabling Security” section of https://jenkins.io/doc/book/system-administration/security/, but that was not enough to fix the problem.

This needs further investigation. Meanwhile, I’ve temporarily disabled (8f58ff066f77b9b3eff550aeeeaffd278dfe1108) the broken step of the service startup, in order to not block on this, so I can test the rest of the CI and possibly identify other problems.

#8 Updated by intrigeri 2020-04-13 09:14:04

When I push branches to the Git repo used by my local Jenkins, notifying Jenkins fails to trigger build jobs:

remote: .--- Notifying Jenkins... -------------------------------------
remote: |
remote: 11
remote: 11
remote: | Notifying Jenkins of Git updates...
remote: | No git jobs found
remote: | No Git consumers using SCM API plugin for: gitolite@jenkins.sib:tails
remote: |
remote: `--------------------------------------------------------------

It looks like the Git plugin, or one of its dependencies, is broken (or requires a config update, or something). It might be caused by the failed jenkins-jobs update: I see no Git repo configured in a build job’s “Source code management” config section in the web interface. This also needs further investigation.

#9 Updated by intrigeri 2020-04-13 09:19:44

I’ve manually configured the Git repo for a build job in the web interface. I had to add Jenkins’ SSH private key to the global credentials store. It looks like credentials / Git / SSH management has changed. I suspect a part of the fix should be done manually in the web interface, and the credentials part in our jenkins-jobs config. Not sure.