Feature #9646

Upgrade jenkins-job-builder to 1.3.0+

Added by bertagaz 2015-06-25 03:50:28 . Updated 2015-09-25 01:31:23 .

Status:
Resolved
Priority:
Normal
Assignee:
Category:
Infrastructure
Target version:
Tails_1.7
Start date:
2015-06-25
Due date:
% Done:

100%

Feature Branch:
Type of work:
Sysadmin
Blueprint:

Starter:
0
Affected tool:
Deliverable for:
267

Description

We’re using Debian’s old jjb v0.9.0 but a lot of changes have happened since then upstream.

It has a better recursive option where you can exclude the .git/ directory (ain’t that nice?). That’s what prevent us to store our automatically generated build_Tails_ISO_* jobs in a subdirectory.

Once 1.2.0 will be in Debian, we should:

  • revert puppet-tails:61f349c and puppet-tails:a4a74c5
  • move the build_Tails_ISO_* jobs in the build_Tails_ISO/ subdirectory of our jenkins-jobs repo (see jenkins-jobs:7e12053)
  • adapt our build_Tails_ISO_* jobs as in jenkins-jobs:b9f5e09 to use the new submodule option syntax.

Subtasks

Related issues

Related to Tails - Feature #10242: Pull jenkins-job-builder related packages from backports or testing Resolved 2015-09-25

History

#1 Updated by bertagaz 2015-06-25 03:53:22

  • Type of work changed from Sysadmin to Wait

For now that’s a wait rather, we just have to track Debian’s bug

#2 Updated by intrigeri 2015-06-28 04:10:23

  • Assignee set to bertagaz

This seems to be part of the work on automated builds and tests, isn’t it?

#3 Updated by bertagaz 2015-06-28 04:55:21

intrigeri wrote:
> This seems to be part of the work on automated builds and tests, isn’t it?

Or sysadmins’ one, as we’re talking about a package upgrade in Debian. I might not be the one that will apply this upgrade, depending on when this version gets into Debian. That’s why I wrote such a detailed description.

#4 Updated by intrigeri 2015-06-28 07:28:31

>> This seems to be part of the work on automated builds and tests, isn’t it?

> Or sysadmins’ one, as we’re talking about a package upgrade in Debian.

It seems to me that the only reasons why we might want to bother getting a new jjb now (as opposed to whenever we upgrade to Stretch) is because it makes implementing new infra features (e.g. files/jenkins/master/update_build_tails_iso_jobs) nicer, which seems totally unrelated to daily sysadmin. As far as daily sysadmin is concerned, jjb works just fine and the upgrade can wait for Stretch. See what I mean? :)

#5 Updated by bertagaz 2015-06-28 08:06:07

intrigeri wrote:
> It seems to me that the only reasons why we might want to bother getting a new jjb now (as opposed to whenever we upgrade to Stretch) is because it makes implementing new infra features (e.g. files/jenkins/master/update_build_tails_iso_jobs) nicer, which seems totally unrelated to daily sysadmin. As far as daily sysadmin is concerned, jjb works just fine and the upgrade can wait for Stretch. See what I mean? :)

Exactly, and that’s what I meant in this ticket: wait for this jjb to enter Debian, there’s no hurry for us to use it as you stated. So this ticket was mainly opened to track down the new jjb availability in Debian, and note what we’ll have to do when it happens.

That’s why it’s not related to the automated tests tickets, is a wait and has no target version.

#6 Updated by bertagaz 2015-08-30 13:52:26

  • Subject changed from Switch to jenkins-job-builder 1.2.0 to Switch to jenkins-job-builder 1.3.0+

Actually, JJB 1.3 has been released, and I just pushed the support of the cucumber-testresult plugin upstream, which is lacking at the moment. We can still use the raw_xml meanwhile.

https://review.openstack.org/#/c/218668/

#7 Updated by intrigeri 2015-08-31 00:42:36

> Actually, JJB 1.3 has been released, and I just pushed the support of the cucumber-testresult plugin upstream, which is lacking at the moment. We can still use the raw_xml meanwhile.

It’s waiting in there: https://ftp-master.debian.org/new.html.

Since we apparently need it for the autotests, we probably need to add corresponding ticket relationships.

#8 Updated by bertagaz 2015-08-31 01:53:47

intrigeri wrote:
> > Actually, JJB 1.3 has been released, and I just pushed the support of the cucumber-testresult plugin upstream, which is lacking at the moment. We can still use the raw_xml meanwhile.
>
> It’s waiting in there: https://ftp-master.debian.org/new.html.
>
> Since we apparently need it for the autotests, we probably need to add corresponding ticket relationships.

Ah, too bad. As they were lacking behind since quite some time, I thought we would have time to get it in before Debian update its package.

But well, that’s not a firm need, we can still use the raw_xml trick.

#9 Updated by intrigeri 2015-08-31 02:49:12

> Ah, too bad. As they were lacking behind since quite some time, I thought we would have time to get it in before Debian update its package.

It’s good news that the package has been taken over by someone who will actually maintain it, and that 1.3 was uploaded to start with: once your stuff is merged upstream + released, there are good chances that they’ll make it into Debian pretty fast :)

Also, a few other people have volunteered to help.

#10 Updated by bertagaz 2015-09-02 05:38:56

  • blocks Feature #9486: Support running multiple instances of the test suite in parallel added

#11 Updated by bertagaz 2015-09-02 05:39:19

  • Target version set to Tails_1.6
  • Deliverable for set to 267

#12 Updated by bertagaz 2015-09-02 07:36:19

  • blocked by deleted (Feature #9486: Support running multiple instances of the test suite in parallel)

#13 Updated by bertagaz 2015-09-02 07:36:48

  • Target version deleted (Tails_1.6)
  • Deliverable for deleted (267)

#14 Updated by bertagaz 2015-09-04 14:05:52

  • Deliverable for set to 267

#15 Updated by bertagaz 2015-09-04 14:07:35

  • Target version set to Tails_1.6

As our current JJB version does not support the raw config option, we’ll need to upgrade ASAP, to be able to support the PrioritySorter and CucumberTestResult plugins.

#16 Updated by bertagaz 2015-09-04 14:07:55

#17 Updated by bertagaz 2015-09-14 07:41:50

  • Status changed from Confirmed to In Progress
  • Priority changed from Normal to Elevated
  • Type of work changed from Wait to Sysadmin

Ok great, I’ve updated JJB to this 1.3.0 version that entered unstable, and hit a bug I’m not yet sure how to resolve. I’ll post more later. Meanwhile I’ll have to restore our build history from the last backups I have (3 or 4 days old), because thanks to this bug, all our jobs were deleted and we lost our build history…

#18 Updated by bertagaz 2015-09-18 03:36:27

  • Assignee changed from bertagaz to intrigeri
  • % Done changed from 0 to 70
  • QA Check set to Info Needed

So the bug is concerning the python3 version of JJB in unstable and has been reported in the Debian BTS.

It’s not that blocking since we can use the python2 version which works fine.

So the upgrade has been done, and the deprecated job options updated.

Also, now that JJB support the recursive option, the build_Tails_ISO_* jobs have been moved in a sub-directory and the scripts adapted to this new layout. The push of this change proved it works, as local testing did.

So shall we close this ticket (and eventually open a new one for the python3 version bug of JJB), or keep it open?

#19 Updated by intrigeri 2015-09-22 01:42:16

  • Assignee changed from intrigeri to bertagaz
  • QA Check changed from Info Needed to Dev Needed

> So the bug is concerning the python3 version of JJB in unstable and has been reported in the Debian BTS.

> It’s not that blocking since we can use the python2 version which works fine.

> So the upgrade has been done, and the deprecated job options updated.

Excellent :)

> Also, now that JJB support the recursive option, the build_Tails_ISO_* jobs have been moved in a sub-directory and the scripts adapted to this new layout. The push of this change proved it works, as local testing did.

Cool, I’ll review that in the next few hours.

> So shall we close this ticket (and eventually open a new one for the python3 version bug of JJB), or keep it open?

I see that for the moment we’re installing jenkins-job-builder and friends from sid. It totally made sense to me as a first step, to test it and adjust whatever we needed to adjust. But do we really want to go on installing it from sid forever? This sounds a bit risky. I don’t think a full-blown official backport is necessary (if it becomes some day, then we’ll see) but IMO we should install those packages from testing once they’ve reached it. What do you think?

#20 Updated by intrigeri 2015-09-22 01:42:36

  • Target version changed from Tails_1.6 to Tails_1.7

#21 Updated by intrigeri 2015-09-22 03:28:20

I see we need python-jenkins >= 0.4 and install it from sid. It’s already in testing so let’s install it from there, no?

#22 Updated by intrigeri 2015-09-22 03:34:47

And for python-six: also installed from sid, while jessie-backports has the same upstream version. Isn’t it good enough?

#23 Updated by intrigeri 2015-09-22 04:00:56

  • Subject changed from Switch to jenkins-job-builder 1.3.0+ to Upgrade to jenkins-job-builder 1.3.0+

#24 Updated by intrigeri 2015-09-22 04:01:05

  • Subject changed from Upgrade to jenkins-job-builder 1.3.0+ to Upgrade jenkins-job-builder to 1.3.0+

#25 Updated by bertagaz 2015-09-23 02:34:08

  • Priority changed from Elevated to Normal

Agree we should install the packages from testing or Jessie-backports.

I’ll do that once jjb 1.3 will be in testing, this way it won’t require multiple changes in our manifests.

Lowering the priority, since it’s less a blocker now.

#26 Updated by intrigeri 2015-09-23 03:45:42

> Lowering the priority, since it’s less a blocker now.

I’d rather see this ticket closed, and a new one filed to track the remaining bits.

#27 Updated by bertagaz 2015-09-25 01:29:32

  • related to Feature #10242: Pull jenkins-job-builder related packages from backports or testing added

#28 Updated by bertagaz 2015-09-25 01:31:09

  • Status changed from In Progress to Resolved
  • Assignee deleted (bertagaz)
  • QA Check deleted (Dev Needed)

Created Feature #10242 to track the remainings then.

#29 Updated by bertagaz 2015-09-25 01:31:23

  • % Done changed from 70 to 100