Tails

#7 Updated by Anonymous 2019-07-08 10:02:27

@intrigeri,

> I have something that works locally. All pieces are deployed in production already except the cronjob, because I’d like your feedback about the content of the email it’s going to send. My draft lives there: https://git.tails.boum.org/puppet-tails/tree/files/redmine/reminder/email_body

Nice!

> A few things that we did not consider initially IIRC, though, and for which I need your input (or consent, in some cases):
>
> * I’m not sure what the From field should be. Possibly you, as our ticket triager? Except this might stop working if your email server ever does DKIM/SPF/whatever. For now, I’ve set From: root$FQDN@, which is not ideal.

My address as from will not work, my mail server migrated last week to a new one and SPF/DKIM will be in place if it’s not already.

Either, we keep using root@FQDN, or we set up some sort of alias (that can forward emails to me). I’m not sure how to request such an alias.

> * Similarly, we need a To field makes it too likely that the message is classified as spam. Here as well, I’d be inclined to set your address there, and set all other recipients as Bcc. Shall I do that?

If I get it right, we do send out one email per person, is that correct? So why do addresses need a Bcc:, when they can have a clear To: field? Not sure why the one or the other would make spam detectors cough more or less.

> * cron does not easily allow doing something every 45 days; the closest cheap options to that are: either every month or every 2 months; I think every 2 months is sufficient; what do you think?

Agreed.

> * Be it wrt. “stalled WIP” or “reviews waiting for a long time”, we have at least as many problematic tickets that this code will not identify as those it will point people to. These problematic tickets, that our setup will miss, are those which have a target version that’s been postponed after the N last releases by the Release Manager: every such Target version bump counts as an update so there’s no chance the ticket will ever be identitied as stalled. That of course does not imply that what we did is useless nor that we should block on this. But I’d like us to think this through and address it if we can do so easily. For example, we could do one pass every year or so on the tickets whose target version is the next Tails release, and whenever the target version was bumped by the RM since 6+ months after each release, with no update by the assignee, we drop the target version and paste a nice comment from a template. I’d be fine with doing the first pass in the next few months myself. Then our queries about stalled WIP and reviews will be able to notice those tickets, starting 6 months later. What do you think?

That sounds like a very good plan. Should I ask enrico if he thinks this could be automated? It would avoid us a lot of boring and recurring work.

#8 Updated by Anonymous 2019-07-08 10:05:12

@intrigeri

> I have something that works locally. All pieces are deployed in production already except the cronjob, because I’d like your feedback about the content of the email it’s going to send. My draft lives there: https://git.tails.boum.org/puppet-tails/tree/files/redmine/reminder/email_body

I like the text and the signature :)

So we don’t include a list of ticket numbers in that email?

#10 Updated by intrigeri 2019-07-09 10:22:31

> So we don’t include a list of ticket numbers in that email?

No, the plan was to do the cheapest possible thing, i.e. point folks to the place where they will find their list (after logging in).

#11 Updated by intrigeri 2019-07-09 10:53:54

Dear @u,

>> * I’m not sure what the From field should be. Possibly you, as our ticket triager? Except this might stop working if your email server ever does DKIM/SPF/whatever. For now, I’ve set From: root$FQDN@, which is not ideal.

> My address as from will not work, my mail server migrated last week to a new one and SPF/DKIM will be in place if it’s not already.

> Either, we keep using root@FQDN, or we set up some sort of alias (that can forward emails to me). I’m not sure how to request such an alias.

Thanks, that’s the info I needed. I’ll try to find & implement a cheap solution, which should be easier now that we host a MX.

>> * Similarly, we need a To field makes it too likely that the message is classified as spam. Here as well, I’d be inclined to set your address there, and set all other recipients as Bcc. Shall I do that?

> If I get it right, we do send out one email per person, is that correct?

Well, I did the cheapest thing i.e. send one single email to all affected folks. Indeed, sending one email per person would fix this problem. I’ll do that. Thanks!

>> * Be it wrt. “stalled WIP” or “reviews waiting for a long time”, we have at least as many problematic tickets that this code will not identify as those it will point people to. These problematic tickets, that our setup will miss, are those which have a target version that’s been postponed after the N last releases by the Release Manager: every such Target version bump counts as an update so there’s no chance the ticket will ever be identitied as stalled. That of course does not imply that what we did is useless nor that we should block on this. But I’d like us to think this through and address it if we can do so easily. For example, we could do one pass every year or so on the tickets whose target version is the next Tails release, and whenever the target version was bumped by the RM since 6+ months after each release, with no update by the assignee, we drop the target version and paste a nice comment from a template. I’d be fine with doing the first pass in the next few months myself. Then our queries about stalled WIP and reviews will be able to notice those tickets, starting 6 months later. What do you think?

> That sounds like a very good plan. Should I ask enrico if he thinks this could be automated? It would avoid us a lot of boring and recurring work.

I’d love to be surprised but I expect that:

• It’s not cheap to specify the exact algorithm the automation should implement in all corner cases (e.g. what to do if the assignee was the RM back then?)
• It’s not cheap to implement said specification (who was the RM back then?)
• In quite a few cases, for human/social/accounting/management-ish reasons, doing the automated thing would cause all sorts of trouble, such as workers confusion (“so, what’s my deadline, in the end?”). So I think that someone will need to review all the resulting Redmine changes and occasionally revert some of them with personalized explanations/apologies. I’m not sure that all in all it’s better (for us and contributors) than doing it by hand in the first place.

Right now that’s 87 tickets we would need to go through for the initial pass. It’s indeed quite some work but the number should drop quickly, once we’ve dealt with the initial backlog and folks get used to a new, stricter meaning of “target version”. Given this and the fact I’m proposing to do it only once a year, I feel more optimistic than “lot of boring and recurring work”.

In any case, if we go for the automated way, I suspect we’ll need to go through a bunch of these tickets by hand first, in order to tell what the algorithm should be and provide examples of what we expect it to do.

Oh, but actually, what about this: the automation does not do any change on Redmine, but instead computes a list of tickets that are candidates for dropping target version? This relaxes quite a bit the need for the algorithm to be excellent and it addresses my concern about automatic changes being undesirable in some cases. It still saves quite some of the boring work and allows humans to focus on what we’re better at than computers. There’s some boring/repetitive work remaining but perhaps not much more than reviewing automatic changes?

#14 Updated by intrigeri 2019-07-18 14:02:55

A first batch of email was sent!

#15 Updated by Anonymous 2019-07-19 12:08:37

intrigeri wrote:
> A first batch of email was sent!

It worked :)

#16 Updated by Anonymous 2019-07-19 12:11:38

@intrigeri

> Right now that’s 87 tickets we would need to go through for the initial pass. It’s indeed quite some work but the number should drop quickly, once we’ve dealt with the initial backlog and folks get used to a new, stricter meaning of “target version”. Given this and the fact I’m proposing to do it only once a year, I feel more optimistic than “lot of boring and recurring work”.

Agreed.

> In any case, if we go for the automated way, I suspect we’ll need to go through a bunch of these tickets by hand first, in order to tell what the algorithm should be and provide examples of what we expect it to do.
>
> Oh, but actually, what about this: the automation does not do any change on Redmine, but instead computes a list of tickets that are candidates for dropping target version? This relaxes quite a bit the need for the algorithm to be excellent and it addresses my concern about automatic changes being undesirable in some cases. It still saves quite some of the boring work and allows humans to focus on what we’re better at than computers. There’s some boring/repetitive work remaining but perhaps not much more than reviewing automatic changes?

Yes, I don’t expect the algorithm to do anything besides computing the list.
If you think it would be doable and have fun doing it, I would say go ahead and clock as ticket triaging work.

#18 Updated by intrigeri 2019-08-30 18:03:23

I’ve gone through 86 tickets that all have target version = 3.16. Most of them won’t be solved in time for 3.16, which will be released in 4 days. I did this:

• I’ve removed the target version on at least 18 tickets: Bug #16566, Bug #15396, #16276, Feature #16143, Bug #15508, Bug #15626, Bug #15390, Feature #13457, Bug #14648, Bug #14647, Bug #14501, Bug #16273, Bug #16060, Bug #16034, Bug #15957, Feature #15120, Bug #15483, Feature #12214… and those I forgot to take note of :]
• I’ve updated a few target versions that did not seem to reflect what the assignee meant, or that did not make any sense to me (as per “better reflect your plans”).
• I’ve closed a few tickets that were waiting for a review for months, when the facts clearly show that the implemented fix worked.

As a result, there’s only 62 tickets left with a 3.16 target version. It’s not ideal, but that’s about 20 more tickets that now have a chance to show up in the stalled work views and trigger email notifications.

The criteria I’ve consciously used are (some of those I realized too late, and did not apply initially):

• When the last action on the ticket by the assignee was performed. For example, if they did not update the ticket since 3 months or more, it suggests the target version should be dropped.
• When the ticket was (re)assigned to the assignee. If the ticket landed on their plate recently, we should probably do nothing.
• Whether there’s been progress but it’s not documented on the ticket. For example, sajolida did steady progress on Feature #16539 (see discussion on -project@) but the ticket does not reflect it. It’s not ideal but in this case, dropping the target version would be inadequate.
• Who postponed the ticket recently. If the assignee did, well, it suggests they have it in mind and are updating their plans, which is good (note, however, that Some People® — sajolida and myself — are usually postponing their tickets themselves before the release, which skews things a bit ;). If a RM did, it suggests the opposite.
• What the ticket state is. We’ve been focused primarily on “In Progress” and “Needs Validation” so far, so they are the best candidates for dropping target version. But when someone is holding a “Confirmed” ticket for 6+ months and it’s postponed every 6 weeks, possibly dropping the target version will help them take a step back and reconsider their commitment to it.
• Whether it’s part of a sponsor deliverable.
  • If the official deadline is in the future, most of the time we don’t want to touch the Target version.
  • Else, if we’ve already delivered the work to the sponsor but the ticket is about a follow-up task: it depends™ :P No, seriously, this sort of situations is too tricky for me to try to formalize criteria by myself right now, so I’ll just ignore these tickets. In the past we’ve generally discussed them on a case-by-case basis, between the PM, accounting team, and other relevant stakeholders.
• Whether there’s an assignee. If there’s none but the ticket was postponed a few times, then:
  • Either it’s important and it should be put on the plate of a core team.
  • Or it’s unimportant and the target version should be dropped: keeping it has little value but forces the RMs to do extra work. If someone wants to track those tickets, there surely are better ways.
• Whether the ticket is blocked by anything (other tickets or what not) that’s out of the hands of the assignee.
• How safe/privileged is the assignee’s position in Tails. I expect some of us to feel comfortable reverting my changes, for example if they’re used to using their own, personal definition of the “Target version” field, which does not match how we’ve documented we collectively use it. So I’ve been a bit bolder with dropping target versions for contributors who hold more power than average.

I propose that we:

1. Do the same thing manually once or twice a year. It took me less than 2 hours, including the time to write this entire comment, including documenting the criteria I’ve used. I think that’s time well invested but I’m curious to see what feedback I get. It may be that doing it twice a year does not take much more time than once a year in total, and it would be more beneficial for Tails. I’m fine with doing the next round or two myself.
2. Document these criteria somewhere under wiki/src/contribute/working_together/roles/ticket_gardener/.
3. Don’t spent time automating this now: many of these criteria are pretty hard to automate, doing it by hand does not require that much time, I did not feel it was painful, and it’s not a good time for us to invest more into Redmine-specific code (upcoming migration to GitLab). We can reconsider next time we do this by hand.

What do you think?

#24 Updated by intrigeri 2019-11-17 05:27:39

intrigeri wrote:
> A first batch of email was sent!

FTR, today’s run notified 12 folks and tried to notify 3 other people (including the “Sysadmins” group) but failed because the automation bits don’t know their email address yet — which I’ll fix right away.