[Jenkins-infra] Pagerduty on-call modifications

Olblak me at olblak.com
Tue Aug 7 19:04:17 UTC 2018


We try to  explain fixes for recurring issues on that (private)
repository -> https://github.com/jenkins-infra/runbooksAnd if we don't know what to do, we raise a Jira ticket and assign to
the person who has the information.A recent example was the last spamming session on #jenkins-infra and #jenkins-
community irc channel (INFRA-1732).We raised a ticket and Tyler filled the documentation and now the next
they come, we'll be ready  (ง •̀_•́)งIt's important to mention when something is not clear otherwise we'll
have to deal with the bus factor :)And if for any reasons, we realise the PD notification is important then
we still have contact information on that repository as well.
---
-> gpg --keyserver keys.gnupg.net --recv-key 52210D3D
---




On Tue, Aug 7, 2018, at 6:00 PM, Arnaud Héritier wrote:
> yes same than KK
> I'm happy to help but with the move to Azure I'm completely lost in
> how I could help (I'm even not sure to still have the required
> rights/permissions)> If I have the instructions of what to do (or not do :-P) I'll continue
> to help with pleasure> 
> On Tue, Aug 7, 2018 at 4:57 PM Kohsuke Kawaguchi
> <kk at kohsuke.org> wrote:>> Thanks for taking the lead on this. I think your plan makes sense.
>> 
>> I'd like to use this opportunity to ask how I should respond to these
>> PD notifications. Most of them I get is indeed disk space issues, but
>> I have no idea how to resolve that, so mostly I just ignore them.>> 
>> On Tue, Aug 7, 2018 at 6:13 AM Olblak <me at olblak.com> wrote:
>>> 
>>> Hello,
>>> 
>>>  I would like to update the Pagerduty rotation, currently every
>>>  people on-call are assigned to the same group and rotated every 6
>>>  hours which leads people after a while to be on call in the middle
>>>  of the night while others are available.>>>  We don't get a lot of notification (disk space issue most of the
>>>  time), but I find it useless to be on-call in the middle of the
>>>  night and not being notified when I am in front of a computer
>>>  during the day.>>> 
>>>  Now I suggest a "follow the sun" strategy with four groups and
>>>  depending on your timezone (or preference) you are assign to a
>>>  specific groups.>>> 
>>>  It would be something like
>>> 
>>>  Group 1 (5AM to 11 AM): Arnaud, Daniel, Olblak (7AM UTC+2 to 1PM
>>>  UTC+2)>>>  Group 2 (11AM to 5PM): Arnaud, Daniel, Olblak (1PM UTC+2 to 7PM
>>>  UTC+2)>>>  Group 3 (5PM to 11PM): Andrew,Tyler, Kohsuke (10AM UTC-7 to 4PM
>>>  UTC-7)>>>  Group 4 (11PM to 5 AM): Andrew, Tyler, Kohsuke (4PM UTC-7 to 10PM
>>>  UTC-7)>>> 
>>>  The global strategy would remain the same
>>> 
>>>  Level1 notification sent to on call people
>>>  After 30minutes
>>>  Level2 notification are sent to Tyler/Daniel/Olivier
>>>  After 30minutes
>>>  Level3 notification is sent to  KK
>>> 
>>>  Because we have at least three people per group, you will only be
>>>  on-call during one slot per day?>>>  Shouldn't we review on call people annually to be sure that they
>>>  are still available?>>> 
>>>  Any suggestions?
>>> 
>>>  ---
>>>  -> gpg --keyserver keys.gnupg.net --recv-key 52210D3D
>>>  ---
>>>  _______________________________________________
>>>  Jenkins-infra mailing list
>>> Jenkins-infra at lists.jenkins-ci.org
>>> http://lists.jenkins-ci.org/mailman/listinfo/jenkins-infra
>> -- 
>> Kohsuke Kawaguchi
>> _______________________________________________
>>  Jenkins-infra mailing list
>> Jenkins-infra at lists.jenkins-ci.org
>> http://lists.jenkins-ci.org/mailman/listinfo/jenkins-infra
> 
> 
> -- 
> -----
> Arnaud Héritier
> http://aheritier.net
> Mail/GTalk: aheritier AT gmail DOT com
> Twitter/Skype : aheritier

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.jenkins-ci.org/pipermail/jenkins-infra/attachments/20180807/1ae183c7/attachment.html>


More information about the Jenkins-infra mailing list