[Jenkins-infra] Pagerduty on-call modifications
Olblak
me at olblak.com
Tue Aug 7 19:04:17 UTC 2018
We try to explain fixes for recurring issues on that (private)
repository -> https://github.com/jenkins-infra/runbooksAnd if we don't know what to do, we raise a Jira ticket and assign to
the person who has the information.A recent example was the last spamming session on #jenkins-infra and #jenkins-
community irc channel (INFRA-1732).We raised a ticket and Tyler filled the documentation and now the next
they come, we'll be ready (ง •̀_•́)งIt's important to mention when something is not clear otherwise we'll
have to deal with the bus factor :)And if for any reasons, we realise the PD notification is important then
we still have contact information on that repository as well.
---
-> gpg --keyserver keys.gnupg.net --recv-key 52210D3D
---
On Tue, Aug 7, 2018, at 6:00 PM, Arnaud Héritier wrote:
> yes same than KK
> I'm happy to help but with the move to Azure I'm completely lost in
> how I could help (I'm even not sure to still have the required
> rights/permissions)> If I have the instructions of what to do (or not do :-P) I'll continue
> to help with pleasure>
> On Tue, Aug 7, 2018 at 4:57 PM Kohsuke Kawaguchi
> <kk at kohsuke.org> wrote:>> Thanks for taking the lead on this. I think your plan makes sense.
>>
>> I'd like to use this opportunity to ask how I should respond to these
>> PD notifications. Most of them I get is indeed disk space issues, but
>> I have no idea how to resolve that, so mostly I just ignore them.>>
>> On Tue, Aug 7, 2018 at 6:13 AM Olblak <me at olblak.com> wrote:
>>>
>>> Hello,
>>>
>>> I would like to update the Pagerduty rotation, currently every
>>> people on-call are assigned to the same group and rotated every 6
>>> hours which leads people after a while to be on call in the middle
>>> of the night while others are available.>>> We don't get a lot of notification (disk space issue most of the
>>> time), but I find it useless to be on-call in the middle of the
>>> night and not being notified when I am in front of a computer
>>> during the day.>>>
>>> Now I suggest a "follow the sun" strategy with four groups and
>>> depending on your timezone (or preference) you are assign to a
>>> specific groups.>>>
>>> It would be something like
>>>
>>> Group 1 (5AM to 11 AM): Arnaud, Daniel, Olblak (7AM UTC+2 to 1PM
>>> UTC+2)>>> Group 2 (11AM to 5PM): Arnaud, Daniel, Olblak (1PM UTC+2 to 7PM
>>> UTC+2)>>> Group 3 (5PM to 11PM): Andrew,Tyler, Kohsuke (10AM UTC-7 to 4PM
>>> UTC-7)>>> Group 4 (11PM to 5 AM): Andrew, Tyler, Kohsuke (4PM UTC-7 to 10PM
>>> UTC-7)>>>
>>> The global strategy would remain the same
>>>
>>> Level1 notification sent to on call people
>>> After 30minutes
>>> Level2 notification are sent to Tyler/Daniel/Olivier
>>> After 30minutes
>>> Level3 notification is sent to KK
>>>
>>> Because we have at least three people per group, you will only be
>>> on-call during one slot per day?>>> Shouldn't we review on call people annually to be sure that they
>>> are still available?>>>
>>> Any suggestions?
>>>
>>> ---
>>> -> gpg --keyserver keys.gnupg.net --recv-key 52210D3D
>>> ---
>>> _______________________________________________
>>> Jenkins-infra mailing list
>>> Jenkins-infra at lists.jenkins-ci.org
>>> http://lists.jenkins-ci.org/mailman/listinfo/jenkins-infra
>> --
>> Kohsuke Kawaguchi
>> _______________________________________________
>> Jenkins-infra mailing list
>> Jenkins-infra at lists.jenkins-ci.org
>> http://lists.jenkins-ci.org/mailman/listinfo/jenkins-infra
>
>
> --
> -----
> Arnaud Héritier
> http://aheritier.net
> Mail/GTalk: aheritier AT gmail DOT com
> Twitter/Skype : aheritier
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.jenkins-ci.org/pipermail/jenkins-infra/attachments/20180807/1ae183c7/attachment.html>
More information about the Jenkins-infra
mailing list