<div dir="ltr">So basically, we're abusing the GH API. Nice. =)<div><br></div><div>A.</div></div><div class="gmail_extra"><br><div class="gmail_quote">On Wed, Oct 5, 2016 at 5:46 PM, R. Tyler Croy <span dir="ltr"><<a href="mailto:tyler@monkeypox.org" target="_blank">tyler@monkeypox.org</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">----- Forwarded message from Ivan ??u??ak <<a href="mailto:support@github.com">support@github.com</a>> -----<br>
<br>
Date: Wed, 05 Oct 2016 04:49:20 -0700<br>
From: Ivan ??u??ak <<a href="mailto:support@github.com">support@github.com</a>><br>
To: "R. Tyler Croy" <<a href="mailto:tyler@monkeypox.org">tyler@monkeypox.org</a>><br>
Subject: Re: Increasing rate-limit for autnenticated activity on busy/large (jenkinsci) org<br>
Message-ID: <discussions/<wbr>07023a048a5a11e69316559b2eb2b6<wbr>a1/comments/<a href="mailto:3256985@github.com">3256985@github.com</a><wbr>><br>
<br>
Hello there, Tyler,<br>
<br>
Thanks for reaching out. API rate limit increases are something we offer rarely, especially permanent ones. The current limits help us keep the API fast and reliable for all our users and applications, not just one. For the same reason, hitting the API rate limit shouldn't be considered abnormal -- it's a core part of using the API and the API returns information about your remaining quota and when it will refresh with every response. In most case, we try to provide some advice on optimizing API usage so that you don't even need a rate limit increase.<br>
<br>
I've just had a look at our API traffic logs for that account you're using, and I wanted to share some stats with you.<br>
<br>
Here are the top 10 API endpoints you're hitting (over the past week):<br>
<br>
nil 204,138 20.349% (these are requests that were rate limited)<br>
/rate_limit 177,343 17.678%<br>
/repositories/:repository_id/<wbr>commits/* 176,145 17.558%<br>
/repositories/:repository_id/<wbr>branches 105,497 10.516%<br>
/repositories/:repository_id/<wbr>events 99,464 9.915%<br>
/repositories/:repository_id/<wbr>collaborators 47,637 4.748%<br>
/repositories/:repository_id/<wbr>pulls/:id/comments 37,171 3.705%<br>
/repositories/:repository_id/<wbr>contents/?* 30,111 3.002%<br>
/repositories/:repository_id/<wbr>pulls 29,056 2.896%<br>
/repositories/:repository_id 18,348 1.829%<br>
<br>
And the top 10 resources:<br>
<br>
/rate_limit 177,343 17.678%<br>
/repositories/1103607/<wbr>collaborators 20,690 2.062%<br>
/repos/jenkinsci/jenkins/<wbr>contents/ 15,998 1.595%<br>
/repos/jenkinsci/jenkins/<wbr>collaborators 10,344 1.031%<br>
/user 7,695 0.767%<br>
/ 7,562 0.754%<br>
/repositories/612587/<wbr>collaborators 7,130 0.711%<br>
/repos/jenkins-infra/<a href="http://jenkins.io" rel="noreferrer" target="_blank">jenkins.<wbr>io</a> 6,642 0.662%<br>
/repos/jenkins-inc/securitay/<wbr>contents/ 5,254 0.524%<br>
/repos/jenkins-inc/borat/<wbr>contents/ 4,007 0.399%<br>
<br>
Top HTTP request methods and response statuses:<br>
<br>
get 996,922 99.376%<br>
post 6,220 0.62%<br>
put 41 0.004%<br>
delete 2 0%<br>
<br>
200 792,833 79.032%<br>
403 204,038 20.339%<br>
201 3,861 0.385%<br>
422 2,319 0.231%<br>
404 69 0.007%<br>
301 42 0.004%<br>
204 13 0.001%<br>
202 10 0.001%<br>
<br>
And the tokens you're using to make those requests:<br>
<br>
52986225 852,452 84.974% Jenkins JIRA (OAuth application)<br>
44790939 92,556 9.226% <a href="http://ci.jenkins.io" rel="noreferrer" target="_blank">ci.jenkins.io</a> pipeline token (personal token)<br>
28723209 47,502 4.735% <a href="http://demo.jenkins-ci.org" rel="noreferrer" target="_blank">demo.jenkins-ci.org</a> (personal token)<br>
<br>
There are a few things here you might consider optimizing. First, you're making a lot of requests over the API rate limit, which is generally considered abuse of the API:<br>
<br>
<a href="https://developer.github.com/guides/best-practices-for-integrators/#dealing-with-rate-limits" rel="noreferrer" target="_blank">https://developer.github.com/<wbr>guides/best-practices-for-<wbr>integrators/#dealing-with-<wbr>rate-limits</a><br>
<br>
Can you add some checks so that you don't make requests after you've hit the rate limit? Some of those requests are API calls to check your rate limit, but I don't see a reason why you'd need to call that endpoint so frequently either -- you can call it once to see when the rate limit will refresh and sleep until that moment.<br>
<br>
Next, it seems that you are fetching and-refetching the same resources over and over again in short periods of time. For example, here are some stats for a 1-hour period when you hit the rate limits:<br>
<br>
/repos/jenkinsci/elasticbox-<wbr>plugin/pulls 62 1.243%<br>
/repos/jenkinsci/jenkins/pulls 49 0.982%<br>
/repositories/1103607/pulls 49 0.982%<br>
/repos/jenkinsci/ec2-plugin/<wbr>pulls 48 0.962%<br>
/repos/jenkinsci/instance-<wbr>identity-module/pulls 39 0.782%<br>
/repositories/1169210/pulls 36 0.722%<br>
/repos/jenkinsci/jacoco-<wbr>plugin/pulls 30 0.601%<br>
/repos/jenkinsci/jenkins/<wbr>pulls/2570/comments 21 0.421%<br>
/repos/jenkinsci/instant-<wbr>messaging-plugin/pulls 19 0.381%<br>
/user 19 0.381%<br>
<br>
All of those requests were actually made within a 10-minute period, which was enough to drain your quota completely. For example, you fetched the list of comments for a single pull request 21 times within those 10 minutes (/repos/jenkinsci/jenkins/<wbr>pulls/2570/comments), and as far as I can tell -- there's only one page of comments there (you weren't fetching multiple pages). The same is true for fetching the list of pull requests for some repositories -- you fetched /repos/jenkinsci/elasticbox-<wbr>plugin/pulls 62 times within a single minute (again, only the first page). When you aggregate such behavior over many pull requests and repositories -- it drains your quota really fast.<br>
<br>
I recommend you look into caching some of that data on your end (the API will reward you for that: <a href="https://developer.github.com/v3/#conditional-requests" rel="noreferrer" target="_blank">https://developer.github.com/<wbr>v3/#conditional-requests</a>), or even better -- using webhook so that you fetch data once after it's modified (and store it on your end) instead of every time you need it.<br>
<br>
I'm guessing that with some optimizations, you might not even need a rate limit increase. If you notice that you do even after optimizing your usage -- let us know and we'd be happy to take a look at our logs again and discuss a possible increase with the team.<br>
<br>
I hope these notes are helpful, and let me know if you need any other information from our end.<br>
<br>
Best,<br>
Ivan<br>
<br>
> Hello, I manage the infrastructure for the Jenkins project (<a href="https://jenkins.io" rel="noreferrer" target="_blank">https://jenkins.io</a>)<br>
> and we're using some GitHub integrations to provide build/test/release services<br>
> for developers who participate in our GitHub organization on<br>
> <a href="https://ci.jenkins.io" rel="noreferrer" target="_blank">https://ci.jenkins.io</a><br>
><br>
> Unfortunately, despite being authenticated (via the jenkinsadmin account) we<br>
> are regularly hitting the rate-limit on API calls.<br>
><br>
> I was hoping we could get the rate-limit raised for calls? I'm not sure what is<br>
> reasonable, but 5000 per hour does seem rather low for the number of commit<br>
> statuses and repo lookups our Jenkins installation needs to perform.<br>
><br>
> Please let me know if this is doable, and if any more information is necessary<br>
> from our side.<br>
><br>
> Cheers<br>
> - R. Tyler Croy<br>
<br>
----- End forwarded message -----<br>
<br>
- R. Tyler Croy<br>
<br>
------------------------------<wbr>------------------------<br>
Code: <<a href="https://github.com/rtyler" rel="noreferrer" target="_blank">https://github.com/rtyler</a>><br>
Chatter: <<a href="https://twitter.com/agentdero" rel="noreferrer" target="_blank">https://twitter.com/agentdero</a><wbr>><br>
<br>
% gpg --keyserver <a href="http://keys.gnupg.net" rel="noreferrer" target="_blank">keys.gnupg.net</a> --recv-key 1426C7DC3F51E16F<br>
------------------------------<wbr>------------------------<br>
<br>______________________________<wbr>_________________<br>
Jenkins-infra mailing list<br>
<a href="mailto:Jenkins-infra@lists.jenkins-ci.org">Jenkins-infra@lists.jenkins-<wbr>ci.org</a><br>
<a href="http://lists.jenkins-ci.org/mailman/listinfo/jenkins-infra" rel="noreferrer" target="_blank">http://lists.jenkins-ci.org/<wbr>mailman/listinfo/jenkins-infra</a><br>
<br></blockquote></div><br></div>