[Jenkins-infra] plugins.jenkins.io Static site deployment

Gavin Mogan gavin at gavinmogan.com
Fri Jan 10 00:04:48 UTC 2020


Or you know, abandon jenkins, use netlify ci, and let it auto deploy to
unique urls every PR

Anyways, azure
https://issues.jenkins-ci.org/browse/INFRA-2414

Azure PR - https://github.com/jenkins-infra/azure/pull/132
Charts PR - https://github.com/jenkins-infra/charts/pull/101 (working on
updating existing PR)



On Thu, Jan 9, 2020 at 4:53 AM Olblak <me at olblak.com> wrote:

> Right now, we are missing a few things for that
>
> 1) At the moment we don't version our helm charts and for that we need to
> configure/deploy a helm chart repository
> 2) We need a dns record to the "beta" version or something like wildcard
> record like '*.pr.ci.jenkins.io'
>
> Then run helm install && helm delete from a ci environment
>
> I'd personally rather get it done, then start adding more issues and
> feature requests, than leave it in a weird stale state for even longer.
>
>  Me too, anyway it's easy to rollback to the current state
>
> ---
> gpg --keyserver keys.gnupg.net --recv-key 52210D3D
> ---
>
>
> On Thu, Jan 9, 2020, at 11:39 AM, Gavin Mogan wrote:
>
> I'm not sure it'll help, it just adds more work :) probably not a lot of
> work, it would just be another domain, but is more work.
>
> I'd personally rather get it done, then start adding more issues and
> feature requests, than leave it in a weird stale state for even longer.
>
> On Thu, Jan 9, 2020 at 2:35 AM Oleg Nenashev <o.v.nenashev at gmail.com>
> wrote:
>
> Would it help if we firstly deploy two instances of the plugin site in
> parallel?
> I mean having the current plugin site + a beta version of a new one so
> that it can get more reviews before replacing the original one.
> We could add a banner to the old site so that users could try it out
>
> On Thu, Jan 9, 2020 at 11:30 AM Olblak <me at olblak.com> wrote:
>
>
> When i get online in the morning I'll create an infra ticket for azure
> blob storage. Based on
> https://github.com/jenkins-infra/javadoc/blob/master/Jenkinsfile#L52 I
> think we need a new account and key setup.
>
> Awesome, don't forget to also describe the blob storage for pluginsite
> similar to the javadoc
> <https://github.com/jenkins-infra/azure/blob/master/plans/javadoc.tf>
> Once provisioned, I'll configure ci.jenkins.io with the right
> credentials
>
> ---
> gpg --keyserver keys.gnupg.net --recv-key 52210D3D
> ---
>
>
> On Thu, Jan 9, 2020, at 10:35 AM, Gavin Mogan wrote:
>
> > we upload the data on network storage like (jenkins.io,
> javadoc.jenkins.io, reports.jenkins.io)
>
> Certainly my preference. Easiest to handle.
>
> Looking at the existing helm charts for jenkinsio/reports/javadocs, this
> should be pretty easy to setup.
>
> When i get online in the morning I'll create an infra ticket for azure
> blob storage. Based on
> https://github.com/jenkins-infra/javadoc/blob/master/Jenkinsfile#L52 I
> think we need a new account and key setup.
>
> I can setup the jenkinsfile to publish there, then update the helm charts
> like the rest to reference it.
>
> On Thu, Jan 9, 2020 at 12:45 AM Olblak <me at olblak.com> wrote:
>
>
> Looks like at least for December the pluginsite was well under the free
> bandwidth of 100GB offered by netlify
>
> Also note that the Jenkins project qualifies for the pro plan for free
> under the open source offering which provides 400gb of bandwidth
>
>
> It appears to me that you looked at the ingress value and not the egress.
> I also had a look at the grafana
> <https://grafana.publick8s.jenkins.io/d/nginx/nginx-ingress-controller>
> dashboard, where you have the transfer (in and out) per second.
> So based on those data, the netlify cdn won't work, especially if we
> consider jenkins.io or javadoc.jenkins.io.
>
> Regarding the pluginsite data, in the current state, I am not a big fan
> neither of a docker images updated every few hours.
> So we could run `mvn -B -PgeneratePluginData && sleep 15m` from a sidecar
> container or we upload the data on network storage like (jenkins.io,
> javadoc.jenkins.io, reports.jenkins.io) or we keep relying on
> ci.jenkins.io for now.
>
>  Keen (https://github.com/keel-hq/keel/blob/master/readme.md) seems like
> the easiest solution, especially if we switch the docker hub tags to semver
> instead of $id-hash
>
> I am not in favor of this tool for philosophical reasons, I prefer the
> GitOps approach where your infrastructure is defined based on git
> repositories, and every change is done through PR, it slower but easier to
> audit...
> My feeling with keen, is that once in place, it can quickly get out of
> control.
>
> So really we are only blocked by the deployment plan.
>
> I have helm charts ready. I have the docker images ready. It just needs to
> be merge and Cron triggered.
>
> I still think the docker image isn't going to work well cause it'll deploy
> a new image every couple hours which we need to update.
>
> Keen (https://github.com/keel-hq/keel/blob/master/readme.md) seems like
> the easiest solution, especially if we switch the docker hub tags to semver
> instead of $id-hash
>
> Or if trusted can commit to GitHub, as part of the job I can make it pull
> down charts. Commit a change and push.
>
> Or if trusted has access to kubectl, I could just update the image field.
>
> Or original suggestion of azure blob / netlify.
>
> I can do the work I just need people with actual access to make decisions.
>
>
> Currently, my only concern with your pr is about the best way to generate
> and consume pluginsite API data.
> All the other improvements are really great and I can't wait to look at
> plugins informations on my phone.
>
> Since you specified a flag with the old-mode, I guess, we can already move
> forward with your PR
>
> ---
> gpg --keyserver keys.gnupg.net --recv-key 52210D3D
> ---
>
>
> On Wed, Jan 8, 2020, at 10:59 AM, Gavin Mogan wrote:
>
> So really we are only blocked by the deployment plan.
>
> I have helm charts ready. I have the docker images ready. It just needs to
> be merge and Cron triggered.
>
> I still think the docker image isn't going to work well cause it'll deploy
> a new image every couple hours which we need to update.
>
> Keen (https://github.com/keel-hq/keel/blob/master/readme.md) seems like
> the easiest solution, especially if we switch the docker hub tags to semver
> instead of $id-hash
>
> Or if trusted can commit to GitHub, as part of the job I can make it pull
> down charts. Commit a change and push.
>
> Or if trusted has access to kubectl, I could just update the image field.
>
> Or original suggestion of azure blob / netlify.
>
> I can do the work I just need people with actual access to make decisions.
>
> Gavin
>
> On Mon., Jan. 6, 2020, 9:04 p.m. Tim Jacomb, <timjacomb1 at gmail.com> wrote:
>
> Looks like at least for December the pluginsite was well under the free
> bandwidth of 100GB offered by netlify
>
> Also note that the Jenkins project qualifies for the pro plan for free
> under the open source offering which provides 400gb of bandwidth
>
> https://www.netlify.com/legal/open-source-policy/
>
> https://www.netlify.com/pricing/#teams
>
> It would be good to try out a CDN imo, our users across the globe should
> benefit from it rather than us just serving from one location
>
> Tim
>
> On Thu, 2 Jan 2020 at 14:08, Olblak <me at olblak.com> wrote:
>
>
> As some of you might know, i've been poking at making a static version of
> plugins.jenkins.io like jenkins.io, that just gets rebuilt periodically
> (PR @ https://github.com/jenkins-infra/plugin-site/pull/76), Its pretty
> much done, just needs jenkinsfile/deployment tweaking.
>
>
> That's really great to hear
>
> all) add plugins-api.jenkins.io to the current helm chart
>
> What's wrong to keep plugins.jenkins.io/api/? we just have to forward to
> a different docker container
>
> 1) Update the helm chart, manually or automatically, every build of the
> site content
>
> Why not if the helm chart is automatically updated, I can finalize a small
> tool to do it.
>
> 2) Upload the content to azure blob storage like jenkins.io
>
>
> Do you have an idea of how often that content will be generated? Once per
> hour?
> Azure Storage makes more sense to me, as we already have a process and
> billing information in place.
>
> To give you an idea
> jenkinsio storage for december 2019 -> 217$
> javadoc storage for december 2019 -> 17$
>
> plugins.jenkins.io traffic seems to be one third of jenkins.io
>
> And here a screenshot of jenkinsio storage traffic over December 2019
>
>
>
> 3) If desired to reduce Azure dependencies, netlify is what I use for the
> demo site, its pretty easy, and they have a free tier for open source
> projects - pro with unlimited users -
> https://www.netlify.com/legal/open-source-policy/
>
>
> At the moment I have some concerned about bringing a new SAAS account in
> the infrastructure as It implies some billing information, and the free
> tier CDN bandwidth will be quickly consumed.
>
>
> ---
> gpg --keyserver keys.gnupg.net --recv-key 52210D3D
> ---
>
>
> On Sat, Dec 28, 2019, at 7:04 AM, Gavin Mogan wrote:
>
> Hey ya'll,
>
> As some of you might know, i've been poking at making a static version of
> plugins.jenkins.io like jenkins.io, that just gets rebuilt periodically
> (PR @ https://github.com/jenkins-infra/plugin-site/pull/76), Its pretty
> much done, just needs jenkinsfile/deployment tweaking.
>
> I started to create a PR to update the helm chart to split the backend
> image (/api) and the frontend image (/) into two different docker images.
> This was supposed to be short term until a new CDN style deployment. About
> 5 minutes into this PR, I realized it'll be a pain if the content is
> generated through cron.
>
> So, options
>
> all) add plugins-api.jenkins.io to the current helm chart
>
> 1) Update the helm chart, manually or automatically, every build of the
> site content
> 2) Upload the content to azure blob storage like jenkins.io (See
> https://github.com/jenkins-infra/jenkins.io/blob/a7cf4218dba77ab07e4e4ef691d5142e986a4949/Jenkinsfile#L94-L101
> )
> 3) If desired to reduce Azure dependencies, netlify is what I use for the
> demo site, its pretty easy, and they have a free tier for open source
> projects - pro with unlimited users -
> https://www.netlify.com/legal/open-source-policy/
>
> I think 1 is really gross and don't want to do it.
>
> But looking for more ideas/feedback.
>
> Gavin
> _______________________________________________
> Jenkins-infra mailing list
> Jenkins-infra at lists.jenkins-ci.org
> http://lists.jenkins-ci.org/mailman/listinfo/jenkins-infra
>
>
> _______________________________________________
> Jenkins-infra mailing list
> Jenkins-infra at lists.jenkins-ci.org
> http://lists.jenkins-ci.org/mailman/listinfo/jenkins-infra
>
> _______________________________________________
> Jenkins-infra mailing list
> Jenkins-infra at lists.jenkins-ci.org
> http://lists.jenkins-ci.org/mailman/listinfo/jenkins-infra
>
>
>
> _______________________________________________
> Jenkins-infra mailing list
> Jenkins-infra at lists.jenkins-ci.org
> http://lists.jenkins-ci.org/mailman/listinfo/jenkins-infra
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.jenkins-ci.org/pipermail/jenkins-infra/attachments/20200109/e0cc4c12/attachment-0001.html>


More information about the Jenkins-infra mailing list