[Jenkins-infra] A random thought for reducing ci.j.io cost

R. Tyler Croy rtyler at brokenco.de
Sat Feb 8 00:10:29 UTC 2020


(replies inline)

On Fri, 07 Feb 2020, Jesse Glick wrote:

> 
> On Fri, Feb 7, 2020 at 4:22 PM R. Tyler Croy <rtyler at brokenco.de> wrote:
> > The downsides that come to mind
> 
> From the developer perspective, the obvious downside would be the
> likelihood that these machines would behave subtly differently from
> dynamically provisioned cloud machines / containers, leading to
> hard-to-diagnose occasional build failures. It is more attractive to
> have a homogeneous pool of agents provisioned from a cloud service,
> especially if the provisioning configuration is ???as code???.


I agree, the same-ness of our hosts in Azure right now is especially useful.
Other than using Puppet, like we have with fixed agents in the past, I don't
have a great solution in mind for how the sameness could be acheived by fixed
and cloud-based agents simultaneously.

The only idea that does come to me would be to revive some of my Packer work
and use Packer to create identical vhd/ami images which can be deployed on KVM
locally, and via AWS dynamically (as an example). That might even address
Mark's concerns about ensuring we have fixed Windows capacity.


> I continue to think that the main issue as outlined in INFRA-1633 is
> that we do way more builds than we really need, and that we could
> easily change CI configuration to skip many of them without much
> impact on workflow.

If the maintainers of core to not wish to practice continuous integration for
Jenkins core, that is not my call. That said, I would be willing to guess that
the slight majority of our workload is coming from the 900+ plugins in the
Plugins/ directory.


Either way, the overall infra problem is that we are burning cash. Our last
month was ~6k in VMs and ~1.8k in containers (ACI). I think fixed agents would
smooth out the spend quite a bit.




Cheers
--
GitHub:  https://github.com/rtyler

GPG Key ID: 0F2298A980EE31ACCA0A7825E5C92681BEF6CEA2
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 839 bytes
Desc: OpenPGP digital signature
URL: <http://lists.jenkins-ci.org/pipermail/jenkins-infra/attachments/20200208/b44ae41a/attachment-0001.asc>


More information about the Jenkins-infra mailing list