[Jenkins-infra] Request for feedback: IEP-002 - Azure Virtual Networks for Cluster Segregation

R. Tyler Croy tyler at monkeypox.org
Fri Nov 18 18:34:00 UTC 2016

(bringing Ben's comments onto  the list for further discussion, replies inline)

On Fri, 18 Nov 2016, Ben Walding wrote:

> Looks sane enough to me.
> (I have an AWS centric view of the world - and Azure is similar but
> different to AWS - so I'm not 100% sure you can do everything below)
> *Networks*
> I would also consider separating your static infrastructure from your
> dynamic CI infrastructure - i.e. Confluence / JIRA etc would be static
> infrastructure, whereas the build farms would be dynamic CI infrastructure.

I hadn't considered this, could you expand more on the reasoning behind this?
Are there limits on public IPs or something from within VPCs that make this a
prudent choice on AWS? I'm not sure what the benefit of peering two Virtual
Networks would be between static and dynamic workloads in "Public Production."

To give you a sense of size, our dynamic Jenkins agents at their max
utilization are only about 10-15 VMs.

> *Accounts*
> You could also consider separating your infrastructure into multiple Azure
> accounts.
> This is related to your network design as your VPCs are per account - so
> your infrastructure for a given VPC would need to live on that account.
> Separate accounts would also let you
>    - separate the users / groups / roles (no idea how Azure does all this
>    with AD - I want to cry in the corner every time I use Azure security - in
>    AWS you can have a separate account people log into, then they assume-role
>    into the working accounts)
>    - limit the impact of a breach (resource level permissions are very
>    limited even in AWS - I imagine Azure would be similar)
>    - limit the impact of a runaway process - I assume Azure has API limits
>    - you don't want developers / CI taking out your production management
>    infrastructure.

In Azure "accounts" are loosely translated to "Subscriptions" and for the
Jenkins project's partnership we have a single Subscription which is sponsored
by Microsoft. This means the same patterns of account balkanization won't work on
Azure, since we're stuck under one subscription.

Azure does have the concept of Resource Groups for all resources, such as
Virtual Networks. To take the pattern you suggest above as an example, in Azure
I would separate concerns by provisioning (as examples):

 * A "static-dev-tools" Resource Group
   * confluence
   * confluence's SQL Server
   * JIRA
   * JIRA's SQL server
   * etc
 * A "jenkins-on-jenkins" Resource Group
   * jenkins
   * Azure Container Service for containerized agents
   * ???

And so on. Resource Groups then integrate with Azure Active Directory (AAD), and
allow role-based access control to specific specific resources or resource

The concerns, summarized, of:

  1. User/group separation
  2. Limit breach impact
  3. Limiting impact of runaway API accesses
  4. Prevent developers/CI from taking out the infrastructure management plane

I believe, in lieu of Account/Subscription separation, 1,2,4 can be addressed
with Resource Groups and RBAC in AAD.

The Virtual Networks themselves, I would propose should be deployed into their
own Resource Groups with heavily locked down access control, instead of trying
to share a Resource Group with "static-dev-tools" for example.

Does this approach address your concerns about separating Virtual Networks out
into separate Accounts/Subscriptions (since we can't actually accomplish that)?

> There's probably more stuff I will think of later.
> I am happy to post this to list as well if you want.

Your input is always welcome on the Jenkins project infrastructure list :)


- R. Tyler Croy

     Code: <https://github.com/rtyler>
  Chatter: <https://twitter.com/agentdero>

  % gpg --keyserver keys.gnupg.net --recv-key 1426C7DC3F51E16F
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 181 bytes
Desc: Digital signature
URL: <http://lists.jenkins-ci.org/pipermail/jenkins-infra/attachments/20161118/158f703c/attachment.asc>

More information about the Jenkins-infra mailing list