Running Azure DevOps Pipeline Build Agent in OpenShift

Adopted from Azure DevOps with Managed OpenShift by Kevin Collins [1] and Hosting an Azure Pipelines Build Agent in OpenShift by Kevin Chung, and Mark Dunnett [2]

In my daily job, I have assisted many of my clients to embark on their modernization journey. One of the key components is the CI/CD pipeline. All clients would have different choices of CI/CD pipelines with their different perspectives and justifications. Because of that, I got the chance to work on different CI/CD pipelines, such as GitLab CI, Jenkins, GitHub Action, etc. They are all meant to achieve the same thing, to deliver the code in an automated and controlled fashion. In turn, it will deliver the business value faster to the market.

Recently, I got the opportunity to work on Azure DevOps Pipeline with my client. The Azure pipeline view is integrated into Azure DevOps, providing a nice seamless view of all the interconnected development components, like the source code repository, Kanban, test plan, etc.

In Azure Pipelines, you have the option to run the build agent in various places. A build agent is the actual worker instance that will execute the pipeline jobs. Here, I am going to run it in OpenShift as a self-hosted agent. Why? Because, in my opinion, it is easier to spin up and manage a pod in OpenShift, as compared to standing up a VM or a bare metal machine. Secondly, we will have the option to scale the build agents according to the build workload later using Keda [3]. Thirdly, because I am a Red Hatter, so I am biased :)

Building the Build Agent Podman Image

First, let’s build the Podman image using the Containerfile as shown below, or you can also grab it from my GitHub repo here [5].

You can use any base image, my choice here is to use a builder image to build .Net applications. The most important thing is that we need to bake in the binary file of the build agent, as shown in line 9. This binary will be the one that connects to the Azure DevOps later, signalling that this machine is a build agent. To build the image, simply execute the Podman build command

podman build -t azp-agent:linux \
     -f Containerfile \
     --platform linux/amd64 \
     --env AZP_AGENT_PACKAGE_LATEST_URL=https://vstsagentpackage.azureedge.net/agent/2.210.1/vsts-agent-linux-x64-2.210.1.tar.gz

Push the Image to the OCP Internal Image Registry

This step right here, is purely my personal preference and for the sake of the quick POC. You can also use an external image registry. So here, I am going to enable the route of my OCP internal image registry, so that I can connect to it from external.

 oc patch configs.imageregistry.operator.openshift.io/cluster --patch '{"spec":{"defaultRoute":true}}' --type=merge

 export REGISTRY=`oc get route default-route -n openshift-image-registry --template='{{ .spec.host }}'`

Next, push the image! Note that REGISTRY is just the placeholder of the image registry URL.

 oc new-project azure-build-1

 oc create imagestream azure-build-agent

 podman login -u <username> -p `oc whoami --show-token` ${REGISTRY}

 podman tag azp-agent:linux ${REGISTRY}/azure-build-1/azure-build-agent:latest

 podman push ${REGISTRY}/azure-build-1/azure-build-agent:latest

Create Service Account

Running the build agent in OpenShift requires certain permission. To do that, we will create a new service account and grant the right permission to it.

oc create serviceaccount azure-build-sa
oc create -f openshift/agent/base/scc.yaml

# you will need cluster admin permission 
oc adm policy add-scc-to-user nonroot-builder -z azure-build-sa

Create Secrets

Next, we will need to create an OpenShift secret to inject a few environment variables into the deployment. These will be used by the build agent binary to connect to the Azure DevOps.

# setting the env var
export AZP_URL=https://dev.azure.com/<Azure DevOps Org Name>
export AZP_TOKEN=<personal access token in Azure DevOps>
export AZP_POOL=<Agent Pool name, by default is 'Default'>

# create the secret using the 3 env var
oc create secret generic azdevops \                 
--from-literal=AZP_URL=${AZP_URL} \
--from-literal=AZP_TOKEN=${AZP_TOKEN} \
--from-literal=AZP_POOL=${AZP_POOL}

Deploy!

If you have Git cloned my GitHub repo, you can use the command below to deploy it into your OpenShift cluster. Here, I am using Kustomize to apply the deployment file, because I am planning to deploy different kinds of build agents in future. Otherwise, you can modify the deployment file yourself and just do a oc apply -f

oc apply -k openshift/agent/azure-build-1

If you have done everything correctly, you should see your Azure build agent pod running in OpenShift, and connected to your Azure Devops. You may check the status of the build agent under your Azure project > Project settings > Agent Pools.

Congratulations on running your Azure Build Agent in OpenShift! Next time, we will also look into building the pipeline itself, stay tune!

References

[1] Azure DevOps with Managed
OpenShifthttps://cloud.redhat.com/experts/misc/azure-dev-ops-with-managed-openshift/

[2] Hosting an Azure Pipelines Build Agent in OpenShift
https://www.redhat.com/en/blog/hosting-an-azure-pipelines-build-agent-in-openshift

[3] Autoscaling Azure Pipelines agents with KEDA
https://keda.sh/blog/2021-05-27-azure-pipelines-scaler/

[4] Run a self-hosted agent in Docker
https://learn.microsoft.com/en-us/azure/devops/pipelines/agents/docker?view=azure-devops

[5] GitHub Repo
https://github.com/wengkee/azure-devops-pipeline/blob/main/podman/agent/Containerfile