Setting Up Jenkins on AWS EKS: A Guide to Containerized CI/CD

Modern software development requires fast, reliable, and scalable deployment pipelines. Continuous Integration and Continuous Deployment (CI/CD) help teams automate application building, testing, and deployment efficiently.

Jenkins is one of the most widely used CI/CD tools, while Amazon Elastic Kubernetes Service (EKS) provides scalable Kubernetes infrastructure. Running Jenkins on AWS EKS combines automation with container orchestration to build highly scalable cloud-native CI/CD pipelines.

Key Takeaways

Running Jenkins on AWS EKS helps organisations:

Build scalable CI/CD pipelines
Improve fault tolerance and reliability
Dynamically scale Jenkins workloads
Optimise infrastructure resource usage
Integrate seamlessly with AWS services

This setup combines Jenkins, Kubernetes, Helm, Amazon EBS, Prometheus, and AWS monitoring services to create a robust environment.

Why Use Jenkins on AWS EKS?

Deploying Jenkins on AWS EKS allows teams to leverage Kubernetes scalability and AWS cloud infrastructure while simplifying CI/CD operations.

Key Benefits

Scalability

Kubernetes automatically scales Jenkins workloads and build agents based on pipeline activity and resource demand. This ensures consistent performance during traffic spikes. Teams can handle more builds without manual intervention. As workloads decrease, resources are scaled down automatically.

High Availability

AWS EKS distributes Jenkins workloads across multiple nodes and Availability Zones. This reduces the risk of downtime caused by infrastructure failures. If one node becomes unavailable, workloads can be rescheduled automatically. The result is a more reliable and resilient CI/CD environment.

Better Resource Optimisation

Containerized Jenkins agents are created only when required and terminated after completing tasks. This prevents idle resources from consuming unnecessary infrastructure capacity. Organizations can optimize compute usage and improve efficiency. It also helps reduce overall cloud operating costs.

AWS Service Integration

Jenkins integrates seamlessly with AWS services such as Amazon S3, AWS CodeCommit, AWS Secrets Manager, and CloudWatch. This simplifies pipeline management and automation. Teams can securely store artifacts, manage secrets, and monitor workloads. The integration creates a streamlined and scalable DevOps workflow.

Prerequisites

Before starting, ensure the following tools and services are available:

An AWS account with sufficient permissions to create and manage EKS resources.
kubectl installed and configured to communicate with Kubernetes clusters.
eksctl installed for simplified Amazon EKS cluster provisioning and management.
Helm installed for deploying and managing Jenkins and other Kubernetes applications.
Basic knowledge of Kubernetes and Jenkins to understand the deployment and configuration process.

Also read : https://cloudzenia.com/blog/kubernetes-autoscaling-core-mechanisms-and-types/

Setting Up Jenkins on AWS EKS

Step 1: Create the EKS Cluster

First, create an EKS cluster that will host Jenkins workloads.

Create the Cluster Using eksctl

eksctl create cluster \
--name jenkins-eks-cluster \
--region us-west-2 \
--nodegroup-name jenkins-nodes \
--nodes 3 \
--nodes-min 1 \
--nodes-max 4 \
--managed

This command creates a managed EKS cluster with autoscaling worker nodes.

Update Kubeconfig

After the cluster is created, configure kubectl to communicate with EKS.

aws eks --region us-west-2 update-kubeconfig --name jenkins-eks-cluster

Configuring Persistent Storage for Jenkins

Jenkins requires persistent storage to retain build history, plugins, and configuration data even after Pod restarts.

Install AWS EBS CSI Driver

The EBS CSI driver enables Kubernetes to dynamically provision Amazon EBS storage volumes.

kubectl apply -k "github.com/kubernetes-sigs/aws-ebs-csi-driver/deploy/kubernetes/overlays/stable/ecr/?ref=release-1.0"

Create Persistent Volume Claim

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: jenkins-pvc
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 20Gi

This PVC allocates 20 GB of persistent storage for Jenkins workloads.

Deploying Jenkins on EKS

Create a Namespace

Using a dedicated namespace improves workload isolation and cluster management.

kubectl create namespace jenkins

Deploy Jenkins Using Helm

Helm simplifies application deployment on Kubernetes.

Add the Jenkins Helm Repository

helm repo add jenkinsci https://charts.jenkins.io
helm repo update

Install Jenkins

helm install jenkins jenkinsci/jenkins \
--namespace jenkins \
--set persistence.existingClaim=jenkins-pvc

Need help with your cloud infrastructure?

Our experts at CloudZenia are ready to help you build, scale, and secure your setup.

Book a Free Consultation

This deploys Jenkins with persistent storage enabled.

Accessing Jenkins

Retrieve the Admin Password

kubectl get secret --namespace jenkins jenkins \
-o jsonpath="{.data.jenkins-admin-password}" | base64 --decode

Access Jenkins Locally

kubectl port-forward svc/jenkins -n jenkins 8080:8080

Jenkins can now be accessed at:

http://localhost:8080

Configuring Jenkins for Kubernetes CI/CD

Install the Jenkins Kubernetes plugin to dynamically create Kubernetes-based build agents during pipeline execution.

Sample Jenkins Pipeline

pipeline {
  agent {
    kubernetes {
      yaml """
apiVersion: v1
kind: Pod
spec:
  containers:
  - name: build
    image: maven:3.6.3-jdk-8
    tty: true
"""
    }
  }

  stages {
    stage('Build') {
      steps {
        container('build') {
          sh 'mvn --version'
        }
      }
    }
  }
}

This pipeline creates temporary containerized build agents inside Kubernetes Pods.

Scaling Jenkins on AWS EKS

Efficient scaling helps improve CI/CD performance and optimise Kubernetes resources.

Enable Cluster Autoscaler

Cluster Autoscaler automatically adjusts node capacity based on Jenkins workload requirements.

helm install cluster-autoscaler stable/cluster-autoscaler

Use Dynamic Jenkins Agents

Dynamic Kubernetes agents are created only during active builds, reducing idle resource consumption.

Configure Horizontal Pod Autoscaler (HPA)

HPA scales Jenkins controller Pods automatically based on CPU and memory usage.

Monitoring Jenkins on AWS EKS

Monitoring helps maintain CI/CD reliability and cluster performance.

AWS CloudWatch

Amazon CloudWatch collects logs and infrastructure metrics for Jenkins workloads.

fields @timestamp, @message
| filter @message like /ERROR/
| sort @timestamp desc

Prometheus and Grafana

Prometheus and Grafana help monitor Pod performance, resource utilisation, and Jenkins build activity.

rate(container_cpu_usage_seconds_total{namespace="jenkins"}[5m])

Best Practices for Jenkins on AWS EKS

Monitor Resource Usage

Regularly monitoring Jenkins and Kubernetes resources helps prevent performance bottlenecks and unexpected failures. Tracking CPU, memory, storage usage, and Jenkins build queue metrics ensures that CI/CD pipelines continue running smoothly even during heavy workloads.

Tools like Prometheus, Grafana, and Amazon CloudWatch can help teams identify resource spikes, failed builds, and infrastructure issues early.

Optimise Build Agents

Using lightweight and containerized Jenkins build agents improves CI/CD pipeline speed and reduces infrastructure costs. Smaller Docker images start faster, consume fewer resources, and allow Kubernetes to schedule builds more efficiently.

Dynamic Kubernetes agents also help eliminate idle infrastructure because Pods are created only during active builds and removed automatically after pipeline execution.

Enable Autoscaling

Autoscaling ensures Jenkins workloads can handle changing build demands automatically without manual intervention. Kubernetes autoscaling mechanisms dynamically adjust Pods and cluster nodes depending on resource usage and pipeline activity.

Combining tools like Horizontal Pod Autoscaler (HPA), Cluster Autoscaler, and dynamic Jenkins agents improves scalability, workload distribution, infrastructure efficiency, and overall CI/CD reliability.

Also read : https://cloudzenia.com/blog/k8s-series-part-7-observability-and-monitoring-with-prometheus-and-grafana

Conclusion

Running Jenkins on Amazon Elastic Kubernetes Service helps organisations build scalable, reliable, and cloud-native CI/CD pipelines.

By combining Kubernetes orchestration with Jenkins automation, teams can improve deployment efficiency, scalability, fault tolerance, and infrastructure optimisation.

For deeper insights into AWS EKS, Kubernetes, Jenkins CI/CD pipelines, and cloud-native DevOps practices, visit CloudZenia.

Frequently Asked Questions

Q. Why should I run Jenkins on AWS EKS instead of traditional servers?

Running Jenkins on AWS EKS provides better scalability, high availability, and automated infrastructure management. Kubernetes dynamically scales Jenkins workloads and build agents, making CI/CD pipelines more efficient and resilient.

Q. Does Jenkins require persistent storage on EKS?

Yes. Jenkins stores build history, job configurations, plugins, and credentials. Using persistent storage such as Amazon EBS ensures that this data remains available even if Jenkins Pods restart or are rescheduled.

Q. How does Jenkins scale on AWS EKS?

Jenkins scales through Kubernetes features such as Dynamic Build Agents, Horizontal Pod Autoscaler (HPA), and Cluster Autoscaler. These tools automatically adjust Pods and worker nodes based on pipeline workload and resource usage.

Q. Which AWS services can be integrated with Jenkins on EKS?

Jenkins integrates with AWS services like Amazon S3 for artifact storage, AWS CodeCommit for source code management, AWS Secrets Manager for secure credential storage, and Amazon CloudWatch for monitoring and logging.

Q. How can I monitor Jenkins running on AWS EKS?

You can use Amazon CloudWatch to collect logs and infrastructure metrics, while Prometheus and Grafana provide detailed visibility into Jenkins performance, Kubernetes resource utilization, and CI/CD pipeline health.

Filed under

cloud