Skip to content

Jenkins persistence on Kubernetes

When deploying Jenkins on Kubernetes, data persistence is critical to ensure that configurations, job histories, and artifacts survive Pod restarts or rescheduling. The Jenkins container image stores all persistent data—defined as the Jenkins home directory—under the /var/jenkins_home path^[400-devops-06-kubernetes-devops-helm-helm-jenkins-readme.md].

Default Persistence Mechanism

By default, the Jenkins Helm chart utilizes a dynamically managed Persistent Volume Claim (PVC) to retain data across deployments^[400-devops-06-kubernetes-devops-helm-helm-jenkins-readme.md]. This automatic provisioning is supported on cloud providers like AWS (using gp2) and GKE, as well as local environments like minikube^[400-devops-06-kubernetes-devops-helm-helm-jenkins-readme.md].

This default behavior ensures that the state of the Jenkins controller is maintained independently of the Pod lifecycle.

Configuration Options

Users can configure persistence behavior through Helm values, specifically using persistence.volumes and persistence.mounts^[400-devops-06-kubernetes-devops-helm-helm-jenkins-readme.md].

Using Existing Claims

For scenarios where a Persistent Volume Claim already exists and should be attached to the Jenkins instance, users can reference the claim by name^[400-devops-06-kubernetes-devops-helm-helm-jenkins-readme.md].

To use an existing PVC: 1. Create the PersistentVolume. 2. Create the PersistentVolumeClaim. 3. Install the chart with persistence.existingClaim set to the name of the PVC^[400-devops-06-kubernetes-devops-helm-helm-jenkins-readme.md].

Storage Classes

The choice of backend storage can be controlled via the persistence.storageClass parameter^[400-devops-06-kubernetes-devops-helm-helm-jenkins-readme.md].

  • Undefined or Null: Uses the cluster's default provisioner (e.g., gp2 on AWS, standard on GKE)^[400-devops-06-kubernetes-devops-helm-helm-jenkins-readme.md].
  • Dash (-): Disables dynamic provisioning^[400-devops-06-kubernetes-devops-helm-helm-jenkins-readme.md].
  • Custom Name: Specifies a particular StorageClass to use^[400-devops-06-kubernetes-devops-helm-helm-jenkins-readme.md].

Performance Considerations

Certain combinations of volume types and filesystem formats may result in extended volume attach and mount times, sometimes exceeding 10 minutes^[400-devops-06-kubernetes-devops-helm-helm-jenkins-readme.md]. This issue often manifests as FailedMount warnings in the Pod event history when using fsGroup within the security context^[400-devops-06-kubernetes-devops-helm-helm-jenkins-readme.md].

To mitigate this on affected providers (such as Azure Disk or Alibaba Cloud), it is recommended to experiment with replacing fsGroup with supplementalGroups^[400-devops-06-kubernetes-devops-helm-helm-jenkins-readme.md]. This can be configured by overriding the Pod security context:

controller:
  podSecurityContextOverride:
    runAsNonRoot: true
    runAsUser: 1000
    supplementalGroups: [1000]

Backup and Restore

Because the persistent volume contains the core state of the CI/CD server, maintaining backups is a crucial operational practice. The Helm chart includes functionality to define a backup CronJob^[400-devops-06-kubernetes-devops-helm-helm-jenkins-readme.md].

Backup Strategy

A common strategy involves creating a CronJob that backs up the /var/jenkins_home directory to cloud object storage (e.g., Google Cloud Storage or AWS S3)^[400-devops-06-kubernetes-devops-helm-helm-jenkins-readme.md]. This requires: 1. Persistence Enabled: persistence.enabled must be true^[400-devops-06-kubernetes-devops-helm-helm-jenkins-readme.md]. 2. Cloud Credentials: A Kubernetes Secret containing service account keys (e.g., sa-credentials.json for GCS) must be created and referenced in the Helm values^[400-devops-06-kubernetes-devops-helm-helm-jenkins-readme.md].

Restore Procedure

Restoring from a backup involves using a Kubernetes Job to copy data from cloud storage back to the Jenkins Pod's /var/jenkins_home directory^[400-devops-06-kubernetes-devops-helm-helm-jenkins-readme.md]. This process typically requires: 1. RBAC Resources: A ServiceAccount and ClusterRole with permissions to get and exec into pods^[400-devops-06-kubernetes-devops-helm-helm-jenkins-readme.md]. 2. Copy Job: A Job (often using a utility image like maorfr/skbn) that transfers files from the storage bucket (e.g., gcs://bucket/path) to the Kubernetes PVC destination (k8s://namespace/pod-name/container/path)^[400-devops-06-kubernetes-devops-helm-helm-jenkins-readme.md].

After the data is restored, Jenkins requires a "Reload Configuration from Disk" to recognize the restored state^[400-devops-06-kubernetes-devops-helm-helm-jenkins-readme.md].

Sources

^[400-devops-06-kubernetes-devops-helm-helm-jenkins-readme.md]