Skip to content

Temporary node pool pattern

The Temporary node pool pattern is an infrastructure strategy used during Kubernetes maintenance to facilitate zero-downtime upgrades of node pools, specifically within Google Kubernetes Engine (GKE) environments^[400-devops__05-Cloud-Provider__GCP升级.md]. Instead of upgrading nodes in-place, this method introduces a temporary resource to handle workload traffic during the transition process^[400-devops__05-Cloud-Provider__GCP升级.md].

Procedure

The pattern begins with the creation of a secondary node pool, often named temp or similar, to serve as a temporary environment^[400-devops__05-Cloud-Provider__GCP升级.md]. While existing node pools (e.g., app) continue to run, the GKE version specified in the infrastructure configuration (such as Terraform variables) is updated^[400-devops__05-Cloud-Provider__GCP升级.md].

Once the temporary pool is provisioned, the workloads are systematically migrated. This is typically achieved by updating the nodeSelector fields in Kubernetes manifests to point from the original pool (e.g., pool: app) to the temporary pool (e.g., pool: temp)^[400-devops__05-Cloud-Provider__GCP升级.md]. To ensure stability, components may be migrated sequentially with intervals—for example, rotating Kafka nodes every three minutes^[400-devops__05-Cloud-Provider__GCP升级.md].

After traffic is fully shifted to the temporary pool, the original node pool is upgraded to the new GKE version^[400-devops__05-Cloud-Provider__GCP升级.md]. Finally, the nodeSelector is reverted to point back to the now-upgraded original pool, completing the cycle^[400-devops__05-Cloud-Provider__GCP升级.md].

Implementation Details

In a typical Infrastructure as Code (IaC) workflow, the temporary pool is often managed by temporarily renaming a configuration file (e.g., from 11-app.tf.back to 11-app.tf) and applying the changes^[400-devops__05-Cloud-Provider__GCP升级.md]. This ensures the temporary infrastructure exists only for the duration of the upgrade operation and is removed once the production pool is stabilized^[400-devops__05-Cloud-Provider__GCP升级.md].

Sources

  • 400-devops__05-Cloud-Provider__GCP升级.md