Skip to content

Apollo containerized deployment on Kubernetes

Apollo is a reliable configuration management center developed by Ctrip. In Kubernetes environments, Apollo is typically deployed in a containerized architecture to separate configuration management from application images, ensuring the immutability of container images^[400-devops__06-Kubernetes__k8s-paas__06.在K8S中集成Apollo配置中心.md].

This page details the process of deploying the Apollo configuration platform onto a Kubernetes cluster, including database preparation, container image creation, resource list management, and multi-environment deployment strategies^[400-devops__06-Kubernetes__k8s-paas__06.在K8S中集成Apollo配置中心.md].

Architecture Overview

The containerized deployment of Apollo consists of several core components:

  • Apollo ConfigService: Provides configuration reading services and acts as a service registration center (Eureka).
  • Apollo AdminService: Provides configuration management and modification services.
  • Apollo Portal: The management Web UI for configuring projects and namespaces.
  • MySQL Database: Stores configuration data and metadata.

Pre-deployment: Database Preparation

Before deploying the Apollo services, the underlying database must be initialized.

  1. Install Database: Deploy a MySQL instance (e.g., MariaDB 10.1) with character set settings configured to utf8mb4^[400-devops__06-Kubernetes__k8s-paas__06.在K8S中集成Apollo配置中心.md].
  2. Initialize Schemas:
    • ConfigDB: Download the initialization SQL (e.g., V1.0.0__initialization.sql) from the official repository to create the ApolloConfigDB^[400-devops__06-Kubernetes__k8s-paas__06.在K8S中集成Apollo配置中心.md].
    • PortalDB: Download the portal SQL (e.g., apolloportaldb.sql) to create the ApolloPortalDB^[400-devops__06-Kubernetes__k8s-paas__06.在K8S中集成Apollo配置中心.md].
  3. Configure Database Access: Create dedicated database users (e.g., apolloconfig, apolloportal) and grant necessary permissions^[400-devops__06-Kubernetes__k8s-paas__06.在K8S中集成Apollo配置中心.md].
  4. Update Eureka URLs: Modify the ServerConfig table in the corresponding databases to set the eureka.service.url to the intended service address (e.g., http://config.od.com/eureka)1^[400-devops__06-Kubernetes__k8s-paas__06.在K8S中集成Apollo配置中心.md].

Deployment Workflow

1. Create Docker Images

Each Apollo component requires a specific Docker image. The general process involves downloading the release package, creating a Dockerfile, and building the image^[400-devops__06-Kubernetes__k8s-paas__06.在K8S中集成Apollo配置中心.md].

Dockerfile Construction: * Base Image: Use a JRE base image (e.g., jre8:8u112). * Application: Add the component's JAR file (e.g., apollo-configservice-1.5.1.jar). * Configuration: Mount application-github.properties via Kubernetes ConfigMap to decouple configuration from the image^[400-devops__06-Kubernetes__k8s-paas__06.在K8S中集成Apollo配置中心.md]. * Startup Script: Modify the default startup.sh script. Key adjustments include: * Setting APOLLO_CONFIG_SERVICE_NAME=$(hostname -i) to ensure the service registers with its Pod IP instead of an unreachable hostname^[400-devops__06-Kubernetes__k8s-paas__06.在K8S中集成Apollo配置中心.md]. * Adjusting JVM memory settings (e.g., -Xms128m -Xmx128m) to suit container resources^[400-devops__06-Kubernetes__k8s-paas__06.在K8S中集成Apollo配置中心.md].

Example Build Commands: 

docker build . -t harbor.od.com/infra/apollo-configservice:v1.5.1
docker push harbor.od.com/infra/apollo-configservice:v1.5.1

2. Define Kubernetes Resources

For each component, define the following Kubernetes Resource Manifests (typically stored in a dedicated directory like /data/k8s-yaml/apollo-configservice)2:

  • ConfigMap (cm.yaml): Contains non-sensitive environment configuration, such as database connection strings and Eureka service URLs^[400-devops__06-Kubernetes__k8s-paas__06.在K8S中集成Apollo配置中心.md].
  • Deployment (dp.yaml): Defines the application deployment.
    • Mounts the ConfigMap to the container's config directory (e.g., /apollo-configservice/config)3^[400-devops__06-Kubernetes__k8s-paas__06.在K8S中集成Apollo配置中心.md].
    • Sets restartPolicy: Always.
  • Service (svc.yaml): Exposes the application within the cluster (typically on port 8080).
  • Ingress (ingress.yaml): Exposes the service via a hostname (e.g., config.od.com, portal.od.com) to external traffic^[400-devops__06-Kubernetes__k8s-paas__06.在K8S中集成Apollo配置中心.md].

3. Component Deployment Sequence

Apply the resources in the following order:

  1. ConfigService:
    • Apply ConfigMap, Deployment, Service, and Ingress.
    • Verify accessibility via the configured URL (e.g., http://config.od.com).
  2. AdminService:
    • Apply resources. It connects to the same ConfigDB and registers with ConfigService.
  3. Portal:
    • Apply resources.
    • Critical Configuration: The Portal's ConfigMap must define apollo-env.properties to map environments (e.g., dev.meta, fat.meta) to the respective ConfigService addresses^[400-devops__06-Kubernetes__k8s-paas__06.在K8S中集成Apollo配置中心.md].

Multi-Environment Deployment

To support multiple environments (e.g., FAT for testing, PRO for production) using a single image:

  1. Namespace Isolation: Create separate Kubernetes Namespaces for each environment (e.g., test, prod)4^[400-devops__06-Kubernetes__k8s-paas__06.在K8S中集成Apollo配置中心.md].
  2. Database Separation: Use distinct database schemas (e.g., ApolloConfigTestDB, ApolloConfigProdDB) for each environment^[400-devops__06-Kubernetes__k8s-paas__06.在K8S中集成Apollo配置中心.md].
  3. Dedicated Deployments: Deploy separate instances of ConfigService and AdminService into the respective namespaces.
    • Update the ConfigMap in each namespace to point to the correct database and Eureka URL (e.g., http://config-test.od.com/eureka).
  4. Portal Configuration: Update the ApolloPortalDB.ServerConfig to support multiple environments (e.g., fat,pro) and configure the Portal ConfigMap apollo-env.properties to map fat.meta and pro.meta to their respective ConfigService URLs^[400-devops__06-Kubernetes__k8s-paas__06.在K8S中集成Apollo配置中心.md].

Application Integration

Applications integrating with Apollo in Kubernetes need specific environment variables set in their Deployment manifests^[400-devops__06-Kubernetes__k8s-paas__06.在K8S中集成Apollo配置中心.md]:

  • C_OPTS:
    • -Denv=dev: Defines the Apollo environment (e.g., fat, pro).
    • -Dapollo.meta=http://config.od.com: Specifies the ConfigService address for the specific environment. In namespaces where ConfigService is local, this can be the internal service URL (e.g., http://apollo-configservice:8080)^[400-devops__06-Kubernetes__k8s-paas__06.在K8S中集成Apollo配置中心.md].

Footnotes

Sources

^[400-devops__06-Kubernetes__k8s-paas__06.在K8S中集成Apollo配置中心.md]

  1. The URL in the database must match the Ingress or Service address exposed by Kubernetes. ↩

  2. File paths like /data/k8s-yaml/... refer to a centralized configuration repository accessible via HTTP (e.g., k8s-yaml.od.com). ↩

  3. Using ConfigMaps allows for configuration changes without rebuilding the image, adhering to the "Configuration over Hard-coding" principle. ↩

  4. Namespaces provide logical isolation, allowing the same application release to be deployed to different environments with different configurations. ↩