multi-cluster
This commit is contained in:
@@ -16,7 +16,7 @@ This Kubernetes cluster uses a **GitOps approach** powered by **ArgoCD**, where
|
||||
|
||||
### Key Characteristics
|
||||
- **Environment**: Production (internal use only)
|
||||
- **Cluster Type**: Single cluster, single environment
|
||||
- **Cluster Type**: Multi-cluster (upc-dev, upc-prod) via Kustomize overlays
|
||||
- **GitOps Tool**: ArgoCD
|
||||
- **Deployment Pattern**: App-of-Apps
|
||||
- **Secret Management**: Sealed Secrets (kubeseal)
|
||||
@@ -62,8 +62,8 @@ This Kubernetes cluster uses a **GitOps approach** powered by **ArgoCD**, where
|
||||
│
|
||||
▼
|
||||
┌────────────────────────────────┐
|
||||
│ Kubernetes Cluster │
|
||||
│ (UpCloud Managed) │
|
||||
│ Kubernetes Clusters │
|
||||
│ (UpCloud: upc-dev, upc-prod) │
|
||||
│ │
|
||||
│ ┌──────────────────────────┐ │
|
||||
│ │ ArgoCD │ │
|
||||
@@ -116,74 +116,68 @@ This Kubernetes cluster uses a **GitOps approach** powered by **ArgoCD**, where
|
||||
```
|
||||
launchpad/
|
||||
├── bootstrap.sh # Cluster initialization script
|
||||
├── _app-of-apps.yaml # Root ArgoCD Application (App-of-Apps pattern)
|
||||
├── _app-of-apps-upc-dev.yaml # Root ArgoCD Application (upc-dev cluster)
|
||||
├── _app-of-apps-upc-prod.yaml # Root ArgoCD Application (upc-prod cluster)
|
||||
│
|
||||
├── infra/ # Infrastructure ArgoCD Applications
|
||||
│ ├── enterprise-apps.yaml # Parent app managing all apps in apps/
|
||||
│ ├── cluster-resources-application.yaml
|
||||
│ ├── traefik-application.yaml
|
||||
│ ├── cert-manager-application.yaml
|
||||
│ ├── kyverno.yaml
|
||||
│ ├── kyverno-policies.yaml
|
||||
│ ├── prometheus.yaml
|
||||
│ ├── grafana.yaml
|
||||
│ ├── loki.yaml
|
||||
│ ├── tempo.yaml
|
||||
│ ├── fluent-bit.yaml
|
||||
│ ├── trivy.yaml
|
||||
│ ├── sealedsecrets.yaml
|
||||
│ ├── secrets.yaml
|
||||
├── infra/ # Infrastructure ArgoCD Applications (Kustomize)
|
||||
│ ├── base/ # Base Application manifests (upc-dev defaults)
|
||||
│ │ ├── kustomization.yaml
|
||||
│ │ ├── traefik-application.yaml
|
||||
│ │ ├── keycloak.yaml
|
||||
│ │ ├── grafana.yaml
|
||||
│ │ ├── gitea.yaml
|
||||
│ │ ├── gitea-actions.yaml
|
||||
│ │ ├── tempo.yaml
|
||||
│ │ ├── renovate.yaml
|
||||
│ │ ├── ... # All other Application manifests
|
||||
│ │ └── secrets.yaml
|
||||
│ ├── overlays/ # Per-cluster overrides
|
||||
│ │ ├── upc-dev/ # UpCloud Dev (uses base as-is)
|
||||
│ │ └── upc-prod/ # UpCloud Prod (patches value paths)
|
||||
│ ├── dashboards/ # Grafana dashboard ConfigMaps
|
||||
│ └── values/ # Helm value overrides for infra
|
||||
│ ├── argocd-values.yaml
|
||||
│ ├── prometheus-values.yaml
|
||||
│ ├── grafana-values.yaml
|
||||
│ ├── loki-values.yaml
|
||||
│ ├── tempo-values.yaml
|
||||
│ └── fluent-bit-values.yaml
|
||||
│ ├── base/ # Shared values (all clusters)
|
||||
│ │ ├── traefik-values.yaml
|
||||
│ │ ├── keycloak-values.yaml
|
||||
│ │ ├── grafana-values.yaml
|
||||
│ │ ├── prometheus-values.yaml
|
||||
│ │ ├── gitea-values.yaml
|
||||
│ │ └── ...
|
||||
│ ├── upc-dev/ # upc-dev cluster-specific values
|
||||
│ │ ├── traefik-values.yaml
|
||||
│ │ ├── keycloak-values.yaml
|
||||
│ │ └── grafana-values.yaml
|
||||
│ └── upc-prod/ # upc-prod cluster-specific values
|
||||
│ ├── traefik-values.yaml
|
||||
│ ├── keycloak-values.yaml
|
||||
│ └── grafana-values.yaml
|
||||
│
|
||||
├── apps/ # Business Application ArgoCD manifests
|
||||
│ ├── mcp10x.yaml # MCP 10X application
|
||||
│ ├── musicman.yaml # Music Man application
|
||||
│ ├── dot-ai-stack.yaml # Dot AI Stack
|
||||
│ └── argo-mcp.yaml # ArgoCD MCP server
|
||||
├── apps/ # Business Application ArgoCD manifests (Kustomize)
|
||||
│ ├── base/ # Base app manifests
|
||||
│ │ ├── kustomization.yaml
|
||||
│ │ ├── dot-ai-stack.yaml
|
||||
│ │ └── ...
|
||||
│ └── overlays/
|
||||
│ ├── upc-dev/ # Uses base as-is
|
||||
│ └── upc-prod/ # Patches value paths
|
||||
│
|
||||
├── cluster-resources/ # Cluster-wide Kubernetes resources
|
||||
│ ├── cert-manager-namespace.yaml
|
||||
│ ├── secrets-namespace.yaml
|
||||
│ ├── letsencrypt-issuer.yaml # Let's Encrypt ClusterIssuer
|
||||
│ ├── kyverno-config.yaml
|
||||
│ ├── argocd-notifications-secret-sealed.yaml
|
||||
│ ├── forte10x-repo-credentials-sealed.yaml
|
||||
│ ├── mcp10x-repo-credentials-sealed.yaml
|
||||
│ ├── ...
|
||||
│ └── policies/ # Kyverno policies
|
||||
│ ├── deployment-verifier.yaml
|
||||
│ ├── label-checker.yaml
|
||||
│ ├── bare-pod-cleaner.yaml
|
||||
│ ├── replicaset-cleaner.yaml
|
||||
│ ├── default-ns-blocker.yaml
|
||||
│ ├── secret-cloner.yaml
|
||||
│ └── auth-sidecar-injector.yaml
|
||||
│
|
||||
├── secrets/ # Application secrets (sealed)
|
||||
│ ├── argocd-mcp-credentials.yaml
|
||||
│ ├── dot-ai-secrets.yaml
|
||||
│ ├── mcp10x-credentials-sealed.yaml
|
||||
│ └── musicman-credentials.yaml
|
||||
├── secrets/ # Application secrets (sealed, per-cluster)
|
||||
│ └── upc-dev/ # Secrets for upc-dev cluster
|
||||
│
|
||||
├── private/ # Local-only files (NOT in Git)
|
||||
│ ├── *.yaml # Unsealed secrets
|
||||
│ └── *.sh # Helper scripts
|
||||
│
|
||||
└── docs/ # Documentation
|
||||
├── GITOPS-ARCHITECTURE.md # This file
|
||||
├── DEVELOPER-GUIDE.md
|
||||
├── OPERATIONS-RUNBOOK.md
|
||||
└── REFERENCE.md
|
||||
```
|
||||
|
||||
**Key Points**:
|
||||
- `_app-of-apps.yaml` is the root Application that ArgoCD monitors
|
||||
- `infra/enterprise-apps.yaml` auto-discovers all apps in `apps/` folder
|
||||
- `_app-of-apps-upc-dev.yaml` and `_app-of-apps-upc-prod.yaml` are the per-cluster root Applications
|
||||
- Kustomize overlays in `infra/overlays/` render base Applications with per-cluster patches
|
||||
- Helm values are split: `values/base/` (shared) + `values/upc-dev/` or `values/upc-prod/` (cluster-specific)
|
||||
- `apps/` follows the same base/overlays pattern for business applications
|
||||
- Changes pushed to this repo trigger automatic syncs in ArgoCD
|
||||
- `private/` folder contains local-only files (Git-ignored)
|
||||
|
||||
@@ -295,7 +289,7 @@ app-repository/
|
||||
### The App-of-Apps Pattern
|
||||
|
||||
```
|
||||
_app-of-apps.yaml (Root)
|
||||
_app-of-apps-{upc-dev,upc-prod}.yaml (Root, per cluster)
|
||||
│
|
||||
├── infrastructure-apps (manages infra/)
|
||||
│ ├── cluster-resources-application
|
||||
@@ -315,10 +309,10 @@ _app-of-apps.yaml (Root)
|
||||
```
|
||||
|
||||
**How It Works**:
|
||||
1. Bootstrap script installs ArgoCD and applies `_app-of-apps.yaml`
|
||||
2. ArgoCD creates the root Application which monitors `infra/` folder
|
||||
3. Each YAML in `infra/` becomes a child Application
|
||||
4. `enterprise-apps.yaml` monitors `apps/` folder and auto-discovers applications
|
||||
1. Bootstrap script installs ArgoCD and applies `_app-of-apps-upc-dev.yaml` (or `upc-prod`)
|
||||
2. ArgoCD creates the root Application which monitors the appropriate `infra/overlays/` folder
|
||||
3. Kustomize renders base Applications with cluster-specific patches
|
||||
4. `enterprise-apps` Application monitors the cluster's `apps/overlays/` folder
|
||||
5. ArgoCD continuously syncs (every 60s) and auto-heals drift
|
||||
|
||||
### Sync Waves & Ordering
|
||||
@@ -363,6 +357,34 @@ spec:
|
||||
- Easy to update all apps by changing the chart
|
||||
- Environment-specific values isolated in separate repo
|
||||
|
||||
### Multi-Cluster Pattern
|
||||
|
||||
Kustomize overlays enable deploying the same Applications across clusters with different configurations:
|
||||
|
||||
```yaml
|
||||
# infra/base/ contains default (upc-dev) Applications
|
||||
# Helm values are layered: base + cluster-specific
|
||||
valueFiles:
|
||||
- $values/infra/values/base/traefik-values.yaml # Shared config
|
||||
- $values/infra/values/upc-dev/traefik-values.yaml # Cluster-specific
|
||||
|
||||
# infra/overlays/upc-prod/kustomization.yaml patches the second valueFile
|
||||
patches:
|
||||
- target:
|
||||
kind: Application
|
||||
name: traefik
|
||||
patch: |
|
||||
- op: replace
|
||||
path: /spec/sources/0/helm/valueFiles/1
|
||||
value: $values/infra/values/upc-prod/traefik-values.yaml
|
||||
```
|
||||
|
||||
**Benefits**:
|
||||
- Single source of truth for Application definitions
|
||||
- Cluster-specific values isolated per overlay
|
||||
- Easy to add new clusters by creating a new overlay
|
||||
- Base values shared across all clusters reduce duplication
|
||||
|
||||
---
|
||||
|
||||
## CI/CD Pipeline
|
||||
|
||||
Reference in New Issue
Block a user