multi-cluster

This commit is contained in:
2026-04-18 19:26:51 +02:00
parent 0d64249858
commit ae1c60cee3
48 changed files with 200 additions and 156 deletions

View File

@@ -16,7 +16,7 @@ This Kubernetes cluster uses a **GitOps approach** powered by **ArgoCD**, where
### Key Characteristics
- **Environment**: Production (internal use only)
- **Cluster Type**: Single cluster, single environment
- **Cluster Type**: Multi-cluster (upc-dev, upc-prod) via Kustomize overlays
- **GitOps Tool**: ArgoCD
- **Deployment Pattern**: App-of-Apps
- **Secret Management**: Sealed Secrets (kubeseal)
@@ -62,8 +62,8 @@ This Kubernetes cluster uses a **GitOps approach** powered by **ArgoCD**, where
┌────────────────────────────────┐
│ Kubernetes Cluster
│ (UpCloud Managed)
│ Kubernetes Clusters
│ (UpCloud: upc-dev, upc-prod)
│ │
│ ┌──────────────────────────┐ │
│ │ ArgoCD │ │
@@ -116,74 +116,68 @@ This Kubernetes cluster uses a **GitOps approach** powered by **ArgoCD**, where
```
launchpad/
├── bootstrap.sh # Cluster initialization script
├── _app-of-apps.yaml # Root ArgoCD Application (App-of-Apps pattern)
├── _app-of-apps-upc-dev.yaml # Root ArgoCD Application (upc-dev cluster)
├── _app-of-apps-upc-prod.yaml # Root ArgoCD Application (upc-prod cluster)
├── infra/ # Infrastructure ArgoCD Applications
│ ├── enterprise-apps.yaml # Parent app managing all apps in apps/
│ ├── cluster-resources-application.yaml
│ ├── traefik-application.yaml
│ ├── cert-manager-application.yaml
│ ├── kyverno.yaml
│ ├── kyverno-policies.yaml
│ ├── prometheus.yaml
│ ├── grafana.yaml
│ ├── loki.yaml
│ ├── tempo.yaml
├── fluent-bit.yaml
│ ├── trivy.yaml
│ ├── sealedsecrets.yaml
├── secrets.yaml
├── infra/ # Infrastructure ArgoCD Applications (Kustomize)
│ ├── base/ # Base Application manifests (upc-dev defaults)
│ ├── kustomization.yaml
│ ├── traefik-application.yaml
│ ├── keycloak.yaml
│ ├── grafana.yaml
│ ├── gitea.yaml
│ ├── gitea-actions.yaml
│ ├── tempo.yaml
│ ├── renovate.yaml
│ ├── ... # All other Application manifests
│ └── secrets.yaml
│ ├── overlays/ # Per-cluster overrides
│ ├── upc-dev/ # UpCloud Dev (uses base as-is)
│ └── upc-prod/ # UpCloud Prod (patches value paths)
│ ├── dashboards/ # Grafana dashboard ConfigMaps
│ └── values/ # Helm value overrides for infra
│ ├── argocd-values.yaml
├── prometheus-values.yaml
├── grafana-values.yaml
├── loki-values.yaml
├── tempo-values.yaml
└── fluent-bit-values.yaml
│ ├── base/ # Shared values (all clusters)
│ ├── traefik-values.yaml
│ ├── keycloak-values.yaml
│ ├── grafana-values.yaml
│ ├── prometheus-values.yaml
│ ├── gitea-values.yaml
│ │ └── ...
│ ├── upc-dev/ # upc-dev cluster-specific values
│ │ ├── traefik-values.yaml
│ │ ├── keycloak-values.yaml
│ │ └── grafana-values.yaml
│ └── upc-prod/ # upc-prod cluster-specific values
│ ├── traefik-values.yaml
│ ├── keycloak-values.yaml
│ └── grafana-values.yaml
├── apps/ # Business Application ArgoCD manifests
│ ├── mcp10x.yaml # MCP 10X application
│ ├── musicman.yaml # Music Man application
│ ├── dot-ai-stack.yaml # Dot AI Stack
│ └── argo-mcp.yaml # ArgoCD MCP server
├── apps/ # Business Application ArgoCD manifests (Kustomize)
│ ├── base/ # Base app manifests
│ ├── kustomization.yaml
│ ├── dot-ai-stack.yaml
│ └── ...
│ └── overlays/
│ ├── upc-dev/ # Uses base as-is
│ └── upc-prod/ # Patches value paths
├── cluster-resources/ # Cluster-wide Kubernetes resources
│ ├── cert-manager-namespace.yaml
│ ├── secrets-namespace.yaml
│ ├── letsencrypt-issuer.yaml # Let's Encrypt ClusterIssuer
│ ├── kyverno-config.yaml
│ ├── argocd-notifications-secret-sealed.yaml
│ ├── forte10x-repo-credentials-sealed.yaml
│ ├── mcp10x-repo-credentials-sealed.yaml
│ ├── ...
│ └── policies/ # Kyverno policies
│ ├── deployment-verifier.yaml
│ ├── label-checker.yaml
│ ├── bare-pod-cleaner.yaml
│ ├── replicaset-cleaner.yaml
│ ├── default-ns-blocker.yaml
│ ├── secret-cloner.yaml
│ └── auth-sidecar-injector.yaml
├── secrets/ # Application secrets (sealed)
── argocd-mcp-credentials.yaml
│ ├── dot-ai-secrets.yaml
│ ├── mcp10x-credentials-sealed.yaml
│ └── musicman-credentials.yaml
├── secrets/ # Application secrets (sealed, per-cluster)
── upc-dev/ # Secrets for upc-dev cluster
├── private/ # Local-only files (NOT in Git)
│ ├── *.yaml # Unsealed secrets
│ └── *.sh # Helper scripts
└── docs/ # Documentation
├── GITOPS-ARCHITECTURE.md # This file
├── DEVELOPER-GUIDE.md
├── OPERATIONS-RUNBOOK.md
└── REFERENCE.md
```
**Key Points**:
- `_app-of-apps.yaml` is the root Application that ArgoCD monitors
- `infra/enterprise-apps.yaml` auto-discovers all apps in `apps/` folder
- `_app-of-apps-upc-dev.yaml` and `_app-of-apps-upc-prod.yaml` are the per-cluster root Applications
- Kustomize overlays in `infra/overlays/` render base Applications with per-cluster patches
- Helm values are split: `values/base/` (shared) + `values/upc-dev/` or `values/upc-prod/` (cluster-specific)
- `apps/` follows the same base/overlays pattern for business applications
- Changes pushed to this repo trigger automatic syncs in ArgoCD
- `private/` folder contains local-only files (Git-ignored)
@@ -295,7 +289,7 @@ app-repository/
### The App-of-Apps Pattern
```
_app-of-apps.yaml (Root)
_app-of-apps-{upc-dev,upc-prod}.yaml (Root, per cluster)
├── infrastructure-apps (manages infra/)
│ ├── cluster-resources-application
@@ -315,10 +309,10 @@ _app-of-apps.yaml (Root)
```
**How It Works**:
1. Bootstrap script installs ArgoCD and applies `_app-of-apps.yaml`
2. ArgoCD creates the root Application which monitors `infra/` folder
3. Each YAML in `infra/` becomes a child Application
4. `enterprise-apps.yaml` monitors `apps/` folder and auto-discovers applications
1. Bootstrap script installs ArgoCD and applies `_app-of-apps-upc-dev.yaml` (or `upc-prod`)
2. ArgoCD creates the root Application which monitors the appropriate `infra/overlays/` folder
3. Kustomize renders base Applications with cluster-specific patches
4. `enterprise-apps` Application monitors the cluster's `apps/overlays/` folder
5. ArgoCD continuously syncs (every 60s) and auto-heals drift
### Sync Waves & Ordering
@@ -363,6 +357,34 @@ spec:
- Easy to update all apps by changing the chart
- Environment-specific values isolated in separate repo
### Multi-Cluster Pattern
Kustomize overlays enable deploying the same Applications across clusters with different configurations:
```yaml
# infra/base/ contains default (upc-dev) Applications
# Helm values are layered: base + cluster-specific
valueFiles:
- $values/infra/values/base/traefik-values.yaml # Shared config
- $values/infra/values/upc-dev/traefik-values.yaml # Cluster-specific
# infra/overlays/upc-prod/kustomization.yaml patches the second valueFile
patches:
- target:
kind: Application
name: traefik
patch: |
- op: replace
path: /spec/sources/0/helm/valueFiles/1
value: $values/infra/values/upc-prod/traefik-values.yaml
```
**Benefits**:
- Single source of truth for Application definitions
- Cluster-specific values isolated per overlay
- Easy to add new clusters by creating a new overlay
- Base values shared across all clusters reduce duplication
---
## CI/CD Pipeline