Forte/launchpad
6
0
Fork 0
Code Issues Pull Requests Actions Packages Wiki Activity

Compare commits

base: Forte:feature/multi-cloud
Branches Tags
Forte:main Forte:fix/drop-duplicate-keycloak-secret Forte:feature/dns01 Forte:feat/forte-drop-infra Forte:feature/ppusher Forte:feature/chibisafe Forte:hotfix/backup Forte:feature/vault-migration Forte:feature/hashicorp-vault Forte:feature/homepage Forte:feature/argocd-rbac Forte:feature/argocd-tls Forte:feature/multi-cloud Forte:feature/karpor Forte:feature/backstage Forte:feature/ai-review Forte:gitea-pages Forte:feature/gitea-docs Forte:feature/multicluster Forte:feature/secret-syncing Forte:feature/smtp Forte:renovate/configure
..
compare: Forte:ac0f464b2a6d772ba77c14d8b32d099c0d5c2d98
Branches Tags
Forte:main Forte:fix/drop-duplicate-keycloak-secret Forte:feature/dns01 Forte:feat/forte-drop-infra Forte:feature/ppusher Forte:feature/chibisafe Forte:hotfix/backup Forte:feature/vault-migration Forte:feature/hashicorp-vault Forte:feature/homepage Forte:feature/argocd-rbac Forte:feature/argocd-tls Forte:feature/multi-cloud Forte:feature/karpor Forte:feature/backstage Forte:feature/ai-review Forte:gitea-pages Forte:feature/gitea-docs Forte:feature/multicluster Forte:feature/secret-syncing Forte:feature/smtp Forte:renovate/configure

2 Commits
feature/mu ... ac0f464b2a

Author SHA1 Message Date
Danijel Simeunovic
ac0f464b2a fixes 2026-03-19 15:42:41 +01:00
Danijel Simeunovic
a681a9ae81 multi cluster 2026-03-18 22:28:38 +01:00
174 changed files with 1682 additions and 9945 deletions
Expand all files Collapse all files

47
.gitea/workflows/ai-review.yaml
View File

@@ -1,47 +0,0 @@
name: AI Code Review
on:
pull_request:
types: [ labeled, synchronize ]
jobs:
ai-review:
if: >-
(github.event.action == 'synchronized' && contains(toJSON(github.event.pull_request.labels), 'ai-review')) || contains(toJSON(gitea.event.changes.added_labels), 'ai-review')
runs-on: ubuntu-latest
env:
AI_REVIEW_CONFIG_FILE_YAML: ./shared-prompts/iac/.ai-review.yaml
# VCS configuration
VCS__PROVIDER: GITEA
VCS__PIPELINE__OWNER: ${{ github.repository_owner }}
VCS__PIPELINE__REPO: ${{ github.event.repository.name }}
VCS__PIPELINE__PULL_NUMBER: ${{ github.event.pull_request.number }}
VCS__HTTP_CLIENT__API_URL: https://git.forteapps.net/api/v1
VCS__HTTP_CLIENT__API_TOKEN: ${{ secrets.AI_REVIEW_TOKEN }}
# Review — disable fallback to see real Gitea API errors
REVIEW__INLINE_COMMENT_FALLBACK: "false"
# LLM configuration
LLM__PROVIDER: CLAUDE
LLM__META__MODEL: claude-sonnet-4-20250514
LLM__META__MAX_TOKENS: "4096"
LLM__HTTP_CLIENT__API_URL: https://api.anthropic.com
LLM__HTTP_CLIENT__API_TOKEN: ${{ secrets.ANTHROPIC_API_KEY }}
steps:
- name: Checkout repository
uses: actions/checkout@v4
with:
submodules: true
fetch-depth: 0
token: ${{ secrets.AI_REVIEW_TOKEN }}
- name: Run inline review
uses: docker://nikitafilonov/ai-review:v0.64.0
with:
args: ai-review run-inline
- name: Run summary review
uses: docker://nikitafilonov/ai-review:v0.64.0
with:
args: ai-review run-summary

2
.gitignore vendored
View File

@@ -6,6 +6,7 @@
# User-specific files (MonoDevelop/Xamarin Studio)
*.userprefs
.github/
private/
.helm/
temp/
@@ -15,4 +16,3 @@ CLAUDE.md
devbox.d/
devbox.lock
.devbox/
bash.exe.stackdump

3
.gitmodules vendored
View File

@@ -1,3 +0,0 @@
[submodule "shared-prompts"]
path = shared-prompts
url = https://git.forteapps.net/Forte/ai-review-prompts.git

96
README.md
View File

@@ -1,9 +1,9 @@
# Kubernetes Cluster - GitOps Configuration
> **Kubernetes cluster bootstrapping and GitOps configuration repository** using ArgoCD for multi-cloud Kubernetes (UpCloud, AWS EKS, Azure AKS, GCP GKE)
> **Kubernetes cluster bootstrapping and GitOps configuration repository** using ArgoCD for UpCloud Managed Kubernetes
[![GitOps](https://img.shields.io/badge/GitOps-ArgoCD-blue)](https://argoproj.github.io/cd/)
[![Kubernetes](https://img.shields.io/badge/Kubernetes-Multi--Cloud-orange)]()
[![Kubernetes](https://img.shields.io/badge/Kubernetes-UpCloud-orange)](https://upcloud.com/)
---
@@ -27,8 +27,8 @@
### For New Developers
```bash
# 1. Clone repositories
git clone https://git.forteapps.net/Forte/launchpad.git
git clone ssh://git@git.forteapps.net:2222/Forte/helm-prod-values.git
git clone https://github.com/fortedigital/sturdy-adventure.git
git clone git@github.com:fortedigital/helm-values.git
# 2. Read the guides
# - Start: docs/GITOPS-ARCHITECTURE.md
@@ -57,10 +57,10 @@ This repository contains the complete GitOps configuration for our Kubernetes cl
### What's Inside
- **Infrastructure Applications**: Traefik, Cert-Manager, Kyverno, Prometheus, Grafana, Loki, Tempo, Sealed Secrets
- **Infrastructure Applications**: Traefik, Cert-Manager, Kyverno, Prometheus, Grafana, Loki, Sealed Secrets
- **Business Applications**: MCP10X, MusicMan, Dot-AI Stack, ArgoCD MCP
- **Policies**: Kyverno security policies for secret management, namespace controls, pod verification
- **Monitoring**: Full observability stack with metrics, logs, traces, and alerting
- **Monitoring**: Full observability stack with metrics, logs, and alerting
- **Secrets**: Sealed Secrets for secure Git storage
### Key Features
@@ -72,7 +72,7 @@ This repository contains the complete GitOps configuration for our Kubernetes cl
✅ **Policy Enforcement**: Kyverno ensures security and compliance
✅ **Authentication**: Automatic sidecar injection (token & OIDC support)
✅ **TLS Everywhere**: Automatic Let's Encrypt certificates
✅ **Full Observability**: Prometheus, Grafana, Loki, Tempo integration
✅ **Full Observability**: Prometheus, Grafana, Loki integration
---
@@ -83,38 +83,18 @@ This repository contains the complete GitOps configuration for our Kubernetes cl
├── bootstrap.sh # Cluster initialization script
├── _app-of-apps.yaml # Root ArgoCD Application (App-of-Apps pattern)
│
├── infra/ # Infrastructure ArgoCD Applications (Kustomize multi-cluster)
│ ├── base/ # Base ArgoCD Application manifests (EU defaults)
│ │ ├── kustomization.yaml
│ │ ├── traefik-application.yaml
│ │ ├── keycloak.yaml
│ │ ├── grafana.yaml
│ │ ├── gitea.yaml
│ │ ├── gitea-actions.yaml
│ │ ├── tempo.yaml
│ │ ├── renovate.yaml
│ │ ├── ... # All other Application manifests
│ │ └── secrets.yaml
│ ├── overlays/ # Per-cluster overrides (Kustomize)
│ │ ├── upc-dev/ # UpCloud Dev (uses base as-is)
│ │ ├── upc-prod/ # UpCloud Prod (patches value paths)
│ │ ├── eks-dev/ # AWS EKS Dev
│ │ ├── eks-prod/ # AWS EKS Prod
│ │ ├── aks-dev/ # Azure AKS Dev
│ │ ├── aks-prod/ # Azure AKS Prod
│ │ ├── gke-dev/ # GCP GKE Dev
│ │ └── gke-prod/ # GCP GKE Prod
│ ├── dashboards/ # Grafana dashboard ConfigMaps
├── infra/ # Infrastructure ArgoCD Applications
│ ├── enterprise-apps.yaml # Manages all apps in apps/ folder
│ ├── traefik-application.yaml
│ ├── cert-manager-application.yaml
│ ├── kyverno.yaml
│ ├── prometheus.yaml
│ ├── grafana.yaml
│ ├── loki.yaml
│ ├── fluent-bit.yaml
│ ├── trivy.yaml
│ ├── sealedsecrets.yaml
│ └── values/ # Helm value overrides
│ ├── base/ # Shared cloud-agnostic values
│ ├── upc-dev/ # UpCloud Dev (storage, LB, pricing)
│ ├── upc-prod/ # UpCloud Prod
│ ├── eks-dev/ # AWS EKS Dev
│ ├── eks-prod/ # AWS EKS Prod
│ ├── aks-dev/ # Azure AKS Dev
│ ├── aks-prod/ # Azure AKS Prod
│ ├── gke-dev/ # GCP GKE Dev
│ └── gke-prod/ # GCP GKE Prod
│
├── apps/ # Business Applications
│ ├── mcp10x.yaml
@@ -156,14 +136,14 @@ This repository contains the complete GitOps configuration for our Kubernetes cl
| Repository | Purpose | Who Edits | How Often |
|------------|---------|-----------|-----------|
| **[launchpad](https://git.forteapps.net/Forte/launchpad)** (this repo) | ArgoCD Applications, cluster resources | Platform / DevOps engineers | ✅ Often |
| **[forte-helm](https://git.forteapps.net/Forte/forte-helm)** | Generic Helm chart templates | Platform engineers | ❌ Rarely |
| **[helm-prod-values](ssh://git@git.forteapps.net:2222/Forte/helm-prod-values.git)** | App-specific configuration & versions | Developers / CI pipelines | ✅ Sometimes |
| **[sturdy-adventure](https://github.com/fortedigital/sturdy-adventure.git)** (this repo) | ArgoCD Applications, cluster resources | Platform / DevOps engineers | ✅ Often |
| **[forte-helm](https://github.com/snothub/forte-helm)** | Generic Helm chart templates | Platform engineers | ❌ Rarely |
| **[helm-values](git@github.com:fortedigital/helm-values.git)** | App-specific configuration & versions | Developers / CI pipelines | ✅ Sometimes |
### GitOps Workflow
```
Developer commits code → CI/CD builds image → Updates helm-prod-values → ArgoCD syncs → Deployed to cluster
Developer commits code → CI/CD builds image → Updates helm-values → ArgoCD syncs → Deployed to cluster
```
**Learn more**: [GitOps Architecture - GitOps Workflow](docs/GITOPS-ARCHITECTURE.md#gitops-workflow)
@@ -178,7 +158,7 @@ Developer commits code → CI/CD builds image → Updates helm-prod-values → A
**Quick version**:
1. Create `apps/myapp.yaml` (ArgoCD Application manifest)
2. Create `helm-prod-values/myapp/values.yaml` (configuration)
2. Create `helm-values/myapp/values.yaml` (configuration)
3. Create sealed secrets if needed
4. Commit and push - ArgoCD auto-syncs!
@@ -187,8 +167,8 @@ Developer commits code → CI/CD builds image → Updates helm-prod-values → A
**See detailed guide**: [Developer Guide - Updating an Existing Application](docs/DEVELOPER-GUIDE.md#updating-an-existing-application)
**Quick version**:
- **Update code**: Push to app repo → CI/CD updates image tag in helm-prod-values
- **Update config**: Edit `helm-prod-values/myapp/values.yaml` → commit → push
- **Update code**: Push to app repo → CI/CD updates image tag in helm-values
- **Update config**: Edit `helm-values/myapp/values.yaml` → commit → push
### Manage Secrets
@@ -216,7 +196,7 @@ git push
**Quick version**:
```yaml
# In helm-prod-values/myapp/values.yaml
# In helm-values/myapp/values.yaml
# Token-based auth (simple)
auth:
@@ -351,10 +331,7 @@ kubectl patch application myapp -n argocd \
| **Prometheus** | Metrics | `monitoring` | 1 |
| **Grafana** | Dashboards | `monitoring` | 1 |
| **Loki** | Logs | `monitoring` | 1 |
| **Tempo** | Distributed tracing | `monitoring` | 1 |
| **Fluent-Bit** | Log shipping | `monitoring` | DaemonSet |
| **OpenCost** | Cost monitoring | `monitoring` | 1 |
| **Renovate** | Dependency updates | `renovate` | CronJob |
| **Trivy** | Vulnerability scanning | `trivy-system` | 1 |
**Full specs**: [Technical Reference - Infrastructure Components](docs/REFERENCE.md#infrastructure-components)
@@ -373,12 +350,12 @@ kubectl patch application myapp -n argocd \
## 📖 Key Concepts
### App-of-Apps Pattern
`_app-of-apps-{cluster}.yaml` is the root Application that manages all other Applications in `infra/`. Kustomize overlays in `infra/overlays/{cluster}/` render the base Applications with per-cluster patches (e.g., swapping value file paths). Supported clusters: `upc-dev`, `upc-prod`, `eks-dev`, `eks-prod`, `aks-dev`, `aks-prod`, `gke-dev`, `gke-prod`.
`_app-of-apps.yaml` is the root Application that manages all other Applications in `infra/`. Each YAML in `infra/` becomes a child Application managed by ArgoCD.
### Multi-Source Pattern
Applications reference both:
1. **Helm charts** from `forte-helm` (templates)
2. **Values** from `helm-prod-values` (configuration)
2. **Values** from `helm-values` (configuration)
This separates reusable templates from environment-specific config.
@@ -447,7 +424,7 @@ Applications deploy in order using `argocd.argoproj.io/sync-wave`:
### Adding a New Application
1. Read [Developer Guide - Deploying Your First Application](docs/DEVELOPER-GUIDE.md#deploying-your-first-application)
2. Create ArgoCD Application manifest in `apps/`
3. Create Helm values in `helm-prod-values/`
3. Create Helm values in `helm-values/`
4. Create sealed secrets if needed
5. Commit and push - ArgoCD handles the rest!
@@ -470,14 +447,16 @@ Documentation lives in `docs/`. To update:
## 📝 Notes
### Current Environment
- **Provider**: Multi-cloud (UpCloud, AWS EKS, Azure AKS, GCP GKE)
- **Active clusters**: UpCloud (upc-dev, upc-prod)
- **Provider**: UpCloud Managed Kubernetes
- **Environment**: Production (internal use only)
- **Cluster**: Single cluster
- **Auth**: Disabled for ArgoCD (internal access)
- **Backup**: Gitea daily backup to S3-compatible storage
- **Backup**: None (cluster rebuildable via GitOps)
### Known Limitations
- No automated backups (yet)
- Secret rotation not automated
- Single cluster (no multi-cluster setup)
- DNS management is manual
**Future improvements**: See [Operations Runbook - Disaster Recovery](docs/OPERATIONS-RUNBOOK.md#disaster-recovery)
@@ -491,12 +470,11 @@ Documentation lives in `docs/`. To update:
- [Kyverno Documentation](https://kyverno.io/docs/)
- [Traefik Documentation](https://doc.traefik.io/traefik/)
- [Cert-Manager Documentation](https://cert-manager.io/docs/)
- [Grafana Tempo Documentation](https://grafana.com/docs/tempo/)
- [Sealed Secrets](https://github.com/bitnami-labs/sealed-secrets)
### Related Repositories
- [forte-helm](https://git.forteapps.net/Forte/forte-helm) - Helm chart templates
- [helm-prod-values](git@github.com:fortedigital/helm-prod-values.git) - Application values
- [forte-helm](https://github.com/snothub/forte-helm) - Helm chart templates
- [helm-values](git@github.com:fortedigital/helm-values.git) - Application values
---
@@ -514,7 +492,7 @@ Internal use only. Not for public distribution.
---
**Last Updated**: 2026-04-22
**Last Updated**: 2026-03-16
**Documentation Version**: 1.0.0
**🚀 Ready to get started? Check out the [Documentation Index](docs/README.md)!**

32
_app-of-apps-aks-dev.yaml
View File

@@ -1,32 +0,0 @@
apiVersion: v1
kind: Namespace
metadata:
name: monitoring
annotations:
argocd.argoproj.io/sync-wave: "-1"
---
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: infrastructure-apps
namespace: argocd
labels:
app.kubernetes.io/name: infrastructure-apps
app.kubernetes.io/part-of: platform
finalizers:
- resources-finalizer.argocd.argoproj.io
spec:
project: default
source:
repoURL: ssh://git@git.forteapps.net:2222/Forte/launchpad.git
targetRevision: HEAD
path: infra/overlays/aks-dev
destination:
server: https://kubernetes.default.svc
namespace: default
syncPolicy:
automated:
prune: true
selfHeal: true
syncOptions:
- CreateNamespace=true

32
_app-of-apps-aks-prod.yaml
View File

@@ -1,32 +0,0 @@
apiVersion: v1
kind: Namespace
metadata:
name: monitoring
annotations:
argocd.argoproj.io/sync-wave: "-1"
---
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: infrastructure-apps
namespace: argocd
labels:
app.kubernetes.io/name: infrastructure-apps
app.kubernetes.io/part-of: platform
finalizers:
- resources-finalizer.argocd.argoproj.io
spec:
project: default
source:
repoURL: ssh://git@git.forteapps.net:2222/Forte/launchpad.git
targetRevision: HEAD
path: infra/overlays/aks-prod
destination:
server: https://kubernetes.default.svc
namespace: default
syncPolicy:
automated:
prune: true
selfHeal: true
syncOptions:
- CreateNamespace=true

32
_app-of-apps-eks-prod.yaml
View File

@@ -1,32 +0,0 @@
apiVersion: v1
kind: Namespace
metadata:
name: monitoring
annotations:
argocd.argoproj.io/sync-wave: "-1"
---
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: infrastructure-apps
namespace: argocd
labels:
app.kubernetes.io/name: infrastructure-apps
app.kubernetes.io/part-of: platform
finalizers:
- resources-finalizer.argocd.argoproj.io
spec:
project: default
source:
repoURL: ssh://git@git.forteapps.net:2222/Forte/launchpad.git
targetRevision: HEAD
path: infra/overlays/eks-prod
destination:
server: https://kubernetes.default.svc
namespace: default
syncPolicy:
automated:
prune: true
selfHeal: true
syncOptions:
- CreateNamespace=true

4
_app-of-apps-eks-dev.yaml → _app-of-apps-eu.yaml
View File

@@ -18,9 +18,9 @@ metadata:
spec:
project: default
source:
repoURL: ssh://git@git.forteapps.net:2222/Forte/launchpad.git
repoURL: git@github.com:fortedigital/sturdy-adventure.git
targetRevision: HEAD
path: infra/overlays/eks-dev
path: infra/overlays/eu
destination:
server: https://kubernetes.default.svc
namespace: default

32
_app-of-apps-gke-prod.yaml
View File

@@ -1,32 +0,0 @@
apiVersion: v1
kind: Namespace
metadata:
name: monitoring
annotations:
argocd.argoproj.io/sync-wave: "-1"
---
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: infrastructure-apps
namespace: argocd
labels:
app.kubernetes.io/name: infrastructure-apps
app.kubernetes.io/part-of: platform
finalizers:
- resources-finalizer.argocd.argoproj.io
spec:
project: default
source:
repoURL: ssh://git@git.forteapps.net:2222/Forte/launchpad.git
targetRevision: HEAD
path: infra/overlays/gke-prod
destination:
server: https://kubernetes.default.svc
namespace: default
syncPolicy:
automated:
prune: true
selfHeal: true
syncOptions:
- CreateNamespace=true

32
_app-of-apps-upc-dev.yaml
View File

@@ -1,32 +0,0 @@
apiVersion: v1
kind: Namespace
metadata:
name: monitoring
annotations:
argocd.argoproj.io/sync-wave: "-1"
---
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: infrastructure-apps
namespace: argocd
labels:
app.kubernetes.io/name: infrastructure-apps
app.kubernetes.io/part-of: platform
finalizers:
- resources-finalizer.argocd.argoproj.io
spec:
project: default
source:
repoURL: ssh://git@git.forteapps.net:2222/Forte/launchpad.git
targetRevision: HEAD
path: infra/overlays/upc-dev
destination:
server: https://kubernetes.default.svc
namespace: default
syncPolicy:
automated:
prune: true
selfHeal: true
syncOptions:
- CreateNamespace=true

32
_app-of-apps-upc-prod.yaml
View File

@@ -1,32 +0,0 @@
apiVersion: v1
kind: Namespace
metadata:
name: monitoring
annotations:
argocd.argoproj.io/sync-wave: "-1"
---
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: infrastructure-apps
namespace: argocd
labels:
app.kubernetes.io/name: infrastructure-apps
app.kubernetes.io/part-of: platform
finalizers:
- resources-finalizer.argocd.argoproj.io
spec:
project: default
source:
repoURL: ssh://git@git.forteapps.net:2222/Forte/launchpad.git
targetRevision: HEAD
path: infra/overlays/upc-prod
destination:
server: https://kubernetes.default.svc
namespace: default
syncPolicy:
automated:
prune: true
selfHeal: true
syncOptions:
- CreateNamespace=true

4
_app-of-apps-gke-dev.yaml → _app-of-apps-us.yaml
View File

@@ -18,9 +18,9 @@ metadata:
spec:
project: default
source:
repoURL: ssh://git@git.forteapps.net:2222/Forte/launchpad.git
repoURL: git@github.com:fortedigital/sturdy-adventure.git
targetRevision: HEAD
path: infra/overlays/gke-dev
path: infra/overlays/us
destination:
server: https://kubernetes.default.svc
namespace: default

4
apps/base/argo-mcp.yaml
View File

@@ -16,14 +16,14 @@ metadata:
spec:
project: default
sources:
- repoURL: ssh://git@git.forteapps.net:2222/Forte/forte-helm.git
- repoURL: https://github.com/snothub/forte-helm
path: forteapp
targetRevision: HEAD
helm:
valueFiles:
- $values/argocd-mcp/values.yaml
- repoURL: ssh://git@git.forteapps.net:2222/Forte/helm-prod-values.git
- repoURL: git@github.com:fortedigital/helm-values.git
targetRevision: HEAD
ref: values

4
apps/base/dot-ai-stack.yaml
View File

@@ -35,9 +35,9 @@ spec:
releaseName: dot-ai-stack
valueFiles:
- $values/infra/values/base/dot-ai-stack-values.yaml
- $values/infra/values/upc-dev/dot-ai-stack-values.yaml
- $values/infra/values/eu/dot-ai-stack-values.yaml
- repoURL: ssh://git@git.forteapps.net:2222/Forte/launchpad.git
- repoURL: git@github.com:fortedigital/sturdy-adventure.git
targetRevision: HEAD
ref: values

2
apps/base/kustomization.yaml
View File

@@ -4,5 +4,5 @@ resources:
- dot-ai-stack.yaml
- mcp10x.yaml
- musicman.yaml
- ts-mcp.yaml
- mcpcoder.yaml
- argo-mcp.yaml

4
apps/base/mcp10x.yaml
View File

@@ -17,14 +17,14 @@ metadata:
spec:
project: default
sources:
- repoURL: ssh://git@git.forteapps.net:2222/Forte/forte-helm.git
- repoURL: https://github.com/snothub/forte-helm
path: forteapp
targetRevision: HEAD
helm:
valueFiles:
- $values/mcp10x/values.yaml
- repoURL: ssh://git@git.forteapps.net:2222/Forte/helm-prod-values.git
- repoURL: git@github.com:fortedigital/helm-values.git
targetRevision: HEAD
ref: values

15
apps/base/ts-mcp.yaml → apps/base/mcpcoder.yaml
View File

@@ -1,15 +1,15 @@
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: ts-mcp
name: mcpcoder
namespace: argocd
annotations:
argocd.argoproj.io/sync-wave: "11"
argocd.argoproj.io/sync-wave: "1"
notifications.argoproj.io/subscribe.on-sync-succeeded.slack: ""
notifications.argoproj.io/subscribe.on-sync-failed.slack: ""
notifications.argoproj.io/subscribe.on-degraded.slack: ""
labels:
app.kubernetes.io/name: ts-mcp
app.kubernetes.io/name: mcpcoder
app.kubernetes.io/part-of: apps
app.kubernetes.io/managed-by: argocd
finalizers:
@@ -17,21 +17,20 @@ metadata:
spec:
project: default
sources:
- repoURL: ssh://git@git.forteapps.net:2222/Forte/forte-helm.git
- repoURL: https://github.com/snothub/forte-helm
path: forteapp
targetRevision: HEAD
helm:
valueFiles:
- $values/ts-mcp/values.yaml
- $values/mcpcoder/values.yaml
- repoURL: ssh://git@git.forteapps.net:2222/Forte/helm-prod-values.git
- repoURL: git@github.com:fortedigital/helm-values.git
targetRevision: HEAD
ref: values
destination:
server: https://kubernetes.default.svc
namespace: ts-mcp
namespace: mcpcoder
syncPolicy:
automated:
prune: true

4
apps/base/musicman.yaml
View File

@@ -17,14 +17,14 @@ metadata:
spec:
project: default
sources:
- repoURL: ssh://git@git.forteapps.net:2222/Forte/forte-helm.git
- repoURL: https://github.com/snothub/forte-helm
path: forteapp
targetRevision: HEAD
helm:
valueFiles:
- $values/musicman/values.yaml
- repoURL: ssh://git@git.forteapps.net:2222/Forte/helm-prod-values.git
- repoURL: git@github.com:fortedigital/helm-values.git
targetRevision: HEAD
ref: values

3
secrets/overlays/eks-dev/kustomization.yaml → apps/overlays/eu/kustomization.yaml
View File

@@ -2,3 +2,6 @@ apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- ../../base
# No patches needed — base already has "eu" paths
# EU is the default/base cluster

7
apps/overlays/upc-dev/kustomization.yaml
View File

@@ -1,7 +0,0 @@
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- ../../base
# No patches needed — base already has "upc-dev" paths
# upc-dev is the default/base cluster

4
apps/overlays/upc-prod/kustomization.yaml → apps/overlays/us/kustomization.yaml
View File

@@ -4,11 +4,11 @@ resources:
- ../../base
patches:
# dot-ai-stack: swap upc-dev → upc-prod
# dot-ai-stack: swap eu → us
- target:
kind: Application
name: dot-ai-stack
patch: |
- op: replace
path: /spec/sources/0/helm/valueFiles/1
value: $values/infra/values/upc-prod/dot-ai-stack-values.yaml
value: $values/infra/values/us/dot-ai-stack-values.yaml

12
bootstrap.sh
View File

@@ -2,7 +2,7 @@
# in case of $'\r': command not found error, run command below first
# sed -i 's/\r$//' ./bootstrap.sh
CLUSTER="${1:?Usage: ./bootstrap.sh <cluster> (upc-dev|upc-prod|aks-dev|aks-prod|eks-dev|eks-prod|gke-dev|gke-prod)}"
CLUSTER="${1:?Usage: ./bootstrap.sh <cluster> (eu|us)}"
echo "running $0 for cluster: ${CLUSTER}..."
@@ -17,18 +17,18 @@ echo "Bootstrapping cluster: ${clusterName} (${CLUSTER})..."
Bootstrap()
{
ArgoCd
# Gitea
Github
}
############################################################
# Gitea #
# Github #
############################################################
Gitea()
Github()
{
echo "Installing secret..."
kubectl apply -f private/gitea-repo-main.yaml
kubectl apply -f private/main.key
kubectl apply -f private/github-${CLUSTER}.yaml
kubectl apply -f private/main-${CLUSTER}.key
}
############################################################

88
cluster-resources/gitea-backup-cronjob.yaml
View File

@@ -1,88 +0,0 @@
apiVersion: batch/v1
kind: CronJob
metadata:
name: gitea-backup
namespace: gitea
spec:
schedule: "0 3 * * *" # daily at 03:00 UTC
concurrencyPolicy: Forbid
successfulJobsHistoryLimit: 3
failedJobsHistoryLimit: 3
jobTemplate:
spec:
backoffLimit: 1
activeDeadlineSeconds: 1800
template:
spec:
restartPolicy: Never
securityContext:
runAsUser: 1000
runAsGroup: 1000
fsGroup: 1000
# Must run on the same node as Gitea to share the RWO volume
affinity:
podAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
- labelSelector:
matchLabels:
app.kubernetes.io/name: gitea
topologyKey: kubernetes.io/hostname
initContainers:
- name: gitea-dump
image: gitea/gitea:1.25.4
command:
- sh
- -c
- |
gitea dump \
-c /data/gitea/conf/app.ini \
-f /backup/gitea-dump.zip \
-t /tmp/gitea-dump && \
echo "Dump completed: $(ls -lh /backup/gitea-dump.zip)"
volumeMounts:
- name: data
mountPath: /data
readOnly: true
- name: backup
mountPath: /backup
- name: tmp
mountPath: /tmp/gitea-dump
containers:
- name: upload
image: minio/mc:latest
env:
- name: HOME
value: /tmp
command:
- sh
- -c
- |
mc alias set s3 "${S3_ENDPOINT}" "${AWS_ACCESS_KEY_ID}" "${AWS_SECRET_ACCESS_KEY}"
TIMESTAMP=$(date +%Y%m%d-%H%M%S)
KEY="gitea-dump-${TIMESTAMP}.zip"
echo "Uploading ${KEY}..."
mc cp /backup/gitea-dump.zip "s3/${S3_BUCKET}/${KEY}" && \
echo "Upload complete."
# Prune backups older than 7 days
echo "Pruning backups older than 7 days..."
mc rm --older-than 7d --force "s3/${S3_BUCKET}/" 2>&1 || true
echo "Pruning complete."
envFrom:
- secretRef:
name: gitea-backup-s3
volumeMounts:
- name: backup
mountPath: /backup
readOnly: true
volumes:
- name: data
persistentVolumeClaim:
claimName: gitea-shared-storage
- name: backup
emptyDir:
sizeLimit: 5Gi
- name: tmp
emptyDir:
sizeLimit: 5Gi

13
cluster-resources/gitea-ssh-ingressroute.yaml
View File

@@ -1,13 +0,0 @@
apiVersion: traefik.io/v1alpha1
kind: IngressRouteTCP
metadata:
name: gitea-ssh
namespace: gitea
spec:
entryPoints:
- giteassh
routes:
- match: HostSNI(`*`)
services:
- name: gitea-ssh
port: 22

37
cluster-resources/network/deny-external-egress-trivy.yaml
View File

@@ -1,37 +0,0 @@
apiVersion: cilium.io/v2
kind: CiliumNetworkPolicy
metadata:
name: deny-external-egress
namespace: trivy-system
labels:
app.kubernetes.io/managed-by: argocd
app.kubernetes.io/part-of: network-policies
spec:
endpointSelector: {}
egress:
# Allow DNS resolution
- toEndpoints:
- matchLabels:
io.kubernetes.pod.namespace: kube-system
k8s-app: kube-dns
toPorts:
- ports:
- port: "53"
protocol: UDP
- port: "53"
protocol: TCP
# Allow cluster-internal traffic (RFC1918)
- toCIDR:
- 10.0.0.0/8
- 172.16.0.0/12
- 192.168.0.0/16
# Allow Trivy vulnerability DB downloads (ghcr.io OCI registry)
- toFQDNs:
- matchName: ghcr.io
- matchName: pkg-containers.githubusercontent.com
toPorts:
- ports:
- port: "443"
protocol: TCP

152
cluster-resources/policies/auth-sidecar-injector.yaml
View File

@@ -10,7 +10,7 @@ metadata:
policies.kyverno.io/severity: medium
policies.kyverno.io/subject: Pod
policies.kyverno.io/description: >-
Injects an auth sidecar container into Pods annotated with policies.forteapps.io/auth: "true". Supports three auth modes controlled by the policies.forteapps.io/auth-type annotation: "token" (default), "oidc", and "mcp". In token mode the sidecar reads credentials from a mounted Secret volume. In OIDC mode the sidecar uses OpenID Connect with authority and client-id provided via required annotations (policies.forteapps.io/auth-oidc-authority and policies.forteapps.io/auth-oidc-client-id) and secrets from an auth-oidc Secret. In MCP mode the sidecar implements OAuth 2.0 for MCP servers per RFC 9728 (Protected Resource Metadata) and RFC 7591 (Dynamic Client Registration), configured via policies.forteapps.io/auth-mcp-resource and policies.forteapps.io/auth-mcp-authority annotations. The sidecar port defaults to 9001 and can be overridden via the policies.forteapps.io/auth-port annotation. A NetworkPolicy is generated to restrict ingress to the sidecar port only.
Injects an auth sidecar container into Pods annotated with policies.forteapps.io/auth: "true". Supports three auth modes controlled by the policies.forteapps.io/auth-type annotation: "token" (default), "oidc", and "mcp". In token mode the sidecar reads credentials from a mounted Secret volume. In OIDC mode the sidecar uses OpenID Connect with authority and client-id provided via required annotations (policies.forteapps.io/auth-oidc-authority and policies.forteapps.io/auth-oidc-client-id) and secrets from an auth-oidc Secret. In MCP mode the sidecar implements OAuth 2.0 for MCP servers per RFC 9728 (Protected Resource Metadata) and RFC 7591 (Dynamic Client Registration), configured via policies.forteapps.io/auth-mcp-resource and policies.forteapps.io/auth-mcp-authority annotations. A NetworkPolicy is generated to restrict ingress to the sidecar port only.
spec:
background: false
rules:
@@ -119,26 +119,21 @@ spec:
- name: appPort
variable:
jmesPath: request.object.spec.containers[?name != 'authn'] | [0].ports[0].containerPort || `3000`
- name: sidecarPort
variable:
jmesPath: to_number(request.object.metadata.annotations."policies.forteapps.io/auth-port" || '9001')
mutate:
patchStrategicMerge:
spec:
containers:
- name: authn
image: "{{ request.object.metadata.annotations.\"policies.forteapps.io/auth-image\" || 'git.forteapps.net/forte/auth-sidecar' }}:{{ request.object.metadata.annotations.\"policies.forteapps.io/auth-image-version\" || 'latest' }}"
image: "{{ request.object.metadata.annotations.\"policies.forteapps.io/auth-image\" || 'ghcr.io/snothub/stunning-memory' }}:{{ request.object.metadata.annotations.\"policies.forteapps.io/auth-image-version\" || 'latest' }}"
ports:
- containerPort: "{{ sidecarPort }}"
- containerPort: 8080
name: auth
protocol: TCP
env:
- name: AUTH_LISTEN_ADDR
value: ":{{ sidecarPort }}"
value: ":8080"
- name: AUTH_UPSTREAM_URL
value: "{{ request.object.metadata.annotations.\"policies.forteapps.io/auth-upstream-url\" || join('', ['http://localhost:', to_string(appPort)]) }}"
- name: AUTH_PUBLIC_PATHS
value: "{{ request.object.metadata.annotations.\"policies.forteapps.io/auth-public-paths\" || '/healthz' }}"
- name: AUTH_TOKEN_FILE
value: "/etc/auth/tokens"
- name: AUTH_MODE
@@ -157,13 +152,13 @@ spec:
readinessProbe:
httpGet:
path: /healthz
port: "{{ sidecarPort }}"
port: 8080
initialDelaySeconds: 2
periodSeconds: 5
livenessProbe:
httpGet:
path: /healthz
port: "{{ sidecarPort }}"
port: 8080
initialDelaySeconds: 5
periodSeconds: 10
securityContext:
@@ -200,25 +195,22 @@ spec:
- name: appPort
variable:
jmesPath: request.object.spec.containers[?name != 'authn'] | [0].ports[0].containerPort || `3000`
- name: sidecarPort
variable:
jmesPath: to_number(request.object.metadata.annotations."policies.forteapps.io/auth-port" || '9001')
mutate:
patchStrategicMerge:
spec:
containers:
- name: authn
image: "{{ request.object.metadata.annotations.\"policies.forteapps.io/auth-image\" || 'git.forteapps.net/forte/auth-sidecar' }}:{{ request.object.metadata.annotations.\"policies.forteapps.io/auth-image-version\" || 'latest' }}"
image: "{{ request.object.metadata.annotations.\"policies.forteapps.io/auth-image\" || 'ghcr.io/snothub/stunning-memory' }}:{{ request.object.metadata.annotations.\"policies.forteapps.io/auth-image-version\" || 'latest' }}"
imagePullPolicy: Always
ports:
- containerPort: "{{ sidecarPort }}"
- containerPort: 8080
name: auth
protocol: TCP
env:
- name: AUTH_MODE
value: "oidc"
- name: AUTH_LISTEN_ADDR
value: ":{{ sidecarPort }}"
value: ":8080"
- name: AUTH_LOG_LEVEL
value: "{{ request.object.metadata.annotations.\"policies.forteapps.io/auth-log-level\" || 'info' }}"
- name: AUTH_UPSTREAM_URL
@@ -233,8 +225,6 @@ spec:
value: "{{ regex_replace_all('https?://[^/]*', request.object.metadata.annotations.\"policies.forteapps.io/auth-oidc-callback-path\", '') }}"
- name: AUTH_OIDC_SCOPES
value: "{{ request.object.metadata.annotations.\"policies.forteapps.io/auth-oidc-scopes\" || 'openid,profile,email' }}"
- name: AUTH_PUBLIC_PATHS
value: "{{ request.object.metadata.annotations.\"policies.forteapps.io/auth-public-paths\" || '/healthz' }}"
- name: AUTH_OIDC_COOKIE_SECRET
valueFrom:
secretKeyRef:
@@ -243,8 +233,8 @@ spec:
- name: AUTH_OIDC_CLIENT_SECRET
valueFrom:
secretKeyRef:
name: "{{ request.object.metadata.annotations.\"policies.forteapps.io/auth-oidc-credentials-secret\" || 'auth-oidc' }}"
key: "{{ request.object.metadata.annotations.\"policies.forteapps.io/auth-oidc-credentials-secret-key\" || 'client-secret' }}"
name: auth-oidc
key: client-secret
resources:
limits:
cpu: 50m
@@ -255,13 +245,13 @@ spec:
readinessProbe:
httpGet:
path: /healthz
port: "{{ sidecarPort }}"
port: 8080
initialDelaySeconds: 2
periodSeconds: 5
livenessProbe:
httpGet:
path: /healthz
port: "{{ sidecarPort }}"
port: 8080
initialDelaySeconds: 5
periodSeconds: 10
securityContext:
@@ -293,25 +283,22 @@ spec:
- name: appPort
variable:
jmesPath: request.object.spec.containers[?name != 'authn'] | [0].ports[0].containerPort || `3000`
- name: sidecarPort
variable:
jmesPath: to_number(request.object.metadata.annotations."policies.forteapps.io/auth-port" || '9001')
mutate:
patchStrategicMerge:
spec:
containers:
- name: authn
image: "{{ request.object.metadata.annotations.\"policies.forteapps.io/auth-image\" || 'git.forteapps.net/forte/auth-sidecar' }}:{{ request.object.metadata.annotations.\"policies.forteapps.io/auth-image-version\" || 'latest' }}"
image: "{{ request.object.metadata.annotations.\"policies.forteapps.io/auth-image\" || 'ghcr.io/snothub/stunning-memory' }}:{{ request.object.metadata.annotations.\"policies.forteapps.io/auth-image-version\" || 'latest' }}"
imagePullPolicy: Always
ports:
- containerPort: "{{ sidecarPort }}"
- containerPort: 8080
name: auth
protocol: TCP
env:
- name: AUTH_MODE
value: "mcp"
- name: AUTH_LISTEN_ADDR
value: ":{{ sidecarPort }}"
value: ":8080"
- name: AUTH_LOG_LEVEL
value: "{{ request.object.metadata.annotations.\"policies.forteapps.io/auth-log-level\" || 'info' }}"
- name: AUTH_UPSTREAM_URL
@@ -320,10 +307,8 @@ spec:
value: "{{ request.object.metadata.annotations.\"policies.forteapps.io/auth-mcp-resource\" }}"
- name: AUTH_MCP_AUTHORIZATION_SERVERS
value: "{{ request.object.metadata.annotations.\"policies.forteapps.io/auth-mcp-authority\" }}"
- name: AUTH_PUBLIC_PATHS
value: "{{ request.object.metadata.annotations.\"policies.forteapps.io/auth-public-paths\" || '/healthz' }}"
- name: AUTH_MCP_SCOPES_SUPPORTED
value: "{{ request.object.metadata.annotations.\"policies.forteapps.io/auth-mcp-scopes\" || 'profile' }}"
value: "{{ request.object.metadata.annotations.\"policies.forteapps.io/auth-mcp-scopes\" || 'read,write' }}"
resources:
limits:
cpu: 50m
@@ -334,106 +319,13 @@ spec:
readinessProbe:
httpGet:
path: /healthz
port: "{{ sidecarPort }}"
port: 8080
initialDelaySeconds: 2
periodSeconds: 5
livenessProbe:
httpGet:
path: /healthz
port: "{{ sidecarPort }}"
initialDelaySeconds: 5
periodSeconds: 10
securityContext:
allowPrivilegeEscalation: false
readOnlyRootFilesystem: true
capabilities:
drop:
- ALL
- name: inject-sidecar-oauth
skipBackgroundRequests: true
match:
any:
- resources:
kinds:
- Pod
annotations:
policies.forteapps.io/auth: "true"
policies.forteapps.io/auth-type: "oauth"
exclude:
any:
- resources:
namespaces:
- kube-system
- kyverno
- argocd
- cert-manager
- monitoring
context:
- name: appPort
variable:
jmesPath: request.object.spec.containers[?name != 'authn'] | [0].ports[0].containerPort || `3000`
- name: sidecarPort
variable:
jmesPath: to_number(request.object.metadata.annotations."policies.forteapps.io/auth-port" || '9001')
mutate:
patchStrategicMerge:
spec:
containers:
- name: authn
image: "{{ request.object.metadata.annotations.\"policies.forteapps.io/auth-image\" || 'git.forteapps.net/forte/auth-sidecar' }}:{{ request.object.metadata.annotations.\"policies.forteapps.io/auth-image-version\" || 'latest' }}"
imagePullPolicy: Always
ports:
- containerPort: "{{ sidecarPort }}"
name: auth
protocol: TCP
env:
- name: AUTH_MODE
value: "oauth"
- name: AUTH_LISTEN_ADDR
value: ":{{ sidecarPort }}"
- name: AUTH_LOG_LEVEL
value: "{{ request.object.metadata.annotations.\"policies.forteapps.io/auth-log-level\" || 'info' }}"
- name: AUTH_UPSTREAM_URL
value: "{{ request.object.metadata.annotations.\"policies.forteapps.io/auth-upstream-url\" || join('', ['http://localhost:', to_string(appPort)]) }}"
- name: AUTH_OAUTH_AUTHORITY
value: "{{ request.object.metadata.annotations.\"policies.forteapps.io/auth-oauth-authority\" }}"
- name: AUTH_OAUTH_CLIENT_ID
value: "{{ request.object.metadata.annotations.\"policies.forteapps.io/auth-oauth-client-id\" }}"
- name: AUTH_OAUTH_SCOPES
value: "{{ request.object.metadata.annotations.\"policies.forteapps.io/auth-oauth-scopes\" || 'openid,profile,email' }}"
- name: AUTH_OAUTH_DELEGATION_ENABLED
value: "{{ request.object.metadata.annotations.\"policies.forteapps.io/auth-oauth-delegation-enabled\" || 'false' }}"
- name: AUTH_OAUTH_DELEGATION_CLIENT_ID
value: "{{ request.object.metadata.annotations.\"policies.forteapps.io/auth-oauth-delegation-client-id\" || '' }}"
- name: AUTH_OAUTH_DELEGATION_SCOPES
value: "{{ request.object.metadata.annotations.\"policies.forteapps.io/auth-oauth-delegation-scopes\" || '' }}"
- name: AUTH_OAUTH_CLIENT_SECRET
valueFrom:
secretKeyRef:
name: "{{ request.object.metadata.annotations.\"policies.forteapps.io/auth-oauth-credentials-secret\" || 'auth-oauth' }}"
key: "{{ request.object.metadata.annotations.\"policies.forteapps.io/auth-oauth-credentials-secret-key\" || 'client-secret' }}"
- name: AUTH_OAUTH_DELEGATION_CLIENT_SECRET
valueFrom:
secretKeyRef:
name: auth-oauth
key: delegation-client-secret
resources:
limits:
cpu: 50m
memory: 64Mi
requests:
cpu: 10m
memory: 32Mi
readinessProbe:
httpGet:
path: /healthz
port: "{{ sidecarPort }}"
initialDelaySeconds: 2
periodSeconds: 5
livenessProbe:
httpGet:
path: /healthz
port: "{{ sidecarPort }}"
port: 8080
initialDelaySeconds: 5
periodSeconds: 10
securityContext:
@@ -466,10 +358,6 @@ spec:
operator: In
value:
- CREATE
context:
- name: sidecarPort
variable:
jmesPath: to_number(request.object.metadata.annotations."policies.forteapps.io/auth-port" || '9001')
generate:
synchronize: false
apiVersion: networking.k8s.io/v1
@@ -488,5 +376,5 @@ spec:
- Ingress
ingress:
- ports:
- port: "{{ sidecarPort }}"
- port: 8080
protocol: TCP

71
cluster-resources/policies/deployment-verifier.yaml Normal file
View File

@@ -0,0 +1,71 @@
apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
name: require-deployment-owner
spec:
validationFailureAction: Audit
background: false
rules:
- name: check-pod-owner-is-replicaset-from-deployment
skipBackgroundRequests: true
match:
any:
- resources:
kinds:
- Pod
exclude:
any:
- resources:
namespaces:
- kube-system
- kyverno
- cert-manager
- monitoring
- argocd
- traefik-system
context:
- name: ownerReplicaSet
apiCall:
method: GET
urlPath: "/apis/apps/v1/namespaces/{{request.namespace}}/replicasets/{{request.object.metadata.ownerReferences[0].name}}"
jmesPath: "@"
preconditions:
all:
- key: "{{ request.object.metadata.ownerReferences || `[]` | [?kind=='ReplicaSet'] | length(@) }}"
operator: GreaterThanOrEquals
value: 1
validate:
allowExistingViolations: true
message: "Pods must be created through a Deployment resource."
deny:
conditions:
any:
- key: "{{ownerReplicaSet.metadata.ownerReferences[0].kind}}"
operator: NotEquals
value: Deployment
- name: deny-pods-without-replicaset-owner
match:
any:
- resources:
kinds:
- Pod
exclude:
any:
- resources:
namespaces:
- kube-system
- kyverno
- cert-manager
- monitoring
- argocd
- traefik-system
skipBackgroundRequests: true
validate:
allowExistingViolations: true
message: "Direct pod creation is not allowed. Pods must come from a Deployment managed by ArgoCD."
deny:
conditions:
all:
- key: "{{ request.object.metadata.ownerReferences || `[]` | [?kind=='ReplicaSet'] | length(@) }}"
operator: LessThan
value: 1

37
cluster-resources/policies/keycloak-client-cloner.yaml
View File

@@ -1,37 +0,0 @@
apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
name: keycloak-client-config-cloner
spec:
rules:
- name: clone-client-config-to-keycloak
skipBackgroundRequests: false
match:
any:
- resources:
kinds:
- Secret
selector:
matchLabels:
keycloak.forteapps.net/client-config: "true"
exclude:
any:
- resources:
namespaces:
- keycloak
generate:
apiVersion: v1
kind: Secret
name: "{{request.object.metadata.name}}"
namespace: keycloak
synchronize: true
data:
metadata:
labels:
keycloak.forteapps.net/client-config: "true"
keycloak.forteapps.net/source-namespace: "{{request.object.metadata.namespace}}"
annotations:
keycloak.forteapps.net/source-name: "{{request.object.metadata.name}}"
keycloak.forteapps.net/source-namespace: "{{request.object.metadata.namespace}}"
data: "{{request.object.data}}"
type: "{{request.object.type}}"

1
cluster-resources/policies/label-checker.yaml
View File

@@ -26,7 +26,6 @@ spec:
- monitoring
- secrets
- kyverno
- trivy-system
match:
any:
- resources:

20
cluster-resources/snothub-repo-credentials-sealed.yaml Normal file
View File

@@ -0,0 +1,20 @@
---
apiVersion: bitnami.com/v1alpha1
kind: SealedSecret
metadata:
creationTimestamp: null
name: snothub-repo-creds
namespace: argocd
spec:
encryptedData:
sshPrivateKey: AgCBd+i8jXBDwvWM0YC9OWvNTKyynVpW6hF0F3aH0GXBIYxzFo1g9kMajE/Ce3bTl8DiVs7VGzPdI6lmJtSC3+fVMm4wavDGrUUbhUHSR/HnEqq51NVjxU1Uj+VRz550z9r6gB0VLAcYqN0WLKd/4Qn7tvQOmKsaXMd5jsIYpSB8nZK87awZ5niFmT8DIKu2DOzoDBeN7Yqrf1aELq3m3kaDxGcDMYSglB0xRU3fF8FYr5inCic1eTrCUdfIwnrBSublNxI9FrGnlylRC40XCPxNpy6zyoQt7yeJTNgRSvtfceqXAwh25mcvga+BfDBTLPQ0EDbCoNDnE9EyMUa4kWcXBTFZ1Qx5NGRz8HAjWoYDbuNOonl9ik5dvv9UKKA+/04ZEuPjsbdkBq9xKuowx0zLL1IVv/jeuViFdvNY6LKNu/hLwMn8aMlTLlIkB+1fsIqV7Vkva9Vk7IXNz575lMwIGUTo1dsK9FQ5+uIe2bsRnY8RJ3lpyndZQ1HDPh7P7KZLZPH8fUuAHH1UF74njMQyve79zCRcisBAewpXdq4UsYAYUQOluS1Ak+sFcIdQ0jHjfklGCcJnTvMyO7obIsPQSv39/bsCqQX6uisrTzcb0s7wnbzhcxf1gm7IyZMIhi2Vub3GoLCIMnb6ViO8k+itLUa5eZpoEeg+BpP1mgL7O2nVfrZYgueULgMSvN89ct/THITsFAR8614An3DCHSnZLv5ZmY7yC6rmO405IlrnjvfFqEt9MdqTgR7uTYDSdI7UjIFJp3rchzikF1pSDMu/siKmD/Vi9+S0KqBcSENz7EUppVuYIk0aRBqHZ9Awoe0qPIjp1AMg947FjIYXzkGk3Kz0P85fGwkktyfHNZdIrXvpDAIP2739Mr7Hde+EHpuajrhuBgozfless5PqLFfGWFnt5COW2HrdSrvrY56MXuJUfiV0nq3eEhCju9f/cA41VbxfO0Hj+KMCjxiL+MCgIt1eWD24P7GQHb0D7+JiuwqgcH0ZqXRaKFYSOJ3/U1o9RRX9v5yebNTm8ErQTSnIN5bNRgE+t8CrTulGVcpaL/rW+XW8cO8MXLC/R4eeNwVjoiK4yXSHpSjf5dF9MZ6bS6SSZPMOa2besEzzQIj0c/EkUsp/GmIF1utDbJVI2VZLFGklwRo92OJs3URXNCqbzzgte43Y9dJp2VdWyi3Zx32pXVWaNdOzeifVOASP7ags7Kbdfvoaa5CmTu17iRUHCOv6164/KFeDCfY=
type: AgCIKe+SzLHIp+6gjOVbD7wcpZkeg5UwgXabFjtonavkPbzX+txHWN7IH2HbDmd7TdpgGFiqMGFSQC8mOVXnj+Pw1XnI8trH6wavvKjQ1SsRWteB8o7lGe2PcG2h2v5yW2vk5rrmuB0ehogeJez3ynlk508HguRtidzxnKdZyc6SHOh2hbWq1clspJicREsHlz3Rfn7upOSUyFmx+Tilfnjuom5FFYGBNZt0aEjaW5S7fjcYJBTEerjGjTz3vUs2DK3C5ymyCKKasA628fVZ20uIZhmn5NUGGK6bKDusFOQhYjRnxRYwS2fToBHiVfC8wo7bWW8ZwOrbguvedJU2q9pwjKvXy9upw8Ra3EcYzXASqwI6rqUQv6htGqzTYtVkRsdVxaQqITc3FYrhKIroru4Iup4xYtcVojppz0+HQxiv5WtxosVtfXXX4Zj4spr9ThoZgzLe3ebILEIK3wJNmK/AxV80NXq7JOI6gaiLK/fbm7gd9G7oW9VuM50OJELDmR4jU2k1KSx6sD5lB+c+Ajp1iw+iS9ETD/je93+eyBRKGM9fqP0+DpNsJBBlVLuWK/BwidBN186pFi4DlMzo13Wd0zYKQce4NVrf5s4f7T01KeBU+nnvvcuI+rW7tsY3R18zgP9D9uX2E5066qJQdrFov2YuXwovS4hCQDttIfz9YoGUKKQ3PObWUVynhtBptoGbhRw=
url: AgCjVnAWNarZplxbAurz++eBiKXIuYUcoleDcsrMdwdflBOOf6ayguCr7CDHzUbuKFnlNKoFvIvjUOqFb1v5Yy4iWju3ajvlo+ncEIezxetYOmQSft3nSeOD3+RCZn4Qf4K2C1D113IyPCo7T2h01KuVLWOItfuWuVwFDpLTvmKfTRAs8eP1L4ohvAcY0/J5mAmi7tXV2hN1r9R2MEuvHOusujtrRqewXHhzBksSb9/wxilruU/BHQAhYKeKHzi7QoIOeXJTnGEYxuTvskwKQhschIOIBPAOLaUbgKkuHuDIf8y1Gv2b8ENu5uNvTb/ZD43jtmx9P4pS+Hwc7OW411TrkRO7XbV7qo/PqYGYpkKYDK4g7ONGnzrsmXbSmip34vXll/jAknY61QQ6D6JbbONw/psX72p+ZFvOedhlKbHRuUnDYXyQgKFWBODLb63RaMYai79qbv8mJcwDLJaPXYXpwLumHeZX91uPPjahxfvOe8VoryTpvHIbxJO85VJJ1q+7uyFy6h7LVLhifYSbb+M55p/e5Ds5gNcgUv6npPUHdbf0yjYbT1tXGjhaqk4Tx93WoOKPQ/j+nPB41akooY0YfL/ZTDzr2iCMByrx/uPQz3JE+m7VrH5BZyYjj9sSASVsULabJGFiSuGpD+u/lcUNJW+WHQetMEU7+wNo/WB3E5iG9J8qPSPFcTiTslEa8cvGPNtNgyCQ2PphlinscQsvcAQ+ALtEd1dWXPhIGbF8udK6Wx6NYt8=
template:
metadata:
creationTimestamp: null
labels:
argocd.argoproj.io/secret-type: repository
name: snothub-repo-creds
namespace: argocd
type: Opaque

12
clusters/aks-dev.yaml
View File

@@ -1,12 +0,0 @@
# Cluster config reference — values must match the corresponding overlay files.
# Read by bootstrap.sh at install time; NOT auto-propagated to ArgoCD value files.
clusterName: dev-aks # → infra/values/aks-dev/argocd-values.yaml (notifications.context.clusterName)
domain: example.com # → infra/values/base/gitea-values.yaml, renovate-values.yaml, keycloak-values.yaml (subdomains)
argocdDomain: argocd.example.com # → infra/values/aks-dev/argocd-values.yaml (global.domain)
grafanaDomain: grafana.example.com # → infra/values/aks-dev/grafana-values.yaml (ingress.hosts)
keycloakDomain: id.example.com # → infra/values/aks-dev/keycloak-values.yaml (ingress.hostname)
dotaiDomain: kubemcp.example.com # → infra/values/aks-dev/dot-ai-stack-values.yaml (dot-ai.ingress.host) — create if needed
dotaiUiDomain: kubemcpui.example.com # → infra/values/aks-dev/dot-ai-stack-values.yaml (dot-ai-ui.ingress.host) — create if needed
letsencryptEmail: admin@example.com # → cluster-resources/letsencrypt-issuer.yaml (spec.acme.email)
trustedIPs: "10.0.0.0/8,168.63.129.16/32" # → infra/values/aks-dev/traefik-values.yaml (ports.*.trustedIPs) — VNet CIDR + Azure health probe
cloudProvider: azure # → determines overlay directory and cloud-specific LB/storage annotations

12
clusters/aks-prod.yaml
View File

@@ -1,12 +0,0 @@
# Cluster config reference — values must match the corresponding overlay files.
# Read by bootstrap.sh at install time; NOT auto-propagated to ArgoCD value files.
clusterName: prod-aks # → infra/values/aks-prod/argocd-values.yaml (notifications.context.clusterName)
domain: example.com # → infra/values/base/gitea-values.yaml, renovate-values.yaml, keycloak-values.yaml (subdomains)
argocdDomain: argocd.example.com # → infra/values/aks-prod/argocd-values.yaml (global.domain)
grafanaDomain: grafana.example.com # → infra/values/aks-prod/grafana-values.yaml (ingress.hosts)
keycloakDomain: id.example.com # → infra/values/aks-prod/keycloak-values.yaml (ingress.hostname)
dotaiDomain: kubemcp.example.com # → infra/values/aks-prod/dot-ai-stack-values.yaml (dot-ai.ingress.host) — create if needed
dotaiUiDomain: kubemcpui.example.com # → infra/values/aks-prod/dot-ai-stack-values.yaml (dot-ai-ui.ingress.host) — create if needed
letsencryptEmail: admin@example.com # → cluster-resources/letsencrypt-issuer.yaml (spec.acme.email)
trustedIPs: "10.0.0.0/8,168.63.129.16/32" # → infra/values/aks-prod/traefik-values.yaml (ports.*.trustedIPs) — VNet CIDR + Azure health probe
cloudProvider: azure # → determines overlay directory and cloud-specific LB/storage annotations

12
clusters/eks-dev.yaml
View File

@@ -1,12 +0,0 @@
# Cluster config reference — values must match the corresponding overlay files.
# Read by bootstrap.sh at install time; NOT auto-propagated to ArgoCD value files.
clusterName: dev-eks # → infra/values/eks-dev/argocd-values.yaml (notifications.context.clusterName)
domain: example.com # → infra/values/base/gitea-values.yaml, renovate-values.yaml, keycloak-values.yaml (subdomains)
argocdDomain: argocd.example.com # → infra/values/eks-dev/argocd-values.yaml (global.domain)
grafanaDomain: grafana.example.com # → infra/values/eks-dev/grafana-values.yaml (ingress.hosts)
keycloakDomain: id.example.com # → infra/values/eks-dev/keycloak-values.yaml (ingress.hostname)
dotaiDomain: kubemcp.example.com # → infra/values/eks-dev/dot-ai-stack-values.yaml (dot-ai.ingress.host) — create if needed
dotaiUiDomain: kubemcpui.example.com # → infra/values/eks-dev/dot-ai-stack-values.yaml (dot-ai-ui.ingress.host) — create if needed
letsencryptEmail: admin@example.com # → cluster-resources/letsencrypt-issuer.yaml (spec.acme.email)
trustedIPs: "10.0.0.0/8" # → infra/values/eks-dev/traefik-values.yaml (ports.*.trustedIPs) — VPC CIDR
cloudProvider: eks # → determines overlay directory and cloud-specific LB/storage annotations

12
clusters/eks-prod.yaml
View File

@@ -1,12 +0,0 @@
# Cluster config reference — values must match the corresponding overlay files.
# Read by bootstrap.sh at install time; NOT auto-propagated to ArgoCD value files.
clusterName: prod-eks # → infra/values/eks-prod/argocd-values.yaml (notifications.context.clusterName)
domain: example.com # → infra/values/base/gitea-values.yaml, renovate-values.yaml, keycloak-values.yaml (subdomains)
argocdDomain: argocd.example.com # → infra/values/eks-prod/argocd-values.yaml (global.domain)
grafanaDomain: grafana.example.com # → infra/values/eks-prod/grafana-values.yaml (ingress.hosts)
keycloakDomain: id.example.com # → infra/values/eks-prod/keycloak-values.yaml (ingress.hostname)
dotaiDomain: kubemcp.example.com # → infra/values/eks-prod/dot-ai-stack-values.yaml (dot-ai.ingress.host) — create if needed
dotaiUiDomain: kubemcpui.example.com # → infra/values/eks-prod/dot-ai-stack-values.yaml (dot-ai-ui.ingress.host) — create if needed
letsencryptEmail: admin@example.com # → cluster-resources/letsencrypt-issuer.yaml (spec.acme.email)
trustedIPs: "10.0.0.0/8" # → infra/values/eks-prod/traefik-values.yaml (ports.*.trustedIPs) — VPC CIDR
cloudProvider: eks # → determines overlay directory and cloud-specific LB/storage annotations

10
clusters/eu.yaml Normal file
View File

@@ -0,0 +1,10 @@
clusterName: dev-fd-no-svg1
domain: forteapps.net
argocdDomain: argocd.127.0.0.1.nip.io
grafanaDomain: grafana.forteapps.net
keycloakDomain: id.forteapps.net
dotaiDomain: kubemcp.forteapps.net
dotaiUiDomain: kubemcpui.forteapps.net
letsencryptEmail: danijels@gmail.com
trustedIPs: "172.16.1.0/24"
cloudProvider: upcloud

12
clusters/gke-dev.yaml
View File

@@ -1,12 +0,0 @@
# Cluster config reference — values must match the corresponding overlay files.
# Read by bootstrap.sh at install time; NOT auto-propagated to ArgoCD value files.
clusterName: dev-gke # → infra/values/gke-dev/argocd-values.yaml (notifications.context.clusterName)
domain: example.com # → infra/values/base/gitea-values.yaml, renovate-values.yaml, keycloak-values.yaml (subdomains)
argocdDomain: argocd.example.com # → infra/values/gke-dev/argocd-values.yaml (global.domain)
grafanaDomain: grafana.example.com # → infra/values/gke-dev/grafana-values.yaml (ingress.hosts)
keycloakDomain: id.example.com # → infra/values/gke-dev/keycloak-values.yaml (ingress.hostname)
dotaiDomain: kubemcp.example.com # → infra/values/gke-dev/dot-ai-stack-values.yaml (dot-ai.ingress.host) — create if needed
dotaiUiDomain: kubemcpui.example.com # → infra/values/gke-dev/dot-ai-stack-values.yaml (dot-ai-ui.ingress.host) — create if needed
letsencryptEmail: admin@example.com # → cluster-resources/letsencrypt-issuer.yaml (spec.acme.email)
trustedIPs: "10.0.0.0/8,35.191.0.0/16,130.211.0.0/22" # → infra/values/gke-dev/traefik-values.yaml (ports.*.trustedIPs) — subnet + GCP health checks
cloudProvider: gke # → determines overlay directory and cloud-specific LB/storage annotations

12
clusters/gke-prod.yaml
View File

@@ -1,12 +0,0 @@
# Cluster config reference — values must match the corresponding overlay files.
# Read by bootstrap.sh at install time; NOT auto-propagated to ArgoCD value files.
clusterName: prod-gke # → infra/values/gke-prod/argocd-values.yaml (notifications.context.clusterName)
domain: example.com # → infra/values/base/gitea-values.yaml, renovate-values.yaml, keycloak-values.yaml (subdomains)
argocdDomain: argocd.example.com # → infra/values/gke-prod/argocd-values.yaml (global.domain)
grafanaDomain: grafana.example.com # → infra/values/gke-prod/grafana-values.yaml (ingress.hosts)
keycloakDomain: id.example.com # → infra/values/gke-prod/keycloak-values.yaml (ingress.hostname)
dotaiDomain: kubemcp.example.com # → infra/values/gke-prod/dot-ai-stack-values.yaml (dot-ai.ingress.host) — create if needed
dotaiUiDomain: kubemcpui.example.com # → infra/values/gke-prod/dot-ai-stack-values.yaml (dot-ai-ui.ingress.host) — create if needed
letsencryptEmail: admin@example.com # → cluster-resources/letsencrypt-issuer.yaml (spec.acme.email)
trustedIPs: "10.0.0.0/8,35.191.0.0/16,130.211.0.0/22" # → infra/values/gke-prod/traefik-values.yaml (ports.*.trustedIPs) — subnet + GCP health checks
cloudProvider: gke # → determines overlay directory and cloud-specific LB/storage annotations

12
clusters/upc-dev.yaml
View File

@@ -1,12 +0,0 @@
# Cluster config reference — values must match the corresponding overlay files.
# Read by bootstrap.sh at install time; NOT auto-propagated to ArgoCD value files.
clusterName: dev-fd-no-svg1 # → infra/values/upc-dev/argocd-values.yaml (notifications.context.clusterName)
domain: forteapps.net # → infra/values/base/gitea-values.yaml, renovate-values.yaml, keycloak-values.yaml (subdomains)
argocdDomain: argocd.127.0.0.1.nip.io # → infra/values/upc-dev/argocd-values.yaml (global.domain)
grafanaDomain: grafana.forteapps.net # → infra/values/upc-dev/grafana-values.yaml (ingress.hosts)
keycloakDomain: id.forteapps.net # → infra/values/upc-dev/keycloak-values.yaml (ingress.hostname)
dotaiDomain: kubemcp.forteapps.net # → infra/values/upc-dev/dot-ai-stack-values.yaml (dot-ai.ingress.host)
dotaiUiDomain: kubemcpui.forteapps.net # → infra/values/upc-dev/dot-ai-stack-values.yaml (dot-ai-ui.ingress.host)
letsencryptEmail: danijels@gmail.com # → cluster-resources/letsencrypt-issuer.yaml (spec.acme.email)
trustedIPs: "172.16.1.0/24" # → infra/values/upc-dev/traefik-values.yaml (ports.*.trustedIPs)
cloudProvider: upcloud # → determines overlay directory and cloud-specific LB/storage annotations

12
clusters/upc-prod.yaml
View File

@@ -1,12 +0,0 @@
# Cluster config reference — values must match the corresponding overlay files.
# Read by bootstrap.sh at install time; NOT auto-propagated to ArgoCD value files.
clusterName: prod-fd-no-svg1 # → infra/values/upc-prod/argocd-values.yaml (notifications.context.clusterName)
domain: fortedigital.com # → infra/values/base/gitea-values.yaml, renovate-values.yaml, keycloak-values.yaml (subdomains)
argocdDomain: argocd.127.0.0.1.nip.io # → infra/values/upc-prod/argocd-values.yaml (global.domain)
grafanaDomain: grafana.fortedigital.com # → infra/values/upc-prod/grafana-values.yaml (ingress.hosts)
keycloakDomain: id.fortedigital.com # → infra/values/upc-prod/keycloak-values.yaml (ingress.hostname)
dotaiDomain: kubemcp.fortedigital.com # → infra/values/upc-prod/dot-ai-stack-values.yaml (dot-ai.ingress.host)
dotaiUiDomain: kubemcpui.fortedigital.com # → infra/values/upc-prod/dot-ai-stack-values.yaml (dot-ai-ui.ingress.host)
letsencryptEmail: danijel.simeunovic@fortedigital.com # → cluster-resources/letsencrypt-issuer.yaml (spec.acme.email)
trustedIPs: "172.16.1.0/24" # → infra/values/upc-prod/traefik-values.yaml (ports.*.trustedIPs)
cloudProvider: upcloud # → determines overlay directory and cloud-specific LB/storage annotations

10
clusters/us.yaml Normal file
View File

@@ -0,0 +1,10 @@
clusterName: dev-fd-us-east1
domain: us.forteapps.net
argocdDomain: argocd.us.forteapps.net
grafanaDomain: grafana.us.forteapps.net
keycloakDomain: id.us.forteapps.net
dotaiDomain: kubemcp.us.forteapps.net
dotaiUiDomain: kubemcpui.us.forteapps.net
letsencryptEmail: danijels@gmail.com
trustedIPs: "10.0.0.0/16"
cloudProvider: tbd

311
docs/DEVELOPER-GUIDE.md
View File

@@ -9,7 +9,6 @@
- [Updating an Existing Application](#updating-an-existing-application)
- [Working with Secrets](#working-with-secrets)
- [Enabling Authentication for Applications](#enabling-authentication-for-applications)
- [Adding a New Keycloak Client](#adding-a-new-keycloak-client)
- [Troubleshooting](#troubleshooting)
- [Best Practices](#best-practices)
@@ -90,21 +89,21 @@ If you do need cluster access, install:
You'll need read/write access to these repositories:
1. **launchpad** (Config repo)
1. **sturdy-adventure** (Config repo)
```bash
git clone https://git.forteapps.net/Forte/launchpad.git
cd launchpad
git clone https://github.com/fortedigital/sturdy-adventure.git
cd sturdy-adventure
```
2. **helm-prod-values** (Values repo)
2. **helm-values** (Values repo)
```bash
git clone https://git.forteapps.net/Forte/helm-prod-values.git
cd helm-prod-values
git clone git@github.com:fortedigital/helm-values.git
cd helm-values
```
3. **forte-helm** (Chart repo - read-only for most developers)
```bash
git clone https://git.forteapps.net/Forte/forte-helm.git
git clone https://github.com/snothub/forte-helm.git
cd forte-helm
```
@@ -133,9 +132,9 @@ mkdir -p ~/dev/k8s
cd ~/dev/k8s
# Clone repositories
git clone https://git.forteapps.net/Forte/launchpad.git launchpad
git clone https://git.forteapps.net/Forte/helm-prod-values helm-prod-values
git clone https://git.forteapps.net/Forte/forte-helm forte-helm
git clone https://github.com/fortedigital/sturdy-adventure.git launchpad
git clone git@github.com:fortedigital/helm-values.git helm-prod-values
git clone https://github.com/snothub/forte-helm.git forte-helm
# Your folder structure:
# ~/dev/k8s/
@@ -175,13 +174,13 @@ npm run dev
│ - GitHub Actions builds image │
│ - Pushes to container registry (GHCR, Docker Hub) │
│ - Tags with version (e.g., v2.0.4) │
│ - Updates helm-prod-values repository with new tag │
│ - Updates helm-values repository with new tag │
└─────────────────────────────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────────┐
│ Step 3: GitOps Sync (Automated) │
│ - ArgoCD detects change in helm-prod-values │
│ - ArgoCD detects change in helm-values │
│ - Pulls updated configuration │
│ - Syncs to Kubernetes cluster │
│ - Sends Slack notification on success/failure │
@@ -201,8 +200,8 @@ Our setup uses three repositories:
| Repository | Purpose | Who Edits | How Often |
|------------|---------|-----------|-----------|
| **forte-helm** | Helm chart templates (generic, reusable) | Platform engineers | ❌ Rarely |
| **helm-prod-values** | Application configuration (image tag, env vars) | Developers / CI pipelines | ✅ Sometimes |
| **launchpad** | ArgoCD Applications (what gets deployed) | Platform / DevOps engineers | ✅ Per new app |
| **helm-values** | Application configuration (image tag, env vars) | Developers / CI pipelines | ✅ Sometimes |
| **sturdy-adventure** | ArgoCD Applications (what gets deployed) | Platform / DevOps engineers | ✅ Per new app |
### Example: Deploying "myapp"
@@ -223,7 +222,7 @@ spec:
value: {{ .Values.app.port }}
```
#### Repository: `helm-prod-values` (Your App Config)
#### Repository: `helm-values` (Your App Config)
```yaml
# myapp/values.yaml
# Your app's specific configuration
@@ -237,7 +236,7 @@ app:
value: https://api.example.com
```
#### Repository: `launchpad` (ArgoCD Application)
#### Repository: `sturdy-adventure` (ArgoCD Application)
```yaml
# apps/myapp.yaml
# Tells ArgoCD to deploy your app
@@ -248,13 +247,13 @@ metadata:
namespace: argocd
spec:
sources:
- repoURL: https://git.forteapps.net/Forte/forte-helm
- repoURL: https://github.com/snothub/forte-helm
path: forteapp
helm:
valueFiles:
- $values/myapp/values.yaml
- repoURL: git@github.com:fortedigital/helm-prod-values.git
- repoURL: git@github.com:fortedigital/helm-values.git
ref: values
destination:
@@ -316,10 +315,10 @@ Ensure your app repository has:
docker build -t ghcr.io/fortedigital/hello-world:${{ steps.version.outputs.VERSION }} .
docker push ghcr.io/fortedigital/hello-world:${{ steps.version.outputs.VERSION }}
- name: Update helm-prod-values
- name: Update helm-values
run: |
git clone git@github.com:fortedigital/helm-prod-values.git
cd helm-prod-values
git clone git@github.com:fortedigital/helm-values.git
cd helm-values
mkdir -p hello-world
cat > hello-world/values.yaml <<EOF
app:
@@ -334,7 +333,7 @@ Ensure your app repository has:
### Step 2: Create Helm Values
Create a folder in `helm-prod-values` repository:
Create a folder in `helm-values` repository:
```bash
cd ~/dev/k8s/helm-prod-values
@@ -387,7 +386,7 @@ git push
### Step 3: Create ArgoCD Application Manifest
In the `launchpad` repository, create `apps/hello-world.yaml`:
In the `sturdy-adventure` repository, create `apps/hello-world.yaml`:
```yaml
apiVersion: argoproj.io/v1alpha1
@@ -412,7 +411,7 @@ spec:
sources:
# Source 1: Helm chart templates
- repoURL: https://git.forteapps.net/Forte/forte-helm
- repoURL: https://github.com/snothub/forte-helm
path: forteapp
targetRevision: HEAD
helm:
@@ -420,7 +419,7 @@ spec:
- $values/hello-world/values.yaml
# Source 2: Helm values
- repoURL: git@github.com:fortedigital/helm-prod-values.git
- repoURL: git@github.com:fortedigital/helm-values.git
targetRevision: HEAD
ref: values
@@ -528,7 +527,7 @@ git push origin main
2. ✅ Builds new Docker image
3. ✅ Tags with new version (e.g., `v20260316-143022`)
4. ✅ Pushes to container registry
5. ✅ Updates `helm-prod-values/myapp/values.yaml` with new tag
5. ✅ Updates `helm-values/myapp/values.yaml` with new tag
6. ✅ ArgoCD detects change
7. ✅ Syncs new version to cluster
8. ✅ Sends Slack notification
@@ -683,7 +682,7 @@ git push
#### Step 4: Reference Secret in Application
Update your `helm-prod-values/myapp/values.yaml`:
Update your `helm-values/myapp/values.yaml`:
```yaml
app:
@@ -791,7 +790,7 @@ Three authentication modes are supported:
#### Step 1: Configure Helm Values
```yaml
# In helm-prod-values/myapp/values.yaml
# In helm-values/myapp/values.yaml
auth:
enabled: true
type: token # Token mode (default)
@@ -913,7 +912,7 @@ rm private/myapp-auth-oidc.yaml
#### Step 3: Configure Helm Values
```yaml
# In helm-prod-values/myapp/values.yaml
# In helm-values/myapp/values.yaml
auth:
enabled: true
type: oidc # OIDC mode
@@ -962,46 +961,6 @@ User sees application (authenticated)
---
### Accessing Authenticated User Information
The auth sidecar handles all authentication before requests reach your application. Your app never sees unauthenticated traffic — the sidecar returns 401 or redirects to the IdP first.
After successful authentication, the sidecar forwards the request to your application with user identity injected as HTTP headers:
| Header | Description | Available in |
|--------|-------------|-------------|
| `X-Auth-User` | Username or display name | Token, OIDC, MCP |
| `X-Auth-Email` | User email address | OIDC |
| `X-Auth-Subject` | OIDC `sub` claim (stable user ID) | OIDC, MCP |
| `X-Auth-Groups` | Comma-separated group memberships | OIDC (if scope includes `groups`) |
| `X-Auth-Token` | The validated access token | All modes |
**Your application reads these headers — no auth library needed:**
```javascript
// Express.js example
app.get('/profile', (req, res) => {
const user = req.headers['x-auth-user'];
const email = req.headers['x-auth-email'];
res.json({ user, email });
});
```
```python
# Flask example
@app.route('/profile')
def profile():
user = request.headers.get('X-Auth-User')
email = request.headers.get('X-Auth-Email')
return jsonify(user=user, email=email)
```
**Why this is safe**: The Kyverno-generated NetworkPolicy restricts ingress to the sidecar port only. Traffic cannot bypass the sidecar to reach the application port directly, so the `X-Auth-*` headers can be trusted unconditionally.
**Key principle**: Your application is zero-trust-unaware by design. It reads headers and renders UI. All authentication complexity lives in the sidecar and Kyverno policy.
---
### Authentication Configuration Reference
#### Helm Values Schema
@@ -1066,7 +1025,7 @@ policies.forteapps.io/auth-upstream-url: "http://localhost:3000"
#### Sidecar Configuration
The auth sidecar container:
- **Image**: `ghcr.io/fortedigital/auth-sidecar:latest`
- **Image**: `ghcr.io/snothub/stunning-memory:latest`
- **Port**: 8080
- **Resources**: 10m CPU / 32Mi memory (requests), 50m CPU / 64Mi memory (limits)
- **Health checks**: `/healthz` endpoint
@@ -1089,7 +1048,7 @@ policies.forteapps.io/auth-image-version: "v1.2.3"
#### Example 1: Internal API with Token Auth
```yaml
# helm-prod-values/internal-api/values.yaml
# helm-values/internal-api/values.yaml
app:
image:
repository: ghcr.io/company/internal-api
@@ -1117,7 +1076,7 @@ curl -H "Authorization: Bearer d4f88f..." \
#### Example 2: User-Facing App with OIDC
```yaml
# helm-prod-values/web-app/values.yaml
# helm-values/web-app/values.yaml
app:
image:
repository: ghcr.io/company/web-app
@@ -1152,7 +1111,7 @@ kubectl create secret generic auth-oidc \
#### Example 3: MCP Server with OAuth 2.0
```yaml
# helm-prod-values/mcp-server/values.yaml
# helm-values/mcp-server/values.yaml
app:
image:
repository: ghcr.io/company/mcp-server
@@ -1176,7 +1135,7 @@ The MCP auth mode implements RFC 9728 (OAuth 2.0 Protected Resource Metadata) fo
#### Example 4: Disabling Authentication
```yaml
# helm-prod-values/public-api/values.yaml
# helm-values/public-api/values.yaml
auth:
enabled: false # No authentication
@@ -1288,202 +1247,6 @@ kubectl logs -n myapp <pod-name> -c authn
---
## Adding a New Keycloak Client
There are two ways to add an OIDC client, depending on your use case:
| Method | Best for | Who edits the infra repo? |
|--------|----------|--------------------------|
| **Self-service** (recommended) | New apps that deploy their own resources | App developer — no infra changes needed |
| **Legacy (realm JSON)** | Existing clients already defined in forte-realm.json (e.g., Gitea) | Platform engineer |
Both methods are served by the **Keycloak Client Registrar** CronJob, which runs every 2 minutes.
### Self-Service OIDC Client Registration
This is the recommended flow for new applications. Your app deploys a labeled config Secret in its own namespace; the platform handles everything else.
#### How It Works
1. You deploy a Secret with label `keycloak.forteapps.net/client-config: "true"` containing a `client.json` definition
2. A **Kyverno ClusterPolicy** (`keycloak-client-config-cloner`) clones it to the `keycloak` namespace
3. The **Client Registrar CronJob** picks it up within 2 minutes:
- Registers (or updates) the client in Keycloak
- Fetches the auto-generated client secret
- Creates a credential Secret in your app's namespace
- Annotates the config Secret with sync status
#### Step 1: Create the Config Secret
Deploy this Secret in your application's namespace (e.g., as part of your Helm chart or Kustomize overlay):
```yaml
apiVersion: v1
kind: Secret
metadata:
name: keycloak-client-myapp
namespace: myapp
labels:
keycloak.forteapps.net/client-config: "true"
stringData:
client.json: |
{
"clientId": "myapp",
"name": "My Application",
"redirectUris": ["https://myapp.forteapps.net/*"],
"webOrigins": ["https://myapp.forteapps.net"],
"defaultClientScopes": ["openid", "email", "profile"],
"protocolMappers": [],
"secret": {
"namespace": "myapp",
"name": "myapp-oidc-credentials",
"keys": { "clientId": "client-id", "clientSecret": "client-secret" }
}
}
```
**`client.json` fields**:
| Field | Required | Description |
|-------|----------|-------------|
| `clientId` | Yes | Keycloak client ID |
| `name` | Yes | Display name in Keycloak |
| `redirectUris` | Yes | Allowed redirect URIs |
| `webOrigins` | Yes | Allowed web origins (CORS) |
| `defaultClientScopes` | No | Scopes (default: `["openid", "email", "profile"]`) |
| `protocolMappers` | No | Custom claim mappers (default: `[]`) |
| `secret.namespace` | No | Namespace for the credential Secret (default: source namespace) |
| `secret.name` | No | Name of the credential Secret (default: `<clientId>-oidc-credentials`) |
| `secret.keys.clientId` | No | Key name for client ID in credential Secret (default: `client-id`) |
| `secret.keys.clientSecret` | No | Key name for client secret in credential Secret (default: `client-secret`) |
#### Step 2: Reference the Credential Secret
In your application's deployment config, reference the credential Secret that the registrar creates:
```yaml
env:
- name: OIDC_CLIENT_ID
valueFrom:
secretKeyRef:
name: myapp-oidc-credentials
key: client-id
- name: OIDC_CLIENT_SECRET
valueFrom:
secretKeyRef:
name: myapp-oidc-credentials
key: client-secret
```
#### Step 3: Deploy and Wait
Commit and push your changes. The credential Secret will appear within 2 minutes:
```bash
# Watch for the credential Secret to be created
kubectl get secret myapp-oidc-credentials -n myapp -w
# Check registrar logs
kubectl logs -n keycloak job/$(kubectl get jobs -n keycloak --sort-by=.metadata.creationTimestamp -o jsonpath='{.items[-1].metadata.name}')
# Check sync status on the config Secret
kubectl get secret keycloak-client-myapp -n keycloak -o jsonpath='{.metadata.annotations}'
```
#### Change Detection
The registrar computes a SHA-256 hash of `client.json` and stores it as an annotation. On subsequent runs, it skips processing if:
- The hash hasn't changed, AND
- The credential Secret already exists in the target namespace
To force a re-sync, update any field in `client.json` (e.g., add a trailing space to `name`).
### Legacy Method: Realm JSON
Existing clients (like Gitea) are defined directly in `forte-realm.json` inside `keycloak-values.yaml`. The registrar syncs their secrets via client attributes.
#### Step 1: Add Client to Realm Config
In `infra/values/base/keycloak-values.yaml`, add a new entry to the `clients` array in `forte-realm.json`:
```json
{
"clientId": "myapp",
"name": "My Application",
"enabled": true,
"protocol": "openid-connect",
"clientAuthenticatorType": "client-secret",
"standardFlowEnabled": true,
"directAccessGrantsEnabled": false,
"publicClient": false,
"redirectUris": ["https://myapp.forteapps.net/*"],
"webOrigins": ["https://myapp.forteapps.net"],
"defaultClientScopes": ["openid", "email", "profile"],
"attributes": {
"k8s.secret.sync": "true",
"k8s.secret.namespace": "myapp",
"k8s.secret.name": "myapp-oidc-credentials",
"k8s.secret.client-id-key": "key",
"k8s.secret.client-secret-key": "secret"
}
}
```
**Important**:
- Do **NOT** include a `"secret"` field — Keycloak generates one automatically
- The `attributes` block tells the registrar where to create the K8s Secret
- Set `client-id-key` / `client-secret-key` to match what the consuming app expects (defaults: `client-id` / `client-secret`)
#### Step 2: Reference the Secret in Your Application
```yaml
existingSecret: myapp-oidc-credentials
```
#### Step 3: Commit and Push
```bash
cd ~/dev/k8s/launchpad
git add infra/values/base/keycloak-values.yaml
git commit -m "Add myapp Keycloak client with auto-sync"
git push
```
ArgoCD will sync the Keycloak config, and the registrar CronJob will pick up the new client within 2 minutes.
#### Legacy Sync Attribute Reference
| Attribute | Required | Default | Description |
|-----------|----------|---------|-------------|
| `k8s.secret.sync` | Yes | — | Set to `"true"` to enable syncing |
| `k8s.secret.namespace` | Yes | — | Target K8s namespace for the secret |
| `k8s.secret.name` | Yes | — | Name of the K8s Secret to create |
| `k8s.secret.client-id-key` | No | `client-id` | Field name for the client ID in the K8s Secret |
| `k8s.secret.client-secret-key` | No | `client-secret` | Field name for the client secret in the K8s Secret |
### Retrieving Secrets for External Deployments
The registrar always writes a **central copy** of every synced secret to the `secrets` namespace, in addition to the target namespace. This allows operators to retrieve client credentials for applications deployed outside this cluster:
```bash
# View the central copy
kubectl get secret gitea-oidc-credentials -n secrets -o yaml
# Extract the client secret for use elsewhere
kubectl get secret myapp-oidc-credentials -n secrets \
-o jsonpath='{.data.client-secret}' | base64 -d
```
### Registrar Behavior Notes
- The registrar runs as a CronJob every 2 minutes (`concurrencyPolicy: Forbid`)
- If the target namespace doesn't exist, the target write is skipped with a warning (the central copy still happens)
- A central copy is **always** written to the `secrets` namespace for every synced client
- The registrar uses the `keycloak-credentials` secret for admin authentication
- Created secrets have the label `app.kubernetes.io/managed-by: keycloak-client-registrar`
---
## Troubleshooting
### Application Not Deploying
@@ -1540,7 +1303,7 @@ kubectl exec -n myapp <pod-name> -- env
# Check if secrets exist
kubectl get secrets -n myapp
# Increase resources in helm-prod-values
# Increase resources in helm-values
vim ~/dev/k8s/helm-prod-values/myapp/values.yaml
```
@@ -1689,7 +1452,7 @@ If you're stuck:
### Configuration Management
✅ **DO**:
- Keep configuration in `helm-prod-values` repository
- Keep configuration in `helm-values` repository
- Use environment variables for config
- Document what each value does
- Use reasonable resource limits
@@ -1816,4 +1579,4 @@ Now that you understand the basics:
- Docs: [Full documentation index](README.md)
- Help: Contact platform team
**Last Updated**: 2026-04-16
**Last Updated**: 2026-03-16

198
docs/GITOPS-ARCHITECTURE.md
View File

@@ -12,16 +12,16 @@
## Overview
This Kubernetes cluster uses a **GitOps approach** powered by **ArgoCD**, where Git repositories serve as the single source of truth for both infrastructure and application deployments. The cluster setup is **cloud-agnostic**, with ready-to-use configurations for **UpCloud**, **AWS EKS**, **Azure AKS**, and **GCP GKE**.
This Kubernetes cluster uses a **GitOps approach** powered by **ArgoCD**, where Git repositories serve as the single source of truth for both infrastructure and application deployments. The cluster is running on **UpCloud Managed Kubernetes** but is designed to be cloud-agnostic.
### Key Characteristics
- **Environment**: Production (internal use only)
- **Cluster Type**: Multi-cloud, multi-cluster via Kustomize overlays (UpCloud, AWS, Azure, GCP)
- **Cluster Type**: Single cluster, single environment
- **GitOps Tool**: ArgoCD
- **Deployment Pattern**: App-of-Apps
- **Secret Management**: Sealed Secrets (kubeseal)
- **Ingress**: Traefik with Let's Encrypt TLS
- **Monitoring**: Prometheus + Grafana + Loki + Tempo + Fluent-Bit
- **Monitoring**: Prometheus + Grafana + Loki + Fluent-Bit
- **Policy Engine**: Kyverno
- **Notifications**: Slack integration for sync status
@@ -47,14 +47,14 @@ This Kubernetes cluster uses a **GitOps approach** powered by **ArgoCD**, where
│ │ │
│ │ │
└────────► Update image tag ─┴──────────────────────────┘
in helm-prod-values │
in helm-values │
│
▼
┌────────────────────────────────┐
│ Config Repository │
│ (ArgoCD Applications) │
│ git.forteapps.net/Forte/ │
│ launchpad │
│ github.com/snothub/ │
│ sturdy-adventure │
└────────────────────────────────┘
│
│
@@ -62,8 +62,8 @@ This Kubernetes cluster uses a **GitOps approach** powered by **ArgoCD**, where
│
▼
┌────────────────────────────────┐
│ Kubernetes Clusters │
│ (UpCloud, AWS, Azure, GCP) │
│ Kubernetes Cluster │
│ (UpCloud Managed) │
│ │
│ ┌──────────────────────────┐ │
│ │ ArgoCD │ │
@@ -83,7 +83,6 @@ This Kubernetes cluster uses a **GitOps approach** powered by **ArgoCD**, where
│ │ - Prometheus │ │
│ │ - Grafana │ │
│ │ - Loki │ │
│ │ - Tempo │ │
│ │ - Fluent-Bit │ │
│ └──────────────────────────┘ │
│ │
@@ -109,78 +108,87 @@ This Kubernetes cluster uses a **GitOps approach** powered by **ArgoCD**, where
## Repository Structure
### 1. **Config Repository** (Current Repo)
**Repository**: `https://git.forteapps.net/Forte/launchpad`
**Repository**: `https://github.com/fortedigital/sturdy-adventure.git`
**Purpose**: GitOps configuration - ArgoCD Applications and cluster resources
**Location**: `C:\dev\k8s\launchpad`
```
launchpad/
sturdy-adventure/
├── bootstrap.sh # Cluster initialization script
├── _app-of-apps-upc-dev.yaml # Root ArgoCD Application (upc-dev cluster)
├── _app-of-apps-upc-prod.yaml # Root ArgoCD Application (upc-prod cluster)
├── _app-of-apps.yaml # Root ArgoCD Application (App-of-Apps pattern)
│
├── infra/ # Infrastructure ArgoCD Applications (Kustomize)
│ ├── base/ # Base Application manifests (upc-dev defaults)
│ │ ├── kustomization.yaml
│ │ ├── traefik-application.yaml
│ │ ├── keycloak.yaml
│ │ ├── grafana.yaml
│ │ ├── gitea.yaml
│ │ ├── gitea-actions.yaml
│ │ ├── tempo.yaml
│ │ ├── renovate.yaml
│ │ ├── ... # All other Application manifests
│ │ └── secrets.yaml
│ ├── overlays/ # Per-cluster Kustomize overrides
│ │ ├── upc-dev/ # UpCloud Dev (uses base as-is)
│ │ ├── upc-prod/ # UpCloud Prod (patches value paths)
│ │ ├── eks-dev/ # AWS EKS Dev
│ │ ├── eks-prod/ # AWS EKS Prod
│ │ ├── aks-dev/ # Azure AKS Dev
│ │ ├── aks-prod/ # Azure AKS Prod
│ │ ├── gke-dev/ # GCP GKE Dev
│ │ └── gke-prod/ # GCP GKE Prod
│ ├── dashboards/ # Grafana dashboard ConfigMaps
├── infra/ # Infrastructure ArgoCD Applications
│ ├── enterprise-apps.yaml # Parent app managing all apps in apps/
│ ├── cluster-resources-application.yaml
│ ├── traefik-application.yaml
│ ├── cert-manager-application.yaml
│ ├── kyverno.yaml
│ ├── kyverno-policies.yaml
│ ├── prometheus.yaml
│ ├── grafana.yaml
│ ├── loki.yaml
│ ├── fluent-bit.yaml
│ ├── trivy.yaml
│ ├── sealedsecrets.yaml
│ ├── secrets.yaml
│ └── values/ # Helm value overrides for infra
│ ├── base/ # Cloud-agnostic shared values
│ ├── upc-{dev,prod}/ # UpCloud: storage class, LB, pricing
│ ├── aws-{dev,prod}/ # AWS: gp3, NLB, CUR pricing
│ ├── aks-{dev,prod}/ # Azure: managed-csi-premium, Standard LB
│ └── gcp-{dev,prod}/ # GCP: premium-rwo, L4 LB
│ ├── argocd-values.yaml
│ ├── prometheus-values.yaml
│ ├── grafana-values.yaml
│ ├── loki-values.yaml
│ └── fluent-bit-values.yaml
│
├── apps/ # Business Application ArgoCD manifests (Kustomize)
│ ├── base/ # Base app manifests
│ │ ├── kustomization.yaml
│ │ ├── dot-ai-stack.yaml
│ │ └── ...
│ └── overlays/
│ ├── upc-dev/ # Uses base as-is
│ └── upc-prod/ # Patches value paths
├── apps/ # Business Application ArgoCD manifests
│ ├── mcp10x.yaml # MCP 10X application
│ ├── musicman.yaml # Music Man application
│ ├── dot-ai-stack.yaml # Dot AI Stack
│ └── argo-mcp.yaml # ArgoCD MCP server
│
├── cluster-resources/ # Cluster-wide Kubernetes resources
│ ├── ...
│ ├── cert-manager-namespace.yaml
│ ├── secrets-namespace.yaml
│ ├── letsencrypt-issuer.yaml # Let's Encrypt ClusterIssuer
│ ├── kyverno-config.yaml
│ ├── argocd-notifications-secret-sealed.yaml
│ ├── snothub-repo-credentials-sealed.yaml
│ ├── forte10x-repo-credentials-sealed.yaml
│ ├── mcp10x-repo-credentials-sealed.yaml
│ └── policies/ # Kyverno policies
│ ├── deployment-verifier.yaml
│ ├── label-checker.yaml
│ ├── bare-pod-cleaner.yaml
│ ├── replicaset-cleaner.yaml
│ ├── default-ns-blocker.yaml
│ ├── secret-cloner.yaml
│ └── auth-sidecar-injector.yaml
│
├── secrets/ # Application secrets (sealed, per-cluster)
│ └── upc-dev/ # Secrets for upc-dev cluster
├── secrets/ # Application secrets (sealed)
│ ├── argocd-mcp-credentials.yaml
│ ├── dot-ai-secrets.yaml
│ ├── mcp10x-credentials-sealed.yaml
│ └── musicman-credentials.yaml
│
├── private/ # Local-only files (NOT in Git)
│ ├── *.yaml # Unsealed secrets
│ └── *.sh # Helper scripts
│
└── docs/ # Documentation
├── GITOPS-ARCHITECTURE.md # This file
├── DEVELOPER-GUIDE.md
├── OPERATIONS-RUNBOOK.md
└── REFERENCE.md
```
**Key Points**:
- `_app-of-apps-upc-dev.yaml` and `_app-of-apps-upc-prod.yaml` are the per-cluster root Applications
- Kustomize overlays in `infra/overlays/` render base Applications with per-cluster patches
- Helm values are split: `values/base/` (shared) + `values/upc-dev/` or `values/upc-prod/` (cluster-specific)
- `apps/` follows the same base/overlays pattern for business applications
- `_app-of-apps.yaml` is the root Application that ArgoCD monitors
- `infra/enterprise-apps.yaml` auto-discovers all apps in `apps/` folder
- Changes pushed to this repo trigger automatic syncs in ArgoCD
- `private/` folder contains local-only files (Git-ignored)
---
### 2. **Helm Charts Repository**
**Repository**: `https://git.forteapps.net/Forte/forte-helm`
**Repository**: `https://github.com/snothub/forte-helm`
**Purpose**: Reusable Helm chart templates for Forte applications
**Location**: `C:\dev\k8s\forte-helm`
@@ -214,7 +222,7 @@ forte-helm/
---
### 3. **Helm Values Repository**
**Repository**: `git@github.com:fortedigital/helm-prod-values.git`
**Repository**: `git@github.com:fortedigital/helm-values.git`
**Purpose**: Environment-specific configuration for each application
**Location**: `C:\dev\k8s\helm-prod-values`
@@ -224,6 +232,8 @@ helm-prod-values/
│ └── values.yaml # MCP 10X configuration
├── musicman/
│ └── values.yaml # Music Man configuration
├── mcpcoder/
│ └── values.yaml # MCP Coder configuration
└── argocd-mcp/
└── values.yaml # ArgoCD MCP configuration
```
@@ -273,7 +283,7 @@ app-repository/
2. Build Docker image
3. Tag with version (e.g., `v2.0.4`)
4. Push to container registry (GHCR, Docker Hub, etc.)
5. Update image tag in `helm-prod-values` repository
5. Update image tag in `helm-values` repository
6. ArgoCD detects change and syncs automatically
---
@@ -283,7 +293,7 @@ app-repository/
### The App-of-Apps Pattern
```
_app-of-apps-{cluster}.yaml (Root, per cluster — e.g. upc-dev, eks-prod, gke-dev)
_app-of-apps.yaml (Root)
│
├── infrastructure-apps (manages infra/)
│ ├── cluster-resources-application
@@ -292,7 +302,6 @@ _app-of-apps-{cluster}.yaml (Root, per cluster — e.g. upc-dev, eks-prod, gke-d
│ ├── kyverno
│ ├── prometheus
│ ├── grafana
│ ├── tempo
│ └── ... (other infra apps)
│
└── enterprise-apps (manages apps/)
@@ -303,10 +312,10 @@ _app-of-apps-{cluster}.yaml (Root, per cluster — e.g. upc-dev, eks-prod, gke-d
```
**How It Works**:
1. Bootstrap script installs ArgoCD and applies `_app-of-apps-upc-dev.yaml` (or `upc-prod`)
2. ArgoCD creates the root Application which monitors the appropriate `infra/overlays/` folder
3. Kustomize renders base Applications with cluster-specific patches
4. `enterprise-apps` Application monitors the cluster's `apps/overlays/` folder
1. Bootstrap script installs ArgoCD and applies `_app-of-apps.yaml`
2. ArgoCD creates the root Application which monitors `infra/` folder
3. Each YAML in `infra/` becomes a child Application
4. `enterprise-apps.yaml` monitors `apps/` folder and auto-discovers applications
5. ArgoCD continuously syncs (every 60s) and auto-heals drift
### Sync Waves & Ordering
@@ -334,13 +343,13 @@ Applications like `mcp10x` and `musicman` use multiple sources:
```yaml
spec:
sources:
- repoURL: https://git.forteapps.net/Forte/forte-helm
- repoURL: https://github.com/snothub/forte-helm
path: forteapp # Helm chart templates
helm:
valueFiles:
- $values/mcp10x/values.yaml # Reference to second source
- repoURL: git@github.com:fortedigital/helm-prod-values.git
- repoURL: git@github.com:fortedigital/helm-values.git
targetRevision: HEAD
ref: values # Named reference
```
@@ -351,43 +360,6 @@ spec:
- Easy to update all apps by changing the chart
- Environment-specific values isolated in separate repo
### Multi-Cluster Pattern
Kustomize overlays enable deploying the same Applications across clusters with different configurations:
```yaml
# infra/base/ contains default (upc-dev) Applications
# Helm values are layered: base + cluster-specific
valueFiles:
- $values/infra/values/base/traefik-values.yaml # Shared config
- $values/infra/values/upc-dev/traefik-values.yaml # Cluster-specific
# infra/overlays/upc-prod/kustomization.yaml patches the second valueFile
patches:
- target:
kind: Application
name: traefik
patch: |
- op: replace
path: /spec/sources/0/helm/valueFiles/1
value: $values/infra/values/upc-prod/traefik-values.yaml
```
Cloud-specific values (storage classes, load balancer annotations, cost model) are isolated in per-cluster value files. Base values are fully cloud-agnostic:
| Cloud | Storage Class | Load Balancer | OpenCost Provider |
|-------|--------------|---------------|-------------------|
| **UpCloud** | `upcloud-block-storage-maxiops` | UpCloud LB (ProxyProtocol v2) | Custom pricing |
| **AWS EKS** | `gp3` (EBS CSI) | NLB (ProxyProtocol v2) | AWS CUR |
| **Azure AKS** | `managed-csi-premium` | Standard LB (`externalTrafficPolicy: Local`) | Azure Billing API |
| **GCP GKE** | `premium-rwo` (PD CSI) | L4 passthrough NLB | GCP Cloud Billing |
**Benefits**:
- Single source of truth for Application definitions
- Cluster-specific values isolated per overlay
- Easy to add new clusters by creating a new overlay
- Base values shared across all clusters reduce duplication
---
## CI/CD Pipeline
@@ -417,8 +389,8 @@ jobs:
- name: Update Helm values
run: |
git clone git@github.com:fortedigital/helm-prod-values.git
cd helm-prod-values/app
git clone git@github.com:fortedigital/helm-values.git
cd helm-values/app
sed -i "s/tag: .*/tag: $VERSION/" values.yaml
git commit -am "Update app to $VERSION"
git push
@@ -430,12 +402,12 @@ jobs:
1. **Config Repo Change**:
- Developer updates `apps/myapp.yaml`
- Pushes to `launchpad` repo
- Pushes to `sturdy-adventure` repo
- ArgoCD detects change (60s reconciliation)
- Syncs application to cluster
2. **Helm Values Change**:
- CI/CD updates `helm-prod-values/myapp/values.yaml`
- CI/CD updates `helm-values/myapp/values.yaml`
- ArgoCD detects change
- Pulls new Helm chart with updated values
- Applies to cluster
@@ -517,6 +489,7 @@ git commit -m "Add app credentials"
**Private Repository Credentials** stored as SealedSecrets:
```yaml
# cluster-resources/snothub-repo-credentials-sealed.yaml
# cluster-resources/forte10x-repo-credentials-sealed.yaml
```
@@ -555,9 +528,8 @@ annotations:
1. **Prometheus**: Metrics collection and storage
2. **Grafana**: Metrics visualization and dashboards
3. **Loki**: Log aggregation
4. **Tempo**: Distributed tracing (OTLP)
5. **Fluent-Bit**: Log shipping from pods to Loki
6. **Trivy**: Container vulnerability scanning
4. **Fluent-Bit**: Log shipping from pods to Loki
5. **Trivy**: Container vulnerability scanning
### Slack Notifications
@@ -586,7 +558,7 @@ Notifications include:
**Rebuild Process**:
1. Provision new Kubernetes cluster
2. Clone `launchpad` repository
2. Clone `sturdy-adventure` repository
3. Run `./bootstrap.sh`
4. ArgoCD installs and syncs all applications
5. Manually recreate unsealed secrets and seal them
@@ -642,7 +614,7 @@ Notifications include:
✅ **DO**:
- Follow the `forteapp` chart pattern
- Use semantic versioning for image tags
- Update helm-prod-values via CI/CD
- Update helm-values via CI/CD
- Test locally with Docker Compose
- Document environment variables
@@ -663,6 +635,6 @@ Notifications include:
---
**Last Updated**: 2026-04-22
**Last Updated**: 2026-03-16
**Maintained By**: Platform Team
**Questions?**: Contact #platform-support on Slack

320
docs/OPERATIONS-RUNBOOK.md
View File

@@ -37,7 +37,7 @@ Bootstrap a new cluster from scratch:
#### Prerequisites
1. **Kubernetes cluster running** (UpCloud, AWS EKS, Azure AKS, GCP GKE, or any K8s cluster)
1. **Kubernetes cluster running** (UpCloud or any K8s cluster)
2. **kubectl configured** with admin access
3. **Repositories cloned** locally
@@ -51,16 +51,14 @@ kubectl get nodes
```bash
# 1. Clone config repository
git clone https://git.forteapps.net/Forte/launchpad
cd launchpad
git clone https://github.com/fortedigital/sturdy-adventure.git
cd sturdy-adventure
# 2. Run bootstrap script with cluster target
# Available clusters: upc-dev, upc-prod, eks-dev, eks-prod,
# aks-dev, aks-prod, gke-dev, gke-prod
./bootstrap.sh upc-dev
# 2. Set cluster name (optional)
export CLUSTER_NAME="prod-cluster-01"
# Cluster config is loaded from clusters/<cluster>.yaml
# (cloudProvider, trustedIPs, domain, etc.)
# 3. Run bootstrap script
./bootstrap.sh
```
**What Happens:**
@@ -87,8 +85,7 @@ kubectl get applications -n argocd
1. **Configure DNS** for ingress domains:
- `argocd.127.0.0.1.nip.io` (local dev)
- `*.forteapps.net` (dev)
- `*.fortedigital.com` (production)
- `*.forteapps.net` (production)
2. **Verify Let's Encrypt certificates**:
```bash
@@ -110,7 +107,7 @@ kubectl get applications -n argocd
### ArgoCD Repository Access Setup
ArgoCD needs SSH access to private Git repositories to pull manifests and Helm values. This section covers setting up deploy keys for Gitea repositories.
ArgoCD needs SSH access to private Git repositories to pull manifests and Helm values. This section covers setting up deploy keys for GitHub repositories.
#### Why Deploy Keys?
@@ -122,7 +119,7 @@ ArgoCD needs SSH access to private Git repositories to pull manifests and Helm v
#### Prerequisites
- kubectl access to the cluster
- Write access to the Gitea repository
- Write access to the GitHub repository
- ArgoCD installed and running
#### Setup Procedure
@@ -133,25 +130,25 @@ Generate a dedicated SSH key for ArgoCD without a passphrase (required for autom
```bash
# Generate ED25519 key (recommended - smaller and more secure)
ssh-keygen -t ed25519 -C "argocd-deploy-key-launchpad" -f argocd-deploy-key -N ""
ssh-keygen -t ed25519 -C "argocd-deploy-key-sturdy-adventure" -f argocd-deploy-key -N ""
# Or RSA key if ED25519 is not supported
ssh-keygen -t rsa -b 4096 -C "argocd-deploy-key-launchpad" -f argocd-deploy-key -N ""
ssh-keygen -t rsa -b 4096 -C "argocd-deploy-key-sturdy-adventure" -f argocd-deploy-key -N ""
```
This creates two files:
- `argocd-deploy-key` - Private key (keep secret)
- `argocd-deploy-key.pub` - Public key (add to Gitea)
- `argocd-deploy-key.pub` - Public key (add to GitHub)
**Step 2: Add Public Key to Gitea**
**Step 2: Add Public Key to GitHub**
1. Copy the public key:
```bash
cat argocd-deploy-key.pub
```
2. Go to Gitea repository settings:
- Navigate to: `https://git.forteapps.net/Forte/launchpad/settings/keys`
2. Go to GitHub repository settings:
- Navigate to: `https://github.com/fortedigital/sturdy-adventure/settings/keys`
- Or: Repository → Settings → Deploy keys
3. Click **"Add deploy key"**
@@ -160,45 +157,90 @@ This creates two files:
- ☐ Allow write access (leave unchecked - read-only is sufficient)
- Click **"Add key"**
4. Repeat for the `helm-prod-values` repository if it's private:
4. Repeat for the `helm-values` repository if it's private:
```bash
# Generate separate key for helm-prod-values repo
ssh-keygen -t ed25519 -C "argocd-deploy-key-helm-prod-values" -f argocd-helm-prod-values-key -N ""
# Generate separate key for helm-values repo
ssh-keygen -t ed25519 -C "argocd-deploy-key-helm-values" -f argocd-helm-values-key -N ""
# Add to: https://git.forteapps.net/Forte/helm-prod-values/settings/keys
# Add to: https://github.com/fortedigital/helm-values/settings/keys
```
**Step 3: Create Kubernetes Secret**
Add the private key to ArgoCD as a repository secret:
Save the following file in private/ (gitignored) folder as secret.yaml
```bash
# Create secret for sturdy-adventure repository
kubectl create secret generic repo-sturdy-adventure \
--from-file=sshPrivateKey=argocd-deploy-key \
--namespace=argocd \
--dry-run=client -o yaml | kubectl apply -f -
# Label it for ArgoCD to recognize
kubectl label secret repo-sturdy-adventure \
-n argocd \
argocd.argoproj.io/secret-type=repository
# Add repository annotations
kubectl annotate secret repo-sturdy-adventure \
-n argocd \
managed-by=argocd.argoproj.io
```
Alternatively, create a complete repository secret with all metadata:
```bash
kubectl apply -f - <<EOF
apiVersion: v1
kind: Secret
metadata:
name: forte-helm-repo
name: repo-sturdy-adventure
namespace: argocd
labels:
argocd.argoproj.io/secret-type: repository
annotations:
managed-by: argocd.argoproj.io
type: Opaque
stringData:
type: git
url: ssh://git@git.forteapps.net:2222/Forte/forte-helm.git
url: git@github.com:fortedigital/sturdy-adventure.git
sshPrivateKey: |
<paste your private key here>
project: default
```
Seal the secret using `kubeseal` command
```bash
kubeseal --format=yaml \
--namespace=argocd \
< private/secret.yaml \
> secrets/forte-helm-repo-secret-sealed.yaml
$(cat argocd-deploy-key | sed 's/^/ /')
EOF
```
**Step 4: Register Repository in ArgoCD**
Check in secrets/forte-helm-repo-secret-sealed.yaml and let Argo sync and create the secret.
Add the repository to ArgoCD's configuration:
```bash
# Via kubectl (recommended for GitOps)
kubectl apply -f - <<EOF
apiVersion: v1
kind: Secret
metadata:
name: repo-sturdy-adventure
namespace: argocd
labels:
argocd.argoproj.io/secret-type: repository
type: Opaque
stringData:
type: git
url: git@github.com:fortedigital/sturdy-adventure.git
sshPrivateKey: |
$(cat argocd-deploy-key | sed 's/^/ /')
insecure: "false"
enableLfs: "false"
EOF
# Or via ArgoCD UI
# 1. Open ArgoCD UI: kubectl port-forward svc/argocd-server -n argocd 8080:443
# 2. Navigate to: Settings → Repositories → Connect Repo
# 3. Connection Method: Via SSH
# 4. Repository URL: git@github.com:fortedigital/sturdy-adventure.git
# 5. SSH private key: Paste private key content
# 6. Click "Connect"
```
**Step 5: Verify Repository Access**
@@ -210,7 +252,7 @@ kubectl get secrets -n argocd -l argocd.argoproj.io/secret-type=repository
# Settings → Repositories → Should show "Successful" status
# Test by creating an application
kubectl apply -f _app-of-apps-upc-dev.yaml # or _app-of-apps-upc-prod.yaml
kubectl apply -f _app-of-apps.yaml
# Check application sync status
kubectl get applications -n argocd
@@ -230,7 +272,7 @@ metadata:
spec:
project: default
source:
repoURL: ssh://git@git.forteapps.net:2222/Forte/launchpad.git
repoURL: git@github.com:fortedigital/sturdy-adventure.git
targetRevision: main
path: cluster-resources
destination:
@@ -273,15 +315,15 @@ rm /tmp/test-repo-access.yaml
# Generate new key
ssh-keygen -t ed25519 -C "argocd-deploy-key-$(date +%Y%m)" -f argocd-new-key -N ""
# Add new public key to Gitea (keep old key for now)
# Add new public key to GitHub (keep old key for now)
# Update Kubernetes secret
kubectl create secret generic repo-launchpad \
kubectl create secret generic repo-sturdy-adventure \
--from-file=sshPrivateKey=argocd-new-key \
--namespace=argocd \
--dry-run=client -o yaml | kubectl apply -f -
# Test access, then remove old deploy key from Gitea
# Test access, then remove old deploy key from GitHub
# Clean up
shred -u argocd-new-key
@@ -292,8 +334,8 @@ rm /tmp/test-repo-access.yaml
# List all repository secrets
kubectl get secrets -n argocd -l argocd.argoproj.io/secret-type=repository
# Review deploy keys in Gitea
# Visit: https://git.forteapps.net/Forte/launchpad/settings/keys
# Review deploy keys in GitHub
# Visit: https://github.com/fortedigital/sturdy-adventure/settings/keys
```
4. **Use Different Keys per Repository**
@@ -307,27 +349,27 @@ rm /tmp/test-repo-access.yaml
```bash
# Check if secret exists
kubectl get secret repo-launchpad -n argocd
kubectl get secret repo-sturdy-adventure -n argocd
# Verify secret has correct label
kubectl get secret repo-launchpad -n argocd -o yaml | grep argocd.argoproj.io/secret-type
kubectl get secret repo-sturdy-adventure -n argocd -o yaml | grep argocd.argoproj.io/secret-type
# Check ArgoCD application controller logs
kubectl logs -n argocd deployment/argocd-application-controller | grep -i "permission denied"
# Verify deploy key is added to Gitea
# Visit: https://git.forteapps.net/Forte/launchpad/settings/keys
# Verify deploy key is added to GitHub
# Visit: https://github.com/fortedigital/sturdy-adventure/settings/keys
```
**Issue: "Host key verification failed"**
```bash
# Add Gitea to known_hosts
# Add GitHub to known_hosts
kubectl exec -n argocd deployment/argocd-repo-server -- \
ssh-keyscan git.forteapps.net >> ~/.ssh/known_hosts
ssh-keyscan github.com >> ~/.ssh/known_hosts
# Or disable strict host key checking (less secure)
kubectl patch secret repo-launchpad -n argocd \
kubectl patch secret repo-sturdy-adventure -n argocd \
--type merge \
-p '{"stringData":{"insecure":"true"}}'
```
@@ -339,7 +381,7 @@ kubectl patch secret repo-launchpad -n argocd \
kubectl logs -n argocd deployment/argocd-repo-server
# Refresh repository connection
kubectl delete secret repo-launchpad -n argocd
kubectl delete secret repo-sturdy-adventure -n argocd
# Recreate secret (see Step 3 above)
# Restart ArgoCD components
@@ -349,34 +391,34 @@ kubectl rollout restart deployment argocd-application-controller -n argocd
#### Multiple Repository Setup
For the three-repository pattern (launchpad, forte-helm, helm-prod-values):
For the three-repository pattern (sturdy-adventure, forte-helm, helm-values):
```bash
# 1. launchpad (main config repo)
ssh-keygen -t ed25519 -C "argocd-launchpad" -f key-sturdy -N ""
# Add key-sturdy.pub to: https://git.forteapps.net/Forte/launchpad/settings/keys
# 1. sturdy-adventure (main config repo)
ssh-keygen -t ed25519 -C "argocd-sturdy-adventure" -f key-sturdy -N ""
# Add key-sturdy.pub to: https://github.com/fortedigital/sturdy-adventure/settings/keys
# 2. helm-prod-values (private values repo)
ssh-keygen -t ed25519 -C "argocd-helm-prod-values" -f key-helm-prod-values -N ""
# Add key-helm-prod-values.pub to: https://git.forteapps.net/Forte/helm-prod-values/settings/keys
# 2. helm-values (private values repo)
ssh-keygen -t ed25519 -C "argocd-helm-values" -f key-helm-values -N ""
# Add key-helm-values.pub to: https://github.com/fortedigital/helm-values/settings/keys
# 3. forte-helm (private helm charts repo)
# 3. forte-helm is public - no key needed (use HTTPS)
# Create secrets
kubectl create secret generic repo-launchpad \
kubectl create secret generic repo-sturdy-adventure \
--from-file=sshPrivateKey=key-sturdy \
--namespace=argocd --dry-run=client -o yaml | \
kubectl label --local -f - argocd.argoproj.io/secret-type=repository --dry-run=client -o yaml | \
kubectl apply -f -
kubectl create secret generic repo-helm-prod-values \
--from-file=sshPrivateKey=key-helm-prod-values \
kubectl create secret generic repo-helm-values \
--from-file=sshPrivateKey=key-helm-values \
--namespace=argocd --dry-run=client -o yaml | \
kubectl label --local -f - argocd.argoproj.io/secret-type=repository --dry-run=client -o yaml | \
kubectl apply -f -
# Clean up keys
shred -u key-sturdy key-helm-prod-values
shred -u key-sturdy key-helm-values
```
#### Converting HTTPS to SSH
@@ -388,12 +430,12 @@ If you're currently using HTTPS and want to switch to SSH:
# 2. Update all Application manifests
# Change from:
# repoURL: https://git.forteapps.net/Forte/launchpad
# repoURL: https://github.com/fortedigital/sturdy-adventure.git
# To:
# repoURL: ssh://git@git.forteapps.net:2222/Forte/launchpad.git
# repoURL: git@github.com:fortedigital/sturdy-adventure.git
# 3. Update and commit
find . -name "*.yaml" -type f -exec sed -i 's|https://git.forteapps.net/Forte/|git@git.forteapps.net:Forte/|g' {} +
find . -name "*.yaml" -type f -exec sed -i 's|https://github.com/fortedigital/|git@github.com:fortedigital/|g' {} +
git add .
git commit -m "Switch from HTTPS to SSH for repository access"
@@ -497,7 +539,7 @@ spec:
See [Developer Guide](DEVELOPER-GUIDE.md#deploying-your-first-application) for detailed steps.
**Quick checklist:**
- [ ] Create `helm-prod-values/myapp/values.yaml`
- [ ] Create `helm-values/myapp/values.yaml`
- [ ] Create `apps/myapp.yaml` in config repo
- [ ] Create SealedSecret if needed
- [ ] Commit and push changes
@@ -562,7 +604,7 @@ kubectl scale deployment myapp -n myapp --replicas=3
#### GitOps Scaling
Update `helm-prod-values/myapp/values.yaml`:
Update `helm-values/myapp/values.yaml`:
```yaml
app:
@@ -576,7 +618,7 @@ Commit and push - ArgoCD will sync.
Enable Horizontal Pod Autoscaler:
```yaml
# In helm-prod-values/myapp/values.yaml
# In helm-values/myapp/values.yaml
app:
hpa:
enabled: true
@@ -625,7 +667,7 @@ kubectl rollout undo deployment myapp -n myapp
#### Option 3: Change Image Tag
```bash
# Edit helm-prod-values
# Edit helm-values
cd ~/dev/k8s/helm-prod-values
vim myapp/values.yaml
@@ -645,7 +687,7 @@ git push
#### Update Resource Limits
```yaml
# In helm-prod-values/myapp/values.yaml
# In helm-values/myapp/values.yaml
app:
resources:
requests:
@@ -659,7 +701,7 @@ app:
#### Enable Database
```yaml
# In helm-prod-values/myapp/values.yaml
# In helm-values/myapp/values.yaml
db:
enabled: true
persistence:
@@ -957,33 +999,6 @@ curl -G -s 'http://localhost:3100/loki/api/v1/query_range' \
--data-urlencode 'start=1h' | jq
```
### Tempo Traces
```bash
# Port forward to Tempo query API
kubectl port-forward -n monitoring svc/tempo 3200:3200
# Access: http://localhost:3200
```
**Query traces via Grafana:**
1. Open Grafana → Explore
2. Select Tempo datasource
3. Use TraceQL or search by service name
**Verify Traefik is sending traces:**
```bash
# Check Traefik logs for OTLP export errors
kubectl logs -n traefik-system -l app.kubernetes.io/name=traefik | grep -i "traces export"
# Check Tempo is receiving data
kubectl logs -n monitoring -l app.kubernetes.io/name=tempo | grep "receiver"
```
**Trace-to-log correlation:**
- Click a trace span in Grafana → linked Loki logs appear (by namespace, pod, container)
- Trace-to-metrics links to Prometheus by service name
### Fluent-Bit Log Shipping
Verify Fluent-Bit is shipping logs:
@@ -1264,21 +1279,13 @@ spec:
### Backup Strategy
**Current State**: Gitea daily backups to S3-compatible storage
**Current State**: No automated backups
**What Is Backed Up**:
- ✅ Gitea repositories + database: Daily CronJob (`cluster-resources/gitea-backup-cronjob.yaml`) uploads to S3-compatible storage with 7-day retention
- ✅ Git repositories: Full cluster config recoverable from Git
- ⚠️ Secrets: Sealed secrets in Git; unseal keys need safekeeping
**What Is NOT Backed Up**:
- ❌ Cluster state (recreate via GitOps)
- ❌ Other persistent volumes (Prometheus, Loki, Tempo data)
**Per-cloud backup scripts** (manual restore helpers):
- UpCloud/AWS: `scripts/gitea-backup.sh` / `scripts/gitea-backup-eks.sh` (MinIO CLI, S3-compatible)
- Azure: `scripts/gitea-backup-aks.sh` (Azure CLI + Blob Storage)
- GCP: `scripts/gitea-backup-gke.sh` (gsutil + GCS)
**What Needs Backup**:
- ❌ Cluster state (not backed up - recreate via GitOps)
- ❌ Persistent volumes (currently not critical)
- ✅ Git repositories (GitHub provides backup)
- ⚠️ Secrets (sealed secrets in Git, unseal keys need safekeeping)
### Cluster Rebuild
@@ -1363,13 +1370,13 @@ kubectl get deployment argocd-server -n argocd \
-o jsonpath='{.spec.template.spec.containers[0].image}'
# Update version in values
vim infra/values/base/argocd-values.yaml
vim infra/values/argocd-values.yaml
# Or upgrade via Helm directly
helm upgrade argocd argo-cd \
--repo https://argoproj.github.io/argo-helm \
--namespace argocd \
--values infra/values/base/argocd-values.yaml \
--values infra/values/argocd-values.yaml \
--version 6.0.0 # New version
# Verify
@@ -1380,9 +1387,6 @@ kubectl get pods -n argocd
```bash
# UpCloud: Upgrade via control panel or CLI
# AWS EKS: eksctl upgrade cluster / AWS Console
# Azure AKS: az aks upgrade / Azure Portal
# GCP GKE: gcloud container clusters upgrade / Cloud Console
# After upgrade, verify cluster
kubectl version
@@ -1468,8 +1472,8 @@ kubectl top pods --all-namespaces --sort-by=cpu
Example: Adding Redis
```bash
# 1. Create application manifest in base/
cat > infra/base/redis-application.yaml <<EOF
# 1. Create application manifest
cat > infra/redis-application.yaml <<EOF
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
@@ -1479,17 +1483,15 @@ metadata:
argocd.argoproj.io/sync-wave: "1"
spec:
project: default
sources:
- repoURL: https://charts.bitnami.com/bitnami
source:
repoURL: https://charts.bitnami.com/bitnami
chart: redis
targetRevision: 18.0.0
helm:
releaseName: redis
valueFiles:
- \$values/infra/values/base/redis-values.yaml
- repoURL: ssh://git@git.forteapps.net:2222/Forte/launchpad.git
targetRevision: HEAD
ref: values
values: |
auth:
enabled: true
password: changeme
destination:
server: https://kubernetes.default.svc
namespace: redis
@@ -1501,54 +1503,30 @@ spec:
- CreateNamespace=true
EOF
# 2. Add to base kustomization
# Edit infra/base/kustomization.yaml and add: - redis-application.yaml
# 3. Create base values file
cat > infra/values/base/redis-values.yaml <<EOF
auth:
enabled: true
EOF
# 4. Commit and push
git add infra/base/redis-application.yaml infra/values/base/redis-values.yaml infra/base/kustomization.yaml
# 2. Commit and push
git add infra/redis-application.yaml
git commit -m "Add Redis infrastructure component"
git push
# 5. ArgoCD will auto-sync within 60 seconds
# 3. ArgoCD will auto-sync within 60 seconds
```
### Multi-Cluster Setup
### Multi-Cluster Setup (Future)
The repository supports multiple clusters across multiple clouds via Kustomize overlays:
For multi-cluster deployments:
**Active clusters:**
- **upc-dev** (default): `infra/overlays/upc-dev/` — uses base Applications as-is
- **upc-prod**: `infra/overlays/upc-prod/` — patches value file paths from `upc-dev` to `upc-prod`
```yaml
# Different destinations per environment
# dev-cluster
destination:
server: https://dev.k8s.example.com
namespace: myapp
**Cloud-ready templates (fill in `clusters/*.yaml` before use):**
- **eks-dev** / **eks-prod**: AWS EKS with NLB, gp3 storage, AWS CUR pricing
- **aks-dev** / **aks-prod**: Azure AKS with Standard LB, managed-csi-premium storage
- **gke-dev** / **gke-prod**: GCP GKE with L4 LB, premium-rwo storage
Each cluster has its own:
- Root app-of-apps: `_app-of-apps-{cluster}.yaml`
- Cluster config: `clusters/{cluster}.yaml` (domain, trustedIPs, cloudProvider)
- Kustomize overlay: `infra/overlays/{cluster}/kustomization.yaml`
- Helm value overrides: `infra/values/{cluster}/` (traefik, gitea, opencost)
- Sealed secrets: `secrets/{cluster}/` (as needed)
- Apps overlay: `apps/overlays/{cluster}/`
Cloud-specific values handled per-cluster:
| Concern | UpCloud | AWS EKS | Azure AKS | GCP GKE |
|---------|---------|---------|-----------|---------|
| **Storage class** | `upcloud-block-storage-maxiops` | `gp3` | `managed-csi-premium` | `premium-rwo` |
| **Load balancer** | UpCloud LB + ProxyProtocol v2 | NLB + ProxyProtocol v2 | Standard LB + `externalTrafficPolicy: Local` | L4 passthrough NLB |
| **Cost monitoring** | Custom pricing | AWS CUR | Azure Billing API | GCP Cloud Billing |
| **Backup storage** | UpCloud S3-compat | AWS S3 (native) | Azure Blob Storage | GCS |
To add a new cluster, create a new overlay directory (e.g., `infra/overlays/eks-staging/`) with patches that swap the value file paths, and a matching `clusters/eks-staging.yaml`.
# prod-cluster
destination:
server: https://prod.k8s.example.com
namespace: myapp
```
### Blue-Green Deployments
@@ -1592,7 +1570,7 @@ git push
kubectl scale deployment myapp -n myapp --replicas=0
# Update Git
vim helm-prod-values/myapp/values.yaml
vim helm-values/myapp/values.yaml
# Set replicaCount: 0
git commit -am "Scale down myapp for maintenance"
git push
@@ -1665,7 +1643,7 @@ echo "Remember to delete: $SECRET_FILE"
- [ ] Application code repository created
- [ ] Dockerfile created and tested
- [ ] Gitea Actions workflow configured
- [ ] GitHub Actions workflow configured
- [ ] Helm values created in `helm-prod-values/`
- [ ] ArgoCD application manifest created in `apps/`
- [ ] Secrets created and sealed
@@ -1691,6 +1669,6 @@ echo "Remember to delete: $SECRET_FILE"
---
**Last Updated**: 2026-04-22
**Last Updated**: 2026-03-16
**Maintained By**: Platform Team
**Emergency Contact**: #platform-support on Slack

11
docs/README.md
View File

@@ -180,7 +180,7 @@ Reference for:
│
▼
┌──────────────────────────────────────────────────────────────┐
│ Kubernetes Clusters (UpCloud, AWS, Azure, GCP) │
│ Kubernetes Cluster (UpCloud) │
│ ┌──────────────────────────────────────────────────────┐ │
│ │ Infrastructure: Traefik, Cert-Manager, Kyverno │ │
│ ├──────────────────────────────────────────────────────┤ │
@@ -194,7 +194,7 @@ Reference for:
### Key Technologies
- **GitOps**: ArgoCD
- **Kubernetes**: Multi-cloud (UpCloud, AWS EKS, Azure AKS, GCP GKE)
- **Kubernetes**: UpCloud Managed Kubernetes
- **Ingress**: Traefik v2
- **Certificates**: Cert-Manager + Let's Encrypt
- **Policies**: Kyverno
@@ -299,16 +299,11 @@ docs/
## 🔄 Documentation Versions
**Current Version**: 1.0.0
**Last Updated**: 2026-04-22
**Last Updated**: 2026-03-16
**Maintained By**: Platform Team
### Changelog
- **v1.1.0 (2026-04-22)**: Multi-cloud support
- Cloud-agnostic base values (storage, LB, pricing moved to per-cluster overlays)
- Added AWS EKS, Azure AKS, GCP GKE configurations
- Per-cloud backup scripts
- Updated all documentation
- **v1.0.0 (2026-03-16)**: Initial comprehensive documentation release
- GitOps Architecture guide
- Developer Onboarding guide

698
docs/REFERENCE.md
View File

@@ -9,7 +9,6 @@
- [Kyverno Policies](#kyverno-policies)
- [Configuration Reference](#configuration-reference)
- [API Endpoints](#api-endpoints)
- [Cloud Overlay Pattern](#cloud-overlay-pattern)
- [Glossary](#glossary)
---
@@ -20,10 +19,9 @@
| Component | Value |
|-----------|-------|
| **Provider** | Multi-cloud (UpCloud, AWS EKS, Azure AKS, GCP GKE) |
| **Environment** | Dev + Production per cloud |
| **Active clusters** | UpCloud (upc-dev, upc-prod) |
| **Cloud-ready templates** | EKS, AKS, GKE (dev + prod each) |
| **Provider** | UpCloud Managed Kubernetes |
| **Environment** | Production (internal use) |
| **Cluster Count** | Single cluster |
| **GitOps Tool** | ArgoCD |
| **Ingress Controller** | Traefik v2 |
| **Certificate Management** | Cert-Manager + Let's Encrypt |
@@ -31,9 +29,7 @@
| **Secret Management** | Sealed Secrets (Bitnami) |
| **Monitoring** | Prometheus + Grafana |
| **Logging** | Loki + Fluent-Bit |
| **Tracing** | Tempo (OTLP) |
| **Container Scanning** | Trivy |
| **Version Control** | Gitea |
### Network Architecture
@@ -44,7 +40,7 @@ Internet
[DNS: *.forteapps.net]
│
▼
[Cloud Load Balancer]
[UpCloud LoadBalancer]
│
▼
[Traefik Ingress Controller]
@@ -64,17 +60,16 @@ Internet
## Repository Reference
### Config Repository: `launchpad`
### Config Repository: `sturdy-adventure`
**URL**: `https://git.forteapps.net/Forte/launchpad`
**URL**: `https://github.com/fortedigital/sturdy-adventure.git`
#### Directory Structure
```
launchpad/
sturdy-adventure/
├── bootstrap.sh # Cluster initialization script
├── _app-of-apps-upc-dev.yaml # Root ArgoCD Application (upc-dev)
├── _app-of-apps-upc-prod.yaml # Root ArgoCD Application (upc-prod)
├── _app-of-apps.yaml # Root ArgoCD Application
│
├── infra/ # Infrastructure applications
│ ├── cluster-resources-application.yaml
@@ -86,42 +81,16 @@ launchpad/
│ ├── prometheus.yaml
│ ├── grafana.yaml
│ ├── loki.yaml
│ ├── tempo.yaml
│ ├── fluent-bit.yaml
│ ├── trivy.yaml
│ ├── gitea.yaml
│ ├── gitea-actions.yaml
│ ├── sealedsecrets.yaml
│ ├── secrets.yaml
│ ├── renovate.yaml
│ ├── base/ # ArgoCD Application manifests (Kustomize base)
│ │ ├── gitea.yaml
│ │ ├── opencost.yaml
│ │ ├── traefik-application.yaml
│ │ ├── keycloak.yaml
│ │ ├── grafana.yaml
│ │ └── ...
│ ├── overlays/
│ │ └── upc-prod/
│ │ └── kustomization.yaml # Patches upc-dev → upc-prod valueFile paths
│ └── values/
│ ├── base/ # Cloud-agnostic Helm values
│ │ ├── gitea-values.yaml
│ │ ├── opencost-values.yaml
│ │ ├── prometheus-values.yaml
│ │ └── ...
│ ├── upc-dev/ # UpCloud dev overlay values
│ │ ├── traefik-values.yaml
│ │ ├── keycloak-values.yaml
│ │ ├── grafana-values.yaml
│ │ ├── gitea-values.yaml
│ │ └── opencost-values.yaml
│ └── upc-prod/ # UpCloud prod overlay values
│ ├── traefik-values.yaml
│ ├── keycloak-values.yaml
│ ├── argocd-values.yaml
│ ├── prometheus-values.yaml
│ ├── grafana-values.yaml
│ ├── gitea-values.yaml
│ └── opencost-values.yaml
│ ├── loki-values.yaml
│ └── fluent-bit-values.yaml
│
├── apps/ # Business applications
│ ├── mcp10x.yaml
@@ -135,6 +104,7 @@ launchpad/
│ ├── letsencrypt-issuer.yaml
│ ├── kyverno-config.yaml
│ ├── argocd-notifications-secret-sealed.yaml
│ ├── snothub-repo-credentials-sealed.yaml
│ ├── forte10x-repo-credentials-sealed.yaml
│ ├── mcp10x-repo-credentials-sealed.yaml
│ └── policies/
@@ -144,43 +114,13 @@ launchpad/
│ ├── replicaset-cleaner.yaml
│ ├── default-ns-blocker.yaml
│ ├── secret-cloner.yaml
│ ├── keycloak-client-cloner.yaml
│ └── auth-sidecar-injector.yaml
│
├── secrets/ # Application secrets (sealed)
│ ├── base/ # All SealedSecrets (shared across clouds)
│ │ ├── kustomization.yaml
│ │ ├── argocd-forte-helm-secret-sealed.yaml
│ │ ├── argocd-mcp-credentials.yaml
│ │ ├── argocdmcp-auth-oidc-sealed.yaml
│ │ ├── dot-ai-secrets.yaml
│ │ ├── forte10x-app-credentials-sealed.yaml
│ │ ├── gitea-backup-s3-sealed.yaml
│ │ ├── gitea-credentials-sealed.yaml
│ │ ├── gitea-runner-token-sealed.yaml
│ │ ├── gitea-smtp-secret-sealed.yaml
│ │ ├── keycloak-credentials-sealed.yaml
│ │ ├── musicman-auth-oidc-sealed.yaml
│ │ ├── musicman-credentials.yaml
│ │ └── renovate-env-sealed.yaml
│ └── overlays/ # Per-cloud overlays (reference base)
│ ├── aks-dev/kustomization.yaml
│ ├── aks-prod/kustomization.yaml
│ ├── eks-dev/kustomization.yaml
│ ├── eks-prod/kustomization.yaml
│ ├── gke-dev/kustomization.yaml
│ ├── gke-prod/kustomization.yaml
│ ├── upc-dev/kustomization.yaml
│ └── upc-prod/kustomization.yaml
│
├── scripts/ # Operational helper scripts
│ ├── gitea-backup.sh # S3 backup helper (list/download)
│ ├── gitea-restore.sh
│ └── backup/ # Per-cloud backup reference scripts
│ ├── s3-minio.sh # S3-compatible (UpCloud, MinIO, Wasabi)
│ ├── aws-s3.sh # Native AWS S3
│ ├── azure-blob.sh # Azure Blob Storage
│ └── gcp-gcs.sh # GCP Cloud Storage
│ ├── argocd-mcp-credentials.yaml
│ ├── dot-ai-secrets.yaml
│ ├── mcp10x-credentials-sealed.yaml
│ └── musicman-credentials.yaml
│
├── private/ # Local-only (Git-ignored)
│ ├── *.yaml
@@ -204,15 +144,15 @@ ArgoCd() {
helm upgrade --install argocd argo-cd \
--repo https://argoproj.github.io/argo-helm \
--namespace argocd --create-namespace \
--values infra/values/base/argocd-values.yaml \
--values infra/values/argocd-values.yaml \
--set notifications.context.clusterName="$CLUSTER_NAME" \
--timeout 60s --atomic
kubectl apply -f _app-of-apps-upc-dev.yaml -n argocd # or _app-of-apps-upc-prod.yaml
kubectl apply -f _app-of-apps.yaml -n argocd
}
```
**`_app-of-apps-upc-dev.yaml`** / **`_app-of-apps-upc-prod.yaml`**
**`_app-of-apps.yaml`**
```yaml
apiVersion: argoproj.io/v1alpha1
kind: Application
@@ -222,7 +162,7 @@ metadata:
spec:
project: default
source:
repoURL: ssh://git@git.forteapps.net:2222/Forte/launchpad.git
repoURL: git@github.com:fortedigital/sturdy-adventure.git
path: infra
destination:
server: https://kubernetes.default.svc
@@ -237,7 +177,7 @@ spec:
### Helm Charts Repository: `forte-helm`
**URL**: `https://git.forteapps.net/Forte/forte-helm`
**URL**: `https://github.com/snothub/forte-helm`
#### Chart: `forteapp`
@@ -384,18 +324,20 @@ configmap: [] # Application ConfigMap key-value pairs
---
### Helm Values Repository: `helm-prod-values`
### Helm Values Repository: `helm-values`
**URL**: `https://git.forteapps.net/Forte/helm-prod-values.git`
**URL**: `https://github.com/fortedigital/helm-values.git`
#### Structure
```
helm-prod-values/
helm-values/
├── mcp10x/
│ └── values.yaml
├── musicman/
│ └── values.yaml
├── mcpcoder/
│ └── values.yaml
└── argocd-mcp/
└── values.yaml
```
@@ -571,14 +513,14 @@ spec:
# Multi-source configuration
sources:
- repoURL: https://git.forteapps.net/Forte/forte-helm
- repoURL: https://github.com/snothub/forte-helm
path: forteapp
targetRevision: HEAD
helm:
valueFiles:
- $values/<app-name>/values.yaml
- repoURL: git@github.com:fortedigital/helm-prod-values.git
- repoURL: git@github.com:fortedigital/helm-values.git
targetRevision: HEAD
ref: values
@@ -649,15 +591,6 @@ retry:
4. 40 seconds
5. 80 seconds (capped at 3 minutes)
### Global Settings (`argocd-cm`)
| Setting | Value | Purpose |
|---------|-------|---------|
| `application.resourceTrackingMethod` | `annotation` | Track resources via annotations |
| `timeout.reconciliation` | `60s` | Reconciliation interval |
| `admin.enabled` | `true` | Enable admin account |
| `git.submodule.enabled` | `false` | Disable git submodule checkout — submodules are not needed for manifest generation |
---
## Infrastructure Components
@@ -670,7 +603,7 @@ retry:
**Configuration**:
```yaml
# infra/base/traefik-application.yaml
# infra/traefik-application.yaml
replicas: 2
service:
@@ -733,10 +666,6 @@ spec:
**Chart**: `sealed-secrets/sealed-secrets-controller`
**Namespace**: `kube-system`
**Directory Structure**: `secrets/base/` contains all SealedSecrets with a `kustomization.yaml`. Per-cloud overlays in `secrets/overlays/<cloud>/` reference the base via Kustomize. The ArgoCD `secrets` Application points to the active overlay (e.g., `secrets/overlays/upc-dev`), and `infra/overlays/upc-prod` patches the path to `secrets/overlays/upc-prod`.
To add cloud-specific secrets, create a new SealedSecret in the overlay directory and add it to the overlay's `kustomization.yaml`.
**Public Certificate**:
```bash
kubeseal --fetch-cert \
@@ -775,7 +704,6 @@ kubeStateMetrics:
**Datasources**:
- Prometheus
- Loki
- Tempo
### Loki
@@ -793,45 +721,6 @@ promtail:
enabled: false # Using Fluent-Bit instead
```
### Tempo
**Chart**: `grafana/tempo`
**Version**: 1.24.4
**Namespace**: `monitoring`
**Purpose**: Distributed tracing backend receiving OTLP traces from Traefik and other instrumented services.
**Configuration**:
```yaml
tempo:
storage:
trace:
backend: local
local:
path: /var/tempo/traces
receivers:
otlp:
protocols:
grpc:
endpoint: "0.0.0.0:4317"
http:
endpoint: "0.0.0.0:4318"
persistence:
enabled: true
size: 10Gi
```
**Endpoints**:
- gRPC OTLP receiver: `:4317`
- HTTP OTLP receiver: `:4318`
- Query API: `:3200`
**Grafana Integration**:
- Trace-to-logs correlation with Loki (by namespace, pod, container)
- Trace-to-metrics correlation with Prometheus (by service name)
- Service graph and node graph visualization
### Fluent-Bit
**Chart**: `fluent/fluent-bit`
@@ -839,377 +728,6 @@ persistence:
**Output**: Loki
### Gitea
**Chart**: `gitea/gitea`
**Version**: 12.5.0 (app v1.25.4)
**Namespace**: `gitea`
**Purpose**: Self-hosted Git repository hosting with pull requests, issues, CI/CD (Gitea Actions), container registry, and package registry.
**Configuration**:
```yaml
# infra/base/gitea.yaml + infra/values/base/gitea-values.yaml
ingress:
host: git.forteapps.net
tls: cert-manager (letsencrypt-prod)
gitea:
admin:
existingSecret: gitea-credentials
config:
service:
DISABLE_REGISTRATION: true
ALLOW_ONLY_EXTERNAL_REGISTRATION: true
actions:
ENABLED: true
packages:
ENABLED: true
metrics:
ENABLED: true
postgresql:
enabled: true
persistence: 8Gi (upcloud-block-storage-maxiops)
```
**Authentication**: Keycloak OIDC via `forte` realm (client ID: `gitea`). Protocol mapper: `email_verified` hardcoded claim (`true`, boolean) on ID token, Access token, and Userinfo.
**External User Sync**: Disabled (`cron.sync_external_users.ENABLED: false`). This Gitea cron job is designed for LDAP and deactivates OIDC-only users because it cannot enumerate them — causing "Sign-in prohibited" errors after the sync runs.
**Email Notifications**: Enabled (`ENABLE_NOTIFY_MAIL: true`). SMTP credentials injected via `gitea-smtp-secret` using `additionalConfigFromEnvs` with `GITEA__mailer__USER` / `GITEA__mailer__PASSWD` environment variables.
**Auto-Watch**: Disabled (`AUTO_WATCH_ON_CHANGES: false`, `AUTO_WATCH_NEW_REPOS: false`). Prevents contributors from being auto-subscribed to repo notifications on push, reducing email noise from CI bots (e.g., ai-review PR comments). Users who were already watching before this change need to manually unwatch or switch to "Only participating".
**Endpoints**:
- Web UI: `https://git.forteapps.net`
- SSH: port 22 (ClusterIP)
- Metrics: `/metrics` (Prometheus scrape)
**Secrets**:
- `gitea-credentials` (SealedSecret) — admin password
- `gitea-oidc-credentials` (registrar-managed) — OIDC client ID + secret
- `gitea-smtp-secret` (SealedSecret) — SMTP username + password
### Gitea Actions Runners
**Chart**: `actions` (from `https://dl.gitea.com/charts`)
**Namespace**: `gitea`
**Sync Wave**: 2 (deploys after Gitea)
**Purpose**: Act runners execute Gitea Actions CI/CD workflows. Deployed as a StatefulSet with a Docker-in-Docker sidecar for container-based job execution.
**Configuration**:
```yaml
# infra/base/gitea-actions.yaml + infra/values/base/gitea-actions-values.yaml
replicaCount: 3
runner:
labels:
- "ubuntu-latest:docker://node:20-bookworm"
- "ubuntu-22.04:docker://node:20-bookworm"
existingSecret: gitea-runner-token
gitea:
instance:
url: http://gitea-http.gitea.svc.cluster.local:3000
dind:
enabled: true # Docker-in-Docker sidecar (privileged)
```
**Resources**:
| Container | CPU Request | Memory Request | CPU Limit | Memory Limit |
|-----------|-------------|----------------|-----------|--------------|
| Runner | 250m | 256Mi | 1 | 1Gi |
| DinD sidecar | 250m | 256Mi | 1 | 1Gi |
**Secrets**: `gitea-runner-token` (SealedSecret) containing `token` (instance-level runner registration token from `/admin/runners`)
**Setup Steps**:
1. Get runner registration token from Gitea admin panel (`/admin/runners`)
2. Fill in `private/gitea-runner-token.yaml` with the token
3. Seal: `kubeseal --format yaml < private/gitea-runner-token.yaml > secrets/gitea-runner-token-sealed.yaml`
4. Commit and push — ArgoCD deploys runners automatically
**Verification**:
- `kubectl get statefulset -n gitea` — 3/3 runners ready
- Gitea admin panel (`/admin/runners`) — runners show as Online
- Create test workflow in `.gitea/workflows/test.yml` — job executes
### AI Code Review (ai-review)
**Type**: Gitea Actions workflow (`.gitea/workflows/ai-review.yaml`)
**Trigger**: `pull_request` events (`opened`, `synchronize`)
**Runner**: `ubuntu-latest` (container: `nikitafilonov/ai-review:latest`)
**Purpose**: Automated AI-powered code review on pull requests using Claude (Anthropic). Posts inline comments on changed lines and a PR summary comment highlighting infrastructure impact.
**Architecture**:
- Uses [xai-review](https://github.com/nicktechnologies/xai-review) Docker image
- Shared configuration and prompts live in the `shared-prompts` Git submodule (→ `Forte/ai-review-prompts`)
- Review mode: `ONLY_ADDED_WITH_CONTEXT` — reviews only new/changed lines plus surrounding context (token-efficient)
- Agent mode: disabled (one-shot review, no multi-turn reasoning)
- LLM: Claude Sonnet (`claude-sonnet-4-20250514`)
**Shared Prompts Structure** (submodule: `Forte/ai-review-prompts`):
```
shared-prompts/
base/
security.md # org-wide security rules (all profiles)
iac/
.ai-review.yaml # IaC/GitOps profile config
inline.md # inline review prompt
summary.md # PR summary prompt
# future profiles: backend/, frontend/, etc.
```
**Configuration** (`shared-prompts/iac/.ai-review.yaml`):
```yaml
llm:
provider: CLAUDE
model: claude-sonnet-4-20250514
vcs:
provider: GITEA
review:
mode: ONLY_ADDED_WITH_CONTEXT
agent:
enabled: false
prompt:
inline_prompt_files: # concatenated in order
- ./shared-prompts/base/security.md
- ./shared-prompts/iac/inline.md
summary_prompt_files:
- ./shared-prompts/iac/summary.md
ignore:
- "*.sealed.yaml"
- "*.lock"
- "docs/**"
```
**Custom Prompts** (IaC profile):
- `shared-prompts/base/security.md` — org-wide security rules, concatenated before every inline review prompt
- `shared-prompts/iac/inline.md` — IaC-specific inline review (YAML, Helm, K8s manifests, shell scripts), max 7 comments
- `shared-prompts/iac/summary.md` — PR summary: affected services/namespaces, infrastructure impact, security flags
**Prompt composition**: ai-review does not support Jinja includes. Instead, list multiple files under `inline_prompt_files` / `summary_prompt_files` — they are concatenated in order with double newlines.
**Adding a new profile**: Create a new directory (e.g., `backend/`) with its own `.ai-review.yaml`, `inline.md`, and `summary.md`. The `inline_prompt_files` list should include `base/security.md` first, then the profile-specific prompt. Reference it in the consuming repo's workflow: `AI_REVIEW_CONFIG_FILE_YAML=./shared-prompts/backend/.ai-review.yaml`
**Required Secrets** (configure in Gitea repo or org settings):
| Secret | Purpose |
|--------|---------|
| `ANTHROPIC_API_KEY` | Claude API key (from Anthropic console) |
| `AI_REVIEW_TOKEN` | Gitea API token with `write:repository` + `read:repository` scopes (use a bot/service account) |
**Setup Steps**:
1. Create a Gitea bot/service account and generate an API token with `write:repository` + `read:repository` scopes
2. Add `AI_REVIEW_TOKEN` secret in Gitea repo settings → Actions → Secrets
3. Add `ANTHROPIC_API_KEY` secret with your Anthropic API key
4. Ensure the `shared-prompts` submodule is initialized (`git submodule update --init`)
5. Push the workflow file — it triggers automatically on PR creation/update
**Verification**:
- Open a PR with infrastructure changes → workflow runs → inline comments + summary appear
- Check Gitea Actions tab for workflow run status and logs
- Monitor Anthropic usage dashboard for token consumption
### Keycloak Client Registrar
**Type**: CronJob (deployed via Keycloak Helm chart `extraDeploy`)
**Namespace**: `keycloak`
**Schedule**: `*/2 * * * *` (every 2 minutes)
**Purpose**: Handles two responsibilities:
1. **Legacy sync** — extracts secrets from Keycloak clients with `k8s.secret.sync: "true"` attribute (same as former PostSync syncer)
2. **Self-service registration** — processes config Secrets (cloned by Kyverno) to register new OIDC clients and sync their credentials
**How It Works**:
*Legacy path (existing clients like Gitea):*
1. Authenticates to Keycloak Admin API using admin credentials from `keycloak-credentials` secret
2. Queries all clients in the `forte` realm
3. Filters clients with `k8s.secret.sync: "true"` attribute
4. For each matching client, retrieves the auto-generated secret via Keycloak Admin API
5. Creates/updates a K8s Secret in the target namespace (from `k8s.secret.namespace` attribute)
6. Always writes a central copy to the `secrets` namespace
*Self-service path (new clients):*
1. Lists Secrets in `keycloak` namespace with label `keycloak.forteapps.net/client-config=true`
2. For each config Secret, parses `client.json` and computes a config hash
3. Skips if hash matches annotation and credential Secret already exists
4. Creates or updates the Keycloak client via Admin API
5. Fetches the generated client secret
6. Upserts credential Secret in target namespace + central `secrets` namespace
7. Annotates config Secret with sync status, config hash, and timestamp
**Resources**:
- `ServiceAccount`: `keycloak-client-registrar` (namespace: `keycloak`)
- `ClusterRole`: `keycloak-client-registrar` (secrets: get/list/create/update/patch; namespaces: get/list)
- `ClusterRoleBinding`: `keycloak-client-registrar`
- `CronJob`: `keycloak-client-registrar`
**Kyverno Policy**: `keycloak-client-config-cloner` — clones labeled Secrets from app namespaces to `keycloak` namespace (see [Kyverno Policies](#kyverno-policies))
**Legacy Client Attributes** (set in `forte-realm.json`):
| Attribute | Required | Default | Description |
|-----------|----------|---------|-------------|
| `k8s.secret.sync` | Yes | — | Set to `"true"` to enable syncing |
| `k8s.secret.namespace` | Yes | — | Target K8s namespace |
| `k8s.secret.name` | Yes | — | Name of the K8s Secret |
| `k8s.secret.client-id-key` | No | `client-id` | Field name for client ID in the Secret |
| `k8s.secret.client-secret-key` | No | `client-secret` | Field name for client secret in the Secret |
**Self-Service Config Secret Schema**:
```yaml
apiVersion: v1
kind: Secret
metadata:
name: keycloak-client-<app>
namespace: <app-namespace>
labels:
keycloak.forteapps.net/client-config: "true"
stringData:
client.json: |
{
"clientId": "<app>",
"name": "<App Name>",
"redirectUris": ["https://<app>.forteapps.net/*"],
"webOrigins": ["https://<app>.forteapps.net"],
"defaultClientScopes": ["openid", "email", "profile"],
"protocolMappers": [],
"secret": {
"namespace": "<app-namespace>",
"name": "<app>-oidc-credentials",
"keys": { "clientId": "client-id", "clientSecret": "client-secret" }
}
}
```
**Created Credential Secret Format**:
```yaml
apiVersion: v1
kind: Secret
metadata:
name: <target-name>
namespace: <target-namespace>
labels:
app.kubernetes.io/managed-by: keycloak-client-registrar
type: Opaque
data:
<client-id-key>: <base64-encoded client ID>
<client-secret-key>: <base64-encoded client secret>
```
**Config Secret Annotations** (set by registrar):
| Annotation | Description |
|-----------|-------------|
| `keycloak.forteapps.net/config-hash` | SHA-256 hash of client.json for change detection |
| `keycloak.forteapps.net/sync-status` | `synced` or `error` |
| `keycloak.forteapps.net/last-sync` | ISO 8601 timestamp of last successful sync |
**Verification**:
```bash
# Check CronJob status
kubectl get cronjobs -n keycloak
# View latest registrar logs
kubectl logs -n keycloak job/$(kubectl get jobs -n keycloak --sort-by=.metadata.creationTimestamp -o jsonpath='{.items[-1].metadata.name}')
# Verify created secret
kubectl get secret <name> -n <namespace> -o yaml
# Check config Secret annotations (self-service)
kubectl get secret keycloak-client-<app> -n keycloak -o jsonpath='{.metadata.annotations}'
```
**See**: [Developer Guide - Adding a New Keycloak Client](DEVELOPER-GUIDE.md#adding-a-new-keycloak-client)
### Karpor
**Chart**: `karpor` from `https://kusionstack.github.io/charts`
**Version**: 0.7.6 (app v0.6.4)
**Namespace**: `karpor`
**Sync Wave**: 1
**Purpose**: Kubernetes visualization and intelligence tool. Provides cross-cluster resource search, compliance checking, and topology visualization. Gives platform engineers a unified view of all cluster resources and their relationships.
**Architecture** (4 components):
- **Server** — main Karpor API/UI (port 7443)
- **Syncer** — syncs cluster state into the search index
- **ElasticSearch** — search backend for resource indexing
- **etcd** — persistent key-value store (10Gi PVC)
**Configuration** (`infra/values/base/karpor-values.yaml`):
- `namespaceEnabled: false` — ArgoCD manages namespace creation
- Default resource limits tuned for small clusters
- ElasticSearch: 2 CPU / 4Gi memory (the heaviest component)
- AI features available but not enabled (requires `server.ai.authToken` + backend config)
**Access**: Port-forward to reach the UI:
```bash
kubectl port-forward svc/karpor-release-server -n karpor 7443:7443
# Open https://localhost:7443
```
### Renovate
**Chart**: `renovate` (OCI: `ghcr.io/renovatebot/charts`)
**Version**: 46.109.0 (app v43.113.0)
**Namespace**: `renovate`
**Sync Wave**: 2
**Purpose**: Automated dependency update bot. Runs as a CronJob that scans Gitea repositories for outdated dependencies and creates pull requests with updates.
**Configuration**:
```yaml
# infra/base/renovate.yaml + infra/values/base/renovate-values.yaml
cronjob:
schedule: "@daily"
concurrencyPolicy: Forbid
renovate:
config:
platform: gitea
endpoint: https://git.forteapps.net
autodiscover: true
gitAuthor: "Renovate Bot <renovate@forteapps.net>"
packageRules:
- matchRepositories: ["**/10x"]
assignees: ["edvard.unsvag"]
reviewers: ["edvard.unsvag"]
- matchRepositories: ["**/auth-sidecar"]
assignees: ["danijel.simeunovic"]
reviewers: ["danijel.simeunovic"]
- matchRepositories: ["**/forte-helm"]
assignees: ["danijel.simeunovic"]
reviewers: ["danijel.simeunovic"]
resources:
requests: { cpu: 500m, memory: 1Gi }
limits: { cpu: "2", memory: 4Gi }
```
**Note**: Assignees and reviewers are only applied at PR creation time. Existing PRs must be closed and recreated for new assignment rules to take effect.
**Secrets**: `renovate-env` (SealedSecret in `secrets` namespace, cloned by Kyverno) containing:
- `RENOVATE_TOKEN` — Gitea PAT with repo write + issue write permissions
- `RENOVATE_GITHUB_COM_TOKEN` — GitHub PAT (public_repo read-only) for changelog fetching
**Setup Steps**:
1. Fill in `private/renovate-env.yaml` with tokens
2. Seal: `kubeseal --format yaml < private/renovate-env.yaml > secrets/renovate-env-sealed.yaml`
3. Commit and push — ArgoCD deploys the CronJob, Kyverno clones the secret
**Verification**:
- `kubectl get cronjob -n renovate` — CronJob exists
- `kubectl create job --from=cronjob/renovate renovate-test -n renovate` — manual trigger
- `kubectl logs -n renovate job/renovate-test` — check logs
---
## Kyverno Policies
@@ -1246,19 +764,6 @@ spec:
**Label Requirement**: Secrets must have `allowedToBeCloned: "true"`
### Keycloak Client Config Cloner
**File**: `cluster-resources/policies/keycloak-client-cloner.yaml`
**Purpose**: Clones Secrets labeled `keycloak.forteapps.net/client-config: "true"` from app namespaces to the `keycloak` namespace. This allows apps to declare their OIDC client configuration in their own namespace, which the [Keycloak Client Registrar](#keycloak-client-registrar) then processes.
**Trigger**: Any Secret with label `keycloak.forteapps.net/client-config: "true"` created outside the `keycloak` namespace.
**Behavior**:
- Generates a copy of the Secret in the `keycloak` namespace with the same name
- Adds source tracking annotations (`keycloak.forteapps.net/source-namespace`, `keycloak.forteapps.net/source-name`)
- `synchronize: true` — changes to the source Secret are reflected in the clone
### Default Namespace Blocker
**File**: `cluster-resources/policies/default-ns-blocker.yaml`
@@ -1348,7 +853,7 @@ policies.forteapps.io/auth-token-secret-name: "auth-tokens"
policies.forteapps.io/auth-upstream-url: "http://localhost:3000"
# Optional customization
policies.forteapps.io/auth-image: "ghcr.io/fortedigital/auth-sidecar"
policies.forteapps.io/auth-image: "ghcr.io/snothub/stunning-memory"
policies.forteapps.io/auth-image-version: "latest"
```
@@ -1364,7 +869,7 @@ policies.forteapps.io/auth-oidc-client-id: "myapp"
policies.forteapps.io/auth-oidc-callback-path: "/auth/callback"
policies.forteapps.io/auth-oidc-scopes: "openid,profile,email"
policies.forteapps.io/auth-upstream-url: "http://localhost:3000"
policies.forteapps.io/auth-image: "ghcr.io/fortedigital/auth-sidecar"
policies.forteapps.io/auth-image: "ghcr.io/snothub/stunning-memory"
policies.forteapps.io/auth-image-version: "latest"
```
@@ -1380,7 +885,7 @@ policies.forteapps.io/auth-mcp-authority: "https://auth.example.com"
policies.forteapps.io/auth-mcp-scopes: "read,write"
policies.forteapps.io/auth-upstream-url: "http://localhost:3000"
policies.forteapps.io/auth-log-level: "info"
policies.forteapps.io/auth-image: "ghcr.io/fortedigital/auth-sidecar"
policies.forteapps.io/auth-image: "ghcr.io/snothub/stunning-memory"
policies.forteapps.io/auth-image-version: "latest"
```
@@ -1389,7 +894,7 @@ policies.forteapps.io/auth-image-version: "latest"
**Token Mode**:
```yaml
name: authn
image: ghcr.io/fortedigital/auth-sidecar:latest
image: ghcr.io/snothub/stunning-memory:latest
ports:
- containerPort: 8080
name: auth
@@ -1424,7 +929,7 @@ securityContext:
**OIDC Mode**:
```yaml
name: authn
image: ghcr.io/fortedigital/auth-sidecar:latest
image: ghcr.io/snothub/stunning-memory:latest
ports:
- containerPort: 8080
name: auth
@@ -1471,7 +976,7 @@ securityContext:
**MCP Mode**:
```yaml
name: authn
image: ghcr.io/fortedigital/auth-sidecar:latest
image: ghcr.io/snothub/stunning-memory:latest
ports:
- containerPort: 8080
name: auth
@@ -1603,23 +1108,7 @@ Forward to Application (localhost:3000)
Application processes request
```
#### Forwarded Headers
After successful authentication, the sidecar injects user identity as HTTP headers before forwarding the request to the application container:
| Header | Description | Auth Modes |
|--------|-------------|------------|
| `X-Auth-User` | Username or display name | Token, OIDC, MCP |
| `X-Auth-Email` | User email address | OIDC |
| `X-Auth-Subject` | OIDC `sub` claim (stable user ID) | OIDC, MCP |
| `X-Auth-Groups` | Comma-separated group memberships | OIDC (if `groups` scope) |
| `X-Auth-Token` | The validated access token | All modes |
These headers are trustworthy because the auto-generated `NetworkPolicy` restricts pod ingress to the sidecar port only — external traffic cannot reach the application container directly, so headers cannot be spoofed.
Applications should read these headers to obtain authenticated user information (e.g. for display, authorisation decisions, or audit logging) instead of implementing their own authentication.
**See**: [Developer Guide - Accessing Authenticated User Information](DEVELOPER-GUIDE.md#accessing-authenticated-user-information) for code examples.
**See**: [Developer Guide - Enabling Authentication](DEVELOPER-GUIDE.md#enabling-authentication-for-applications) for usage examples.
---
@@ -1653,22 +1142,14 @@ Recommended resource allocation:
### Storage Classes
Storage classes are cloud-specific and configured in per-cluster value overrides (`infra/values/{cluster}/gitea-values.yaml`):
| Cloud | Storage Class | Driver |
|-------|--------------|--------|
| **UpCloud** | `upcloud-block-storage-maxiops` | UpCloud CSI |
| **AWS EKS** | `gp3` | EBS CSI |
| **Azure AKS** | `managed-csi-premium` | Azure Disk CSI |
| **GCP GKE** | `premium-rwo` | PD CSI |
Default storage class used: **UpCloud default** (varies by provider)
```yaml
# Example: base values omit storageClass (set in per-cluster overlay)
persistence:
enabled: true
storageClass: "" # Uses default
accessMode: ReadWriteOnce
size: 5Gi
# storageClass set by infra/values/{cluster}/gitea-values.yaml
```
---
@@ -1704,19 +1185,6 @@ GET /api/v1/query_range?query={promql}&start={time}&end={time}&step={duration}
GET /api/v1/label/__name__/values
```
### Tempo API
```
# Search traces
GET /api/search?q={traceql}
# Get trace by ID
GET /api/traces/{traceID}
# Service tag values
GET /api/v2/search/tag/resource.service.name/values
```
### Loki API
```
@@ -1732,88 +1200,6 @@ POST /loki/api/v1/push
---
## Cloud Overlay Pattern
### Overview
Cloud-specific configuration (StorageClass, LoadBalancer annotations, pricing models, etc.) lives in per-cloud overlay value files, **not** in `base/`. Adding a new cloud provider only requires a new overlay directory — no base changes.
### Supported Clouds
| Cloud | Dev overlay | Prod overlay | StorageClass | LB type |
|-------|-----------|-------------|-------------|---------|
| **UpCloud** | `upc-dev` | `upc-prod` | `upcloud-block-storage-maxiops` | UpCloud LB (proxy protocol v2) |
| **Azure AKS** | `aks-dev` | `aks-prod` | `managed-csi-premium` | Azure LB |
| **AWS EKS** | `eks-dev` | `eks-prod` | `gp3` | AWS NLB (proxy protocol) |
| **GCP GKE** | `gke-dev` | `gke-prod` | `premium-rwo` | GCP NEG |
Bootstrap any cluster with: `./bootstrap.sh <cluster>` (e.g., `./bootstrap.sh aks-dev`)
### How It Works
Each ArgoCD Application uses **multi-source Helm values** with two value files:
```yaml
# infra/base/gitea.yaml (example)
helm:
valueFiles:
- $values/infra/values/base/gitea-values.yaml # [0] cloud-agnostic
- $values/infra/values/upc-dev/gitea-values.yaml # [1] cloud-specific (default: upc-dev)
```
The `upc-prod` Kustomize overlay patches index `[1]` to swap the cloud-specific file:
```yaml
# infra/overlays/upc-prod/kustomization.yaml
- target:
kind: Application
name: gitea
patch: |
- op: replace
path: /spec/sources/0/helm/valueFiles/1
value: $values/infra/values/upc-prod/gitea-values.yaml
```
### Components Using Cloud Overlays
| Component | Cloud-specific config | Overlay value file |
|-----------|----------------------|-------------------|
| **Traefik** | LB annotations, proxy protocol IPs | `traefik-values.yaml` |
| **Keycloak** | Hostname, TLS settings | `keycloak-values.yaml` |
| **Grafana** | Hostname, datasource URLs | `grafana-values.yaml` |
| **Gitea** | StorageClass (persistence + PostgreSQL) | `gitea-values.yaml` |
| **OpenCost** | Custom pricing model (CPU/RAM/storage rates) | `opencost-values.yaml` |
### Backup CronJob
The `gitea-backup` CronJob uses a generic `s3` alias for `minio/mc`. The actual endpoint and credentials come from the `gitea-backup-s3` Sealed Secret, which is per-cloud. Reference scripts for different cloud providers are in `scripts/backup/`:
| Script | Provider | Tool |
|--------|----------|------|
| `s3-minio.sh` | S3-compatible (UpCloud, MinIO, Wasabi) | `minio/mc` |
| `aws-s3.sh` | AWS S3 | `aws` CLI |
| `azure-blob.sh` | Azure Blob Storage | `az` CLI |
| `gcp-gcs.sh` | GCP Cloud Storage | `gsutil` |
### Adding a New Cloud Provider
To add support for a new cloud (e.g., `oci-dev` for Oracle Cloud):
1. **Cluster config**: `clusters/oci-dev.yaml` — clusterName, domain, trustedIPs, cloudProvider
2. **Overlay value files** in `infra/values/oci-dev/`:
- `traefik-values.yaml` — LB annotations, proxy protocol config
- `keycloak-values.yaml` — hostname
- `grafana-values.yaml` — hostname
- `gitea-values.yaml` — `storageClass` for persistence + PostgreSQL
- `opencost-values.yaml` — pricing model or cloud billing integration
3. **Kustomize overlay**: `infra/overlays/oci-dev/kustomization.yaml` — patch `valueFiles[1]` for each Application
4. **App-of-apps**: `_app-of-apps-oci-dev.yaml` — points to `infra/overlays/oci-dev`
5. **Secrets overlay**: `secrets/overlays/oci-dev/kustomization.yaml` — references `../../base`, add cloud-specific SealedSecrets if needed
6. **Secrets patch**: Add patch to `infra/overlays/oci-dev/kustomization.yaml` to swap secrets path to `secrets/overlays/oci-dev`
7. **Bootstrap**: `./bootstrap.sh oci-dev`
---
## Glossary
### Terms
@@ -1930,11 +1316,7 @@ team: platform
| **Prometheus** | 2.47.0+ | Latest |
| **Grafana** | 10.0.0+ | Latest |
| **Loki** | 2.9.0+ | Latest |
| **Tempo** | 2.6.0+ | 1.24.4 |
| **Fluent-Bit** | 2.1.0+ | Latest |
| **Gitea** | 1.25.4 | 12.5.0 |
| **Gitea Act Runner** | Latest | Latest |
| **Renovate** | v43.113.0 | 46.109.0 |
| **PostgreSQL** | 16-alpine | N/A |
| **Trivy** | Latest | Latest |
@@ -1946,6 +1328,6 @@ team: platform
---
**Last Updated**: 2026-04-22
**Last Updated**: 2026-03-16
**Maintained By**: Platform Team
**Version**: 1.0.0

4
infra/base/cert-manager-application.yaml
View File

@@ -48,10 +48,10 @@ spec:
resources:
requests:
cpu: 50m
memory: 128Mi
memory: 64Mi
limits:
cpu: 100m
memory: 256Mi
memory: 128Mi
# Service account
serviceAccount:

4
infra/base/cluster-resources-application.yaml
View File

@@ -15,11 +15,9 @@ spec:
project: default
source:
repoURL: ssh://git@git.forteapps.net:2222/Forte/launchpad.git
repoURL: git@github.com:fortedigital/sturdy-adventure.git
targetRevision: HEAD
path: cluster-resources
directory:
exclude: 'network'
destination:
server: https://kubernetes.default.svc

4
infra/base/enterprise-apps.yaml
View File

@@ -16,9 +16,9 @@ metadata:
spec:
project: default
source:
repoURL: ssh://git@git.forteapps.net:2222/Forte/launchpad.git
repoURL: git@github.com:fortedigital/sturdy-adventure.git
targetRevision: HEAD
path: apps/overlays/upc-dev
path: apps/overlays/eu
destination:
server: https://kubernetes.default.svc
namespace: apps

2
infra/base/fluent-bit.yaml
View File

@@ -23,7 +23,7 @@ spec:
valueFiles:
- $values/infra/values/base/fluent-bit-values.yaml
- repoURL: ssh://git@git.forteapps.net:2222/Forte/launchpad.git
- repoURL: git@github.com:fortedigital/sturdy-adventure.git
targetRevision: HEAD
ref: values

48
infra/base/gitea-actions.yaml
View File

@@ -1,48 +0,0 @@
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: gitea-actions
namespace: argocd
annotations:
argocd.argoproj.io/sync-wave: "2"
labels:
app.kubernetes.io/name: gitea-actions
app.kubernetes.io/part-of: platform
app.kubernetes.io/managed-by: argocd
finalizers:
- resources-finalizer.argocd.argoproj.io
spec:
project: default
sources:
- repoURL: https://dl.gitea.com/charts
chart: actions
targetRevision: "0.0.5"
helm:
releaseName: gitea-actions
valueFiles:
- $values/infra/values/base/gitea-actions-values.yaml
- repoURL: ssh://git@git.forteapps.net:2222/Forte/launchpad.git
targetRevision: HEAD
ref: values
destination:
server: https://kubernetes.default.svc
namespace: gitea
syncPolicy:
automated:
prune: true
selfHeal: true
allowEmpty: false
syncOptions:
- CreateNamespace=true
- Validate=true
- ServerSideApply=true
ignoreDifferences:
- group: apps
kind: StatefulSet
jsonPointers:
- /spec/volumeClaimTemplates

53
infra/base/gitea.yaml
View File

@@ -1,53 +0,0 @@
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: gitea
namespace: argocd
annotations:
argocd.argoproj.io/sync-wave: "1"
labels:
app.kubernetes.io/name: gitea
app.kubernetes.io/part-of: platform
app.kubernetes.io/managed-by: argocd
finalizers:
- resources-finalizer.argocd.argoproj.io
spec:
project: default
sources:
- repoURL: https://dl.gitea.com/charts
chart: gitea
targetRevision: "12.5.0"
helm:
releaseName: gitea
valueFiles:
- $values/infra/values/base/gitea-values.yaml
- $values/infra/values/upc-dev/gitea-values.yaml
- repoURL: ssh://git@git.forteapps.net:2222/Forte/launchpad.git
targetRevision: HEAD
ref: values
destination:
server: https://kubernetes.default.svc
namespace: gitea
syncPolicy:
automated:
prune: true
selfHeal: true
allowEmpty: false
syncOptions:
- CreateNamespace=true
- Validate=true
- ServerSideApply=true
ignoreDifferences:
- group: apps
kind: StatefulSet
jsonPointers:
- /spec/volumeClaimTemplates
- group: v1
kind: Secret
jsonPointers:
- /data/postgres-password

34
infra/base/grafana-dashboards.yaml
View File

@@ -1,34 +0,0 @@
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: grafana-dashboards
namespace: argocd
annotations:
argocd.argoproj.io/sync-wave: "2"
labels:
app.kubernetes.io/name: grafana-dashboards
app.kubernetes.io/part-of: monitoring
app.kubernetes.io/managed-by: argocd
finalizers:
- resources-finalizer.argocd.argoproj.io
spec:
project: default
source:
repoURL: ssh://git@git.forteapps.net:2222/Forte/launchpad.git
targetRevision: HEAD
path: infra/dashboards
destination:
server: https://kubernetes.default.svc
namespace: monitoring
syncPolicy:
automated:
prune: true
selfHeal: true
allowEmpty: false
syncOptions:
- CreateNamespace=true
- Validate=true
- ServerSideApply=true

4
infra/base/grafana.yaml
View File

@@ -22,9 +22,9 @@ spec:
releaseName: grafana
valueFiles:
- $values/infra/values/base/grafana-values.yaml
- $values/infra/values/upc-dev/grafana-values.yaml
- $values/infra/values/eu/grafana-values.yaml
- repoURL: ssh://git@git.forteapps.net:2222/Forte/launchpad.git
- repoURL: git@github.com:fortedigital/sturdy-adventure.git
targetRevision: HEAD
ref: values

48
infra/base/karpor.yaml
View File

@@ -1,48 +0,0 @@
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: karpor
namespace: argocd
annotations:
argocd.argoproj.io/sync-wave: "1"
labels:
app.kubernetes.io/name: karpor
app.kubernetes.io/part-of: developer-portal
app.kubernetes.io/managed-by: argocd
finalizers:
- resources-finalizer.argocd.argoproj.io
spec:
project: default
sources:
- repoURL: https://kusionstack.github.io/charts
chart: karpor
targetRevision: "0.7.6"
helm:
releaseName: karpor
valueFiles:
- $values/infra/values/base/karpor-values.yaml
- repoURL: ssh://git@git.forteapps.net:2222/Forte/launchpad.git
targetRevision: HEAD
ref: values
destination:
server: https://kubernetes.default.svc
namespace: karpor
syncPolicy:
automated:
prune: true
selfHeal: true
allowEmpty: false
syncOptions:
- CreateNamespace=true
- Validate=true
- ServerSideApply=true
ignoreDifferences:
- group: apps
kind: StatefulSet
jsonPointers:
- /spec/volumeClaimTemplates

10
infra/base/keycloak.yaml
View File

@@ -22,9 +22,9 @@ spec:
releaseName: keycloak
valueFiles:
- $values/infra/values/base/keycloak-values.yaml
- $values/infra/values/upc-dev/keycloak-values.yaml
- $values/infra/values/eu/keycloak-values.yaml
- repoURL: ssh://git@git.forteapps.net:2222/Forte/launchpad.git
- repoURL: git@github.com:fortedigital/sturdy-adventure.git
targetRevision: HEAD
ref: values
@@ -41,9 +41,3 @@ spec:
- CreateNamespace=true
- Validate=true
- ServerSideApply=true
ignoreDifferences:
- group: batch
kind: CronJob
jsonPointers:
- /spec/jobTemplate/spec/template/spec/containers/0/args

8
infra/base/kustomization.yaml
View File

@@ -15,11 +15,3 @@ resources:
- cluster-resources-application.yaml
- kyverno-policies.yaml
- secrets.yaml
- gitea.yaml
- gitea-actions.yaml
- opencost.yaml
- renovate.yaml
- tempo.yaml
- grafana-dashboards.yaml
- network-policies-application.yaml
- karpor.yaml

2
infra/base/kyverno-policies.yaml
View File

@@ -15,7 +15,7 @@ spec:
project: default
source:
repoURL: ssh://git@git.forteapps.net:2222/Forte/launchpad.git
repoURL: git@github.com:fortedigital/sturdy-adventure.git
targetRevision: HEAD
path: cluster-resources/policies

4
infra/base/kyverno.yaml
View File

@@ -39,10 +39,6 @@ spec:
targetRevision: v3.7.0 # Update to latest stable version
helm:
releaseName: kyverno
valuesObject:
grafana:
enabled: true
namespace: monitoring
destination:
server: https://kubernetes.default.svc

2
infra/base/loki.yaml
View File

@@ -23,7 +23,7 @@ spec:
valueFiles:
- $values/infra/values/base/loki-values.yaml
- repoURL: ssh://git@git.forteapps.net:2222/Forte/launchpad.git
- repoURL: git@github.com:fortedigital/sturdy-adventure.git
targetRevision: HEAD
ref: values

33
infra/base/network-policies-application.yaml
View File

@@ -1,33 +0,0 @@
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: network-policies
namespace: argocd
labels:
app.kubernetes.io/name: network-policies
app.kubernetes.io/part-of: platform
app.kubernetes.io/managed-by: argocd
annotations:
argocd.argoproj.io/sync-wave: "1"
finalizers:
- resources-finalizer.argocd.argoproj.io
spec:
project: default
source:
repoURL: ssh://git@git.forteapps.net:2222/Forte/launchpad.git
targetRevision: HEAD
path: cluster-resources/network
destination:
server: https://kubernetes.default.svc
syncPolicy:
automated:
prune: true
selfHeal: true
allowEmpty: false
syncOptions:
- Validate=true
- ServerSideApply=true

43
infra/base/opencost.yaml
View File

@@ -1,43 +0,0 @@
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: opencost
namespace: argocd
annotations:
argocd.argoproj.io/sync-wave: "1"
labels:
app.kubernetes.io/name: opencost
app.kubernetes.io/part-of: monitoring
app.kubernetes.io/managed-by: argocd
finalizers:
- resources-finalizer.argocd.argoproj.io
spec:
project: default
sources:
- repoURL: https://opencost.github.io/opencost-helm-chart
chart: opencost
targetRevision: "1.42.0"
helm:
releaseName: opencost
valueFiles:
- $values/infra/values/base/opencost-values.yaml
- $values/infra/values/upc-dev/opencost-values.yaml
- repoURL: ssh://git@git.forteapps.net:2222/Forte/launchpad.git
targetRevision: HEAD
ref: values
destination:
server: https://kubernetes.default.svc
namespace: monitoring
syncPolicy:
automated:
prune: true
selfHeal: true
allowEmpty: false
syncOptions:
- CreateNamespace=true
- Validate=true
- ServerSideApply=true

2
infra/base/prometheus.yaml
View File

@@ -23,7 +23,7 @@ spec:
valueFiles:
- $values/infra/values/base/prometheus-values.yaml
- repoURL: ssh://git@git.forteapps.net:2222/Forte/launchpad.git
- repoURL: git@github.com:fortedigital/sturdy-adventure.git
targetRevision: HEAD
ref: values

42
infra/base/renovate.yaml
View File

@@ -1,42 +0,0 @@
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: renovate
namespace: argocd
annotations:
argocd.argoproj.io/sync-wave: "2"
labels:
app.kubernetes.io/name: renovate
app.kubernetes.io/part-of: platform
app.kubernetes.io/managed-by: argocd
finalizers:
- resources-finalizer.argocd.argoproj.io
spec:
project: default
sources:
- repoURL: ghcr.io/renovatebot/charts
chart: renovate
targetRevision: "46.109.0"
helm:
releaseName: renovate
valueFiles:
- $values/infra/values/base/renovate-values.yaml
- repoURL: ssh://git@git.forteapps.net:2222/Forte/launchpad.git
targetRevision: HEAD
ref: values
destination:
server: https://kubernetes.default.svc
namespace: renovate
syncPolicy:
automated:
prune: true
selfHeal: true
allowEmpty: false
syncOptions:
- CreateNamespace=true
- Validate=true
- ServerSideApply=true

4
infra/base/secrets.yaml
View File

@@ -17,8 +17,8 @@ metadata:
spec:
project: default
source:
repoURL: ssh://git@git.forteapps.net:2222/Forte/launchpad.git
path: secrets/overlays/upc-dev
repoURL: git@github.com:fortedigital/sturdy-adventure.git
path: secrets/eu
destination:
server: https://kubernetes.default.svc
namespace: secrets

42
infra/base/tempo.yaml
View File

@@ -1,42 +0,0 @@
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: tempo
namespace: argocd
annotations:
argocd.argoproj.io/sync-wave: "1"
labels:
app.kubernetes.io/name: tempo
app.kubernetes.io/part-of: monitoring
app.kubernetes.io/managed-by: argocd
finalizers:
- resources-finalizer.argocd.argoproj.io
spec:
project: default
sources:
- repoURL: https://grafana.github.io/helm-charts
chart: tempo
targetRevision: "1.24.4"
helm:
releaseName: tempo
valueFiles:
- $values/infra/values/base/tempo-values.yaml
- repoURL: ssh://git@git.forteapps.net:2222/Forte/launchpad.git
targetRevision: HEAD
ref: values
destination:
server: https://kubernetes.default.svc
namespace: monitoring
syncPolicy:
automated:
prune: true
selfHeal: true
allowEmpty: false
syncOptions:
- CreateNamespace=true
- Validate=true
- ServerSideApply=true

4
infra/base/traefik-application.yaml
View File

@@ -29,9 +29,9 @@ spec:
releaseName: traefik
valueFiles:
- $values/infra/values/base/traefik-values.yaml
- $values/infra/values/upc-dev/traefik-values.yaml
- $values/infra/values/eu/traefik-values.yaml
- repoURL: ssh://git@git.forteapps.net:2222/Forte/launchpad.git
- repoURL: git@github.com:fortedigital/sturdy-adventure.git
targetRevision: HEAD
ref: values

148
infra/dashboards/dot-ai-logs.json
View File

@@ -1,148 +0,0 @@
{
"annotations": {
"list": []
},
"editable": true,
"fiscalYearStartMonth": 0,
"graphTooltip": 0,
"links": [],
"panels": [
{
"title": "Log Volume",
"type": "timeseries",
"gridPos": {
"h": 6,
"w": 24,
"x": 0,
"y": 0
},
"datasource": {
"type": "loki",
"uid": "loki"
},
"targets": [
{
"expr": "sum(count_over_time({namespace=\"dot-ai\"} [1m])) by (pod)",
"legendFormat": "{{pod}}",
"refId": "A"
}
],
"fieldConfig": {
"defaults": {
"custom": {
"drawStyle": "bars",
"fillOpacity": 50,
"stacking": {
"mode": "normal"
}
}
},
"overrides": []
}
},
{
"title": "Logs by Pod",
"type": "logs",
"gridPos": {
"h": 16,
"w": 24,
"x": 0,
"y": 6
},
"datasource": {
"type": "loki",
"uid": "loki"
},
"targets": [
{
"expr": "{namespace=\"dot-ai\", pod=~\"$pod\"} | json log=\"log\", message=\"message\", msg=\"msg\", level=\"level\", stream=\"stream\" | label_format level=`{{if .level}}{{.level}}{{else if eq .stream \"stderr\"}}error{{else}}info{{end}}` | line_format `{{.pod}} |{{or .message .msg .log}}`",
"refId": "A"
}
],
"options": {
"showTime": true,
"showLabels": false,
"showCommonLabels": false,
"wrapLogMessage": true,
"prettifyLogMessage": false,
"enableLogDetails": true,
"sortOrder": "Descending",
"dedupStrategy": "none",
"displayedFields": [
"pod",
"level"
]
}
},
{
"title": "Errors & Warnings",
"type": "logs",
"gridPos": {
"h": 10,
"w": 24,
"x": 0,
"y": 22
},
"datasource": {
"type": "loki",
"uid": "loki"
},
"targets": [
{
"expr": "{namespace=\"dot-ai\", pod=~\"$pod\"} | json log=\"log\", message=\"message\", msg=\"msg\", level=\"level\", stream=\"stream\" | label_format level=`{{if .level}}{{.level}}{{else if eq .stream \"stderr\"}}error{{else}}info{{end}}` | level=~`error|warn|warning|fatal|panic` | line_format `{{.pod}} |{{or .message .msg .log}}`",
"refId": "A"
}
],
"options": {
"showTime": true,
"showLabels": false,
"showCommonLabels": false,
"wrapLogMessage": true,
"prettifyLogMessage": false,
"enableLogDetails": true,
"sortOrder": "Descending",
"dedupStrategy": "none",
"displayedFields": [
"pod",
"level"
]
}
}
],
"schemaVersion": 39,
"tags": [
"dot-ai",
"logs",
"loki"
],
"templating": {
"list": [
{
"name": "pod",
"type": "query",
"datasource": {
"type": "loki",
"uid": "loki"
},
"query": {
"label": "pod",
"stream": "{namespace=\"dot-ai\"}",
"type": 1
},
"includeAll": true,
"multi": true,
"current": {
"selected": true,
"text": "All",
"value": "$__all"
}
}
]
},
"time": {
"from": "now-1h",
"to": "now"
},
"title": "dot-ai Logs",
"uid": "dot-ai-logs"
}

25
infra/dashboards/kustomization.yaml
View File

@@ -1,25 +0,0 @@
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
namespace: monitoring
generatorOptions:
disableNameSuffixHash: true
labels:
grafana_dashboard: "1"
configMapGenerator:
- name: grafana-dashboard-trivy
files:
- trivy.json
- name: grafana-dashboard-traefik-loki
files:
- traefik-loki.json
- name: grafana-dashboard-dot-ai-logs
files:
- dot-ai-logs.json
- name: grafana-dashboard-opencost
files:
- opencost.json
- name: grafana-dashboard-pod-security
files:
- pod-security.json

1510
infra/dashboards/opencost.json
View File

File diff suppressed because it is too large Load Diff

399
infra/dashboards/pod-security.json
View File

@@ -1,399 +0,0 @@
{
"annotations": {
"list": []
},
"editable": true,
"fiscalYearStartMonth": 0,
"graphTooltip": 1,
"links": [],
"panels": [
{
"title": "Enforced Denials",
"description": "Pods rejected by Pod Security Standards (enforce mode)",
"type": "stat",
"datasource": { "type": "prometheus" },
"gridPos": { "h": 5, "w": 6, "x": 0, "y": 0 },
"targets": [
{
"expr": "sum(increase(pod_security_evaluations_total{decision=\"deny\", mode=\"enforce\"}[$__range])) or vector(0)",
"refId": "A"
}
],
"fieldConfig": {
"defaults": {
"noValue": "0",
"thresholds": {
"mode": "absolute",
"steps": [
{ "value": null, "color": "green" },
{ "value": 1, "color": "red" }
]
}
},
"overrides": []
},
"options": {
"reduceOptions": { "calcs": ["lastNotNull"] },
"colorMode": "background",
"textMode": "auto"
}
},
{
"title": "Audit Violations",
"description": "Pods that violate audit-level policy (allowed but logged)",
"type": "stat",
"datasource": { "type": "prometheus" },
"gridPos": { "h": 5, "w": 6, "x": 6, "y": 0 },
"targets": [
{
"expr": "sum(increase(pod_security_evaluations_total{decision=\"deny\", mode=\"audit\"}[$__range])) or vector(0)",
"refId": "A"
}
],
"fieldConfig": {
"defaults": {
"noValue": "0",
"thresholds": {
"mode": "absolute",
"steps": [
{ "value": null, "color": "green" },
{ "value": 1, "color": "orange" }
]
}
},
"overrides": []
},
"options": {
"reduceOptions": { "calcs": ["lastNotNull"] },
"colorMode": "background",
"textMode": "auto"
}
},
{
"title": "Warnings",
"description": "Pods that triggered warn-level policy (allowed with warning)",
"type": "stat",
"datasource": { "type": "prometheus" },
"gridPos": { "h": 5, "w": 6, "x": 12, "y": 0 },
"targets": [
{
"expr": "sum(increase(pod_security_evaluations_total{decision=\"deny\", mode=\"warn\"}[$__range])) or vector(0)",
"refId": "A"
}
],
"fieldConfig": {
"defaults": {
"noValue": "0",
"thresholds": {
"mode": "absolute",
"steps": [
{ "value": null, "color": "green" },
{ "value": 1, "color": "yellow" }
]
}
},
"overrides": []
},
"options": {
"reduceOptions": { "calcs": ["lastNotNull"] },
"colorMode": "background",
"textMode": "auto"
}
},
{
"title": "Total Evaluations",
"description": "All pod security evaluations across all modes",
"type": "stat",
"datasource": { "type": "prometheus" },
"gridPos": { "h": 5, "w": 6, "x": 18, "y": 0 },
"targets": [
{
"expr": "sum(increase(pod_security_evaluations_total[$__range])) or vector(0)",
"refId": "A"
}
],
"fieldConfig": {
"defaults": {
"noValue": "0",
"thresholds": {
"mode": "absolute",
"steps": [
{ "value": null, "color": "blue" }
]
}
},
"overrides": []
},
"options": {
"reduceOptions": { "calcs": ["lastNotNull"] },
"colorMode": "background",
"textMode": "auto"
}
},
{
"title": "Violation Rate by Mode",
"description": "Rate of policy violations over time, grouped by enforcement mode",
"type": "timeseries",
"datasource": { "type": "prometheus" },
"gridPos": { "h": 8, "w": 12, "x": 0, "y": 5 },
"targets": [
{
"expr": "sum(rate(pod_security_evaluations_total{decision=\"deny\", mode=\"enforce\"}[5m]))",
"legendFormat": "enforce (denied)",
"refId": "A"
},
{
"expr": "sum(rate(pod_security_evaluations_total{decision=\"deny\", mode=\"audit\"}[5m]))",
"legendFormat": "audit",
"refId": "B"
},
{
"expr": "sum(rate(pod_security_evaluations_total{decision=\"deny\", mode=\"warn\"}[5m]))",
"legendFormat": "warn",
"refId": "C"
}
],
"fieldConfig": {
"defaults": {
"custom": {
"drawStyle": "line",
"lineWidth": 2,
"fillOpacity": 15,
"pointSize": 5,
"showPoints": "auto"
},
"unit": "ops"
},
"overrides": [
{
"matcher": { "id": "byName", "options": "enforce (denied)" },
"properties": [{ "id": "color", "value": { "fixedColor": "red", "mode": "fixed" } }]
},
{
"matcher": { "id": "byName", "options": "audit" },
"properties": [{ "id": "color", "value": { "fixedColor": "orange", "mode": "fixed" } }]
},
{
"matcher": { "id": "byName", "options": "warn" },
"properties": [{ "id": "color", "value": { "fixedColor": "yellow", "mode": "fixed" } }]
}
]
}
},
{
"title": "Violations by Policy Level",
"description": "Violation rate grouped by the PSS level that was violated",
"type": "timeseries",
"datasource": { "type": "prometheus" },
"gridPos": { "h": 8, "w": 12, "x": 12, "y": 5 },
"targets": [
{
"expr": "sum(rate(pod_security_evaluations_total{decision=\"deny\"}[5m])) by (policy_level)",
"legendFormat": "{{ policy_level }}",
"refId": "A"
}
],
"fieldConfig": {
"defaults": {
"custom": {
"drawStyle": "line",
"lineWidth": 2,
"fillOpacity": 15,
"pointSize": 5,
"showPoints": "auto"
},
"unit": "ops"
},
"overrides": [
{
"matcher": { "id": "byName", "options": "restricted" },
"properties": [{ "id": "color", "value": { "fixedColor": "yellow", "mode": "fixed" } }]
},
{
"matcher": { "id": "byName", "options": "baseline" },
"properties": [{ "id": "color", "value": { "fixedColor": "orange", "mode": "fixed" } }]
},
{
"matcher": { "id": "byName", "options": "privileged" },
"properties": [{ "id": "color", "value": { "fixedColor": "red", "mode": "fixed" } }]
}
]
}
},
{
"title": "Enforced Denials by Namespace",
"description": "Pods blocked per namespace (enforce mode only)",
"type": "timeseries",
"datasource": { "type": "prometheus" },
"gridPos": { "h": 8, "w": 12, "x": 0, "y": 13 },
"targets": [
{
"expr": "sum(rate(pod_security_evaluations_total{decision=\"deny\", mode=\"enforce\"}[5m])) by (resource_namespace)",
"legendFormat": "{{ resource_namespace }}",
"refId": "A"
}
],
"fieldConfig": {
"defaults": {
"custom": {
"drawStyle": "bars",
"lineWidth": 1,
"fillOpacity": 80,
"stacking": { "mode": "normal" }
},
"unit": "ops"
},
"overrides": []
}
},
{
"title": "Audit + Warn Violations by Namespace",
"description": "Non-enforced violations per namespace — candidates for tightening",
"type": "timeseries",
"datasource": { "type": "prometheus" },
"gridPos": { "h": 8, "w": 12, "x": 12, "y": 13 },
"targets": [
{
"expr": "sum(rate(pod_security_evaluations_total{decision=\"deny\", mode=~\"audit|warn\"}[5m])) by (resource_namespace)",
"legendFormat": "{{ resource_namespace }}",
"refId": "A"
}
],
"fieldConfig": {
"defaults": {
"custom": {
"drawStyle": "bars",
"lineWidth": 1,
"fillOpacity": 80,
"stacking": { "mode": "normal" }
},
"unit": "ops"
},
"overrides": []
}
},
{
"title": "Violations Breakdown",
"description": "Detailed breakdown of all policy violations",
"type": "table",
"datasource": { "type": "prometheus" },
"gridPos": { "h": 10, "w": 24, "x": 0, "y": 21 },
"targets": [
{
"expr": "sum(increase(pod_security_evaluations_total{decision=\"deny\"}[$__range])) by (resource_namespace, policy_level, mode, request_operation) > 0",
"format": "table",
"instant": true,
"refId": "A"
}
],
"transformations": [
{
"id": "organize",
"options": {
"excludeByName": { "Time": true },
"renameByName": {
"resource_namespace": "Namespace",
"policy_level": "Policy Level",
"mode": "Mode",
"request_operation": "Operation",
"Value": "Violations"
},
"indexByName": {
"resource_namespace": 0,
"policy_level": 1,
"mode": 2,
"request_operation": 3,
"Value": 4
}
}
},
{
"id": "sortBy",
"options": {
"fields": {},
"sort": [
{ "field": "Violations", "desc": true }
]
}
}
],
"fieldConfig": {
"defaults": {},
"overrides": [
{
"matcher": { "id": "byName", "options": "Mode" },
"properties": [
{
"id": "mappings",
"value": [
{ "type": "value", "options": { "enforce": { "text": "Enforce", "color": "red" }, "audit": { "text": "Audit", "color": "orange" }, "warn": { "text": "Warn", "color": "yellow" } } }
]
}
]
},
{
"matcher": { "id": "byName", "options": "Violations" },
"properties": [
{
"id": "custom.cellOptions",
"value": { "type": "color-background", "mode": "gradient" }
},
{
"id": "thresholds",
"value": {
"mode": "absolute",
"steps": [
{ "value": null, "color": "transparent" },
{ "value": 1, "color": "orange" },
{ "value": 100, "color": "red" }
]
}
}
]
}
]
}
},
{
"title": "Exemptions",
"description": "Pods exempted from policy evaluation",
"type": "timeseries",
"datasource": { "type": "prometheus" },
"gridPos": { "h": 8, "w": 24, "x": 0, "y": 31 },
"targets": [
{
"expr": "sum(rate(pod_security_exemptions_total[5m])) by (request_namespace)",
"legendFormat": "{{ request_namespace }}",
"refId": "A"
}
],
"fieldConfig": {
"defaults": {
"custom": {
"drawStyle": "line",
"lineWidth": 2,
"fillOpacity": 10
},
"unit": "ops"
},
"overrides": []
}
}
],
"schemaVersion": 39,
"tags": [
"security",
"pod-security",
"pss",
"compliance"
],
"templating": {
"list": []
},
"time": {
"from": "now-24h",
"to": "now"
},
"title": "Pod Security Violations",
"uid": "pod-security-violations"
}

1192
infra/dashboards/traefik-loki.json
View File

File diff suppressed because it is too large Load Diff

1841
infra/dashboards/trivy.json
View File

File diff suppressed because it is too large Load Diff

68
infra/overlays/aks-dev/kustomization.yaml
View File

@@ -1,68 +0,0 @@
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- ../../base
patches:
# Traefik: swap upc-dev → aks-dev
- target:
kind: Application
name: traefik
patch: |
- op: replace
path: /spec/sources/0/helm/valueFiles/1
value: $values/infra/values/aks-dev/traefik-values.yaml
# Keycloak: swap upc-dev → aks-dev
- target:
kind: Application
name: keycloak
patch: |
- op: replace
path: /spec/sources/0/helm/valueFiles/1
value: $values/infra/values/aks-dev/keycloak-values.yaml
# Grafana: swap upc-dev → aks-dev
- target:
kind: Application
name: grafana
patch: |
- op: replace
path: /spec/sources/0/helm/valueFiles/1
value: $values/infra/values/aks-dev/grafana-values.yaml
# Gitea: swap upc-dev → aks-dev
- target:
kind: Application
name: gitea
patch: |
- op: replace
path: /spec/sources/0/helm/valueFiles/1
value: $values/infra/values/aks-dev/gitea-values.yaml
# OpenCost: swap upc-dev → aks-dev
- target:
kind: Application
name: opencost
patch: |
- op: replace
path: /spec/sources/0/helm/valueFiles/1
value: $values/infra/values/aks-dev/opencost-values.yaml
# Secrets: change path to aks-dev
- target:
kind: Application
name: secrets
patch: |
- op: replace
path: /spec/source/path
value: secrets/aks-dev
# Enterprise-apps: point to aks-dev overlay
- target:
kind: Application
name: enterprise-apps
patch: |
- op: replace
path: /spec/source/path
value: apps/overlays/aks-dev

68
infra/overlays/aks-prod/kustomization.yaml
View File

@@ -1,68 +0,0 @@
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- ../../base
patches:
# Traefik: swap upc-dev → aks-prod
- target:
kind: Application
name: traefik
patch: |
- op: replace
path: /spec/sources/0/helm/valueFiles/1
value: $values/infra/values/aks-prod/traefik-values.yaml
# Keycloak: swap upc-dev → aks-prod
- target:
kind: Application
name: keycloak
patch: |
- op: replace
path: /spec/sources/0/helm/valueFiles/1
value: $values/infra/values/aks-prod/keycloak-values.yaml
# Grafana: swap upc-dev → aks-prod
- target:
kind: Application
name: grafana
patch: |
- op: replace
path: /spec/sources/0/helm/valueFiles/1
value: $values/infra/values/aks-prod/grafana-values.yaml
# Gitea: swap upc-dev → aks-prod
- target:
kind: Application
name: gitea
patch: |
- op: replace
path: /spec/sources/0/helm/valueFiles/1
value: $values/infra/values/aks-prod/gitea-values.yaml
# OpenCost: swap upc-dev → aks-prod
- target:
kind: Application
name: opencost
patch: |
- op: replace
path: /spec/sources/0/helm/valueFiles/1
value: $values/infra/values/aks-prod/opencost-values.yaml
# Secrets: change path to aks-prod
- target:
kind: Application
name: secrets
patch: |
- op: replace
path: /spec/source/path
value: secrets/aks-prod
# Enterprise-apps: point to aks-prod overlay
- target:
kind: Application
name: enterprise-apps
patch: |
- op: replace
path: /spec/source/path
value: apps/overlays/aks-prod

68
infra/overlays/eks-dev/kustomization.yaml
View File

@@ -1,68 +0,0 @@
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- ../../base
patches:
# Traefik: swap upc-dev → eks-dev
- target:
kind: Application
name: traefik
patch: |
- op: replace
path: /spec/sources/0/helm/valueFiles/1
value: $values/infra/values/eks-dev/traefik-values.yaml
# Keycloak: swap upc-dev → eks-dev
- target:
kind: Application
name: keycloak
patch: |
- op: replace
path: /spec/sources/0/helm/valueFiles/1
value: $values/infra/values/eks-dev/keycloak-values.yaml
# Grafana: swap upc-dev → eks-dev
- target:
kind: Application
name: grafana
patch: |
- op: replace
path: /spec/sources/0/helm/valueFiles/1
value: $values/infra/values/eks-dev/grafana-values.yaml
# Gitea: swap upc-dev → eks-dev
- target:
kind: Application
name: gitea
patch: |
- op: replace
path: /spec/sources/0/helm/valueFiles/1
value: $values/infra/values/eks-dev/gitea-values.yaml
# OpenCost: swap upc-dev → eks-dev
- target:
kind: Application
name: opencost
patch: |
- op: replace
path: /spec/sources/0/helm/valueFiles/1
value: $values/infra/values/eks-dev/opencost-values.yaml
# Secrets: change path to eks-dev
- target:
kind: Application
name: secrets
patch: |
- op: replace
path: /spec/source/path
value: secrets/eks-dev
# Enterprise-apps: point to eks-dev overlay
- target:
kind: Application
name: enterprise-apps
patch: |
- op: replace
path: /spec/source/path
value: apps/overlays/eks-dev

68
infra/overlays/eks-prod/kustomization.yaml
View File

@@ -1,68 +0,0 @@
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- ../../base
patches:
# Traefik: swap upc-dev → eks-prod
- target:
kind: Application
name: traefik
patch: |
- op: replace
path: /spec/sources/0/helm/valueFiles/1
value: $values/infra/values/eks-prod/traefik-values.yaml
# Keycloak: swap upc-dev → eks-prod
- target:
kind: Application
name: keycloak
patch: |
- op: replace
path: /spec/sources/0/helm/valueFiles/1
value: $values/infra/values/eks-prod/keycloak-values.yaml
# Grafana: swap upc-dev → eks-prod
- target:
kind: Application
name: grafana
patch: |
- op: replace
path: /spec/sources/0/helm/valueFiles/1
value: $values/infra/values/eks-prod/grafana-values.yaml
# Gitea: swap upc-dev → eks-prod
- target:
kind: Application
name: gitea
patch: |
- op: replace
path: /spec/sources/0/helm/valueFiles/1
value: $values/infra/values/eks-prod/gitea-values.yaml
# OpenCost: swap upc-dev → eks-prod
- target:
kind: Application
name: opencost
patch: |
- op: replace
path: /spec/sources/0/helm/valueFiles/1
value: $values/infra/values/eks-prod/opencost-values.yaml
# Secrets: change path to eks-prod
- target:
kind: Application
name: secrets
patch: |
- op: replace
path: /spec/source/path
value: secrets/eks-prod
# Enterprise-apps: point to eks-prod overlay
- target:
kind: Application
name: enterprise-apps
patch: |
- op: replace
path: /spec/source/path
value: apps/overlays/eks-prod

3
secrets/overlays/aks-prod/kustomization.yaml → infra/overlays/eu/kustomization.yaml
View File

@@ -2,3 +2,6 @@ apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- ../../base
# No patches needed — base already has "eu" paths
# EU is the default/base cluster

68
infra/overlays/gke-dev/kustomization.yaml
View File

@@ -1,68 +0,0 @@
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- ../../base
patches:
# Traefik: swap upc-dev → gke-dev
- target:
kind: Application
name: traefik
patch: |
- op: replace
path: /spec/sources/0/helm/valueFiles/1
value: $values/infra/values/gke-dev/traefik-values.yaml
# Keycloak: swap upc-dev → gke-dev
- target:
kind: Application
name: keycloak
patch: |
- op: replace
path: /spec/sources/0/helm/valueFiles/1
value: $values/infra/values/gke-dev/keycloak-values.yaml
# Grafana: swap upc-dev → gke-dev
- target:
kind: Application
name: grafana
patch: |
- op: replace
path: /spec/sources/0/helm/valueFiles/1
value: $values/infra/values/gke-dev/grafana-values.yaml
# Gitea: swap upc-dev → gke-dev
- target:
kind: Application
name: gitea
patch: |
- op: replace
path: /spec/sources/0/helm/valueFiles/1
value: $values/infra/values/gke-dev/gitea-values.yaml
# OpenCost: swap upc-dev → gke-dev
- target:
kind: Application
name: opencost
patch: |
- op: replace
path: /spec/sources/0/helm/valueFiles/1
value: $values/infra/values/gke-dev/opencost-values.yaml
# Secrets: change path to gke-dev
- target:
kind: Application
name: secrets
patch: |
- op: replace
path: /spec/source/path
value: secrets/gke-dev
# Enterprise-apps: point to gke-dev overlay
- target:
kind: Application
name: enterprise-apps
patch: |
- op: replace
path: /spec/source/path
value: apps/overlays/gke-dev

68
infra/overlays/gke-prod/kustomization.yaml
View File

@@ -1,68 +0,0 @@
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- ../../base
patches:
# Traefik: swap upc-dev → gke-prod
- target:
kind: Application
name: traefik
patch: |
- op: replace
path: /spec/sources/0/helm/valueFiles/1
value: $values/infra/values/gke-prod/traefik-values.yaml
# Keycloak: swap upc-dev → gke-prod
- target:
kind: Application
name: keycloak
patch: |
- op: replace
path: /spec/sources/0/helm/valueFiles/1
value: $values/infra/values/gke-prod/keycloak-values.yaml
# Grafana: swap upc-dev → gke-prod
- target:
kind: Application
name: grafana
patch: |
- op: replace
path: /spec/sources/0/helm/valueFiles/1
value: $values/infra/values/gke-prod/grafana-values.yaml
# Gitea: swap upc-dev → gke-prod
- target:
kind: Application
name: gitea
patch: |
- op: replace
path: /spec/sources/0/helm/valueFiles/1
value: $values/infra/values/gke-prod/gitea-values.yaml
# OpenCost: swap upc-dev → gke-prod
- target:
kind: Application
name: opencost
patch: |
- op: replace
path: /spec/sources/0/helm/valueFiles/1
value: $values/infra/values/gke-prod/opencost-values.yaml
# Secrets: change path to gke-prod
- target:
kind: Application
name: secrets
patch: |
- op: replace
path: /spec/source/path
value: secrets/gke-prod
# Enterprise-apps: point to gke-prod overlay
- target:
kind: Application
name: enterprise-apps
patch: |
- op: replace
path: /spec/source/path
value: apps/overlays/gke-prod

7
infra/overlays/upc-dev/kustomization.yaml
View File

@@ -1,7 +0,0 @@
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- ../../base
# No patches needed — base already has "upc-dev" paths
# upc-dev is the default/base cluster

68
infra/overlays/upc-prod/kustomization.yaml
View File

@@ -1,68 +0,0 @@
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- ../../base
patches:
# Traefik: swap upc-dev → upc-prod in valueFiles
- target:
kind: Application
name: traefik
patch: |
- op: replace
path: /spec/sources/0/helm/valueFiles/1
value: $values/infra/values/upc-prod/traefik-values.yaml
# Keycloak: swap upc-dev → upc-prod
- target:
kind: Application
name: keycloak
patch: |
- op: replace
path: /spec/sources/0/helm/valueFiles/1
value: $values/infra/values/upc-prod/keycloak-values.yaml
# Grafana: swap upc-dev → upc-prod
- target:
kind: Application
name: grafana
patch: |
- op: replace
path: /spec/sources/0/helm/valueFiles/1
value: $values/infra/values/upc-prod/grafana-values.yaml
# Gitea: swap upc-dev → upc-prod
- target:
kind: Application
name: gitea
patch: |
- op: replace
path: /spec/sources/0/helm/valueFiles/1
value: $values/infra/values/upc-prod/gitea-values.yaml
# OpenCost: swap upc-dev → upc-prod
- target:
kind: Application
name: opencost
patch: |
- op: replace
path: /spec/sources/0/helm/valueFiles/1
value: $values/infra/values/upc-prod/opencost-values.yaml
# Secrets: change path to upc-prod
- target:
kind: Application
name: secrets
patch: |
- op: replace
path: /spec/source/path
value: secrets/overlays/upc-prod
# Enterprise-apps: point to upc-prod overlay
- target:
kind: Application
name: enterprise-apps
patch: |
- op: replace
path: /spec/source/path
value: apps/overlays/upc-prod

50
infra/overlays/us/kustomization.yaml Normal file
View File

@@ -0,0 +1,50 @@
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- ../../base
patches:
# Traefik: swap eu → us in valueFiles
- target:
kind: Application
name: traefik
patch: |
- op: replace
path: /spec/sources/0/helm/valueFiles/1
value: $values/infra/values/us/traefik-values.yaml
# Keycloak: swap eu → us
- target:
kind: Application
name: keycloak
patch: |
- op: replace
path: /spec/sources/0/helm/valueFiles/1
value: $values/infra/values/us/keycloak-values.yaml
# Grafana: swap eu → us
- target:
kind: Application
name: grafana
patch: |
- op: replace
path: /spec/sources/0/helm/valueFiles/1
value: $values/infra/values/us/grafana-values.yaml
# Secrets: change path to us
- target:
kind: Application
name: secrets
patch: |
- op: replace
path: /spec/source/path
value: secrets/us
# Enterprise-apps: point to us overlay
- target:
kind: Application
name: enterprise-apps
patch: |
- op: replace
path: /spec/source/path
value: apps/overlays/us

7
infra/values/aks-dev/gitea-values.yaml
View File

@@ -1,7 +0,0 @@
# AKS-specific: Azure managed disk storage class
persistence:
storageClass: managed-csi-premium
postgresql:
primary:
persistence:
storageClass: managed-csi-premium

4
infra/values/aks-dev/grafana-values.yaml
View File

@@ -1,4 +0,0 @@
# AKS-specific: Grafana hostname
ingress:
hosts:
- grafana.forteapps.net

3
infra/values/aks-dev/keycloak-values.yaml
View File

@@ -1,3 +0,0 @@
# AKS-specific: Keycloak hostname
ingress:
hostname: id.forteapps.net

8
infra/values/aks-dev/opencost-values.yaml
View File

@@ -1,8 +0,0 @@
# AKS-specific: Azure pricing via Cloud Billing API
opencost:
exporter:
cloudProviderApiKey: ""
customPricing:
enabled: false
azure:
secretName: opencost-azure-billing

11
infra/values/aks-dev/traefik-values.yaml
View File

@@ -1,11 +0,0 @@
# AKS-specific: Azure Load Balancer for Traefik
service:
annotations:
service.beta.kubernetes.io/azure-load-balancer-health-probe-request-path: /ping
ports:
web:
forwardedHeaders:
trustedIPs: "10.0.0.0/8"
websecure:
forwardedHeaders:
trustedIPs: "10.0.0.0/8"

7
infra/values/aks-prod/gitea-values.yaml
View File

@@ -1,7 +0,0 @@
# AKS-specific: Azure managed disk storage class (prod)
persistence:
storageClass: managed-csi-premium
postgresql:
primary:
persistence:
storageClass: managed-csi-premium

4
infra/values/aks-prod/grafana-values.yaml
View File

@@ -1,4 +0,0 @@
# AKS-specific: Grafana hostname (prod)
ingress:
hosts:
- grafana.fortedigital.com

3
infra/values/aks-prod/keycloak-values.yaml
View File

@@ -1,3 +0,0 @@
# AKS-specific: Keycloak hostname (prod)
ingress:
hostname: id.fortedigital.com

8
infra/values/aks-prod/opencost-values.yaml
View File

@@ -1,8 +0,0 @@
# AKS-specific: Azure pricing via Cloud Billing API (prod)
opencost:
exporter:
cloudProviderApiKey: ""
customPricing:
enabled: false
azure:
secretName: opencost-azure-billing

12
infra/values/aks-prod/traefik-values.yaml
View File

@@ -1,12 +0,0 @@
# AKS-specific: Azure Load Balancer for Traefik (prod)
service:
annotations:
service.beta.kubernetes.io/azure-load-balancer-health-probe-request-path: /ping
service.beta.kubernetes.io/azure-load-balancer-internal: "false"
ports:
web:
forwardedHeaders:
trustedIPs: "10.0.0.0/8"
websecure:
forwardedHeaders:
trustedIPs: "10.0.0.0/8"

23
infra/values/base/argocd-values.yaml
View File

@@ -2,21 +2,16 @@ configs:
secret:
createSecret: true
argocdServerAdminPassword: "$2b$12$Tmb1jH7ADvwWoUoNPXXsfOf6JqEluqhq8mL06a8DGT2AP1GzbNsCm"
ssh:
knownHosts: |
[git.forteapps.net]:2222 ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDTwi40de8yTGUuRT0i/XGicQ672BLhYR6D/lDquJrp/tdrWoZhVVPy0wxSkWsq1V92iiAUuQnXagOGsLBGZT9uDLWKvEmNDnCfjzTMq3J1iA3vk2rQ8WBlCzhvmeCV/r0ufl6vsgfwxSRomLZeqa2UkLHx69gy2Njb1S2/aZK1Q53f466hCUfDULZrTn2Nn5Sj8cEbJ8EyvVN2YG9HYBxQdzKRPZEmS1vyzmn8YrYIkZseIRQElabzWGh86owuaaqnwJhTJj1j2sEUeIet04sGKJcnxx2UL4H90N66LKMldmMiuli+ve/CjJmMwDl0zGkjIniT3XR8CyEXYHli7B1hR8Z+dbK6DBgjz+28lFgMIRY70KkZJNsJcBNZLZ5fHwCI13a9U3Uhg3Pu/6s0zlosM4CrAQNQCRe95ZPtCpdFhlGrOl4m1rdSK2meL6rND0TBBuZbaFF6Py7TawLCAiO2KRaVqhu9OFVjwJ/nifgLzFGwWj+WcYmpuR+DwozrF/Hl7QYsz1x4GO1SONY07KbIFkUCHOMAh0AELY5YE4eGI4mtG6SecdPaAdLREGZYK4IcyP5i1QW9g0wmfRSsV9jy+r0ivBxixxh4yJiNpkg6NXak40gQtGIme9EJ+DxrRLruNsfDILWcdSuH/wvuorv56NpQFGB0FzB6LXMloSYptQ==
cm:
application.resourceTrackingMethod: annotation
timeout.reconciliation: 60s
admin.enabled: "true"
repositories: |
- type: git
url: https://github.com/snothub
name: github-repo
params:
"server.insecure": true
repoServer:
env:
# Disable git submodule checkout - submodules (e.g. shared-prompts)
# are not needed for K8s manifest generation
- name: ARGOCD_GIT_MODULES_ENABLED
value: "false"
server:
ingress:
enabled: false
@@ -37,7 +32,7 @@ notifications:
method: POST
body: |
{
"payload": "🖥️ {{ .context.clusterName }}: 🔄 *{{ .app.metadata.name }}* is syncing...\n📦 Revision: {{ .app.status.sync.revision | default `n/a` | substr 0 7 }}"
"payload": "🖥️ {{ .context.clusterName }}: 🔄 *{{ .app.metadata.name }}* is syncing...\n📦 Revision: {{ .app.status.sync.revision | substr 0 7 }}"
}
template.app-sync-succeeded: |
webhook:
@@ -45,7 +40,7 @@ notifications:
method: POST
body: |
{
"payload": "🖥️ {{ .context.clusterName }}: ✅ *{{ .app.metadata.name }}* sync succeeded\n📦 Revision: {{ .app.status.sync.revision | default `n/a` | substr 0 7 }}{{ range .app.status.summary.images }}\n🏷️ Image: {{ . }}{{ end }}"
"payload": "🖥️ {{ .context.clusterName }}: ✅ *{{ .app.metadata.name }}* sync succeeded\n📦 Revision: {{ .app.status.sync.revision | substr 0 7 }}{{ range .app.status.summary.images }}\n🏷️ Image: {{ . }}{{ end }}"
}
template.app-sync-failed: |
webhook:
@@ -53,7 +48,7 @@ notifications:
method: POST
body: |
{
"payload": "🖥️ {{ .context.clusterName }}: ❌ *{{ .app.metadata.name }}* sync failed\n📦 Revision: {{ .app.status.sync.revision | default `n/a` | substr 0 7 }}\n⚠️ Message: {{ .app.status.operationState.message }}"
"payload": "🖥️ {{ .context.clusterName }}: ❌ *{{ .app.metadata.name }}* sync failed\n📦 Revision: {{ .app.status.sync.revision | substr 0 7 }}\n⚠️ Message: {{ .app.status.operationState.message }}"
}
template.app-degraded: |
webhook:
@@ -61,7 +56,7 @@ notifications:
method: POST
body: |
{
"payload": "🖥️ {{ .context.clusterName }}: ⚠️ *{{ .app.metadata.name }}* is degraded\n🏥 Health: {{ .app.status.health.status }}\n📦 Revision: {{ .app.status.sync.revision | default `n/a` | substr 0 7 }}{{ range .app.status.summary.images }}\n🏷️ Image: {{ . }}{{ end }}"
"payload": "🖥️ {{ .context.clusterName }}: ⚠️ *{{ .app.metadata.name }}* is degraded\n🏥 Health: {{ .app.status.health.status }}\n💬 Message: {{ .app.status.health.message }}"
}
# Define notification triggers
@@ -70,7 +65,7 @@ notifications:
- when: app.status.operationState.phase in ['Running']
send: [app-syncing]
trigger.on-sync-succeeded: |
- when: app.status.operationState.phase in ['Succeeded'] and app.status.health.status == 'Healthy'
- when: app.status.operationState.phase in ['Succeeded']
send: [app-sync-succeeded]
trigger.on-sync-failed: |
- when: app.status.operationState.phase in ['Failed']

3
infra/values/base/fluent-bit-values.yaml
View File

@@ -48,8 +48,7 @@ config:
Match kube.*
Host loki-gateway.monitoring.svc.cluster.local
Port 80
Labels job=fluent-bit, namespace=$kubernetes['namespace_name'], pod=$kubernetes['pod_name'], container=$kubernetes['container_name'], stream=$stream
Auto_Kubernetes_Labels Off
Labels job=fluent-bit, namespace=$kubernetes['namespace_name'], pod=$kubernetes['pod_name'], container=$kubernetes['container_name']
Line_Format json
[OUTPUT]

36
infra/values/base/gitea-actions-values.yaml
View File

@@ -1,36 +0,0 @@
## Gitea Act Runner - Helm values
## Chart: actions v0.0.5 (https://dl.gitea.com/charts)
enabled: true
giteaRootURL: https://git.forteapps.net
existingSecret: gitea-runner-token
existingSecretKey: token
statefulset:
replicas: 3
resources:
requests:
cpu: 250m
memory: 256Mi
limits:
cpu: "1"
memory: 1Gi
actRunner:
config: |
log:
level: info
cache:
enabled: false
container:
require_docker: true
docker_timeout: 300s
runner:
labels:
- "ubuntu-latest:docker://catthehacker/ubuntu:act-22.04"
- "ubuntu-22.04:docker://catthehacker/ubuntu:act-22.04"
dind:
rootless: false

182
infra/values/base/gitea-values.yaml
View File

@@ -1,182 +0,0 @@
# Gitea Helm Chart Values
# Host: git.forteapps.net
# Chart: gitea v12.5.0 (app v1.25.4)
# Repo: https://dl.gitea.com/charts
# -- Admin account (password from sealed secret)
gitea:
admin:
existingSecret: gitea-credentials
email: admin@forteapps.net
# -- Gitea app.ini configuration
config:
APP_NAME: "Forte Git"
server:
DOMAIN: git.forteapps.net
ROOT_URL: https://git.forteapps.net
SSH_DOMAIN: git.forteapps.net
SSH_PORT: 2222
LFS_START_SERVER: true
ENABLE_GITEA_PAGES: true
ENABLE_BASIC_AUTH_CHALLENGE: true
service:
DISABLE_REGISTRATION: false
DEFAULT_ALLOW_CREATE_ORGANIZATION: false
REQUIRE_SIGNIN_VIEW: false
ALLOW_ONLY_EXTERNAL_REGISTRATION: true
ENABLE_BASIC_AUTHENTICATION: true
ENABLE_PASSWORD_SIGNIN_FORM: false
AUTO_WATCH_ON_CHANGES: false
AUTO_WATCH_NEW_REPOS: false
ENABLE_NOTIFY_MAIL: false
ENABLE_TIMETRACKING: false
openid:
ENABLE_OPENID_SIGNIN: false
ENABLE_OPENID_SIGNUP: false
oauth2:
ENABLED: true
ENABLE_AUTO_REGISTRATION: true
USERNAME: email
session:
PROVIDER: db
cache:
ADAPTER: memory
database:
DB_TYPE: postgres
metrics:
ENABLED: true
repository:
DEFAULT_BRANCH: main
DEFAULT_PRIVATE: last
actions:
ENABLED: true
packages:
ENABLED: true
indexer:
ISSUE_INDEXER_TYPE: bleve
REPO_INDEXER_ENABLED: true
mailer:
ENABLED: true
PROTOCOL: smtp+starttls
SMTP_ADDR: smtp.office365.com
SMTP_PORT: 587
FROM: "noreply@fortedigital.com"
admin:
DEFAULT_EMAIL_NOTIFICATIONS: enabled
# -- SMTP credentials injected from secret (USER and PASSWD)
additionalConfigFromEnvs:
- name: GITEA__mailer__USER
valueFrom:
secretKeyRef:
name: gitea-smtp-secret
key: username
- name: GITEA__mailer__PASSWD
valueFrom:
secretKeyRef:
name: gitea-smtp-secret
key: password
# -- OIDC authentication via Forte
oauth:
- name: "Forte"
provider: "openidConnect"
existingSecret: gitea-oidc-credentials
key: gitea
autoDiscoverUrl: "https://id.forteapps.net/realms/forte/.well-known/openid-configuration"
scopes: "openid email profile organization"
groupClaimName: "groups"
adminGroup: ""
restrictedGroup: ""
# -- Prometheus metrics (scraped via annotations, no ServiceMonitor CRD needed)
metrics:
enabled: true
serviceMonitor:
enabled: false
# -- Ingress via Traefik with Let's Encrypt TLS
ingress:
enabled: true
className: traefik
annotations:
cert-manager.io/cluster-issuer: letsencrypt-prod
hosts:
- host: git.forteapps.net
paths:
- path: /
pathType: Prefix
tls:
- secretName: gitea-tls
hosts:
- git.forteapps.net
# -- Git repository storage
persistence:
enabled: true
size: 10Gi
accessModes:
- ReadWriteOnce
# -- Recreate strategy to avoid Multi-Attach errors with RWO volumes
strategy:
type: Recreate
# -- Pod resources
resources:
requests:
cpu: 100m
memory: 256Mi
limits:
cpu: 500m
memory: 512Mi
# -- Embedded PostgreSQL (Bitnami subchart)
# Password auto-generated by the subchart; Gitea chart auto-wires the connection.
postgresql:
enabled: true
auth:
username: gitea
database: gitea
primary:
persistence:
enabled: true
size: 8Gi
resources:
requests:
cpu: 100m
memory: 256Mi
limits:
cpu: 500m
memory: 512Mi
# -- Disable PostgreSQL HA (using single-instance postgresql above)
postgresql-ha:
enabled: false
# -- Disable Redis cluster (use in-memory cache instead)
redis-cluster:
enabled: false
# -- Disable test pod
test:
enabled: false
# -- SSH service (ClusterIP, exposed externally via Traefik TCP IngressRoute on port 2222)
service:
ssh:
type: ClusterIP
port: 22

1066
infra/values/base/grafana-values.yaml
View File

File diff suppressed because it is too large Load Diff

44
infra/values/base/karpor-values.yaml
View File

@@ -1,44 +0,0 @@
# Karpor - Kubernetes Visualization & Intelligence Tool
# Helm chart: https://github.com/KusionStack/charts/tree/master/charts/karpor
# Let the ArgoCD Application manage the namespace
namespaceEnabled: false
server:
replicas: 1
port: 7443
resources:
requests:
cpu: 250m
memory: 256Mi
limits:
cpu: 500m
memory: 1Gi
syncer:
replicas: 1
port: 7443
resources:
requests:
cpu: 250m
memory: 256Mi
limits:
cpu: 500m
memory: 1Gi
elasticsearch:
replicas: 1
port: 9200
resources:
requests:
cpu: 500m
memory: 2Gi
limits:
cpu: "2"
memory: 4Gi
etcd:
replicas: 1
port: 2379
persistence:
size: 5Gi

400
infra/values/base/keycloak-values.yaml
View File

@@ -19,18 +19,12 @@ ingress:
annotations:
cert-manager.io/cluster-issuer: letsencrypt-prod
metrics:
enabled: true
prometheusRule:
namespace: monitoring
enabled: true
resources:
requests:
cpu: 250m
memory: 512Mi
limits:
cpu: 500m
cpu: "1"
memory: 1Gi
postgresql:
@@ -62,395 +56,5 @@ keycloakConfigCli:
"registrationAllowed": false,
"loginWithEmailAllowed": true,
"resetPasswordAllowed": true,
"rememberMe": true,
"clients": [
{
"clientId": "gitea",
"name": "Gitea",
"enabled": true,
"protocol": "openid-connect",
"clientAuthenticatorType": "client-secret",
"standardFlowEnabled": true,
"directAccessGrantsEnabled": false,
"publicClient": false,
"redirectUris": ["https://git.forteapps.net/*"],
"webOrigins": ["https://git.forteapps.net"],
"defaultClientScopes": ["openid", "email", "profile"],
"attributes": {
"k8s.secret.sync": "true",
"k8s.secret.namespace": "gitea",
"k8s.secret.name": "gitea-oidc-credentials",
"k8s.secret.client-id-key": "key",
"k8s.secret.client-secret-key": "secret"
},
"protocolMappers": [
{
"name": "email_verified",
"protocol": "openid-connect",
"protocolMapper": "oidc-hardcoded-claim-mapper",
"config": {
"claim.name": "email_verified",
"claim.value": "true",
"jsonType.label": "boolean",
"id.token.claim": "true",
"access.token.claim": "true",
"userinfo.token.claim": "true"
"rememberMe": true
}
}
]
}
]
}
extraDeploy:
# -- ServiceAccount for the client registrar CronJob
- apiVersion: v1
kind: ServiceAccount
metadata:
name: keycloak-client-registrar
namespace: keycloak
# -- ClusterRole granting access to secrets and namespaces
- apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: keycloak-client-registrar
rules:
- apiGroups: [""]
resources: ["secrets"]
verbs: ["get", "list", "create", "update", "patch"]
- apiGroups: [""]
resources: ["namespaces"]
verbs: ["get", "list"]
# -- ClusterRoleBinding for the registrar ServiceAccount
- apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: keycloak-client-registrar
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: keycloak-client-registrar
subjects:
- kind: ServiceAccount
name: keycloak-client-registrar
namespace: keycloak
# -- CronJob: registers Keycloak clients and syncs secrets
- apiVersion: batch/v1
kind: CronJob
metadata:
name: keycloak-client-registrar
namespace: keycloak
spec:
schedule: "*/2 * * * *"
concurrencyPolicy: Forbid
successfulJobsHistoryLimit: 1
failedJobsHistoryLimit: 3
jobTemplate:
spec:
backoffLimit: 3
template:
spec:
serviceAccountName: keycloak-client-registrar
restartPolicy: Never
containers:
- name: registrar
image: alpine:3.20
command: ["/bin/sh", "-c"]
args:
- |
set -e
apk add --no-cache curl jq > /dev/null 2>&1
KEYCLOAK_URL="http://keycloak:80"
REALM="forte"
K8S_API="https://kubernetes.default.svc"
SA_TOKEN=$(cat /var/run/secrets/kubernetes.io/serviceaccount/token)
CA_CERT="/var/run/secrets/kubernetes.io/serviceaccount/ca.crt"
CENTRAL_NS="secrets"
# --- Authenticate to Keycloak Admin API ---
ADMIN_USER="admin"
ADMIN_PASS=$(cat /secrets/admin-password)
echo "Authenticating to Keycloak..."
TOKEN=$(curl -sf -X POST "${KEYCLOAK_URL}/realms/master/protocol/openid-connect/token" \
-d "client_id=admin-cli" \
-d "username=${ADMIN_USER}" \
-d "password=${ADMIN_PASS}" \
-d "grant_type=password" | jq -r '.access_token')
if [ -z "$TOKEN" ] || [ "$TOKEN" = "null" ]; then
echo "ERROR: Failed to authenticate to Keycloak"
exit 1
fi
# --- Helper functions ---
# Upsert a K8s Secret: try PUT (update), fall back to POST (create)
upsert_secret() {
local ns="$1" name="$2" manifest="$3"
local code
code=$(curl -sf -o /dev/null -w "%{http_code}" \
--cacert "$CA_CERT" \
-H "Authorization: Bearer ${SA_TOKEN}" \
-H "Content-Type: application/json" \
-X PUT -d "$manifest" \
"${K8S_API}/api/v1/namespaces/${ns}/secrets/${name}")
if [ "$code" = "200" ]; then
echo " Updated secret '${ns}/${name}'"
elif [ "$code" = "404" ]; then
code=$(curl -sf -o /dev/null -w "%{http_code}" \
--cacert "$CA_CERT" \
-H "Authorization: Bearer ${SA_TOKEN}" \
-H "Content-Type: application/json" \
-X POST -d "$manifest" \
"${K8S_API}/api/v1/namespaces/${ns}/secrets")
if [ "$code" = "201" ]; then
echo " Created secret '${ns}/${name}'"
else
echo " ERROR: Failed to create secret '${ns}/${name}' (HTTP ${code})"
return 1
fi
else
echo " ERROR: Failed to update secret '${ns}/${name}' (HTTP ${code})"
return 1
fi
}
# Build a credential Secret JSON manifest
build_credential_secret() {
local ns="$1" name="$2" id_key="$3" secret_key="$4" b64_id="$5" b64_secret="$6"
cat <<MANIFEST
{
"apiVersion": "v1",
"kind": "Secret",
"metadata": {
"name": "${name}",
"namespace": "${ns}",
"labels": {
"app.kubernetes.io/managed-by": "keycloak-client-registrar"
}
},
"type": "Opaque",
"data": {
"${id_key}": "${b64_id}",
"${secret_key}": "${b64_secret}"
}
}
MANIFEST
}
# Sync credentials to target + central namespace
sync_credentials() {
local client_id="$1" client_uuid="$2" target_ns="$3" target_name="$4" id_key="$5" secret_key="$6"
# Get the client secret from Keycloak
local secret_value
secret_value=$(curl -sf -H "Authorization: Bearer ${TOKEN}" \
"${KEYCLOAK_URL}/admin/realms/${REALM}/clients/${client_uuid}/client-secret" \
| jq -r '.value')
if [ -z "$secret_value" ] || [ "$secret_value" = "null" ]; then
echo " WARNING: No secret found for client '${client_id}', skipping"
return 0
fi
local b64_id b64_secret
b64_id=$(printf '%s' "$client_id" | base64 | tr -d '\n')
b64_secret=$(printf '%s' "$secret_value" | base64 | tr -d '\n')
# Write to target namespace (if it exists)
local ns_status
ns_status=$(curl -sf -o /dev/null -w "%{http_code}" \
--cacert "$CA_CERT" \
-H "Authorization: Bearer ${SA_TOKEN}" \
"${K8S_API}/api/v1/namespaces/${target_ns}")
if [ "$ns_status" = "200" ]; then
local manifest
manifest=$(build_credential_secret "$target_ns" "$target_name" "$id_key" "$secret_key" "$b64_id" "$b64_secret")
upsert_secret "$target_ns" "$target_name" "$manifest" || return 1
else
echo " WARNING: Namespace '${target_ns}' does not exist, skipping target"
fi
# Always write a central copy to the secrets namespace
local central_manifest
central_manifest=$(build_credential_secret "$CENTRAL_NS" "$target_name" "$id_key" "$secret_key" "$b64_id" "$b64_secret")
upsert_secret "$CENTRAL_NS" "$target_name" "$central_manifest" || return 1
}
# Annotate a K8s Secret with sync status
annotate_secret() {
local ns="$1" name="$2" key="$3" value="$4"
local patch
patch=$(printf '{"metadata":{"annotations":{"%s":"%s"}}}' "$key" "$value")
curl -sf -o /dev/null \
--cacert "$CA_CERT" \
-H "Authorization: Bearer ${SA_TOKEN}" \
-H "Content-Type: application/strategic-merge-patch+json" \
-X PATCH -d "$patch" \
"${K8S_API}/api/v1/namespaces/${ns}/secrets/${name}"
}
# =============================================
# LEGACY PATH — sync existing realm clients
# =============================================
echo "=== Legacy sync: clients with k8s.secret.sync=true ==="
CLIENTS=$(curl -sf -H "Authorization: Bearer ${TOKEN}" \
"${KEYCLOAK_URL}/admin/realms/${REALM}/clients")
SYNC_CLIENTS=$(echo "$CLIENTS" | jq -c '[.[] | select(.attributes["k8s.secret.sync"] == "true")]')
COUNT=$(echo "$SYNC_CLIENTS" | jq 'length')
echo "Found ${COUNT} legacy client(s) with sync enabled"
echo "$SYNC_CLIENTS" | jq -c '.[]' | while read -r CLIENT; do
CLIENT_ID=$(echo "$CLIENT" | jq -r '.clientId')
CLIENT_UUID=$(echo "$CLIENT" | jq -r '.id')
TARGET_NS=$(echo "$CLIENT" | jq -r '.attributes["k8s.secret.namespace"]')
TARGET_NAME=$(echo "$CLIENT" | jq -r '.attributes["k8s.secret.name"]')
ID_KEY=$(echo "$CLIENT" | jq -r '.attributes["k8s.secret.client-id-key"] // "client-id"')
SECRET_KEY=$(echo "$CLIENT" | jq -r '.attributes["k8s.secret.client-secret-key"] // "client-secret"')
echo "Processing legacy client '${CLIENT_ID}' -> '${TARGET_NS}/${TARGET_NAME}' (keys: ${ID_KEY}, ${SECRET_KEY})"
sync_credentials "$CLIENT_ID" "$CLIENT_UUID" "$TARGET_NS" "$TARGET_NAME" "$ID_KEY" "$SECRET_KEY"
done
# =============================================
# NEW PATH — self-service config Secrets
# =============================================
echo ""
echo "=== Self-service: config Secrets with label keycloak.forteapps.net/client-config=true ==="
CONFIG_SECRETS=$(curl -sf \
--cacert "$CA_CERT" \
-H "Authorization: Bearer ${SA_TOKEN}" \
"${K8S_API}/api/v1/namespaces/keycloak/secrets?labelSelector=keycloak.forteapps.net/client-config=true")
CONFIG_COUNT=$(echo "$CONFIG_SECRETS" | jq '.items | length')
echo "Found ${CONFIG_COUNT} config Secret(s) to process"
echo "$CONFIG_SECRETS" | jq -c '.items[]' | while read -r CONFIG_SECRET; do
CONFIG_NAME=$(echo "$CONFIG_SECRET" | jq -r '.metadata.name')
SOURCE_NS=$(echo "$CONFIG_SECRET" | jq -r '.metadata.annotations["keycloak.forteapps.net/source-namespace"] // .metadata.labels["keycloak.forteapps.net/source-namespace"] // "unknown"')
# Decode client.json from the Secret data
CLIENT_JSON_B64=$(echo "$CONFIG_SECRET" | jq -r '.data["client.json"] // empty')
if [ -z "$CLIENT_JSON_B64" ]; then
echo "WARNING: Config Secret '${CONFIG_NAME}' missing client.json field, skipping"
continue
fi
CLIENT_JSON=$(printf '%s' "$CLIENT_JSON_B64" | base64 -d)
CLIENT_ID=$(echo "$CLIENT_JSON" | jq -r '.clientId')
echo "Processing self-service client '${CLIENT_ID}' from config '${CONFIG_NAME}'"
# Compute config hash for change detection
CONFIG_HASH=$(printf '%s' "$CLIENT_JSON" | sha256sum | cut -d' ' -f1)
EXISTING_HASH=$(echo "$CONFIG_SECRET" | jq -r '.metadata.annotations["keycloak.forteapps.net/config-hash"] // ""')
# Extract secret delivery config from client.json
CRED_NS=$(echo "$CLIENT_JSON" | jq -r '.secret.namespace // "'"${SOURCE_NS}"'"')
CRED_NAME=$(echo "$CLIENT_JSON" | jq -r '.secret.name // "'"${CLIENT_ID}"'-oidc-credentials"')
CRED_ID_KEY=$(echo "$CLIENT_JSON" | jq -r '.secret.keys.clientId // "client-id"')
CRED_SECRET_KEY=$(echo "$CLIENT_JSON" | jq -r '.secret.keys.clientSecret // "client-secret"')
# Check if credential Secret already exists in target namespace
CRED_EXISTS=$(curl -sf -o /dev/null -w "%{http_code}" \
--cacert "$CA_CERT" \
-H "Authorization: Bearer ${SA_TOKEN}" \
"${K8S_API}/api/v1/namespaces/${CRED_NS}/secrets/${CRED_NAME}")
# Skip if hash matches and credential Secret exists
if [ "$CONFIG_HASH" = "$EXISTING_HASH" ] && [ "$CRED_EXISTS" = "200" ]; then
echo " No changes detected, skipping"
continue
fi
# Build Keycloak client representation (strip our secret delivery config)
KC_CLIENT=$(echo "$CLIENT_JSON" | jq '{
clientId: .clientId,
name: .name,
enabled: true,
protocol: "openid-connect",
clientAuthenticatorType: "client-secret",
standardFlowEnabled: true,
directAccessGrantsEnabled: false,
publicClient: false,
redirectUris: .redirectUris,
webOrigins: .webOrigins,
defaultClientScopes: .defaultClientScopes,
protocolMappers: (.protocolMappers // [])
}')
# Check if client already exists
EXISTING=$(curl -sf -H "Authorization: Bearer ${TOKEN}" \
"${KEYCLOAK_URL}/admin/realms/${REALM}/clients?clientId=${CLIENT_ID}" \
| jq -r '.[0].id // empty')
if [ -n "$EXISTING" ]; then
echo " Updating existing Keycloak client (uuid: ${EXISTING})"
HTTP_CODE=$(curl -sf -o /dev/null -w "%{http_code}" \
-H "Authorization: Bearer ${TOKEN}" \
-H "Content-Type: application/json" \
-X PUT -d "$KC_CLIENT" \
"${KEYCLOAK_URL}/admin/realms/${REALM}/clients/${EXISTING}")
if [ "$HTTP_CODE" != "204" ] && [ "$HTTP_CODE" != "200" ]; then
echo " ERROR: Failed to update client '${CLIENT_ID}' (HTTP ${HTTP_CODE})"
annotate_secret "keycloak" "$CONFIG_NAME" "keycloak.forteapps.net/sync-status" "error"
continue
fi
CLIENT_UUID="$EXISTING"
else
echo " Creating new Keycloak client '${CLIENT_ID}'"
HTTP_CODE=$(curl -sf -o /dev/null -w "%{http_code}" \
-H "Authorization: Bearer ${TOKEN}" \
-H "Content-Type: application/json" \
-X POST -d "$KC_CLIENT" \
"${KEYCLOAK_URL}/admin/realms/${REALM}/clients")
if [ "$HTTP_CODE" != "201" ]; then
echo " ERROR: Failed to create client '${CLIENT_ID}' (HTTP ${HTTP_CODE})"
annotate_secret "keycloak" "$CONFIG_NAME" "keycloak.forteapps.net/sync-status" "error"
continue
fi
# Fetch the newly created client's UUID
CLIENT_UUID=$(curl -sf -H "Authorization: Bearer ${TOKEN}" \
"${KEYCLOAK_URL}/admin/realms/${REALM}/clients?clientId=${CLIENT_ID}" \
| jq -r '.[0].id')
fi
# Sync credentials to target namespace
sync_credentials "$CLIENT_ID" "$CLIENT_UUID" "$CRED_NS" "$CRED_NAME" "$CRED_ID_KEY" "$CRED_SECRET_KEY"
# Annotate config Secret with hash and sync status
annotate_secret "keycloak" "$CONFIG_NAME" "keycloak.forteapps.net/config-hash" "$CONFIG_HASH"
annotate_secret "keycloak" "$CONFIG_NAME" "keycloak.forteapps.net/sync-status" "synced"
TIMESTAMP=$(date -u +"%Y-%m-%dT%H:%M:%SZ")
annotate_secret "keycloak" "$CONFIG_NAME" "keycloak.forteapps.net/last-sync" "$TIMESTAMP"
echo " Synced successfully"
done
echo ""
echo "Client registrar run complete"
volumeMounts:
- name: keycloak-credentials
mountPath: /secrets
readOnly: true
resources:
requests:
cpu: 50m
memory: 64Mi
limits:
cpu: 200m
memory: 128Mi
volumes:
- name: keycloak-credentials
secret:
secretName: keycloak-credentials
items:
- key: admin-password
path: admin-password

14
infra/values/base/loki-values.yaml
View File

@@ -20,22 +20,20 @@ loki:
limits_config:
reject_old_samples: true
reject_old_samples_max_age: 168h
ingestion_rate_mb: 15
ingestion_burst_size_mb: 30
ingestion_rate_mb: 10
ingestion_burst_size_mb: 20
max_line_size: 512KB
chunksCache:
enabled: false
resultsCache:
enabled: false
singleBinary:
replicas: 1
resources:
requests:
cpu: 50m
memory: 256Mi
limits:
cpu: 100m
memory: 1Gi
memory: 512Mi
limits:
cpu: 200m
memory: 2Gi
read:
replicas: 0
backend:

Some files were not shown because too many files have changed in this diff Show More