This commit is contained in:
2026-04-18 23:10:18 +02:00
parent 395ca70c2a
commit eed53006c1
5 changed files with 84 additions and 83 deletions

View File

@@ -146,12 +146,12 @@ This repository contains the complete GitOps configuration for our Kubernetes cl
|------------|---------|-----------|-----------|
| **[launchpad](https://git.forteapps.net/Forte/launchpad)** (this repo) | ArgoCD Applications, cluster resources | Platform / DevOps engineers | ✅ Often |
| **[forte-helm](https://git.forteapps.net/Forte/forte-helm)** | Generic Helm chart templates | Platform engineers | ❌ Rarely |
| **[helm-values](ssh://git@git.forteapps.net:2222/Forte/helm-prod-values.git)** | App-specific configuration & versions | Developers / CI pipelines | ✅ Sometimes |
| **[helm-prod-values](ssh://git@git.forteapps.net:2222/Forte/helm-prod-values.git)** | App-specific configuration & versions | Developers / CI pipelines | ✅ Sometimes |
### GitOps Workflow
```
Developer commits code → CI/CD builds image → Updates helm-values → ArgoCD syncs → Deployed to cluster
Developer commits code → CI/CD builds image → Updates helm-prod-values → ArgoCD syncs → Deployed to cluster
```
**Learn more**: [GitOps Architecture - GitOps Workflow](docs/GITOPS-ARCHITECTURE.md#gitops-workflow)
@@ -166,7 +166,7 @@ Developer commits code → CI/CD builds image → Updates helm-values → ArgoCD
**Quick version**:
1. Create `apps/myapp.yaml` (ArgoCD Application manifest)
2. Create `helm-values/myapp/values.yaml` (configuration)
2. Create `helm-prod-values/myapp/values.yaml` (configuration)
3. Create sealed secrets if needed
4. Commit and push - ArgoCD auto-syncs!
@@ -175,8 +175,8 @@ Developer commits code → CI/CD builds image → Updates helm-values → ArgoCD
**See detailed guide**: [Developer Guide - Updating an Existing Application](docs/DEVELOPER-GUIDE.md#updating-an-existing-application)
**Quick version**:
- **Update code**: Push to app repo → CI/CD updates image tag in helm-values
- **Update config**: Edit `helm-values/myapp/values.yaml` → commit → push
- **Update code**: Push to app repo → CI/CD updates image tag in helm-prod-values
- **Update config**: Edit `helm-prod-values/myapp/values.yaml` → commit → push
### Manage Secrets
@@ -204,7 +204,7 @@ git push
**Quick version**:
```yaml
# In helm-values/myapp/values.yaml
# In helm-prod-values/myapp/values.yaml
# Token-based auth (simple)
auth:
@@ -366,7 +366,7 @@ kubectl patch application myapp -n argocd \
### Multi-Source Pattern
Applications reference both:
1. **Helm charts** from `forte-helm` (templates)
2. **Values** from `helm-values` (configuration)
2. **Values** from `helm-prod-values` (configuration)
This separates reusable templates from environment-specific config.
@@ -435,7 +435,7 @@ Applications deploy in order using `argocd.argoproj.io/sync-wave`:
### Adding a New Application
1. Read [Developer Guide - Deploying Your First Application](docs/DEVELOPER-GUIDE.md#deploying-your-first-application)
2. Create ArgoCD Application manifest in `apps/`
3. Create Helm values in `helm-values/`
3. Create Helm values in `helm-prod-values/`
4. Create sealed secrets if needed
5. Commit and push - ArgoCD handles the rest!
@@ -485,8 +485,8 @@ Documentation lives in `docs/`. To update:
- [Sealed Secrets](https://github.com/bitnami-labs/sealed-secrets)
### Related Repositories
- [forte-helm](https://github.com/fortedigital/forte-helm) - Helm chart templates
- [helm-values](git@github.com:fortedigital/helm-values.git) - Application values
- [forte-helm](https://git.forteapps.net/Forte/forte-helm) - Helm chart templates
- [helm-prod-values](git@github.com:fortedigital/helm-prod-values.git) - Application values
---

View File

@@ -96,10 +96,10 @@ You'll need read/write access to these repositories:
cd launchpad
```
2. **helm-values** (Values repo)
2. **helm-prod-values** (Values repo)
```bash
git clone https://git.forteapps.net/Forte/helm-prod-values.git
cd helm-values
cd helm-prod-values
```
3. **forte-helm** (Chart repo - read-only for most developers)
@@ -175,13 +175,13 @@ npm run dev
│ - GitHub Actions builds image │
│ - Pushes to container registry (GHCR, Docker Hub) │
│ - Tags with version (e.g., v2.0.4) │
│ - Updates helm-values repository with new tag │
│ - Updates helm-prod-values repository with new tag │
└─────────────────────────────────────────────────────────────────┘
┌─────────────────────────────────────────────────────────────────┐
│ Step 3: GitOps Sync (Automated) │
│ - ArgoCD detects change in helm-values │
│ - ArgoCD detects change in helm-prod-values │
│ - Pulls updated configuration │
│ - Syncs to Kubernetes cluster │
│ - Sends Slack notification on success/failure │
@@ -201,7 +201,7 @@ Our setup uses three repositories:
| Repository | Purpose | Who Edits | How Often |
|------------|---------|-----------|-----------|
| **forte-helm** | Helm chart templates (generic, reusable) | Platform engineers | ❌ Rarely |
| **helm-values** | Application configuration (image tag, env vars) | Developers / CI pipelines | ✅ Sometimes |
| **helm-prod-values** | Application configuration (image tag, env vars) | Developers / CI pipelines | ✅ Sometimes |
| **launchpad** | ArgoCD Applications (what gets deployed) | Platform / DevOps engineers | ✅ Per new app |
### Example: Deploying "myapp"
@@ -223,7 +223,7 @@ spec:
value: {{ .Values.app.port }}
```
#### Repository: `helm-values` (Your App Config)
#### Repository: `helm-prod-values` (Your App Config)
```yaml
# myapp/values.yaml
# Your app's specific configuration
@@ -248,13 +248,13 @@ metadata:
namespace: argocd
spec:
sources:
- repoURL: https://github.com/fortedigital/forte-helm
- repoURL: https://git.forteapps.net/Forte/forte-helm
path: forteapp
helm:
valueFiles:
- $values/myapp/values.yaml
- repoURL: git@github.com:fortedigital/helm-values.git
- repoURL: git@github.com:fortedigital/helm-prod-values.git
ref: values
destination:
@@ -316,10 +316,10 @@ Ensure your app repository has:
docker build -t ghcr.io/fortedigital/hello-world:${{ steps.version.outputs.VERSION }} .
docker push ghcr.io/fortedigital/hello-world:${{ steps.version.outputs.VERSION }}
- name: Update helm-values
- name: Update helm-prod-values
run: |
git clone git@github.com:fortedigital/helm-values.git
cd helm-values
git clone git@github.com:fortedigital/helm-prod-values.git
cd helm-prod-values
mkdir -p hello-world
cat > hello-world/values.yaml <<EOF
app:
@@ -334,7 +334,7 @@ Ensure your app repository has:
### Step 2: Create Helm Values
Create a folder in `helm-values` repository:
Create a folder in `helm-prod-values` repository:
```bash
cd ~/dev/k8s/helm-prod-values
@@ -412,7 +412,7 @@ spec:
sources:
# Source 1: Helm chart templates
- repoURL: https://github.com/fortedigital/forte-helm
- repoURL: https://git.forteapps.net/Forte/forte-helm
path: forteapp
targetRevision: HEAD
helm:
@@ -420,7 +420,7 @@ spec:
- $values/hello-world/values.yaml
# Source 2: Helm values
- repoURL: git@github.com:fortedigital/helm-values.git
- repoURL: git@github.com:fortedigital/helm-prod-values.git
targetRevision: HEAD
ref: values
@@ -528,7 +528,7 @@ git push origin main
2. ✅ Builds new Docker image
3. ✅ Tags with new version (e.g., `v20260316-143022`)
4. ✅ Pushes to container registry
5. ✅ Updates `helm-values/myapp/values.yaml` with new tag
5. ✅ Updates `helm-prod-values/myapp/values.yaml` with new tag
6. ✅ ArgoCD detects change
7. ✅ Syncs new version to cluster
8. ✅ Sends Slack notification
@@ -683,7 +683,7 @@ git push
#### Step 4: Reference Secret in Application
Update your `helm-values/myapp/values.yaml`:
Update your `helm-prod-values/myapp/values.yaml`:
```yaml
app:
@@ -791,7 +791,7 @@ Three authentication modes are supported:
#### Step 1: Configure Helm Values
```yaml
# In helm-values/myapp/values.yaml
# In helm-prod-values/myapp/values.yaml
auth:
enabled: true
type: token # Token mode (default)
@@ -913,7 +913,7 @@ rm private/myapp-auth-oidc.yaml
#### Step 3: Configure Helm Values
```yaml
# In helm-values/myapp/values.yaml
# In helm-prod-values/myapp/values.yaml
auth:
enabled: true
type: oidc # OIDC mode
@@ -1049,7 +1049,7 @@ policies.forteapps.io/auth-image-version: "v1.2.3"
#### Example 1: Internal API with Token Auth
```yaml
# helm-values/internal-api/values.yaml
# helm-prod-values/internal-api/values.yaml
app:
image:
repository: ghcr.io/company/internal-api
@@ -1077,7 +1077,7 @@ curl -H "Authorization: Bearer d4f88f..." \
#### Example 2: User-Facing App with OIDC
```yaml
# helm-values/web-app/values.yaml
# helm-prod-values/web-app/values.yaml
app:
image:
repository: ghcr.io/company/web-app
@@ -1112,7 +1112,7 @@ kubectl create secret generic auth-oidc \
#### Example 3: MCP Server with OAuth 2.0
```yaml
# helm-values/mcp-server/values.yaml
# helm-prod-values/mcp-server/values.yaml
app:
image:
repository: ghcr.io/company/mcp-server
@@ -1136,7 +1136,7 @@ The MCP auth mode implements RFC 9728 (OAuth 2.0 Protected Resource Metadata) fo
#### Example 4: Disabling Authentication
```yaml
# helm-values/public-api/values.yaml
# helm-prod-values/public-api/values.yaml
auth:
enabled: false # No authentication
@@ -1500,7 +1500,7 @@ kubectl exec -n myapp <pod-name> -- env
# Check if secrets exist
kubectl get secrets -n myapp
# Increase resources in helm-values
# Increase resources in helm-prod-values
vim ~/dev/k8s/helm-prod-values/myapp/values.yaml
```
@@ -1649,7 +1649,7 @@ If you're stuck:
### Configuration Management
✅ **DO**:
- Keep configuration in `helm-values` repository
- Keep configuration in `helm-prod-values` repository
- Use environment variables for config
- Document what each value does
- Use reasonable resource limits

View File

@@ -47,7 +47,7 @@ This Kubernetes cluster uses a **GitOps approach** powered by **ArgoCD**, where
│ │ │
│ │ │
└────────► Update image tag ─┴──────────────────────────┘
in helm-values │
in helm-prod-values │
┌────────────────────────────────┐
@@ -184,7 +184,7 @@ launchpad/
---
### 2. **Helm Charts Repository**
**Repository**: `https://github.com/fortedigital/forte-helm`
**Repository**: `https://git.forteapps.net/Forte/forte-helm`
**Purpose**: Reusable Helm chart templates for Forte applications
**Location**: `C:\dev\k8s\forte-helm`
@@ -218,7 +218,7 @@ forte-helm/
---
### 3. **Helm Values Repository**
**Repository**: `git@github.com:fortedigital/helm-values.git`
**Repository**: `git@github.com:fortedigital/helm-prod-values.git`
**Purpose**: Environment-specific configuration for each application
**Location**: `C:\dev\k8s\helm-prod-values`
@@ -279,7 +279,7 @@ app-repository/
2. Build Docker image
3. Tag with version (e.g., `v2.0.4`)
4. Push to container registry (GHCR, Docker Hub, etc.)
5. Update image tag in `helm-values` repository
5. Update image tag in `helm-prod-values` repository
6. ArgoCD detects change and syncs automatically
---
@@ -340,13 +340,13 @@ Applications like `mcp10x` and `musicman` use multiple sources:
```yaml
spec:
sources:
- repoURL: https://github.com/fortedigital/forte-helm
- repoURL: https://git.forteapps.net/Forte/forte-helm
path: forteapp # Helm chart templates
helm:
valueFiles:
- $values/mcp10x/values.yaml # Reference to second source
- repoURL: git@github.com:fortedigital/helm-values.git
- repoURL: git@github.com:fortedigital/helm-prod-values.git
targetRevision: HEAD
ref: values # Named reference
```
@@ -414,8 +414,8 @@ jobs:
- name: Update Helm values
run: |
git clone git@github.com:fortedigital/helm-values.git
cd helm-values/app
git clone git@github.com:fortedigital/helm-prod-values.git
cd helm-prod-values/app
sed -i "s/tag: .*/tag: $VERSION/" values.yaml
git commit -am "Update app to $VERSION"
git push
@@ -432,7 +432,7 @@ jobs:
- Syncs application to cluster
2. **Helm Values Change**:
- CI/CD updates `helm-values/myapp/values.yaml`
- CI/CD updates `helm-prod-values/myapp/values.yaml`
- ArgoCD detects change
- Pulls new Helm chart with updated values
- Applies to cluster
@@ -639,7 +639,7 @@ Notifications include:
✅ **DO**:
- Follow the `forteapp` chart pattern
- Use semantic versioning for image tags
- Update helm-values via CI/CD
- Update helm-prod-values via CI/CD
- Test locally with Docker Compose
- Document environment variables

View File

@@ -85,7 +85,8 @@ kubectl get applications -n argocd
1. **Configure DNS** for ingress domains:
- `argocd.127.0.0.1.nip.io` (local dev)
- `*.forteapps.net` (production)
- `*.forteapps.net` (dev)
- `*.fortedigital.com` (production)
2. **Verify Let's Encrypt certificates**:
```bash
@@ -107,7 +108,7 @@ kubectl get applications -n argocd
### ArgoCD Repository Access Setup
ArgoCD needs SSH access to private Git repositories to pull manifests and Helm values. This section covers setting up deploy keys for GitHub repositories.
ArgoCD needs SSH access to private Git repositories to pull manifests and Helm values. This section covers setting up deploy keys for Gitea repositories.
#### Why Deploy Keys?
@@ -119,7 +120,7 @@ ArgoCD needs SSH access to private Git repositories to pull manifests and Helm v
#### Prerequisites
- kubectl access to the cluster
- Write access to the GitHub repository
- Write access to the Gitea repository
- ArgoCD installed and running
#### Setup Procedure
@@ -138,16 +139,16 @@ ssh-keygen -t rsa -b 4096 -C "argocd-deploy-key-launchpad" -f argocd-deploy-key
This creates two files:
- `argocd-deploy-key` - Private key (keep secret)
- `argocd-deploy-key.pub` - Public key (add to GitHub)
- `argocd-deploy-key.pub` - Public key (add to Gitea)
**Step 2: Add Public Key to GitHub**
**Step 2: Add Public Key to Gitea**
1. Copy the public key:
```bash
cat argocd-deploy-key.pub
```
2. Go to GitHub repository settings:
2. Go to Gitea repository settings:
- Navigate to: `https://git.forteapps.net/Forte/launchpad/settings/keys`
- Or: Repository → Settings → Deploy keys
@@ -157,12 +158,12 @@ This creates two files:
- ☐ Allow write access (leave unchecked - read-only is sufficient)
- Click **"Add key"**
4. Repeat for the `helm-values` repository if it's private:
4. Repeat for the `helm-prod-values` repository if it's private:
```bash
# Generate separate key for helm-values repo
ssh-keygen -t ed25519 -C "argocd-deploy-key-helm-values" -f argocd-helm-values-key -N ""
# Generate separate key for helm-prod-values repo
ssh-keygen -t ed25519 -C "argocd-deploy-key-helm-prod-values" -f argocd-helm-prod-values-key -N ""
# Add to: https://github.com/fortedigital/helm-values/settings/keys
# Add to: https://git.forteapps.net/Forte/helm-prod-values/settings/keys
```
**Step 3: Create Kubernetes Secret**
@@ -270,7 +271,7 @@ rm /tmp/test-repo-access.yaml
# Generate new key
ssh-keygen -t ed25519 -C "argocd-deploy-key-$(date +%Y%m)" -f argocd-new-key -N ""
# Add new public key to GitHub (keep old key for now)
# Add new public key to Gitea (keep old key for now)
# Update Kubernetes secret
kubectl create secret generic repo-launchpad \
@@ -278,7 +279,7 @@ rm /tmp/test-repo-access.yaml
--namespace=argocd \
--dry-run=client -o yaml | kubectl apply -f -
# Test access, then remove old deploy key from GitHub
# Test access, then remove old deploy key from Gitea
# Clean up
shred -u argocd-new-key
@@ -289,7 +290,7 @@ rm /tmp/test-repo-access.yaml
# List all repository secrets
kubectl get secrets -n argocd -l argocd.argoproj.io/secret-type=repository
# Review deploy keys in GitHub
# Review deploy keys in Gitea
# Visit: https://git.forteapps.net/Forte/launchpad/settings/keys
```
@@ -312,16 +313,16 @@ kubectl get secret repo-launchpad -n argocd -o yaml | grep argocd.argoproj.io/se
# Check ArgoCD application controller logs
kubectl logs -n argocd deployment/argocd-application-controller | grep -i "permission denied"
# Verify deploy key is added to GitHub
# Verify deploy key is added to Gitea
# Visit: https://git.forteapps.net/Forte/launchpad/settings/keys
```
**Issue: "Host key verification failed"**
```bash
# Add GitHub to known_hosts
# Add Gitea to known_hosts
kubectl exec -n argocd deployment/argocd-repo-server -- \
ssh-keyscan github.com >> ~/.ssh/known_hosts
ssh-keyscan git.forteapps.net >> ~/.ssh/known_hosts
# Or disable strict host key checking (less secure)
kubectl patch secret repo-launchpad -n argocd \
@@ -346,16 +347,16 @@ kubectl rollout restart deployment argocd-application-controller -n argocd
#### Multiple Repository Setup
For the three-repository pattern (launchpad, forte-helm, helm-values):
For the three-repository pattern (launchpad, forte-helm, helm-prod-values):
```bash
# 1. launchpad (main config repo)
ssh-keygen -t ed25519 -C "argocd-launchpad" -f key-sturdy -N ""
# Add key-sturdy.pub to: https://git.forteapps.net/Forte/launchpad/settings/keys
# 2. helm-values (private values repo)
ssh-keygen -t ed25519 -C "argocd-helm-values" -f key-helm-values -N ""
# Add key-helm-values.pub to: https://github.com/fortedigital/helm-values/settings/keys
# 2. helm-prod-values (private values repo)
ssh-keygen -t ed25519 -C "argocd-helm-prod-values" -f key-helm-prod-values -N ""
# Add key-helm-prod-values.pub to: https://git.forteapps.net/Forte/helm-prod-values/settings/keys
# 3. forte-helm (private helm charts repo)
@@ -366,14 +367,14 @@ kubectl create secret generic repo-launchpad \
kubectl label --local -f - argocd.argoproj.io/secret-type=repository --dry-run=client -o yaml | \
kubectl apply -f -
kubectl create secret generic repo-helm-values \
--from-file=sshPrivateKey=key-helm-values \
kubectl create secret generic repo-helm-prod-values \
--from-file=sshPrivateKey=key-helm-prod-values \
--namespace=argocd --dry-run=client -o yaml | \
kubectl label --local -f - argocd.argoproj.io/secret-type=repository --dry-run=client -o yaml | \
kubectl apply -f -
# Clean up keys
shred -u key-sturdy key-helm-values
shred -u key-sturdy key-helm-prod-values
```
#### Converting HTTPS to SSH
@@ -390,7 +391,7 @@ If you're currently using HTTPS and want to switch to SSH:
# repoURL: ssh://git@git.forteapps.net:2222/Forte/launchpad.git
# 3. Update and commit
find . -name "*.yaml" -type f -exec sed -i 's|https://github.com/fortedigital/|git@github.com:fortedigital/|g' {} +
find . -name "*.yaml" -type f -exec sed -i 's|https://git.forteapps.net/Forte/|git@git.forteapps.net:Forte/|g' {} +
git add .
git commit -m "Switch from HTTPS to SSH for repository access"
@@ -494,7 +495,7 @@ spec:
See [Developer Guide](DEVELOPER-GUIDE.md#deploying-your-first-application) for detailed steps.
**Quick checklist:**
- [ ] Create `helm-values/myapp/values.yaml`
- [ ] Create `helm-prod-values/myapp/values.yaml`
- [ ] Create `apps/myapp.yaml` in config repo
- [ ] Create SealedSecret if needed
- [ ] Commit and push changes
@@ -559,7 +560,7 @@ kubectl scale deployment myapp -n myapp --replicas=3
#### GitOps Scaling
Update `helm-values/myapp/values.yaml`:
Update `helm-prod-values/myapp/values.yaml`:
```yaml
app:
@@ -573,7 +574,7 @@ Commit and push - ArgoCD will sync.
Enable Horizontal Pod Autoscaler:
```yaml
# In helm-values/myapp/values.yaml
# In helm-prod-values/myapp/values.yaml
app:
hpa:
enabled: true
@@ -622,7 +623,7 @@ kubectl rollout undo deployment myapp -n myapp
#### Option 3: Change Image Tag
```bash
# Edit helm-values
# Edit helm-prod-values
cd ~/dev/k8s/helm-prod-values
vim myapp/values.yaml
@@ -642,7 +643,7 @@ git push
#### Update Resource Limits
```yaml
# In helm-values/myapp/values.yaml
# In helm-prod-values/myapp/values.yaml
app:
resources:
requests:
@@ -656,7 +657,7 @@ app:
#### Enable Database
```yaml
# In helm-values/myapp/values.yaml
# In helm-prod-values/myapp/values.yaml
db:
enabled: true
persistence:
@@ -1266,7 +1267,7 @@ spec:
**What Needs Backup**:
- ❌ Cluster state (not backed up - recreate via GitOps)
- ❌ Persistent volumes (currently not critical)
- ✅ Git repositories (GitHub provides backup)
- ✅ Git repositories (Gitea provides backup)
- ⚠️ Secrets (sealed secrets in Git, unseal keys need safekeeping)
### Cluster Rebuild
@@ -1561,7 +1562,7 @@ git push
kubectl scale deployment myapp -n myapp --replicas=0
# Update Git
vim helm-values/myapp/values.yaml
vim helm-prod-values/myapp/values.yaml
# Set replicaCount: 0
git commit -am "Scale down myapp for maintenance"
git push
@@ -1634,7 +1635,7 @@ echo "Remember to delete: $SECRET_FILE"
- [ ] Application code repository created
- [ ] Dockerfile created and tested
- [ ] GitHub Actions workflow configured
- [ ] Gitea Actions workflow configured
- [ ] Helm values created in `helm-prod-values/`
- [ ] ArgoCD application manifest created in `apps/`
- [ ] Secrets created and sealed

View File

@@ -190,7 +190,7 @@ spec:
### Helm Charts Repository: `forte-helm`
**URL**: `https://github.com/fortedigital/forte-helm`
**URL**: `https://git.forteapps.net/Forte/forte-helm`
#### Chart: `forteapp`
@@ -337,14 +337,14 @@ configmap: [] # Application ConfigMap key-value pairs
---
### Helm Values Repository: `helm-values`
### Helm Values Repository: `helm-prod-values`
**URL**: `https://github.com/fortedigital/helm-values.git`
**URL**: `https://git.forteapps.net/Forte/helm-prod-values.git`
#### Structure
```
helm-values/
helm-prod-values/
├── mcp10x/
│ └── values.yaml
├── musicman/
@@ -526,14 +526,14 @@ spec:
# Multi-source configuration
sources:
- repoURL: https://github.com/fortedigital/forte-helm
- repoURL: https://git.forteapps.net/Forte/forte-helm
path: forteapp
targetRevision: HEAD
helm:
valueFiles:
- $values/<app-name>/values.yaml
- repoURL: git@github.com:fortedigital/helm-values.git
- repoURL: git@github.com:fortedigital/helm-prod-values.git
targetRevision: HEAD
ref: values