diff --git a/docs/DEVELOPER-GUIDE.md b/docs/DEVELOPER-GUIDE.md index dc06e3e..f812c5b 100644 --- a/docs/DEVELOPER-GUIDE.md +++ b/docs/DEVELOPER-GUIDE.md @@ -9,6 +9,7 @@ - [Updating an Existing Application](#updating-an-existing-application) - [Working with Secrets](#working-with-secrets) - [Enabling Authentication for Applications](#enabling-authentication-for-applications) +- [Adding a New Keycloak Client](#adding-a-new-keycloak-client) - [Troubleshooting](#troubleshooting) - [Best Practices](#best-practices) @@ -1247,6 +1248,135 @@ kubectl logs -n myapp -c authn --- +## Adding a New Keycloak Client + +When you need an application to authenticate via Keycloak (OIDC), you can add a client definition to the realm config. The secret syncer automatically extracts the Keycloak-generated client secret into a Kubernetes Secret that your application can reference — no manual secret management needed. + +### How It Works + +1. You define a client in `forte-realm.json` (inside `keycloak-values.yaml`) **without** a `secret` field +2. Keycloak auto-generates a cryptographically strong secret on first creation +3. An ArgoCD **PostSync Job** (`keycloak-secret-syncer`) runs after each Keycloak sync: + - Authenticates to the Keycloak Admin API + - Finds clients with `k8s.secret.sync: "true"` in their attributes + - Extracts the auto-generated secret for each client + - Creates/updates a K8s Secret in the target namespace with `client-id` and `client-secret` keys +4. Your application references the syncer-created Secret + +### Step 1: Add Client to Realm Config + +In `infra/values/keycloak-values.yaml`, add a new entry to the `clients` array in `forte-realm.json`: + +```json +{ + "clientId": "myapp", + "name": "My Application", + "enabled": true, + "protocol": "openid-connect", + "clientAuthenticatorType": "client-secret", + "standardFlowEnabled": true, + "directAccessGrantsEnabled": false, + "publicClient": false, + "redirectUris": ["https://myapp.forteapps.net/*"], + "webOrigins": ["https://myapp.forteapps.net"], + "defaultClientScopes": ["openid", "email", "profile"], + "attributes": { + "k8s.secret.sync": "true", + "k8s.secret.namespace": "myapp", + "k8s.secret.name": "myapp-oidc-credentials" + } +} +``` + +**Important**: +- Do **NOT** include a `"secret"` field — Keycloak generates one automatically +- The `attributes` block tells the syncer where to create the K8s Secret +- The target namespace must exist before the syncer runs (ArgoCD creates it via `CreateNamespace=true`) + +### Step 2: Reference the Secret in Your Application + +In your application's Helm values, reference the syncer-created secret: + +```yaml +# In helm-values/myapp/values.yaml (or inline in values file) +# The secret will have keys: client-id, client-secret +existingSecret: myapp-oidc-credentials +key: client-secret +``` + +For Gitea-style oauth config: +```yaml +oauth: +- name: "Forte" + provider: "openidConnect" + existingSecret: myapp-oidc-credentials + key: client-secret + autoDiscoverUrl: "https://id.forteapps.net/realms/forte/.well-known/openid-configuration" +``` + +### Step 3: Commit and Push + +```bash +cd ~/dev/k8s/launchpad +git add infra/values/keycloak-values.yaml +git commit -m "Add myapp Keycloak client with auto-sync" +git push +``` + +ArgoCD will: +1. Sync the Keycloak config (keycloakConfigCli creates the client) +2. Run the PostSync syncer Job +3. The syncer creates `myapp-oidc-credentials` in the `myapp` namespace + +### Step 4: Verify + +```bash +# Check the syncer job ran successfully +kubectl get jobs -n keycloak +kubectl logs -n keycloak job/keycloak-secret-syncer + +# Verify the secret was created +kubectl get secret myapp-oidc-credentials -n myapp -o yaml + +# Check the secret has the expected keys +kubectl get secret myapp-oidc-credentials -n myapp -o jsonpath='{.data.client-id}' | base64 -d +kubectl get secret myapp-oidc-credentials -n myapp -o jsonpath='{.data.client-secret}' | base64 -d +``` + +### Sync Attribute Reference + +| Attribute | Required | Description | +|-----------|----------|-------------| +| `k8s.secret.sync` | Yes | Set to `"true"` to enable syncing | +| `k8s.secret.namespace` | Yes | Target K8s namespace for the secret | +| `k8s.secret.name` | Yes | Name of the K8s Secret to create | + +### Retrieving Secrets for External Deployments + +The syncer always writes a **central copy** of every synced secret to the `secrets` namespace, in addition to the target namespace. This allows operators to retrieve client credentials for applications deployed outside this cluster: + +```bash +# View the central copy +kubectl get secret gitea-oidc-credentials -n secrets -o yaml + +# Extract the client secret for use elsewhere +kubectl get secret myapp-oidc-credentials -n secrets \ + -o jsonpath='{.data.client-secret}' | base64 -d +``` + +This is useful when an application runs on a separate cluster or external infrastructure and needs the Keycloak-generated OIDC credentials provisioned manually (e.g., via a SealedSecret on the remote side). + +### Syncer Behavior Notes + +- The syncer runs as an ArgoCD **PostSync hook** — it executes after all Keycloak resources are healthy +- `BeforeHookCreation` delete policy ensures old Job is cleaned up before each run +- If the target namespace doesn't exist, the target write is skipped with a warning (the central copy still happens) +- A central copy is **always** written to the `secrets` namespace for every synced client +- The syncer uses the `keycloak-credentials` secret for admin authentication +- Created secrets have the label `app.kubernetes.io/managed-by: keycloak-secret-syncer` + +--- + ## Troubleshooting ### Application Not Deploying @@ -1579,4 +1709,4 @@ Now that you understand the basics: - Docs: [Full documentation index](README.md) - Help: Contact platform team -**Last Updated**: 2026-03-16 +**Last Updated**: 2026-04-16 diff --git a/docs/REFERENCE.md b/docs/REFERENCE.md index 715e554..3cee384 100644 --- a/docs/REFERENCE.md +++ b/docs/REFERENCE.md @@ -869,6 +869,65 @@ dind: - Gitea admin panel (`/admin/runners`) — runners show as Online - Create test workflow in `.gitea/workflows/test.yml` — job executes +### Keycloak Secret Syncer + +**Type**: ArgoCD PostSync Job (deployed via Keycloak Helm chart `extraDeploy`) +**Namespace**: `keycloak` + +**Purpose**: Automatically extracts Keycloak-generated client secrets and syncs them into Kubernetes Secrets in target namespaces. Eliminates the need to manually manage OIDC client secrets. + +**How It Works**: +1. Runs as an ArgoCD PostSync hook after Keycloak resources are healthy +2. Authenticates to Keycloak Admin API using admin credentials from `keycloak-credentials` secret +3. Queries all clients in the `forte` realm +4. Filters clients with `k8s.secret.sync: "true"` attribute +5. For each matching client, retrieves the auto-generated secret via Keycloak Admin API +6. Creates/updates a K8s Secret in the target namespace (from `k8s.secret.namespace` attribute) +7. Always writes a central copy to the `secrets` namespace (for external deployment retrieval) + +**Resources**: +- `ServiceAccount`: `keycloak-secret-syncer` (namespace: `keycloak`) +- `ClusterRole`: `keycloak-secret-syncer` (secrets: get/create/update/patch; namespaces: get/list) +- `ClusterRoleBinding`: `keycloak-secret-syncer` +- `Job`: `keycloak-secret-syncer` (PostSync hook) + +**Client Attributes** (set in `forte-realm.json`): + +| Attribute | Description | +|-----------|-------------| +| `k8s.secret.sync` | Set to `"true"` to enable syncing | +| `k8s.secret.namespace` | Target K8s namespace | +| `k8s.secret.name` | Name of the K8s Secret | + +**Created Secret Format**: +```yaml +apiVersion: v1 +kind: Secret +metadata: + name: + namespace: + labels: + app.kubernetes.io/managed-by: keycloak-secret-syncer +type: Opaque +data: + client-id: + client-secret: +``` + +**Verification**: +```bash +# Check job status +kubectl get jobs -n keycloak + +# View syncer logs +kubectl logs -n keycloak job/keycloak-secret-syncer + +# Verify created secret +kubectl get secret -n -o yaml +``` + +**See**: [Developer Guide - Adding a New Keycloak Client](DEVELOPER-GUIDE.md#adding-a-new-keycloak-client) + ### Renovate **Chart**: `renovate` (OCI: `ghcr.io/renovatebot/charts`) @@ -1528,6 +1587,6 @@ team: platform --- -**Last Updated**: 2026-04-14 +**Last Updated**: 2026-04-16 **Maintained By**: Platform Team **Version**: 1.0.0 diff --git a/infra/keycloak.yaml b/infra/keycloak.yaml index 66b38fa..be9d5ef 100644 --- a/infra/keycloak.yaml +++ b/infra/keycloak.yaml @@ -40,3 +40,9 @@ spec: - CreateNamespace=true - Validate=true - ServerSideApply=true + + ignoreDifferences: + - group: batch + kind: Job + jsonPointers: + - /spec/template/spec/containers/0/args diff --git a/infra/values/gitea-values.yaml b/infra/values/gitea-values.yaml index d5a33cf..b20f588 100644 --- a/infra/values/gitea-values.yaml +++ b/infra/values/gitea-values.yaml @@ -69,8 +69,8 @@ gitea: oauth: - name: "Forte" provider: "openidConnect" - existingSecret: gitea-credentials - key: gitea + existingSecret: gitea-oidc-credentials + key: client-secret autoDiscoverUrl: "https://id.forteapps.net/realms/forte/.well-known/openid-configuration" scopes: "openid email profile organization" groupClaimName: "groups" diff --git a/infra/values/keycloak-values.yaml b/infra/values/keycloak-values.yaml index 3f07394..d709513 100644 --- a/infra/values/keycloak-values.yaml +++ b/infra/values/keycloak-values.yaml @@ -72,13 +72,17 @@ keycloakConfigCli: "enabled": true, "protocol": "openid-connect", "clientAuthenticatorType": "client-secret", - "secret": "382ed413580cb79d0f54813e5da87007b28fe766a8903d378b9e1c266405a784", "standardFlowEnabled": true, "directAccessGrantsEnabled": false, "publicClient": false, "redirectUris": ["https://git.forteapps.net/*"], "webOrigins": ["https://git.forteapps.net"], "defaultClientScopes": ["openid", "email", "profile"], + "attributes": { + "k8s.secret.sync": "true", + "k8s.secret.namespace": "gitea", + "k8s.secret.name": "gitea-oidc-credentials" + }, "protocolMappers": [ { "name": "email_verified", @@ -97,3 +101,210 @@ keycloakConfigCli: } ] } + +extraDeploy: +# -- ServiceAccount for the secret syncer Job +- apiVersion: v1 + kind: ServiceAccount + metadata: + name: keycloak-secret-syncer + namespace: keycloak + +# -- ClusterRole granting access to secrets and namespaces +- apiVersion: rbac.authorization.k8s.io/v1 + kind: ClusterRole + metadata: + name: keycloak-secret-syncer + rules: + - apiGroups: [""] + resources: ["secrets"] + verbs: ["get", "create", "update", "patch"] + - apiGroups: [""] + resources: ["namespaces"] + verbs: ["get", "list"] + +# -- ClusterRoleBinding for the syncer ServiceAccount +- apiVersion: rbac.authorization.k8s.io/v1 + kind: ClusterRoleBinding + metadata: + name: keycloak-secret-syncer + roleRef: + apiGroup: rbac.authorization.k8s.io + kind: ClusterRole + name: keycloak-secret-syncer + subjects: + - kind: ServiceAccount + name: keycloak-secret-syncer + namespace: keycloak + +# -- PostSync Job: extracts Keycloak client secrets into K8s Secrets +- apiVersion: batch/v1 + kind: Job + metadata: + name: keycloak-secret-syncer + namespace: keycloak + annotations: + argocd.argoproj.io/hook: PostSync + argocd.argoproj.io/hook-delete-policy: BeforeHookCreation + spec: + backoffLimit: 3 + template: + spec: + serviceAccountName: keycloak-secret-syncer + restartPolicy: Never + containers: + - name: syncer + image: alpine:3.20 + command: ["/bin/sh", "-c"] + args: + - | + set -e + apk add --no-cache curl jq > /dev/null 2>&1 + + KEYCLOAK_URL="http://keycloak:80" + REALM="forte" + + # Read admin credentials from the keycloak-credentials secret + ADMIN_USER="admin" + ADMIN_PASS=$(cat /secrets/admin-password) + + # Authenticate to Keycloak Admin API + echo "Authenticating to Keycloak..." + TOKEN=$(curl -sf -X POST "${KEYCLOAK_URL}/realms/master/protocol/openid-connect/token" \ + -d "client_id=admin-cli" \ + -d "username=${ADMIN_USER}" \ + -d "password=${ADMIN_PASS}" \ + -d "grant_type=password" | jq -r '.access_token') + + if [ -z "$TOKEN" ] || [ "$TOKEN" = "null" ]; then + echo "ERROR: Failed to authenticate to Keycloak" + exit 1 + fi + + # Get all clients in the realm + echo "Fetching clients from realm '${REALM}'..." + CLIENTS=$(curl -sf -H "Authorization: Bearer ${TOKEN}" \ + "${KEYCLOAK_URL}/admin/realms/${REALM}/clients") + + # Filter clients with k8s.secret.sync=true + SYNC_CLIENTS=$(echo "$CLIENTS" | jq -c '[.[] | select(.attributes["k8s.secret.sync"] == "true")]') + COUNT=$(echo "$SYNC_CLIENTS" | jq 'length') + echo "Found ${COUNT} client(s) with sync enabled" + + K8S_API="https://kubernetes.default.svc" + SA_TOKEN=$(cat /var/run/secrets/kubernetes.io/serviceaccount/token) + CA_CERT="/var/run/secrets/kubernetes.io/serviceaccount/ca.crt" + CENTRAL_NS="secrets" + + # Upsert a K8s Secret: try PUT (update), fall back to POST (create) + upsert_secret() { + local ns="$1" name="$2" manifest="$3" + local code + code=$(curl -sf -o /dev/null -w "%{http_code}" \ + --cacert "$CA_CERT" \ + -H "Authorization: Bearer ${SA_TOKEN}" \ + -H "Content-Type: application/json" \ + -X PUT -d "$manifest" \ + "${K8S_API}/api/v1/namespaces/${ns}/secrets/${name}") + if [ "$code" = "200" ]; then + echo " Updated secret '${ns}/${name}'" + elif [ "$code" = "404" ]; then + code=$(curl -sf -o /dev/null -w "%{http_code}" \ + --cacert "$CA_CERT" \ + -H "Authorization: Bearer ${SA_TOKEN}" \ + -H "Content-Type: application/json" \ + -X POST -d "$manifest" \ + "${K8S_API}/api/v1/namespaces/${ns}/secrets") + if [ "$code" = "201" ]; then + echo " Created secret '${ns}/${name}'" + else + echo " ERROR: Failed to create secret '${ns}/${name}' (HTTP ${code})" + return 1 + fi + else + echo " ERROR: Failed to update secret '${ns}/${name}' (HTTP ${code})" + return 1 + fi + } + + # Build a Secret JSON manifest + build_manifest() { + local ns="$1" name="$2" b64_id="$3" b64_secret="$4" + cat < secret '${TARGET_NS}/${TARGET_NAME}'" + + # Get the client secret from Keycloak + SECRET_VALUE=$(curl -sf -H "Authorization: Bearer ${TOKEN}" \ + "${KEYCLOAK_URL}/admin/realms/${REALM}/clients/${CLIENT_UUID}/client-secret" \ + | jq -r '.value') + + if [ -z "$SECRET_VALUE" ] || [ "$SECRET_VALUE" = "null" ]; then + echo " WARNING: No secret found for client '${CLIENT_ID}', skipping" + continue + fi + + B64_CLIENT_ID=$(printf '%s' "$CLIENT_ID" | base64 | tr -d '\n') + B64_SECRET=$(printf '%s' "$SECRET_VALUE" | base64 | tr -d '\n') + + # 1. Write to target namespace (if it exists) + NS_STATUS=$(curl -sf -o /dev/null -w "%{http_code}" \ + --cacert "$CA_CERT" \ + -H "Authorization: Bearer ${SA_TOKEN}" \ + "${K8S_API}/api/v1/namespaces/${TARGET_NS}") + + if [ "$NS_STATUS" = "200" ]; then + MANIFEST=$(build_manifest "$TARGET_NS" "$TARGET_NAME" "$B64_CLIENT_ID" "$B64_SECRET") + upsert_secret "$TARGET_NS" "$TARGET_NAME" "$MANIFEST" || exit 1 + else + echo " WARNING: Namespace '${TARGET_NS}' does not exist, skipping target" + fi + + # 2. Always write a central copy to the secrets namespace + CENTRAL_MANIFEST=$(build_manifest "$CENTRAL_NS" "$TARGET_NAME" "$B64_CLIENT_ID" "$B64_SECRET") + upsert_secret "$CENTRAL_NS" "$TARGET_NAME" "$CENTRAL_MANIFEST" || exit 1 + done + + echo "Secret sync complete" + volumeMounts: + - name: keycloak-credentials + mountPath: /secrets + readOnly: true + resources: + requests: + cpu: 50m + memory: 64Mi + limits: + cpu: 200m + memory: 128Mi + volumes: + - name: keycloak-credentials + secret: + secretName: keycloak-credentials + items: + - key: admin-password + path: admin-password