launchpad/APPLICATIONSET_GUIDE.md

# ApplicationSet Guide

## Overview

This repository uses **ApplicationSet** (instead of the traditional app-of-apps pattern) to manage infrastructure applications. ApplicationSet automatically discovers and creates ArgoCD Applications based on a git directory pattern.

## How It Works

### ApplicationSet Definition
Located in `argocd/_app-of-apps.yaml`:

```yaml
apiVersion: argoproj.io/v1alpha1
kind: ApplicationSet
metadata:
  name: infrastructure-apps
  namespace: argocd
spec:
  goTemplate: true
  generators:
  - git:
      repoURL: https://github.com/fortedigital/sturdy-adventure.git
      revision: HEAD
      directories:
      - path: argocd/infra/*.yaml
  template:
    # Template applied to each discovered file
```

### Key Components

1. **Generator**: Scans `argocd/infra/*.yaml` for all Application manifests
2. **Path Variables**: Each match provides template variables:
   - `{{ .path.basenameNormalized }}` - filename without extension (e.g., "prometheus")
   - `{{ .path.dir }}` - directory path (e.g., "argocd/infra")
3. **Template**: Used to create consistent Applications from the discovered files

## How Applications Are Created

When you create a new file `argocd/infra/my-app.yaml`:

```yaml
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: my-app
  # ... other config
spec:
  # ... your application spec
```

The ApplicationSet:
1. Discovers the file matching `argocd/infra/*.yaml`
2. Extracts metadata: `basenameNormalized=my-app`, `dir=argocd/infra`
3. Renders the template with these variables
4. Creates an Application resource with:
   - Name: `my-app`
   - Labels applied from the template
   - Sync policy inherited from template
   - Source path: `argocd/infra`

## Adding New Applications

### Step 1: Create Application YAML
Create `argocd/infra/my-new-app.yaml`:

```yaml
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: my-new-app
  annotations:
    argocd.argoproj.io/sync-wave: "2"  # Optional: control sync ordering
spec:
  project: default
  source:
    repoURL: https://my-helm-repo.com
    chart: my-chart
    targetRevision: "1.0.0"
    helm:
      releaseName: my-new-app
  destination:
    server: https://kubernetes.default.svc
    namespace: my-namespace
  # Note: syncPolicy is applied from the ApplicationSet template
  # No need to duplicate it here
```

### Step 2: Push to Repository
```bash
git add argocd/infra/my-new-app.yaml
git commit -m "Add my-new-app infrastructure application"
git push
```

### Step 3: Verify Discovery
```bash
# View all generated applications
kubectl get applications -n argocd

# Check ApplicationSet status
kubectl describe applicationset infrastructure-apps -n argocd

# Watch for your new application
kubectl get application my-new-app -n argocd -w
```

## Template Variables

The ApplicationSet template uses these variables for each discovered file:

| Variable | Example | Description |
|----------|---------|-------------|
| `{{ .path.basenameNormalized }}` | `prometheus` | Filename without extension |
| `{{ .path.dir }}` | `argocd/infra` | Directory containing the file |
| `{{ .path.path }}` | `argocd/infra/prometheus.yaml` | Full path to the file |

## ApplicationSet Features Used

### 1. Git Directory Generator
```yaml
generators:
- git:
    repoURL: https://github.com/fortedigital/sturdy-adventure.git
    revision: HEAD
    directories:
    - path: argocd/infra/*.yaml  # Match all YAML files
```

Automatically discovers applications in the git repository.

### 2. Go Templating
```yaml
goTemplate: true
```

Enables Go template syntax for variable interpolation (e.g., `{{ .path.basenameNormalized }}`).

### 3. Dynamic Application Generation
```yaml
template:
  metadata:
    name: "{{ .path.basenameNormalized }}"
    labels:
      app.kubernetes.io/name: "{{ .path.basenameNormalized }}"
```

Each discovered file generates an Application with consistent configuration.

## Benefits Over App-of-Apps Pattern

| Feature | ApplicationSet | App-of-Apps |
|---------|---|---|
| Auto-discovery | ✅ Automatic | ❌ Manual list required |
| New app onboarding | 1 file created | 1 file created + parent update |
| Consistency | ✅ Template enforced | ❌ Manual consistency |
| Scalability | ✅ Grows automatically | ❌ Manual maintenance |
| RBAC per app | ✅ Supported | ❌ Limited |
| Drift detection | ✅ Per app | ✅ Per app |

## Sync Policy Applied by ApplicationSet

All generated Applications inherit this sync policy from the template:

```yaml
syncPolicy:
  automated:
    prune: true
    selfHeal: true
  syncOptions:
  - CreateNamespace=true
  - Validate=true
  - ServerSideApply=true
  timeout: 300s
  retry:
    limit: 5
    backoff:
      duration: 5s
      factor: 2
      maxDuration: 3m
```

**What this means:**
- ✅ Automatic syncing when repository changes
- ✅ Automatic pruning of deleted resources
- ✅ Self-healing if cluster drifts from git
- ✅ Namespace auto-creation if missing
- ✅ Manifest validation before applying
- ✅ Server-side apply for field ownership
- ✅ 5-minute timeout per application sync
- ✅ Up to 5 retry attempts with exponential backoff

## Monitoring ApplicationSet

### View ApplicationSet Status
```bash
kubectl describe applicationset infrastructure-apps -n argocd
```

Output shows:
- Condition status (Healthy, Progressing, etc.)
- Number of applications created
- Last sync time
- Error messages if any

### View Generated Applications
```bash
# All applications created by this ApplicationSet
kubectl get applications -n argocd -l app.kubernetes.io/managed-by=argocd

# View a specific application
kubectl get application prometheus -n argocd -o yaml
```

### Check Sync Status
```bash
# Watch all applications
kubectl get applications -n argocd -w

# Detailed sync status
argocd app list

# Application health
kubectl get applications -n argocd -o wide
```

## Troubleshooting

### ApplicationSet Not Creating Applications

**Problem**: Files in `argocd/infra/` but no Applications created

**Solutions**:
1. Check file naming: Must end with `.yaml` (not `.yml`)
2. Verify path: Files must be in `argocd/infra/` (not subdirectories)
3. Check permissions: Repository URL must be accessible
4. Review ApplicationSet status: `kubectl describe applicationset infrastructure-apps -n argocd`

### Application Created But Not Syncing

**Problem**: Application exists but stays in "OutOfSync" state

**Solutions**:
1. Check application spec is valid YAML
2. Verify `destination.server` is accessible: `https://kubernetes.default.svc`
3. Check `destination.namespace` exists or `CreateNamespace=true` is set
4. Review application logs: `kubectl logs -n argocd deployment/argocd-application-controller`

### ApplicationSet Generates Duplicate Applications

**Problem**: Same application created multiple times

**Solutions**:
1. Check for duplicate files in `argocd/infra/` with same name
2. Clear git cache: `git clean -fd`
3. ApplicationSet may take time to reconcile; wait 60 seconds

## Best Practices

1. **File Organization**
   - Keep only Application manifests in `argocd/infra/`
   - Don't mix other resource types (ConfigMaps, Secrets, etc.)
   - Use consistent naming: `lowercase-with-hyphens.yaml`

2. **Application Design**
   - Keep each Application spec in the discovered file
   - Don't rely on syncPolicy being defined in the Application (it comes from ApplicationSet)
   - Use sync waves for dependency ordering

3. **Repository Management**
   - All files in `argocd/infra/` should be valid Kubernetes manifests
   - Regular commits to track changes
   - Use branches for testing new applications before merging

4. **Monitoring**
   - Regularly check ApplicationSet status
   - Monitor generated applications for sync status
   - Set up alerts for Failed or Degraded applications

5. **Updates**
   - Update application specs directly in `argocd/infra/` files
   - ApplicationSet changes take effect within 60 seconds
   - Test in dev environment first

## Related Documentation

- [ArgoCD ApplicationSet Docs](https://argocd-applicationset.readthedocs.io/)
- [ArgoCD Application Spec](https://argo-cd.readthedocs.io/en/stable/operator-manual/declarative-setup/#applications)
- [Kubernetes Application Convention](https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/#recommended-labels)