Deploy App Workflow

End-to-end orchestration for deploying applications to the Kubernetes homelab with full monitoring integration.

Workflow Overview

┌─────────────────────────────────────────────────────────────────────┐
│                      /deploy-app Workflow                           │
├─────────────────────────────────────────────────────────────────────┤
│  1. RESEARCH                                                        │
│     ├─ Invoke kubesearch skill for real-world patterns              │
│     ├─ Check if native Helm chart exists (helm search hub)          │
│     ├─ Determine: native chart vs app-template                      │
│     └─ AskUserQuestion: Present findings, confirm approach          │
│                                                                     │
│  2. SETUP                                                           │
│     └─ task wt:new -- deploy-<app-name>                             │
│                                                                     │
│  3. CONFIGURE (in worktree)                                         │
│     ├─ kubernetes/platform/versions.env (add version)               │
│     ├─ kubernetes/platform/namespaces.yaml (add namespace)          │
│     ├─ kubernetes/platform/helm-charts.yaml (add input)             │
│     ├─ kubernetes/platform/charts/<app>.yaml (create values)        │
│     ├─ kubernetes/platform/kustomization.yaml (register)            │
│     └─ kubernetes/platform/config/<app>/ (optional extras)          │
│        ├─ route.yaml, canary.yaml, prometheus-rules.yaml            │
│                                                                     │
│  4. VALIDATE: task k8s:validate && task renovate:validate           │
│                                                                     │
│  5. TEST ON DEV (direct helm install, bypass Flux)                  │
│     ├─ Wait for pods ready, verify network + monitoring             │
│     └─ AskUserQuestion: Report status, confirm proceed              │
│                                                                     │
│  6. CLEANUP & PR                                                    │
│     ├─ helm uninstall, Flux reconcile-validate, commit, PR          │
│     └─ Report PR URL to user                                        │
└─────────────────────────────────────────────────────────────────────┘

---

Phase 1: Research

Invoke the kubesearch skill to find real-world chart configurations: /kubesearch <chart-name>

Check for a native chart: helm search hub <app-name> --max-col-width=100

Scenario	Approach
Official/community chart exists	Use native Helm chart
Only container image available	Use app-template
Chart is unmaintained (>1 year)	Consider app-template

Use AskUserQuestion to confirm: chart selection, exposure type (internal/external/none), namespace, persistence requirements.

---

Phase 2: Setup

Create an isolated worktree: task wt:new -- deploy-<app-name>

This creates branch deploy-<app-name> and worktree ../homelab-deploy-<app-name>/. Work exclusively in the worktree.

---

Phase 3: Configure

3.1 Add Version to versions.env

Add a version entry with a Renovate annotation. For annotation syntax and datasource selection, see the versions-renovate skill.

3.2 Add Namespace to namespaces.yaml

Add to kubernetes/platform/namespaces.yaml inputs array:

- name: <namespace>
  dataplane: ambient
  security: baseline       # restricted | baseline | privileged
  networkPolicy: false     # or object with profile/enforcement

PodSecurity level:

Level	Use When
restricted	Standard controllers, databases, simple apps — requires full security context on all containers
baseline	Apps needing elevated capabilities (e.g., NET_BIND_SERVICE)
privileged	Host access, BPF, device access

Network policy profile:

Profile	Use When
isolated	No inbound traffic needed
internal	Internal gateway only
internal-egress	Internal + calls external APIs
standard	Public-facing (both gateways + HTTPS egress)

Access labels (add as needed):

    access.network-policy.homelab/postgres: "true"
    access.network-policy.homelab/garage-s3: "true"
    access.network-policy.homelab/kube-api: "true"

For PostgreSQL provisioning, see the cnpg-database skill.

For Garage S3 storage, see Section 3.8 below — the access.network-policy.homelab/garage-s3 label alone is not sufficient.

3.3 Add to helm-charts.yaml

Add to kubernetes/platform/helm-charts.yaml inputs array:

- name: "<app-name>"
  namespace: "<namespace>"
  chart:
    name: "<chart-name>"
    version: "${<APP>_VERSION}"
    url: "https://charts.example.com"  # or oci://registry.io/path
  dependsOn: [cilium]

3.4 Create Values File

Create kubernetes/platform/charts/<app-name>.yaml. See references/file-templates.md for complete templates.

Security context for restricted namespaces (cert-manager, external-secrets, system, database, kromgo): add full restricted context to all containers. task k8s:validate does NOT catch PodSecurity violations — only admission time reveals them.

# Pod-level
podSecurityContext:       # key varies by chart
  runAsNonRoot: true
  seccompProfile:
    type: RuntimeDefault

# Container-level (every container and init container)
securityContext:
  allowPrivilegeEscalation: false
  capabilities:
    drop: ["ALL"]
  readOnlyRootFilesystem: true
  runAsNonRoot: true
  seccompProfile:
    type: RuntimeDefault

Check the image's default user — if it runs as root, add runAsUser: 65534.

3.5 Register in kustomization.yaml

Add to kubernetes/platform/kustomization.yaml configMapGenerator files list: - charts/<app-name>.yaml

3.6 Configure Renovate Tracking

Renovate tracks versions.env entries automatically via inline # renovate: annotations added in step 3.1. No changes to .github/renovate.json5 are needed unless adding grouping or automerge overrides. See the versions-renovate skill.

3.7 Optional: Additional Configuration

Create kubernetes/platform/config/<app-name>/ for extra resources. See references/file-templates.md for HTTPRoute, secret, and ExternalSecret templates. See references/monitoring-patterns.md for Canary, PrometheusRule, and Grafana dashboard examples.

For gateway routing and TLS, see the gateway-routing skill.

3.8 Garage S3 Object Storage

Garage S3 requires three resources across two locations. Missing any one causes an admission webhook denial.

Step 1: GarageBucket — in the cluster config, app namespace, under garage namespace:

# kubernetes/clusters/live/config/<app>/garage-bucket.yaml
apiVersion: garage.rajsingh.info/v1beta1
kind: GarageBucket
metadata:
  name: <app>
  namespace: garage
spec:
  clusterRef:
    name: garage

Step 2: GarageKey — in the cluster config, app namespace:

# kubernetes/clusters/live/config/<app>/garage-key.yaml
apiVersion: garage.rajsingh.info/v1beta1
kind: GarageKey
metadata:
  name: <app>
  namespace: <app>
spec:
  clusterRef:
    name: garage
    namespace: garage
  neverExpires: true
  bucketPermissions:
    - bucketRef:
        name: <app>
        namespace: garage
      read: true
      write: true
  secretTemplate:
    name: <app>-s3-credentials

Step 3: GarageReferenceGrant — PLATFORM-level, must go in kubernetes/platform/config/garage/:

# kubernetes/platform/config/garage/reference-grant-<app>.yaml
apiVersion: garage.rajsingh.info/v1beta1
kind: GarageReferenceGrant
metadata:
  name: allow-<app>
spec:
  from:
    - kind: GarageKey
      namespace: <app>

Also register it in kubernetes/platform/config/garage/kustomization.yaml. Without this, the Garage admission webhook (vgaragekey.kb.io) will deny the GarageKey with cross-namespace reference not permitted.

Namespace label (also required for network access):

access.network-policy.homelab/garage-s3: "true"

S3 env vars in the app (GarageKey generates the <app>-s3-credentials secret automatically):

S3_ACCESS_KEY_ID:
  valueFrom:
    secretKeyRef:
      name: <app>-s3-credentials
      key: access-key-id
S3_SECRET_ACCESS_KEY:
  valueFrom:
    secretKeyRef:
      name: <app>-s3-credentials
      key: secret-access-key
S3_BUCKET: <app>
S3_REGION: garage
S3_ENDPOINT: "http://${garage_s3_endpoint}"
S3_FORCE_PATH_STYLE: "true"

---

Phase 4: Validate

task k8s:validate → task renovate:validate. Fix all errors before proceeding.

---

Phase 5: Test on Dev

The dev cluster is a sandbox — iterate freely.

Deploy directly (suspend Flux first if needed: task k8s:flux-suspend -- <kustomization-name>):

helm install <app-name> <repo>/<chart> \
  -n <namespace> --create-namespace \
  -f kubernetes/platform/charts/<app-name>.yaml \
  --version <version>

kubectl -n <namespace> \
  wait --for=condition=Ready pod -l app.kubernetes.io/name=<app-name> --timeout=300s

For OCI charts, use oci://registry/<path>/<chart>. For iteration, use helm upgrade instead of helm install.

Verify network connectivity (CRITICAL — network policies are enforced):

kubectl port-forward -n kube-system svc/hubble-relay 4245:80 &
hubble observe --verdict DROPPED --namespace <namespace> --since 5m
hubble observe --from-namespace istio-gateway --to-namespace <namespace> --since 2m
hubble observe --from-namespace <namespace> --to-namespace database --since 2m

Common causes: missing profile label (gateway blocked), missing access label (database/S3 blocked), wrong profile (external API calls blocked).

Verify monitoring using the helper scripts:

.claude/skills/deploy-app/scripts/check-deployment-health.sh <namespace> <app-name>
.claude/skills/deploy-app/scripts/check-servicemonitor.sh <app-name>
.claude/skills/deploy-app/scripts/check-alerts.sh
.claude/skills/deploy-app/scripts/check-canary.sh <app-name>  # if canary created

Iterate until all checks pass. AskUserQuestion to report status and confirm proceeding.

---

Phase 6: Validate GitOps & PR

Uninstall direct helm install → reconcile via Flux → validate clean convergence:

helm uninstall <app-name> -n <namespace>
task k8s:reconcile-validate

If reconciliation fails, fix manifests and retry.

Commit using conventional commits format, push, and create the PR:

git push -u origin deploy-<app-name>
gh pr create --title "feat(k8s): deploy <app-name>" --body "..."

The worktree is kept until PR is merged. User cleans up with task wt:remove -- deploy-<app-name>.

---

Secrets Handling

For detailed workflows, see the secrets skill.

App needs a secret?
├─ Random/generated → secret-generator annotation
│     secret-generator.v1.mittwald.de/autogenerate: "key"
├─ External service (OAuth, third-party API) → ExternalSecret → AWS SSM
└─ Unclear → AskUserQuestion: "Can this be randomly generated?"

---

Error Handling

Error	Response
No chart found	Suggest app-template, ask user
Validation fails	Show error, fix, retry
CrashLoopBackOff	Show logs, propose fix, ask user
Alerts firing	Show alerts, determine if related, ask user
Namespace exists	Ask user: reuse or new name
Secret needed	Apply decision tree above
Pods rejected by PodSecurity	Add restricted security context (see step 3.4)

---

References

• File Templates - Copy-paste templates for all config files
• Monitoring Patterns - ServiceMonitor, PrometheusRule, Canary examples
• flux-gitops skill - ResourceSet patterns
• app-template skill - For apps without native charts
• kubesearch skill - Research workflow