Install Genesis — from zero to ready

CLI install — step by step

The CLI path is the standard path for technical operators. No wizard, no port-forward, fully scriptable.

Pick your host type — every step below switches to match. Connected uses public installers; air-gapped pulls all tools from the signed portal bundle.

Prefer GitOps or can't use Zarf? The platform bundle can also be deployed via an ArgoCD Application or plain Helm — see Deployment models for the ArgoCD/GitOps hybrid, the CRD pre-apply workaround, and the Helm-only path. (The ops bundle is always deployed imperatively; only the platform bundle supports these alternatives.)

1. 01
   Bootstrap gap-host toolchain.

   Genesis shells out to cosign, zstd, tar, skopeo, helm, kubectl, zarf (+ zarf-init), kubelogin, and the trivy CVE DB. On an air-gapped host these are unreachable from GitHub / dl.k8s.io / raw.githubusercontent.com — Genesis publishes them as one cosign-signed tool bundle that genesis pull --tools downloads and verifies (audit A3/A5/A8).

   # Minimal OS packages (from your internal apt mirror if offline)
   sudo apt-get install -y curl jq zstd tar python3-pip pipx skopeo
   
   # cosign — the one binary needed to verify the wheels + tool bundle.
   # Pull it from your internal mirror (it is ALSO inside the tool bundle).
   # … then install the genesis CLI (step 2) and log in (step 3), and:
   genesis pull --tools                        # writes ./tools/<os-arch>/
   # zarf + zarf-init, helm, kubectl, kubelogin, cosign, trivy CVE DB —
   # each verified against a cosign-signed SHA256SUMS. No GitHub, no internet.
   export PATH="$PWD/tools/linux-amd64:$PATH"  # or install them onto PATH
   
   pipx ensurepath && exec $SHELL -l

2. 01b
   Connect to the cluster.

   The cluster-side host needs a working cluster-admin kubeconfig pointed at your cluster. On an air-gapped host, this kubeconfig is pre-provisioned by whoever owns the cluster (it does not require internet); the az login flow below is for connected setup hosts only — az login reaches Azure AD and will not work air-gapped.

   # Connected setup host (reaches Azure AD) — produces a kubeconfig you can
   # then carry to the gap-host. Air-gapped hosts: use the pre-provisioned one.
   az login
   az account set --subscription <subscription-id>
   az aks get-credentials --resource-group <rg-name> --name <cluster-name> --overwrite-existing
   # AKS + AAD: convert to a non-interactive auth mode kubectl/helm can use
   kubelogin convert-kubeconfig -l azurecli
   
   kubectl get nodes          # confirm connectivity + cluster-admin

3. 02
   Install the genesis CLI from the portal (cosign-verified).

   The CLI ships as a signed Python wheel plus a dependency wheelhouse (third-party deps as pre-built linux-amd64 wheels). Verification uses the same cosign key as the bundles. No PyPI access needed — everything downloads from the portal.

   VER=3.9.10      # replace with your entitled version (see `genesis releases`)
   BASE=https://downloads.genesis.autonomize.ai/cli/${VER}
   
   mkdir ~/cli-install && cd ~/cli-install
   curl -sSLO ${BASE}/SHA256SUMS
   curl -sSLO ${BASE}/SHA256SUMS.sig
   curl -sSLO https://downloads.genesis.autonomize.ai/cli/cosign.pub
   
   # Verify the integrity manifest signature BEFORE trusting the names inside it
   cosign verify-blob \
     --key cosign.pub \
     --signature SHA256SUMS.sig \
     --insecure-ignore-tlog \
     SHA256SUMS
   
   # Download exactly what the signed manifest lists. Do NOT build wheel names
   # from ${VER}: PEP 427 normalises a pre-release, so 3.10.1-rc.1 is published as
   # genesis_cli-3.10.1rc1-py3-none-any.whl and a constructed URL 404s on every RC.
   awk '{print $2}' SHA256SUMS | sed 's|^\*||' | while read -r f; do
     curl -sSLO "${BASE}/${f}"
   done
   
   # Verify wheel bytes against the signed manifest
   sha256sum -c SHA256SUMS --ignore-missing
   
   # Install — wheels + wheelhouse resolve everything locally, no PyPI
   tar -xzf genesis_cli_wheelhouse-*-linux-amd64.tar.gz   # → ./wheelhouse/
   pipx install --pip-args="--find-links=. --find-links=wheelhouse --no-index" ./genesis_cli-*-py3-none-any.whl
   genesis --version

4. 03
   Authenticate with your license key.

   genesis login          # paste sk_yourorg_*
   genesis whoami         # prints your customer slug + entitled channels
   genesis releases       # lists available versions

   If genesis login hangs after you paste the key (headless Linux only):

   1. Cancel the hung command with Ctrl+C.
   2. Tell the CLI to skip the system keyring for this shell and persist it for future shells:

      echo 'export PYTHON_KEYRING_BACKEND=keyring.backends.fail.Keyring' >> ~/.bashrc
      source ~/.bashrc

   3. Re-run genesis login. It returns in ~2 seconds and prints stored via ~/.genesis/creds instead of stored via keyring.

   Why this happens. The CLI prefers the OS keyring (Keychain on macOS, gnome-keyring/kwallet on Linux desktops). A bare Linux server has no desktop session, so the Python keyring library blocks waiting for a D-Bus reply that never comes. The env var above tells keyring to fail immediately; the CLI then writes the key to ~/.genesis/creds (mode 0600, owner-only) — the documented headless-host path.

5. 04
   Pull both bundles.

   Each release publishes an ops bundle (CRDs + operators + agents — stateless) and a platform bundle (Keycloak, APISIX, AI Studio, ~18 services). Both are cosign-signed and ship with a CycloneDX SBOM.

   mkdir ~/bundles && cd ~/bundles
   genesis pull ${VER}
   # Downloads: genesis-ops-${VER}.tar.zst + genesis-platform-${VER}.tar.zst
   #            + .sig sidecars + sbom-${VER}.cdx.json + SHA256 manifests
   ls -lh

   A third, optional bundle kind exists — inference (model-serving images for ModelHub, carried separately since it's multi-GiB). The command above does not fetch it; add --kind inference (or --kind all) only if you're enabling ModelHub. It has no chart — relocate it with push-images (step 6) and enable it with --set genesis-modelhub.enabled=true on the platform deploy (step 11), never with genesis deploy directly.

6. 05
   Verify cosign signatures & SBOM.

   Fully offline. Nothing crosses to the cluster until both bundles pass.

   genesis verify genesis-ops-${VER}.tar.zst
   genesis verify genesis-platform-${VER}.tar.zst
   # Both must print: ✓ cosign verify OK · sha256 match
   
   # CVE-gate the bundle (HIGH+CRITICAL → exit 1; CI-gateable).
   # Extracts the per-image SBOM sidecar and runs trivy against each image.
   genesis scan genesis-ops-${VER}.tar.zst
   genesis scan genesis-platform-${VER}.tar.zst

   Scanner prereq. genesis scan shells out to trivy (to read the per-image CycloneDX SBOM sidecar and emit a HIGH/CRITICAL gate) plus the trivy CVE DB. Both ship in the signed portal tool bundle — if you ran genesis pull --tools in step 1 you already have them, no internet needed:

   # Air-gapped (default): trivy binary + CVE DB come from the tool bundle.
   # genesis scan auto-detects ./tools/<os-arch>/trivy-db.tar.gz, or pass it:
   genesis scan genesis-ops-${VER}.tar.zst --trivy-db ./tools/linux-amd64/trivy-db.tar.gz
   
   # An internal trivy-db mirror also works:
   genesis scan genesis-ops-${VER}.tar.zst -- --db-repository <internal>/trivy-db:2 --skip-db-update
   
   # Connected hosts only — public installer (NOT reachable air-gapped):
   #   curl -sfL https://raw.githubusercontent.com/aquasecurity/trivy/main/contrib/install.sh | sudo sh -s -- -b /usr/local/bin
   #   brew install trivy   (macOS)
   trivy --version

   Prefer Grype / Snyk / Black Duck? Extract <bundle>.sboms.tar.gz and loop your scanner over each .cdx.json — see Scan the bundle.

7. 06
   Push all images to your registry.

   This is the step that makes your CISO happy: every image in the bundle — Genesis-owned and the bundled third-party dependencies (the Chainguard preflight probe images) — is relocated into your registry before any pod ever runs in your cluster. Your scanner runs against your registry. Pods pull from your registry, never from Docker Hub or ours.

   Third-party images too Not everything in the bundle is authored by Autonomize — the preflight probe pods use upstream Chainguard images. They ship inside the ops bundle and are relocated by push-images like the rest, so they stay scannable and air-gap-safe. Every chart honors global.imageRegistry (and global.imagePullSecrets), so a plain helm install or ArgoCD sync relocates them with the same single override as every other image — no per-image flag. The full list is in the ops-bundle image table on the Prerequisites page.

   Push both bundles Push the ops bundle first, then the platform bundle. Both must be in your registry before either genesis deploy call.

   REGISTRY=your-registry.azurecr.io    # ECR, ACR, Harbor, or any OCI registry
   
   # Authenticate to your registry (Azure ACR example — no Docker needed)
   TOKEN=$(az acr login --name ${REGISTRY%%.*} --expose-token --query accessToken -o tsv)
   mkdir -p ~/.config/containers
   cat > ~/.config/containers/auth.json <<EOF
   {"auths":{"${REGISTRY}":{"auth":"$(printf '00000000-0000-0000-0000-000000000000:%s' $TOKEN | base64 -w0)"}}}
   EOF
   
   # Push ops images
   genesis push-images --to ${REGISTRY} --bundle genesis-ops-${VER}.tar.zst
   
   # Push platform images
   genesis push-images --to ${REGISTRY} --bundle genesis-platform-${VER}.tar.zst

   After this step your CISO team can run Trivy / Aqua / Twistlock / Snyk against ${REGISTRY}. Continue to step 7 only after images clear your policy.

8. 07
   Install External Secrets Operator, then apply the ExternalSecret CRs.

   Two ExternalSecret CRs must exist in the genesis namespace before platform pods start. If they're missing, pods crash immediately with CreateContainerConfigError. The CRs depend on External Secrets Operator (ESO) being installed and a ClusterSecretStore pointing at your vault — install both now if you don't already have them. Apply everything before the ops deploy.

   7.1 — Install ESO (skip if already installed). It must serve external-secrets.io/v1 before preflight (step 10) — the eso-version check hard-FAILs when the CRD is absent (audit E3).

   # EXAMPLE — from your INTERNAL mirror, or the genesis-external-secrets
   # subchart shipped inside the platform bundle. No public Helm repo.
   # The OCI ref, --version, and image.repository below reflect ONE mirror
   # layout; substitute the path/tag/repo your registry actually uses (mirror
   # conventions vary). What matters: install ESO 0.18.2+ from a source your
   # air-gapped cluster can reach, with installCRDs=true.
   helm upgrade --install external-secrets \
     oci://<your-registry>/charts/external-secrets --version 0.18.2 \
     -n external-secrets --create-namespace \
     --set installCRDs=true \
     --set image.repository=<your-registry>/external-secrets/external-secrets \
     --wait

   
   # 7.2 — Create a ClusterSecretStore pointing at your vault.
   # Stub below is Azure Key Vault + Workload Identity — substitute your
   # vault URL + ServiceAccount, or use the AWS / Vault / GCP variant from
   # the secrets setup guide linked below.
   cat <<'YAML' | kubectl apply -f -
   apiVersion: external-secrets.io/v1
   kind: ClusterSecretStore
   metadata:
     name: customer-akv     # ← matches secretStoreRef.name in both ExternalSecret templates
   spec:
     provider:
       azurekv:
         authType: WorkloadIdentity
         vaultUrl: "https://YOUR-KEYVAULT.vault.azure.net"
         serviceAccountRef:
           name: external-secrets
           namespace: external-secrets
   YAML
   
   # Wait until the store reports Ready=True before applying ExternalSecrets.
   kubectl wait --for=condition=Ready clustersecretstore/customer-akv --timeout=60s
   
   # 7.3 — Apply the two ExternalSecret CRs.
   kubectl create namespace genesis 2>/dev/null || true
   
   # Edit the templates IN PLACE in docs/customer/templates/ — set
   # secretStoreRef.name to "customer-akv" (or whatever you named the
   # ClusterSecretStore above), uncomment the oidc-client-secret line only
   # when auth_mode=client_secret. Then apply from the repo root:
   kubectl apply -f docs/customer/templates/customer-genesis-secrets-externalsecret.yaml
   kubectl apply -f docs/customer/templates/ai-studio-secrets-externalsecret.yaml
   
   # Wait for ESO to sync both into K8s Secrets
   kubectl -n genesis get externalsecret -w
   # Status column must show: SecretSynced for both

   Pre-seed every key first Every remoteRef.key referenced by either template must already exist in your vault — even as a blank placeholder — or ESO logs SecretSyncedError until it does. Seed anything you're not populating yet as blank so sync succeeds, then fill it in for real once you enable that feature:

   az keyvault secret set --vault-name <your-keyvault-name> --name <KEY-NAME> --file /dev/null --query name -o tsv

   Per-vault auth setup (Azure Key Vault Workload Identity, AWS Secrets Manager IRSA, HashiCorp Vault, GCP Secret Manager): see required-secrets.md.

   Ready-to-edit CR templates: customer-genesis-secrets-externalsecret.yaml · ai-studio-secrets-externalsecret.yaml. Full key list + AWS / Azure / GCP setup walkthrough: required-secrets.md.

9. 08
   One-time cluster bootstrap, then deploy the ops bundle.

   zarf init creates the zarf-state secret and an in-cluster registry mirror (used as a relay even when you have an external registry). Run once per cluster lifetime.

   Ops first, always The ops bundle installs the 5 CRDs and the deploy operator. Without this, genesis preflight and genesis deploy --bundle genesis-platform-* will fail.

   # One-time per cluster. The zarf-init package ships in the signed tool
   # bundle (genesis pull --tools, step 1) — pass the LOCAL file; a bare
   # `zarf init` pulls init images from ghcr.io and fails air-gapped (audit A3).
   zarf init ./tools/linux-amd64/zarf-init-amd64-*.tar.zst --confirm
   kubectl -n zarf get secret zarf-state   # must exist before ops deploy
   
   # Deploy ops bundle (uses images already in your registry from step 6).
   # --local-helm is required here, not optional: this is the one bootstrap
   # moment before the deploy operator exists (the ops bundle IS the
   # operator), so `genesis deploy` can't drive it through the operator yet.
   # You'll see a "--local-helm is deprecated" warning — expected, not an
   # error; it's the only currently-shipping way to get a registry target
   # into this one step.
   genesis deploy --bundle genesis-ops-${VER}.tar.zst --local-helm --registry ${REGISTRY} -n genesis
   kubectl -n genesis get pods             # all pods must reach 1/1 Running

   Why --local-helm --registry and not a bare zarf package deploy? Without it, the ops bundle's relocate-umbrella-chart component's helm push step reads Zarf's built-in ZARF_REGISTRY variable, which defaults to Zarf's own internal registry address (127.0.0.1:31999) when no explicit target is given — nothing is bound to that port on a plain deploy, so it fails with a guaranteed connection refused.

   If zarf-docker-registry's pod gets stuck instead of reaching 1/1 Running — typically FailedAttachVolume / LinkedAuthorizationFailed on a cluster whose managed disks use a customer-managed key — see Customer-managed disk encryption blocks zarf init before continuing.

10. 09
    Configure: write the PlatformConfig ConfigMap.

    Populate your genesis.yaml with DB host, Redis host, registry URL, base URL, OIDC settings, and Secret references (never raw credentials). Then save to the cluster and emit the Helm values overlay.

    genesis configure \
      --from genesis.yaml \
      --save \
      --emit-helm-values /tmp/genesis-platform-values.yaml
    # Writes: genesis-platform-config ConfigMap (preflight reads this)
    # Emits:  /tmp/genesis-platform-values.yaml (deploy reads this)

    Redis URL trap networking.redis.url must never include credentials — but watch for a stray @ left with nothing before it, e.g. rediss://@your-redis-host:10000/0. That parses as an explicit empty-string ACL username, so preflight's Redis check sends a 2-argument AUTH "" <password> that managed Redis (e.g. Azure Managed Redis) rejects with WRONGPASS — even when the password itself is correct. Drop the @ entirely (rediss://your-redis-host:10000/0); it's still schema-valid and falls back to the single-argument AUTH <password> form.

    Full field reference: configure.md

11. 10
    Run preflight — every check must PASS.

    Preflight reads the ConfigMap saved in step 9. It checks Postgres reachability, Redis, ESO secrets, image registry pull, OIDC discovery, cert-manager, DNS, TLS, and more — around 46 checks today (see the phase-3 card below for the full breakdown; the exact count grows as checks are added, so treat it as "all of them," not a fixed number). Platform deploy is blocked on any FAIL. Any FAIL/WARN row prints inline Vin Advisor advice.

    genesis preflight -n genesis
    # Prints per-check table (PASS / WARN / FAIL)
    # Exit 1 = one or more FAIL — fix and re-run
    # Exit 0 = no FAILs — but this does NOT distinguish a clean pass from a
    # WARN-only report. Read the table before proceeding to step 11; pass
    # --allow-warnings to `genesis deploy` if you're knowingly proceeding
    # over warnings.

12. 11
    Deploy the platform bundle.

    The deploy operator installs the full Genesis umbrella (~18 services including Keycloak, APISIX, AI Studio, Knowledge Center) by reading the genesis-platform-config ConfigMap you saved in step 9 — it is not a direct helm install from this host, and it does not take --registry/--values flags on this default path (those two now only exist for the deprecated --local-helm escape hatch used in step 8; passing them here without --local-helm is a hard refusal). It also stays gated on human approval unless you pass --approve.

    cd bundles first so the --bundle path resolves to the downloaded tarball.

    cd bundles
    genesis deploy \
      --bundle genesis-platform-${VER}.tar.zst \
      -n genesis \
      --approve
    # --preflight-report is optional — it defaults to the newest
    # PreflightReport in the namespace, i.e. the one step 10 just created.

13. 11b
    Grant schema CREATE to the RLS migrate role — RLS databases only.

    RLS databases only: ai_studio and genesis_authz. The provisioner mints the RLS migrate role but does not grant it CREATE on schema public. Run grant-schema.sql as the Postgres admin once the provisioner pod is Ready (roles exist) and before the app pods finish migrating — the script \connects into both RLS databases, so one invocation covers both.

    psql "postgresql://<admin>:<pw>@<host>:5432/postgres?sslmode=require" \
      -v ON_ERROR_STOP=1 -f docs/customer/scripts/grant-schema.sql

    Without it, genesis-be / genesis-be-runtime / genesis-worker / genesis-dev-worker / genesis-deploy-worker / genesis-authz crash-loop on permission denied for schema public. There is no automated trigger — preflight is read-only and ran before deploy. It is idempotent; if the app pods have already crash-looped, they self-recover on their next migration retry once the grant lands. Deep-dive: Prerequisites § “Grant schema CREATE to the RLS migrate role”.

14. 12
    Verify the install.

    genesis status -n genesis
    # Prints the wizard phase, then a table of CR rows (GenesisDeployment,
    # PreflightReport, HealthReport, etc.) each with its own status/age.
    # Add --output json for a scriptable form.
    kubectl get pods -n genesis    # all 1/1 Running

    Your platform is now available at the base_url you configured. OIDC/SSO access is live; bootstrap mode is permanently disabled.

1. 01
   Bastion toolchain.

   Run everything on the bastion/jump host that has outbound internet access to GitHub, dl.k8s.io, and aka.ms (one-way — nothing needs to reach back in).

   # Add deadsnakes PPA for Python 3.12
   sudo apt-get update && sudo apt-get install -y software-properties-common
   sudo add-apt-repository -y ppa:deadsnakes/ppa
   
   # Install Python 3.12 and core utilities
   sudo apt-get update && sudo apt-get install -y \
     python3.12 python3.12-venv python3.12-dev \
     curl jq zstd tar skopeo unzip
   
   # Azure CLI
   curl -sL https://aka.ms/InstallAzureCLIDeb | sudo bash
   
   # Python 3.12 as the user-level default
   mkdir -p ~/.local/bin
   ln -sf /usr/bin/python3.12 ~/.local/bin/python3
   ln -sf /usr/bin/python3.12 ~/.local/bin/python
   export PATH="$HOME/.local/bin:$PATH"
   hash -r
   
   # kubectl
   curl -LO "https://dl.k8s.io/release/$(curl -L -s https://dl.k8s.io/release/stable.txt)/bin/linux/amd64/kubectl"
   chmod +x kubectl && mv kubectl ~/.local/bin/
   
   # Helm
   curl -fsSL https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-3 | bash
   
   # kubelogin
   curl -LO https://github.com/Azure/kubelogin/releases/latest/download/kubelogin-linux-amd64.zip
   unzip -o kubelogin-linux-amd64.zip -d /tmp/
   mv /tmp/bin/linux_amd64/kubelogin ~/.local/bin/
   rm -rf kubelogin-linux-amd64.zip /tmp/bin
   
   # Zarf (with API fallback in case the GitHub API rate-limits)
   ZARF_VER=$(curl -s https://api.github.com/repos/zarf-dev/zarf/releases/latest | jq -r .tag_name 2>/dev/null)
   if [[ -z "$ZARF_VER" || "$ZARF_VER" == "null" || "$ZARF_VER" == "Not"* ]]; then
     ZARF_VER="v0.83.0"
   fi
   curl -sL "https://github.com/zarf-dev/zarf/releases/download/${ZARF_VER}/zarf_${ZARF_VER}_Linux_amd64" -o ~/.local/bin/zarf
   chmod +x ~/.local/bin/zarf
   
   # cosign
   curl -OL https://github.com/sigstore/cosign/releases/latest/download/cosign-linux-amd64
   sudo install cosign-linux-amd64 /usr/local/bin/cosign && rm cosign-linux-amd64
   
   # (Optional) k9s
   curl -sS https://webinstall.dev/k9s | bash
   
   # Verify
   python3 --version
   az version
   kubectl version --client
   helm version
   kubelogin --version
   zarf version

2. 02
   Download + verify the Genesis CLI wheels.

   VER=3.9.13       # replace with your entitled version (see `genesis releases` once logged in)
   BASE=https://downloads.genesis.autonomize.ai/cli/${VER}
   
   mkdir ~/cli-install && cd ~/cli-install
   curl -sSLO ${BASE}/SHA256SUMS
   curl -sSLO ${BASE}/SHA256SUMS.sig
   curl -sSLO https://downloads.genesis.autonomize.ai/cli/cosign.pub
   
   # Verify the integrity manifest signature BEFORE trusting the names inside it
   cosign verify-blob \
     --key cosign.pub \
     --signature SHA256SUMS.sig \
     --insecure-ignore-tlog \
     SHA256SUMS
   
   # Download exactly what the signed manifest lists. Do NOT build wheel names
   # from ${VER}: PEP 427 normalises a pre-release, so 3.10.1-rc.1 is published as
   # genesis_cli-3.10.1rc1-py3-none-any.whl and a constructed URL 404s on every RC.
   awk '{print $2}' SHA256SUMS | sed 's|^\*||' | while read -r f; do
     curl -sSLO "${BASE}/${f}"
   done
   
   # Verify wheel bytes against the signed manifest
   sha256sum -c SHA256SUMS --ignore-missing

3. 03
   Install the CLI into its own venv.

   tar -xzf genesis_cli_wheelhouse-*-linux-amd64.tar.gz   # -> ./wheelhouse/
   
   python3.12 -m venv ~/.local/share/genesis-venv
   
   ~/.local/share/genesis-venv/bin/pip install \
     --force-reinstall \
     --no-index \
     --find-links="$PWD" \
     --find-links="$PWD/wheelhouse" \
     "$PWD"/genesis_cli-*-py3-none-any.whl
   
   mkdir -p ~/.local/bin
   ln -sf ~/.local/share/genesis-venv/bin/genesis ~/.local/bin/genesis
   export PATH="$HOME/.local/bin:$PATH"
   
   genesis --version

4. 04
   Log in and pull the bundles.

   export GENESIS_PORTAL_URL=https://cp.autonomize.ai
   genesis login          # paste sk_yourorg_*
   genesis whoami         # prints your customer slug + entitled channels
   genesis releases        # lists available versions
   mkdir ~/bundles && cd ~/bundles
   genesis pull ${VER}

5. 05
   Push both bundles' images to your own registry.

   Both bundles need pushing — the ops bundle carries the Bastion, operators, agents, and Vin Agent; the platform bundle carries everything else. Relocating only one leaves the other pulling from the vendor ACR (unreachable air-gapped, and unscanned).

   REGISTRY=<your-registry>.azurecr.io   # your own ACR/ECR/Harbor host -- reused as ${REGISTRY} in every later step
   az login
   # Authenticate to ACR without needing Docker installed
   TOKEN=$(az acr login --name ${REGISTRY%%.*} --expose-token --query accessToken -o tsv)
   mkdir -p ~/.config/containers
   cat > ~/.config/containers/auth.json <<EOF
   {"auths":{"${REGISTRY}":{"auth":"$(printf '00000000-0000-0000-0000-000000000000:%s' $TOKEN | base64 -w0)"}}}
   EOF
   
   genesis push-images --to ${REGISTRY} --bundle genesis-ops-${VER}.tar.zst
   genesis push-images --to ${REGISTRY} --bundle genesis-platform-${VER}.tar.zst

6. 06
   Get cluster credentials.

   az account set --subscription <your-subscription-id>
   az aks get-credentials --resource-group <your-resource-group> --name <your-cluster-name> --overwrite-existing
   kubelogin convert-kubeconfig -l azurecli
   
   kubectl get po -A

7. 07
   ClusterSecretStore + ExternalSecrets.

   Naming note The ClusterSecretStore's name is arbitrary, but whatever you choose must match exactly in three places: this ClusterSecretStore's own metadata.name, secretStoreRef.name in both ExternalSecret manifests below, and secrets.secret_store_name in genesis.yaml (step 09). The example below uses azure-keyvault.

   cat <<'YAML' | kubectl apply -f -
   apiVersion: external-secrets.io/v1
   kind: ClusterSecretStore
   metadata:
     name: azure-keyvault
   spec:
     provider:
       azurekv:
         authType: WorkloadIdentity
         vaultUrl: "https://<your-keyvault-name>.vault.azure.net/"
         serviceAccountRef:
           name: external-secrets
           namespace: external-secrets
   YAML
   
   kubectl wait --for=condition=Ready clustersecretstore/azure-keyvault --timeout=60s
   
   kubectl create namespace genesis 2>/dev/null || true

   Both ExternalSecret manifests below already have secretStoreRef.name set to azure-keyvault, matching the ClusterSecretStore created above. If you're starting from a template with a placeholder store name instead, replace it everywhere — an unedited placeholder fails Kubernetes' own name-format validation, not just an ESO sync error.

   customer-genesis-secrets-externalsecret.yaml — the small Secret the Bastion wizard references by name/key when you supply the Postgres admin password and (optionally) an OIDC client secret:

   apiVersion: external-secrets.io/v1
   kind: ExternalSecret
   metadata:
     name: customer-genesis-secrets
     namespace: genesis
   spec:
     refreshInterval: 1h
     secretStoreRef:
       kind: ClusterSecretStore
       name: azure-keyvault
     target:
       name: customer-genesis-secrets
       creationPolicy: Owner
       deletionPolicy: Retain
     data:
       - { secretKey: db-admin-password,  remoteRef: { key: db-admin-password } }
       # Uncomment only when oidc.auth_mode = "client_secret" in genesis.yaml.
       # Workload Identity Federation (auth_mode: wif, what this environment
       # uses) does not need this key -- leaving it active causes "key not
       # found" log noise on every ESO refresh until the vault key exists.
       # - { secretKey: oidc-client-secret, remoteRef: { key: oidc-client-secret } }

   ai-studio-secrets-externalsecret.yaml — the large shared Secret every platform subchart reads via envFrom/secretKeyRef. Comment out any entry for a feature you're not using (Azure Marketplace, Kafka, OTEL Event Hub, LLM provider keys) — ESO otherwise logs key not found noise on every refresh for a vault key that will never exist:

   apiVersion: external-secrets.io/v1
   kind: ExternalSecret
   metadata:
     name: ai-studio-secrets
     namespace: genesis
   spec:
     refreshInterval: 1h
     secretStoreRef:
       kind: ClusterSecretStore
       name: azure-keyvault
     target:
       name: ai-studio-secrets
       creationPolicy: Owner
       deletionPolicy: Retain
     data:
       # ── Database ────────────────────────────────────────────────
       - { secretKey: DB_USER,                                        remoteRef: { key: DB-USER } }
       - { secretKey: DB_PASSWORD,                                    remoteRef: { key: DB-PASSWORD } }
   
       # ── Temporal (genesis-de) DB env-var aliases -- all three resolve
       # to the same DB password ───────────────────────────────────
       - { secretKey: TEMPORAL_STORE_PASSWORD,                        remoteRef: { key: DB-PASSWORD } }
       - { secretKey: TEMPORAL_VISIBILITY_STORE_PASSWORD,             remoteRef: { key: DB-PASSWORD } }
   
       # ── Redis ───────────────────────────────────────────────────
       - { secretKey: REDIS_PASSWORD,                                 remoteRef: { key: REDIS-PASSWORD } }
   
       # ── App secrets ─────────────────────────────────────────────
       - { secretKey: SECRET_KEY,                                     remoteRef: { key: SECRET-KEY } }
       - { secretKey: AUTH_SECRET,                                    remoteRef: { key: AUTH-SECRET } }
       # Singular + plural are BOTH intentional -- different services read
       # different env-var names for the same value.
       - { secretKey: CREDENTIAL_ENCRYPTION_KEY,                      remoteRef: { key: CREDENTIAL-ENCRYPTION-KEY } }
       - { secretKey: CREDENTIALS_ENCRYPTION_KEY,                     remoteRef: { key: CREDENTIALS-ENCRYPTION-KEY } }
       - { secretKey: SECURITY_ENCRYPTION_KEY,                        remoteRef: { key: SECURITY-ENCRYPTION-KEY } }
       - { secretKey: SECURITY_ENCRYPTION_SALT,                       remoteRef: { key: SECURITY-ENCRYPTION-SALT } }
       - { secretKey: JWT_SECRET,                                     remoteRef: { key: JWT-SECRET } }
       - { secretKey: ADMIN_PASSWORD,                                 remoteRef: { key: ADMIN-PASSWORD } }
       - { secretKey: INTERNAL_API_KEYS,                              remoteRef: { key: INTERNAL-API-KEYS } }
       - { secretKey: GENESIS_API_KEY,                                remoteRef: { key: GENESIS-API-KEY } }
   
       # ── Keycloak ────────────────────────────────────────────────
       - { secretKey: AUTH_KEYCLOAK_SECRET,                           remoteRef: { key: AUTH-KEYCLOAK-SECRET } }
       - { secretKey: KEYCLOAK_CLIENT_ID,                             remoteRef: { key: KEYCLOAK-CLIENT-ID } }
       - { secretKey: KEYCLOAK_CLIENT_SECRET,                         remoteRef: { key: KEYCLOAK-CLIENT-SECRET } }
       - { secretKey: KEYCLOAK_FRONTEND_CLIENT_SECRET,                remoteRef: { key: KEYCLOAK-FRONTEND-CLIENT-SECRET } }
       - { secretKey: KC_BOOTSTRAP_ADMIN_USERNAME,                    remoteRef: { key: KC-BOOTSTRAP-ADMIN-USERNAME } }
       - { secretKey: KC_BOOTSTRAP_ADMIN_PASSWORD,                    remoteRef: { key: KC-BOOTSTRAP-ADMIN-PASSWORD } }
       - { secretKey: KC_SPI_EVENTS_LISTENER_GENESIS_AUTHZ_EVENT_LISTENER_INTERNAL_API_KEY,
           remoteRef: { key: KC-SPI-EVENTS-LISTENER-GENESIS-AUTHZ-EVENT-LISTENER-INTERNAL-API-KEY } }
   
       # ── Keycloak SMTP ───────────────────────────────────────────
       - { secretKey: KC_SMTP_HOST,                                   remoteRef: { key: KC-SMTP-HOST } }
       - { secretKey: KC_SMTP_PORT,                                   remoteRef: { key: KC-SMTP-PORT } }
       - { secretKey: KC_SMTP_USER,                                   remoteRef: { key: KC-SMTP-USER } }
       - { secretKey: KC_SMTP_PASSWORD,                               remoteRef: { key: KC-SMTP-PASSWORD } }
       - { secretKey: KC_SMTP_FROM,                                   remoteRef: { key: KC-SMTP-FROM } }
       - { secretKey: KC_SMTP_FROM_DISPLAY_NAME,                      remoteRef: { key: KC-SMTP-FROM-DISPLAY-NAME } }
       - { secretKey: KC_SMTP_SSL,                                    remoteRef: { key: KC-SMTP-SSL } }
       - { secretKey: KC_SMTP_STARTTLS,                               remoteRef: { key: KC-SMTP-STARTTLS } }
   
       # ── Gateway (APISIX) ────────────────────────────────────────
       - { secretKey: APISIX_ADMIN_KEY,                               remoteRef: { key: APISIX-ADMIN-KEY } }
   
       # ── Azure AD federation (Keycloak SSO; NOT Workload Identity) ───
       # AZURE_CLIENT_ID / AZURE_TENANT_ID are injected by the Workload
       # Identity webhook from the ServiceAccount annotation -- don't map
       # them here.
       - { secretKey: AZURE_AD_CLIENT_ID,                             remoteRef: { key: AZURE-AD-CLIENT-ID } }
       - { secretKey: AZURE_AD_CLIENT_SECRET,                         remoteRef: { key: AZURE-AD-CLIENT-SECRET } }
       - { secretKey: AZURE_AD_TENANT_ID,                             remoteRef: { key: AZURE-AD-TENANT-ID } }
   
       # ── Azure AI / Document services ─────────────────────────────
       - { secretKey: AZURE_OPENAI_API_KEY,                           remoteRef: { key: AZURE-OPENAI-API-KEY } }
       - { secretKey: AZURE_DOCUMENT_INTELLIGENCE_KEY,                remoteRef: { key: AZURE-DOCUMENT-INTELLIGENCE-KEY } }
       - { secretKey: AZURE_FORM_RECOGNIZER_API_KEY,                  remoteRef: { key: AZURE-FORM-RECOGNIZER-API-KEY } }
   
       # ── Optional: LLM providers -- uncomment + provision when enabling ──
       # - { secretKey: OPENAI_API_KEY,                               remoteRef: { key: OPENAI-API-KEY } }
       # - { secretKey: ANTHROPIC_API_KEY,                            remoteRef: { key: ANTHROPIC-API-KEY } }
   
       # ── Vector store + web search ────────────────────────────────
       - { secretKey: QDRANT_API_KEY,                                 remoteRef: { key: QDRANT-API-KEY } }
       - { secretKey: TAVILY_API_KEY,                                 remoteRef: { key: TAVILY-API-KEY } }
   
       # ── Azure Marketplace (genesis-tenant-mgmt) ─────────────────
       - { secretKey: AZURE_MARKETPLACE_CLIENT_SECRET,                remoteRef: { key: AZURE-MARKETPLACE-CLIENT-SECRET } }
       - { secretKey: AZURE_MARKETPLACE_WEBHOOK_SECRET,               remoteRef: { key: AZURE-MARKETPLACE-WEBHOOK-SECRET } }
   
       # ── Observability fan-out ────────────────────────────────────
       - { secretKey: KAFKA_SASL_USERNAME,                            remoteRef: { key: KAFKA-SASL-USERNAME } }
       - { secretKey: KAFKA_SASL_PASSWORD,                            remoteRef: { key: KAFKA-SASL-PASSWORD } }
       - { secretKey: OTEL_EVENTHUB_CONNECTION_STRING,                remoteRef: { key: OTEL-EVENTHUB-CONNECTION-STRING } }
       - { secretKey: OTEL_EVENTHUB_NAMESPACE,                        remoteRef: { key: OTEL-EVENTHUB-NAMESPACE } }

   Apply both, then wait for sync:

   kubectl apply -f customer-genesis-secrets-externalsecret.yaml
   kubectl apply -f ai-studio-secrets-externalsecret.yaml
   
   kubectl -n genesis get externalsecret -w
   # Status column must show: SecretSynced for both

   Every remoteRef.key above needs to actually exist in the Key Vault first (even as a blank placeholder) or ESO logs SecretSyncedError until it does. These split into two very different groups.

   Safe to leave blank Feature-specific credentials — SSO federation, SMTP, observability fan-out, LLM/vector-store providers, Azure Marketplace, plus KEYCLOAK-CLIENT-ID (the chart supplies its own sensible default when this is blank). Safe to leave blank until you're ready to wire up that specific feature:

   VAULT=<your-keyvault-name>
   for key in KC-SMTP-HOST KC-SMTP-PORT KC-SMTP-USER KC-SMTP-PASSWORD KC-SMTP-FROM \
     KC-SMTP-FROM-DISPLAY-NAME KC-SMTP-SSL KC-SMTP-STARTTLS KEYCLOAK-CLIENT-ID \
     AZURE-AD-CLIENT-ID AZURE-AD-CLIENT-SECRET AZURE-AD-TENANT-ID \
     AZURE-OPENAI-API-KEY AZURE-DOCUMENT-INTELLIGENCE-KEY AZURE-FORM-RECOGNIZER-API-KEY \
     QDRANT-API-KEY TAVILY-API-KEY AZURE-MARKETPLACE-CLIENT-SECRET AZURE-MARKETPLACE-WEBHOOK-SECRET \
     KAFKA-SASL-USERNAME KAFKA-SASL-PASSWORD OTEL-EVENTHUB-CONNECTION-STRING OTEL-EVENTHUB-NAMESPACE \
   ; do
     az keyvault secret set --vault-name "$VAULT" --name "$key" --file /dev/null --query name -o tsv
   done

   Cannot stay blank Internal platform secrets are different — these cannot stay blank. They aren't external credentials; they're the platform's own signing/encryption keys and internal service-to-service auth, so nothing external depends on their specific value, but the platform won't boot without real ones. Left blank, Keycloak's own bootstrap hangs indefinitely rather than failing loudly (bootstrap-admin-username available only when bootstrap admin password is set, then a silent hang with no further log output), and several services crash outright (RuntimeError: Required environment variable not set: GENESIS_API_KEY). Generate real random values instead:

   VAULT=<your-keyvault-name>
   FERNET() { python3 -c "import base64,os; print(base64.urlsafe_b64encode(os.urandom(32)).decode())"; }
   HEX() { openssl rand -hex 32; }
   
   # Fernet-format keys -- some services use these for symmetric (Fernet)
   # encryption of stored credentials, which requires this exact 32-byte
   # urlsafe-base64 format, not an arbitrary string.
   CRED_KEY=$(FERNET)
   az keyvault secret set --vault-name "$VAULT" --name CREDENTIAL-ENCRYPTION-KEY --value "$CRED_KEY" --query name -o tsv
   az keyvault secret set --vault-name "$VAULT" --name CREDENTIALS-ENCRYPTION-KEY --value "$CRED_KEY" --query name -o tsv
   # ^ singular + plural are BOTH intentional -- different services read
   # different env-var names for the SAME value, not independent secrets.
   az keyvault secret set --vault-name "$VAULT" --name SECRET-KEY --value "$(FERNET)" --query name -o tsv
   
   # Plain random secrets
   for key in KC-BOOTSTRAP-ADMIN-PASSWORD GENESIS-API-KEY AUTH-SECRET AUTH-KEYCLOAK-SECRET \
     JWT-SECRET ADMIN-PASSWORD APISIX-ADMIN-KEY KC-SPI-EVENTS-LISTENER-GENESIS-AUTHZ-EVENT-LISTENER-INTERNAL-API-KEY \
     KEYCLOAK-CLIENT-SECRET KEYCLOAK-FRONTEND-CLIENT-SECRET SECURITY-ENCRYPTION-KEY SECURITY-ENCRYPTION-SALT \
   ; do
     az keyvault secret set --vault-name "$VAULT" --name "$key" --value "$(HEX)" --query name -o tsv
   done
   az keyvault secret set --vault-name "$VAULT" --name KC-BOOTSTRAP-ADMIN-USERNAME --value "admin" --query name -o tsv
   
   # INTERNAL_API_KEYS must be a JSON array, not a bare string -- the
   # consuming service's settings parser rejects a plain value with a Pydantic
   # parsing error ("error parsing value for field internal_api_keys"), not a
   # "missing value" error, so this one is easy to get wrong even when you
   # remember to set it.
   az keyvault secret set --vault-name "$VAULT" --name INTERNAL-API-KEYS --value "[\"$(openssl rand -hex 32)\"]" --query name -o tsv
   
   # db-admin-password must be the exact SAME value as DB-PASSWORD below -- a
   # second copy of your Postgres admin password under the key name
   # customer-genesis-secrets expects, not an independent secret.
   DB_PW=$(az keyvault secret show --vault-name "$VAULT" --name DB-PASSWORD --query value -o tsv)
   az keyvault secret set --vault-name "$VAULT" --name db-admin-password --value "$DB_PW" --query name -o tsv

   (DB-USER, DB-PASSWORD, and REDIS-PASSWORD already carry real values from Terraform; don't overwrite them. AZURE-OPENAI-API-KEY/AZURE-DOCUMENT-INTELLIGENCE-KEY can be populated with real values via az cognitiveservices account keys list instead of staying blank, once you're ready to wire up those features.)

   Per-vault auth setup, full key list: required-secrets.md.

8. 08
   One-time cluster bootstrap: zarf init.

   ZARF_VER=v0.83.0   # match your installed `zarf version` exactly
   curl -sSLO https://github.com/zarf-dev/zarf/releases/download/${ZARF_VER}/zarf-init-amd64-${ZARF_VER}.tar.zst
   
   zarf init zarf-init-amd64-${ZARF_VER}.tar.zst --confirm
   kubectl -n zarf get secret zarf-state      # confirm: 1 row
   kubectl -n zarf get pods                  # zarf-docker-registry should reach 1/1 Running

   This should just work on a properly-configured cluster. If your cluster encrypts managed disks with a customer-managed key (CMK / Disk Encryption Set) and zarf-docker-registry's pod gets stuck instead of reaching 1/1 Running, see the Troubleshooting reference at the bottom — it's almost always a missing role assignment on the AKS identity, not a Zarf problem.

9. 09
   genesis.yaml and saving PlatformConfig.

   A working example, based on a real Azure deployment — replace the values with your own. Save as ~/bundles/genesis.yaml:

   namespace: genesis
   
   cloud: azure
   
   # must be one of: development, staging, production, integration -- NOT a
   # free-form name (e.g. your resource-group name). genesis-tenant-mgmt's own
   # Settings validator rejects anything else, even though `genesis configure`'s
   # schema validation won't catch the mismatch itself.
   environment: development
   keycloak_realm: autonomize
   
   service_account:
     # If your infrastructure already created this ServiceAccount with the
     # Workload Identity annotation baked in, set create: false so the chart
     # doesn't try to create/re-annotate it.
     create: false
     name: workload-identity-sa
     annotations: {}
   
   azure:
     tenant_id: "<your-azure-tenant-id>"
     client_id: "<your-managed-identity-client-id>"   # the managed identity federated for Workload Identity in this namespace
     vault_url: "https://<your-keyvault-name>.vault.azure.net/"
     acr_attached_to_aks: true   # set true if your AKS cluster's kubelet identity already has AcrPull on this registry
     storage_account: <your-storage-account>   # populated so preflight's storage-container check actually runs instead of skipping
   
   database:
     host: "<your-postgres-host>.postgres.database.azure.com"
     port: 5432
     database: postgres   # default system DB -- no dedicated "genesis" DB was provisioned
     admin_username: psqladmin
     auth_method: password
     tls_mode: require
     admin_password_ref:
       name: customer-genesis-secrets
       key: db-admin-password
   
   registry:
     # No "/genesis-platform" suffix -- images are pushed flat under the
     # registry root (<registry>/postgres, /genesis-be, etc).
     url: "<your-registry>.azurecr.io"
     image_pull_secret: ""   # acr_attached_to_aks handles pulls -- no imagePullSecret needed
     pattern: cloud-native   # node (kubelet) identity pulls, no pull secret
   
   networking:
     base_url: "https://<your-platform-hostname>"
     ingress_class: alb   # match whatever Ingress controller your cluster actually runs -- e.g. alb (Azure App Gateway), nginx, etc.
     tls_cert_secret: genesis-tls
     cluster_dns_suffix: cluster.local
     redis:
       # Azure Managed Redis (Microsoft.Cache/redisEnterprise), not classic
       # Azure Cache for Redis -- hence .redis.azure.net and port 10000
       # rather than .redis.cache.windows.net:6380.
       # No "@" before the host -- an empty-but-present userinfo segment
       # ("rediss://@host") makes `genesis preflight`'s redis check parse an
       # empty-string ACL username and send a 2-arg AUTH that Azure rejects
       # with WRONGPASS, even with the correct password. Omit "@" entirely.
       url: "rediss://<your-redis-host>.centralus.redis.azure.net:10000/0"
       password_ref:
         name: ai-studio-secrets
         key: "REDIS_PASSWORD"
       auth: true
       ssl_verify: true
       mode: cluster   # the database's clusteringPolicy is OSSCluster, not standalone
   
   oidc:
     # Control Plane's own admin login (genesis-bastion) -- separate from the
     # platform's internal Keycloak (genesis-idp), which needs no entry here.
     issuer_url: "https://login.microsoftonline.com/<your-azure-tenant-id>/v2.0"
     client_id: "<your-azure-ad-app-registration-client-id>"
     auth_mode: wif   # Workload Identity Federation -- no long-lived client secret
     wif_service_account: genesis-bastion
     required_role: genesis-platform-admin
   
   secrets:
     provider: azure-keyvault
     secret_store_name: azure-keyvault   # must match the ClusterSecretStore's metadata.name from step 07
     secret_store_kind: ClusterSecretStore
     k8s_secret_name: ai-studio-secrets
     refresh_interval: 1h
     external_secrets_enabled: true
   
   guardrails:
     change_freeze_start: ""
     change_freeze_end: ""
     break_glass_reason: ""
     confirm_prod: false   # set true for production environments
     helm_release_name: genesis-platform

   genesis configure \
     --from genesis.yaml \
     --save \
     --emit-helm-values /tmp/genesis-platform-values.yaml
   # ✓ genesis.yaml: PlatformConfig valid
   # ✓ genesis-platform-config created/updated in genesis

   Full field reference: configure.md

10. 10
    Install the ops bundle.

    The ops bundle is what installs the deploy operator, so it cannot go through the operator-driven genesis deploy path (chicken-and-egg — the error is explicit about this: "genesis-ops-*.tar.zst is an ops bundle — genesis deploy drives the platform install through the deploy operator, which the ops bundle itself provides").

    A bare zarf package deploy genesis-ops-${VER}.tar.zst gets further but fails on the relocate-umbrella-chart component: its helm push step reads Zarf's own built-in ZARF_REGISTRY variable, which without an explicit registry target defaults to Zarf's internal registry address (127.0.0.1:31999) — nothing is actually bound to that port for a plain deploy, so it's a guaranteed connection refused. The fix is supplying a real registry target, which currently requires the (release-scoped, soon-to-be-replaced) --local-helm flag:

    genesis deploy --local-helm \
      --bundle genesis-ops-${VER}.tar.zst \
      --registry ${REGISTRY} \
      -n genesis
    
    kubectl -n genesis get pods              # all Running

    You'll see a deprecation warning (--local-helm is deprecated... removed next release) — expected, not an error. It's the only currently-shipping way to get a registry target into this one bootstrap step; every install after this one goes through the operator normally.

    If zarf-docker-registry's pod gets stuck instead of reaching 1/1 Running, see the Troubleshooting reference at the bottom before continuing.

11. 11
    Run preflight.

    Preflight validates the cluster and your genesis.yaml before any platform component is installed. It reads the PlatformConfig saved in step 09 and checks things like Postgres reachability, Redis connectivity, ESO secret sync status, registry pull access, OIDC discovery, cert-manager, DNS, and TLS. Platform deploy is blocked until preflight passes; any FAIL/WARN row prints inline troubleshooting advice alongside it.

    genesis preflight -n genesis
    # Prints a per-check table (PASS / WARN / FAIL)
    # Exit 0 = every check PASSes -- proceed to step 12
    # Exit 1 = one or more FAIL -- fix and re-run

12. 12
    Deploy the platform bundle.

    Helm installs the full Genesis umbrella (~18 services, including Keycloak, APISIX, AI Studio, and Knowledge Center). Images pull from your registry; configuration comes from the values overlay emitted in step 09.

    Known first-install gaps A handful of the chart's default values don't work out of the box yet. Fix them with a small customer-overrides.yaml layered on top of the emitted values, plus one direct edit to the emitted file itself:

    # customer-overrides.yaml
    global:
      # genesis-lib's shared DB-init/schema-wait helper defaults every
      # subchart's init container to postgres:16-alpine. `genesis push-images`
      # doesn't relocate this image -- it's a chart-hardcoded utility
      # reference, not part of the bundle's own image manifest. Check what
      # actually landed (`az acr repository show-tags --name <registry>
      # --repository postgres`) and point this at whatever tag you have. This
      # override always wins per the chart's own _image.tpl comment.
      dbInitImage: "${REGISTRY}/postgres:latest"
    
    genesis-connectors:
      env:
        # Defaults to Azure Blob storage for dynamically-loaded connector
        # bundles, which needs a real Storage Account + container wired up
        # first. Disabling is the template's own documented option for first
        # install/non-prod -- see the vendor's customer-overrides.yaml template
        # for the production (Azure Blob) alternative.
        GENESIS_DYNAMIC_CONNECTOR_STORAGE_TYPE: "disabled"
    
    genesis-provisioner:
      database:
        # DB_HOST here is a plain literal from database.host, not derived from
        # genesis.yaml/PlatformConfig at all -- the chart's own default leaves
        # it blank and the pod crashes on startup without this.
        host: "<your-postgres-host>.postgres.database.azure.com"
    
    genesis-fe:
      env:
        # genesis-fe uses Auth.js v5 with AUTH_TRUST_HOST=true, which is meant
        # to derive the public base URL dynamically from the incoming request's
        # Host/X-Forwarded-* headers -- but those don't reliably survive the
        # Gateway -> in-cluster gateway -> genesis-fe hop chain, so it falls
        # back to its own bind address (0.0.0.0:3000) when building OIDC
        # callback URLs. See step 14's Troubleshooting entry for the exact
        # symptom this produces (an unrelated-looking WAF 403).
        AUTH_URL: "https://genesis.<your-dns-zone>"

    Separately, edit /tmp/genesis-platform-values.yaml itself (the file genesis configure --emit-helm-values just wrote) and flip one line:

    sed -i "s/KC_HOSTNAME_STRICT: 'true'/KC_HOSTNAME_STRICT: 'false'/" /tmp/genesis-platform-values.yaml

    genesis configure emits KC_HOSTNAME_STRICT: true whenever networking.base_url is set, but never emits a corresponding KC_HOSTNAME — and genesis-idp's chart hardcodes each Keycloak env var individually with no template slot for KC_HOSTNAME at all, so there's no values-based way to supply one. Left as true, Keycloak refuses to start (hostname is not configured), and the failure is silent from the outside — the pod just sits at Init:0/1 forever with no error visible in kubectl get pods. Restoring the chart's own default (false) is the fix. This has to be a direct edit to the emitted file, not a customer-overrides.yaml entry — the --local-helm deploy command below loads the emitted file after your overrides, so anything set there for this one field gets silently overwritten back to true.

    With both in place:

    cd ~/bundles   # so --bundle resolves to the downloaded tarball
    
    genesis deploy --local-helm \
      --bundle genesis-platform-${VER}.tar.zst \
      --registry ${REGISTRY} \
      --values /tmp/genesis-platform-values.yaml \
      --values customer-overrides.yaml \
      -n genesis

    You'll see the same --local-helm is deprecated warning as step 10 — expected. The fully operator-driven genesis deploy --bundle ... -n genesis (no extra flags) has no mechanism to layer in extra values at all — its GenesisDeployment CR has no field for it — so it only works once none of the overrides above are needed. Prefer it once these gaps are fixed upstream.

13. 12b
    RLS database roles (app_migrate / app_runtime).

    Some subcharts (confirmed: genesis-authz, genesis-be; likely others sharing the same SDK) enforce Row-Level Security and refuse to run against Postgres roles that can't guarantee tenant isolation. Two separate, easy-to-miss gaps show up here, both per-database — this platform spreads across ~10 separate databases on one Postgres server (keycloak, ai_studio, genesis_authz, genesis_tenant, genesis_prompts, knowledge_center, command_center, openfga, temporal, temporal_visibility) — a fix applied in one doesn't carry to the others. Treat a failure surfacing in one service as a signal to check the rest too, not just the one that happened to fail first.

    Gap 1 — migrations fail with permission denied for schema public: the migrate role lacks CREATE on schema public in that database. Run grant-schema.sql as the Postgres admin once the provisioner pod is Ready (roles exist) and before the app pods finish migrating — the script \connects into both RLS databases (ai_studio, genesis_authz), so one invocation covers both. It is idempotent; if the app pods have already crash-looped on this, they self-recover on their next migration retry once the grant lands.

    psql "postgresql://<admin>:<pw>@<host>:5432/postgres?sslmode=require" \
      -v ON_ERROR_STOP=1 -f docs/customer/scripts/grant-schema.sql

    Gap 2 — the app refuses to start with an RLS-bypass RuntimeError:

    RuntimeError: Configuration error: connected as Postgres role 'psqladmin' which can bypass RLS: rolbypassrls=true;
    member of cloud superuser group (rds_superuser / azure_pg_admin / cloudsqlsuperuser / pg_read_all_data / pg_write_all_data).
    App pods must connect as a role without RLS-bypass privileges (typically app_runtime).
    Refusing to start to avoid silent tenant-isolation bypass.

    The chart supports a dedicated non-bypass runtime role for exactly this — global.database.runtimeUser / per-service database.runtimeUser. Its password is always the same DB_PASSWORD secret used everywhere else (no separate runtime-password key exists — don't look for one). Fix, connected as admin:

    -- One-time, server-wide (roles aren't per-database, unlike schema grants).
    ALTER ROLE app_runtime WITH LOGIN PASSWORD '<same value as DB-PASSWORD>'
      NOSUPERUSER NOCREATEDB NOCREATEROLE NOBYPASSRLS NOREPLICATION;
    
    -- Per-database.
    GRANT USAGE ON SCHEMA public TO app_runtime;
    GRANT SELECT, INSERT, UPDATE, DELETE ON ALL TABLES IN SCHEMA public TO app_runtime;
    GRANT USAGE, SELECT ON ALL SEQUENCES IN SCHEMA public TO app_runtime;
    ALTER DEFAULT PRIVILEGES FOR ROLE app_migrate IN SCHEMA public
      GRANT SELECT, INSERT, UPDATE, DELETE ON TABLES TO app_runtime;
    ALTER DEFAULT PRIVILEGES FOR ROLE app_migrate IN SCHEMA public
      GRANT USAGE, SELECT ON SEQUENCES TO app_runtime;

    Then set the per-service override in customer-overrides.yaml (step 12) and redeploy:

    genesis-authz:
      database:
        runtimeUser: "app_runtime"

    Deep-dive: Prerequisites § “Grant schema CREATE to the RLS migrate role”.

14. 13
    Verify the install.

    genesis status -n genesis
    # Prints a phase summary: ops ✓  platform ✓  preflight ✓
    
    kubectl get pods -n genesis   # every pod should show 1/1 Running

15. 14
    Expose the platform (Gateway API, not Ingress).

    If your cluster's ingress path is Azure Application Gateway for Containers (the ALB Controller), it implements Gateway API (Gateway/HTTPRoute), not classic Ingress — there's no AGIC-compatible IngressClass, so a classic Ingress object (even with TLS/wildcard-cert annotations) would never be seen by the controller at all. Confirm which model your cluster uses (kubectl get gatewayclass vs kubectl get ingressclass) before assuming either way.

    For the Gateway API case: a shared Gateway (name/namespace depend on your setup, e.g. platform-gateway/azure-alb-system) typically already exists with an HTTP listener and an HTTPS listener backed by one combined cert-manager Certificate covering an explicit hostname list — not a wildcard. Every app needing TLS at this Gateway adds its hostname to that list rather than provisioning its own certificate/listener.

    1. Add your hostname to the shared certificate's coverage (wherever your Terraform manages the Gateway — e.g. a tls_hostnames input) and apply.
    2. Create an HTTPRoute pointing at your app's Service:

       apiVersion: gateway.networking.k8s.io/v1
       kind: HTTPRoute
       metadata:
         name: genesis
         namespace: genesis
       spec:
         parentRefs:
           - name: platform-gateway
             namespace: azure-alb-system
             sectionName: https
         hostnames:
           - genesis.<your-dns-zone>
         rules:
           - backendRefs:
               - name: genesis-gateway   # the in-cluster APISIX gateway Service
                 port: 80

    3. Point DNS at the Gateway's address. If External DNS isn't actually running in your cluster (check kubectl get pods -A | grep external-dns — Terraform may only provision its identity/RBAC, with the controller itself deployed separately, e.g. via a GitOps tool from a different repo), create the record yourself:

       GATEWAY_ADDR=$(kubectl get gateway platform-gateway -n azure-alb-system -o jsonpath='{.status.addresses[0].value}')
       az network dns record-set cname set-record -g <rg> -z <your-dns-zone> -n genesis -c "$GATEWAY_ADDR"

    Verify end-to-end:

    curl -v https://genesis.<your-dns-zone>/
    # Expect a valid TLS handshake (a cert covering your hostname) and an HTTP
    # response from the app -- not a connection error or a cert mismatch.