New nametfv is now teriac. See what changed and how to move over.
v0.2.1
teriac server Guide: EKS end to end

Guide: teriac server on EKS

From an empty namespace to your first reviewed apply: teriac server on Amazon EKS with IRSA, an ALB with HTTPS, Okta sign-in, per-project rules and drift alerts.

This guide takes you from nothing to a pull request that plans, gets approved, and applies, with teriac server running on Amazon EKS. It uses:

  • IRSA (IAM roles for service accounts), so Terraform gets AWS credentials without any keys;
  • an Application Load Balancer with an ACM certificate for HTTPS;
  • Okta (or any OIDC provider) for dashboard sign-in;
  • per-project rules, and a nightly drift check that posts to Slack.

Allow about an hour. The names used throughout:

Example
AWS account 123456789012
Region eu-central-1
Cluster platform
Hostname teriac.acme.dev
GitHub org and repo acme, acme/infra
Namespace teriac

1. Before you start

You need:

  • an EKS cluster on Kubernetes 1.27 or newer, with an IAM OIDC provider (needed for IRSA);
  • the AWS Load Balancer Controller, for the ALB;
  • the Amazon EBS CSI driver and a gp3 storage class, for teriac's volume;
  • an ACM certificate for teriac.acme.dev;
  • aws, kubectl and helm on your machine.

Check the OIDC provider, and create it if it's missing:

aws eks describe-cluster --name platform --query "cluster.identity.oidc.issuer" --output text
# https://oidc.eks.eu-central-1.amazonaws.com/id/EXAMPLED539D4633E53DE1B71EXAMPLE

eksctl utils associate-iam-oidc-provider --cluster platform --approve   # if not set up yet

2. An IAM role for Terraform

teriac runs Terraform inside its pod, so the pod needs the AWS access your Terraform needs. With IRSA, the pod's service account assumes an IAM role. The chart names the service account <release>-teriac-server, so with the release name teriac it's teriac-teriac-server.

Trust policy (trust.json): only that service account may assume the role.

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "Federated": "arn:aws:iam::123456789012:oidc-provider/oidc.eks.eu-central-1.amazonaws.com/id/EXAMPLED539D4633E53DE1B71EXAMPLE"
      },
      "Action": "sts:AssumeRoleWithWebIdentity",
      "Condition": {
        "StringEquals": {
          "oidc.eks.eu-central-1.amazonaws.com/id/EXAMPLED539D4633E53DE1B71EXAMPLE:sub": "system:serviceaccount:teriac:teriac-teriac-server",
          "oidc.eks.eu-central-1.amazonaws.com/id/EXAMPLED539D4633E53DE1B71EXAMPLE:aud": "sts.amazonaws.com"
        }
      }
    }
  ]
}
aws iam create-role --role-name teriac --assume-role-policy-document file://trust.json

Permissions. At minimum, Terraform needs your state backend. For an S3 backend:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": ["s3:ListBucket"],
      "Resource": "arn:aws:s3:::acme-terraform-state"
    },
    {
      "Effect": "Allow",
      "Action": ["s3:GetObject", "s3:PutObject", "s3:DeleteObject"],
      "Resource": "arn:aws:s3:::acme-terraform-state/*"
    }
  ]
}
aws iam put-role-policy --role-name teriac --policy-name terraform-state --policy-document file://state.json

Then attach what your Terraform manages. Keep it as narrow as you can. Many teams use one role per environment and have Terraform assume it through the provider's assume_role block, so prod access stays separate:

provider "aws" {
  region = "eu-central-1"
  assume_role {
    role_arn = "arn:aws:iam::210987654321:role/terraform-prod"
  }
}

Prefer EKS Pod Identity? Create a pod identity association for the teriac-teriac-server service account in the teriac namespace, and skip the role annotation in step 5. The AWS SDKs in Terraform pick up either one.

3. A GitHub App

Create the App as in the quick start:

  • Webhook URL: https://teriac.acme.dev/events/github
  • Events: pull request, issue comment, check run
  • Permissions: Checks, Issues and Pull requests (read & write), Contents (read), organization Members (read). The permissions table says what each is for

Install it on acme/infra, and keep the App ID, the private key (.pem) and the webhook secret.

4. Namespace and secrets

kubectl create namespace teriac

# GitHub App
kubectl -n teriac create secret generic teriac-github \
  --from-file=private-key=app.pem \
  --from-literal=webhook-secret="$WEBHOOK_SECRET"

# Okta app's client secret (for dashboard sign-in)
kubectl -n teriac create secret generic teriac-okta \
  --from-literal=client-secret="$OKTA_CLIENT_SECRET"

# Slack bot token (for drift alerts)
kubectl -n teriac create secret generic teriac-chat-tokens \
  --from-literal=TERIAC_SLACK_TOKEN="$SLACK_BOT_TOKEN"

In Okta, create a Web app with the sign-in redirect URI https://teriac.acme.dev/auth/oidc/callback, and add a groups claim to its ID token.

5. values.yaml

github:
  appId: "123456"
  existingSecret: teriac-github

config:
  repoAllowlist: [github.com/acme/infra]
  publicUrl: https://teriac.acme.dev     # set it: TLS ends at the ALB, so the chart can't tell it's https
  requireApplyProject: true              # apply one project at a time: teriac apply -p <project>
  preinstall: [terraform@1.9.8]          # optional: warm the cache with the versions you pin

serviceAccount:
  annotations:
    eks.amazonaws.com/role-arn: arn:aws:iam::123456789012:role/teriac

extraEnv:
  - { name: AWS_REGION, value: eu-central-1 }
extraEnvFrom:
  - secretRef: { name: teriac-chat-tokens }

persistence:
  storageClass: gp3
  size: 20Gi

ingress:
  enabled: true
  className: alb
  annotations:
    alb.ingress.kubernetes.io/scheme: internet-facing
    alb.ingress.kubernetes.io/target-type: ip
    alb.ingress.kubernetes.io/listen-ports: '[{"HTTPS": 443}]'
    alb.ingress.kubernetes.io/certificate-arn: arn:aws:acm:eu-central-1:123456789012:certificate/1111-2222
    alb.ingress.kubernetes.io/healthcheck-path: /healthz
  hosts:
    - host: teriac.acme.dev
      paths: [{ path: /, pathType: Prefix }]

dashboard:
  oidc:
    enabled: true
    issuer: https://acme.okta.com/oauth2/default
    clientId: 0oa1b2c3d4
    existingSecret: teriac-okta
    adminGroups: [platform-admins]
    allowedGroups: [engineering]

podDisruptionBudget:
  enabled: true                          # node drains wait for a running apply

repos:
  channels:
    infra-alerts: { type: slack, channel: "#infra-drift", token_env: TERIAC_SLACK_TOKEN }
  repos:
    - id: github.com/acme/infra
      apply_requirements: [mergeable]
      projects:
        - match: live/prod/**
          permissions:
            apply: [team:platform-admins]
            apply_destroy: [admin]
            force_unlock: []
          approvals: [{ from: team:platform-admins, count: 2 }]
        - match: "**"
          approvals: [{ from: write, count: 1 }]
      drift:
        - name: prod-nightly
          projects: ["live/prod/**"]
          schedule: "0 6 * * 1-5"
          timezone: Europe/Berlin
          notify: [infra-alerts]

Want the dashboard off the internet? Route only the webhooks through the public ALB, with paths: [{ path: /events/, pathType: Prefix }], and reach the dashboard with kubectl -n teriac port-forward svc/teriac-teriac-server 8080:80 or through an internal load balancer.

6. Install

helm install teriac oci://ghcr.io/marcdomain/charts/teriac-server \
  --version 0.2.1 -n teriac -f values.yaml

kubectl -n teriac rollout status statefulset/teriac-teriac-server
helm test teriac -n teriac

Point DNS at the ALB: a Route 53 alias record for teriac.acme.dev to the load balancer shown by kubectl -n teriac get ingress (or let external-dns do it). Then check:

curl https://teriac.acme.dev/healthz     # {"ok":true}

Check IRSA works. The role's details are injected into the pod:

kubectl -n teriac exec teriac-teriac-server-0 -- env | grep AWS_
# AWS_ROLE_ARN=arn:aws:iam::123456789012:role/teriac
# AWS_WEB_IDENTITY_TOKEN_FILE=/var/run/secrets/eks.amazonaws.com/serviceaccount/token

7. Connect GitHub

The webhook URL is already https://teriac.acme.dev/events/github. In the GitHub App's Advanced tab, Recent deliveries should show 200 responses from now on. A 401 means the webhook secret doesn't match.

Open https://teriac.acme.dev and sign in with Okta. Members of platform-admins are admins.

8. Get the repo ready

teriac finds projects on its own, so a typical repo needs nothing new. Two habits help:

Pin versions, so every plan uses the version you tested:

cd live/prod/vpc
teriac pin 1.9.8          # writes .terraform-version

Use a remote backend with locking. For example S3, with its native lock file (Terraform 1.10+):

terraform {
  backend "s3" {
    bucket       = "acme-terraform-state"
    key          = "live/prod/vpc/terraform.tfstate"
    region       = "eu-central-1"
    use_lockfile = true
  }
}

9. Your first pull request

Change something in live/prod/vpc and open a PR. Within moments:

  1. 👀 appears: teriac picked it up.
  2. A comment lists live/prod/vpc with its plan and the command to apply it.
  3. A check run, teriac: live/prod/vpc, appears with an Apply button.

Two members of platform-admins approve, since that's the rule for live/prod/**. Then apply:

teriac apply -p live/prod/vpc

teriac runs the saved plan, the one that was reviewed, and posts the apply report. Merge the PR, and its lock and plans are cleaned up.

If you click Apply too early, teriac says what's missing: approvals of the current commit, a mergeable PR, or a newer plan.

10. Drift alerts

The prod-nightly schedule checks live/prod/** at 06:00 on weekdays (Berlin time) with read-only plans, and posts to #infra-drift when something changed outside Terraform, or was merged but never applied. Each finding tags who last changed that resource.

Don't want to wait until tomorrow? Run it now from the dashboard's Drift page.

Troubleshooting

Symptom Check
Pod Pending kubectl -n teriac describe pod teriac-teriac-server-0. Usually the volume: is the EBS CSI driver installed, and does gp3 exist?
AccessDenied / no valid credential sources in a plan AWS_ROLE_ARN in the pod (step 6), the trust policy's sub (system:serviceaccount:teriac:teriac-teriac-server) and aud, and the role's permissions
No ALB created kubectl -n teriac describe ingress. The AWS Load Balancer Controller logs say why (subnet tags, certificate ARN, permissions)
GitHub deliveries fail DNS for teriac.acme.dev, the ALB's security group allows 443, and the webhook secret
Okta sign-in refused Your groups are in adminGroups or allowedGroups, and the ID token carries the groups claim
Links in comments use http:// Set config.publicUrl: https://teriac.acme.dev

Next