> For the complete documentation index, see [llms.txt](https://docs.p0.dev/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.p0.dev/agentic-access/agentic-gateway/gateway/deploy.md). # Deploying the gateway Deploy the self-hosted P0 AI Gateway and Auth Server from the P0 console, with the batteries-included Helm chart, or with the basic Helm chart. The P0 AI Gateway is the self-hosted runtime enforcement layer for [P0 Runtime Access for AI](/readme/agentic-control-plane.md). You deploy it into your own Kubernetes cluster, where it sits in the data path between your AI agents and your MCP servers. Because it runs in your environment, your sensitive data and upstream credentials never leave your perimeter. This guide covers how to install the self-hosted components. For how they fit together with the P0 Controller, see the [P0 Runtime Access for AI](/readme/agentic-control-plane.md#architecture). ## Prerequisites * An account with AWS, Google Cloud, or Azure, with billing enabled and permission to create a Kubernetes cluster. The gateway runs on your infrastructure, so P0 provisions nothing here. * An administrator account at the identity provider your people sign in with, such as Google Workspace, Okta, or Entra ID. You register an OAuth client there below. * A P0 organization. If you don't have one, see [Set up P0 for your organization](/getting-started/p0-security-onboarding.md#account-setup). * A dedicated Kubernetes cluster (EKS, GKE, or AKS). EKS Auto Mode is supported. Its storage and load balancer setup differ from standard EKS, so read [#eks-auto-mode-storage](#eks-auto-mode-storage "mention") and [#eks-load-balancer](#eks-load-balancer "mention") before you install. Some of what the chart installs applies to the whole cluster, so plan on a cluster that runs nothing else. See [#what-the-chart-installs-cluster-wide](#what-the-chart-installs-cluster-wide "mention") if you need to share one, and [#cluster-size](#cluster-size "mention") for what the bundled components need. * A block-storage `StorageClass` for the bundled PostgreSQL, VictoriaLogs and Valkey. A `StorageClass` is the cluster's recipe for handing disk to a pod that needs to keep data. Valkey's volume holds OAuth credentials for SaaS MCP servers, so use a class that encrypts disks. For example, `gp3` on EKS once you install the EBS CSI driver add-on (see [#standard-eks-storage](#standard-eks-storage "mention")), `standard-rwo` on GKE, or `managed-csi` on AKS. On an EKS cluster in Auto Mode (see [#eks-auto-mode-storage](#eks-auto-mode-storage "mention")), none of these work until you create a storage class yourself. If your cluster has no storage class at all, see [#evaluating-without-a-storage-class](#evaluating-without-a-storage-class "mention"). * An OAuth 2.0 client (client ID and client secret) for user sign-in, registered at your identity provider. Any provider works that serves an OpenID Connect discovery document and supports standard Authorization Code with `client_secret_post` (for example, Google or Okta). The client must allow the `openid`, `email`, and `profile` scopes, which are the ones the gateway requests. On providers that have a consent screen, such as Google, configure it before you install: an unconfigured screen blocks sign-in, and the failure shows up the first time someone signs in rather than during the install. * `kubectl`, the Kubernetes command-line tool, configured against your cluster. You use it to read the gateway's address and to set your OIDC client secret after the install. * `terraform` (≥ 1.7, with the `hashicorp/helm` provider ≥ 3.0). Terraform applies a configuration file describing what to install, so your gateway deployment is reproducible and reviewable. The Helm path in [#deployment-options](#deployment-options "mention") uses `helm` (v3, with OCI support) instead, which installs the same packaged application from the command line. * Control over a public DNS record for the gateway's hostname, if you bring your own domain. With a P0-hosted domain, P0 creates the DNS record for you. * Network access as listed in [#network-requirements](#network-requirements "mention"): outbound from the cluster to P0, your identity provider, Let's Encrypt and the image registries, and inbound to the gateway's load balancer on ports 80 and 443. ## Deployment options | Option | What it is | When to use it | | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------- | | [Install from the P0 console](/agentic-access/agentic-gateway/gateway/deploy/install-from-the-p0-console.md) | A guided install in the P0 console. You enter your environment's details, and P0 generates a Terraform snippet that deploys the batteries-included chart, registers the gateway, and (with a P0-hosted domain) creates its DNS record. | Standard installs. **Recommended.** | | [Install the batteries-included Helm chart](/agentic-access/agentic-gateway/gateway/deploy/install-the-batteries-included-chart.md): [`p0security/ai-gateway-stack`](https://hub.docker.com/r/p0security/ai-gateway-stack) | The umbrella chart the console installer deploys, run by hand. It deploys the gateway and Auth Server **plus** everything they depend on: Envoy Gateway, cert-manager (with Let's Encrypt), PostgreSQL, Valkey, VictoriaLogs, and an OpenTelemetry collector. | Installs that need values the console doesn't expose, or that must be driven entirely from your own pipeline. | Every path deploys the same workloads. The console installer differs in that it also registers the gateway with P0 as you go, so you don't configure the [Gateway](/agentic-access/agentic-gateway/gateway.md) separately afterward. For every value you can set on either path, see the [Helm chart values reference](/agentic-access/agentic-gateway/gateway/deploy/chart-values.md). To move a running gateway to a newer chart version, see [Upgrade the gateway](/agentic-access/agentic-gateway/gateway/deploy/upgrade-the-gateway.md). ## Secrets the chart creates The chart creates the `app-secrets` Secret for you, from a job that runs before anything that reads it. Every key is generated. The OIDC client secret is the exception you must always set yourself, since only you have it: [/pages/dTpYGsFvf4WWGEvrXyvA#id-3.-set-your-oidc-client-secret](https://docs.p0.dev/agentic-access/agentic-gateway/gateway/pages/dTpYGsFvf4WWGEvrXyvA#id-3.-set-your-oidc-client-secret "mention") gives the command. If you use your own database instead of the bundled PostgreSQL, replace the generated `POSTGRESQL_PASSWORD` with the real one the same way. | Key | Description | | ------------------------------------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `JWT_PRIVATE_KEY` / `JWT_PUBLIC_KEY` | Key pair the Auth Server uses to sign and verify tokens. PEM block or base64-encoded PEM block. | | `KMS_LOCAL_ENCRYPTION_KEY` | 32-byte (64 hex character) key for local encryption. | | `REFRESH_TOKEN_SECRET` | Base64-encoded secret (≥32 bytes) for refresh tokens. | | `POSTGRESQL_PASSWORD` | Password for the bundled PostgreSQL. | | `OIDC_CLIENT_SECRET` | The client secret for your OAuth client at your identity provider. Starts as a placeholder, but sign-in fails until you set the real value, as in [/pages/dTpYGsFvf4WWGEvrXyvA#id-3.-set-your-oidc-client-secret](https://docs.p0.dev/agentic-access/agentic-gateway/gateway/pages/dTpYGsFvf4WWGEvrXyvA#id-3.-set-your-oidc-client-secret "mention"). | The job runs again on every upgrade, filling in only missing keys. It checks existing keys and fails the install or upgrade if it finds malformed values. The job log names the key to fix or remove. The same job also creates a second, separate `braekhus-client-jwk` Secret holding the EC P-384 keypair (`BRAEKHUS_JWK_PRIVATE`/`BRAEKHUS_JWK_PUBLIC`) for the gateway's outbound management tunnel (see [Gateway](/agentic-access/agentic-gateway/gateway.md#an-outbound-management-tunnel)). Registering the tunnel with P0 is required for every new gateway. Read back the public half and paste it into the console: ```bash kubectl -n p0-agentic-gateway exec deploy/braekhus-client -- cat /p0-files/jwk.public.json ``` See [Install from the P0 console](/agentic-access/agentic-gateway/gateway/deploy/install-from-the-p0-console.md#provide-the-management-tunnel-public-key) for where this goes in the console installer. {% hint style="info" %} To manage the `app-secrets` Secret yourself, from External Secrets, Vault, or a sealed-secrets pipeline, set `ai-gateway.secretsJob.enabled: false` in your values and create your namespace with `app-secrets` in it before installing. {% endhint %} ## What the chart installs cluster-wide Most of what the chart deploys lives in its own namespace. Three things do not, and they are why the cluster should be dedicated: * **Envoy Gateway v1.9.0**, which routes outside traffic to the gateway, and its `GatewayClass`. * **cert-manager v1.16.1**, which obtains and renews the gateway's TLS certificate from Let's Encrypt. * **24 custom resource definitions** that those two depend on, covering the Gateway API, Envoy, and cert-manager. Installing them needs cluster administrator rights. The chart also creates cluster roles and two admission webhooks for those components. A cluster that already runs cert-manager is the case to watch: a second copy conflicts with the one already there, and the conflict surfaces as certificates that never issue. Another ingress controller is usually fine. Envoy Gateway only serves the `GatewayClass` the chart creates, so a controller that answers to a different class runs alongside it. An existing Envoy Gateway is the exception, since both copies claim the same controller name. If you must install onto a shared cluster, you can turn the bundled copies off and let the cluster's own components serve the gateway: ```yaml cert-manager: enabled: false envoy-gateway: enabled: false ai-gateway: gateway: className: my-release ``` These flags only stop the chart installing its own copies. The cluster has to already run a cert-manager that can issue the chart's `Certificate` resources and a Gateway API controller that serves the `GatewayClass` named above, or certificates never issue and the gateway never gets an address. `GatewayClass` names are cluster-wide, so give each install its own, such as the Helm release name. Set these in the module's `values` block for a console install, or in `my-values.yaml` for a Helm install. You then own keeping those components at versions the gateway works with. ## Cluster size The batteries-included chart runs twelve long-lived pods: the gateway and Auth Server, the braekhus management tunnel client, PostgreSQL, Valkey, VictoriaLogs, an OpenTelemetry collector, Envoy Gateway and its proxy, and cert-manager's three. Install and upgrade also run a few short-lived Jobs. A [Custom](/agentic-access/agentic-gateway/mcp-server/custom.md) of the **Container** type adds a pod for each session that uses it, for as long as the session lasts. Each component's CPU and memory requests and limits, its storage, and the memory it used on a lightly used test install. Expect more under real traffic. | Component | Pods | Requests (CPU, memory) | Limits (CPU, memory) | Memory used | Storage | | ----------------------- | ------ | ---------------------- | -------------------- | ------------------ | -------- | | Gateway | 1 | None | None | 224Mi | None | | Auth Server | 1 | None | None | 96Mi | None | | braekhus tunnel client | 1 | 50m, 64Mi | None, 256Mi | 162Mi | None | | OpenTelemetry collector | 1 | 100m, 256Mi | 500m, 512Mi | 37Mi | None | | PostgreSQL | 1 | None | None | 24Mi | 8Gi | | Valkey | 1 | None | None | 4Mi | 1Gi | | VictoriaLogs | 1 | None | None | 53Mi | 20Gi | | Envoy Gateway | 1 | 100m, 256Mi | None, 1Gi | 74Mi | None | | Envoy proxy | 1 | 110m, 544Mi | None | 40Mi | None | | cert-manager | 3 | None | None | 65Mi for all three | None | | **Total** | **12** | **360m, 1.1Gi** | | **About 800Mi** | **29Gi** | Install and upgrade Jobs run one at a time and finish within minutes. The largest requests 50m CPU and 64Mi of memory. The three volumes come from the `StorageClass` above. VictoriaLogs holds the gateway's tool-call logs, so it is the volume that grows with use. The cluster reserves the requested CPU and memory for the stack whether the pods use it or not. Pods with no requests take what the node has free, so leave room above the totals for them to grow under traffic. Size nodes like this: | Use | Nodes | Each node | EKS example | | ------------------ | ----- | ------------- | --------------------------------- | | Trying the gateway | 1 | 2 vCPU, 8 GiB | `t3.large`, or `t4g.large` on Arm | | Production | 2 | 2 vCPU, 8 GiB | `t3.large`, or `t4g.large` on Arm | On EKS, small instance types also cap pods per node. A `t3.medium` allows 17, and the stack's twelve pods plus the cluster's own system pods can exceed that. The extra pods then sit `Pending` with no resource pressure to explain it. A `t3.large` allows 35. Each session on a **Container** custom MCP server also counts against that cap, and its pod sets no CPU or memory requests. If many people use Container servers at once, add nodes for those sessions on top of the sizes above. The chart needs Kubernetes 1.33 or later. P0's images are built for amd64 and arm64, so Graviton and other Arm nodes work. On EKS Auto Mode, the cluster adds nodes as pods need them, so there is nothing to size beyond the storage above. ## Network requirements The gateway dials out to P0 and keeps that connection open, so P0 needs no inbound access to your cluster. Two things do connect in from the internet, and the gateway's load balancer has to accept them: | Port | Who connects | Why | | ---- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | 80 | Let's Encrypt | Checks that you control the hostname before issuing the certificate, and again at each renewal. Let's Encrypt publishes no IP ranges, so leave the port open to the internet. | | 443 | MCP clients and the people who sign in through them. AWS and Google Cloud too, once you install [AWS OIDC](/agentic-access/credential-sources/aws-oidc.md) or [GCP Workload Identity Federation](/agentic-access/credential-sources/gcp-wif.md). | Tool calls and sign-in. The cloud integrations fetch the gateway's signing keys from this port to verify the tokens it mints. | Outbound, the cluster's nodes need port 443 to these hosts: | Host | Used for | | ------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `api.p0.app` | The P0 API and the management tunnel. | | Your identity provider's issuer. For Google, `accounts.google.com`, `oauth2.googleapis.com` and `www.googleapis.com`. | User sign-in: the provider's discovery document, token exchange and signing keys. The issuer is the `authServer.oidcIssuer` value (see [#user-sign-in-settings](#user-sign-in-settings "mention")). | | `www.googleapis.com` | Verifying P0's management calls, which always use a Google identity. Needed whichever identity provider you use. | | `acme-v02.api.letsencrypt.org`, or `acme-staging-v02.api.letsencrypt.org` when `letsEncrypt.env` is `staging` | Certificate issuance and renewal. | | `registry-1.docker.io`, `auth.docker.io`, `production.cloudflare.docker.com` and `production.cloudfront.docker.com` | Image pulls from Docker Hub: P0's images, PostgreSQL, Valkey, VictoriaLogs, the OpenTelemetry collector and Envoy. Docker Hub serves image layers from the last two hosts. | | `quay.io`, `cdn.quay.io`, and `cdn01.quay.io` through `cdn06.quay.io` | Image pulls for cert-manager. A `*.quay.io` rule covers them all. | | `sts.googleapis.com` and `iamcredentials.googleapis.com` | Only with GCP Workload Identity Federation. Add `secretmanager.googleapis.com` if P0 keeps any MCP server's credentials in Google Secret Manager. | | `sts..amazonaws.com`, which is `sts.us-east-1.amazonaws.com` unless you change it | Only with AWS OIDC. The region is the `authServer.awsRegion` value (see [AWS OIDC](/agentic-access/credential-sources/aws-oidc.md#sts-endpoint-region)). | | `identitytoolkit.googleapis.com` and `securetoken.google.com` | Only with `authServer.loginMode: gcip` (see [Gateway chart values](/agentic-access/agentic-gateway/gateway/deploy/chart-values/gateway-chart.md#user-sign-in)), where the gateway shows its own password form and checks it against Google Cloud Identity Platform instead of redirecting to your identity provider. | | The MCP servers and A2A agents you put behind the gateway | Whatever host and port each one listens on. | | The registries that hold your custom MCP servers' **Container** images (see [Custom](/agentic-access/agentic-gateway/mcp-server/custom.md)) | Image pulls when a session starts one of those servers. | The nodes also need port 80 to the gateway's own public address. Before Let's Encrypt checks the hostname, cert-manager fetches the same challenge from inside the cluster. If that request can't get out and back in through the load balancer, the certificate never issues. ## Evaluating without a storage class Some clusters have no storage class at all, and no way to add one quickly. You can still stand the gateway up to try it, with both bundled datastores writing to scratch space on the node instead of to a volume. {% hint style="warning" %} Scratch space is erased whenever a pod is rescheduled, which a node upgrade, an eviction, or a change to the pod's own settings all cause. The gateway loses its registered MCP clients, its sessions, and its tool-call history, and there is no way to recover them. It also loses all OAuth credentials whenever the Auth Server pod restarts. Users have to sign in to their SaaS MCP servers again, and admins have to register their headless agent credentials again. Use this to evaluate the gateway, not to run anything you depend on. {% endhint %} In the console installer (see [Install from the P0 console](/agentic-access/agentic-gateway/gateway/deploy/install-from-the-p0-console.md)), choose **No persistent storage (proof of concept only)** for **Gateway storage**. If you install the chart directly (see [Install the batteries-included Helm chart](/agentic-access/agentic-gateway/gateway/deploy/install-the-batteries-included-chart.md)), turn persistence off for both datastores in `my-values.yaml`: ```yaml postgresql: persistence: enabled: false victorialogs: persistence: enabled: false valkey: enabled: false ai-gateway: authServer: caches: '{"defaults":[{"type":"memory"}]}' ``` When you install the chart directly, these values also turn off the bundled Valkey and keep the Auth Server's cache in memory. VictoriaLogs drops its oldest logs once they fill 80% of its space, which `victorialogs.storageSize` sets and which defaults to `20Gi`. It always keeps the two newest days, so if those outgrow that 80%, the pod is evicted and its logs are lost. Neither datastore reserves disk on the node, so leave the node room for both, or a full disk can evict the gateway too. After the data is erased, PostgreSQL restarts empty. The gateway and the Auth Server keep reporting healthy, because their health checks do not query the database, so the first sign of trouble is usually a failed sign-in. Wait for the database, then restart both to rebuild the schema: ```bash kubectl -n p0-agentic-gateway rollout status statefulset/postgresql kubectl -n p0-agentic-gateway rollout restart deployment/agentic-gateway-server deployment/agentic-auth-server ``` Once your cluster has a storage class, you can move the gateway onto volumes. In `my-values.yaml`, set `persistence.enabled` back to `true` and name the class in `storageClass` for both datastores. To move to the bundled Valkey too, set `valkey.enabled` to `true`, set `valkey.storageClass` to an encrypted class, and remove the `caches` line. Kubernetes cannot move a running datastore from scratch space onto a volume, so the upgrade fails with a `Forbidden: updates to statefulset spec` error unless you delete both datastores first: ```bash kubectl -n p0-agentic-gateway delete statefulset postgresql victorialogs ``` Then run the upgrade. If it already failed, delete the datastores and run it again. The gateway is down until the upgrade recreates them, and the database comes back empty, so run the two recovery commands above once it finishes. ## Standard EKS storage A standard EKS cluster lists a `gp2` storage class from the day you create it, but no volume can be provisioned from it until you install the Amazon EBS CSI driver add-on. Without the add-on, the PostgreSQL, VictoriaLogs and Valkey volumes stay `Pending` and the install times out. On an Auto Mode cluster, skip this section and see [#eks-auto-mode-storage](#eks-auto-mode-storage "mention") instead. On a cluster your team already uses, someone may have installed the add-on. Check first: ```bash aws eks describe-addon --cluster-name --addon-name aws-ebs-csi-driver \ --query "addon.[status,serviceAccountRoleArn]" ``` If it shows `ACTIVE` and a role ARN, skip to step 4. If it shows a status but no role, the driver can't create volumes: do steps 1 to 3, and in step 3 run `aws eks update-addon` with the same arguments instead of `create-addon`. A `ResourceNotFoundException` means the add-on isn't installed, so start at step 1. 1. **Give the cluster an IAM OIDC provider.** The driver signs in to AWS through it, and AWS doesn't create one with the cluster. This [`eksctl`](https://eksctl.io/installation) command creates it, and only reports it if one already exists: ```bash eksctl utils associate-iam-oidc-provider --cluster --approve ``` 2. **Create the driver's IAM role.** The add-on creates EBS volumes on your behalf, so it needs a role with the `AmazonEBSCSIDriverPolicyV2` managed policy that only the driver's `ebs-csi-controller-sa` service account can use. This command creates the role with that policy and trust: ```bash eksctl create iamserviceaccount --name ebs-csi-controller-sa --namespace kube-system \ --cluster --role-name AmazonEKS_EBS_CSI_DriverRole --role-only \ --attach-policy-arn arn:aws:iam::aws:policy/AmazonEBSCSIDriverPolicyV2 --approve ``` To use EKS Pod Identity instead, follow AWS's [Use Kubernetes volume storage with Amazon EBS](https://docs.aws.amazon.com/eks/latest/userguide/ebs-csi.html) guide, and pass `--pod-identity-associations` in place of `--service-account-role-arn` in the next step. 3. **Install the add-on** with that role: ```bash aws eks create-addon --cluster-name --addon-name aws-ebs-csi-driver \ --service-account-role-arn arn:aws:iam:::role/AmazonEKS_EBS_CSI_DriverRole ``` 4. **Create a `gp3` storage class.** It gives you AWS's current general-purpose volume type with encryption at rest. The stock `gp2` class works once the add-on runs, but it doesn't encrypt Valkey's volume: ```yaml apiVersion: storage.k8s.io/v1 kind: StorageClass metadata: name: gp3 provisioner: ebs.csi.aws.com volumeBindingMode: WaitForFirstConsumer parameters: type: gp3 encrypted: "true" ``` ```bash kubectl apply -f storage-class.yaml ``` 5. **Name the class when you install.** Enter `gp3` in the **Kubernetes storage class** field of the console installer (see [Install from the P0 console](/agentic-access/agentic-gateway/gateway/deploy/install-from-the-p0-console.md)), or set `postgresql.storageClass`, `victorialogs.storageClass` and `valkey.storageClass` to it in `my-values.yaml` if you install the chart directly (see [Install the batteries-included Helm chart](/agentic-access/agentic-gateway/gateway/deploy/install-the-batteries-included-chart.md)). After the install, all three volume claims should read `Bound`: ```bash kubectl -n p0-agentic-gateway get pvc ``` ## EKS Auto Mode storage EKS Auto Mode clusters need a storage class that you create yourself. Auto Mode ships none, so the chart's PostgreSQL, VictoriaLogs and Valkey volumes stay unbound until you add one. What makes this hard to spot is that the cluster looks ready. The `gp2` class listed on an Auto Mode cluster comes from standard EKS cluster setup rather than from you, and Auto Mode can't provision against it. So `kubectl get storageclass` shows something plausible and the install proceeds. The pods then sit in `Pending` with their claims never bound, and nothing reports an error, because from Kubernetes' side nothing's gone wrong. To check whether this applies to your cluster: ```bash kubectl get storageclass ``` If the only entry is `gp2`, or the list is empty, create a storage class before installing. AWS's [Create a storage class](https://docs.aws.amazon.com/eks/latest/userguide/create-storage-class.html) guide names it `auto-ebs-sc`: ```yaml apiVersion: storage.k8s.io/v1 kind: StorageClass metadata: name: auto-ebs-sc annotations: storageclass.kubernetes.io/is-default-class: "true" allowedTopologies: - matchLabelExpressions: - key: eks.amazonaws.com/compute-type values: - auto provisioner: ebs.csi.eks.amazonaws.com volumeBindingMode: WaitForFirstConsumer parameters: type: gp3 encrypted: "true" ``` ```bash kubectl apply -f storage-class.yaml ``` The provisioner must be `ebs.csi.eks.amazonaws.com`. That's Auto Mode's built-in block storage, which isn't the same thing as the EBS CSI driver add-on. Don't install that add-on on an Auto Mode cluster. If you would rather not change the cluster default, leave the annotation out. In either case you must name the class explicitly: enter `auto-ebs-sc` in the **Kubernetes storage class** field of the console installer (see [Install from the P0 console](/agentic-access/agentic-gateway/gateway/deploy/install-from-the-p0-console.md)), or set it in `my-values.yaml` if you install the chart directly (see [Install the batteries-included Helm chart](/agentic-access/agentic-gateway/gateway/deploy/install-the-batteries-included-chart.md)). The chart does not fall back to the cluster default. {% hint style="warning" %} `allowedTopologies` restricts volumes to Auto Mode nodes. On a cluster that mixes Auto Mode and standard node groups, pods that need these volumes must also be scheduled onto Auto Mode nodes, or they stay `Pending` for the same reason. {% endhint %} ## EKS load balancer The chart exposes the gateway through a Kubernetes Service of type `LoadBalancer`, and EKS turns that into an AWS load balancer. Which kind depends on the cluster: * **EKS Auto Mode** creates a Network Load Balancer itself, with no controller to install. Auto Mode makes a new load balancer internal unless told otherwise, so the chart sets `service.beta.kubernetes.io/aws-load-balancer-scheme: internet-facing` on the Service. Auto Mode also needs your public subnets tagged `kubernetes.io/role/elb` to place it. Without the tag, the gateway never gets an address. * **Standard EKS** creates a Classic Load Balancer through the load balancer support built into EKS, public by default. The chart's `service.beta.kubernetes.io/aws-load-balancer-scheme: internet-facing` annotation is ignored. No Helm values need to change if a public Classic Load Balancer is acceptable. * To get a Network Load Balancer instead, install the [AWS Load Balancer Controller](https://docs.aws.amazon.com/eks/latest/userguide/aws-load-balancer-controller.html), tag your public subnets `kubernetes.io/role/elb`, and add its annotations through `envoyProxy.serviceAnnotations`. They merge with the chart's default: ```yaml envoyProxy: serviceAnnotations: service.beta.kubernetes.io/aws-load-balancer-type: external service.beta.kubernetes.io/aws-load-balancer-nlb-target-type: ip ``` An internal load balancer is not supported with this chart. Let's Encrypt has to reach port 80 from the internet to issue and renew the certificate, and MCP clients, AWS and Google Cloud reach port 443 the same way. Behind an internal load balancer the certificate never issues, so the install never finishes. If your policy rules out a public load balancer, the gateway chart on its own (see [Gateway chart values](/agentic-access/agentic-gateway/gateway/deploy/chart-values/gateway-chart.md)) lets you bring a certificate and a Gateway API controller you run. Contact P0 before you start down that path. ### Load balancer idle timeout The load balancer closes any connection that carries no traffic for a set time. A tool call or agent call carries none while the upstream server works on its reply, the same gap that [#stream-timeouts-for-long-running-calls](#stream-timeouts-for-long-running-calls "mention") covers for Envoy. A Network Load Balancer allows 350 seconds of quiet. A Classic Load Balancer allows 60 seconds, less than Envoy's 5-minute default, so on standard EKS it is the first limit a quiet call hits. Set the load balancer's idle timeout to at least your `streamIdleTimeout`, through the same `envoyProxy.serviceAnnotations` value as above. A Network Load Balancer, from Auto Mode or from the AWS Load Balancer Controller version 2.9 or later, accepts up to 6000 seconds. The `TCP-443` in the annotation key names the listener that carries the gateway's traffic: ```yaml envoyProxy: serviceAnnotations: service.beta.kubernetes.io/aws-load-balancer-listener-attributes.TCP-443: tcp.idle_timeout.seconds=3600 ``` This works because the chart ends TLS at Envoy and the listener is plain TCP. If you add `aws-load-balancer-ssl-cert` to end TLS at the Network Load Balancer, the listener's idle timeout is fixed at 350 seconds and can't be raised. A Classic Load Balancer accepts up to 4000 seconds: ```yaml envoyProxy: serviceAnnotations: service.beta.kubernetes.io/aws-load-balancer-connection-idle-timeout: "3600" ``` ## Stream timeouts for long-running calls MCP tool calls and A2A agent calls stream their responses back through Envoy. The gateway doesn't send heartbeats on these streams, so a stream carries no traffic while a tool or agent is working on a reply. Envoy closes any stream that stays idle longer than its stream idle timeout, which is 5 minutes by default. The chart disables Envoy's request timeout (`requestTimeout: "0s"`), but that is a separate setting: the stream idle timeout still applies, and it cuts off any call that goes 5 minutes without sending anything back. The agent sees the call fail partway through, even though the upstream server may still be working. Set `streamIdleTimeout` to much larger than the longest call you expect. If your slowest tool call or agent task can run for 20 minutes, set it to an hour rather than 25 minutes, so a slower-than-usual call still finishes: ```yaml ai-gateway: gateway: streamIdleTimeout: 1h ``` Set this in the module's `values` block for a console install, or in `my-values.yaml` for a Helm install. The value takes a duration such as `30m` or `1h`. The chart applies it as an Envoy Gateway `BackendTrafficPolicy` only when the value is set, so it has no effect if you've turned off the bundled Envoy Gateway (see [#what-the-chart-installs-cluster-wide](#what-the-chart-installs-cluster-wide "mention")) in favor of another Gateway API controller. In that case, raise the equivalent timeout in that controller instead. On EKS, the load balancer in front of Envoy has an idle timeout of its own, and it can be shorter than this one. Raise it as well, as described under [#load-balancer-idle-timeout](#load-balancer-idle-timeout "mention"). On AKS, the Azure load balancer closes a connection after 4 minutes with no traffic. If you raise `streamIdleTimeout`, raise the load balancer's timeout to match through `envoyProxy.serviceAnnotations`. The value is in minutes, from 4 to 100: ```yaml envoyProxy: serviceAnnotations: service.beta.kubernetes.io/azure-load-balancer-tcp-idle-timeout: "60" ``` A2A calls have idle timeouts of their own in the gateway, which reset each time the stream carries data. To let a long A2A call through, raise these alongside `streamIdleTimeout`: | Value | Default | Purpose | | ------------------------------------------ | ---------- | --------------------------------------------------------------------------------------------------- | | `accessProxy.a2aProxyIdleTimeoutMs` | 60 seconds | Idle limit between the gateway and the A2A bridge. | | `accessProxy.a2aUpstreamIdleTimeoutMs` | 60 seconds | Idle limit between the A2A bridge and the upstream agent. | | `accessProxy.a2aAgentHandlerIdleTimeoutMs` | 5 minutes | How long the gateway keeps an idle upstream agent connection. Keep it equal to `streamIdleTimeout`. | These take milliseconds, and sit under `ai-gateway:` in the batteries-included chart's values, next to `gateway`. MCP calls have no gateway-side timeout, so for MCP, `streamIdleTimeout` is the only limit. ## User sign-in settings Every path defaults to Google as the upstream identity provider. To use another provider, set these under `authServer`, in the module's `values` block for a console install or in your values file for a Helm install: | Value | Purpose | | -------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `oidcIssuer` | Your provider's issuer URL, exactly as its discovery document states it. Defaults to `https://accounts.google.com`. | | `oidcRequireEmailVerified` | Set to `"false"` for providers that omit `email_verified` from the ID token. The gateway still matches `openIdDomain` against the `email` claim, so with this off, `openIdDomain` is what limits who can sign in. | | `oidcPrompt` | The `prompt` sent on the authorize leg. Set to `""` to send none, so the IdP reuses an existing session instead of showing an account chooser. | | `openIdDomain` | Login allowlist: a regex matched against the whole `id_token` `email` claim, for example \`\[^@]\*@(example\[.]com | Register the gateway's URL, `https://`, as the OAuth callback URL at your provider. The gateway always uses that URL. What to set for each provider: * **Google**: the defaults are Google's. Set `openIdDomain` to your domain, because an empty value admits any verified Google account. * **Okta**: if you use the org authorization server, the issuer is `https://.okta.com`, with no path. The custom `default` server's issuer is `https://.okta.com/oauth2/default`. A custom server needs an access policy with a rule that allows your sign-in app, or every sign-in fails. See Okta's [Create access policies](https://help.okta.com/oie/en-us/content/topics/security/api-config-access-policies.htm). Okta doesn't send `email_verified`, so set `oidcRequireEmailVerified` to `"false"` and set `openIdDomain`. * **Entra ID**: the issuer is `https://login.microsoftonline.com//v2.0`, with your directory (tenant) ID. Set `oidcRequireEmailVerified` to `"false"` and set `openIdDomain`. Entra ID sends an `email` claim only for users with an email address on their account, so a user without one can't sign in. For a Helm install, Google looks like this in `my-values.yaml`. The module's `values` block takes the same keys: ```yaml ai-gateway: authServer: oidcClientId: openIdDomain: "[^@]*@example[.]com" ``` Okta looks like this: ```yaml ai-gateway: authServer: oidcClientId: oidcIssuer: https://.okta.com/oauth2/default oidcRequireEmailVerified: "false" openIdDomain: "[^@]*@example[.]com" ``` Entra ID looks like this: ```yaml ai-gateway: authServer: oidcClientId: oidcIssuer: https://login.microsoftonline.com//v2.0 oidcRequireEmailVerified: "false" openIdDomain: "[^@]*@example[.]com" ``` ## Next steps Once the gateway is deployed, federate cloud credentials so it can obtain short-lived credentials for the agents it serves. For AWS, install [AWS OIDC](/agentic-access/credential-sources/aws-oidc.md). For Google Cloud, install [GCP Workload Identity Federation](/agentic-access/credential-sources/gcp-wif.md). Install the one that matches the servers you are about to configure, or both.