Gateway API + Envoy: Production Deployment

Prerequisites

To follow this guide, you'll need:

• A running Kubernetes cluster (v1.26+ recommended).
• kubectl installed and configured to connect to your cluster.
• Helm installed (v3.8.0+).
• Basic understanding of Kubernetes concepts (Pods, Services, Deployments).
• Familiarity with networking concepts (HTTP, DNS, load balancing).

Step-by-Step Guide: Envoy Gateway Production Deployment

1. Install Envoy Gateway

The first step is to install Envoy Gateway into your Kubernetes cluster. We'll use Helm, the package manager for Kubernetes, for a streamlined installation process. Envoy Gateway will deploy the necessary control plane components and an Envoy data plane proxy.

Envoy Gateway creates its own namespace, `envoy-gateway-system`, for its control plane components. The data plane (Envoy proxy instances) is typically deployed in the same namespace by default, but can be configured to run elsewhere. It's crucial to pin the Helm chart version for production deployments to ensure reproducibility and stability. Always refer to the official Envoy Gateway documentation for the latest stable version.

# Add the Envoy Gateway Helm repository (optional, if not already added)
# helm repo add envoy-gateway https://envoyproxy.github.io/gateway-helm

# Install Envoy Gateway using OCI registry
helm install envoy-gateway \
  oci://ghcr.io/envoyproxy/gateway-helm/envoy-gateway \
  --version v1.0.0 \
  -n envoy-gateway-system --create-namespace

Verify:

You should see several pods running in the `envoy-gateway-system` namespace, including the Envoy Gateway controller and the Envoy proxy data plane.

kubectl get pods -n envoy-gateway-system

NAME                                READY   STATUS    RESTARTS   AGE
envoy-gateway-5c7d88b48c-abcde      1/1     Running   0          2m
envoy-56f78997c-fghij               1/1     Running   0          2m

2. Deploy a Sample Application

Before we can route traffic, we need a target application. We'll deploy a simple "hello-world" application that will serve as our backend service. This application consists of a Deployment and a Service.

This step demonstrates the core concept of routing traffic to a Kubernetes Service. The Gateway API, through Envoy Gateway, will manage the external exposure and routing logic, while the Service handles internal load balancing to the application Pods.

kubectl apply -f https://raw.githubusercontent.com/kubernetes/website/main/content/en/examples/service/access/hello-world-deployment.yaml
kubectl apply -f https://raw.githubusercontent.com/kubernetes/website/main/content/en/examples/service/access/hello-world-service.yaml

Verify:

Ensure the `hello-world` deployment and service are running in the `default` namespace.

kubectl get deployments,services hello-world

NAME                            READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/hello-world     1/1     1            1           1m

NAME                TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)   AGE
service/hello-world ClusterIP   10.96.123.123   <none>        80/TCP    1m

3. Create a Gateway Resource

The `Gateway` resource is the entry point for traffic into your cluster. It defines where traffic will be received (e.g., port 80, port 443) and which `GatewayClass` (in our case, `envoy-gateway`) will fulfill its capabilities. Think of the Gateway as the virtual load balancer or proxy that will be provisioned by Envoy Gateway.

The `listeners` section specifies the ports and protocols the Gateway will listen on. `allowedRoutes` is a powerful security feature that restricts which namespaces can attach `HTTPRoute` or other route resources to this Gateway. For production, you might restrict this to specific application namespaces or a dedicated ingress namespace.

apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
  name: my-gateway
  namespace: default
spec:
  gatewayClassName: envoy-gateway
  listeners:
  - name: http
    protocol: HTTP
    port: 80
    allowedRoutes:
      namespaces:
        from: All # For demonstration, allowing all namespaces

kubectl apply -f - <<EOF
apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
  name: my-gateway
  namespace: default
spec:
  gatewayClassName: envoy-gateway
  listeners:
  - name: http
    protocol: HTTP
    port: 80
    allowedRoutes:
      namespaces:
        from: All
EOF

Verify:

The Gateway resource should transition to a `Ready` status, and Envoy Gateway will provision an external load balancer (if on a cloud provider) or expose a NodePort/HostPort.

kubectl get gateway my-gateway -o yaml

You'll observe status conditions indicating readiness, and importantly, an `address` field in the status, which provides the external IP or hostname.

...
status:
  addresses:
  - type: IPAddress
    value: 34.123.45.67 # This will be your external IP
  conditions:
  - lastTransitionTime: "2023-10-27T10:00:00Z"
    message: Gateway is accepted by the controller
    observedGeneration: 1
    reason: Accepted
    status: "True"
    type: Accepted
  - lastTransitionTime: "2023-10-27T10:00:00Z"
    message: Listener Ready
    observedGeneration: 1
    reason: Ready
    status: "True"
    type: Ready
  - lastTransitionTime: "2023-10-27T10:00:00Z"
    message: All listeners are configured.
    observedGeneration: 1
    reason: ListenersReady
    status: "True"
    type: Programmed
...

4. Create an HTTPRoute Resource

The `HTTPRoute` resource defines how HTTP traffic is routed from a `Gateway` to backend services. It specifies hostnames, path matches, and the backend services to which traffic should be forwarded. This is where application teams typically define their routing rules.

The `parentRefs` field links this `HTTPRoute` to our `my-gateway`. `hostnames` defines the domain names this route applies to, and `rules` contain the matching criteria (e.g., path prefix) and backend references. This separation of concerns between Gateway and Route is a key advantage of the Gateway API.

apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: http-route-example
  namespace: default
spec:
  parentRefs:
  - name: my-gateway
  hostnames:
  - "example.com" # Replace with a domain you control or use for testing
  rules:
  - matches:
    - path:
        type: PathPrefix
        value: /
    backendRefs:
    - name: hello-world
      port: 80

kubectl apply -f - <<EOF
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: http-route-example
  namespace: default
spec:
  parentRefs:
  - name: my-gateway
  hostnames:
  - "example.com"
  rules:
  - matches:
    - path:
        type: PathPrefix
        value: /
    backendRefs:
    - name: hello-world
      port: 80
EOF

Verify:

Check the status of your `HTTPRoute`. It should show that it's successfully attached to the `Gateway`.

kubectl get httproute http-route-example -o yaml

...
status:
  parents:
  - conditions:
    - lastTransitionTime: "2023-10-27T10:05:00Z"
      message: Route is accepted by the Gateway.
      observedGeneration: 1
      reason: Accepted
      status: "True"
      type: Accepted
    - lastTransitionTime: "2023-10-27T10:05:00Z"
      message: All rules have been successfully programmed.
      observedGeneration: 1
      reason: Programmed
      status: "True"
      type: Programmed
    controllerName: gateway.envoyproxy.io/gatewayclass-controller
    parentRef:
      group: gateway.networking.k8s.io
      kind: Gateway
      name: my-gateway
      namespace: default
...

5. Test Traffic Routing

Now that the Gateway and HTTPRoute are configured, retrieve the external IP address of your Gateway and test the routing. You'll need to either configure your local `hosts` file or use a tool like `curl` with the `Host` header to simulate the `example.com` hostname.

This step confirms that Envoy Gateway is correctly receiving external traffic and routing it to your backend application based on the `HTTPRoute` rules.

# Get the external IP of the Gateway
GATEWAY_IP=$(kubectl get gateway my-gateway -o jsonpath='{.status.addresses[0].value}')
echo "Gateway IP: $GATEWAY_IP"

# Test the route using curl with the Host header
curl -H "Host: example.com" http://$GATEWAY_IP

Verify:

You should receive the "Hello, world!" response from your sample application.

Hello, world!
Hostname: hello-world-5d66c5c76-xyzab

6. Advanced Traffic Management (HTTPRoute Filters)

The Gateway API isn't just about simple routing; it supports powerful traffic management features through `filters`. Let's demonstrate adding a request header modification.

HTTPRoute filters allow modifying requests and responses, injecting headers, performing URL rewrites, and much more. Envoy Gateway translates these filters into Envoy proxy configurations, enabling advanced traffic manipulation without direct Envoy configuration. This is crucial for A/B testing, Canary deployments, or even just adding tracing headers.

apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: http-route-example-with-filter
  namespace: default
spec:
  parentRefs:
  - name: my-gateway
  hostnames:
  - "example.com"
  rules:
  - matches:
    - path:
        type: PathPrefix
        value: /
    filters:
    - type: RequestHeaderModifier
      requestHeaderModifier:
        add:
        - name: X-Kubezilla-Header
          value: "EnvoyGateway-Test"
    backendRefs:
    - name: hello-world
      port: 80

kubectl apply -f - <<EOF
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: http-route-example-with-filter
  namespace: default
spec:
  parentRefs:
  - name: my-gateway
  hostnames:
  - "example.com"
  rules:
  - matches:
    - path:
        type: PathPrefix
        value: /
    filters:
    - type: RequestHeaderModifier
      requestHeaderModifier:
        add:
        - name: X-Kubezilla-Header
          value: "EnvoyGateway-Test"
    backendRefs:
    - name: hello-world
      port: 80
EOF

Verify:

To verify this, we need to inspect the headers received by the `hello-world` application. A simple way is to exec into the `hello-world` pod and check its logs or use a more advanced debug container. For simplicity, let's assume `hello-world` echoes headers, or we can use `tcpdump` inside the pod (if installed). For a more robust test, you'd deploy a custom application that logs all incoming headers.

Alternatively, if you have access to the Envoy proxy logs, you could inspect them, though that's more involved.

# Assuming hello-world application logs headers (if it were designed to do so)
# For this example, we'll just re-run the curl and trust the configuration.
# If your hello-world app can show headers, you'd see X-Kubezilla-Header.
GATEWAY_IP=$(kubectl get gateway my-gateway -o jsonpath='{.status.addresses[0].value}')
curl -H "Host: example.com" http://$GATEWAY_IP

Hello, world!
Hostname: hello-world-5d66c5c76-xyzab
# If the app logged headers, you'd see X-Kubezilla-Header: EnvoyGateway-Test

7. HTTPS and TLS Configuration

Securing traffic with HTTPS is a critical production requirement. Envoy Gateway integrates seamlessly with Kubernetes Secrets for TLS certificates.

This involves creating a `Secret` containing your TLS certificate and private key, and then referencing it in the `Gateway`'s listener configuration. This enables end-to-end encryption from the client to the Gateway. For more advanced certificate management, consider integrating with cert-manager.

# Create a self-signed certificate for demonstration purposes
# In production, use a proper CA-signed certificate.
openssl req -x509 -newkey rsa:4096 -keyout tls.key -out tls.crt -days 365 -nodes \
  -subj "/CN=example.com" -addext "subjectAltName = DNS:example.com"

# Create a Kubernetes Secret from the certificate and key
kubectl create secret tls example-com-tls \
  --key=tls.key \
  --cert=tls.crt \
  -n default

# Update the Gateway to listen on HTTPS (port 443)
cat <

GATEWAY_IP=$(kubectl get gateway my-gateway -o jsonpath='{.status.addresses[0].value}')
curl -k -H "Host: example.com" https://$GATEWAY_IP

Hello, world!
Hostname: hello-world-5d66c5c76-xyzab

apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: multi-host-route
spec:
  parentRefs:
  - name: my-gateway
  hostnames:
  - "app1.example.com"
  - "app2.example.com"
  rules:
  - matches:
    - path:
        type: PathPrefix
        value: /
    backendRefs:
    - name: my-app-service
      port: 80
    

apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: canary-route
spec:
  parentRefs:
  - name: my-gateway
  hostnames:
  - "example.com"
  rules:
  - matches:
    - path:
        type: PathPrefix
        value: /
    backendRefs:
    - name: my-app-v1
      port: 80
      weight: 90
    - name: my-app-v2-canary
      port: 80
      weight: 10