Secure Passthrough: Gateway API TLSRoute

Introduction

In the dynamic world of Kubernetes, managing external traffic securely and efficiently is paramount. Historically, Kubernetes Ingress has served as the de facto standard for routing HTTP/S traffic into clusters. However, as applications become more complex and security demands stricter, Ingress often falls short, particularly when dealing with advanced traffic management scenarios like TLS passthrough. When you need to terminate TLS at the application pod itself, rather than at the edge proxy, Ingress’s capabilities become restrictive, forcing workarounds or complex annotations that are vendor-specific and non-portable.

Enter the Kubernetes Gateway API, a powerful, expressive, and extensible evolution of Kubernetes’ networking capabilities. Designed to overcome the limitations of Ingress, the Gateway API introduces a more granular and role-oriented approach to traffic management. Among its many features, the `TLSRoute` resource stands out as a game-changer for secure passthrough configurations. This resource allows operators to declare how TLS traffic should be routed directly to backend services, enabling end-to-end encryption without the Gateway itself decrypting the traffic, thus preserving the integrity and confidentiality of the connection all the way to the application.

This comprehensive guide will walk you through setting up `TLSRoute` for secure passthrough in your Kubernetes cluster. We’ll explore its architecture, demonstrate practical configurations, and provide you with the knowledge to leverage this powerful API for robust, secure, and flexible traffic management. Whether you’re migrating from Ingress or building a new, secure-by-design application, understanding `TLSRoute` is crucial for modern Kubernetes networking. For a broader understanding of migrating to this new API, consider our Kubernetes Gateway API vs Ingress: The Complete Migration Guide.

# Install Gateway API CRDs (if not already present with controller)
kubectl apply -f https://github.com/kubernetes-sigs/gateway-api/releases/download/v1.0.0/standard-install.yaml

# Example GatewayClass
kubectl apply -f - <<EOF
apiVersion: gateway.networking.k8s.io/v1
kind: GatewayClass
metadata:
  name: example-gatewayclass
spec:
  controllerName: example.com/gateway-controller
EOF

# Example Gateway for TLS Passthrough
kubectl apply -f - <<EOF
apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
  name: my-tls-gateway
  namespace: default
spec:
  gatewayClassName: example-gatewayclass
  listeners:
  - name: https-passthrough
    port: 443
    protocol: TLS
    tls:
      mode: Passthrough
EOF

# Example TLSRoute
kubectl apply -f - <<EOF
apiVersion: gateway.networking.k8s.io/v1
kind: TLSRoute
metadata:
  name: my-app-tls-route
  namespace: default
spec:
  parentRefs:
  - name: my-tls-gateway
  rules:
  - backendRefs:
    - name: my-backend-service
      port: 443
EOF

Prerequisites

Before diving into the configuration, ensure you have the following:

• A Kubernetes Cluster: Version 1.20+ is recommended, as Gateway API has evolved significantly. Any cloud provider (AWS EKS, GCP GKE, Azure AKS) or on-premise cluster will work.
• kubectl: The Kubernetes command-line tool, configured to connect to your cluster.
• Gateway API CRDs: The Custom Resource Definitions for Gateway API must be installed in your cluster. Many Gateway API controllers install these automatically.
• A Gateway API Controller: You need an implementation of the Gateway API. Popular choices include:
  • Istio (with Istio Ambient Mesh for sidecar-less operation, see our Istio Ambient Mesh Production Guide)
  • Nginx Gateway Fabric
  • GKE Gateway (Google Cloud)
  • AWS Load Balancer Controller (for certain use cases)

  For this tutorial, we will use a generic controller setup, but the principles apply to any compliant implementation.

• Basic Kubernetes Knowledge: Familiarity with Deployments, Services, and basic networking concepts.
• TLS Certificates: For your backend application, you’ll need a valid TLS certificate and key. We’ll use a self-signed certificate for demonstration purposes.

Step-by-Step Guide: Secure Passthrough with TLSRoute

Step 1: Install Gateway API CRDs and a Gateway Controller

First, ensure the core Gateway API Custom Resource Definitions (CRDs) are present in your cluster. Many Gateway controllers install these automatically, but it’s good practice to ensure they are up-to-date. Then, deploy your chosen Gateway controller. For this guide, we’ll demonstrate using a generic approach, but you would replace `example.com/gateway-controller` with your actual controller’s name.

The Gateway API CRDs define the `GatewayClass`, `Gateway`, `HTTPRoute`, `TLSRoute`, `TCPRoute`, `UDPRoute`, and `ReferenceGrant` resources. These are fundamental for the API’s operation. After installing the CRDs, you need an actual controller (like Istio, Nginx Gateway Fabric, etc.) that watches these resources and translates them into underlying infrastructure (e.g., Load Balancers, proxies). Without a controller, the Gateway API resources are just definitions.

# Install Gateway API CRDs (if not already present or if your controller doesn't install them)
# Using v1.0.0, adjust as needed for newer versions.
kubectl apply -f https://github.com/kubernetes-sigs/gateway-api/releases/download/v1.0.0/standard-install.yaml

# Verify CRDs are installed
kubectl get crd | grep gateway.networking.k8s.io

# Expected Output (may vary slightly based on version)
gatewayclasses.gateway.networking.k8s.io           2023-10-26T12:00:00Z
gateways.gateway.networking.k8s.io                 2023-10-26T12:00:00Z
httproutes.gateway.networking.k8s.io               2023-10-26T12:00:00Z
referencegrants.gateway.networking.k8s.io          2023-10-26T12:00:00Z
tlsroutes.gateway.networking.k8s.io                2023-10-26T12:00:00Z
tcproutes.gateway.networking.k8s.io                2023-10-26T12:00:00Z
udproutes.gateway.networking.k8s.io                2023-10-26T12:00:00Z

Step 2: Create a GatewayClass and a Gateway

A `GatewayClass` defines a template for Gateways, specifying which controller manages them. A `Gateway` then represents the actual entry point for traffic, such as a load balancer or a proxy, configured with listeners for specific protocols and ports. For TLS passthrough, we’ll define a listener with `protocol: TLS` and `tls.mode: Passthrough`.

The `GatewayClass` acts as a cluster-scoped resource that allows different implementations of the Gateway API to be registered. Think of it as defining the “type” of Gateway you want to provision (e.g., “NginxGateway”, “IstioGateway”, “GKEGateway”). The `controllerName` field links to the specific controller responsible for fulfilling Gateways of this class. The `Gateway` resource itself is a namespace-scoped resource that provisions a specific instance of an ingress point. Its `gatewayClassName` field refers to the `GatewayClass` that dictates its behavior. For TLS passthrough, the critical configuration is the `listeners` section, where we explicitly set `tls.mode: Passthrough`. This instructs the Gateway controller *not* to terminate TLS at the Gateway level, but instead to forward the raw TLS stream to the backend.

# Create a GatewayClass (replace controllerName with your actual controller)
kubectl apply -f - <<EOF
apiVersion: gateway.networking.k8s.io/v1
kind: GatewayClass
metadata:
  name: my-passthrough-gatewayclass
spec:
  controllerName: example.com/gateway-controller # Replace with your controller, e.g., gateway.nginx.org/controller
EOF

# Create a Gateway for TLS Passthrough
kubectl apply -f - <<EOF
apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
  name: my-tls-gateway
  namespace: default
spec:
  gatewayClassName: my-passthrough-gatewayclass
  listeners:
  - name: https-passthrough
    port: 443
    protocol: TLS
    tls:
      mode: Passthrough # This is the key for TLS passthrough
EOF

# Verify GatewayClass and Gateway status
kubectl get gatewayclass my-passthrough-gatewayclass
kubectl get gateway my-tls-gateway -o wide

# Expected Output for GatewayClass
NAME                        CONTROLLER                          ACCEPTED   AGE
my-passthrough-gatewayclass   example.com/gateway-controller      True       Xs

# Expected Output for Gateway (Address might take some time to provision)
NAME             CLASS                         ADDRESS         PROGRAMMED   AGE
my-tls-gateway   my-passthrough-gatewayclass   <EXTERNAL_IP>   True         Xs

*Note: The `ADDRESS` field for the Gateway will populate with an external IP or hostname once your Gateway controller provisions the underlying load balancer or proxy.*

Step 3: Prepare Your Backend Application with TLS

Since `TLSRoute` performs passthrough, your backend application must be capable of handling TLS termination itself. We’ll deploy a simple Nginx server configured to serve HTTPS. We need to generate a self-signed certificate for this demonstration. In a production environment, you would use certificates issued by a trusted Certificate Authority (CA).

For production environments, consider using `cert-manager` to automate certificate issuance and renewal. This tool can integrate with various CAs, including Let’s Encrypt, and store certificates as Kubernetes Secrets. For securing container supply chains and ensuring trust, you might also look into solutions like Sigstore and Kyverno.

# 1. Generate a self-signed certificate and key
# Common Name (CN) should match the hostname you expect clients to use.
# For simplicity, we'll use a generic name, but in reality, it would be your domain.
openssl req -x509 -nodes -days 365 -newkey rsa:2048 \
    -keyout tls.key -out tls.crt \
    -subj "/CN=my-app.example.com/O=MyOrg"

# 2. Create a Kubernetes Secret from the certificate and key
kubectl create secret tls my-app-tls-secret --key tls.key --cert tls.crt

# 3. Deploy a simple Nginx application configured for HTTPS
kubectl apply -f - <<EOF
apiVersion: apps/v1
kind: Deployment
metadata:
  name: my-tls-app
  namespace: default
spec:
  selector:
    matchLabels:
      app: my-tls-app
  replicas: 1
  template:
    metadata:
      labels:
        app: my-tls-app
    spec:
      containers:
      - name: nginx
        image: nginx:latest
        ports:
        - containerPort: 443
        volumeMounts:
        - name: tls-cert
          mountPath: /etc/nginx/certs
          readOnly: true
        - name: nginx-conf
          mountPath: /etc/nginx/conf.d
          readOnly: true
      volumes:
      - name: tls-cert
        secret:
          secretName: my-app-tls-secret
      - name: nginx-conf
        configMap:
          name: nginx-tls-config
---
apiVersion: v1
kind: ConfigMap
metadata:
  name: nginx-tls-config
  namespace: default
data:
  default.conf: |
    server {
        listen 443 ssl;
        server_name my-app.example.com; # This should match your certificate's CN

        ssl_certificate /etc/nginx/certs/tls.crt;
        ssl_certificate_key /etc/nginx/certs/tls.key;

        location / {
            return 200 "Hello from TLS Passthrough Nginx!\n";
            add_header Content-Type text/plain;
        }
    }
---
apiVersion: v1
kind: Service
metadata:
  name: my-backend-service
  namespace: default
spec:
  selector:
    app: my-tls-app
  ports:
  - name: https
    protocol: TCP
    port: 443
    targetPort: 443
EOF

# Verify deployment and service
kubectl get deployment my-tls-app
kubectl get service my-backend-service
kubectl get secret my-app-tls-secret

# Expected Output
NAME           READY   UP-TO-DATE   AVAILABLE   AGE
my-tls-app     1/1     1            1           Xs

NAME                 TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)    AGE
my-backend-service   ClusterIP   10.X.X.X         <none>        443/TCP    Xs

NAME                  TYPE     DATA   AGE
my-app-tls-secret     kubernetes.io/tls   2      Xs

Step 4: Create a TLSRoute for Passthrough

Finally, we define the `TLSRoute` resource. This resource binds to our `Gateway` and specifies how TLS traffic received on the `https-passthrough` listener should be routed to our `my-backend-service`. The key here is that the Gateway will not inspect the TLS handshake; it simply forwards the encrypted TCP stream.

The `TLSRoute` resource is designed specifically for routing TLS traffic. Unlike `HTTPRoute`, which operates at Layer 7 and inspects HTTP headers, `TLSRoute` can operate at Layer 4 (TCP with TLS SNI) or Layer 5 (TLS handshake). When `tls.mode: Passthrough` is set on the Gateway listener, the Gateway simply forwards the raw TCP stream (which contains the TLS handshake) to the backend. The `TLSRoute` then acts as a selector, allowing you to specify `hostnames` to match against the Server Name Indication (SNI) presented by the client during the TLS handshake. This enables multiple `TLSRoute` resources to share a single passthrough listener on the Gateway, routing traffic based on the requested hostname.

kubectl apply -f - <<EOF
apiVersion: gateway.networking.k8s.io/v1
kind: TLSRoute
metadata:
  name: my-app-tls-route
  namespace: default
spec:
  parentRefs:
  - name: my-tls-gateway # Reference the Gateway created in Step 2
  hostnames:
  - "my-app.example.com" # This should match the certificate's CN and what clients use
  rules:
  - backendRefs:
    - name: my-backend-service # Reference the Service created in Step 3
      port: 443
EOF

# Verify TLSRoute status
kubectl get tlsroute my-app-tls-route -o yaml

# Expected Output (truncated for brevity)
apiVersion: gateway.networking.k8s.io/v1
kind: TLSRoute
metadata:
  name: my-app-tls-route
  namespace: default
spec:
  hostnames:
  - my-app.example.com
  parentRefs:
  - name: my-tls-gateway
  rules:
  - backendRefs:
    - name: my-backend-service
      port: 443
status:
  parents:
  - accepted: true
    conditions:
    - lastTransitionTime: "2023-10-26T12:30:00Z"
      message: Route is successfully attached to the Gateway
      reason: Accepted
      status: "True"
      type: Accepted
    controllerName: example.com/gateway-controller
    parentRef:
      name: my-tls-gateway

Step 5: Test the TLS Passthrough Configuration

Now that everything is set up, you can test the end-to-end TLS passthrough. You’ll need the external IP address of your `my-tls-gateway`.

# Get the external IP of your Gateway
GATEWAY_IP=$(kubectl get gateway my-tls-gateway -o jsonpath='{.status.addresses[0].value}')
echo "Gateway IP: $GATEWAY_IP"

# Test with curl, specifying the hostname and allowing insecure SSL for self-signed cert
curl -v --resolve my-app.example.com:443:$GATEWAY_IP https://my-app.example.com --insecure

# Expected Output (look for TLS handshake details and the backend response)
* Added my-app.example.com:443:GATEWAY_IP to DNS cache
* Hostname my-app.example.com was found in DNS cache
*   Trying GATEWAY_IP:443...
* Connected to my-app.example.com (GATEWAY_IP) port 443 (#0)
* ALPN: offers h2
* ALPN: offers http/1.1
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
* TLSv1.3 (IN), TLS handshake, Finished (20):
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.3 (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / AEAD-CHACHA20-POLY1305-SHA256
* ALPN: server accepted http/1.1
* Server certificate:
*  subject: CN=my-app.example.com; O=MyOrg
*  start date: Oct 26 12:30:00 2023 GMT
*  expire date: Oct 26 12:30:00 2024 GMT
*  issuer: CN=my-app.example.com; O=MyOrg
*  SSL certificate verify result: self signed certificate (18), continuing anyway.
> GET / HTTP/1.1
> Host: my-app.example.com
> User-Agent: curl/7.81.0
> Accept: */*
>
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* old SSL session ID is stale, removing
* Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK
< Server: nginx/1.25.3
< Date: Thu, 26 Oct 2023 12:35:00 GMT
< Content-Type: text/plain
< Content-Length: 35
< Connection: keep-alive
<
Hello from TLS Passthrough Nginx!
* Connection #0 to host my-app.example.com left intact

The `SSL certificate verify result: self signed certificate (18), continuing anyway.` and `subject: CN=my-app.example.com; O=MyOrg` lines confirm that the TLS termination happened at our Nginx backend, not at the Gateway, and that it used our self-signed certificate. This verifies successful TLS passthrough!

Production Considerations

While `TLSRoute` offers powerful capabilities for secure passthrough, deploying it in production requires careful consideration of several factors:

• Certificate Management: Manually managing certificates for backend applications is cumbersome and error-prone. Integrate `cert-manager` for automated certificate issuance (e.g., from Let's Encrypt or your internal CA) and renewal. This ensures your applications always have valid, trusted certificates.
• Gateway Controller Choice: The performance, scalability, and features of your chosen Gateway API controller are critical. Evaluate options like Istio, Nginx Gateway Fabric, or cloud-provider-specific Gateways based on your needs. For advanced service mesh capabilities, Istio is a strong contender, and its Ambient Mesh offers a sidecar-less approach.
• Observability: Monitoring the health and performance of your Gateway and backend services is essential. Implement robust logging, metrics, and tracing. For eBPF-based controllers like Cilium, tools like Hubble can provide deep insights into network traffic, as discussed in our eBPF Observability: Building Custom Metrics with Hubble guide.
• Security Best Practices:
  • TLS Versions and Ciphers: Configure your backend applications to use strong TLS versions (e.g., TLS 1.2, TLS 1.3) and secure cipher suites.
  • Network Policies: Even with TLS passthrough, implement Kubernetes Network Policies to restrict traffic between namespaces and to backend services, providing a layered defense.
  • Authentication/Authorization: TLS only encrypts communication. Implement proper authentication and authorization at the application layer.
  • Vulnerability Scanning: Regularly scan your application images and dependencies for vulnerabilities.
• High Availability and Scalability:
  • Ensure your Gateway controller and backend applications are deployed with sufficient replicas and anti-affinity rules.
  • Configure horizontal pod autoscaling (HPA) for your backend applications based on CPU, memory, or custom metrics.
  • Leverage node autoscaling solutions like Karpenter for efficient resource management and cost optimization.
• DNS Management: Ensure your domain's DNS records (A/AAAA records) point to the external IP/hostname of your Gateway.
• Egress Traffic: While this guide focuses on ingress, remember to secure your egress traffic as well, potentially using solutions like Cilium WireGuard Encryption for Pod-to-Pod Traffic if your CNI supports it.

Troubleshooting

Here are some common issues you might encounter with `TLSRoute` passthrough and their solutions:

1. Issue: Gateway `ADDRESS` field is Pending or Empty.

   Explanation: The Gateway controller hasn't yet provisioned an external IP or hostname for your Gateway. This is common if the controller is not running, not configured correctly, or if there's a delay in cloud load balancer provisioning.

   Solution:

   • Check the logs of your Gateway controller deployment (e.g., `kubectl logs -n `).
   • Ensure the `GatewayClass` `controllerName` matches the actual controller deployed.
   • Verify your cluster has sufficient permissions/resources to provision external load balancers (if applicable, e.g., in cloud environments).
   • Wait a few minutes and recheck `kubectl get gateway my-tls-gateway -o wide`.
2. Issue: `TLSRoute` `status.parents` shows `Accepted: false` or `Reason: NoMatchingParent`.

   Explanation: The `TLSRoute` is not successfully binding to the `Gateway`. This usually means the `parentRefs` in the `TLSRoute` do not correctly reference an existing `Gateway` or the `Gateway`'s listener configuration doesn't match.

   Solution:

   • Double-check the `name` and `namespace` in `TLSRoute.spec.parentRefs` against your `Gateway` resource.
   • Ensure the `Gateway` listener's `protocol` is `TLS` and `tls.mode` is `Passthrough`.
   • Check the `hostnames` in your `TLSRoute`. If your Gateway listener has `hostname` defined, the `TLSRoute` `hostnames` must be a subset of the Gateway listener's hostnames.
   • Inspect `kubectl describe tlsroute my-app-tls-route` for detailed status messages.
3. Issue: `curl` command fails with "SSL_connect: SSL_ERROR_SYSCALL" or "Connection reset by peer".

   Explanation: The TLS handshake is failing or being interrupted. This often indicates an issue with the backend application's TLS configuration or the `TLSRoute` not directing traffic correctly.

   Solution:

   • Verify your backend application (Nginx in our example) is listening on port 443 and is correctly configured for TLS (correct `tls.crt` and `tls.key` paths, `ssl_certificate`, `ssl_certificate_key`).
   • Check the logs of your backend application pods for TLS-related errors.
   • Ensure the `Service` `targetPort` for your backend matches the `containerPort` your application is listening on (both should be 443 for HTTPS).
   • Confirm the `TLSRoute` `backendRefs` correctly points to the `Service` and `port`.
   • Use `openssl s_client -connect :443 -servername my-app.example.com` to debug the TLS handshake directly and see the certificate presented.
4. Issue: `curl` works, but the certificate is not the one from my backend application (e.g., a default certificate from the Gateway).

   Explanation: This means TLS termination is happening at the Gateway, not at your backend. The `tls.mode: Passthrough` might not be correctly applied or respected by your controller.

   Solution:

   • Re-verify `Gateway.spec.listeners[*].tls.mode` is explicitly set to `Passthrough`.
   • Ensure your Gateway controller fully supports `TLSRoute` with passthrough mode. Some controllers might have specific versions or configurations required. Consult your controller's official documentation (e.g., Istio Gateway API docs).
   • Check `kubectl describe gateway my-tls-gateway` for any warnings or conditions related to the listener configuration.
5. Issue: `curl` command hangs or times out.

   Explanation: The connection is not reaching the Gateway or the backend, or there's a firewall/network issue blocking traffic.

   Solution:

   • Verify the `GATEWAY_IP` is correct and reachable from your client machine (e.g., `ping $GATEWAY_IP`).
   • Check firewall rules on your cloud provider or cluster that might block ingress traffic to the Gateway's external IP on port 443.
   • Ensure your backend `Service` is healthy and its pods are running (e.g., `kubectl get pods -l app=my-tls-app`).
   • If using a CNI like Cilium, ensure there are no Network Policies blocking traffic between the Gateway proxy and your backend pods.
6. Issue: `TLSRoute` `hostnames` not working as expected.

   Explanation: The `hostnames` field in `TLSRoute` is used for Server Name Indication (SNI) matching. If the client doesn't send the correct SNI, or if there's a mismatch, the route won't be selected.

   Solution:

   • Ensure the client (e.g., `curl`) is sending the correct `Host` header and SNI. For `curl`, use `--resolve my-app.example.com:443:$GATEWAY_IP` and `https://my-app.example.com`.
   • Verify the `hostnames` in your `TLSRoute` exactly match the SNI you expect from clients.
   • If you have multiple `TLSRoute` resources, ensure their `hostnames` are distinct to avoid conflicts.

FAQ Section

1. What is the difference between `TLSRoute` and `HTTPRoute`?

   HTTPRoute is designed for Layer 7 (HTTP/HTTPS) traffic management, allowing for routing based on HTTP headers, paths, methods, and hostnames, often with TLS termination at the Gateway. TLSRoute is for Layer 4/5 TLS traffic. In passthrough mode, it simply forwards the encrypted TLS stream to the backend, with routing based on SNI. In terminate mode (not covered here), it terminates TLS at the Gateway but then routes based on SNI.

2. Why would I use TLS Passthrough instead of terminating TLS at the Gateway?

   TLS passthrough offers several benefits:

   • End-to-End Encryption: TLS is terminated at the application, ensuring encryption from the client all the way to the application pod, which can be critical for highly sensitive data or compliance requirements.
   • Application-Specific TLS Features: Allows applications to use specific TLS features, client certificate authentication, or custom cipher suites that the Gateway might not support or expose.
   • Simplicity for Specific Use Cases: For services like databases (e.g., PostgreSQL with SSL), MQTT brokers, or gRPC services that inherently use TLS, passthrough simplifies configuration by letting the application handle its own TLS.
   • Reduced Gateway Overhead: The Gateway doesn't need to perform computationally intensive TLS decryption/encryption.
3. Can `TLSRoute` handle multiple hostnames?

   Yes, `TLSRoute` can specify multiple hostnames in its `spec.hostnames` array. The Gateway controller will use the Server Name Indication (SNI) presented by the client during the TLS handshake to match the incoming connection to the appropriate `TLSRoute` and thus to the correct backend service.

4. Is `TLSRoute` compatible