Orchestration

Scale Apps with KEDA: Event-Driven Autoscaling

August 3, 2026 Kubezilla Team 17 min read

Introduction

In the dynamic world of cloud-native applications, efficient resource utilization is paramount. Traditional Kubernetes autoscaling, primarily driven by CPU and memory metrics via the Horizontal Pod Autoscaler (HPA), often falls short for event-driven architectures. Imagine a scenario where your application processes messages from a Kafka queue or handles a sudden surge in HTTP requests. Relying solely on CPU utilization to scale up might introduce significant latency as pods only scale out after the CPU load increases, rather than proactively responding to the incoming event stream.

This is where KEDA (Kubernetes Event-driven Autoscaling) steps in. KEDA extends the capabilities of the HPA, allowing your Kubernetes workloads to scale based on the number of events needing to be processed. Instead of reacting to resource consumption, KEDA enables your applications to scale reactively to the demand, whether that demand comes from message queues, streaming platforms, databases, or even external HTTP services. By integrating with a vast array of external event sources, KEDA ensures your applications are always right-sized, leading to optimal performance, reduced operational costs, and improved responsiveness to user demand.

TL;DR: Event-Driven Autoscaling with KEDA

KEDA extends Kubernetes HPA to scale workloads based on external event sources like Kafka, RabbitMQ, SQS, Prometheus, etc. It allows for more efficient resource utilization by scaling pods proactively based on actual demand rather than reactive CPU/memory metrics.

Key Steps:

  1. Install KEDA: Use Helm to deploy KEDA to your cluster.
  2. Deploy a Sample Application: Create a basic Kubernetes Deployment.
  3. Define a ScaledObject: Configure KEDA to monitor an event source (e.g., a message queue) and scale your Deployment.
  4. Generate Load: Simulate events to trigger autoscaling.
  5. Observe Scaling: Monitor HPA and pod counts.

Key Commands:


# Install KEDA via Helm
helm repo add kedacore https://kedacore.github.io/charts
helm repo update
helm install keda kedacore/keda --namespace keda --create-namespace

# Apply a sample deployment and ScaledObject
kubectl apply -f my-app-deployment.yaml
kubectl apply -f my-scaledobject.yaml

# Check KEDA status
kubectl get scaledobject
kubectl get hpa
kubectl get pods

Prerequisites

Before we dive into the practical aspects of KEDA, ensure you have the following:

  • Kubernetes Cluster: A running Kubernetes cluster (version 1.16+). This could be a local cluster like Minikube or Kind, or a managed service like EKS, GKE, or AKS. For cloud-specific considerations, refer to their respective documentation (e.g., AWS EKS, Google GKE).
  • kubectl: The Kubernetes command-line tool, configured to connect to your cluster. You can find installation instructions in the official Kubernetes documentation.
  • Helm: The Kubernetes package manager, version 3+. If you don’t have it, follow the Helm installation guide.
  • Basic Kubernetes Knowledge: Familiarity with Deployments, Services, and Horizontal Pod Autoscalers (HPAs) is assumed.
  • (Optional) Message Queue: For demonstration purposes, we’ll use a simple in-cluster queue. For real-world scenarios, you might need access to an external message queue like RabbitMQ, Kafka, or AWS SQS.

Step-by-Step Guide: Event-Driven Autoscaling with KEDA

Step 1: Install KEDA

The easiest way to install KEDA into your Kubernetes cluster is by using Helm. KEDA will deploy its controller and associated Custom Resource Definitions (CRDs) into a dedicated namespace. This ensures a clean separation of KEDA’s components from your application workloads.

First, add the KEDA Helm repository and update your local Helm charts. Then, install KEDA into a new namespace called keda. This process typically takes a few moments as Helm fetches the chart and deploys the necessary Kubernetes resources.


# Add the KEDA Helm repository
helm repo add kedacore https://kedacore.github.io/charts

# Update your Helm repositories
helm repo update

# Install KEDA into the 'keda' namespace
helm install keda kedacore/keda --namespace keda --create-namespace

Verify that KEDA is installed and running correctly by checking the pods in the keda namespace. You should see at least one keda-operator pod and one keda-admission-webhooks pod in a Running state. These pods are responsible for managing the autoscaling logic and handling admission control for KEDA resources, respectively.


# Verify KEDA installation
kubectl get pods -n keda

Expected Output:


NAME                                       READY   STATUS    RESTARTS   AGE
keda-operator-868695886d-abcde             1/1     Running   0          2m
keda-admission-webhooks-6789f4b6d-vwxyz    1/1     Running   0          2m

Step 2: Deploy a Sample Application

For this demonstration, we’ll use a simple Nginx deployment that we want to scale based on an external trigger. This application doesn’t inherently process events, but it serves as a placeholder for any application you might want to scale. In a real-world scenario, this would be your worker application that processes messages from a queue.

Create a file named nginx-deployment.yaml with the following content. This defines a basic Nginx deployment and a corresponding service to expose it.


apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-app
  labels:
    app: nginx-app
spec:
  replicas: 1 # Start with 1 replica
  selector:
    matchLabels:
      app: nginx-app
  template:
    metadata:
      labels:
        app: nginx-app
    spec:
      containers:
      - name: nginx
        image: nginx:latest
        ports:
        - containerPort: 80
---
apiVersion: v1
kind: Service
metadata:
  name: nginx-app-service
spec:
  selector:
    app: nginx-app
  ports:
    - protocol: TCP
      port: 80
      targetPort: 80

Apply this deployment to your cluster. We start with a single replica, which KEDA will then manage and scale up or down based on the defined triggers. This deployment acts as the target for KEDA’s autoscaling actions.


# Apply the sample Nginx deployment
kubectl apply -f nginx-deployment.yaml

Verify that the deployment is created and the pod is running. You should see one pod for nginx-app in a Running state.


# Verify the deployment
kubectl get deployment nginx-app
kubectl get pods -l app=nginx-app

Expected Output:


NAME          READY   UP-TO-DATE   AVAILABLE   AGE
nginx-app     1/1     1            1           1m

NAME                          READY   STATUS    RESTARTS   AGE
nginx-app-7b8c7b8c7b-xxxxx    1/1     Running   0          1m

Step 3: Define a ScaledObject for HTTP Trigger

Now, let’s configure KEDA to scale our Nginx application. KEDA introduces a custom resource called ScaledObject. This resource links a Kubernetes Deployment (or StatefulSet, Job) to one or more external event sources. KEDA then creates an HPA resource under the hood, translating the event source metrics into scaling actions.

For this example, we’ll use the HTTP scaler, which allows KEDA to scale based on pending requests to an HTTP endpoint. This is particularly useful for webhooks or APIs that receive varying loads. We’ll set up an HTTP endpoint that KEDA will monitor. When requests are sent to this endpoint, KEDA will scale the nginx-app deployment.

Create a file named http-scaledobject.yaml. This ScaledObject will instruct KEDA to scale the nginx-app based on the number of pending HTTP requests. It specifies a minimum of 0 pods (allowing complete scale-down to save costs, similar to serverless functions) and a maximum of 5 pods. The targetPendingRequests value defines the threshold for scaling.


apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata:
  name: nginx-http-scaledobject
  namespace: default
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: nginx-app
  minReplicaCount: 0 # Allow scaling down to zero
  maxReplicaCount: 5
  pollingInterval: 30 # Check for new events every 30 seconds
  cooldownPeriod: 300 # Wait 5 minutes before scaling down after last activity
  triggers:
  - type: http
    metadata:
      # This is the KEDA HTTP scaler service that will expose an endpoint
      # You can find more details here: https://keda.sh/docs/latest/scalers/http/
      targetPendingRequests: "10" # Scale out if more than 10 pending requests
      scalerAddress: "keda-http-add-on.keda:8080" # Default KEDA HTTP add-on address
      # You need to deploy the KEDA HTTP add-on for this to work

Important Note: The HTTP scaler requires the KEDA HTTP Add-on to be installed. This add-on provides the proxy service that KEDA monitors. If you haven’t installed it, KEDA will report errors. For simplicity in this tutorial, we will demonstrate basic scaling that doesn’t strictly require the HTTP add-on to function (KEDA will still create the HPA), but to fully utilize the HTTP scaler, you’d deploy the add-on:


# Install KEDA HTTP Add-on (if you want to use the HTTP scaler fully)
helm install keda-http-add-on kedacore/keda-http-add-on --namespace keda

Apply the ScaledObject. KEDA will detect this resource and automatically create a Horizontal Pod Autoscaler (HPA) for your nginx-app deployment. This HPA will be configured to use custom metrics provided by KEDA, reflecting the state of your event source.


# Apply the ScaledObject
kubectl apply -f http-scaledobject.yaml

Verify that the ScaledObject and the underlying HPA have been created. KEDA creates an HPA with a special metric type, usually external, which it manages.


# Check the ScaledObject
kubectl get scaledobject nginx-http-scaledobject

# Check the HPA created by KEDA
kubectl get hpa

Expected Output:


NAME                      SCALETARGETREF     METRICS   TRIGGERS   AUTHENTICATION   READY   ACTIVE   MIN   MAX   TARGET   ACTUAL   AGE
nginx-http-scaledobject   Deployment/nginx-app             http                        True    True     0     5     10       0        1m

NAME                      REFERENCE              TARGETS    MINPODS   MAXPODS   REPLICAS   AGE
keda-hpa-nginx-http-scaledobject   Deployment/nginx-app   0/10 (avg)   0         5         0          1m

Notice the REPLICAS count for the HPA is 0, as our minReplicaCount is 0 and there are no pending requests yet. The TARGETS shows 0/10 (avg), indicating 0 pending requests against a target of 10.

Step 4: Simulate Load and Observe Scaling

Now that KEDA is configured, let’s simulate some load to see the autoscaling in action. To trigger the HTTP scaler, we need to send requests to the KEDA HTTP add-on service. This add-on acts as a proxy, queuing requests and exposing metrics that KEDA monitors.

First, let’s expose the KEDA HTTP add-on service. If you installed it, you can port-forward to its service:


# Port-forward to the KEDA HTTP Add-on service (run in a separate terminal)
kubectl port-forward service/keda-http-add-on -n keda 8080:8080

Now, let’s send some requests. We’ll use curl to simulate incoming HTTP requests to the KEDA HTTP add-on. Each request will contribute to the pending request count that KEDA monitors. We need to specify the Host header to route the request to our nginx-app target. The Host header should match the name of your ScaledObject.


# Simulate load by sending multiple HTTP requests
# Replace  with 'nginx-http-scaledobject'
# Replace  with 'default'
for i in $(seq 1 50); do \
  curl -v -H "Host: nginx-http-scaledobject.default" http://localhost:8080/ & \
done

After sending the requests, observe the HPA and the pods. KEDA’s polling interval (set to 30 seconds in our ScaledObject) means it will take a short while for it to detect the increased pending requests and trigger scaling. You should see the HPA’s REPLICAS count increase, and new nginx-app pods being created.


# Monitor the HPA
kubectl get hpa -w

# Monitor the pods
kubectl get pods -l app=nginx-app -w

Expected Output (after some time):


NAME                      REFERENCE              TARGETS        MINPODS   MAXPODS   REPLICAS   AGE
keda-hpa-nginx-http-scaledobject   Deployment/nginx-app   40/10 (avg)    0         5         4          3m

NAME                          READY   STATUS    RESTARTS   AGE
nginx-app-7b8c7b8c7b-xxxxx    1/1     Running   0          5m
nginx-app-7b8c7b8c7b-yyyyy    1/1     Running   0          30s
nginx-app-7b8c7b8c7b-zzzzz    1/1     Running   0          25s
nginx-app-7b8c7b8c7b-aaaaa    1/1     Running   0          20s

You can see the TARGETS for the HPA has increased significantly, and KEDA has scaled up the REPLICAS to handle the load. The number of pods should also reflect this increase, up to the maxReplicaCount defined in the ScaledObject. This demonstrates KEDA’s ability to react to external event-driven metrics.

Step 5: Observe Scale Down

Once the load subsides, KEDA will eventually scale down your application. The cooldownPeriod (set to 300 seconds or 5 minutes in our ScaledObject) dictates how long KEDA waits after the last activity before it begins to scale down. This prevents thrashing and ensures that temporary dips in load don’t immediately trigger scale-down actions.

Stop the curl commands you initiated in the previous step. Wait for the cooldownPeriod to elapse. During this time, KEDA will continuously monitor the event source. Once the pending requests fall below the target and the cooldown period is over, KEDA will instruct the HPA to reduce the replica count.


# Continue monitoring the HPA and pods
kubectl get hpa -w
kubectl get pods -l app=nginx-app -w

Expected Output (after cooldown period):


NAME                      REFERENCE              TARGETS       MINPODS   MAXPODS   REPLICAS   AGE
keda-hpa-nginx-http-scaledobject   Deployment/nginx-app   0/10 (avg)    0         5         0          10m


# After scale down, no nginx-app pods should be running if minReplicaCount is 0
kubectl get pods -l app=nginx-app

Expected Output:


No resources found in default namespace.

This demonstrates KEDA’s full lifecycle management, from scaling up to handle demand to scaling down to zero when there’s no activity, effectively saving resources and costs. For further cost optimization in Kubernetes, consider exploring tools like Karpenter Cost Optimization.

Production Considerations

Deploying KEDA in a production environment requires careful planning beyond a simple demonstration. Here are key considerations:

  1. Security:
    • RBAC: Ensure KEDA’s Service Accounts have only the necessary permissions. KEDA creates ClusterRoles and ClusterRoleBindings. Review these to align with your organization’s security policies.
    • Secrets Management: When connecting to external event sources (e.g., Kafka, Azure Service Bus), KEDA often requires credentials. Use Kubernetes Secrets to store these securely and reference them in your ScaledObject definitions. Avoid hardcoding credentials.
    • Network Policies: Implement Kubernetes Network Policies to restrict KEDA’s access to only the necessary services and endpoints, both within the cluster and to external event sources. This is crucial for isolating components and reducing the attack surface.
  2. High Availability:
    • KEDA Operator Replicas: Deploy KEDA with multiple replicas (e.g., 3) for its controller and admission webhooks. This ensures that KEDA remains operational even if a node or pod fails.
    • Resource Limits: Set appropriate CPU and memory requests/limits for KEDA’s pods to prevent them from consuming excessive resources or being evicted during high load.
  3. Monitoring and Alerting:
    • KEDA Metrics: KEDA exposes Prometheus metrics. Integrate these into your monitoring stack (e.g., Prometheus, Grafana) to observe the state of your ScaledObjects, trigger metrics, and KEDA’s internal operations.
    • HPA Metrics: Monitor the HPAs created by KEDA. Pay attention to the target and actual metrics to understand scaling behavior.
    • Event Source Metrics: Crucially, monitor the event sources themselves (e.g., queue depth, message lag). KEDA reacts to these, so understanding their state is vital for debugging and performance tuning.
    • Alerting: Set up alerts for failed scaling operations, KEDA pod failures, or when target metrics consistently exceed desired thresholds. Consider using eBPF Observability with Hubble for deep network insights if your scaling depends on network-level events.
  4. Scaler Choice and Configuration:
    • Choose Wisely: Select the most appropriate scaler for your event source. KEDA supports a wide range of scalers.
    • Fine-Tuning: Adjust pollingInterval, cooldownPeriod, minReplicaCount, maxReplicaCount, and scaler-specific metadata parameters carefully. These settings directly impact responsiveness, cost, and stability.
    • Authentication: Configure authentication for external event sources using TriggerAuthentication and ClusterTriggerAuthentication resources.
  5. Zero-Scale Considerations:
    • Cold Start: Scaling down to 0 can introduce “cold start” latency when scaling back up. For latency-sensitive applications, consider setting minReplicaCount to 1.
    • Stateful Applications: KEDA can scale StatefulSets, but scaling stateful applications down to 0 requires careful consideration of data persistence and rehydration.
  6. Integration with Service Mesh:
    • If you’re using a service mesh like Istio Ambient Mesh, ensure KEDA’s traffic to external event sources or internal metrics endpoints is correctly configured and not blocked by the mesh’s policies.

Troubleshooting

Even with careful configuration, issues can arise. Here are common troubleshooting scenarios and their solutions for KEDA:

  1. KEDA Pods Not Running:

    Issue: KEDA operator or webhook pods are stuck in Pending, CrashLoopBackOff, or Error states.

    Solution:

    • Check pod logs: kubectl logs -n keda <keda-pod-name>
    • Check pod descriptions for events: kubectl describe pod -n keda <keda-pod-name>. Look for issues related to image pull failures, insufficient resources, or volume mounting problems.
    • Ensure your cluster has enough resources (CPU, Memory) for KEDA pods.
  2. ScaledObject Not Ready / HPA Not Created:

    Issue: Your ScaledObject shows READY: False or no HPA is created after applying the ScaledObject.

    Solution:

    • Inspect the ScaledObject status: kubectl get scaledobject <scaledobject-name> -o yaml. Look for error messages in the status field.
    • Check KEDA operator logs: kubectl logs -n keda -l app=keda-operator. This is the primary source for KEDA-specific errors during ScaledObject processing.
    • Verify the scaleTargetRef points to an existing and correct Kubernetes resource (Deployment, StatefulSet, Job).
    • Ensure the scaler configuration in triggers.metadata is correct and matches the KEDA scaler documentation.
  3. Application Not Scaling Up or Down:

    Issue: The HPA is created, but the application pods are not scaling as expected, even with load.

    Solution:

    • Check the HPA status: kubectl get hpa <hpa-name-created-by-keda> -o yaml. Look at the currentReplicas, desiredReplicas, and especially the currentMetricValue and targetMetricValue.
    • Verify the metric source. Is KEDA correctly reading metrics from your event source?
      • For Prometheus scaler: Check if Prometheus is scraping KEDA’s metrics endpoint and if the query returns expected values.
      • For queue scalers (Kafka, RabbitMQ, SQS): Verify connectivity and credentials. Check if the queue depth or message lag is actually increasing.
    • Check KEDA operator logs for any errors related to metric gathering.
    • Review pollingInterval and cooldownPeriod in your ScaledObject. A long polling interval will delay scaling actions.
    • Ensure there are no resource quotas or limits preventing scaling.
  4. minReplicaCount: 0 Not Working (Pods remain at 1):

    Issue: You’ve set minReplicaCount: 0, but KEDA keeps at least one replica running.

    Solution:

    • This often happens if the target metric value is never truly zero or below the scaling threshold. Ensure your event source is completely empty and no events are being processed.
    • Check for subtle background processes in your application that might be keeping the metric non-zero.
    • The cooldownPeriod must elapse entirely after the last event before scaling down to zero.
    • Some scalers (e.g., CPU, Memory if combined) might have their own minimum replica counts or default behaviors that override KEDA’s. Ensure only KEDA-managed metrics are dictating the scale-down.
  5. Authentication Issues with External Event Sources:

    Issue: KEDA fails to connect to an external event source due to authentication errors.

    Solution:

    • Verify your Kubernetes Secret containing credentials. Ensure the keys match what the TriggerAuthentication expects.
    • Check the TriggerAuthentication or ClusterTriggerAuthentication resource. Ensure it correctly references the secret and specifies the correct authentication method (e.g., secretTargetRef, podIdentity).
    • Ensure the KEDA operator has the necessary RBAC permissions to read the specified Secret.
    • Check network connectivity from KEDA pods to the external event source (firewall rules, security groups, VPNs, etc.). For advanced network debugging, tools leveraging eBPF, like Cilium WireGuard Encryption, can be helpful for understanding traffic flows.
  6. Memory/CPU Usage Spikes in KEDA Operator:

    Issue: The KEDA operator pod consumes excessive CPU or memory.

    Solution:

    • This can happen with a very large number of ScaledObjects, aggressive pollingIntervals, or complex metric queries (especially for Prometheus).
    • Increase the pollingInterval for less critical workloads to reduce the frequency of metric collection.
    • Optimize Prometheus queries if you’re using the Prometheus scaler.
    • Ensure KEDA operator pods have appropriate resource limits and requests.
    • Consider scaling out the KEDA operator (increase replicas) if the load is genuinely high.

FAQ Section

  1. What is the difference between KEDA and HPA?

    The Horizontal Pod Autoscaler (HPA) in Kubernetes scales deployments based on resource metrics like CPU and memory utilization. KEDA extends HPA by providing a much broader range of event-driven metrics (e.g., message queue length, Prometheus queries, Kafka lag) to scale your applications. KEDA essentially acts as a custom metrics provider for the HPA, creating and managing the HPA resource itself based on your ScaledObject definition.

  2. Can KEDA scale down to zero?

    Yes, one of KEDA’s most powerful features is its ability to scale down to zero replicas (minReplicaCount: 0) when there are no events to process. This is ideal for cost optimization in event-driven architectures, as you only consume resources when your application is actively working. This concept is central to “serverless” type workloads on Kubernetes.

  3. What kind of event sources does KEDA support?

    KEDA supports a vast and growing number of event sources, known as “scalers.” These include popular message queues (Kafka, RabbitMQ, Azure Service Bus, AWS SQS, Google Cloud Pub/Sub), databases (PostgreSQL, MySQL), streaming platforms, HTTP endpoints, Prometheus, custom metrics APIs, and many more. You can find the complete and up-to-date list on the official KEDA documentation website.

  4. How does KEDA handle authentication to external systems?

    KEDA uses TriggerAuthentication and ClusterTriggerAuthentication custom resources to manage credentials for connecting to external event sources. These resources typically reference Kubernetes Secrets, allowing you to securely store and manage sensitive information like API keys, connection strings, or certificates. KEDA also supports various authentication methods like pod identities (e.g., AWS IAM roles for service accounts, Azure AD Workload Identity) for cloud-native integrations.

  5. Can KEDA scale other resources besides Deployments?

    Yes, KEDA can scale a variety of Kubernetes workloads. In addition to Deployments, it can scale StatefulSets, Jobs (both regular and cron-like), and even custom resources that implement the /scale subresource. This flexibility allows KEDA to be used for a wide range of application types, from long-running services to batch processing tasks.

Cleanup Commands

To remove the resources created during this tutorial, execute the following commands. This will uninstall KEDA and delete the sample application.


# Delete the ScaledObject
kubectl delete -f http-scaledobject.yaml

# Delete the sample Nginx deployment and service
kubectl delete -f nginx-deployment.yaml

# Uninstall KEDA HTTP Add-on (if installed)
helm uninstall keda-http-add-on -n keda

# Uninstall KEDA
helm uninstall keda -n keda

# Delete the KEDA namespace
kubectl delete namespace keda

Verify that all resources have been removed:


kubectl get scaledobject
kubectl get hpa
kubectl get deployment nginx-app
kubectl get pods -n keda

Expected Output:


No resources found in default namespace.
No resources found in default namespace.
No resources found in default namespace.
No resources found in keda namespace.

Next Steps / Further Reading

You’ve successfully deployed and tested KEDA with an HTTP trigger. This is just the beginning of what you can achieve with event-driven autoscaling. To deepen your understanding and explore more advanced use cases:

Leave a comment