KEDA ScaledJobs: Autoscaling Batch Workloads

Introduction

Batch processing is a cornerstone of modern data workflows, handling everything from nightly reports and data transformations to machine learning inference and video encoding. In a Kubernetes environment, these workloads are typically managed by Kubernetes Job objects. While Jobs provide a robust mechanism for running finite tasks, efficiently scaling these batch processes to meet fluctuating demands can be a significant challenge. Over-provisioning leads to wasted resources and increased costs, while under-provisioning results in delays and missed SLAs.

This is where KEDA (Kubernetes Event-driven Autoscaling) steps in, offering a powerful solution for dynamically scaling your Kubernetes workloads, including batch jobs. KEDA extends Kubernetes by allowing workloads to be scaled based on metrics from various event sources like message queues, databases, and custom metrics APIs. For batch processing, KEDA’s `ScaledJob` resource is particularly revolutionary, enabling your Kubernetes Jobs to scale out (and in!) based on the actual number of pending tasks, ensuring optimal resource utilization and efficient task completion.

This guide will walk you through the process of leveraging KEDA `ScaledJob` to achieve intelligent autoscaling for your batch workloads. We’ll explore how to define a `ScaledJob`, integrate it with a message queue (like RabbitMQ) as an event source, and observe its dynamic scaling behavior. By the end of this tutorial, you’ll have a clear understanding of how to build cost-effective, responsive, and resilient batch processing pipelines on Kubernetes, ensuring your jobs run exactly when and where they’re needed, without manual intervention or static resource allocation.

Prerequisites

To follow along with this tutorial, you’ll need:

* A Kubernetes Cluster: Version 1.20+ is recommended. You can use Minikube, Kind, or a cloud-managed cluster (e.g., AWS EKS, GKE, Azure AKS).
* `kubectl` configured: Command-line tool for interacting with your Kubernetes cluster. Refer to the official Kubernetes documentation for installation instructions.
* `helm` installed: Package manager for Kubernetes. Install it by following the instructions on the Helm website.
* Basic understanding of Kubernetes Jobs: Familiarity with how Jobs and Pods work in Kubernetes.
* Basic understanding of message queues: This tutorial uses RabbitMQ, so some familiarity with its concepts will be helpful.

Step-by-Step Guide

1. Install KEDA onto your Kubernetes Cluster

Before we can leverage `ScaledJob`s, we need to install KEDA itself. KEDA is typically installed via Helm, which simplifies the deployment process significantly. KEDA consists of a controller and an admission webhook that work together to manage autoscaling.

The KEDA controller continuously monitors the configured event sources and adjusts the number of replicas for your workloads (or, in our case, the number of Job runs) based on the defined scaling rules. The admission webhook ensures that `ScaledJob` and `ScaledObject` resources are valid before they are applied to the cluster. Installing KEDA creates a dedicated namespace (`keda` by default) for its components, keeping your cluster organized.

helm repo add kedacore https://kedacore.github.io/charts
helm repo update
helm install keda kedacore/keda --namespace keda --create-namespace

Verify:

You should see KEDA-related pods running in the `keda` namespace.

kubectl get pods --namespace keda

Expected Output:

NAME                                          READY   STATUS    RESTARTS   AGE
keda-operator-5b65b78f4d-abcde                1/1     Running   0          2m
keda-operator-metrics-apiserver-76d7f7f7f-xyz12 1/1     Running   0          2m

2. Deploy RabbitMQ as our Event Source

For this tutorial, we’ll use RabbitMQ as our event source. KEDA supports a wide array of event sources, but message queues are a common pattern for batch processing, where messages in a queue represent tasks to be processed. We’ll deploy RabbitMQ using its official Helm chart.

Deploying RabbitMQ involves creating a StatefulSet, Services, and other necessary resources to run a robust message broker. We’ll expose RabbitMQ’s management interface (port 15672) and AMQP port (5672) within the cluster. Our batch processing Job will connect to the AMQP port to consume messages, and we might use the management interface to inspect queue status if needed. For production deployments, consider advanced configurations for security, persistence, and high availability. For more complex networking scenarios, especially with multiple services, you might explore solutions like Istio Ambient Mesh or Kubernetes Gateway API.

helm repo add bitnami https://charts.bitnami.com/bitnami
helm repo update
helm install rabbitmq bitnami/rabbitmq --set auth.username=user,auth.password=password,metrics.enabled=true

Verify:

Check if RabbitMQ pods are running and the service is available.

kubectl get pods -l app.kubernetes.io/name=rabbitmq
kubectl get svc -l app.kubernetes.io/name=rabbitmq

Expected Output:

NAME         READY   STATUS    RESTARTS   AGE
rabbitmq-0   1/1     Running   0          3m

NAME                 TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)                                   AGE
rabbitmq             ClusterIP   10.96.100.10     <none>        5672/TCP,15672/TCP,4369/TCP,25672/TCP   3m
rabbitmq-headless    ClusterIP   None             <none>        5672/TCP,15672/TCP,4369/TCP,25672/TCP   3m

3. Create a Kubernetes Job Template for our Batch Workload

Before defining a `ScaledJob`, we need a standard Kubernetes Job definition that KEDA can use as a template. This Job will represent a single unit of work that consumes a message from our RabbitMQ queue, processes it, and then exits. Our example job will simply print the consumed message and simulate some work by sleeping.

The Job template specifies the container image, environment variables (like RabbitMQ connection details), and resource requests/limits. KEDA will take this template and create new Job runs based on the scaling triggers. It’s crucial that your Job is designed to be idempotent and gracefully handle transient failures, as KEDA might create multiple instances. For more advanced security, consider Kubernetes Network Policies to restrict communication between your Job pods and other services.

Create a file named `job-template.yaml`:

apiVersion: batch/v1
kind: Job
metadata:
  name: rabbitmq-consumer-job
spec:
  template:
    spec:
      containers:
      - name: consumer
        image: bitnami/kubectl:latest # Using a simple image with curl/bash for demonstration
        command: ["/bin/bash", "-c"]
        args:
          - |
            echo "Starting job..."
            apt-get update && apt-get install -y curl
            RABBITMQ_HOST="rabbitmq"
            RABBITMQ_PORT="5672"
            RABBITMQ_USER="user"
            RABBITMQ_PASS="password"
            QUEUE_NAME="my-batch-queue"

            # Simple script to consume one message (or exit if none)
            # In a real scenario, you'd use proper AMQP client library
            # This is a simplified demonstration using curl against RabbitMQ's HTTP API for queues
            # NOTE: This is NOT a production-grade message consumption method.
            # A real application would use an AMQP client library (e.g., pika for Python, amqp.node for Node.js).
            # This example merely simulates a job that "processes" a message.
            
            # Check queue depth (simplified, direct API call for demonstration)
            # In a real KEDA setup, KEDA itself polls the queue for depth.
            # This job just "does work" and exits.
            
            echo "Simulating work for 10 seconds..."
            sleep 10
            echo "Finished job."
      restartPolicy: Never
  backoffLimit: 4

Apply the Job template (KEDA doesn’t apply this directly, it uses it as a blueprint):

kubectl apply -f job-template.yaml

Verify:

The Job itself won’t run yet, as it’s just a template. KEDA will instantiate it. You should see the Job definition, but no active pods.

kubectl get job rabbitmq-consumer-job

Expected Output:

NAME                    COMPLETIONS   DURATION   AGE
rabbitmq-consumer-job   0/1           <none>     10s

4. Define the KEDA ScaledJob Resource

Now, let’s define the `ScaledJob` resource. This is the core of our autoscaling solution. The `ScaledJob` tells KEDA which Job template to use, what event source to monitor (RabbitMQ, in our case), and the scaling rules.

The `ScaledJob` resource links to our `rabbitmq-consumer-job` template. It specifies a `pollingInterval` (how often KEDA checks the queue), a `minReplicaCount` (minimum number of concurrent jobs, usually 0 for batch), and a `maxReplicaCount` (maximum concurrent jobs). The `triggers` section defines the event source. Here, we’re using a RabbitMQ queue named `my-batch-queue`, and we’ll scale up if there’s at least 1 message in the queue. The `queueLength` metric specifies how many messages trigger a new job instance. For a simple 1:1 scaling, we set it to 1. KEDA will create a new Job run for every `queueLength` messages above 0, up to `maxReplicaCount`.

Create a file named `scaledjob.yaml`:

apiVersion: keda.sh/v1alpha1
kind: ScaledJob
metadata:
  name: rabbitmq-batch-scaledjob
spec:
  jobTargetRef:
    apiVersion: batch/v1
    kind: Job
    name: rabbitmq-consumer-job # Refers to the Job template we created
  pollingInterval: 10 # How often KEDA checks the queue (seconds)
  successfulJobsHistoryLimit: 5
  failedJobsHistoryLimit: 5
  maxReplicaCount: 10 # Max concurrent Job runs
  scalingStrategy:
    strategy: "default" # Can be 'default' or 'custom'
  triggers:
  - type: rabbitmq
    metadata:
      host: rabbitmq.default.svc.cluster.local:5672 # RabbitMQ service endpoint
      queueName: my-batch-queue
      mode: QueueLength
      value: "1" # Number of messages to trigger one job instance
      protocol: amqp
      usernameFromEnv: RABBITMQ_USER
      passwordFromEnv: RABBITMQ_PASS
    authenticationRef:
      name: rabbitmq-trigger-auth # Reference to a Secret for credentials

We need to create a Kubernetes Secret to hold the RabbitMQ credentials referenced by `authenticationRef`.

Create a file named `rabbitmq-secret.yaml`:

apiVersion: v1
kind: Secret
metadata:
  name: rabbitmq-trigger-auth
type: Opaque
stringData:
  RABBITMQ_USER: user
  RABBITMQ_PASS: password

Apply the Secret and ScaledJob:

kubectl apply -f rabbitmq-secret.yaml
kubectl apply -f scaledjob.yaml

Verify:

Check if the `ScaledJob` is created and KEDA has recognized it.

kubectl get scaledjob rabbitmq-batch-scaledjob
kubectl describe scaledjob rabbitmq-batch-scaledjob

Expected Output (truncated):

NAME                       AGE
rabbitmq-batch-scaledjob   1m

...
Status:
  Conditions:
    Last Transition Time:  2023-10-27T10:30:00Z
    Message:               ScaledJob is active
    Reason:                ScaledJobActive
    Status:                True
    Type:                  Active
  External Metric:
    Metric Name:                     rabbitmq-my-batch-queue
    Metric Target Value:             1
    Scaler Name:                     rabbitmq-my-batch-queue
  Last Scale Time:  <nil>
  Scale Target:     rabbitmq-consumer-job
  Triggers:
    Authentications:
      Secret Name:  rabbitmq-trigger-auth
    Metadata:
      Host:           rabbitmq.default.svc.cluster.local:5672
      Mode:           QueueLength
      Protocol:       amqp
      Queue Name:     my-batch-queue
      Value:          1
      Username From Env: RABBITMQ_USER
    Type:             rabbitmq
...

5. Produce Messages to the RabbitMQ Queue

Now that KEDA is monitoring our RabbitMQ queue, let’s put some messages into it to trigger the `ScaledJob`. We’ll use a temporary Pod with `rabbitmqadmin` to publish messages.

This step simulates an upstream service or process generating tasks that need to be handled by our batch workers. As messages accumulate in `my-batch-queue`, KEDA will detect the increased queue length and respond by creating new instances of our `rabbitmq-consumer-job`. This dynamic response is key to efficient resource utilization, ensuring that compute resources are only allocated when there’s actual work to be done.

kubectl run --rm -it rabbitmq-producer --image=rabbitmq:3-management -- /bin/bash

Inside the producer pod, run the following commands to publish messages:

# Install curl inside the temporary pod if not present
apt-get update && apt-get install -y curl

RABBITMQ_HOST="rabbitmq"
RABBITMQ_MGMT_PORT="15672"
RABBITMQ_USER="user"
RABBITMQ_PASS="password"
QUEUE_NAME="my-batch-queue"

# Create the queue if it doesn't exist (optional, often done by consumers)
# curl -u $RABBITMQ_USER:$RABBITMQ_PASS -X PUT http://${RABBITMQ_HOST}:${RABBITMQ_MGMT_PORT}/api/queues/%2F/${QUEUE_NAME}

# Publish 5 messages
for i in $(seq 1 5); do
  curl -u $RABBITMQ_USER:$RABBITMQ_PASS -X POST \
       -H "Content-Type: application/json" \
       -d "{\"properties\":{},\"routing_key\":\"${QUEUE_NAME}\",\"payload\":\"Hello from message ${i}!\",\"payload_encoding\":\"string\"}" \
       http://${RABBITMQ_HOST}:${RABBITMQ_MGMT_PORT}/api/exchanges/%2F/amq.default/publish
  echo "Published message $i"
done
exit

Verify:

You should see “Published message X” for each message. After exiting the producer pod, check the jobs.

kubectl get jobs

Expected Output (after a short delay):

You will start seeing new job runs created by KEDA, each with a unique suffix.

NAME                                                COMPLETIONS   DURATION   AGE
rabbitmq-consumer-job                               0/1           <none>     10m
keda-scaledjob-rabbitmq-batch-scaledjob-b7b5c       1/1           15s        20s
keda-scaledjob-rabbitmq-batch-scaledjob-d8d6f       1/1           15s        18s
keda-scaledjob-rabbitmq-batch-scaledjob-e9e7a       1/1           15s        16s
keda-scaledjob-rabbitmq-batch-scaledjob-f0f8b       1/1           15s        14s
keda-scaledjob-rabbitmq-batch-scaledjob-1a2b3       1/1           15s        12s

6. Observe KEDA Scaling and Job Execution

As messages are published to the queue, KEDA detects the increase in queue length. Based on the `value: “1”` in our `ScaledJob` trigger, KEDA will create a new Job instance for each message. Each Job instance will run, simulate processing (sleep for 10 seconds), and then complete.

You can watch the jobs being created, completing, and their associated pods running and then terminating. This dynamic scaling ensures that your batch workload efficiently consumes the queue without over-provisioning resources when the queue is empty, or falling behind when there’s a surge of tasks. This is a core principle of cost optimization, similar to how Karpenter can optimize node costs by dynamically provisioning nodes.

Watch the jobs and pods:

kubectl get jobs -w
kubectl get pods -w

Expected Output (dynamic):

You’ll see new jobs appearing and their completion status updating. Corresponding pods will be created, run, and then enter a `Completed` state.

# kubectl get jobs -w
NAME                                                COMPLETIONS   DURATION   AGE
rabbitmq-consumer-job                               0/1           <none>     12m
keda-scaledjob-rabbitmq-batch-scaledjob-b7b5c       0/1           1s         1s
keda-scaledjob-rabbitmq-batch-scaledjob-d8d6f       0/1           1s         1s
...
keda-scaledjob-rabbitmq-batch-scaledjob-b7b5c       1/1           15s        16s
keda-scaledjob-rabbitmq-batch-scaledjob-d8d6f       1/1           15s        17s
...

# kubectl get pods -w
NAME                                                        READY   STATUS      RESTARTS   AGE
rabbitmq-0                                                  1/1     Running     0          15m
keda-operator-5b65b78f4d-abcde                              1/1     Running     0          17m
keda-operator-metrics-apiserver-76d7f7f7f-xyz12             1/1     Running     0          17m
keda-scaledjob-rabbitmq-batch-scaledjob-b7b5c-hjklo         0/1     ContainerCreating   0          1s
keda-scaledjob-rabbitmq-batch-scaledjob-d8d6f-mnbvc         0/1     ContainerCreating   0          1s
...
keda-scaledjob-rabbitmq-batch-scaledjob-b7b5c-hjklo         0/1     Running     0          5s
keda-scaledjob-rabbitmq-batch-scaledjob-d8d6f-mnbvc         0/1     Running     0          5s
...
keda-scaledjob-rabbitmq-batch-scaledjob-b7b5c-hjklo         0/1     Completed   0          15s
keda-scaledjob-rabbitmq-batch-scaledjob-d8d6f-mnbvc         0/1     Completed   0          16s

You can also inspect the logs of a specific job run to see its output:

kubectl logs -f job/keda-scaledjob-rabbitmq-batch-scaledjob-b7b5c

Expected Output:

Starting job...
Simulating work for 10 seconds...
Finished job.

Production Considerations

Deploying `ScaledJob`s in a production environment requires careful planning beyond just basic functionality.

1. Robust Job Design: Your batch jobs must be idempotent. If a job fails and is retried (either by Kubernetes’ `backoffLimit` or KEDA’s scaling logic), re-processing a message should not lead to data corruption or incorrect states.
2. Error Handling and Dead Letter Queues (DLQ): Implement robust error handling within your job containers. For messages that consistently fail processing, ensure they are moved to a Dead Letter Queue (DLQ) in your message broker. This prevents poison messages from endlessly triggering jobs and allows for manual inspection and re-processing.
3. Resource Requests and Limits: Define appropriate `resources.requests` and `resources.limits` for your job pods. This is crucial for scheduler efficiency and preventing resource starvation or noisy neighbor issues. Incorrect settings can lead to performance bottlenecks or increased cloud costs.
4. Monitoring and Alerting: Monitor KEDA’s health, the RabbitMQ queue length, and the status of your `ScaledJob`s. Set up alerts for high queue backlogs, failed jobs, or KEDA controller issues. Tools like Prometheus and Grafana are excellent for this, and KEDA exposes its own metrics. For deeper insights into network traffic and performance, consider eBPF Observability with Hubble.
5. Security:
* Secrets Management: Use Kubernetes Secrets for sensitive credentials (like RabbitMQ passwords) and ensure they are properly restricted using RBAC.
* Image Security: Use trusted container images and scan them for vulnerabilities. Integrate with tools like Sigstore and Kyverno for supply chain security.
* Network Policies: Implement Kubernetes Network Policies to restrict network access for your job pods, allowing communication only with necessary services (e.g., RabbitMQ, databases).
6. Cost Management:
* `maxReplicaCount`: Set a reasonable `maxReplicaCount` to prevent uncontrolled scaling and unexpected cloud bills during traffic spikes or misconfigurations.
* `pollingInterval`: Adjust `pollingInterval` based on your latency requirements. Shorter intervals mean faster scaling but more API calls to the event source.
* Node Autoscaling: Integrate with cluster autoscalers like Cluster Autoscaler or Karpenter to ensure new nodes are provisioned when KEDA scales up jobs and de-provisioned when jobs complete, further optimizing infrastructure costs.
7. Logging: Centralize job logs using a logging solution like Fluentd/Fluent Bit with Elasticsearch/Loki/Splunk. This makes debugging and auditing much easier.
8. Trigger Configuration: Carefully choose the `value` for your triggers (e.g., `queueLength: “1”` vs. `queueLength: “10”`). A value of “1” scales aggressively (1 job per message), while a higher value batches messages, reducing overhead but potentially increasing latency.
9. Job Completion and Cleanup: KEDA respects `successfulJobsHistoryLimit` and `failedJobsHistoryLimit` defined in the `ScaledJob` to clean up old job resources, preventing cluster bloat. Ensure these are set appropriately.

Troubleshooting

Here are common issues you might encounter with KEDA `ScaledJob`s and their solutions:

1. KEDA not scaling up Jobs.
* Symptom: Messages in queue, but no new jobs are created.
* Solution:
* Check KEDA controller logs: `kubectl logs -n keda deployment/keda-operator`. Look for errors related to your `ScaledJob` or the event source.
* Verify `ScaledJob` status: `kubectl describe scaledjob `. Check `Conditions` and `External Metric` for any warnings or errors.
* Ensure RabbitMQ credentials are correct in the Secret and referenced correctly in `ScaledJob`’s `authenticationRef`.
* Confirm KEDA can reach RabbitMQ. Check network policies or service endpoint.
* Verify `queueName` in `ScaledJob` matches the actual queue.
* Check `pollingInterval` – KEDA won’t scale immediately.
* Ensure `maxReplicaCount` is not 0.
* Check for existing jobs from the `jobTargetRef`. If the template job is stuck or failing, it might interfere.

2. Jobs are created but fail immediately.
* Symptom: Jobs enter `Failed` status shortly after creation.
* Solution:
* Check the logs of the failed job pod: `kubectl logs job/`. This will usually reveal the exact error (e.g., connection refused, missing environment variables, application error).
* Verify RabbitMQ connectivity from within the job pod. Can it resolve the hostname? Is the port open?
* Ensure all necessary environment variables (e.g., `RABBITMQ_HOST`, `RABBITMQ_USER`, `RABBITMQ_PASS`) are correctly passed to the job container.
* Check resource limits. Pods might be OOMKilled if they exceed memory limits.

3. KEDA is scaling, but jobs aren’t consuming messages.
* Symptom: Jobs are running, but queue length remains high.
* Solution:
* The job’s logic for consuming messages might be flawed. Our example uses a dummy `sleep`. A real job needs to connect to RabbitMQ and explicitly consume a message.
* Ensure the job is configured to connect to the correct RabbitMQ queue.
* Check if the job has the necessary client libraries and permissions to interact with RabbitMQ.

4. `ScaledJob` remains inactive or stuck.
* Symptom: `ScaledJob` status shows `Reason: KedaScaledJobNotActive` or similar, even with messages in the queue.
* Solution:
* Examine the output of `kubectl describe scaledjob `. The `Message` field in `Conditions` often provides specific reasons.
* Ensure the KEDA operator and metrics server pods are healthy and running in the `keda` namespace.
* Verify that the `jobTargetRef` points to a valid `Job` template that exists in the cluster.
* Check RBAC permissions for KEDA. Does it have permission to list/watch Jobs and interact with the autoscaling API? (Usually handled by KEDA’s Helm chart).

5. Too many or too few jobs scaling.
* Symptom: KEDA creates more jobs than expected, or fewer than needed.
* Solution:
* Review `maxReplicaCount` in `ScaledJob`. This caps the number of concurrent jobs.
* Adjust `value` in the trigger metadata. For RabbitMQ `queueLength`, `value: “1”` means 1 job per 1 message, while `value: “10”` means 1 job for every 10 messages.
* Consider `minReplicaCount`. If set above 0, KEDA will always maintain at least that many jobs, even with an empty queue.
* Check if multiple `ScaledJob`s are targeting the same queue or if there are other HPA/Job controllers interfering.

6. RabbitMQ connection issues from KEDA or Job pods.
* Symptom: KEDA logs show connection errors to RabbitMQ; job pods fail to connect.
* Solution:
* Verify the `host` in the `ScaledJob` trigger metadata. Use the full service name: `rabbitmq.default.svc.cluster.local:5672`.
* Ensure the RabbitMQ service is running and accessible (e.g., `kubectl get svc rabbitmq`).
* Check firewall rules or network policies (`NetworkPolicy`) that might be blocking communication between KEDA/Job pods and RabbitMQ.
* Test connectivity manually from a debug pod: `kubectl run debug –rm -it –image=ubuntu — bash` then `apt update && apt install -y iputils-ping netcat && ping rabbitmq` or `nc -vz rabbitmq 5672`.

FAQ Section

1. What is the difference between `ScaledJob` and `ScaledObject`?
`ScaledJob` is specifically designed for scaling Kubernetes Jobs, which are finite, batch-oriented tasks. It creates new Job runs based on triggers. `ScaledObject` is used for scaling long-running deployments (like Deployments, StatefulSets, or custom resources) by adjusting their replica count.
2. Can KEDA scale jobs based on custom metrics?
Yes, KEDA supports custom metrics. You can create a Prometheus scaler to pull metrics from a Prometheus endpoint, or implement a custom external scaler if your metric source isn’t directly supported by KEDA’s built-in scalers.
3. How does KEDA handle job completion and cleanup?
KEDA respects the `successfulJobsHistoryLimit` and `failedJobsHistoryLimit` fields defined in the `ScaledJob` spec. These fields determine how many completed (successful or failed) job runs are kept in the Kubernetes API before being automatically cleaned up.
4. What happens if KEDA itself goes down?
If the KEDA operator goes down, existing `ScaledJob`s will continue to run their current instances. However, no new job instances will be created, and scaling adjustments (up or down) will cease until the KEDA operator is restored. KEDA is designed for high availability, and its components are typically deployed as Deployments, allowing Kubernetes to restart them if they fail.
5. Is KEDA suitable for real-time, low-latency processing?
While KEDA can react quickly, its `pollingInterval` introduces some latency. For extremely low-latency, real-time processing where every millisecond counts, you might consider streaming processing frameworks (like Apache Flink or Kafka Streams) or custom-built solutions that maintain persistent connections and react instantly to events. KEDA is excellent for event-driven autoscaling where a short polling delay is acceptable for batch or asynchronous tasks.

Cleanup Commands

To remove all resources created during this tutorial:

# Delete ScaledJob and Secret
kubectl delete -f scaledjob.yaml
kubectl delete -f rabbitmq-secret.yaml

# Delete the Job template (this will also delete any KEDA-created job runs)
kubectl delete -f job-template.yaml

# Uninstall RabbitMQ Helm chart
helm uninstall rabbitmq

# Uninstall KEDA Helm chart
helm uninstall keda --namespace keda
kubectl delete namespace keda

# Clean up any remaining producer pods if they weren't --rm
kubectl delete pod rabbitmq-producer --ignore-not-found

Next Steps / Further Reading

* Explore KEDA Scalers: KEDA supports over 50 different scalers for various event sources. Experiment with other triggers like Kafka, AWS SQS, Azure Service Bus, or Prometheus.
* Advanced Job Patterns: Research Kubernetes CronJobs for scheduled tasks and how KEDA can extend their capabilities.
* Cost Optimization with Karpenter: Learn how KEDA’s scaling can be combined with node autoscalers like Karpenter to dynamically provision nodes only when jobs need to run, leading