Scaling Out

You can scale out your app to multiple containers by adding autoscaling.

Scaling Horizontally (Adding More Containers)

When you deploy a Task Queue or endpoint, Beam creates a queueing system that manages each task that’s created when your API is called.

You can configure how Beam will scale based on how many things are in the task queue.

Scale by Queue Depth

Our simplest autoscaling strategy allows you to scale by the number of tasks in the queue.

This allows you to control how many tasks each container can process before scaling up. For example, you could setup an autoscaler to run 30 tasks per container. When you pass 30 tasks in your queue, we will add a container. When you pass 60, we’ll add another containers (up until max_containers is reached).

from beam import QueueDepthAutoscaler, endpoint

autoscaling_config = QueueDepthAutoscaler(
    max_containers=5,
    tasks_per_container=30,
)

@endpoint(autoscaler=autoscaling_config)
def function():
    ...

Setting Always-On Containers

Any running containers count towards billable usage. Take care to avoid setting min_containers unless you’re comfortable paying for usage 24/7.

You can configure the number of containers running at baseline using the min_containers field.

By setting min_containers=1, the autoscaler maintains a target of at least one running container while the deployment is active, subject to capacity and runtime health.

from beam import endpoint, QueueDepthAutoscaler


@endpoint(
    autoscaler=QueueDepthAutoscaler(
        min_containers=1, max_containers=3, tasks_per_container=1
    ),
)
def handler():
    return {"success": "true"}

Review older active versions after redeploying an app with a warm minimum. Whether a prior version is stopped depends on the rollout strategy; see deployment lifecycle. Stop versions you no longer need.