How to Implement Horizontal Scaling

How to scale your Rudder infra without event ordering guarantee

Overview

A single instance of backend guarantees delivery of a given user events in the same order as they were received. This is achieved by caching the user events on the running instance and pausing the later events if any prior user event fails.

In future versions, Rudder will support a clustered architecture where the event ordering is guaranteed in a multi-node cluster. Many analytics destinations don't need ordering guarantee since they use the timestamp of the event(Eg. Amplitude, MixPanel). If your use case fits the same, the following article guides you how to set up a scale out version of Rudder.

Components

  1. Backend

  2. Database

  3. Transformer

Backend and Database will always scale together, think of them as a Single pod in your Kubernetes cluster

Scaling Backend and Database

There should be a one-to-one mapping between a backend instance and a database instance. For the following article, please refer a node as a tuple of backend instance and database instance.

Every Rudder node handles an estimated load for a given instance type. Please refer to the scaling stats page for more details. For example, one m4.2xlarge machine on AWS can handle 3k requests per second i.e., 250M requests per day (each request can have more than 1 event).

As always, the real traffic can spike based on various events in your product. If the load is more than the expected value, you can spawn more servers to handle the load. One can subscribe to application stats and figure out when to scale as mentioned in the monitoring page.

How to scale

Every node requires a unique INSTANCE_ID. Eg. node1 has INSTANCE_ID=1 and node2 has INSTANCE_ID=2.

  1. Create a new database instance

  2. Create a new backend instance

  3. Add the INSTANCE_ID in the .env file before starting the service

  4. Add the node to your load balancer

  5. If the backend instance goes down, bring up a new instance with the same INSTANCE_ID. You can automate this with your custom infra cluster setups.

Scaling Transformer

Transformer has two variants

  • Destination transformer

  • Destination transformer + User Functions

Destination transformer is a NodeJS/Koa application and stateless. You can run multiple containers/instances and scale based on the load. Please refer to the setup for Docker containers or native installation in the Github page. This can also be scaled with Lambda. Rudder recommends Up for easy deployment of the app to into your own AWS Lambda.

Destination transformer + User Functions uses isolate-vm for running user javascript functions in a secure and isolated environment. This has OS dependencies which don't work in the AWS Lambda straight away. You can follow the setup instructions on the Github page for native installation and docker containers. This is as easy as running plugging in your docker image.

Docker Compose Example

In the following example, we create

  • 2 backend containers and a NGINX reverse proxy that load balances among the backend containers

  • 2 database containers one for each backend container

  • 3 transformer containers and a NGINX reverse proxy that load balances among them

# Start dependenct containers first
$ docker-compose -f docker-compose-multi.yml run --rm dependencies
# Start the client facing load balancers
$ docker-compose -f docker-compose-multi.yml up