Node.js

Detailed technical documentation on using RudderStack’s Node.js SDK to send server-side events to various destinations.

RudderStack’s Node.js SDK lets you track your customer event data from your Node.js app and route them to your specified destination platforms.

Find the code for this SDK in our GitHub repository.

Installing the Node.js SDK

To install the RudderStack Node.js SDK using npm, run the following command:

npm install @rudderstack/rudder-sdk-node

Using the SDK

To use the Node.js SDK, run the following code snippet:

const Analytics = require("@rudderstack/rudder-sdk-node");
// RudderStack requires the batch endpoint of the server you are running
const client = new Analytics(WRITE_KEY, `${DATA_PLANE_URL}/v1/batch`);

Running the above snippet creates a global RudderStack client object that can be used for all the subsequent event requests.

Supported Events

The RudderStack Node.js SDK supports the following API calls:

  • Identify

  • Track

  • Page

  • Group

  • Alias

For a detailed information on each of these calls, refer to our RudderStack API Specification guide.

RudderStack does not store the user state in any of the server-side SDKs. Unlike the client-side SDKs that deal with only a single user at a given time, the server-side SDKs deal with multiple users at the same time. Therefore, for any of the calls supported by the Node.js SDK, you need to specify either userId or anonymousId every time.

Identify

The identify call lets you associate a user to their actions as well as captures the relevant traits or properties related to that user.

An example identify call is as shown:

client.identify({
userId: '123456',
traits: {
name: 'Name Username',
plan: 'Free',
friends: 21
}
});

The identify parameters are as described below:

Field

Type

Presence

Description

anonymousId

String

Optional

A user identifier for cases where there is no userId set for the user. Either userId or anonymousId is required.

userId

String

Required, if anonymousId is not present

The unique identifier for a user in your database.

context

Object

Optional

The dictionary of information that provides context about a message. Note that it is not directly related to the API call.

integrations

Object

Optional

The dictionary of destinations to be either enabled or disabled.

timestamp

Date

Optional

The timestamp of the message's arrival.

``traits``

Object

Optional

The dictionary of the traits associated with the user, such as nameor email.

Track

The track call lets you record the users' actions along with their associated properties. Each user action is called an 'event'.

An example track call is as shown:

client.track({
userId: '123456',
event: 'Item Viewed',
properties: {
revenue: 19.95,
shippingMethod: 'Premium'
}
});

The track method parameters are as described below:

Name

Type

Presence

Description

userId

String

Required, if anonymousId is not present.

The unique identifier for a user in your database.

anonymousId

String

Optional

A user identifier for cases where there is no userId set for the user. Either userId or anonymousId is required.

event

String

Required

Name of the user event.

properties

Object

Optional

The dictionary of the properties associated with the particular event.

context

Object

Optional

The dictionary of information that provides context about a message. Note that it is not directly related to the API call.

timestamp

Date

Optional

The timestamp of the message's arrival.

integrations

Object

Optional

A dictionary of destinations to be either enabled or disabled.

Page

The page call allows you to record the page views on your website. It also records the other relevant information about the page that is being viewed.

An example page call is as shown:

client.page({
userId: '12345',
category: 'Food',
name: 'Pizza',
properties: {
url: 'https://dominos.com',
title: 'Pizza',
referrer: 'https://google.com'
}
});

The page method parameters are as described below:

Field

Type

Presence

Description

anonymousId

String

Optional

A user identifier for cases where there is no userId set for the user. Either userId or anonymousId is required.

userId

String

Required, if anonymousId is not present.

The unique identifier for a user in your database.

context

Object

Optional

The dictionary of information that provides context about a message. Note that it is not directly related to the API call.

integrations

Object

Optional

A dictionary of destinations to be either enabled or disabled.

name

String

Required

Name of the viewed page.

properties

Object

Optional

A dictionary of the properties associated with the viewed page, like url and referrer.

timestamp

Date

Optional

The timestamp of the message's arrival.

Group

The group call allows you to link an identified user with a group, such as a company, organization, or an account. It also lets you record any custom traits associated with that group, such as the name of the company, the number of employees, etc.

An example group call is as shown:

client.group({
userId: '12345',
groupId: '1',
traits: {
name: 'Company',
description: 'Google'
}
});

The group method parameters are as follows:

Field

Type

Presence

Description

anonymousId

String

Optional

A user identifier for cases where there is no userId set for the user. Either userId or anonymousId is required.

userId

String

Required, if anonymousId is not present.

The unique identifier for a user in your database.

context

Object

Optional

A dictionary of information that provides context about a message. Note that it is not directly related to the API call.

integrations

Object

Optional

A dictionary of the destinations to be either enabled or disabled.

groupId

String

Required

Unique identifier for the group present in your database.

traits

Object

Optional

A dictionary of the group's properties or traits, like email or name.

timestamp

Date

Optional

The timestamp of the message's arrival.

Alias

The alias call lets you merge different identities of a known user.

alias is an advanced method that lets you change the tracked user's ID explicitly. This method is useful when managing identities for some of the downstream destinations.

For a detailed explanation of the alias call, refer to our RudderStack API Specification guide.

An example alias call is as shown:

client.alias({
previousId: 'old_id',
userId: 'new_id'
});

The alias method parameters are as mentioned below:

Field

Type

Presence

Description

userId

String

Required, if anonymousId is not present.

The unique identifier for a user in your database.

context

Object

Optional

A dictionary of information that provides context about a message. Note that it is not directly related to the API call.

integrations

Object

Optional

A dictionary of the destinations to be either enabled or disabled.

previousId

String

Required

The previous unique identifier of the user.

traits

Object

Optional

The dictionary of the traits associated with the user, like nameor email.

timestamp

Date

Optional

The timestamp of the message's arrival.

Node.js SDK Data Persistence

This feature is still in beta. Contact us on our Slack channel if you face any issues.

If the Node.js SDK fails to successfully deliver the event data to RudderStack at the first attempt, it retries the event delivery. However, if, for some reason, RudderStack is unavailable for a longer duration, there is a possibility of data loss. To prevent this scenario, RudderStack has a data persistence feature to persist the events in Redis, leading to better event delivery guarantees. Also, the SDK can retry multiple times as the queue is maintained in a different process space (in this case, Redis).

To use this feature, you will need to host a Redis server and use it as the intermediary data storage queue.

RudderStack uses Bull as the interface layer between the Node.js SDK and Redis.

A sample initialization of the SDK is as shown:

const client = new Analytics(WRITE_KEY,`${DATA_PLANE_URL}/v1/batch`,{
flushAt:20,
flushInterval:20000,
maxInternalQueueSize:20000
});
client.createPersistenceQueue(queueOpts, callback)

To achieve the data persistence, you need to call the createPersistenceQueue method which takes two parameters as input - queueOpts and a callback. This will initialize the persistent queue.

If the createPersistenceQueue method is not called after initializing the SDK, the SDK will work without any persistence.

Configurable Parameters

Parameter

Description

Default Value

flushAt

The maximum number of events to batch and send to the server

20

flushInterval

The maximum timespan (in milliseconds) after which the events from the in-memory queue is flushed to Redis' persistence queue

20000

maxInternalQueueSize

The maximum size of the in-memory queue

20000

JobOpts.maxAttempts

The maximum number of retry attempts

10

isMultiProcessor

Determines whether to handle previously active jobs. If set to false, the previously active job will be picked up first by the processor. Otherwise, Bull moves this job to the back of the Redis queue to be picked up after the already pushed event.

false

If the same queue (RudderStack SDK initialized with the same queue name) is used in case of multiple servers (server-side SDKs), set the value of isMultiProcessor to true as event ordering is not applicable in this case.

How to ensure that all my events in the queue are processed?

You can use the flush() method to ensure that all the events in the queue are processed. The following example highlights the use of flush() with a callback:

client.flush(function(err, batch){
console.log('Flushing done');
}
)

queueOpts

As mentioned in the previous section, you need to call the createPersistenceQueue method which takes two parameters as input - queueOpts and a callback- to achieve data persistence. This method will initialize the persistent queue.

client.createPersistenceQueue(QueueOpts, callback)

Calling the createPersistenceQueue method will initialize a Redis list by calling Bull's utility methods. It will also add a single job processor for the processing (making requests to RudderStack) jobs that are pushed into the list. Any error encountered while doing this leads to a callback with the error.

A sample queueOpts initialization is shown below:

queueOpts {
queueName ?: string = rudderEventsQueue,
isMultiProcessor ? : boolean = false
// for prefix, pass a value without the {}, this will used as prefix to Redis keys,
// needed to support Redis cluster slots.
prefix ? : string = {rudder},
redisOpts : RedisOpts,
jobOpts ?: JobOpts
}

The specification of the different queueOpts parameters is listed in the following table:

Parameter

Description

Default Value

queueName

Name of the queue.

20

isMultiProcessor

Determines whether to handle previously active jobs. If set to false, the previously active job will be picked up first by the processor. Otherwise, Bull moves this job to the back of the Redis queue to be picked up after the already pushed event.

false

prefix

Used as the prefix to the Redis keys needed to support the Redis cluster slots.

20000

redisOpts

Refer to the RedisOpts section below

RedisOpts

jobOpts

Refer to the JobOpts section below

JobOpts

More information on this parameter can be found in the Bull docs.

RedisOpts

RedisOpts {
port?: number = 6379;
host?: string = localhost;
db?: number = 0;
password?: string;
}

More information on this parameter can be found in the Bull docs.

JobOpts

JobOpts {
maxAttempts?: number = 10
}

callback

In case of an error, the createPersistenceQueue method returns a callback. You should retry sending the events in this scenario.

// createPersistenceQueue calls this with error or nothing(in case of success), // user should retry in case of error
callback: function(error) || function()

Calling the createPersistenceQueue method will initialize a Redis list by calling Bull's utility methods. It will also add a single job processor for the processing (making requests to RudderStack) jobs that are pushed into the list. Any error encountered while doing this leads to calling callback with the error.

If the callback returns with an error, we recommend retrying calling createPersistenceQueue with a backoff.

Event Flow

  • Calling the SDK methods like track, page, identify, etc. pushes the events to an in-memory array.

  • The events from the array are flushed as a batch to the Redis persistence based on the flushAt and flushInterval settings. The in-memory array has a maximum size of maxInternalQueueSize. Once this size limit is reached, the events won't be accepted if not drained to the queue (applicable in cases where the Redis connection is slow or the Redis server is not reachable).

  • The processor will take the batch from the Redis list and make a request to RudderStack. In case of an error, the processor will retry sending the data again if the error can be retried (in case of errors with the status code 5xx and 429). The retry will go up to JobOpts.maxAttempts with an exponential backoff of power 2 with a max backoff of 30s.

  • If the job fails even after the JobOpts.maxAttempts limit is reached, it will not be retried again and pushed to a failed queue. You can retry sending these events manually using Bull’s utility methods listed here or get them from Redis directly.

  • There might be a scenario where the node process dies with the jobs still in active state (not completed nor failed but in the process of sending or retrying). Since the RudderStack SDK has only 1 processor for sending events (this count should always be 1), the next time the SDK is initialized and createPersistenceQueue is called, the jobs will be picked up first by the processor to maintain the event ordering - based on the value set for queueOpts.isMultiProcessor.

  • For multiple server-side SDKs connecting to the same queue (queueOpts.queueName), there will be multiple processors fetching events from the same queue and event ordering won’t be implemented. In this case, queueOpts.isMultiProcessor should be set to true.

Contact Us

For more information on any of the sections covered in this guide, you can contact us or start a conversation on our Slack channel.