Skip to content

# Erlang Distribution

## Overview

ejabberd uses the Distributed Erlang feature for two purposes:

- `ejabberdctl` shell script that connects to your running ejabberd node to allow you executing API commands in it, opening an interactive erlang shell, ...

- clustering several ejabberd nodes that you deploy in separate machines to serve a large XMPP domain with fault-tolerance and scalability.

There are three topics involved in erlang distribution:

- Cookie, automatically generated, shared by your ejabberd nodes, and secret to the world
- Node Name that you pick for each ejabberd node to identify them in a cluster
- Port number to use, determined using either:
  - ERL_DIST_PORT environment variable that lets you assign your desired port number
  - epmd program that assigns a random port number and maps names with ports

## Cookie

cookie

Random alphanumeric string assigned to each erlang node used to secure connections between erlang nodes. See Security in distributed erlang.

An Erlang node reads the cookie at startup from the command-line parameter `-setcookie`. If not indicated, the cookie is read from the file `$HOME/.erlang.cookie`.

If this file does not exist, it is created immediately with a random cookie in the user `$HOME` path. This means the user running ejabberd must have a `$HOME`, and have write access to that path. So, when you create a new account in your system for running ejabberd, either allow it to have a `$HOME`, or set as `$HOME` a path where ejabberd will have write access. Depending on your setup, examples could be:

    adduser --home /usr/local/var/lib/ejabberd ejabberd

or

    adduser --home /var/lib/ejabberd ejabberd

Two Erlang nodes communicate only if they have the same cookie. Setting a cookie on the Erlang node allows you to structure your Erlang network and define which nodes are allowed to connect to which.

Thanks to Erlang cookies, you can prevent access to the Erlang node by mistake, for example when there are several Erlang nodes running different programs in the same machine.

Setting a secret cookie is a simple method to difficult unauthorized access to your Erlang node. However, the cookie system is not ultimately effective to prevent unauthorized access or intrusion to an Erlang node. The communication between Erlang nodes are not encrypted, so the cookie could be read sniffing the traffic on the network. The recommended way to secure the Erlang node is to block the port 4369.

## Node Name

erlang node

A node in erlang connected to a cluster. See Erlang node name

Variable in ejabberdctl.cfg | Corresponding argument in Erlang/OTP
--- | ---
`ERLANG_NODE=ejabberd@localhost` | -name `ejabberd@localhost` -sname `ejabberd`

An Erlang node may have a node name. The name can be short (if indicated with the command-line parameter `-sname`) or long (if indicated with the parameter `-name`). Starting an Erlang node with `-sname` limits the communication between Erlang nodes to the LAN.

Using the argument `-sname` instead of `-name` is a simple method to difficult unauthorized access to your Erlang node. However, it is not ultimately effective to prevent access to the Erlang node, because it may be possible to fake the fact that you are on another network using a modified version of Erlang epmd. The recommended way to secure the Erlang node is to block the port 4369.

## `ERL_DIST_PORT`

Variable in ejabberdctl.cfg | Corresponding argument in Erlang/OTP
--- | ---
`INET_DIST_INTERFACE` | -kernel inet_dist_use_interface
`ERL_DIST_PORT=5210` | -erl_epmd_port `5210` -start_epmd `false`

The TCP port where your Erlang node listens for Erlang distribution connections can be set with the environment variable `ERL_DIST_PORT` in the file `ejabberdctl.cfg`. When attempting to connect to other remote nodes, it will also try to connect to that port number.

By default this variable is set to `5210`, but you can set any number you prefer as long as it isn't used in the machine, and it is the same for all the other ejabberd nodes that will build the cluster.

When building a cluster of several Erlang nodes, all of them must have the same `ERL_DIST_PORT`, because this is used for listening and also for connecting. Consequently, if you want to build a cluster of several nodes in the same machine, each node must have a different IP address in `INET_DIST_INTERFACE`.

Just as a small note: since Erlang/OTP 27.2, the argument `-erl_epmd_port` is obsoleted by -kernel erl_epmd_node_listen_port. However, ejabberd does not yet use the new argument because ejabberd supports Erlang/OTP 25+.

## epmd

Variable in ejabberdctl.cfg | Corresponding argument in Erlang/OTP
--- | ---
`INET_DIST_INTERFACE` | -kernel inet_dist_use_interface
`FIREWALL_WINDOW=XX-YY` | -kernel inet_dist_listen_min `XX` inet_dist_listen_max `YY`
`ERL_EPMD_ADDRESS` | ERL_EPMD_ADDRESS

epmd

Small name server included in Erlang/OTP and used by Erlang programs when establishing distributed Erlang communications. See `epmd (Erlang Port Mapper Daemon)`

When ERL_DIST_PORT is not set in `ejabberdctl.cfg`, Erlang starts the `epmd` program (which listens in port `4369`), and connects to that program to ask for a random port number to use.

You should block the port `4369` in the firewall in such a way that only the programs in your machine can access it, or configure the environment variable `ERL_EPMD_ADDRESS` in the file `ejabberdctl.cfg`.

When building a cluster of several ejabberd nodes, if you don't set ERL_DIST_PORT with the same port number in `ejabberdctl.cfg` in all the nodes, then you must open the port `4369` for all the machines involved in the cluster so their epmd programs can be accessed. Remember to block the port so Internet doesn't have access to it.

Once an Erlang node solved the node name of another Erlang node using EPMD and port `4369`, the nodes communicate directly. The ports used in this case by default are random, and can be restricted with the environment variable `FIREWALL_WINDOW` in the file `ejabberdctl.cfg`.

The network interface where the Erlang node will listen and accept connections can be configured with the environment variable `INET_DIST_INTERFACE` in the file `ejabberdctl.cfg`.