User Guide¶

Authentication¶

The EMQ broker supports to authenticate MQTT clients with ClientID, Username/Password, IpAddress and even HTTP Cookies.

The authentication is provided by a list of plugins such as MySQL, PostgreSQL and Redis…

If we enable several authentication plugins at the same time, the authentication process:

           ----------------           ----------------           -------------
Client --> |   Username   | -ignore-> |   ClientID   | -ignore-> | Anonymous |
           ----------------           ----------------           -------------
                  |                         |                         |
                 \|/                       \|/                       \|/
            allow | deny              allow | deny              allow | deny

The authentication plugins implemented by default:

Plugin	Description
emq_auth_clientid	ClientId Auth Plugin
emq_auth_username	Username Auth Plugin
emq_auth_ldap	LDAP Auth Plugin
emq_auth_http	HTTP Auth/ACL Plugin
emq_auth_mysql	MySQL Auth/ACL Plugin
emq_auth_pgsql	Postgre Auth/ACL Plugin
emq_auth_redis	Redis Auth/ACL Plugin
emq_auth_mongo	MongoDB Auth/ACL Plugin

Allow Anonymous¶

Configure etc/emq.conf to allow anonymous authentication:

## Allow Anonymous authentication
mqtt.allow_anonymous = true

Username/Password¶

Authenticate MQTT client with Username/Password:

Configure default users in etc/plugins/emq_auth_username.conf:

auth.user.$N.username = admin
auth.user.$N.password = public

Enable emq_auth_username plugin:

./bin/emqttd_ctl plugins load emq_auth_username

Add user by ‘./bin/emqttd_ctl users’ command:

$ ./bin/emqttd_ctl users add <Username> <Password>

ClientId¶

Authentication with MQTT ClientId.

Configure Client Ids in etc/plugins/emq_auth_clientid.conf:

auth.client.$N.clientid = clientid
auth.client.$N.password = passwd

Enable emq_auth_clientid plugin:

./bin/emqttd_ctl plugins load emq_auth_clientid

LDAP¶

etc/plugins/emq_auth_ldap.conf:

auth.ldap.servers = 127.0.0.1

auth.ldap.port = 389

auth.ldap.timeout = 30

auth.ldap.user_dn = uid=%u,ou=People,dc=example,dc=com

auth.ldap.ssl = false

Enable LDAP plugin:

./bin/emqttd_ctl plugins load emq_auth_ldap

HTTP¶

etc/plugins/emq_auth_http.conf:

## Variables: %u = username, %c = clientid, %a = ipaddress, %P = password, %t = topic

auth.http.auth_req = https://127.0.0.1:8080/mqtt/auth
auth.http.auth_req.method = post
auth.http.auth_req.params = clientid=%c,username=%u,password=%P

auth.http.super_req = https://127.0.0.1:8080/mqtt/superuser
auth.http.super_req.method = post
auth.http.super_req.params = clientid=%c,username=%u

Enable HTTP Plugin:

./bin/emqttd_ctl plugins load emq_auth_http

MySQL¶

Authenticate with MySQL database. Suppose that we create a mqtt_user table:

CREATE TABLE `mqtt_user` (
  `id` int(11) unsigned NOT NULL AUTO_INCREMENT,
  `username` varchar(100) DEFAULT NULL,
  `password` varchar(100) DEFAULT NULL,
  `salt` varchar(20) DEFAULT NULL,
  `created` datetime DEFAULT NULL,
  PRIMARY KEY (`id`),
  UNIQUE KEY `mqtt_username` (`username`)
) ENGINE=MyISAM DEFAULT CHARSET=utf8;

Configure the ‘auth_query’ and ‘password_hash’ in etc/plugins/emq_auth_mysql.conf:

## Mysql Server
auth.mysql.server = 127.0.0.1:3306

## Mysql Pool Size
auth.mysql.pool = 8

## Mysql Username
## auth.mysql.username =

## Mysql Password
## auth.mysql.password =

## Mysql Database
auth.mysql.database = mqtt

## Variables: %u = username, %c = clientid

## Authentication Query: select password only
auth.mysql.auth_query = select password from mqtt_user where username = '%u' limit 1

## Password hash: plain, md5, sha, sha256, pbkdf2
auth.mysql.password_hash = sha256

## %% Superuser Query
auth.mysql.super_query = select is_superuser from mqtt_user where username = '%u' limit 1

Enable MySQL plugin:

./bin/emqttd_ctl plugins load emq_auth_mysql

PostgreSQL¶

Authenticate with PostgreSQL database. Create a mqtt_user table:

CREATE TABLE mqtt_user (
  id SERIAL primary key,
  username character varying(100),
  password character varying(100),
  salt character varying(40)
);

Configure the ‘auth_query’ and ‘password_hash’ in etc/plugins/emq_auth_pgsql.conf:

## Postgre Server
auth.pgsql.server = 127.0.0.1:5432

auth.pgsql.pool = 8

auth.pgsql.username = root

#auth.pgsql.password =

auth.pgsql.database = mqtt

auth.pgsql.encoding = utf8

auth.pgsql.ssl = false

## Variables: %u = username, %c = clientid, %a = ipaddress

## Authentication Query: select password only
auth.pgsql.auth_query = select password from mqtt_user where username = '%u' limit 1

## Password hash: plain, md5, sha, sha256, pbkdf2
auth.pgsql.password_hash = sha256

## sha256 with salt prefix
## auth.pgsql.password_hash = salt sha256

## sha256 with salt suffix
## auth.pgsql.password_hash = sha256 salt

## Superuser Query
auth.pgsql.super_query = select is_superuser from mqtt_user where username = '%u' limit 1

Enable the plugin:

./bin/emqttd_ctl plugins load emq_auth_pgsql

Redis¶

Authenticate with Redis. MQTT users could be stored in redis HASH, the key is “mqtt_user:<Username>”.

Configure ‘auth_cmd’ and ‘password_hash’ in etc/plugins/emq_auth_redis.conf:

## Redis Server
auth.redis.server = 127.0.0.1:6379

## Redis Pool Size
auth.redis.pool = 8

## Redis Database
auth.redis.database = 0

## Redis Password
## auth.redis.password =

## Variables: %u = username, %c = clientid

## Authentication Query Command
auth.redis.auth_cmd = HGET mqtt_user:%u password

## Password hash: plain, md5, sha, sha256, pbkdf2
auth.redis.password_hash = sha256

## Superuser Query Command
auth.redis.super_cmd = HGET mqtt_user:%u is_superuser

Enable the plugin:

./bin/emqttd_ctl plugins load emq_auth_redis

MongoDB¶

Create a mqtt_user collection:

{
    username: "user",
    password: "password hash",
    is_superuser: boolean (true, false),
    created: "datetime"
}

Configure super_query, auth_query in etc/plugins/emq_auth_mongo.conf:

## Mongo Server
auth.mongo.server = 127.0.0.1:27017

## Mongo Pool Size
auth.mongo.pool = 8

## Mongo User
## auth.mongo.user =

## Mongo Password
## auth.mongo.password =

## Mongo Database
auth.mongo.database = mqtt

## auth_query
auth.mongo.auth_query.collection = mqtt_user

auth.mongo.auth_query.password_field = password

auth.mongo.auth_query.password_hash = sha256

auth.mongo.auth_query.selector = username=%u

## super_query
auth.mongo.super_query.collection = mqtt_user

auth.mongo.super_query.super_field = is_superuser

auth.mongo.super_query.selector = username=%u

Enable the plugin:

./bin/emqttd_ctl plugins load emq_auth_mongo

ACL¶

The ACL of EMQ broker is responsible for authorizing MQTT clients to publish/subscribe topics.

The ACL rules define:

Allow|Deny Who Publish|Subscribe Topics

Access Control Module of EMQ broker will match the rules one by one:

          ---------              ---------              ---------
Client -> | Rule1 | --nomatch--> | Rule2 | --nomatch--> | Rule3 | --> Default
          ---------              ---------              ---------
              |                      |                      |
            match                  match                  match
             \|/                    \|/                    \|/
        allow | deny           allow | deny           allow | deny

Internal¶

The default ACL of EMQ broker is implemented by an ‘internal’ module.

Enable the ‘internal’ ACL module in etc/emq.conf:

## ACL nomatch
mqtt.acl_nomatch = allow

## Default ACL File
mqtt.acl_file = etc/acl.conf

The ACL rules of ‘internal’ module are defined in ‘etc/acl.conf’ file:

%% Allow 'dashboard' to subscribe '$SYS/#'
{allow, {user, "dashboard"}, subscribe, ["$SYS/#"]}.

%% Allow clients from localhost to subscribe any topics
{allow, {ipaddr, "127.0.0.1"}, pubsub, ["$SYS/#", "#"]}.

%% Deny clients to subscribe '$SYS#' and '#'
{deny, all, subscribe, ["$SYS/#", {eq, "#"}]}.

%% Allow all by default
{allow, all}.

HTTP API¶

ACL by HTTP API: https://github.com/emqtt/emq_auth_http

Configure etc/plugins/emq_auth_http.conf and enable the plugin:

## 'access' parameter: sub = 1, pub = 2
auth.http.acl_req = https://127.0.0.1:8080/mqtt/acl
auth.http.acl_req.method = get
auth.http.acl_req.params = access=%A,username=%u,clientid=%c,ipaddr=%a,topic=%t

MySQL¶

ACL with MySQL database. The mqtt_acl table and default data:

CREATE TABLE `mqtt_acl` (
  `id` int(11) unsigned NOT NULL AUTO_INCREMENT,
  `allow` int(1) DEFAULT NULL COMMENT '0: deny, 1: allow',
  `ipaddr` varchar(60) DEFAULT NULL COMMENT 'IpAddress',
  `username` varchar(100) DEFAULT NULL COMMENT 'Username',
  `clientid` varchar(100) DEFAULT NULL COMMENT 'ClientId',
  `access` int(2) NOT NULL COMMENT '1: subscribe, 2: publish, 3: pubsub',
  `topic` varchar(100) NOT NULL DEFAULT '' COMMENT 'Topic Filter',
  PRIMARY KEY (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;

INSERT INTO mqtt_acl (id, allow, ipaddr, username, clientid, access, topic)
VALUES
    (1,1,NULL,'$all',NULL,2,'#'),
    (2,0,NULL,'$all',NULL,1,'$SYS/#'),
    (3,0,NULL,'$all',NULL,1,'eq #'),
    (5,1,'127.0.0.1',NULL,NULL,2,'$SYS/#'),
    (6,1,'127.0.0.1',NULL,NULL,2,'#'),
    (7,1,NULL,'dashboard',NULL,1,'$SYS/#');

Configure ‘acl-query’ and ‘acl_nomatch’ in etc/plugins/emq_auth_mysql.conf:

## ACL Query Command
auth.mysql.acl_query = select allow, ipaddr, username, clientid, access, topic from mqtt_acl where ipaddr = '%a' or username = '%u' or username = '$all' or clientid = '%c'

PostgreSQL¶

ACL with PostgreSQL database. The mqtt_acl table and default data:

CREATE TABLE mqtt_acl (
  id SERIAL primary key,
  allow integer,
  ipaddr character varying(60),
  username character varying(100),
  clientid character varying(100),
  access  integer,
  topic character varying(100)
);

INSERT INTO mqtt_acl (id, allow, ipaddr, username, clientid, access, topic)
VALUES
    (1,1,NULL,'$all',NULL,2,'#'),
    (2,0,NULL,'$all',NULL,1,'$SYS/#'),
    (3,0,NULL,'$all',NULL,1,'eq #'),
    (5,1,'127.0.0.1',NULL,NULL,2,'$SYS/#'),
    (6,1,'127.0.0.1',NULL,NULL,2,'#'),
    (7,1,NULL,'dashboard',NULL,1,'$SYS/#');

Configure ‘acl_query’ and ‘acl_nomatch’ in etc/plugins/emq_auth_pgsql.conf:

## ACL Query. Comment this query, the acl will be disabled.
auth.pgsql.acl_query = select allow, ipaddr, username, clientid, access, topic from mqtt_acl where ipaddr = '%a' or username = '%u' or username = '$all' or clientid = '%c'

Redis¶

ACL with Redis. The ACL rules are stored in a Redis HashSet:

HSET mqtt_acl:<username> topic1 1
HSET mqtt_acl:<username> topic2 2
HSET mqtt_acl:<username> topic3 3

Configure acl_cmd and acl_nomatch in etc/plugins/emq_auth_redis.conf:

## ACL Query Command
auth.redis.acl_cmd = HGETALL mqtt_acl:%u

MongoDB¶

Store ACL Rules in a mqtt_acl collection:

{
    username: "username",
    clientid: "clientid",
    publish: ["topic1", "topic2", ...],
    subscribe: ["subtop1", "subtop2", ...],
    pubsub: ["topic/#", "topic1", ...]
}

For example, insert rules into mqtt_acl collection:

db.mqtt_acl.insert({username: "test", publish: ["t/1", "t/2"], subscribe: ["user/%u", "client/%c"]})
db.mqtt_acl.insert({username: "admin", pubsub: ["#"]})

Configure acl_query and acl_nomatch in etc/plugins/emq_auth_mongo.conf:

## acl_query
auth.mongo.acl_query.collection = mqtt_user

auth.mongo.acl_query.selector = username=%u

HTTP Publish API¶

The EMQ broker provides a HTTP API to help application servers publish messages to MQTT clients.

HTTP API: POST https://host:8080/mqtt/publish

Web servers such as PHP, Java, Python, NodeJS and Ruby on Rails could use HTTP POST to publish MQTT messages to the broker:

curl -v --basic -u user:passwd -d "qos=1&retain=0&topic=/a/b/c&message=hello from http..." -k http://localhost:8080/mqtt/publish

Parameters of the HTTP API:

Name	Description
client	clientid
qos	QoS(0, 1, 2)
retain	Retain(0, 1)
topic	Topic
message	Payload

Note

The API uses HTTP Basic Authentication.

The url of this API has been changed to ‘api/v2/mqtt/publish’ in v2.3-beta.2 release. Read the doc in REST API.

MQTT Over WebSocket¶

Web browsers could connect to the emqttd broker directly by MQTT Over WebSocket.

WebSocket URI:	ws(s)://host:8083/mqtt
Sec-WebSocket-Protocol:	‘mqttv3.1’ or ‘mqttv3.1.1’

The Dashboard plugin provides a test page for WebSocket:

http://127.0.0.1:18083/websocket.html

Listener of WebSocket and HTTP Publish API is configured in etc/emq.config:

## MQTT/WebSocket Listener
listener.ws.external = 8083
listener.ws.external.acceptors = 4
listener.ws.external.max_clients = 64

$SYS Topics¶

The EMQ broker periodically publishes internal status, MQTT statistics, metrics and client online/offline status to $SYS/# topics.

For the EMQ broker could be clustered, the $SYS topic path is started with:

$SYS/brokers/${node}/

‘${node}’ is the erlang node name of emqttd broker. For example:

$SYS/brokers/emqttd@127.0.0.1/version

$SYS/brokers/emqttd@host2/uptime

Note

The broker only allows clients from localhost to subscribe $SYS topics by default.

Sys Interval of publishing $SYS messages, could be configured in etc/emqttd.config:

## System Interval of publishing broker $SYS Messages
mqtt.broker.sys_interval = 60

Broker Version, Uptime and Description¶

Topic	Description
$SYS/brokers	Broker nodes
$SYS/brokers/${node}/version	Broker Version
$SYS/brokers/${node}/uptime	Broker Uptime
$SYS/brokers/${node}/datetime	Broker DateTime
$SYS/brokers/${node}/sysdescr	Broker Description

Online/Offline Status of MQTT Client¶

The topic path started with: $SYS/brokers/${node}/clients/

Topic	Payload(JSON)	Description
${clientid}/connected	{ipaddress: “127.0.0.1”, username: “test”, session: false, version: 3, connack: 0, ts: 1432648482}	Publish when a client connected
${clientid}/disconnected	{reason: “keepalive_timeout”, username: “test”, ts: 1432749431}	Publish when a client disconnected

Properties of ‘connected’ Payload:

ipaddress: "127.0.0.1",
username:  "test",
session:   false,
protocol:  3,
connack:   0,
ts:        1432648482

Properties of ‘disconnected’ Payload:

reason: normal,
ts:     1432648486

Broker Statistics¶

Topic path started with: $SYS/brokers/${node}/stats/

Clients¶

Topic	Description
clients/count	Count of current connected clients
clients/max	Max number of cocurrent connected clients

Sessions¶

Topic	Description
sessions/count	Count of current sessions
sessions/max	Max number of sessions

Subscriptions¶

Topic	Description
subscriptions/count	Count of current subscriptions
subscriptions/max	Max number of subscriptions

Topics¶

Topic	Description
topics/count	Count of current topics
topics/max	Max number of topics

Broker Metrics¶

Topic path started with: $SYS/brokers/${node}/metrics/

Bytes Sent/Received¶

Topic	Description
bytes/received	MQTT Bytes Received since broker started
bytes/sent	MQTT Bytes Sent since the broker started

Packets Sent/Received¶

Topic	Description
packets/received	MQTT Packets received
packets/sent	MQTT Packets sent
packets/connect	MQTT CONNECT Packet received
packets/connack	MQTT CONNACK Packet sent
packets/publish/received	MQTT PUBLISH packets received
packets/publish/sent	MQTT PUBLISH packets sent
packets/subscribe	MQTT SUBSCRIBE Packets received
packets/suback	MQTT SUBACK packets sent
packets/unsubscribe	MQTT UNSUBSCRIBE Packets received
packets/unsuback	MQTT UNSUBACK Packets sent
packets/pingreq	MQTT PINGREQ packets received
packets/pingresp	MQTT PINGRESP Packets sent
packets/disconnect	MQTT DISCONNECT Packets received

Messages Sent/Received¶

Topic	Description
messages/received	Messages Received
messages/sent	Messages Sent
messages/retained	Messages Retained
messages/stored	TODO: Messages Stored
messages/dropped	Messages Dropped

Broker Alarms¶

Topic path started with: $SYS/brokers/${node}/alarms/

Topic	Description
${alarmId}/alert	New Alarm
${alarmId}/clear	Clear Alarm

Broker Sysmon¶

Topic path started with: ‘$SYS/brokers/${node}/sysmon/’

Topic	Description
long_gc	Long GC Warning
long_schedule	Long Schedule
large_heap	Large Heap Warning
busy_port	Busy Port Warning
busy_dist_port	Busy Dist Port

Trace¶

The emqttd broker supports to trace MQTT packets received/sent from/to a client, or trace MQTT messages published to a topic.

Trace a client:

./bin/emqttd_ctl trace client "clientid" "trace_clientid.log"

Trace a topic:

./bin/emqttd_ctl trace topic "topic" "trace_topic.log"

Lookup Traces:

./bin/emqttd_ctl trace list

Stop a Trace:

./bin/emqttd_ctl trace client "clientid" off

./bin/emqttd_ctl trace topic "topic" off

User Guide¶

Authentication¶

Allow Anonymous¶

Username/Password¶

ClientId¶

LDAP¶

HTTP¶

MySQL¶

PostgreSQL¶

Redis¶

MongoDB¶

ACL¶

Internal¶

HTTP API¶

MySQL¶

PostgreSQL¶

Redis¶

MongoDB¶

MQTT Publish/Subscribe¶

HTTP Publish API¶

MQTT Over WebSocket¶

$SYS Topics¶

Broker Version, Uptime and Description¶

Online/Offline Status of MQTT Client¶

Broker Statistics¶

Clients¶

Sessions¶

Subscriptions¶

Topics¶

Broker Metrics¶

Bytes Sent/Received¶

Packets Sent/Received¶

Messages Sent/Received¶

Broker Alarms¶

Broker Sysmon¶

Trace¶