MQTT and Websocket


I start to read some solution to build a “web interface” in order to display the data from the MQTT flow.

So I found at least two way to realize this web interface ( There are certainly more ;))

The first more complex in my point of view, create and use a database mySQL which will be fill by the MQTT flow. This database will be interrogates with php script or other, might be it’s possible with javascript, I don’t know I’m not an expert. The php script will build also the design of my web interface by adding css script…
Others installation are necessary as web server (apache / nginx) dns server (bind) and lot of configuration…

I hope I do not go far wrong in my explanations…

The second way that I started to look at is Websocket computer communication, this method seems to be really very attractive because from an implementation point of view it’s easier than the first method mentioned above, but certainly more limited but it reach fully my goal.

I must specify that it is for local use.

So if someone have any experience about Websocket and loraserver project I would actually be glad to hear him

Thank you & good W.E

The options you are mentioning aren’t exclusive at all. You can persist your data and then serve it through regular http (be it through templating, a json API, etc., doesn’t matter), allowing you to fetch and view the data you’ve collected so far in any given moment. Also, you may continously update and present the data you receive at the same location by means of websockets, which allow bidirectional communication and thus may push information to the client without needing a request. This can be done with bare websockets, or as you mention, with MQTT over websockets, in which case you subscribe to topics from a javascript MQTT client and update your view when messages are published to the relevant topics.

The thing is you may do both or any of them by themselves. Just as an example, at work we modified lora-app-server to persis any received messages already decoded to the same Postgres DB lora-app-server works on, and also modified the MQTT message that gets published on data up and then subscribe from the React frontend to relevant topics, so we are able to check historical data and also see live data coming in.

From lora-app-server’s implementation it is pretty straight forward to do this things, though there are some implementation details. One of those is that if you are using mosquitto and going with MQTT over websockets for live data, you need to build it with websockets support, which is usually (or maybe always) disabled when installing from packages. Other than that, some googling and forum searches should get you there in little time.

Good luck!

1 Like

Hello here,

Thank you @iegomez good point of view and advice.

To be more precise we are building an application which monitor the occupancy of the meeting rooms.

Here are the highlights of our project:

First step give real-time state of a meeting room (free or busy for example with color code through Web Interface).
Second step, analyze the occupancy rate over a period of time (one year for example).

This completely fulfills the conditions of the first step mentioned above.

This completely fulfills the conditions of the second step mentioned above.

This exactly corresponds to what we want to do! It’s official Ignacio you’r my new best friend :wink:

If I understood correctly we have to modify lora-app-server to persist all received message already decoded and then apply a request on Postgres DB to fetch all data to examine so don’t need to install a new database (mySQL) ?

Regarding MQTT over websockets we have to reinstall Mosquitto broker with websoquets support then we have to install a javascript mqtt.client as paho.mqtt.client which works in a web browser?

I found several tutorial to do it ==> - -

Could you give some example or way to modify lora-app-server in order to be compliant with my purpose.

Thank you for your help.

That’s a way of doing and the one we went with, but it’s not really necessary. You may create a whole new application that gets the message either from the http integration or by subscribing to the broker, without ever needing to touch lora-app-server’s source code. You may even use lora-app-server’s encoding/decoding features so that you get the real decoded data at your end as to only handle storing and serving data. Also, you could interact with lora-app-server through its API and never even bother with the included frontend. There are lot of ways achieve what you are looking for.

In my mosquitto auth plugin I wrote some instructions for building mosquitto, including websockets support. Check it here.

Sure, here’s the function that handles a message and stores it at our DB. Have in mind that it deals with a whole lot more than just storing a simple message (it checks different data types, a preamble byte for our custom encoding, fires alerts depending on the data, etc.), but it could give you an idea of what you could end up doing when customizing lora-app-server.

//CreateRxMessage stores a received device message.
//It creates an rx_mesage and all children data.
//Return an error and a stuct containing a message to be published to the client to render on the front-end.
func CreateRxMessage(db sqlx.Ext, spClient *sp.Client, rxMessage *RxMessage) (ClientData, error) {

	now := time.Now()

	err := sqlx.Get(db, &rxMessage.ID, `
		insert into rx_message (
		) values ($1, $2, $3, $4, $5, $6, $7) returning id`,

	if err != nil {
		return ClientData{}, InsertionError(err)

	//Get device
	device, devErr := GetDevice(db, rxMessage.DevEUI)
	if devErr != nil {
		return ClientData{}, devErr

	* Now create message data and a ClientData to be sent.
	clientData := ClientData{
		ReceivedAt: now,
		DevEUI:     rxMessage.DevEUI,
		RSSI:       rxMessage.Rssi,
		SNR:        rxMessage.Snr,

	//Try to retrieve gateway (may not exist).
	gateway, gErr := GetGateway(db, rxMessage.GatewayMAC, false)
	if gErr != nil {
		clientData.GatewayMAC = "NA"
		clientData.GatewayName = "NA"
	} else {
		clientData.GatewayMAC = gateway.MAC.String()
		clientData.GatewayName = gateway.Name

	//Decode base64 message
	decodedData, decodeErr := base64.StdEncoding.DecodeString(rxMessage.Message)

	if decodeErr != nil {
		return ClientData{}, errors.Wrap(decodeErr, "decode message error")

	//Extract preamble from decodedData.
	preambleByte := decodedData[0]
	log.Infof("Read preamble byte: %v", preambleByte)

	// TEST //

	//Assume there's fix, no panic and dataGroup is 0

	gpsFix := true        //(preambleByte & 128) != 0
	devicePanic := false  //(preambleByte & 64) != 0
	dataGroup := int32(0) //int32(preambleByte & 0x0F)

	log.Infof("Preamble: fix is %t, panic is %t and group is %d.", gpsFix, devicePanic, dataGroup)

	//Adjust decodedData

	// TEST //

	//Assume there's no preamble byte and group is 0
	decodedData = decodedData[:]

	* Check if message length equals expected length.
	* If not, discard the message and send an error.

	//Get data bytes count.
	dBytes, dbErr := DataBytes(db, rxMessage.DevEUI, dataGroup)
	if dbErr != nil {
		return ClientData{}, errors.Wrapf(dbErr, "couldn't get bytes amount")

	if len(decodedData) != dBytes {
		return ClientData{}, errors.New(fmt.Sprintf("message has a different number of bytes than expected - got %d but wanted %d", len(decodedData), dBytes))

	//Loop through data descriptors to save data.
	//Check if it has location in order to treat them separately.
	dataDescriptors, ndtErr := GetDataDescriptorsByGroup(db, rxMessage.DevEUI, dataGroup)

	if ndtErr != nil {
		return ClientData{}, errors.Wrap(ndtErr, "GetDataDescriptors error")

	hasLocation, locationErr := HasGroupLocation(db, rxMessage.DevEUI, dataGroup)

	if locationErr != nil {
		log.WithField("locError", locationErr).Info("Error")
		return ClientData{}, errors.Wrap(locationErr, "has_location error")

	//Create locationDatum in case the device has location.
	locationDatum := LocationDatum{
		Lat:         0.0,
		Lng:         0.0,
		RxMessageID: rxMessage.ID,

	var arrayIndex int32

	arrayIndex = 0

	* Make a map to hold values.

	dataValues := make(map[string]interface{})
	dataAlarms := make(map[string]AlarmMap)

	for _, dataDescriptor := range dataDescriptors {

		//Get the underlying data type
		dataType, dtErr := GetDataTypeForDataDescriptor(db, dataDescriptor.ID)

		if dtErr != nil {
			log.WithField("dtErr", dtErr).Info("dtErr")
			return ClientData{}, errors.Wrap(dtErr, "GetDataTypeForDataDescriptor error")

		//Move the arrayIndex by ndt's left offset and get the subslice.
		arrayIndex += dataDescriptor.LeftOffset
		dataArray := decodedData[arrayIndex:(arrayIndex + dataType.NumBytes)]

		log.WithField("dataType", dataType).Info("Data type")

		if dataType.ValueType == "integer" {

			//Try to convert the subslice to an int.
			intValue, intErr := ReadIntData(dataType, dataArray)

			if intErr != nil {
				log.WithField("Err", intErr).Info("IntErr")
				return ClientData{}, errors.Wrap(intErr, "ReadIntData error")

			intDatum := IntDatum{
				DataDescriptorID: dataDescriptor.ID,
				Value:            intValue,
				RxMessageID:      rxMessage.ID,

			indErr := CreateIntDatum(db, &intDatum)

			if indErr != nil {
				log.WithField("indErr", indErr).Info("indErr")
				return ClientData{}, errors.Wrap(indErr, "int_datum creation error")

			hasDataAlarm, showDataAlarm, daErr := CheckDataAlarm(db, spClient, dataDescriptor.ID, rxMessage.ID, float64(intValue), rxMessage.CreatedAt)
			if daErr != nil {
				log.Errorf("error on check data alarm %v", daErr)
			dataValues[dataDescriptor.Name] = intDatum
			//Assign alarm if there was one
			alarmMessage := ""
			alarmType := "none"
			if showDataAlarm {
				if hasDataAlarm {
					alarmMessage = fmt.Sprintf("Alert: received %s with value %d for device %s (EUI: %s).", dataDescriptor.Name, intValue, device.Name, device.DevEUI.String())
					alarmType = "on"
				} else {
					alarmMessage = fmt.Sprintf("Alarm defused for device %s (EUI: %s): received %s with value %d.", device.Name, device.DevEUI.String(), dataDescriptor.Name, intValue)
					alarmType = "off"
			dataAlarms[dataDescriptor.Name] = AlarmMap{
				Triggered: hasDataAlarm,
				Show:      showDataAlarm,
				Message:   alarmMessage,
				Type:      alarmType,

		} else if dataType.ValueType == "float" {

			floatValue, floatErr := ReadFloatData(dataType, dataArray)

			if floatErr != nil {
				log.WithField("floatErr", floatErr).Info("floatErr")
				return ClientData{}, errors.Wrap(floatErr, "ReadFloatData error")

			floatDatum := FloatDatum{
				DataDescriptorID: dataDescriptor.ID,
				Value:            floatValue,
				RxMessageID:      rxMessage.ID,

			fndErr := CreateFloatDatum(db, &floatDatum)

			if fndErr != nil {
				log.WithField("fndErr", fndErr).Info("fndErr")
				return ClientData{}, errors.Wrap(fndErr, "float_datum creation error")

			if hasLocation && (dataType.Name == latConst || dataType.Name == lngConst) {
				if dataType.Name == latConst {
					locationDatum.Lat = floatValue
				} else {
					locationDatum.Lng = floatValue
			} else {
				hasDataAlarm, showDataAlarm, daErr := CheckDataAlarm(db, spClient, dataDescriptor.ID, rxMessage.ID, float64(floatValue), rxMessage.CreatedAt)
				if daErr != nil {
					log.Errorf("error on check data alarm %v", daErr)
				dataValues[dataDescriptor.Name] = floatDatum
				//Assign alarm if there was one
				alarmMessage := ""
				alarmType := "none"
				if showDataAlarm {
					if hasDataAlarm {
						alarmMessage = fmt.Sprintf("Alert: received %s with value %d for device %s (EUI: %s).", dataDescriptor.Name, floatValue, device.Name, device.DevEUI.String())
						alarmType = "on"
					} else {
						alarmMessage = fmt.Sprintf("Alarm defused for device %s (EUI: %s): received %s with value %d.", device.Name, device.DevEUI.String(), dataDescriptor.Name, floatValue)
						alarmType = "off"
				dataAlarms[dataDescriptor.Name] = AlarmMap{
					Triggered: hasDataAlarm,
					Show:      showDataAlarm,
					Message:   alarmMessage,
					Type:      alarmType,

		} else if dataType.ValueType == "string" {

			strValue, strErr := ReadStringData(dataType, dataArray)

			if strErr != nil {
				log.WithField("strErr", strErr).Info("strErr")
				return ClientData{}, errors.Wrap(strErr, "ReadStringData error")

			stringDatum := StringDatum{
				DataDescriptorID: dataDescriptor.ID,
				Value:            strValue,
				RxMessageID:      rxMessage.ID,

			sndErr := CreateStringDatum(db, &stringDatum)

			if sndErr != nil {
				log.WithField("sndErr", sndErr).Info("sndErr")
				return ClientData{}, errors.Wrap(sndErr, "string_datum creation error")

			dataValues[dataDescriptor.Name] = stringDatum

		} else {
			log.WithField("WTF", "AAHHH").Info("Weird")

		//Move arrayIndex according to num bytes and right offset.
		arrayIndex += dataType.NumBytes + dataDescriptor.RightOffset


	//Create locationDatum
	if hasLocation {
		locAlarm, showLocAlarm, locErr := CreateLocationDatum(db, spClient, &locationDatum, rxMessage.DevEUI)
		if locErr != nil {
			log.WithField("locErr", locErr).Info("locErr")
			return ClientData{}, errors.Wrap(locErr, "location_datum creation error")
		/*locationMap := make(map[string]float64)
		locationMap["Lat"] = locationDatum.Lat
		locationMap["Lng"] = locationDatum.Lng
		jsonLocation, _ := json.Marshal(locationMap)*/
		dataValues["Location"] = locationDatum
		alarmMessage := ""
		alarmType := "none"

		if showLocAlarm {
			if locAlarm {
				alarmMessage = fmt.Sprintf("Alert: a location alarm was triggered for device %s (EUI: %s).", device.Name, device.DevEUI.String())
				alarmType = "on"
			} else {
				alarmMessage = fmt.Sprintf("A location alarm has stopped for device %s (EUI: %s).", device.Name, device.DevEUI.String())
				alarmType = "off"

		dataAlarms["Location"] = AlarmMap{
			Triggered: locAlarm,
			Show:      showLocAlarm,
			Message:   alarmMessage,
			Type:      alarmType,

	* Done creating data.

	clientData.Data = dataValues
	clientData.Alarms = dataAlarms

	rxMessage.CreatedAt = now
	rxMessage.UpdatedAt = now

	//Check for timeAlarm
	timeAlarm, naErr := GetTimeAlarmForDevice(db, rxMessage.DevEUI)
	if naErr == nil {
		if timeAlarm.AlarmSent {
			err := SendTimeAlarmDefuse(db, spClient, &timeAlarm, rxMessage.CreatedAt)
			if err != nil {
				log.Errorf("Error on rx_message creation on alarm defuse attempt: %v", err)

		"id":     rxMessage.ID,
		"devEUI": rxMessage.DevEUI[:],
	}).Info("rx_message created")

	return clientData, nil

1 Like

Thank you for your reply @iegomez.

I 'm looking your “Mosquitto auth plugin” project with attention and I will try to implement it rapidly.

@iegomez could you tell me plz what is the difference between libwebsockets3 and libwebsockets8, I can install only libwebsockets3 with debian 8.5 jessie ???

In second part I will look your script with attention.

On another side I need to improve my understanding about HTTP integration I still have not understood everything and how it works


Here my first issue (during this step ==> Then build mosquitto, add a mosquitto user and set ownership for /var/log/mosquitto and /var/lib/mosquitto/ (default log and persistence locations).:

lora@Debian02:~$ cd mosquitto-1.4.15/
lora@Debian02:~/mosquitto-1.4.15$ make
set -e; for d in lib client src; do make -C ${d}; done
make[1]: Entering directory ‘/home/lora/mosquitto-1.4.15/lib’
cc -Wall -ggdb -O2 -I. -I… -I…/lib -fPIC -DWITH_TLS -DWITH_TLS_PSK -DWITH_THREADING -DWITH_SOCKS -DWITH_SRV -c mosquitto.c -o mosquitto.o
In file included from mosquitto.c:33:0:
./mosquitto_internal.h:27:27: fatal error: openssl/ssl.h: No such file or directory
#include <openssl/ssl.h>
compilation terminated.
Makefile:51: recipe for target ‘mosquitto.o’ failed
make[1]: *** [mosquitto.o] Error 1
make[1]: Leaving directory ‘/home/lora/mosquitto-1.4.15/lib’
Makefile:21: recipe for target ‘mosquitto’ failed
make: *** [mosquitto] Error 2

it seems missing ssl.h files, however i have not had any error during openssl installation.

Thank you

You are probably missing a dev package, but you can build openssl yourself to be sure (remove the current openssl installation before):

git clone git://
cd openssl
make test
sudo make install

After that, the header will be available. Note that for websockets it’s the same story, you can build it like this:

sudo apt-get install cmake
git clone
cd libwebsockets
mkdir build
cd build
cmake ..
make install
1 Like

Hello Ignacio,

I 'm doing “Build” paragraph, when you say “Modify, setting websockets support” it’s:

#Build with websockets support on the broker.

then “set ownership for /var/log/mosquitto and /var/lib/mosquitto/ (default log and persistence locations).”

I have to create mosquitto directory to /var/log/ & /var/lib/ path because the directory mosquitto doesn’t exist?
Would it have to be created alone, something is wrong?

See log:
chown: cannot access ‘/var/log/mosquitto/’: No such file or directory

Thank you

After building mosquitto (i.e., make and sudo make install), you should create the user and then chown the dirs. If for some reason they weren’t created, just create them yourself and chown them later. Those are the default paths for mosquitto’s logging and persistence that will appear at /etc/mosquitto/mosquitto.conf, though you can always modify mosquitto options as you wish (see for configuration options).

Hello @iegomez and all,

I continued to implement mosquitto-go-auth but I’m little lost, as well I did:
sudo chown -R mosquitto:mosquitto /var/log/mosquitto/ sudo chown -R mosquitto:mosquitto /var/lib/mosquitto/

it’s ok then I create the file /etc/systemd/system/mosquitto.service as recommended and I added all annotations required (I didn’t yet undesrtand what is the aim of this service…)

As I had not installed go yet, I installed the version linux / amd64 go1.10.1 and hereinafter the setting of go environment variables:

GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build030793080=/tmp/go-build -gno-record-gcc-switches"

Obviously I cloned mosquitto-go-auth under:

Then from this directory when I’m trying to run the commands to generate go-auth.h and then

I got some log error:

lora@Debian02:~/go/src/$ go build -buildmode=c-archive go-auth.go
backends/jwt.go:20:2: cannot find package "" in any of:
        /usr/local/go/src/ (from $GOROOT)
        /home/lora/go/src/ (from $GOPATH)
go-auth.go:17:2: cannot find package "" in any of:
        /usr/local/go/src/ (from $GOROOT)
        /home/lora/go/src/ (from $GOPATH)
backends/mysql.go:16:2: cannot find package "" in any of:
        /usr/local/go/src/ (from $GOROOT)
        /home/lora/go/src/ (from $GOPATH)
common/utils.go:18:2: cannot find package "" in any of:
        /usr/local/go/src/ (from $GOROOT)
        /home/lora/go/src/ (from $GOPATH)
backends/postgres.go:11:2: cannot find package "" in any of:
        /usr/local/go/src/ (from $GOROOT)
        /home/lora/go/src/ (from $GOPATH)
backends/sqlite.go:10:2: cannot find package "" in any of:
        /usr/local/go/src/ (from $GOROOT)
        /home/lora/go/src/ (from $GOPATH)
common/utils.go:15:2: cannot find package "" in any of:
        /usr/local/go/src/ (from $GOROOT)
        /home/lora/go/src/ (from $GOPATH)
backends/files.go:9:2: cannot find package "" in any of:
        /usr/local/go/src/ (from $GOROOT)
        /home/lora/go/src/ (from $GOPATH)
common/utils.go:16:2: cannot find package "" in any of:
        /usr/local/go/src/ (from $GOROOT)
        /home/lora/go/src/ (from $GOPATH)
backends/mongo.go:14:2: cannot find package "" in any of:
        /usr/local/go/src/ (from $GOROOT)
        /home/lora/go/src/ (from $GOPATH)
backends/mongo.go:15:2: cannot find package "" in any of:
        /usr/local/go/src/ (from $GOROOT)
        /home/lora/go/src/ (from $GOPATH)

I have again lot of questions regarding but I prefer split in several part.

thank you

Hi, Julien.

You need to get all required packages before building the plugin. Just run make requirements to get them, and then run make (it’ll run the go builds for you).

@iegomez thank you for your fast answer, I did it, however I got:

lora@Debian02:~/go/src/$ sudo make requirements
Installing development tools
make: go: Command not found
Makefile:7: recipe for target 'requirements' failed
make: *** [requirements] Error 127

Sorry I’m linux beginner particularly with compilation method

That’s saying that Go isn’t installed. If you already installed it and exported Go path, maybe you only need to source .profile to take effect (you may run go version -which should output your version- later to confirm it works):

source ~/.profile
go version

If that works, then just make requirements, no sudo needed (same for make later).

EDIT: Have in mind that you don’t need to use my plugin for mosquitto to work, you may go with plain files auth from mosquitto, use jpmens’ plugin or roll your own. Of course, if you choose to use mine, I’ll be happy to help you configure it. Consider also that these plugins are for authentication and authorization and are not needed for mosquitto to work at all, so you could test that mosquitto is working fine by allowing anonymous users and skipping auth to make sure that any error isn’t caused by mosquitto itself and later apply some auth method.

hello @iegomez one more time thank you for your answer.

the command go version works:

lora@Debian02:~$ go version
go version go1.10.1 linux/amd64


# /etc/profile: system-wide .profile file for the Bourne shell (sh(1))
# and Bourne compatible shells (bash(1), ksh(1), ash(1), ...).

if [ "`id -u`" -eq 0 ]; then
export PATH

This is same way to config etc/profile instead $HOME/.profile ???

I have a doubt about the location of my Go workspace …

Thank you for this focus, when I would have finished with the build part of your project I will switch on brocaar’s loraserver project in order to make sure that mosquitto works. Then I will apply (with your help :slight_smile: ) mosquitto-go-auth so that to improve the safety of the network.

make requirements will install some packages use in Brocaar’s loraserver project such as postgres, can I find easily which ones?

Thank you

What you mentioned should work as a system-wide installation. If you only want Go for your user, you may add this to $HOME/profile¨

export PATH=$PATH:/usr/local/go/bin

If go version works, then make requirements from the same terminal should too.

It won’t install Postgres, Redis (they have to be installed by you), or anything like that, but it will install some Go packages to handle those and also for testing, logging, etc. Some are shared with loraserver, others not. You mays check them here.