MQTT Integration and Provisioning

ChirpStack (v4) Integrations
1

Hello,

I’m writing to get some clarification on Chirpstack’s MQTT Integration feature. I have set up TLS and can generate certificates by navigating to Tenant>MyTenant>Applications>MyApplication, then clicking Integrations and ‘Generate Certificate’ - I see the CA and TLS certs as well as the TLS key. However, I’m curious if there’s a way to provision data from this MQTT integration. For context, we’re running docker-compose on an Azure server and are hoping to integrate it with a website.

We’re hoping to have data separated by tenants, or at least by application, to keep things fairly separated. However after configuring a MQTT node in Node-RED and adding a debug node, I can see all traffic on the MQTT ports 1883 and 8883 (running Mosquitto with normal and TLS functions).

As I understand it, Chirpstack isn’t able to partition this data based on tenant or application, and just passes the LoRaWAN data to the Mosquitto ports. From what I’ve read, the next best option is to create an ACL file in Mosquitto or use Node-RED to filter by tenants or applications.

Is there a function in Chirpstack to keep the data separate for integration, or will we need to proceed with Mosquitto/Node-RED/other configuration?

Thank you!

1 Like
2

If you did the TLS guide and mosquitto_sub from a separate host, you should only see the data associated to that application. This is due to the line in the ACL:

pattern readwrite application/%u/#

Perhaps this was on the same machine? Still shouldn’t happen using 8883 though, maybe share your mosquitto.conf.

2 Likes
3

Hi Liam,

Thank you so much for your response - that’s a great catch as I only checked data flow from the host machine. I tried the mosquitto_sub example in the guide (on the host) and got the following error:

/ # mosquitto_sub  -h localhost -t "#" -v -d
Client null sending CONNECT
Client null received CONNACK (5)
Connection error: Connection Refused: not authorised.
Client null sending DISCONNECT

I’ll try it from a separate machine and see how that fares.

I’ve attached some configuration files for reference/troubleshooting:

/etc/mosquitto/conf.d/listeners.conf

per_listener_settings true

listener 1883 127.0.0.1
allow_anonymous true

listener 8883 0.0.0.0
cafile /etc/mosquitto/certs/ca.pem
certfile /etc/mosquitto/certs/mqtt-server.pem
keyfile /etc/mosquitto/certs/mqtt-server-key.pem
allow_anonymous false
require_certificate true
use_identity_as_username true
acl_file /etc/mosquitto/acl

/etc/mosquitto/acl

pattern readwrite +/gateway/%u/#
pattern readwrite application/%u/#

~/chirpstack-docker/configuration/mosquitto/config/mosquitto.conf

persistence true 
persistence_location /mosquitto/data/ 
log_dest file /mosquitto/log/mosquitto.log 
log_dest stdout
password_file /mosquitto/config/passwd_file

allow_anonymous false
listener 1883 0.0.0.0

listener 8883 0.0.0.0
cafile /etc/mosquitto/certs/ca.pem
certfile /etc/mosquitto/certs/mqtt-server.pem
keyfile /etc/mosquitto/certs/mqtt-server-key.pem

And some of the docker-compose file:

services:
  chirpstack:
    image: chirpstack/chirpstack:4
    command: -c /etc/chirpstack
    restart: unless-stopped
    volumes:
      - ./configuration/chirpstack:/etc/chirpstack
      - ./lorawan-devices:/opt/lorawan-devices
      - /etc/chirpstack/certs:/etc/chirpstack/certs:ro
    depends_on:
      - postgres
      - mosquitto
      - redis
    environment:
      - MQTT_BROKER_HOST=mosquitto
      - REDIS_HOST=redis
      - POSTGRESQL_HOST=postgres
    ports:
      - 8080:8080
....
  mosquitto:
    image: eclipse-mosquitto:2
    restart: unless-stopped
    ports:
      - 1883:1883
      - 8883:8883
    volumes: 
      - ./configuration/mosquitto/config/:/mosquitto/config/
      - ./configuration/mosquitto/config/passwd_file:/mosquitto/config/passwd_file
      - ./configuration/mosquitto/data:/mosquitto/data
      - ./configuration/mosquitto/log:/mosquitto/log
      - ./configuration/mosquitto/config/mosquitto.conf:/mosquitto/config/mosquitto.conf
      - /etc/mosquitto/certs:/etc/mosquitto/certs
      - /etc/mosquitto/conf.d/listeners.conf:/etc/mosquitto/conf.d/listeners.conf
      - /etc/mosquitto/acl:/etc/mosquitto/acl
1 Like
4

If you are using the TLS described in the guide (looks like you are) any external connections will have to use the certificates generated through the WebUI to connect to the broker (and it will only show the application info associated to the application you generated the certificate in). So you will have to get those certs and reference them in the mosquitto sub to connect.

I am confused by a few of your files though. Why is there two references to mosquitto.conf and listeners.conf? Only ~/chirpstack-docker/configuration/mosquitto/config/mosquitto.conf is required, and the mosquitto.conf you shared has some issues. this might be causing some strange behaviour like being able to connect to 8883 locally, as your mosquitto.conf does not have the per_listener_settings true. Maybe you got confused as the guide is for Ubuntu not docker but it seems like you took both routes.

I’d suggest copy-pasting your current /etc/mosquitto/conf.d/listeners.conf into the ~/chirpstack-docker/configuration/mosquitto/config/mosquitto.conf, then removing any mounting of the /etc files in your docker-compose, and move your certs + acl to ~/chirpstack-docker/configuration/mosquitto on host. That way, all of your configuration will be in the docker-compose file instead of having some scattered to /etc.

Another aside is that you do not need to mount each directory in the /configuration/mosquitto folder, you could mount the whole directory with just

volumes: 
      - ./configuration/mosquitto:/mosquitto
1 Like
5

Great catches - yes I tried following the TLS guide as instructed, and certificates generated in the WebUI just fine. I was able to save those to local files and will use those for securing a connection from a separate computer. What you described is a perfect scenario for how we’re hoping to have our data provisioned.

Yeah, it’s definitely been a learning process for me. I appreciate all of the help to simplify and remove duplicates! I got confused with the Ubuntu guides vs Docker guides & the configurations on the local machine vs what’s inside the containers & volumes. I’ve hesitated cleaning up because I was afraid of breaking things. It’s been a hodge-podge of work, so I’ll definitely make the corrections you suggested & will only use the mosquitto.conf file.

Beautiful simplifications and asides are much appreciated - I’ll make those changes and update as I progress. Many thanks!

6

:+1:

Ya theres definitely some reading between the lines required in a lot of the guides. And host vs container is always a struggle when starting docker. I was in a similar position just a few months ago.

Happy to help if you encounter other issues.

7

Glad I’m not alone on that one, it’s gotten easier with time and exposure though.

Your recommendation worked like a charm - I was able to subscribe on the host computer and read all MQTT events. I set up Mosquitto on my personal computer, and using the certs & key from Chirpstack, was able to see only traffic from devices belonging to that application. It works perfectly!

sudo docker exec -it chirpstack-docker_mosquitto_1 sh
/ # mosquitto_sub  -h localhost -t "#" -v -d
Client null sending CONNECT
Client null received CONNACK (0)
Client null sending SUBSCRIBE (Mid: 1, Topic: #, QoS: 0, Options: 0x00)
Client null received SUBACK
Subscribed (mid: 1): 0
Client null received PUBLISH (d0, q0, r1, m0, 'us915_1/gateway/2cf7f11153100124/state/conn', ... (20 bytes))
us915_1/gateway/2cf7f11153100124/state/conn �2cf7f11153100124
Client null received PUBLISH (d0, q0, r1, m0, 'us915_1/gateway/2cf7f111546000e5/state/conn', ... (20 bytes))
us915_1/gateway/2cf7f111546000e5/state/conn �2cf7f111546000e5
Client null received PUBLISH (d0, q0, r0, m0, 'us915_1/gateway/2cf7f11153100124/event/stats', ... (27 bytes))
us915_1/gateway/2cf7f11153100124/event/stats��ݺ�2cf7f11153100124
Client null received PUBLISH (d0, q0, r0, m0, 'us915_1/gateway/2cf7f111546000e5/event/up', ... (94 bytes))
us915_1/gateway/2cf7f111546000e5/event/up 
�6G"��y�]?ސ����
��
(*1
2cf7f111546000e5��0���������=33#A@jƇ���
Client null received PUBLISH (d0, q0, r0, m0, 'application/188a3b08-965d-4de8-8348-0e4bad6ac368/device/2cf7f120526014e0/event/up', ... (1377 bytes))
application/188a3b08-965d-4de8-8348-0e4bad6ac368/device/2cf7f120526014e0/event/up {"deduplicationId":"2fa91952-7f85-4b98-8cc6-2dc82dcb8802","time":"2024-12-09T22:33:13.959954246+00:00","deviceInfo":{"tenantId":"52f14cd4-c6f1-4fbd-8f87-4025e1d49242",

Thank you so very much for your suggestions and guidance.

8

One more question if you have the chance: do you know if there’s a way to generate an MQTT certificate for all devices from all tenants and applications? Or some other way to get into the 1883 and/or 8883 ports to see all data flowing for use with another software for integration?

1 Like
9

Interesting question, I am certain there is but I am not experienced enough in mosquitto ACLs to know the cleanest way to do that.

Off the top of my head one way you could implement this is to create another MQTT listener in your mosquitto.conf (on port 8882 for example) that is a copy/paste of your 8883 listener but without the acl_file line, and with different certificates so clients cant connect to it with the certificates they use for 8883. You would have to generate those certificates yourself (pretty easy) but then if you use those certificates to connect to that port you would be able to see all the topics as the ACL is not restricting it.

1 Like
10

Great solution! I’ll try that out to keep the general traffic locked down. Many thanks for all your help!

1 Like