I read now a lot of documentation about the MQTT integration, but somehow I don’t understand how I have to configure it for my case. I am using the Docker Image of Chirpstack on my local server. I want to send the application data to an external MQTT server outside of my local network. The host of the MQTT server provided me the following certificate files: ca_certificate.pem client_server-main_certificate.pem client_server-main_key.pem. I also have user and password from the MQTT server and the URL and port.
I tried to incorporate everything into the chirpstack.toml file as follows:
I only added the part of the file which is different to the original in the chirpstack docker container.
[integration]
enabled=["mqtt"]
[integration.mqtt]
# Event topic template.
event_topic="application/{{application_id}}/device/{{dev_eui}}/event/{{event}}"
# Command topic.
#
# This is the topic on which the MQTT subscribes for receiving (enqueue) commands.
command_topic="application/{{application_id}}/device/{{dev_eui}}/command/{{command}}
# Use JSON encoding instead of Protobuf (binary).
json=true
# MQTT server (e.g. scheme://host:port where scheme is tcp, ssl or ws)
server="ssl://mqtt-server-address.de:8883"
# Connect with the given username (optional)
username="username_mqtt_server"
# Connect with the given password (optional)
password="password_mqtt_server"
# Quality of service level
#
# 0: at most once
# 1: at least once
# 2: exactly once
#
# Note: an increase of this value will decrease the performance.
# For more information: https://www.hivemq.com/blog/mqtt-essentials-part-6-mqtt-quality-of-service-levels
qos=0
# Clean session
#
# Set the "clean session" flag in the connect message when this client
# connects to an MQTT broker. By setting this flag you are indicating
# that no messages saved by the broker for this client should be delivered.
clean_session=false
# Client ID
#
# Set the client id to be used by this client when connecting to the MQTT
# broker. A client id must be no longer than 23 characters. If left blank,
# a random id will be generated by ChirpStack.
client_id=""
# Keep alive interval.
#
# This defines the maximum time that that should pass without communication
# between the client and server.
keep_alive_interval="30s"
# CA certificate file (optional)
#
# Use this when setting up a secure connection (when server uses ssl://...)
# but the certificate used by the server is not trusted by any CA certificate
# on the server (e.g. when self generated).
ca_cert="configuration/certs/ca_certificate.pem"
# TLS certificate file (optional)
tls_cert="configuration/certs/client_server-main_certificate.pem"
# TLS key file (optional)
tls_key="configuration/certs/client_server-main_key.pem"
I get the error that the connect to the external MQTT server did not work. Do you know why? Since you can enter the MQTT settings in so many files, I don’t really know where its relevant for my usecase. I only adapted the chirpstack.toml file. The other files are kept with default values.
All guides so far confuse me, because they generate the MQTT certificates themself and I got them from the external MQTT server provider.
Thanks in advance.
PS: This is the current error message. But I think it is quiet generic.
chirpstack-docker-chirpstack-1 | 2023-03-18T15:08:01.585067Z INFO chirpstack::cmd::root: Starting ChirpStack LoRaWAN Network Server version="4.2.0" docs="https://www.chirpstack.io/"
chirpstack-docker-chirpstack-1 | 2023-03-18T15:08:01.585139Z INFO chirpstack::region: Setting up regions
chirpstack-docker-chirpstack-1 | 2023-03-18T15:08:01.586266Z INFO setup{common_name=EU868 region_id=eu868}: chirpstack::region: Configuring region
chirpstack-docker-chirpstack-1 | 2023-03-18T15:08:01.589982Z INFO chirpstack::backend::joinserver: Setting up Join Server clients
chirpstack-docker-chirpstack-1 | 2023-03-18T15:08:01.590091Z INFO chirpstack::backend::roaming: Setting up roaming clients
chirpstack-docker-chirpstack-1 | 2023-03-18T15:08:01.590176Z INFO chirpstack::adr: Setting up adr algorithms
chirpstack-docker-chirpstack-1 | 2023-03-18T15:08:01.590293Z INFO chirpstack::integration: Setting up global integrations
chirpstack-docker-chirpstack-1 | 2023-03-18T15:08:01.590382Z INFO chirpstack::integration::redis: Initializing Redis integration
chirpstack-docker-chirpstack-1 | 2023-03-18T15:08:01.590474Z INFO chirpstack::integration::mqtt: Initializing MQTT integration
chirpstack-docker-chirpstack-1 | 2023-03-18T15:08:01.592732Z INFO chirpstack::integration::mqtt: Configuring connection with TLS certificate ca_cert=configuration/certs/ca_certificate.pem tls_cert=configuration/certs/client_server-main_certificate.pem tls_key=configuration/certs/client_server-main_key.pem
chirpstack-docker-chirpstack-1 | 2023-03-18T15:08:01.597374Z INFO chirpstack::integration::mqtt: Connecting to MQTT broker server_uri=tcp://mqtt-server-address.de:8883 client_id=7865f996aa4cdf0e clean_session=false
chirpstack-docker-chirpstack-gateway-bridge-eu868-1 | time="2023-03-18T15:08:02.869585042Z" level=info msg="integration/mqtt: publishing event" event=stats qos=0 topic=eu868/gateway/a84041ffff22943c/event/stats
chirpstack-docker-chirpstack-1 | Error: Setup MQTT integration
chirpstack-docker-chirpstack-1 |
chirpstack-docker-chirpstack-1 | Caused by:
chirpstack-docker-chirpstack-1 | 0: Connect to MQTT broker
chirpstack-docker-chirpstack-1 | 1: [-1] TCP connect completion failure
Just to make sure that MQTT works in general with external mqtt servers I tried it with a public mqtt server test.mosquitto.org and I received payload messages on the topic
application/applID/device/devID/event/#
But when I change the server URL to my external non-public mqtt server (not on my localhost) including tls certificates, chirpstack keeps crashing due to the error in my first post.
Can anybody give me a hint why my config is not working? I can’t use the tutorials because the explain how to connect an external mqtt server to the chirpstack server (available online). My chirpstack server is only available on localhost.
which is a stupid error, because the first two are ignored, so my configuration changes were not applied.
I managed to get it to work, the process does not exit with the error message and I get this in the log
2023-03-23T21:28:48.849849Z INFO chirpstack::integration::mqtt: Configuring connection with TLS certificate ca_cert=/opt/CA/mondas.crt tls_cert=/opt/CA/chirpstack.crt tls_key=/opt/CA/chirpstack.key
2023-03-23T21:28:48.866840Z INFO chirpstack::integration::mqtt: Connecting to MQTT broker server_uri=ssl://10.2.0.2:8883/ client_id=5d4b388415ec83cd clean_session=false
2023-03-23T21:28:49.018712Z INFO chirpstack::gateway::backend: Setting up gateway backends for the different regions
This is what I did:
I made sure to add SAN to my MQTT broker certificate, I created a file fd.ext
(the 10.2.0.0/16 network is a private network that my VMs are connected to on my cloud provider) and then I executed openssl x509 -req -in broker.csr -CA ca.crt -CAkey ca.key -CAcreateserial -out broker.crt -days 10000 -extfile fd.ext to created the broker certificate.
made sure that the host is able to connect with the external MQTT broker. I did:
I created the directory configuration/CA and put the files in there and changed the owner:
cd configuration/CA
chown 65534 private_key.key
since the chirpstack image runs as the nobody user with the uid 65534.
I changed configuration/chirpstack/chirpstack.toml to
[integration]
enabled=["mqtt"]
[integration.mqtt]
# Event topic template.
event_topic="chirpstack/application/{{application_id}}/device/{{dev_eui}}/event/{{event}}"
# This is the topic on which the MQTT subscribes for receiving (enqueue) commands.
command_topic="chirpstack/application/{{application_id}}/device/{{dev_eui}}/command/{{command}}"
server="ssl://$MQTT_BROKER_HOST:$MQTT_BROKER_PORT/"
json=true
ca_cert="/opt/CA/mondas.crt"
tls_cert="/opt/CA/chirpstack.crt"
tls_key="/opt/CA/chirpstack.key"
I changed configuration/chirpstack/region_eu868.toml to
[regions.gateway.backend]
# The enabled backend type.
enabled="mqtt"
# MQTT configuration.
[regions.gateway.backend.mqtt]
# Event topic template.
event_topic="chirpstack/eu868/gateway/+/event/+"
# Command topic template.
command_topic="chirpstack/eu868/gateway/{{ gateway_id }}/command/{{ command }}"
# MQTT server (e.g. scheme://host:port where scheme is tcp, ssl or ws)
server="ssl://$MQTT_BROKER_HOST:$MQTT_BROKER_PORT/"
# Connect with the given username (optional)
username=""
# Connect with the given password (optional)
password=""
# Quality of service level
#
# 0: at most once
# 1: at least once
# 2: exactly once
#
# Note: an increase of this value will decrease the performance.
# For more information: https://www.hivemq.com/blog/mqtt-essentials-part-6-mqtt-quality-of-service-levels
qos=0
# Clean session
#
# Set the "clean session" flag in the connect message when this client
# connects to an MQTT broker. By setting this flag you are indicating
# that no messages saved by the broker for this client should be delivered.
clean_session=false
# Client ID
#
# Set the client id to be used by this client when connecting to the MQTT
# broker. A client id must be no longer than 23 characters. If left blank,
# a random id will be generated by ChirpStack.
client_id=""
ca_cert="/opt/CA/mondas.crt"
tls_cert="/opt/CA/chirpstack.crt"
tls_key="/opt/CA/chirpstack.key"
then I did docker-compose up -d. I also was watching the logs of my MQTT broker to see whether a connection was made. That seems to work. The only thing I haven’t done yet is connect a gateway to my chirpstack installation. I’ll do that tomorrow. Hope this helps you find the error.
One last thing, please note that I don’t use username & password entries. That’s it because I configured my MQTT broker to do the authentication with the SSL certificates, it gets the user from CN entry of the MQTT client certificate. My mosquitto configuration looks like this:
###################################
# authentication via CERTIFICATES #
###################################
listener 8883
allow_anonymous false
# password_file is ignored when use_identity_as_username is set to true
acl_file /etc/mosquitto/file.acl
cafile /etc/mosquitto/ca_certificates/mondas.crt
keyfile /etc/mosquitto/ca_certificates/server.key
certfile /etc/mosquitto/ca_certificates/server.crt
tls_version tlsv1.2
#client certifcate settings
require_certificate true
use_identity_as_username true
Perhaps this is also similar to your broker. Like I said, use mosquitto_sub to check that you can subscribe to your broker with the exact same parameters as chirpstack is going to use.
Why did you have to change the rights of the certificates? If I docker exec into my running chirpstack network server, the rights of the certificates look like this. Their owner is not nobody. Am I forgetting something.
$ pwd
/etc/chirpstack/certs
$ ls -al
total 20
drwxr-xr-x 2 1000 1000 4096 Mar 24 18:05 .
drwxr-xr-x 3 1000 1000 4096 Mar 23 18:15 ..
-rw-r--r-- 1 1000 1000 1281 Mar 14 21:42 ca_certificate.pem
-rw-r--r-- 1 1000 1000 1371 Mar 14 21:42 client_butler-main-1_certificate.pem
-rw-r--r-- 1 1000 1000 1704 Mar 14 21:42 client_butler-main-1_key.pem
One more question to your latest post:
Am I right that you used the mosquitto config in your latest post only for testing. I thought you mentioned that you deactivated mosquitto in the docker-compose file. Did I understand that correctly or do I have to fill in the mosquitto conf as well? I didn’t do that so far.
Why did you have to change the rights of the certificates?
because chirpstack was not able to start because it did not have read permission to open the key file.
So I entered the in the running image with docker exec -it chirpstack-docker_chirpstack_1 /bin/bash -l and checked with which user the process is running and it was running with user nobody and with id nobody I was able to get the numerical user id. I don’t know which image you are using, so yours may be a little bit different.
Am I right that you used the mosquitto config in your latest post only for testing
No, the config I posted is the config of my external MQTT broker running on a different server.
I thought you mentioned that you deactivated mosquitto in the docker-compose file.
yes I did, because I want to use an external MQTT broker, so there is no need to run the mosquitto server in the same docker-compose.yml file.
But you also have to make sure that all components use the correct domain/IP-address of your external broker. I do that via the MQTT_BROKER_HOST environment variable of the chirpstack service.
Thank you for helping me with this. I was not able to solve it with your suggestions, so I gave up.
But I changend my original plan and connect now the Chirpstack-Mosquitto-MQTT Server from my MQTT-Server and not the other way round. Then I was able to connect using the standard chirpstack documentation.
I created an app in ChirpStack v3 with MQTT integration and was able to publish events to an MQTT client residin another server without requiring TLS certificates.
However, when I recently upgraded to ChirpStack v4, I have been frustrated and stuck for days trying unsuccessfully to configure MQTT clients with certificates. As a newbie to TLS certifcates, searching Google for help has been fruitless.
When I created my ChirpStack application and selected integration with MQTT, I clicked on “Get Certificate” to create the ca.crt, ca.key, and cert.crt. I assumed these certificates were all that were required to configure my MQTT clients (node-red and MQTTX), but I have been unsuccessful after trying numerous configurations.
In addition to certificates, what changes to chirpstack.toml are required to allow MQTT clients to connect?
(“tcp://“ changed to “ssl:// ?)
What certificates are required by MQTT clients?….just the ca.crt (or ca.pem)? When are the ca.key and cert.crt also required?
My lone success was connecting to a node-red MQTT client residing on the same server as ChirpStack, where TLS certificates were not required.