What is the suggested way to to log and view all messages from devices that are registered to different applications?
I’m looking for similar functionality that Actility’s Wireless Logger module in Thingpark gives, ie. to see latest (and history of) message reveiced by all the devices.
It seems that web ui can visualise messages, but only per gateway. But what if we want to see all lora uplinks/downlinks regardless of the gateway?
You could achieve this by subscribing to the application/# MQTT topic. This would mean subscribe to all MQTT topics starting with application/, thus it would match all applications and devices.
What I would suggest is enabling the postgresql integration, this way Chipstack will forward all device events to a sql database that will store them indefinitely, with all the data and metadata. All thats required is to set up another PSQL database (or just another table with the same database/user in your postgres container) and then add the following to your [integration] section of your Chirpstack.toml.
As a side note it seems strange that you are not seeing the MQTT messages in your broker, have you already changed the [integration.mqtt] section above? Or perhaps you have an atypical TLS setup (ACL and mosquitto.conf) on your MQTT broker such that the mqtt integration isn’t posting the events to your MQTT broker. As your Chirpstack setup is still working you should at the very least be able to see the raw still encrypted messages coming from the gateway bridge if you subscribe to the topic “#” rather than “application/#”.
The %u in the ACL gets replaced by the MQTT client’s (in this scenario thats you using mosquitto_sub) username, restricting a client to only view the topics with their username in them.
use_identity_as_username means it uses the CN on the certificates as your username. Presumably you are using the gateway’s certificates to connect with mosquitto_sub, thus it is only showing you the events following the pattern eu868/gateway/%u/# which are messages directly to/from the MQTT forwarder, rather than application/%u/#, which contains the application events after they have been processed and decrypted by chirpstack.
What you should do is go into the UI, under the application you want to monitor and retrieve the certificate from the “mqtt integration” button there, that will have the proper CN to view that applications MQTT events.
The reason you can view all events from inside the container or using 1883 is because of the other listener set up:
listener 1883 0.0.0.0
allow_anonymous true
Which allows any other connection on the same machine to connect without TLS and without the ACL restrictions. If you tried to connect to the mosquitto broker over port 1883 from another machine it would fail.
will allow connections from any interface in the internet?
If I change the setting to as per the instructions:
listener 1883 127.0.0.1
and try to start the stack, I get lots of access denied errors, as other services cant access mosquitto.
Do I need to setup a network in the docker-compose file, and how can I create a mosquitto listerner on port 1883 that allows chirpstack service to connect?
Sorry you are right, as per the instructions you should use:
listener 1883 127.0.0.1
allow_anonymous true
Which allows connections from the same device to be unencrypted. If you also have the allow_anonymous below the listener it should let the other containers connect without issue. If not please share your full mosquitto.conf.
Are all of your components on the same machine? If so I don’t see how that configuration difference could cause those issues. It would be a good idea to set all of your services onto the same docker network although if you haven’t defined a network yet they should all default to the same one.
I believe that should be fine. As long as when you try to mosquitto_sub to the broker from outside the machine on 1883 it fails. I have a similar setup, where I just use:
listener 1883
configs...
listener 8883
configs...
And then using a Traefik reverse-proxy I only route external traffic to the 8883 port. For most purposes it should be fine to leave the listeners as wildcards and then only expose the 8883 port to WAN.
Thanks @Liam_Philipp for the advice, this is now sorted!
But the original question about getting logs from all devices under different apps:
You could achieve this by subscribing to the application/# MQTT topic. This would mean subscribe to all MQTT topics starting with application/ , thus it would match all applications and devices.
So I made an an mqtt integration under one application
Then I subscibe with app’s generated certificates to the topic application/#, but I only see device log data from devices under that specific application. I suppose that is the way it is intended to work?
Yes the “mqtt integration” through the UI just gives you the certificates signed for that specific application. If you needed to subscribe to all the MQTT events regardless of application there are a few methods I know of:
Use mosquitto_sub on the local machine through port 1883, since the listener is not bound by the ACL rules you can see all the events.
If you need subscribe from a remote machine, a slightly more in depth but certainly possible option would be to generate a certificate that is signed by the MQTT broker’s CA with some admin type username and then in your ACL allow access for that user to all topics.
Last but not least (infact I’d say the best option) is if you set up an external MQTT broker on the server you wish to monitor the events from, you can simply set the “server=” line in the [integration.mqtt] section of your chirpstack.toml to the addres of your remote MQTT broker, and chirpstack will forward all “integration” mqtt events to that broker, regardless of application. There is even the option to set up certificates for that as well if you require TLS.
I suppose you could also just delete your ACL file and the reference to it in your mosquitto.conf, that way any certificate signed by your CA could view any applications devices. This would reduce security, as in the event one of your gateways was compromised, it could be used to view/post on any application on the server. A considerable risk if you have multiple tenants with their own gateways but trivial in some smaller applications, depends on you.