Storing Recent Device Values in REDIS

I’m a little curious about the use of the REDIS database in Chirpstack - I know it’s used for a lot of internal housekeeping and storage of non-persistent data, but I’m wondering if it is also used for device data.

Basically what I’m looking for is the ability to store the JSON-parsed payload from my LoRaWAN devices (from a custom Device Profile Codec) to the REDIS store, so I can get the most recent value in my applications, rather than having to connect via MQTT and waiting for the next value.

Is this already supported? (I doubt it, since I haven’t seen many resources on accessing REDIS data in Chirpstack).

If not, I can certainly write something in Go or Python to do it from the MQTT messages, but I felt this would be something that could be done fairly efficiently as a Chirpstack global integration, since there is already a live session with the REDIS server.

The basic idea is that if I get a payload like this…

{ “applicationName”: “my_app”, “deviceName”: “TE101”, “object”: { “temperature”: 22.5, “battery”: 3.6 } }

I’d like nodes in REDIS that look like this…

my_app.TE101.temperature = 22.5
my_app.TE101.battery = 3.6

Then I could either do my whole subscription via REDIS, or take a hybrid approach and get ‘snapshot’ (ie. latest) data from REDIS on initial connection, then maintain my subscription via MQTT.

There are currently no Redis integrations for that purpose, but you could use the PostgreSQL integration: https://www.chirpstack.io/application-server/integrate/sending-receiving/

Not quite what I’m after.

I’m only looking for the latest value rather than a long history of values - getting the most recent value for multiple points can bc complex and time-consuming with SQL databases. There are some issues relational DBs and large quantities of timeseries data. You can get around a couple of them by using Timescale (a PostGRESQL extension designed for time series data), but only some. And Influx has one or two critical issues that prevent me from using it.

The idea is that for current values, you’re better off with an in-memory data store - REDIS is designed this way. So when you make a request no data has to be loaded from the hard-drive. This saves an insane amount of time and lets other systems subscribe or poll for results with negligible cost to the server.

Easier way, I’m doing this by setting uplink_retained_message for MQTT on server integration, on each MQTT subscribe to a topic connection, you get in batch latest values of all your devices (if the device communicated with MQTT since MQTT started) .You can even set MQTT to use persistent file to be reboot/restart proof

[application_server.integration.mqtt]
server="tcp://mosquitto:1883"
uplink_retained_message=true

Oh, THAT is interesting. Saves me from having to make hybrid solutions.

Thanks!

1 Like