Guidance on Data Flow & Best Practices for a LoRaWAN Network with ChirpStack

[arthursd]

Hello everyone,

I’m currently an apprentice working on a large IoT project, where I’m setting up a complete LoRaWAN network—from end devices to web applications—centered around ChirpStack. However, since I’m relatively new to networking and IoT, I find the vast amount of documentation and possible implementations a bit overwhelming.

I’m reaching out to the community for advice on best practices, particularly regarding data flow and infrastructure setup. Right now, I’m working on a demo with five different sensors configured within a Demo application in ChirpStack.

I’ve tried to organize my questions as clearly as possible:

1) Data Flow & Storage

Currently, uplinks are decoded using the Codec section in ChirpStack and stored via the InfluxDB integration. However, I’m unsure if this is the best approach.

• a) Should I:
  • Decode uplinks first and then store them in a database (if so, which type of database is best suited)?
  • Store uplinks as raw payloads and decode them later in my application (again, what type of database would be appropriate for this)?
• b) How is this usually handled in real-world implementations?
  • Do you use Redis Streams to retrieve uplinks, process them, and then store them?
  • Do you rely on MQTT to send uplinks to an application that handles decoding and storage?
  • Do you use the gRPC API to retrieve uplinks and process them before storage?
  • Maybe HTTP?
  • What kind of database(s) do you prefer? InfluxDB, PostgreSQL, MongoDB, or something else?
• c) Handling device data:
  • If multiple sensors of the same type exist, what’s the best way to retrieve and differentiate their data? Should I tag them somewhere (e.g., in ChirpStack, within the database itself, or in a separate table storing additional device information)?
  • How can I efficiently retrieve data from devices located in the same area?

2) MQTT Integration

Would you consider MQTT a must-use technology in modern ChirpStack implementations? How is it typically used between the LNS and applications?

3) gRPC API

What are the practical use cases for gRPC in a ChirpStack-based setup?

• Is it commonly used for retrieving uplinks (from wich source?) and sending downlinks?
• Should it be custom developed for these interactions, or is it intended for other purposes?

4) Best Practices for Data Flow

What is considered a good or best-practice data flow for a LoRaWAN network?

• Option 1: Devices ⟷ Gateways ⟷ LNS ⟷ Persistent Database ⟷ Application
• Option 2: Devices ⟷ Gateways ⟷ LNS ⟷ Application ⟷ Persistent Database
• Other? More complex architectures?

---

I realize this is quite a few questions, but I’d greatly appreciate any insights or guidance on where to focus my efforts. Despite my research, the range of options is overwhelming, and I’m struggling to determine which approach is most effective. I believe answers to these questions could also help others who are trying to get a clearer understanding of ChirpStack and its components.

Looking forward to your thoughts!
Best regards,
Arthur

[sp193]

1a) What’s your rationale of saving only the raw data? In database design, we would save only the necessary information. It’s a choice to decode the data in your application, but I would still decode it before saving.

Note that Chirpstack’s codecs are written in JavaScript, so you could be limited by what you can do with it.

Chirpstack provides you with a UI to freely change the codec. So if you find that more flexible than what your application can do, it could be a reason to use Chirpstack’s functionality.

As for the database, you should evaluate on how you will use the data and how you will access it. Different database kinds have different strengths.

1b) You need to figure out what role your application will take. I am personally of the opinion that applications should take the role (or more specifically, a subset of) of the Application Server and leave Chirpstack as the Network Server.

This means that MAC commands and lower-level messaging shall terminate at Chirpstack. This will exactly happen if you just use Chirpstack’s “integrations”, like MQTT. If you integrate with its Redis stream, you’ll get everything and it’s a tighter coupling with Chirpstack. Therefore, I feel that it’s sufficient and appropriate to terminate the LoRaWAN layer at the Chirpstack LoRa Network Server.

1c) You need to tag your device data with some geographic details. Chirpstack has tags and variables. Variables will be automatically included in the event data that is made available via integrations.

2. Whether you use it or not, it’ll be part of Chirpstack’s deployment. So I do use it.

3. It’s the internal API for making requests with Chirpstack, for actions not covered by the integration functionality. Such as programmatically managing the inventory. It’s also necessary for making multicast downlinks.

4. I feel that it’s more of option 2. How does the link from the database to your Application work? Do you tail the database? Why not have data pushed directly to your application? If you choose to have Chirpstack persist directly to your database, will you use Chirpstack’s data format?

[Liam_Philipp]

@sp193 has very good answers to these but I couldn’t help giving my own 2 cents.

@arthursd

1a) Decode uplinks first and then store them in a database (if so, which type of database is best suited)?

Decode the uplinks in Chirpstack, then use an integration to pass the JSON data to your application. The database choice is up to preference and whatever is best suited for your application.

@arthursd

1b) How is this usually handled in real-world implementations?

Using the redis streams is typically a layer deeper and more complicated then is necessary. For our application we use the EMQX integration, which is essentially MQTT++. This allows for more easily activating real-time triggers like alerts based off uplinks, instead of needing to constantly monitor a database for changes. That being said we do then store necessary data from the EMQX messages into SQL for our application to pull in the future. Do not use the gRPC API for retrieving uplinks, Chirpstack is not meant to store data itself and deletes it after 10 or so uplinks, the proper way to get the uplinks out of Chirpstack is with an integration.

@arthursd

1c) If multiple sensors of the same type exist, what’s the best way to retrieve and differentiate their data? Should I tag them somewhere (e.g., in ChirpStack, within the database itself, or in a separate table storing additional device information)?

Sp’s answers here are probably the best solution. For our purposes we do not need to pull geographically but instead by client, so we are using a separate application in Chirpstack for each client, typically that also ends up being geographical.

@arthursd

Would you consider MQTT a must-use technology in modern ChirpStack implementations?

I wouldn’t say it is must use as an integration (obviously it is a must use on the backend), but I’d say it’s the most useful for real-time data processing. If there are no real-time actions that need to be done from your application based on uplinks (for example if your application is only for summarizing data when needed) you could consider other integrations. EMQX also works for the real-time trigger requirements.

@arthursd

What are the practical use cases for gRPC in a ChirpStack-based setup?

I use the gRPC API for any interaction our portal must make to control Chirpstack, but not to monitor it. For example when a new user signs up, we create an application in Chirpstack via gRPC. When they want to change their device settings, we send a downlink to the device. When they want to add a device to their account, we use the gRPC to create one in Chirpstack. Etc…

@arthursd

What is considered a good or best-practice data flow for a LoRaWAN network?

Really depends on your application but I would say 2 as well. For example our application receives the data from Chirpstack via EMQX, then translates what it needs into a persistent database. So it falls into #2.

As an aside to these questions. If you’re running a large project you also likely want to look into some high availability and redundancy measures. I am currently working on this for our own deployment and if you’re interested you could get some ideas from my plan. It certainly is not the be all end all, or even the most practical setup for many deployments, but it will work well for our purposes: HA and redundancy for Chirpstack - #2 by Liam_Philipp

[arthursd]

@sp193

What’s your rationale of saving only the raw data?

I was considering storing the raw data to ensure persistence while only decoding it when necessary within an application. This might be a matter of storage efficiency and computation time. However, the question arose primarily when evaluating different methods of retrieving uplinks—Redis Streams (raw) vs. InfluxDB integration (decoded but seemingly missing some data like tags and variables) vs. MQTT, etc.

@sp193

How does the link from the database to your Application work? Do you tail the database? Why not have data pushed directly to your application? If you choose to have Chirpstack persist directly to your database, will you use Chirpstack’s data format?

Currently, the data is queried every few seconds to refresh the display. I’m not very familiar with InfluxDB yet, but tailing would likely be a more efficient approach. While the current setup works for my current needs, I agree that it’s not a long-term solution.

@Liam_Philipp

For our purposes we do not need to pull geographically but instead by client, so we are using a separate application in Chirpstack for each client, typically that also ends up being geographical.

Due to the way values are stored in InfluxDB via the integration, I cannot differentiate sensors by type or tags (or maybe I could?). To address this, I initially considered creating a separate ChirpStack application for each client and sensor type. However, this would result in a large number of applications. Is ChirpStack designed for this kind of structure, or is it better to have, say, a single application per client and use tags and variables for sorting?

@Liam_Philipp

If you’re running a large project you also likely want to look into some high availability and redundancy measures. I am currently working on this for our own deployment and if you’re interested you could get some ideas from my plan.

High availability and redundancy will definitely be important considerations for this project. Thanks for bringing up the topic—I’ll need to study it further!

---

At this point, I believe that using MQTT to forward data to an application server upon receiving an uplink—where decoding occurs before storing it in a dedicated database—would be a more flexible and scalable solution. ChirpStack’s JS codecs should be sufficient for decoding in most cases, but if necessary, decoding could also be handled in a separate application before storing the data. Other applications could then access this database (tailing it) to process and display the data, potentially storing additional information as needed.

Additionally, there would be a management layer over ChirpStack (the “portal” you are mentionning @Liam_Philipp, maybe it is something similar to the Actility ThingsPark?) to handle clients, applications, and tags using the gRPC API. gRPC would also be used for sending downlinks and multicast commands.

It could look like this:

Thank you both for your valuable insights. This discussion has really clarified things for me already!

[Liam_Philipp]

@arthursd

I initially considered creating a separate ChirpStack application for each client and sensor type

Chirpstack can support many applications, it really depends on what is the best for the purposes of your integration and you application. One thing to keep in mind is that only some integrations are “global” integrations, these are the integrations that can be enabled in the chirpstack.toml and they pull data from every application. The integrations that can only be configured in the UI, need to be configured for each application separately, so if you use one of these you should refrain from creating many applications.