Azure Service Bus Integration: Are duplicate uplinks expected?

ChirpStack (v4) Integrations
1

Hi Everyone,

I’m hoping to get some advice on a deduplication question.
I’ve got about 400 pulse-counting LoRaWAN devices installed in a building, and as expected, many are picked up by multiple gateways.

When I check the ChirpStack UI, I can see deduplication is happening — the frame counter (FCnt) only increments once per real uplink, and every second event shows:

Code: UPLINK_F_CNT_RETRANSMISSION
Level: WARNING

However, I’m using the Azure Service Bus integration and I’m still seeing all the duplicated events being sent to the queue.

My question is:
Does ChirpStack publish every received packet to the queue (even retransmissions or multiple gateway receptions), and it’s up to us to deduplicate on the Azure side?
Or is it meant to only send messages when FCnt increments (deduplicated uplinks)?

Thanks in advance for any clarification!

2

Never used the azure service but I believe all device type errors get sent with most integrations. The real issue is why you are seeing so many of these.

The uplink_f_cnt_retransmission error will only appear when the delay between the duplicate uplinks is larger than the deduplication delay. Otherwise Chirpstack will silently ignore the duplicates.

There is a setting in the [network] section of your chirpstack.toml called deduplication_delay, which defaults to 200ms, maybe try increasing that.

3

thanks - makes sense it definitely exceeds the default 200ms

image
image974×603 32.9 KB

I haven’t seen it before as we were using embedded NS in the gateways and having multiple gateways in a fleet with 1 master GW forwarding to IOT hub - now switching to chirpstack network server might need to tune this delay.

ive made some adjustments but still seeing the issue i can see alot of retransmission error
currently using these settings there are still some that are 2-3 second delays but increasing more seemed to then create downlink errors.

Time to wait for uplink de-duplication.

deduplication_delay=“1000ms”

Get downlink data delay.

get_downlink_data_delay=“500ms”

2025-04-23T07:27:56.094840Z INFO up{deduplication_id=a78ba597-6328-439f-9729-a97b6abcb258}:data_up{dev_eui=“24e124136e147750”}: chirpstack::maccommand::link_check: Received LinkCheckReq dev_eui=24e124136e147750
2025-04-23T07:27:56.095250Z INFO up{deduplication_id=a78ba597-6328-439f-9729-a97b6abcb258}:data_up{dev_eui=“24e124136e147750”}: chirpstack::storage::device_gateway: Gateway rx-info saved dev_eui=24e124136e147750
2025-04-23T07:27:56.104229Z INFO chirpstack::integration::azure_service_bus: Publishing event event=up dev_eui=24e124136e147750
2025-04-23T07:27:56.115452Z INFO up{deduplication_id=a78ba597-6328-439f-9729-a97b6abcb258}:data_up{dev_eui=“24e124136e147750”}: chirpstack::storage::device: Device partially updated dev_eui=24e124136e147750
2025-04-23T07:27:56.622086Z INFO up{deduplication_id=a78ba597-6328-439f-9729-a97b6abcb258}:data_up{dev_eui=“24e124136e147750”}:data_down{downlink_id=1736775781}: chirpstack::storage::device: Device partially updated dev_eui=24e124136e147750
2025-04-23T07:27:56.684821Z INFO tx_ack{downlink_id=1736775781}: chirpstack::storage::device: Device partially updated dev_eui=24e124136e147750

I also noticed there might be a bigger delay between the “time” in the payload to the EnqueuedTimeUtc from my understanding the time in payload is the time the packet is received from the sensor and the enqueued time is when the service bus is delivering the message on the queue?

for these devices with retransmission alerts I see some high latency like this one received 2025-04-22T00:49:04.153+00:00 but sent to the queue 2025-04-23T02:01:49.5730031Z

“rxInfo”: [
{
“gatewayId”: “24e124fffef6ecb6”,
“rssi”: -57.0,
“snr”: 14.0,
“channel”: null,
“metadata”: null
},
{
“gatewayId”: “24e124fffef71910”,
“rssi”: -100.0,
“snr”: 10.5,
“channel”: null,
“metadata”: null
}
],
“time”: “2025-04-22T00:49:04.153+00:00”,
“txInfo”: {
“frequency”: 916800000,
“modulation”: {
“lora”: {
“bandwidth”: 125000,
“spreadingFactor”: 7,
“codeRate”: “CR_4_5”
}

checking logs it seems like chirpstack does publish every uplink to the service bus integration as soon as they arrive though…

4

the retransmission errors have improved - but is possible to apply ChirpStack’s internal de-duplication logic to filter outbound integration Azure Service Bus? so the messages sent to queue match what is shown in the chirpstack UI?

5

I’m sorry man, I really don’t know. I find it shocking that it doesn’t de-duplicate them in the first place. I have only used the MQTT and SQL integrations, and they certainly de-duplicate the messages before posting them to the integration. I assumed all the integrations worked the same way.

Worst case, is the deduplication_id posted along with the messages to the Azure service bus? Could you de-duplicate them yourself? Or depending on what Azure service bus requires, maybe you want to create a middle man that uses the MQTT integration and posts that to Azure?

6

no problem, same here I’m unsure why the azure integration would just send everything, we can deduplicate when receiving the messages of the queue was just hoping to simplify some of that logic seeing chirpstack already does it… and also trying to identify if there is some other root cause

I also notice large delays between duplicates on the message bus where like the First message comes in and we process/ingest it : EnqueuedTimeUtc: 2025-05-14T01:36:11.3189557Z
Then exact same message again at: 2025-05-14T05:03:15.0083704Z (over 3.5 hours later)

thanks for your replies though it has pointed us in the right direction!

7

Hi @Liam_Philipp some good news, so I thought I would update you…
I done some more testing after you mentioned the SQL integration I went and added the Postgres integration as well.
Confirmed that SQL working perfectly and matches the UI logs.

I think the service bus integration is actually working fine, logs show its only writing the up once and TS matching the sql integration.

image
image1350×201 145 KB

So we have either a code issue receiving the messages or the service bus is requeuing the messages (im still looking into that).