IMST Lite GW, after update, double fcnt messages

ChirpStack Gateway Bridge
1

Dear Community,
IMST Lite GW was running smoothly, but after updating

  • standard Packet Forwarder (no update, not using Concentratord)
  • Gateway-Bridge 3.10.0 → 3.13.2
  • Network-Server 3.12.3 → 3.15.5
  • Application-Server 3.14.0 → 3.17.4
    there are four weird things I observed, but couldn’t find a reason for:
  1. Now the amber LED is lighting up on each received message, what did not happen before the update
  2. before the updating my node was sending exactly every 3 min (e.g. 00:00:05, 00:03:05, 00:06:05, 00:09:05, …). After updating the timestamps changed to 00:00:05, 00:03:38, 00:06:05, 00:09:38, …)
  3. Every message with the timestamp 00:0x:38 (=every second message) is showing exactly the same payload like the 00:0x:05 message. RSSi and SNR values are different and ok.
  4. The 00:0x:05 and 00:0x:38 message are sharing the same fcnt value (corresponds to the same payload)

The node is based on adafruit’s feather M0 and LMIC lib, but untouched and still sending, so shouldn’t be the issue.

Idea: Since I didn’t change the node and didn’t update the standard Packet Forwarder, it must be the Gateway-Bridge handling something different, but I didn’t find out what.
==>Any idea or direction?

2

I checked a lot of logs and tracked down many posts - nothing that matched to my issue.

After a lot of useless action I pressed ‘(Re-) Activate Device’ on the ‘Activation’ tab. The result was doubled messages being replaced by error messages ‘frame-counter did not increment’. Still I had only half of the messages with valid content (every second one).

Then I went out to reset the node. Coming back the log showed only error frames ‘frame-counter reset or rollover occured’, but fcnt showed an increase with every frame. (Made sense)

Final step was to press ‘(Re-) Activate Device’ on the ‘Activation’ tab again, so that the frame counter on ChirpStack was reset and everything was perfect again.

The amber LED is still lighting up at every message, but shorter now (I’m 100% sure that this was not the case before).
Logging with ‘sudo tcpdump -Aq port 1700 -i lo’ shows…

19:51:58.173939 IP localhost.1700 > localhost.41955: UDP, length 202
E…@.@…{“txpk”:{“imme”:false,“rfch”:0,“powe”:14,“ant”:0,“brd”:0,“tmst”:3167341299,“freq”:867.3,“modu”:“LORA”,“datr”:“SF7BW125”,“codr”:“4/5”,“ipol”:true,“size”:24,“data”:“YKgbASaMqQoHBohmhFAHB1huhFDVzsN7”}}

…every time after a message is received. I do not remember that this was the case before. I have no idea what is causing these downlinks. I’m not aware of doing that actively.

I assume that the downlink is causing the amber LED to light up, but I’m not sure. But ok, now the Lite Gateay looks more attractive+active, instead of showing the green LED only.

Conclusion:
It is very unlikely that the node caused that issue, because it happened exactly after the ChirpStack update. Obviously the updated ChirpStack struggled to continue on the running node. Although the issue is solved I would like to understand what happened exactly, so somebody feel free to explain :slight_smile:
==>After Updating Chirpstack it might happen that some nodes need a reset

Dear Brocaar, thank you so much for that awesome work on ChirpStack. I wonder how you manage to keep such a broad knowledge ready that it takes to make ChirpStack possible.

Cheers

3

This could be caused by the ADR algorithm and NbTrans that is set by the ADR algorithm. E.g. upgrade to new ChirpStack version caused some downtime. When everything came back only the ADR algorithm detected packet-loss and increased the NbTrans number (number of times the device repeats the uplink). Depending your configuration (skip frame-counter) this will either cause duplicated uplinks or errors that the frame-counter did not increment if the same uplink was already processed.

Over time, the ADR algorithm will decrease the NbTrans number when the packet-loss decreases.