Duplicated uplinks from gateway topic

Hi,

I am encountering an issue with my devices that are connected to Chirpstack and NodeRed on an AWS EC2 instance using RFM95. All data is stored in AWS IoT Core.

The problem I am experiencing is that NodeRed is processing uplinks twice, resulting in double downlinks. This is causing the device queue to fill with packets that will never be sent, causing the device to receive old data.

After using mosquitto_sub in my EC2 instance, I discovered that the topics gateway/+/event/up and application/+/device/+/event/up are being duplicated when a message is received. The delay between duplicated messages is short (around 40-50ms).

I activated the frame counter of the device at chirpstack, but I believe this is not the issue since I am only using one gateway for testing purposes.

I have made changes in the chirpstack-network-server.toml file to increase the RX1 delay to 5 seconds to avoid missing packets due to lack of gateway connectivity (the gateways will be used in farms with low connectivity). After making this change, I believe I have enough time to process the uplink and send a downlink with the updated changes before the time of the RX1 delay is reached so I expect that the downlink with the response would not sent after the next uplink (if the queue remains empty).

I have checked and the delay between an uplink and downlink is less than 400ms (NodeRed + AWS IoT core).

Therefore, I have configured the .toml file accordingly:

Network-server settings.

[network_server]

Network identifier (NetID, 3 bytes) encoded as HEX (e.g. 010203)

net_id=“000000”

Time to wait for uplink de-duplication.

deduplication_delay=“400ms”

Device session expiration.

device_session_ttl=“744h0m0s”

Get downlink data delay.

get_downlink_data_delay=“1000ms”

LoRaWAN regional band configuration.

[network_server.band]
name=“AU915”

Enforce 400ms dwell time.

uplink_dwell_time_400ms=false
downlink_dwell_time_400ms=false

Uplink max. EIRP.

uplink_max_eirp=-1

Enforce repeater compatibility.

repeater_compatible=false

LoRaWAN network related settings.

[network_server.network_settings]

Installation margin (dB) used by the ADR engine.

installation_margin=10

RX window (Class-A).

Set this to:

0: RX1 / RX2

1: RX1 only

2: RX2 only

rx_window=0

Class A RX1 delay

0=1sec, 1=1sec, … 15=15sec.

rx1_delay=5

RX1 data-rate offset

rx1_dr_offset=0

RX2 data-rate

When set to -1, the default RX2 data-rate will be used for the configured

rx2_dr=-1

RX2 frequency

When set to -1, the default RX2 frequency will be used.

rx2_frequency=-1

Prefer RX2 on RX1 data-rate less than.

rx2_prefer_on_rx1_dr_lt=0

Prefer RX2 on link budget.

rx2_prefer_on_link_budget=false

Prefer gateways for downlink with given uplink (SNR) margin.

gateway_prefer_min_margin=10

Downlink TX Power (dBm)

downlink_tx_power=-1

Disable mac-commands

disable_mac_commands=false

Disable ADR

disable_adr=false

Max mac-command error count.

max_mac_command_error_count=3

Enable only a given sub-set of channels

enabled_uplink_channels=[]

ADR plugins.

adr_plugins=[]

# Class B settings
[network_server.network_settings.class_b]
# Ping-slot data-rate.
ping_slot_dr=0

# Ping-slot frequency (Hz)
ping_slot_frequency=0

Network-server API

[network_server.api]

ip:port to bind the api server

bind=“0.0.0.0:8000”

Backend defines the gateway backend settings.

[network_server.gateway.backend]

Backend

type=“mqtt”

# MQTT gateway backend settings.
#
# This is the backend communicating with the LoRa gateways over a MQTT broker.
[network_server.gateway.backend.mqtt]
# MQTT topic templates for the different MQTT topics.
#
# The meaning of these topics are documented at:
# https://www.chirpstack.io/gateway-bridge/
#
# The default values match the default expected configuration of the
# ChirpStack Gateway Bridge MQTT backend. Therefore only change these values when
# absolutely needed.

# Event topic template.
event_topic="gateway/+/event/+"

# Command topic template.
#
# Use:
#   * "{{ .GatewayID }}" as an substitution for the LoRa gateway ID
#   * "{{ .CommandType }}" as an substitution for the command type
command_topic_template="gateway/{{ .GatewayID }}/command/{{ .CommandType }}"

# MQTT server (e.g. scheme://host:port where scheme is tcp, ssl or ws)
server="tcp://localhost:1883"

Metrics collection settings.

[metrics]

Timezone

timezone=“Local”

Join-server settings.

[join_server]

Default join-server settings.

This join-server will be used when resolving the JoinEUI is set to false

or as a fallback when resolving the JoinEUI fails.

[join_server.default]

hostname:port of the default join-server

This API is provided by ChirpStack Application Server.

server=“http://localhost:8003

Here there are some information of the live frames of a device and its queue:


On the live lora frames the duplication is not shown.

I’ve been trying to find the cause of the problem for a few days, please tell me if you see something weird in my setup.

Also, how can I get more information about how is originated the “gateway” topic to try find the cause of this?

Thanks!

I’ve been searching for a solution about this issue and I found this thread Duplicate messages from RAK4631 - #9 by carlrowan - WisBlock - RAKwireless Forum that says if the gateway is too close of the end device will have non-linear distortion that will make the uplink received on the Gateway from the lora network is received in two different frequencies. This is not the case because the RSSI timestamp and Frequency is the same in both packets.
Now I have more information to provide, here is the output of the Semtech Packet Forwarder in the Gateway:

JSON up: {“stat”:{“time”:“2023-03-15 01:12:41 GMT”,“rxnb”:1,“rxok”:0,“rxfw”:0,“ackr”:0.0,“dwnb”:0,“txnb”:0}}
WARNING: [gps] could not get a valid message from GPS (no time)

INFO: [down] PULL_ACK received in 336 ms
WARNING: [gps] could not get a valid message from GPS (no time)
INFO: [down] PULL_ACK received in 337 ms
WARNING: [gps] could not get a valid message from GPS (no time)
INFO: [down] PULL_RESP received - token[219:6] :slight_smile:

JSON down: {“txpk”:{“imme”:false,“rfch”:0,“powe”:27,“ant”:0,“brd”:0,“tmst”:2399495948,“freq”:926.9,“modu”:“LORA”,“datr”:“SF11BW500”,“codr”:“4/5”,“ipol”:true,“size”:29,“data”:“YOCC1AGwFQABD3dI22UNDkwn1FL3GljL8L+Nwcc=”}}
INFO: == used txlut index:15
INFO: [down] PULL_RESP received - token[219:6] :slight_smile:

JSON down: {“txpk”:{“imme”:false,“rfch”:0,“powe”:27,“ant”:0,“brd”:0,“tmst”:2399495948,“freq”:926.9,“modu”:“LORA”,“datr”:“SF11BW500”,“codr”:“4/5”,“ipol”:true,“size”:29,“data”:“YOCC1AGwFQABD3dI22UNDkwn1FL3GljL8L+Nwcc=”}}
INFO: == used txlut index:15
src/jitqueue.c:280:jit_enqueue(): ERROR: Packet (type=0) REJECTED, collision with packet already programmed at 2399495948 (2399495948)
ERROR: Packet REJECTED (jit error=5)
INFO: [down] PULL_RESP received - token[219:6] :slight_smile:

JSON down: {“txpk”:{“imme”:false,“rfch”:0,“powe”:27,“ant”:0,“brd”:0,“tmst”:2400495948,“freq”:923.3,“modu”:“LORA”,“datr”:“SF12BW500”,“codr”:“4/5”,“ipol”:true,“size”:29,“data”:“YOCC1AGwFQABD3dI22UNDkwn1FL3GljL8L+Nwcc=”}}
INFO: == used txlut index:15
non-linear distortionWARNING: [gps] could not get a valid message from GPS (no time)
WARNING: [gps] could not get a valid message from GPS (no time)

INFO: tx_start_delay=1497 (1497.000000) - (1497, bw_delay=0.000000, notch_delay=0.000000)
INFO: tx_start_delay=1497 (1497.000000) - (1497, bw_delay=0.000000, notch_delay=0.000000)
WARNING: [gps] could not get a valid message from GPS (no time)

INFO: [down] PULL_ACK received in 337 ms
WARNING: [gps] could not get a valid message from GPS (no time)

INFO: Disabling GPS mode for concentrator’s counter…
INFO: host/sx1301 time offset=(1678840388s:237643µs) - drift=-76µs
INFO: Enabling GPS mode for concentrator’s counter.

WARNING: [gps] could not get a valid message from GPS (no time)
WARNING: [gps] could not get a valid message from GPS (no time)

2023-03-15 01:13:11 GMT

[UPSTREAM]

RF packets received by concentrator: 0

CRC_OK: 0.00%, CRC_FAIL: 0.00%, NO_CRC: 0.00%

RF packets forwarded: 0 (0 bytes)

PUSH_DATA datagrams sent: 1 (111 bytes)

PUSH_DATA acknowledged: 0.00%

[DOWNSTREAM]

PULL_DATA sent: 3 (100.00% acknowledged)

PULL_RESP(onse) datagrams received: 3 (633 bytes)

RF packets sent to concentrator: 2 (87 bytes)

TX errors: 0

TX rejected (collision packet): 33.33% (req:54, rej:18)

TX rejected (collision beacon): 0.00% (req:54, rej:0)

TX rejected (too late): 0.00% (req:54, rej:0)

TX rejected (too early): 0.00% (req:54, rej:0)

BEACON queued: 0

BEACON sent so far: 0

BEACON rejected: 0

[JIT]

SX1301 time (PPS): 2402888131

src/jitqueue.c:448:jit_print_queue(): INFO: [jit] queue is empty

[GPS]

Invalid time reference (age: 1220 sec)

no valid GPS coordinates available yet

END

So the duplication does not seems to be related to the Gateway. But you can see that it receives the duplicated replies as a downlink that are due to the duplication of the packets that now seems to be related in the Gateway Bridge.