Hi, I need help. I am new to Chirpstack (CS). I have several Multitech MTCDTIP gateways running in network server mode. To consolidate the keys, I decide to try CS. The installation of CS4 on debian11 was flawless, once I input the region id, the Multitech gateway in packet forwarder mode and sensors: worked. I can seeing the gateway and device on the CS web page. However a few days later it failed, I didn’t make any changes, I am only seeing the “join” event.
In looking at the tcpdump on port 1700, I can see both rx and tx pks
192.168.100.14.48470 > 192.168.100.21.1700: UDP, length 305
E…M…@.@…d…d…V…9. .^K…{“rxpk”:[{“jver”:1,“tmst”:524036726,“time”:“2025-05-03T18:12:54.726735Z”,“tmms”:1430331192726,“chan”:8,“rfch”:0,“freq”:903.000000,“mid”:16,“stat”:1,“modu”:“LORA”,“datr”:“SF8BW500”,“codr”:“4/5”,“rssis”:-36,“lsnr”:11.0,“foff”:699,“rssi”:-35,“size”:23,“data”:“ADYxMDJPQk9WHmYCAAAAgAByAZZOQD4=”}]}
14:12:54.784362 IP (tos 0x0, ttl 64, id 17, offset 0, flags [DF], proto UDP (17), length 32)
debianv4.1700 > 192.168.100.14.48470: UDP, length 4
E… …@.@…C…d…d…V…I…^K.
14:12:55.031739 IP (tos 0x0, ttl 64, id 71, offset 0, flags [DF], proto UDP (17), length 241)
debianv4.1700 > 192.168.100.14.47838: UDP, length 213
E…G@.@…<…d…d…Jg.n…{“txpk”:{“imme”:false,“rfch”:0,“powe”:21,“ant”:0,“brd”:0,“tmst”:529036726,“freq”:923.3,“modu”:“LORA”,“datr”:“SF7BW500”,“codr”:“4/5”,“ipol”:true,“size”:33,“data”:“IJ/lrN3jKLWdjBl3rRzKdUs6UxyhVc69TpZJiotNoTzL”}}
Also the CS trace (below) is showing that it has sent the join-accept response, however, the device is not joining. I reverted the system to multitech network server mode just to make sure there is no failure in the gateway or sensor device: they connected. I would appreciate any assistance on how understand where the error is occurring. What should I check?
May 03 14:14:07 debianv4 chirpstack[25122]: 2025-05-03T18:14:07.002224Z TRACE chirpstack::gateway::backend::mqtt: MQTT event event=Incoming(Publish(Publish { dup: false, qos: AtMostOnce, retain: false, topic: b"us915_0/gateway/00800000a000dc06/event/up", pkid: 0, payload: b"{"phyPayload":"ADYxMDJPQk9WHmYCAAAAgAB5AXV0/5k=","txInfo":{"frequency":903100000,"modulation":{"lora":{"bandwidth":125000,"spreadingFactor":10,"codeRate":"CR_4_5"}}},"rxInfo":{"gatewayId":"00800000a000dc06","uplinkId":23573,"gwTime":"2025-05-03T18:14:06.907412Z","timeSinceGpsEpoch":"1430331264.907s","rssi":-108,"snr":-16.8,"channel":4,"rfChain":1,"context":"I4mN6g==","crcStatus":"CRC_OK"}}", properties: None }))
May 03 14:14:07 debianv4 chirpstack[25122]: 2025-05-03T18:14:07.002284Z INFO chirpstack::gateway::backend::mqtt: Message received from gateway region_id=“us915_0” topic=us915_0/gateway/00800000a000dc06/event/up qos=AtMostOnce json=true
May 03 14:14:07 debianv4 chirpstack[25122]: 2025-05-03T18:14:07.002475Z TRACE chirpstack::uplink: Adding uplink event to deduplication set and getting lock key=“up:collect:{us915_0:08e0ecd0ae03120a1a0808c8d007100a2801:00363130324f424f561e6602000000800079017574ff99}”
May 03 14:14:07 debianv4 chirpstack[25122]: 2025-05-03T18:14:07.003201Z TRACE chirpstack::uplink: Waiting for more uplink events to receive key=“up:collect:{us915_0:08e0ecd0ae03120a1a0808c8d007100a2801:00363130324f424f561e6602000000800079017574ff99}”
May 03 14:14:07 debianv4 chirpstack[25122]: 2025-05-03T18:14:07.153464Z TRACE chirpstack::downlink::scheduler: Starting class_b_c_scheduler_loop run
May 03 14:14:07 debianv4 chirpstack[25122]: 2025-05-03T18:14:07.153507Z TRACE chirpstack::downlink::scheduler: Getting devices that have schedulable queue-items
May 03 14:14:07 debianv4 chirpstack[25122]: 2025-05-03T18:14:07.153819Z TRACE chirpstack::downlink::scheduler: Starting multicast-group queue scheduler loop run
May 03 14:14:07 debianv4 chirpstack[25122]: 2025-05-03T18:14:07.153837Z TRACE chirpstack::downlink::scheduler: Getting schedulable multicast-group queue items
May 03 14:14:07 debianv4 chirpstack[25122]: 2025-05-03T18:14:07.158020Z TRACE chirpstack::downlink::scheduler: Got this number of devices with schedulable queue-items device_count=0
May 03 14:14:07 debianv4 chirpstack[25122]: 2025-05-03T18:14:07.158095Z TRACE chirpstack::downlink::scheduler: class_b_c_scheduler_loop completed successfully
May 03 14:14:07 debianv4 chirpstack[25122]: 2025-05-03T18:14:07.158151Z TRACE chirpstack::downlink::scheduler: Got this number of multicast-group queue items count=0
May 03 14:14:07 debianv4 chirpstack[25122]: 2025-05-03T18:14:07.158166Z TRACE chirpstack::downlink::scheduler: Multicast-group queue scheduler run completed successfully
May 03 14:14:07 debianv4 chirpstack[25122]: 2025-05-03T18:14:07.204554Z TRACE chirpstack::uplink: Collecting received uplink events key=“up:collect:{us915_0:08e0ecd0ae03120a1a0808c8d007100a2801:00363130324f424f561e6602000000800079017574ff99}”
May 03 14:14:07 debianv4 chirpstack[25122]: 2025-05-03T18:14:07.205068Z INFO up{deduplication_id=b9ebb637-f529-48bf-bdd8-497f56e9aecc}: chirpstack::uplink: Uplink received m_type=JoinRequest
May 03 14:14:07 debianv4 chirpstack[25122]: 2025-05-03T18:14:07.205088Z DEBUG up{deduplication_id=b9ebb637-f529-48bf-bdd8-497f56e9aecc}: chirpstack::uplink: Updating gateway meta-data for uplink frame-set
May 03 14:14:07 debianv4 chirpstack[25122]: 2025-05-03T18:14:07.205959Z DEBUG up{deduplication_id=b9ebb637-f529-48bf-bdd8-497f56e9aecc}: chirpstack::uplink: Logging uplink frame to Redis Stream
May 03 14:14:07 debianv4 chirpstack[25122]: 2025-05-03T18:14:07.206752Z TRACE up{deduplication_id=b9ebb637-f529-48bf-bdd8-497f56e9aecc}:join_request: chirpstack::uplink::join: Getting JoinRequestPayload
May 03 14:14:07 debianv4 chirpstack[25122]: 2025-05-03T18:14:07.206781Z TRACE up{deduplication_id=b9ebb637-f529-48bf-bdd8-497f56e9aecc}:join_request{dev_eui=“008000000002661e”}: chirpstack::uplink::join: Getting device
May 03 14:14:07 debianv4 chirpstack[25122]: 2025-05-03T18:14:07.210464Z TRACE up{deduplication_id=b9ebb637-f529-48bf-bdd8-497f56e9aecc}:join_request{dev_eui=“008000000002661e”}: chirpstack::uplink::join: Getting device keys
May 03 14:14:07 debianv4 chirpstack[25122]: 2025-05-03T18:14:07.211974Z TRACE up{deduplication_id=b9ebb637-f529-48bf-bdd8-497f56e9aecc}:join_request{dev_eui=“008000000002661e”}: chirpstack::uplink::join: Validating region_config_id against device-profile
May 03 14:14:07 debianv4 chirpstack[25122]: 2025-05-03T18:14:07.212003Z TRACE up{deduplication_id=b9ebb637-f529-48bf-bdd8-497f56e9aecc}:join_request{dev_eui=“008000000002661e”}: chirpstack::uplink::join: Filtering rx_info by tenant_id
May 03 14:14:07 debianv4 chirpstack[25122]: 2025-05-03T18:14:07.212021Z TRACE up{deduplication_id=b9ebb637-f529-48bf-bdd8-497f56e9aecc}:join_request{dev_eui=“008000000002661e”}: chirpstack::uplink::join: Logging uplink frame-set
May 03 14:14:07 debianv4 chirpstack[25122]: 2025-05-03T18:14:07.213425Z TRACE up{deduplication_id=b9ebb637-f529-48bf-bdd8-497f56e9aecc}:join_request{dev_eui=“008000000002661e”}: chirpstack::uplink::join: Setting random DevAddr
May 03 14:14:07 debianv4 chirpstack[25122]: 2025-05-03T18:14:07.213470Z TRACE up{deduplication_id=b9ebb637-f529-48bf-bdd8-497f56e9aecc}:join_request{dev_eui=“008000000002661e”}: chirpstack::uplink::join: Validate dev-nonce and get device-keys
May 03 14:14:07 debianv4 chirpstack[25122]: 2025-05-03T18:14:07.221743Z INFO up{deduplication_id=b9ebb637-f529-48bf-bdd8-497f56e9aecc}:join_request{dev_eui=“008000000002661e”}: chirpstack::storage::device_keys: Device-nonce validated, join-nonce incremented and stored dev_eui=008000000002661e dev_nonce=377
May 03 14:14:07 debianv4 chirpstack[25122]: 2025-05-03T18:14:07.221793Z TRACE up{deduplication_id=b9ebb637-f529-48bf-bdd8-497f56e9aecc}:join_request{dev_eui=“008000000002661e”}: chirpstack::uplink::join: Constructing JoinAccept payload
May 03 14:14:07 debianv4 chirpstack[25122]: 2025-05-03T18:14:07.221839Z TRACE up{deduplication_id=b9ebb637-f529-48bf-bdd8-497f56e9aecc}:join_request{dev_eui=“008000000002661e”}: chirpstack::uplink::join: Setting session-keys
May 03 14:14:07 debianv4 chirpstack[25122]: 2025-05-03T18:14:07.221862Z TRACE up{deduplication_id=b9ebb637-f529-48bf-bdd8-497f56e9aecc}:join_request{dev_eui=“008000000002661e”}: chirpstack::uplink::join: Logging uplink meta
May 03 14:14:07 debianv4 chirpstack[25122]: 2025-05-03T18:14:07.223311Z TRACE up{deduplication_id=b9ebb637-f529-48bf-bdd8-497f56e9aecc}:join_request{dev_eui=“008000000002661e”}: chirpstack::uplink::join: Setting device-session
May 03 14:14:07 debianv4 chirpstack[25122]: 2025-05-03T18:14:07.223357Z TRACE up{deduplication_id=b9ebb637-f529-48bf-bdd8-497f56e9aecc}:join_request{dev_eui=“008000000002661e”}: chirpstack::uplink::join: Flushing device-queue
May 03 14:14:07 debianv4 chirpstack[25122]: 2025-05-03T18:14:07.224365Z INFO up{deduplication_id=b9ebb637-f529-48bf-bdd8-497f56e9aecc}:join_request{dev_eui=“008000000002661e”}: chirpstack::storage::device_queue: Device queue flushed dev_eui=008000000002661e count=0
May 03 14:14:07 debianv4 chirpstack[25122]: 2025-05-03T18:14:07.224399Z TRACE up{deduplication_id=b9ebb637-f529-48bf-bdd8-497f56e9aecc}:join_request{dev_eui=“008000000002661e”}: chirpstack::uplink::join: Updating device
May 03 14:14:07 debianv4 chirpstack[25122]: 2025-05-03T18:14:07.230814Z INFO up{deduplication_id=b9ebb637-f529-48bf-bdd8-497f56e9aecc}:join_request{dev_eui=“008000000002661e”}: chirpstack::storage::device: Device partially updated dev_eui=008000000002661e
May 03 14:14:07 debianv4 chirpstack[25122]: 2025-05-03T18:14:07.231307Z TRACE up{deduplication_id=b9ebb637-f529-48bf-bdd8-497f56e9aecc}:join_request{dev_eui=“008000000002661e”}: chirpstack::uplink::join: Starting downlink join-accept flow
May 03 14:14:07 debianv4 chirpstack[25122]: 2025-05-03T18:14:07.231577Z TRACE up{deduplication_id=b9ebb637-f529-48bf-bdd8-497f56e9aecc}:join_request{dev_eui=“008000000002661e”}:join_accept{downlink_id=3332594922}: chirpstack::downlink::join: Set device-gateway rx-info
May 03 14:14:07 debianv4 chirpstack[25122]: 2025-05-03T18:14:07.231827Z TRACE up{deduplication_id=b9ebb637-f529-48bf-bdd8-497f56e9aecc}:join_request{dev_eui=“008000000002661e”}:join_accept{downlink_id=3332594922}: chirpstack::downlink::join: Select downlink gateway
May 03 14:14:07 debianv4 chirpstack[25122]: 2025-05-03T18:14:07.231859Z TRACE up{deduplication_id=b9ebb637-f529-48bf-bdd8-497f56e9aecc}:join_request{dev_eui=“008000000002661e”}:join_accept{downlink_id=3332594922}: chirpstack::downlink::join: Setting tx-info
May 03 14:14:07 debianv4 chirpstack[25122]: 2025-05-03T18:14:07.231872Z TRACE up{deduplication_id=b9ebb637-f529-48bf-bdd8-497f56e9aecc}:join_request{dev_eui=“008000000002661e”}:join_accept{downlink_id=3332594922}: chirpstack::downlink::join: Setting tx-info for RX1
May 03 14:14:07 debianv4 chirpstack[25122]: 2025-05-03T18:14:07.231888Z TRACE up{deduplication_id=b9ebb637-f529-48bf-bdd8-497f56e9aecc}:join_request{dev_eui=“008000000002661e”}:join_accept{downlink_id=3332594922}: chirpstack::downlink::join: Setting tx-info for RX2
May 03 14:14:07 debianv4 chirpstack[25122]: 2025-05-03T18:14:07.231901Z TRACE up{deduplication_id=b9ebb637-f529-48bf-bdd8-497f56e9aecc}:join_request{dev_eui=“008000000002661e”}:join_accept{downlink_id=3332594922}: chirpstack::downlink::join: Setting downlink frame
May 03 14:14:07 debianv4 chirpstack[25122]: 2025-05-03T18:14:07.231918Z TRACE up{deduplication_id=b9ebb637-f529-48bf-bdd8-497f56e9aecc}:join_request{dev_eui=“008000000002661e”}:join_accept{downlink_id=3332594922}: chirpstack::downlink::join: Saving downlink frame
May 03 14:14:07 debianv4 chirpstack[25122]: 2025-05-03T18:14:07.233665Z INFO up{deduplication_id=b9ebb637-f529-48bf-bdd8-497f56e9aecc}:join_request{dev_eui=“008000000002661e”}:join_accept{downlink_id=3332594922}: chirpstack::storage::downlink_frame: Downlink-frame saved downlink_id=3332594922
May 03 14:14:07 debianv4 chirpstack[25122]: 2025-05-03T18:14:07.233696Z TRACE up{deduplication_id=b9ebb637-f529-48bf-bdd8-497f56e9aecc}:join_request{dev_eui=“008000000002661e”}:join_accept{downlink_id=3332594922}: chirpstack::downlink::join: Sending join-accept response
May 03 14:14:07 debianv4 chirpstack[25122]: 2025-05-03T18:14:07.233771Z INFO up{deduplication_id=b9ebb637-f529-48bf-bdd8-497f56e9aecc}:join_request{dev_eui=“008000000002661e”}:join_accept{downlink_id=3332594922}: chirpstack::gateway::backend::mqtt: Sending downlink frame region_id=us915_0 gateway_id=00800000a000dc06 topic=us915_0/gateway/00800000a000dc06/command/down json=true
May 03 14:14:07 debianv4 chirpstack[25122]: 2025-05-03T18:14:07.233809Z TRACE up{deduplication_id=b9ebb637-f529-48bf-bdd8-497f56e9aecc}:join_request{dev_eui=“008000000002661e”}:join_accept{downlink_id=3332594922}: chirpstack::gateway::backend::mqtt: Message published
May 03 14:14:07 debianv4 chirpstack[25122]: 2025-05-03T18:14:07.233830Z TRACE up{deduplication_id=b9ebb637-f529-48bf-bdd8-497f56e9aecc}:join_request{dev_eui=“008000000002661e”}: chirpstack::uplink::join: Sending join event
May 03 14:14:07 debianv4 chirpstack[25122]: 2025-05-03T18:14:07.240926Z TRACE chirpstack::gateway::backend::mqtt: MQTT event event=Outgoing(Publish(0))
May 03 14:14:07 debianv4 chirpstack[25122]: 2025-05-03T18:14:07.241865Z INFO chirpstack::integration::mqtt: Publishing event topic=heritage/008000000002661e/up
May 03 14:14:07 debianv4 chirpstack[25122]: 2025-05-03T18:14:07.242707Z TRACE chirpstack::integration::mqtt: MQTT event event=Outgoing(Publish(0))
May 03 14:14:08 debianv4 chirpstack[25122]: 2025-05-03T18:14:08.160387Z TRACE chirpstack::downlink::scheduler: Starting multicast-group queue scheduler loop run
May 03 14:14:08 debianv4 chirpstack[25122]: 2025-05-03T18:14:08.160463Z TRACE chirpstack::downlink::scheduler: Getting schedulable multicast-group queue items
May 03 14:14:08 debianv4 chirpstack[25122]: 2025-05-03T18:14:08.160655Z TRACE chirpstack::downlink::scheduler: Starting class_b_c_scheduler_loop run
May 03 14:14:08 debianv4 chirpstack[25122]: 2025-05-03T18:14:08.160676Z TRACE chirpstack::downlink::scheduler: Getting devices that have schedulable queue-items
May 03 14:14:08 debianv4 chirpstack[25122]: 2025-05-03T18:14:08.172464Z TRACE chirpstack::downlink::scheduler: Got this number of multicast-group queue items count=0
May 03 14:14:08 debianv4 chirpstack[25122]: 2025-05-03T18:14:08.172525Z TRACE chirpstack::downlink::scheduler: Multicast-group queue scheduler run completed successfully
May 03 14:14:08 debianv4 chirpstack[25122]: 2025-05-03T18:14:08.173972Z TRACE chirpstack::downlink::scheduler: Got this number of devices with schedulable queue-items device_count=0