Handle uplink error in V4

zeara · September 7, 2022, 9:58am

i running chirpstack v4.rc4, i can see the data uplinks and loraFrames
but after a while i got the error chirpstack::uplink::data: Handle uplink error error=Get device-session and cannot receive the data from http integration
then after this error log, i got many these logs

Sep 07 08:46:25.535 ERROR up{deduplication_id=35bac6a2-f12e-4f7a-ae19-7b2ef58748ad}: chirpstack::uplink::data: Handle uplink error error=Get device-session
Sep 07 08:46:25.535  INFO chirpstack::gateway::backend::mqtt: Message received from gateway region_name="as923" topic="as923/gateway/24e124fffef45e82/event/up" qos=0 json=true
Sep 07 08:46:25.536  INFO chirpstack::gateway::backend::mqtt: Message received from gateway region_name="as923" topic="as923/gateway/24e124fffef45e82/event/up" qos=0 json=true
Sep 07 08:46:25.536  INFO chirpstack::gateway::backend::mqtt: Message received from gateway region_name="as923" topic="as923/gateway/24e124fffef45e82/event/stats" qos=0 json=true
Sep 07 08:46:25.538  INFO chirpstack::gateway::backend::mqtt: Message received from gateway region_name="as923" topic="as923/gateway/24e124fffef45e82/event/up" qos=0 json=true
Sep 07 08:46:25.538  INFO stats{gateway_id=24e124fffef45e82}: chirpstack::storage::gateway: Gateway state updated gateway_id=24e124fffef45e82
Sep 07 08:46:25.538  INFO chirpstack::gateway::backend::mqtt: Message received from gateway region_name="as923" topic="as923/gateway/24e124fffef45e82/event/up" qos=0 json=true
Sep 07 08:46:25.539  INFO stats{gateway_id=24e124fffef45e82}: chirpstack::storage::metrics: Metrics saved name=gw:24e124fffef45e82 aggregation=HOUR
Sep 07 08:46:25.539  INFO chirpstack::gateway::backend::mqtt: Message received from gateway region_name="as923" topic="as923/gateway/24e124fffef45e82/event/up" qos=0 json=true
Sep 07 08:46:25.539  INFO stats{gateway_id=24e124fffef45e82}: chirpstack::storage::metrics: Metrics saved name=gw:24e124fffef45e82 aggregation=DAY
Sep 07 08:46:25.540  INFO chirpstack::gateway::backend::mqtt: Message received from gateway region_name="as923" topic="as923/gateway/24e124fffef45e82/event/up" qos=0 json=true
Sep 07 08:46:25.540  INFO stats{gateway_id=24e124fffef45e82}: chirpstack::storage::metrics: Metrics saved name=gw:24e124fffef45e82 aggregation=MONTH
Sep 07 08:46:25.738  INFO up{deduplication_id=a4711d02-a694-4e70-8450-ab214edf4480}: chirpstack::uplink: Uplink received m_type="ConfirmedDataUp"
Sep 07 08:46:25.738  INFO up{deduplication_id=efa7f866-f380-4e97-a889-fe3e1786e8d5}: chirpstack::uplink: Uplink received m_type="ConfirmedDataUp"
Sep 07 08:46:25.743  INFO up{deduplication_id=7df18943-47f9-4b63-80f8-957d6eac560d}: chirpstack::uplink: Uplink received m_type="ConfirmedDataUp"
Sep 07 08:46:25.744  INFO up{deduplication_id=8105159f-0b2d-44ae-8a9c-ef52810bdd12}: chirpstack::uplink: Uplink received m_type="ConfirmedDataUp"
Sep 07 08:46:25.744  INFO up{deduplication_id=4be70b80-3dff-42d7-9d27-7607d2925f04}: chirpstack::uplink: Uplink received m_type="ConfirmedDataUp"
Sep 07 08:46:25.744  INFO up{deduplication_id=d588b8cc-69ee-4607-8573-71ff2e7582ef}: chirpstack::uplink: Uplink received m_type="ConfirmedDataUp"
Sep 07 08:46:55.747 ERROR up{deduplication_id=a4711d02-a694-4e70-8450-ab214edf4480}: chirpstack::uplink::data: Handle uplink error error=Get device-session
Sep 07 08:46:55.748 ERROR up{deduplication_id=8105159f-0b2d-44ae-8a9c-ef52810bdd12}: chirpstack::uplink::data: Handle uplink error error=Get device-session
Sep 07 08:47:25.748 ERROR up{deduplication_id=7df18943-47f9-4b63-80f8-957d6eac560d}: chirpstack::uplink::data: Handle uplink error error=Get device-session
Sep 07 08:47:25.748 ERROR up{deduplication_id=d588b8cc-69ee-4607-8573-71ff2e7582ef}: chirpstack::uplink::data: Handle uplink error error=Get device-session
Sep 07 08:47:25.748 ERROR up{deduplication_id=efa7f866-f380-4e97-a889-fe3e1786e8d5}: chirpstack::uplink::data: Handle uplink error error=Get device-session
Sep 07 08:47:25.770  INFO gRPC{uri=/api.InternalService/StreamDeviceEvents}: chirpstack::api: Finished processing request status="200" latency=6.732709ms
Sep 07 08:47:55.749 ERROR chirpstack::gateway::backend::mqtt: Processing gateway event error: timed out waiting for connection topic="as923/gateway/24e124fffef45e82/event/up" qos=0
Sep 07 08:47:55.755 ERROR up{deduplication_id=4be70b80-3dff-42d7-9d27-7607d2925f04}: chirpstack::uplink::data: Handle uplink error error=Get device-session
Sep 07 08:47:57.094  INFO gRPC{uri=/api.InternalService/StreamDeviceEvents}: chirpstack::api: Finished processing request status="200" latency=3.271341ms
Sep 07 08:48:25.749 ERROR chirpstack::gateway::backend::mqtt: Processing gateway event error: timed out waiting for connection topic="as923/gateway/24e124fffef45e82/event/up" qos=0
Sep 07 08:48:27.095 ERROR chirpstack::api::internal: Reading event-log returned error: timed out waiting for connection
Sep 07 08:48:28.811  INFO gRPC{uri=/api.InternalService/StreamDeviceEvents}: chirpstack::api: Finished processing request status="200" latency=1.264578ms
Sep 07 08:48:55.750 ERROR chirpstack::gateway::backend::mqtt: Processing gateway event error: timed out waiting for connection topic="as923/gateway/24e124fffef45e82/event/up" qos=0
Sep 07 08:48:58.811 ERROR chirpstack::api::internal: Reading event-log returned error: timed out waiting for connection

chirpstack can not handle the uplink data after the error occured
but i try to restart chirpstack(only chirpstack, i didn’t restart gateway-bridge), then all works fine for a while; it will accidently get the same problem

anyone face the same problem?
is this a bug?

zeara · September 8, 2022, 1:40am

the same phenomenon(chirpstack can not handle messgae, it looks just like gateway lost connection, but actually it connected), but i got these error
ERROR chirpstack::uplink: Deduplication error error=log_uplink_for_gateways error
data_up: chirpstack::storage::device_session: Get device-session for DevEUI error dev_addr=007338d8 dev_eui=0080e1150518580c error=timed out waiting for connection
ERROR chirpstack::uplink: Deduplication error error=log_uplink_for_gateways error
ERROR chirpstack::uplink: Deduplication error error=Zero items in collect set

Sep 07 13:37:09.596 ERROR chirpstack::uplink: Deduplication error error=log_uplink_for_gateways error
Sep 07 13:37:09.596 ERROR up{deduplication_id=94e10137-0c64-4d07-9a53-0cadecfca442}: chirpstack::uplink::data: Handle uplink error error=Get device-session
Sep 07 13:37:09.596 ERROR up{deduplication_id=b3b51823-fd4d-4d50-ba9d-48a57d37cab0}:data_up: chirpstack::storage::device_session: Get device-session for DevEUI error dev_addr=007338d8 dev_eui=0080e1150518580c error=timed out waiting for connection
Sep 07 13:37:09.596  WARN up{deduplication_id=b3b51823-fd4d-4d50-ba9d-48a57d37cab0}:data_up: chirpstack::uplink::data: No device-session exists for dev_addr dev_addr=007338d8
Sep 07 13:37:09.597 ERROR chirpstack::uplink: Deduplication error error=timed out waiting for connection
Sep 07 13:37:09.597 ERROR chirpstack::uplink: Deduplication error error=log_uplink_for_gateways error
Sep 07 13:37:09.597 ERROR chirpstack::uplink: Deduplication error error=Zero items in collect set
Sep 07 13:37:09.597 ERROR up{deduplication_id=0030dfc3-1f9f-4c3e-b56c-cbfa517d014a}: chirpstack::uplink::data: Handle uplink error error=Get device-session
Sep 07 13:37:09.597 ERROR chirpstack::uplink: Deduplication error error=Zero items in collect set
Sep 07 13:37:09.597 ERROR chirpstack::uplink: Deduplication error error=Zero items in collect set
Sep 07 13:37:09.598 ERROR chirpstack::uplink: Deduplication error error=Zero items in collect set
Sep 07 13:37:09.598  INFO chirpstack::gateway::backend::mqtt: Message received from gateway region_name="as923" topic="as923/gateway/ac1f09fffe015ed3/event/up" qos=0 json=true
Sep 07 13:37:39.598 ERROR chirpstack::uplink: Deduplication error error=timed out waiting for connection
Sep 07 13:37:39.598 ERROR chirpstack::gateway::backend::mqtt: Processing gateway event error: timed out waiting for connection topic="as923/gateway/24e124fffef45e82/event/up" qos=0
Sep 07 13:38:09.598 ERROR chirpstack::gateway::backend::mqtt: Processing gateway event error: timed out waiting for connection topic="as923/gateway/ac1f09fffe015ed3/event/up" qos=0
Sep 07 13:38:11.748  INFO gRPC{uri=/api.InternalService/StreamDeviceEvents}: chirpstack::api: Finished processing request status="200" latency=1.27513ms
Sep 07 13:38:39.599 ERROR chirpstack::gateway::backend::mqtt: Processing gateway event error: timed out waiting for connection topic="as923/gateway/24e124fffef45e82/event/up" qos=0
Sep 07 13:38:41.748 ERROR chirpstack::api::internal: Reading event-log returned error: timed out waiting for connection
Sep 07 13:38:43.420  INFO gRPC{uri=/api.InternalService/StreamDeviceEvents}: chirpstack::api: Finished processing request status="200" latency=1.255849ms
Sep 07 13:39:09.599 ERROR chirpstack::gateway::backend::mqtt: Processing gateway event error: timed out waiting for connection topic="as923/gateway/ac1f09fffe015ed3/event/up" qos=0
Sep 07 13:39:13.420 ERROR chirpstack::api::internal: Reading event-log returned error: timed out waiting for connection
Sep 07 13:39:15.635  INFO gRPC{uri=/api.InternalService/StreamDeviceEvents}: chirpstack::api: Finished processing request status="200" latency=219.637058ms
Sep 07 13:39:39.599 ERROR chirpstack::gateway::backend::mqtt: Processing gateway event error: timed out waiting for connection topic="as923/gateway/24e124fffef45e82/event/up" qos=0
Sep 07 13:39:45.636 ERROR chirpstack::api::internal: Reading event-log returned error: timed out waiting for connection

zeara · September 9, 2022, 5:03am

and i try downgrade the postgresql to 9.6 and redis to 5 according to Server running on raspberrypi, network server log show "zero items in collect set" when device join - #13 by Dan
the error also existed
here are the trace logs

Sep 09 04:27:49.477 DEBUG gRPC{uri=/api.InternalService/StreamDeviceEvents}: chirpstack::api: Started processing request
Sep 09 04:27:49.478 DEBUG gRPC{uri=/api.InternalService/StreamDeviceEvents}: chirpstack::api: Started processing request
Sep 09 04:27:49.479  INFO gRPC{uri=/api.InternalService/StreamDeviceEvents}: chirpstack::api: Finished processing request status="200" latency=1.745125ms
Sep 09 04:27:49.480  INFO gRPC{uri=/api.InternalService/StreamDeviceEvents}: chirpstack::api: Finished processing request status="200" latency=1.96641ms
Sep 09 04:27:49.936 TRACE chirpstack::downlink::scheduler: Starting class_b_c_scheduler_loop run
Sep 09 04:27:49.936 TRACE chirpstack::downlink::scheduler: Getting devices that have schedulable queue-items
Sep 09 04:27:49.938 TRACE chirpstack::downlink::scheduler: Got this number of devices with schedulable queue-items device_count=0
Sep 09 04:27:49.938 TRACE chirpstack::downlink::scheduler: class_b_c_scheduler_loop completed successfully
Sep 09 04:27:50.079 TRACE chirpstack::downlink::scheduler: Starting multicast-group queue scheduler loop run
Sep 09 04:27:50.079 TRACE chirpstack::downlink::scheduler: Getting schedulable multicast-group queue items
Sep 09 04:27:50.080 TRACE chirpstack::downlink::scheduler: Got this number of multicast-group queue items count=0
Sep 09 04:27:50.080 TRACE chirpstack::downlink::scheduler: Multicast-group queue scheduler run completed successfully
Sep 09 04:27:50.512 ERROR chirpstack::uplink: Deduplication error error=timed out waiting for connection
Sep 09 04:27:50.513 ERROR chirpstack::uplink: Deduplication error error=timed out waiting for connection
Sep 09 04:27:50.513 ERROR chirpstack::uplink: Deduplication error error=timed out waiting for connection
Sep 09 04:27:50.513 ERROR chirpstack::uplink: Deduplication error error=timed out waiting for connection
Sep 09 04:27:50.513 ERROR chirpstack::uplink: Deduplication error error=log_uplink_for_gateways error
Sep 09 04:27:50.513 ERROR up{deduplication_id=4e8a7e30-e713-4b78-92f7-fe2f7d268d20}: chirpstack::uplink::data: Handle uplink error error=Get device-session
Sep 09 04:27:50.513 ERROR chirpstack::uplink: Deduplication error error=log_uplink_for_gateways error
Sep 09 04:27:50.513 ERROR chirpstack::uplink: Deduplication error error=Zero items in collect set
Sep 09 04:27:50.514 ERROR chirpstack::uplink: Deduplication error error=Zero items in collect set
Sep 09 04:27:50.514 ERROR chirpstack::uplink: Deduplication error error=Zero items in collect set
Sep 09 04:27:50.514 ERROR chirpstack::uplink: Deduplication error error=Zero items in collect set
Sep 09 04:27:50.514 ERROR chirpstack::uplink: Deduplication error error=Zero items in collect set
Sep 09 04:27:50.514 ERROR chirpstack::uplink: Deduplication error error=Zero items in collect set
Sep 09 04:27:50.515 ERROR chirpstack::uplink: Deduplication error error=Zero items in collect set
Sep 09 04:27:50.515 ERROR chirpstack::uplink: Deduplication error error=Zero items in collect set
Sep 09 04:27:50.515 ERROR chirpstack::uplink: Deduplication error error=Zero items in collect set
Sep 09 04:27:50.515 ERROR chirpstack::uplink: Deduplication error error=Zero items in collect set
Sep 09 04:27:50.515 ERROR chirpstack::uplink: Deduplication error error=Zero items in collect set
Sep 09 04:27:50.516 ERROR chirpstack::uplink: Deduplication error error=Zero items in collect set
Sep 09 04:27:50.515 ERROR chirpstack::uplink: Deduplication error error=Zero items in collect set
Sep 09 04:27:50.516 ERROR chirpstack::uplink: Deduplication error error=Zero items in collect set
Sep 09 04:27:50.516 ERROR chirpstack::uplink: Deduplication error error=Zero items in collect set
Sep 09 04:27:50.516 ERROR chirpstack::uplink: Deduplication error error=Zero items in collect set
Sep 09 04:27:50.516 ERROR chirpstack::uplink: Deduplication error error=Zero items in collect set
Sep 09 04:27:50.516 ERROR chirpstack::uplink: Deduplication error error=Zero items in collect set
Sep 09 04:27:50.517 ERROR chirpstack::uplink: Deduplication error error=Zero items in collect set
Sep 09 04:27:50.517  INFO chirpstack::gateway::backend::mqtt: Message received from gateway region_name="as923" topic="as923/gateway/ac1f09fffe015ed3/event/up" qos=0 json=false
Sep 09 04:27:50.517 TRACE chirpstack::uplink: Adding uplink event to deduplication set key="up:collect:08c0eea7b803120a1a0808c8d00710072801:805350200080b3160a7d8855fc52a222"
Sep 09 04:27:50.517 TRACE chirpstack::eventlog: Event-log received from stream key=status id=1662690659222-0
Sep 09 04:27:50.517 TRACE chirpstack::eventlog: Event-log received from stream key=up id=1662690659229-0
Sep 09 04:27:50.517 TRACE chirpstack::api::internal: Message received from Redis Stream channel
Sep 09 04:27:50.518 TRACE chirpstack::eventlog: Event-log received from stream key=up id=1662690662929-0
Sep 09 04:27:50.518 TRACE chirpstack::api::internal: Message received from Redis Stream channel
Sep 09 04:27:50.518 TRACE chirpstack::eventlog: Event-log received from stream key=up id=1662690666934-0
Sep 09 04:27:50.518 TRACE chirpstack::api::internal: Message received from Redis Stream channel
Sep 09 04:27:50.518 TRACE chirpstack::eventlog: Event-log received from stream key=join id=1662691825952-0
Sep 09 04:27:50.518 TRACE chirpstack::eventlog: Event-log received from stream key=up id=1662691831858-0
Sep 09 04:27:50.518 TRACE chirpstack::api::internal: Message received from Redis Stream channel
Sep 09 04:27:50.518 TRACE chirpstack::api::internal: Message received from Redis Stream channel
Sep 09 04:27:50.519 TRACE chirpstack::eventlog: Event-log received from stream key=status id=1662691835757-0
Sep 09 04:27:50.519 TRACE chirpstack::api::internal: Message received from Redis Stream channel
Sep 09 04:27:50.519 TRACE chirpstack::eventlog: Event-log received from stream key=up id=1662691835764-0
Sep 09 04:27:50.519 TRACE chirpstack::api::internal: Message received from Redis Stream channel
Sep 09 04:27:50.519 TRACE chirpstack::eventlog: Event-log received from stream key=up id=1662691839474-0
Sep 09 04:27:50.519 TRACE chirpstack::api::internal: Message received from Redis Stream channel
Sep 09 04:27:50.519 TRACE chirpstack::eventlog: Event-log received from stream key=up id=1662691843468-0
Sep 09 04:27:50.519 TRACE chirpstack::api::internal: Message received from Redis Stream channel
Sep 09 04:27:50.519 TRACE chirpstack::api::internal: Message received from Redis Stream channel
Sep 09 04:27:50.939 TRACE chirpstack::downlink::scheduler: Starting class_b_c_scheduler_loop run
Sep 09 04:27:50.939 TRACE chirpstack::downlink::scheduler: Getting devices that have schedulable queue-items
Sep 09 04:27:50.941 TRACE chirpstack::downlink::scheduler: Got this number of devices with schedulable queue-items device_count=0
Sep 09 04:27:50.941 TRACE chirpstack::downlink::scheduler: class_b_c_scheduler_loop completed successfully
Sep 09 04:27:51.082 TRACE chirpstack::downlink::scheduler: Starting multicast-group queue scheduler loop run
Sep 09 04:27:51.082 TRACE chirpstack::downlink::scheduler: Getting schedulable multicast-group queue items
Sep 09 04:27:51.083 TRACE chirpstack::downlink::scheduler: Got this number of multicast-group queue items count=0
Sep 09 04:27:51.083 TRACE chirpstack::downlink::scheduler: Multicast-group queue scheduler run completed successfully
Sep 09 04:27:51.941 TRACE chirpstack::downlink::scheduler: Starting class_b_c_scheduler_loop run
Sep 09 04:27:51.941 TRACE chirpstack::downlink::scheduler: Getting devices that have schedulable queue-items
Sep 09 04:27:51.943 TRACE chirpstack::downlink::scheduler: Got this number of devices with schedulable queue-items device_count=0
Sep 09 04:27:51.943 TRACE chirpstack::downlink::scheduler: class_b_c_scheduler_loop completed successfully
Sep 09 04:27:52.084 TRACE chirpstack::downlink::scheduler: Starting multicast-group queue scheduler loop run
Sep 09 04:27:52.084 TRACE chirpstack::downlink::scheduler: Getting schedulable multicast-group queue items
Sep 09 04:27:52.085 TRACE chirpstack::downlink::scheduler: Got this number of multicast-group queue items count=0
Sep 09 04:27:52.085 TRACE chirpstack::downlink::scheduler: Multicast-group queue scheduler run completed successfully
Sep 09 04:27:52.944 TRACE chirpstack::downlink::scheduler: Starting class_b_c_scheduler_loop run
Sep 09 04:27:52.944 TRACE chirpstack::downlink::scheduler: Getting devices that have schedulable queue-items
Sep 09 04:27:52.946 TRACE chirpstack::downlink::scheduler: Got this number of devices with schedulable queue-items device_count=0
Sep 09 04:27:52.946 TRACE chirpstack::downlink::scheduler: class_b_c_scheduler_loop completed successfully
Sep 09 04:27:53.086 TRACE chirpstack::downlink::scheduler: Starting multicast-group queue scheduler loop run
Sep 09 04:27:53.086 TRACE chirpstack::downlink::scheduler: Getting schedulable multicast-group queue items
Sep 09 04:27:53.087 TRACE chirpstack::downlink::scheduler: Got this number of multicast-group queue items count=0
Sep 09 04:27:53.088 TRACE chirpstack::downlink::scheduler: Multicast-group queue scheduler run completed successfully
Sep 09 04:27:53.947 TRACE chirpstack::downlink::scheduler: Starting class_b_c_scheduler_loop run
Sep 09 04:27:53.947 TRACE chirpstack::downlink::scheduler: Getting devices that have schedulable queue-items
Sep 09 04:27:53.949 TRACE chirpstack::downlink::scheduler: Got this number of devices with schedulable queue-items device_count=0
Sep 09 04:27:53.949 TRACE chirpstack::downlink::scheduler: class_b_c_scheduler_loop completed successfully
Sep 09 04:27:54.089 TRACE chirpstack::downlink::scheduler: Starting multicast-group queue scheduler loop run
Sep 09 04:27:54.089 TRACE chirpstack::downlink::scheduler: Getting schedulable multicast-group queue items
Sep 09 04:27:54.090 TRACE chirpstack::downlink::scheduler: Got this number of multicast-group queue items count=0
Sep 09 04:27:54.090 TRACE chirpstack::downlink::scheduler: Multicast-group queue scheduler run completed successfully
Sep 09 04:27:54.951 TRACE chirpstack::downlink::scheduler: Starting class_b_c_scheduler_loop run
Sep 09 04:27:54.951 TRACE chirpstack::downlink::scheduler: Getting devices that have schedulable queue-items
Sep 09 04:27:54.953 TRACE chirpstack::downlink::scheduler: Got this number of devices with schedulable queue-items device_count=0
Sep 09 04:27:54.953 TRACE chirpstack::downlink::scheduler: class_b_c_scheduler_loop completed successfully
Sep 09 04:27:55.091 TRACE chirpstack::downlink::scheduler: Starting multicast-group queue scheduler loop run
Sep 09 04:27:55.091 TRACE chirpstack::downlink::scheduler: Getting schedulable multicast-group queue items
Sep 09 04:27:55.092 TRACE chirpstack::downlink::scheduler: Got this number of multicast-group queue items count=0
Sep 09 04:27:55.092 TRACE chirpstack::downlink::scheduler: Multicast-group queue scheduler run completed successfully
Sep 09 04:27:55.954 TRACE chirpstack::downlink::scheduler: Starting class_b_c_scheduler_loop run
Sep 09 04:27:55.955 TRACE chirpstack::downlink::scheduler: Getting devices that have schedulable queue-items
Sep 09 04:27:55.956 TRACE chirpstack::downlink::scheduler: Got this number of devices with schedulable queue-items device_count=0
Sep 09 04:27:55.956 TRACE chirpstack::downlink::scheduler: class_b_c_scheduler_loop completed successfully
Sep 09 04:27:56.094 TRACE chirpstack::downlink::scheduler: Starting multicast-group queue scheduler loop run
Sep 09 04:27:56.094 TRACE chirpstack::downlink::scheduler: Getting schedulable multicast-group queue items
Sep 09 04:27:56.095 TRACE chirpstack::downlink::scheduler: Got this number of multicast-group queue items count=0
Sep 09 04:27:56.095 TRACE chirpstack::downlink::scheduler: Multicast-group queue scheduler run completed successfully
Sep 09 04:27:56.958 TRACE chirpstack::downlink::scheduler: Starting class_b_c_scheduler_loop run
Sep 09 04:27:56.958 TRACE chirpstack::downlink::scheduler: Getting devices that have schedulable queue-items
Sep 09 04:27:56.960 TRACE chirpstack::downlink::scheduler: Got this number of devices with schedulable queue-items device_count=0
Sep 09 04:27:56.960 TRACE chirpstack::downlink::scheduler: class_b_c_scheduler_loop completed successfully
Sep 09 04:27:57.097 TRACE chirpstack::downlink::scheduler: Starting multicast-group queue scheduler loop run
Sep 09 04:27:57.097 TRACE chirpstack::downlink::scheduler: Getting schedulable multicast-group queue items
Sep 09 04:27:57.098 TRACE chirpstack::downlink::scheduler: Got this number of multicast-group queue items count=0
Sep 09 04:27:57.098 TRACE chirpstack::downlink::scheduler: Multicast-group queue scheduler run completed successfully
Sep 09 04:27:57.960 TRACE chirpstack::downlink::scheduler: Starting class_b_c_scheduler_loop run
Sep 09 04:27:57.960 TRACE chirpstack::downlink::scheduler: Getting devices that have schedulable queue-items
Sep 09 04:27:57.962 TRACE chirpstack::downlink::scheduler: Got this number of devices with schedulable queue-items device_count=0
Sep 09 04:27:57.962 TRACE chirpstack::downlink::scheduler: class_b_c_scheduler_loop completed successfully
Sep 09 04:27:58.100 TRACE chirpstack::downlink::scheduler: Starting multicast-group queue scheduler loop run
Sep 09 04:27:58.100 TRACE chirpstack::downlink::scheduler: Getting schedulable multicast-group queue items
Sep 09 04:27:58.101 TRACE chirpstack::downlink::scheduler: Got this number of multicast-group queue items count=0
Sep 09 04:27:58.101 TRACE chirpstack::downlink::scheduler: Multicast-group queue scheduler run completed successfully
Sep 09 04:27:58.964 TRACE chirpstack::downlink::scheduler: Starting class_b_c_scheduler_loop run
Sep 09 04:27:58.964 TRACE chirpstack::downlink::scheduler: Getting devices that have schedulable queue-items
Sep 09 04:27:58.966 TRACE chirpstack::downlink::scheduler: Got this number of devices with schedulable queue-items device_count=0
Sep 09 04:27:58.966 TRACE chirpstack::downlink::scheduler: class_b_c_scheduler_loop completed successfully
Sep 09 04:27:59.103 TRACE chirpstack::downlink::scheduler: Starting multicast-group queue scheduler loop run
Sep 09 04:27:59.103 TRACE chirpstack::downlink::scheduler: Getting schedulable multicast-group queue items
Sep 09 04:27:59.104 TRACE chirpstack::downlink::scheduler: Got this number of multicast-group queue items count=0
Sep 09 04:27:59.104 TRACE chirpstack::downlink::scheduler: Multicast-group queue scheduler run completed successfully
Sep 09 04:27:59.966 TRACE chirpstack::downlink::scheduler: Starting class_b_c_scheduler_loop run
Sep 09 04:27:59.966 TRACE chirpstack::downlink::scheduler: Getting devices that have schedulable queue-items
Sep 09 04:27:59.968 TRACE chirpstack::downlink::scheduler: Got this number of devices with schedulable queue-items device_count=0
Sep 09 04:27:59.968 TRACE chirpstack::downlink::scheduler: class_b_c_scheduler_loop completed successfully
Sep 09 04:28:00.105 TRACE chirpstack::downlink::scheduler: Starting multicast-group queue scheduler loop run
Sep 09 04:28:00.105 TRACE chirpstack::downlink::scheduler: Getting schedulable multicast-group queue items
Sep 09 04:28:00.106 TRACE chirpstack::downlink::scheduler: Got this number of multicast-group queue items count=0
Sep 09 04:28:00.106 TRACE chirpstack::downlink::scheduler: Multicast-group queue scheduler run completed successfully
Sep 09 04:28:00.969 TRACE chirpstack::downlink::scheduler: Starting class_b_c_scheduler_loop run
Sep 09 04:28:00.969 TRACE chirpstack::downlink::scheduler: Getting devices that have schedulable queue-items
Sep 09 04:28:00.971 TRACE chirpstack::downlink::scheduler: Got this number of devices with schedulable queue-items device_count=0
Sep 09 04:28:00.971 TRACE chirpstack::downlink::scheduler: class_b_c_scheduler_loop completed successfully
Sep 09 04:28:01.107 TRACE chirpstack::downlink::scheduler: Starting multicast-group queue scheduler loop run
Sep 09 04:28:01.107 TRACE chirpstack::downlink::scheduler: Getting schedulable multicast-group queue items
Sep 09 04:28:01.109 TRACE chirpstack::downlink::scheduler: Got this number of multicast-group queue items count=0
Sep 09 04:28:01.109 TRACE chirpstack::downlink::scheduler: Multicast-group queue scheduler run completed successfully
Sep 09 04:28:01.972 TRACE chirpstack::downlink::scheduler: Starting class_b_c_scheduler_loop run
Sep 09 04:28:01.972 TRACE chirpstack::downlink::scheduler: Getting devices that have schedulable queue-items
Sep 09 04:28:01.974 TRACE chirpstack::downlink::scheduler: Got this number of devices with schedulable queue-items device_count=0
Sep 09 04:28:01.974 TRACE chirpstack::downlink::scheduler: class_b_c_scheduler_loop completed successfully
Sep 09 04:28:02.109 TRACE chirpstack::downlink::scheduler: Starting multicast-group queue scheduler loop run
Sep 09 04:28:02.110 TRACE chirpstack::downlink::scheduler: Getting schedulable multicast-group queue items
Sep 09 04:28:02.111 TRACE chirpstack::downlink::scheduler: Got this number of multicast-group queue items count=0
Sep 09 04:28:02.111 TRACE chirpstack::downlink::scheduler: Multicast-group queue scheduler run completed successfully
Sep 09 04:28:02.975 TRACE chirpstack::downlink::scheduler: Starting class_b_c_scheduler_loop run
Sep 09 04:28:02.975 TRACE chirpstack::downlink::scheduler: Getting devices that have schedulable queue-items
Sep 09 04:28:02.977 TRACE chirpstack::downlink::scheduler: Got this number of devices with schedulable queue-items device_count=0
Sep 09 04:28:02.977 TRACE chirpstack::downlink::scheduler: class_b_c_scheduler_loop completed successfully
Sep 09 04:28:03.111 TRACE chirpstack::downlink::scheduler: Starting multicast-group queue scheduler loop run
Sep 09 04:28:03.112 TRACE chirpstack::downlink::scheduler: Getting schedulable multicast-group queue items
Sep 09 04:28:03.113 TRACE chirpstack::downlink::scheduler: Got this number of multicast-group queue items count=0
Sep 09 04:28:03.113 TRACE chirpstack::downlink::scheduler: Multicast-group queue scheduler run completed successfully
Sep 09 04:28:03.979 TRACE chirpstack::downlink::scheduler: Starting class_b_c_scheduler_loop run
Sep 09 04:28:03.979 TRACE chirpstack::downlink::scheduler: Getting devices that have schedulable queue-items
Sep 09 04:28:03.981 TRACE chirpstack::downlink::scheduler: Got this number of devices with schedulable queue-items device_count=0
Sep 09 04:28:03.981 TRACE chirpstack::downlink::scheduler: class_b_c_scheduler_loop completed successfully
Sep 09 04:28:04.115 TRACE chirpstack::downlink::scheduler: Starting multicast-group queue scheduler loop run
Sep 09 04:28:04.115 TRACE chirpstack::downlink::scheduler: Getting schedulable multicast-group queue items
Sep 09 04:28:04.116 TRACE chirpstack::downlink::scheduler: Got this number of multicast-group queue items count=0
Sep 09 04:28:04.116 TRACE chirpstack::downlink::scheduler: Multicast-group queue scheduler run completed successfully
Sep 09 04:28:04.981 TRACE chirpstack::downlink::scheduler: Starting class_b_c_scheduler_loop run
Sep 09 04:28:04.981 TRACE chirpstack::downlink::scheduler: Getting devices that have schedulable queue-items
Sep 09 04:28:04.983 TRACE chirpstack::downlink::scheduler: Got this number of devices with schedulable queue-items device_count=0
Sep 09 04:28:04.983 TRACE chirpstack::downlink::scheduler: class_b_c_scheduler_loop completed successfully
Sep 09 04:28:05.117 TRACE chirpstack::downlink::scheduler: Starting multicast-group queue scheduler loop run
Sep 09 04:28:05.117 TRACE chirpstack::downlink::scheduler: Getting schedulable multicast-group queue items
Sep 09 04:28:05.119 TRACE chirpstack::downlink::scheduler: Got this number of multicast-group queue items count=0
Sep 09 04:28:05.119 TRACE chirpstack::downlink::scheduler: Multicast-group queue scheduler run completed successfully
Sep 09 04:28:05.984 TRACE chirpstack::downlink::scheduler: Starting class_b_c_scheduler_loop run
Sep 09 04:28:05.984 TRACE chirpstack::downlink::scheduler: Getting devices that have schedulable queue-items
Sep 09 04:28:05.986 TRACE chirpstack::downlink::scheduler: Got this number of devices with schedulable queue-items device_count=0
Sep 09 04:28:05.986 TRACE chirpstack::downlink::scheduler: class_b_c_scheduler_loop completed successfully
Sep 09 04:28:06.120 TRACE chirpstack::downlink::scheduler: Starting multicast-group queue scheduler loop run
Sep 09 04:28:06.120 TRACE chirpstack::downlink::scheduler: Getting schedulable multicast-group queue items
Sep 09 04:28:06.121 TRACE chirpstack::downlink::scheduler: Got this number of multicast-group queue items count=0
Sep 09 04:28:06.121 TRACE chirpstack::downlink::scheduler: Multicast-group queue scheduler run completed successfully
Sep 09 04:28:06.988 TRACE chirpstack::downlink::scheduler: Starting class_b_c_scheduler_loop run
Sep 09 04:28:06.988 TRACE chirpstack::downlink::scheduler: Getting devices that have schedulable queue-items
Sep 09 04:28:06.990 TRACE chirpstack::downlink::scheduler: Got this number of devices with schedulable queue-items device_count=0
Sep 09 04:28:06.990 TRACE chirpstack::downlink::scheduler: class_b_c_scheduler_loop completed successfully
Sep 09 04:28:07.122 TRACE chirpstack::downlink::scheduler: Starting multicast-group queue scheduler loop run
Sep 09 04:28:07.122 TRACE chirpstack::downlink::scheduler: Getting schedulable multicast-group queue items
Sep 09 04:28:07.123 TRACE chirpstack::downlink::scheduler: Got this number of multicast-group queue items count=0
Sep 09 04:28:07.123 TRACE chirpstack::downlink::scheduler: Multicast-group queue scheduler run completed successfully
Sep 09 04:28:07.990 TRACE chirpstack::downlink::scheduler: Starting class_b_c_scheduler_loop run
Sep 09 04:28:07.990 TRACE chirpstack::downlink::scheduler: Getting devices that have schedulable queue-items
Sep 09 04:28:07.993 TRACE chirpstack::downlink::scheduler: Got this number of devices with schedulable queue-items device_count=0
Sep 09 04:28:07.993 TRACE chirpstack::downlink::scheduler: class_b_c_scheduler_loop completed successfully
Sep 09 04:28:08.125 TRACE chirpstack::downlink::scheduler: Starting multicast-group queue scheduler loop run
Sep 09 04:28:08.125 TRACE chirpstack::downlink::scheduler: Getting schedulable multicast-group queue items
Sep 09 04:28:08.126 TRACE chirpstack::downlink::scheduler: Got this number of multicast-group queue items count=0
Sep 09 04:28:08.126 TRACE chirpstack::downlink::scheduler: Multicast-group queue scheduler run completed successfully
Sep 09 04:28:08.993 TRACE chirpstack::downlink::scheduler: Starting class_b_c_scheduler_loop run
Sep 09 04:28:08.993 TRACE chirpstack::downlink::scheduler: Getting devices that have schedulable queue-items
Sep 09 04:28:08.995 TRACE chirpstack::downlink::scheduler: Got this number of devices with schedulable queue-items device_count=0
Sep 09 04:28:08.995 TRACE chirpstack::downlink::scheduler: class_b_c_scheduler_loop completed successfully
Sep 09 04:28:09.127 TRACE chirpstack::downlink::scheduler: Starting multicast-group queue scheduler loop run
Sep 09 04:28:09.127 TRACE chirpstack::downlink::scheduler: Getting schedulable multicast-group queue items
Sep 09 04:28:09.129 TRACE chirpstack::downlink::scheduler: Got this number of multicast-group queue items count=0
Sep 09 04:28:09.129 TRACE chirpstack::downlink::scheduler: Multicast-group queue scheduler run completed successfully
Sep 09 04:28:09.996 TRACE chirpstack::downlink::scheduler: Starting class_b_c_scheduler_loop run
Sep 09 04:28:09.996 TRACE chirpstack::downlink::scheduler: Getting devices that have schedulable queue-items
Sep 09 04:28:09.998 TRACE chirpstack::downlink::scheduler: Got this number of devices with schedulable queue-items device_count=0
Sep 09 04:28:09.998 TRACE chirpstack::downlink::scheduler: class_b_c_scheduler_loop completed successfully
Sep 09 04:28:10.129 TRACE chirpstack::downlink::scheduler: Starting multicast-group queue scheduler loop run
Sep 09 04:28:10.129 TRACE chirpstack::downlink::scheduler: Getting schedulable multicast-group queue items
Sep 09 04:28:10.131 TRACE chirpstack::downlink::scheduler: Got this number of multicast-group queue items count=0
Sep 09 04:28:10.131 TRACE chirpstack::downlink::scheduler: Multicast-group queue scheduler run completed successfully
Sep 09 04:28:10.999 TRACE chirpstack::downlink::scheduler: Starting class_b_c_scheduler_loop run
Sep 09 04:28:10.999 TRACE chirpstack::downlink::scheduler: Getting devices that have schedulable queue-items
Sep 09 04:28:11.001 TRACE chirpstack::downlink::scheduler: Got this number of devices with schedulable queue-items device_count=0
Sep 09 04:28:11.001 TRACE chirpstack::downlink::scheduler: class_b_c_scheduler_loop completed successfully
Sep 09 04:28:11.131 TRACE chirpstack::downlink::scheduler: Starting multicast-group queue scheduler loop run
Sep 09 04:28:11.132 TRACE chirpstack::downlink::scheduler: Getting schedulable multicast-group queue items
Sep 09 04:28:11.133 TRACE chirpstack::downlink::scheduler: Got this number of multicast-group queue items count=0
Sep 09 04:28:11.133 TRACE chirpstack::downlink::scheduler: Multicast-group queue scheduler run completed successfully
Sep 09 04:28:12.002 TRACE chirpstack::downlink::scheduler: Starting class_b_c_scheduler_loop run
Sep 09 04:28:12.002 TRACE chirpstack::downlink::scheduler: Getting devices that have schedulable queue-items
Sep 09 04:28:12.004 TRACE chirpstack::downlink::scheduler: Got this number of devices with schedulable queue-items device_count=0
Sep 09 04:28:12.004 TRACE chirpstack::downlink::scheduler: class_b_c_scheduler_loop completed successfully
Sep 09 04:28:12.134 TRACE chirpstack::downlink::scheduler: Starting multicast-group queue scheduler loop run
Sep 09 04:28:12.134 TRACE chirpstack::downlink::scheduler: Getting schedulable multicast-group queue items
Sep 09 04:28:12.135 TRACE chirpstack::downlink::scheduler: Got this number of multicast-group queue items count=0
Sep 09 04:28:12.136 TRACE chirpstack::downlink::scheduler: Multicast-group queue scheduler run completed successfully
Sep 09 04:28:13.005 TRACE chirpstack::downlink::scheduler: Starting class_b_c_scheduler_loop run
Sep 09 04:28:13.005 TRACE chirpstack::downlink::scheduler: Getting devices that have schedulable queue-items
Sep 09 04:28:13.007 TRACE chirpstack::downlink::scheduler: Got this number of devices with schedulable queue-items device_count=0
Sep 09 04:28:13.007 TRACE chirpstack::downlink::scheduler: class_b_c_scheduler_loop completed successfully
Sep 09 04:28:13.137 TRACE chirpstack::downlink::scheduler: Starting multicast-group queue scheduler loop run
Sep 09 04:28:13.137 TRACE chirpstack::downlink::scheduler: Getting schedulable multicast-group queue items
Sep 09 04:28:13.138 TRACE chirpstack::downlink::scheduler: Got this number of multicast-group queue items count=0
Sep 09 04:28:13.138 TRACE chirpstack::downlink::scheduler: Multicast-group queue scheduler run completed successfully
Sep 09 04:28:14.008 TRACE chirpstack::downlink::scheduler: Starting class_b_c_scheduler_loop run
Sep 09 04:28:14.008 TRACE chirpstack::downlink::scheduler: Getting devices that have schedulable queue-items
Sep 09 04:28:14.010 TRACE chirpstack::downlink::scheduler: Got this number of devices with schedulable queue-items device_count=0
Sep 09 04:28:14.010 TRACE chirpstack::downlink::scheduler: class_b_c_scheduler_loop completed successfully
Sep 09 04:28:14.140 TRACE chirpstack::downlink::scheduler: Starting multicast-group queue scheduler loop run
Sep 09 04:28:14.140 TRACE chirpstack::downlink::scheduler: Getting schedulable multicast-group queue items
Sep 09 04:28:14.141 TRACE chirpstack::downlink::scheduler: Got this number of multicast-group queue items count=0
Sep 09 04:28:14.141 TRACE chirpstack::downlink::scheduler: Multicast-group queue scheduler run completed successfully
Sep 09 04:28:15.012 TRACE chirpstack::downlink::scheduler: Starting class_b_c_scheduler_loop run
Sep 09 04:28:15.012 TRACE chirpstack::downlink::scheduler: Getting devices that have schedulable queue-items
Sep 09 04:28:15.014 TRACE chirpstack::downlink::scheduler: Got this number of devices with schedulable queue-items device_count=0
Sep 09 04:28:15.014 TRACE chirpstack::downlink::scheduler: class_b_c_scheduler_loop completed successfully
Sep 09 04:28:15.142 TRACE chirpstack::downlink::scheduler: Starting multicast-group queue scheduler loop run
Sep 09 04:28:15.142 TRACE chirpstack::downlink::scheduler: Getting schedulable multicast-group queue items
Sep 09 04:28:15.143 TRACE chirpstack::downlink::scheduler: Got this number of multicast-group queue items count=0
Sep 09 04:28:15.143 TRACE chirpstack::downlink::scheduler: Multicast-group queue scheduler run completed successfully
Sep 09 04:28:16.015 TRACE chirpstack::downlink::scheduler: Starting class_b_c_scheduler_loop run
Sep 09 04:28:16.015 TRACE chirpstack::downlink::scheduler: Getting devices that have schedulable queue-items
Sep 09 04:28:16.021 TRACE chirpstack::downlink::scheduler: Got this number of devices with schedulable queue-items device_count=0
Sep 09 04:28:16.021 TRACE chirpstack::downlink::scheduler: class_b_c_scheduler_loop completed successfully
Sep 09 04:28:16.145 TRACE chirpstack::downlink::scheduler: Starting multicast-group queue scheduler loop run
Sep 09 04:28:16.145 TRACE chirpstack::downlink::scheduler: Getting schedulable multicast-group queue items
Sep 09 04:28:16.147 TRACE chirpstack::downlink::scheduler: Got this number of multicast-group queue items count=0
Sep 09 04:28:16.147 TRACE chirpstack::downlink::scheduler: Multicast-group queue scheduler run completed successfully
Sep 09 04:28:17.021 TRACE chirpstack::downlink::scheduler: Starting class_b_c_scheduler_loop run
Sep 09 04:28:17.021 TRACE chirpstack::downlink::scheduler: Getting devices that have schedulable queue-items
Sep 09 04:28:17.023 TRACE chirpstack::downlink::scheduler: Got this number of devices with schedulable queue-items device_count=0
Sep 09 04:28:17.023 TRACE chirpstack::downlink::scheduler: class_b_c_scheduler_loop completed successfully
Sep 09 04:28:17.148 TRACE chirpstack::downlink::scheduler: Starting multicast-group queue scheduler loop run
Sep 09 04:28:17.148 TRACE chirpstack::downlink::scheduler: Getting schedulable multicast-group queue items
Sep 09 04:28:17.150 TRACE chirpstack::downlink::scheduler: Got this number of multicast-group queue items count=0
Sep 09 04:28:17.150 TRACE chirpstack::downlink::scheduler: Multicast-group queue scheduler run completed successfully
Sep 09 04:28:18.024 TRACE chirpstack::downlink::scheduler: Starting class_b_c_scheduler_loop run
Sep 09 04:28:18.024 TRACE chirpstack::downlink::scheduler: Getting devices that have schedulable queue-items
Sep 09 04:28:18.026 TRACE chirpstack::downlink::scheduler: Got this number of devices with schedulable queue-items device_count=0
Sep 09 04:28:18.026 TRACE chirpstack::downlink::scheduler: class_b_c_scheduler_loop completed successfully
Sep 09 04:28:18.151 TRACE chirpstack::downlink::scheduler: Starting multicast-group queue scheduler loop run
Sep 09 04:28:18.151 TRACE chirpstack::downlink::scheduler: Getting schedulable multicast-group queue items
Sep 09 04:28:18.152 TRACE chirpstack::downlink::scheduler: Got this number of multicast-group queue items count=0
Sep 09 04:28:18.152 TRACE chirpstack::downlink::scheduler: Multicast-group queue scheduler run completed successfully
Sep 09 04:28:19.028 TRACE chirpstack::downlink::scheduler: Starting class_b_c_scheduler_loop run
Sep 09 04:28:19.028 TRACE chirpstack::downlink::scheduler: Getting devices that have schedulable queue-items
Sep 09 04:28:19.030 TRACE chirpstack::downlink::scheduler: Got this number of devices with schedulable queue-items device_count=0
Sep 09 04:28:19.030 TRACE chirpstack::downlink::scheduler: class_b_c_scheduler_loop completed successfully
Sep 09 04:28:19.153 TRACE chirpstack::downlink::scheduler: Starting multicast-group queue scheduler loop run
Sep 09 04:28:19.153 TRACE chirpstack::downlink::scheduler: Getting schedulable multicast-group queue items
Sep 09 04:28:19.154 TRACE chirpstack::downlink::scheduler: Got this number of multicast-group queue items count=0
Sep 09 04:28:19.154 TRACE chirpstack::downlink::scheduler: Multicast-group queue scheduler run completed successfully
Sep 09 04:28:19.480 ERROR chirpstack::api::internal: Reading event-log returned error: timed out waiting for connection
Sep 09 04:28:19.480 TRACE chirpstack::api::internal: DropReceiver drop method called
Sep 09 04:28:20.030 TRACE chirpstack::downlink::scheduler: Starting class_b_c_scheduler_loop run
Sep 09 04:28:20.030 TRACE chirpstack::downlink::scheduler: Getting devices that have schedulable queue-items
Sep 09 04:28:20.032 TRACE chirpstack::downlink::scheduler: Got this number of devices with schedulable queue-items device_count=0
Sep 09 04:28:20.032 TRACE chirpstack::downlink::scheduler: class_b_c_scheduler_loop completed successfully
Sep 09 04:28:20.156 TRACE chirpstack::downlink::scheduler: Starting multicast-group queue scheduler loop run
Sep 09 04:28:20.156 TRACE chirpstack::downlink::scheduler: Getting schedulable multicast-group queue items
Sep 09 04:28:20.157 TRACE chirpstack::downlink::scheduler: Got this number of multicast-group queue items count=0
Sep 09 04:28:20.157 TRACE chirpstack::downlink::scheduler: Multicast-group queue scheduler run completed successfully
Sep 09 04:28:20.520 ERROR chirpstack::uplink: Deduplication error error=timed out waiting for connection
Sep 09 04:28:20.520 ERROR chirpstack::gateway::backend::mqtt: Processing gateway event error: timed out waiting for connection topic="as923/gateway/ac1f09fffe015ed3/event/up" qos=0

system · December 8, 2022, 5:03am

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.