Frames stuck in queue due to security sync

ChirpStack Network Server
1

I ran into an issue where a frame was stuck in the downlink queue for a device, and the following messages kept repeating over and over. If the device rejoins and gets a new DevAddr, but a message is still in the queue for its old DevAddr, is that what is going on here?

If the DevAddr for a frame in the queue is invalid, should it be dropped?

chirpstack-networkserver-67d6764664-nj6zf      Sep 3, 2021 @ 17:20:42.077	time="2021-09-03T17:20:42.077255771Z" level=warning msg="finished client unary call" ctx_id=59bf9abb-a423-473b-96b2-6906a42b6421 error="rpc error: code = FailedPrecondition desc = devaddr does not match, the security-context is out of sync" grpc.code=FailedPrecondition grpc.ctx_id=555bc323-4707-45f5-8566-b7274094eee4 grpc.duration=191.97887ms grpc.method=ReEncryptDeviceQueueItems grpc.service=as.ApplicationServerService span.kind=client system=grpc
chirpstack-networkserver-67d6764664-nj6zf      Sep 3, 2021 @ 17:20:42.077	time="2021-09-03T17:20:42.077337963Z" level=error msg="schedule next device-queue item error" ctx_id=59bf9abb-a423-473b-96b2-6906a42b6421 dev_eui=aaaaaa0001000033 error="application-server client error: rpc error: code = FailedPrecondition desc = devaddr does not match, the security-context is out of sync"
chirpstack-applicationserver-65b9c59fbf-8s48k  Sep 3, 2021 @ 17:20:42.075	time="2021-09-03T17:20:42.075522770Z" level=warning msg="finished unary call with code FailedPrecondition" ctx_id=555bc323-4707-45f5-8566-b7274094eee4 error="rpc error: code = FailedPrecondition desc = devaddr does not match, the security-context is out of sync" grpc.code=FailedPrecondition grpc.method=ReEncryptDeviceQueueItems grpc.service=as.ApplicationServerService grpc.start_time="2021-09-03T17:20:41Z" grpc.time_ms=179.133 peer.address="127.0.0.1:41240" span.kind=server system=grpc
2

You will see the out-of-sync error when a device re-joined but not yet sent its first uplink, in which case the new security context was not yet sent to the AS. As the AS receives the new context after the first device uplink, in this case the AS tries to enqueue a downlink using outdated session-keys (and DevAddr).

1 Like