Azure IOT Hub negotiate claim error

Currently have our chirpstack setup in docker swarm.
Redeployed my chirpstack network server container yesterday and its now going into a boot loop.

This network server follows the Azure QuickStart guide and utilizes an Azure IOT hub and Service Bus.
The error its giving does not seem to have much info on the net but seems to maybe be a dependency issue with AQMP.

This network server had been running fine for 8 months.

More context on what i tried

  • switching to secondary keys for the events_connection_string and commands_connection_string
  • Regenerated the primary keys and tested with those
  • Tested the connection strings outside chirpstack and they seem to work fine.

No modification was done to the network server toml before the reboot so this is very strange.
The redeploy did update the version but even reverting that did not resolve the issue.

Would appreciate any assistance on the matter.

Error
time=“2022-08-09T20:12:07.35505132Z” level=info msg=“gateway/azure_iot_hub: starting queue consumer” queue=us915-gateway-events
time=“2022-08-09T20:12:07.443098005Z” level=info msg=“gateway/azure_iot_hub: negotiating amqp cbs claim”
time=“2022-08-09T20:12:07.446436404Z” level=fatal msg=“gateway-backend setup failed: new c2d session and link error: cbs negotiate claim error: protocol error: received flow without next-incoming-id after session established”

I woke up to the same exact issue today. Same setup: Azure / IoT / Service Bus. Same error message. It’s been running fine for months. Need help.

Thanks @ralessi for confirming you also had the issue.

Since i am not the only one, I created this issue on GitHub regarding the issue.