Help testing ChirpStack v4 (test releases)!

Will this allow me to use Semtech Basic Station to connect to the server?
I need to test the latest SBS on a LoRa Gateway.

The latest release of the Chirpstack 3.x server results in my gateway displaying this error message when connecting using SBS. (Error message is from SBS itself)

“Sep 8 14:16:45 RG1xx29D9CE user.notice lora: 2022-09-08 14:16:45.188 [S2E:WARN] Received ‘dnmsg’ before ‘router_config’ - dropped”

(From the log snippet below)

MHdr=00 JoinEui=70b3:d57e:f000:390a DevEui=1234:aaaa:7890:bb55 DevNonce=32609 MIC=691823539
Sep 8 14:16:45 RG1xx29D9CE user.notice lora: 2022-09-08 14:16:44.934 [AIO:XDEB] [3|WS] > {“msgtype”:“jreq”,“MHdr”:0,“JoinEui”:“70-B3-D5-7E-F0-00-39-0A”,“DevEui”:“12-34-AA-AA-78-90-BB-55”,“DevNonce”:32609,“MIC”:691823539,“RefTime”:0.000000,“DR”:0,“Freq”:903500000,“upinfo”:{“rctx”:0,“xtime”:57420895471269764,“gpstime”:0,“fts”:-1,“rssi”:-46,“snr”:11.75,“rxtime”:1662646604.934659}}
Sep 8 14:16:45 RG1xx29D9CE user.notice lora: 2022-09-08 14:16:44.935 [AIO:XDEB] [3] socket write bytes=299
Sep 8 14:16:45 RG1xx29D9CE user.info event_mon[449]: Event: SDC_E_SCAN
Sep 8 14:16:45 RG1xx29D9CE user.notice lora: 2022-09-08 14:16:45.187 [AIO:XDEB] [3] socket read bytes=242
Sep 8 14:16:45 RG1xx29D9CE user.notice lora: 2022-09-08 14:16:45.188 [AIO:XDEB] [3|WS] < {“msgtype”:“dnmsg”,“DevEui”:“01-01-01-01-01-01-01-01”,“dC”:0,“diid”:58697,“pdu”:“202f7f28b9a2b634b28ce39f5fcc11a4cf”,“priority”:1,“RxDelay”:5,“RX1DR”:10,“RX1Freq”:926900000,“RX2DR”:8,“RX2Freq”:923300000,“xtime”:57420895471269764,“rctx”:0}
Sep 8 14:16:45 RG1xx29D9CE user.notice lora: 2022-09-08 14:16:45.188 [S2E:WARN] Received ‘dnmsg’ before ‘router_config’ - dropped
Sep 8 14:16:45 RG1xx29D9CE user.info php-cgi[3382]: lrd-sdk: LRD_WF_GetSSID():2991 returned SDCERR_FAIL

Let me start by saying I’m not sure who I should be bringing this report to (RAKWireless vs. here). I decided to start here because Chirpstack v4 docs indicate that the v3 gateway bridge should work with Chirpstack v4 in protobuf mode—Am I correct in understanding that backwards compatibility with protobuf formats is expected?

Our problem: We have many RAK7289 gateways running the WisGateOS. I don’t believe–but don’t know for sure–that WisGate uses chirpstack-gateway-bridge internally; however, one way or another they support both Chirpstack’s JSON and protobuf protocols. I have had much success with this gateway and Chirpstack v3.

When upgrading to Chirpstack v4 things have mostly worked. I see uplinks for other nearby devices and JoinRequests from my sensors. I see Chirpstack command a downlink over MQTT, but that is where things end. When the command is received by the gateway, it reports:

Fri Sep 23 15:47:48 2022 daemon.info gwBridge[12210]: Pb txinfo is NULL

Which isn’t that useful, but indicates the protobuf message was somehow not as expected. I did notice that the downlink protobuf definitions have had some changes in the code and there is now a “legacy” structure. I’m still trying to grok how that is involved, but I suspect it is part of the issue.

Does Chirpstack v4 somehow detect old versions of the protocol? Perhaps that check is not sufficient for RAK’s bridge? Should everything be backwards compatible and the bug on RAK’s side? Should this be reported on GitHub?

We are very excited about Chirpstack v4. The new UI is great and everything is very easy to deploy. Thank you for working on and gifting us this work!

@abalmos
Not 100% tested. But Chirpstack V4 requires that the MQTT topic starts with the region.

Chirpstack V3 event_topic_template="gateway/{{ .GatewayID }}/event/{{ .EventType }}"
Chirpstack V4 event_topic_template="eu868/gateway/{{ .GatewayID }}/event/{{ .EventType }}"

On the RAK gateway, In LoRaWAN Network Settings try to add the region to the MQTT Topi Template Setup

e.g. if region is US915_1

V3 V4
gateway/{{eui}}/event/up us915_1/gateway/{{eui}}/event/up
gateway/{{eui}}/command/down us915_1/gateway/{{eui}}/command/down
gateway/{{eui}}/event/ack us915_1/gateway/{{eui}}/event/ack
gateway/{{eui}}/event/stats us915_1/gateway/{{eui}}/event/stats

Hi @beegee_1926

Thanks for the response, I do appreciate it!

Unfortunately, we already have the topics configured that way. Uplink (from sensors not part of this network) and gateway stats work fine. I’m not sure about ACKs because I can’t get anything to join without downlinks.

I am relatively confident that the gateway is receiving the downlink protobuf messages because the gateway log message

Fri Sep 23 15:47:48 2022 daemon.info gwBridge[12210]: Pb txinfo is NULL

has a 1-to-1 correspondence with Chirpstack sending a downlink command over MQTT. Also note that txinfo was part of that message.

Thanks!
Andrew

Only if you are running the latest ChirpStack Gateway Bridge v3. Have you confirmed this?

@brocaar Yikes. The protocol is in fact not backwards compatible. That is pretty unfortunate, because now to move to the new code base, every gateway in the network first needs to be updated.

Thanks for the quick response. I have asked RAK about supporting this here: Can't upgrade to Chirpstack v4 - Support new MQTT protocol - LPWAN Industrial Gateways / Concentrators - RAKwireless Forum

Andrew

Please note that to ease the migation to v4 that the latest ChirpStack Gateway Bridge v3 version is forward compatible with ChirpStack v4 (thus you can upgrade your gateways while continue using v3, and then migrate your NS to ChirpStack v4).

I know that it can be a hassle to upgrade all gateways, but there were a couple of things that I wanted to cleanup in the protocol. Doing this at a major release seemed like the best way to do this :slight_smile:

2 Likes

@brocaar I fully understand and appreciate the cleanup. I hope my last note didn’t come off as a complaint! I was intending it to be a thinking-out-loud on if that will be a blocker to moving everyone over to the new code base. I’m not sure what your intentions are in terms of supporting v3 long-term.

In my case, the RAK firmware does not appear to use your gateway bridge, but rather they have some custom implementation of the protocol. (Maybe it is possible to build the gateway bridge for the device?) As far as I can tell, RAK’s lorasrv binary implements most of the functionality and is closed source. As a workaround, I brought up a Gateway Bridge instance on our server and used the semtech_udp protocol. This has unblocked us for the time being.

Perhaps this issue is not widespread and the RAK design is just annoying? It was easy to migrate our Tektelic Macro gateway by simply updating the Chirpstack Gateway Bridge that runs on it.

Side question: I believe Semtech UDP is not well suited to be routed over the Internet. If using the gateway bridge on the gateway is not an option, is Semtch UDP or Basic Station a better choice?

Hi @brocaar,
i want to pick up that question, what is the best way to migrate from a running production ChirpStack v3 to v4? Will the migration tool support container in the future or do you already have some scripts?
If i can help with beta testing this usecase, let me know.

Best regards and thank you

Note that in some cases, you can load your own packages on the gateway (I’m not sure if this is possible with the RAK firmware), most embedded osses have a package-manager called opkg.

If not, then you could use both the UDP and Basics Station forwarders. There might be some issues with the UDP protocol (I’ve seen issues in combination with using Docker).

@AndreasW79 have you seen: GitHub - chirpstack/chirpstack-v3-to-v4: Utility to migate ChirpStack v3 data into ChirpStack v4. ?

1 Like

Hi @brocaar,

thank you for the reply. Yes i did - but im not sure how to use it with two different docker stacks - but if this is the way i will try to figure it out how to use it with two isolated docker stacks.