ChirpStack service restarts when using CODEC

Hello,
I’m using ChirpStack OS v4.1.1 (full image with latest SWU) with an RPi1 model B (quite old, but in excellent shape) and a RAK2287 Hat. When using no CODEC (uplink decoder), everything works just fine, and trying to send unconfirmed data uplink, I get device LoRaWAN frames and Events updating immediately.

Then if I turn Javascipt CODEC on, either writing my own code, or even just leaving the basic template that opens in the code editor, something strange happens. The dashboard seems to “hang” (not responding) for about a minute every time an uplink is transmitted, and then I finally got the uplink frame showing up in the LoRaWAN frames tab, but nothing in the Events tab, and the payload is not decoded.

After some investigations, I observed that the ChirpStack service actually stops for a few seconds, and then restarts everytime a uplink is received…

Jul 16 21:51:55 raspberrypi user.info chirpstack-concentratord-sx1302[726]: Frame received, uplink_id: 1366906989, count_us: 4094594633, freq: 867700000, bw: 125000, mod: LoRa, dr: SF7
Jul 16 21:51:55 raspberrypi user.info chirpstack-mqtt-forwarder[485]: Received uplink frame, uplink_id: 1366906989
Jul 16 21:51:55 raspberrypi user.info chirpstack-mqtt-forwarder[485]: Sending uplink event, uplink_id: 1366906989, topic: eu868/gateway/0016c001ff1e7a14/event/up
Jul 16 21:51:59 raspberrypi user.err monit[452]: 'chirpstack' process is not running
Jul 16 21:51:59 raspberrypi user.info monit[452]: 'chirpstack' trying to restart
Jul 16 21:51:59 raspberrypi user.info monit[452]: 'chirpstack' start: '/etc/init.d/chirpstack start'
Jul 16 21:52:09 raspberrypi user.info monit[452]: 'chirpstack' process is running with pid 2082
Jul 16 21:52:21 raspberrypi user.err chirpstack-concentratord-sx1302[726]: Get concentrator temperature error, error: lgw_get_temperature failed
Jul 16 21:52:21 raspberrypi user.info chirpstack-concentratord-sx1302[726]: Publishing stats event, rx_received: 1, rx_received_ok: 1, tx_received: 0, tx_emitted: 0
Jul 16 21:52:21 raspberrypi user.info chirpstack-mqtt-forwarder[485]: Received gateway stats
Jul 16 21:52:21 raspberrypi user.info chirpstack-mqtt-forwarder[485]: Sending gateway stats event, topic: eu868/gateway/0016c001ff1e7a14/event/stats

If I keep checking monit status (like crazy), I get this for a couple of seconds (everytime an uplink is received):

Process 'chirpstack'
  status                       Does not exist
  monitoring status            Monitored
  monitoring mode              active
  on reboot                    start
  data collected               Sun, 16 Jul 2023 21:59:41

Then right after

Process 'chirpstack'
  status                       Initializing
  monitoring status            Initializing
  monitoring mode              active
  on reboot                    start
  data collected               Sun, 16 Jul 2023 21:59:41

And finally

Process 'chirpstack'
  status                       OK
  monitoring status            Monitored
  monitoring mode              active
  on reboot                    start
  pid                          2228
  parent pid                   1
...

I really doubt that the cause is the CODEC javascipt code. I’ve tried many ways to write my simple decoder according to the v4 rules, and even the basic template produces this behavior:

function decodeUplink(input) {
  return {
    data: {
      temp: 22.5
    }
  };
}

BTW, I can decode my payloads with NodeRED very well if I deactivate the CODEC.

Any ideas ? Since the dashboard hangs, I can get no clue from the event log…

Thanks !

That is odd… Could you stop chirpstack and run it manually? For GW OS < v4.2.0 this is the command to use (for v4.2.0 and up it has changed):

chirpstack -c /etc/chirpstack

Thanks Brocaar,

I first stopped chirpstack using

sudo /etc/init.d/chirpstack stop

Then after I monit told me chirpstack was actually off, I entered your suggested command, and what I got is longer than what I can put here…

Some lines that might ring a bell

...
2023-07-24T13:37:54.461383Z  INFO chirpstack::gateway::backend: Setting up gateway backend for region region_id=eu868 region_common_name=EU868
2023-07-24T13:37:54.460383Z  INFO chirpstack::gateway::backend::mqtt: Starting MQTT consumer loop
2023-07-24T13:37:54.468374Z  INFO chirpstack::gateway::backend::mqtt: Connected to MQTT broker region_config_id=eu433
2023-07-24T13:37:54.469995Z  INFO chirpstack::gateway::backend::mqtt: Subscribing to gateway event topic region_config_id=eu433 event_topic=eu433/gateway/+/event/+
2023-07-24T13:37:54.477827Z  INFO chirpstack::gateway::backend::mqtt: Connecting to MQTT broker region_config_id=eu868 server_uri=tcp://localhost:1883 clean_session=true client_id=356991090b05a1e2
2023-07-24T13:37:56.581974Z  INFO chirpstack::gateway::backend::mqtt: Connected to MQTT broker region_config_id=eu868
...
2023-07-24T13:37:56.583325Z  INFO chirpstack::gateway::backend: Setting up gateway backend for region region_id=us915_7 region_common_name=US915
2023-07-24T13:37:56.583582Z  INFO chirpstack::gateway::backend::mqtt: Starting MQTT consumer loop
2023-07-24T13:37:56.586984Z  INFO chirpstack::gateway::backend::mqtt: Subscribing to gateway event topic region_config_id=eu868 event_topic=eu868/gateway/+/event/+
2023-07-24T13:37:56.593702Z  INFO chirpstack::gateway::backend::mqtt: Connecting to MQTT broker region_config_id=us915_7 server_uri=tcp://localhost:1883 clean_session=true client_id=734efa92fa503878
thread 'tokio-runtime-worker' panicked at 'called `Option::unwrap()` on a `None` value', chirpstack/src/storage/mod.rs:235:32
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
2023-07-24T13:37:57.561628Z ERROR chirpstack::gateway::backend::mqtt: Processing gateway event error: task 24 panicked topic="eu868/gateway/0016c001ff1e7a14/event/up" qos=0
...
2023-07-24T13:38:32.421552Z  INFO chirpstack::storage: Applying schema migrations
2023-07-24T13:38:32.459578Z  INFO chirpstack::storage: Setting up Redis client
2023-07-24T13:38:32.948193Z  INFO chirpstack::downlink: Setting up Class-B/C scheduler loop
2023-07-24T13:38:32.950055Z  INFO chirpstack::downlink: Setting up multicast scheduler loop
2023-07-24T13:38:32.957419Z  INFO chirpstack::api: Setting up API interface bind=0.0.0.0:8080
thread '2023-07-24T13:38:32.972586Z  WARN chirpstack::api::backend: Backend interfaces API is disabled
main' panicked at 'error binding to 0.0.0.0:8080: error creating server listener: Address already in use (os error 98)', /usr/src/debug/chirpstack/4.3.0-r0/cargo_home/bitbake/hyper-0.14.24/src/server/server.rs:79:13

Yet monit status says everything is OK.

It seems Chirpstack is setting up MQTT topics for ALL regions whatever my channel plan (eu868 only). Is that expected ?

Also, I tried my CODEC on an RPi3 last WE (Same HAT, Chirpstack OS 4.1.1 full image), and it works well.

Thanks for your help.

You probably want to stop it using monit stop ..., because monit will else automatically restart the process as it detects it is no longer running. This is probably why you see:

main' panicked at 'error binding to 0.0.0.0:8080: 

Have you tried the GW OS v4.2.0-test.1 image? Would you be able to see if you can reproduce the issue with that image?

Hello @brocaar

Not sure what to download, and maybe a lead to the problem. My RPi is exactly an RPi 1 model B+ V1.2. It seems the SOC is a BCM2835.

From your download repo, I can see that support goes only for BCM2708 or BCM2709. Isn’t there a possible explanation ?

If I’m wrong, could you please suggest what image I should try and provide me with a link ?

Thanks.

Please see this page:
https://www.chirpstack.io/docs/chirpstack-gateway-os/install/raspberry-pi.html

(The bcm27xx/bcm2708 platform configuration in OpenWrt supports the following models:
Raspberry Pi B/B+/CM/Zero/ZeroW)

This topic was automatically closed after 90 days. New replies are no longer allowed.