Working V4 server failed and no longer starts - error: Setup MQTT Integration

My Chirpstack V4 server has been running since the end of Nov 2022 without any issues and then today I saw no messages being received from it on my mqtt broker.

I have rebooted but see in the logs an issue caused by setup mqtt integration.

I am at a loss as no config changes have been made and I cannot see what is wrong.

Help very much appreciated!!

Jan 07 14:08:57 chirpstack chirpstack[837]: 2023-01-07T14:08:57.943196Z  INFO chirpstack::backend::joinserver: Setting up Join Server clients
Jan 07 14:08:57 chirpstack chirpstack[837]: 2023-01-07T14:08:57.943203Z  INFO chirpstack::backend::roaming: Setting up roaming clients
Jan 07 14:08:57 chirpstack chirpstack[837]: 2023-01-07T14:08:57.943207Z  INFO chirpstack::adr: Setting up adr algorithms
Jan 07 14:08:57 chirpstack chirpstack[837]: 2023-01-07T14:08:57.943223Z  INFO chirpstack::integration: Setting up global integrations
Jan 07 14:08:57 chirpstack chirpstack[837]: 2023-01-07T14:08:57.943229Z  INFO chirpstack::integration::redis: Initializing Redis integration
Jan 07 14:08:57 chirpstack chirpstack[837]: 2023-01-07T14:08:57.943242Z  INFO chirpstack::integration::mqtt: Initializing MQTT integration
Jan 07 14:08:57 chirpstack chirpstack[837]: Error: Setup MQTT integration
Jan 07 14:08:57 chirpstack chirpstack[837]: Caused by:
Jan 07 14:08:57 chirpstack chirpstack[837]:     0: Create MQTT client
Jan 07 14:08:57 chirpstack chirpstack[837]:     1: Persistence error
Jan 07 14:08:58 chirpstack systemd[1]: chirpstack.service: Scheduled restart job, restart counter is at 5.
Jan 07 14:08:58 chirpstack systemd[1]: Stopped ChirpStack open-source LoRaWAN Network Server.
Jan 07 14:08:58 chirpstack systemd[1]: chirpstack.service: Start request repeated too quickly.
Jan 07 14:08:58 chirpstack systemd[1]: chirpstack.service: Failed with result 'exit-code'.
Jan 07 14:08:58 chirpstack systemd[1]: Failed to start ChirpStack open-source LoRaWAN Network Server.
maurice@chirpstack:~$ sudo systemctl status chirpstack
× chirpstack.service - ChirpStack open-source LoRaWAN Network Server
     Loaded: loaded (/lib/systemd/system/chirpstack.service; enabled; vendor preset: enabled)
     Active: failed (Result: exit-code) since Sat 2023-01-07 14:34:38 GMT; 7min ago
       Docs: https://www.chirpstack.io/
    Process: 1358 ExecStart=/usr/bin/chirpstack -c /etc/chirpstack/ (code=exited, status=1/FAILURE)
   Main PID: 1358 (code=exited, status=1/FAILURE)
        CPU: 18ms

Jan 07 14:34:38 chirpstack systemd[1]: chirpstack.service: Scheduled restart job, restart counter is at 5.
Jan 07 14:34:38 chirpstack systemd[1]: Stopped ChirpStack open-source LoRaWAN Network Server.
Jan 07 14:34:38 chirpstack systemd[1]: chirpstack.service: Start request repeated too quickly.
Jan 07 14:34:38 chirpstack systemd[1]: chirpstack.service: Failed with result 'exit-code'.
Jan 07 14:34:38 chirpstack systemd[1]: Failed to start ChirpStack open-source LoRaWAN Network Server.
maurice@chirpstack:~$

Look into the log; it’s the place to find reasons for any errors.

Duh my problem, I ran out of disk space

1 Like