New install. Working but with errors

ChirpStack (v4)
1

Hi! I have just installed a new setup of Chirpstack v4-10 on an Ubuntu Server (cloud) in order to later migrate an existing setup I have running on a R-PI locally.
All installation went fine, except I had port set to 8080 and I already had a local instance of Thingsboard running on that port, so after changing this on .toml file, it started ok
But following all steps on the doc, running

journalctl -f -n 100 -u chirpstack

I got all these errors non stop:

Nov 04 20:46:11 ubuntu chirpstack[1123411]: 2024-11-04T20:46:11.896549Z ERROR chirpstack::integration::mqtt: MQTT error error=Connection refused, return code: ClientIdentifierNotValid

Can you identify these errors? Google can’t help much.
Thanks a lot!

2

Hello again,

Did you make any changes to the Chirpstack.toml besides the API bind (port swap)? Or perhaps to the mosquitto listener settings?

The -

seems to imply the error is not with the backend, but with the integration section. To narrow it down maybe try removing mqtt from the enabled integrations array in your chirpstack.toml, restart Chirpstack, and see if that causes the errors to go away.

3

Good to see you again yes :slight_smile:
No, all I changed was the port. Nothing in the mosquitto section.
This may help as I realized that there are actually 2 errors alternating:

Nov 04 21:25:21 ubuntu chirpstack[1123573]: 2024-11-04T21:25:21.808975Z ERROR chirpstack::gateway::backend::mqtt: MQTT error error=Connection refused, return code: ClientIdentifierNotValid

Nov 04 21:25:22 ubuntu chirpstack[1123573]: 2024-11-04T21:25:22.210968Z ERROR chirpstack::integration::mqtt: MQTT error error=Connection refused, return code: ClientIdentifierNotValid

Also I’m trying to setup a gateway (Dragino LPS8) and either I forgot how to do it , or it’s not communicatiig, probably due to this error? I opened port 1700 UDP on the cloud management and added semtech UDP section with server URL and port 1700…Neither chirpstack sees the gateway nor the dragino shows the active LoRaWAN green check… :frowning:

4

Ah that is good to know, it is failing on both the backend and the integration.

Definitely this error, Chirpstack cannot connect to your MQTT broker at all. A weird issue for a fresh install. MQTT is the only way for the gateway to communicate to Chirpstack so without it it cannot connect at all.

If you check the status of your MQTT broker is it running? Perhaps another port conflict or something similar caused it to exit out on boot.

5

Thanks, at this point Im starting to reach my knowledge limit LOL…
Most of my install has been done following the guidelines at the Ubuntu/Debian section, but TBH not really fully understanding all the steps.
In these steps I don’t see any mention to installing anything realted to MQTT, other than this section:

Configuration

The configuration file is located at /etc/chirpstack-gateway-bridge/chirpstack-gateway-bridge.toml. Please update the [integration.mqtt] section to match the region prefix for the region that applies to this ChirpStack Gateway Bridge instance.

Example for EU868:

[integration.mqtt]
event_topic_template="eu868/gateway/{{ .GatewayID }}/event/{{ .EventType }}"
command_topic_template="eu868/gateway/{{ .GatewayID }}/command/#"

In this section I made sure to add eu868 to each of those lines as noted, and that’s all I’ve done.
Did I miss any step?
Update: I found mosquitto is installed but the guide doesn’t provide any specific configuration for it, and my journald for mosquitto shows:
Nov 04 21:41:58 ubuntu systemd[1]: Stopped Mosquitto MQTT Broker.
░░ Subject: A stop job for unit mosquitto.service has finished
░░ Defined-By: systemd
░░ Support: Enterprise open source support | Ubuntu
░░
░░ A stop job for unit mosquitto.service has finished.
░░
░░ The job identifier is 78144 and the job result is done.
Nov 04 21:41:58 ubuntu systemd[1]: mosquitto.service: Start request repeated too quickly.
Nov 04 21:41:58 ubuntu systemd[1]: mosquitto.service: Failed with result ‘exit-code’.
░░ Subject: Unit failed
░░ Defined-By: systemd
░░ Support: Enterprise open source support | Ubuntu
░░
░░ The unit mosquitto.service has entered the ‘failed’ state with result ‘exit-code’.
Nov 04 21:41:58 ubuntu systemd[1]: Failed to start Mosquitto MQTT Broker.
░░ Subject: A start job for unit mosquitto.service has failed
░░ Defined-By: systemd
░░ Support: Enterprise open source support | Ubuntu
░░
░░ A start job for unit mosquitto.service has finished with a failure.
░░
░░ The job identifier is 78144 and the job result is failed.
Thanks

6

I found this:
root@ubuntu:/etc/mosquitto/conf.d# tail //var/log/mosquitto/mosquitto.log
1730756517: Error: Address already in use
1730756518: mosquitto version 2.0.11 starting
1730756518: Config loaded from /etc/mosquitto/mosquitto.conf.
1730756518: Starting in local only mode. Connections will only be possible from clients running on this machine.
1730756518: Create a configuration file which defines a listener to allow remote access.
1730756518: For more details see Authentication methods | Eclipse Mosquitto
1730756518: Opening ipv4 listen socket on port 1883.
1730756518: Error: Address already in use
1730756518: Opening ipv6 listen socket on port 1883.
1730756518: Error: Address already in use

So I checked what process is using port 1883 and found it is Thingsboard…
249472 ? Sl 1711:05 /usr/bin/java -Dsun.misc.URLClassPath.disableJarChecking=true -Dplatform=deb -Dinstall.data_dir=/usr/share/thingsboard/data -Xlog:gc*,heap*,age*,safepoint=debug:file=/var/log/thingsboard/gc.log:time,uptime,level,tags:filecount=10,filesize=10M -XX:+IgnoreUnrecognizedVMOptions -XX:+HeapDumpOnOutOfMemoryError -XX:-UseBiasedLocking -XX:+UseTLAB -XX:+ResizeTLAB -XX:+PerfDisableSharedMem -XX:+UseCondCardMark -XX:+UseG1GC -XX:MaxGCPauseMillis=500 -XX:+UseStringDeduplication -XX:+ParallelRefProcEnabled -XX:MaxTenuringThreshold=10 -Xms2G -Xmx2G -jar /usr/share/thingsboard/bin/thingsboard.jar

Is Chirpstack incompatible to coexist with Thingsboard?
Thanks

7

Interesting, as I thought it was exited with failure.

As an aside, you downloaded the MQTT broker with the first command of the guide:

sudo apt install
mosquitto
mosquitto-clients
redis-server
redis-tools
postgresql

Likely this is just another port conflict, as it is saying the address is already in use. Strange that 1883 would be being used already but perhaps whatever you are hosting your server on has a broker of its own? (seems suspicious)

I haven’t used the Ubuntu install in over a year but you should find a file called listeners.conf or mosquitto.conf, likely in /etc/mosquitto if that is a directory. Change the listener in there to 1884 and see if it solves your issues.

8

Ah perfect, they are compatible, either change MQTT port or thingsboards port and you should be good to go. If you can change thingsboards port it would be easier as you would not have to change your Chirpstack.toml and gateway bridges to point at the new MQTT port, but either is fine, my mosquitto currently runs on 1884 as well.

9

Ok, yes I found the thingsboard.yml conf file adding 1883 as mqtt port…

MQTT bind port

bind_port: "${MQTT_BIND_PORT:1883}"

At this point, like I mentioned on the other post…Im a little scared to screw up something… If I change Thingsboard bind port 1883 to 1884 do I have to change it anywhere else to point to this new port, or is this connection not in use normally for TB? I don’t want to take any risk as you know…
Well I guess the procedure would be change .yml file to bind 1884, systemclt restart thingsboard, test everything, and if something fails, revert change and restart TB again right?

10

I’m not sure, I’ve never used thingsboard before. Do you use thingsboard for something? If not it doesn’t really matter what you do to it, it is not part of Chirpstack.

Regardless -

This will never break anything and would be the correct way to test. You’ll likely have to restart mosquitto as well now that the port is cleared.

Again, I’m not certain, but it doesn’t seem likely to me. If I had to wager I’d say this is simply for an MQTT integration, similar to Chirpstacks, where you have the option to view the thingsboard data or interact with it over MQTT (many services have their own MQTT broker for this reason), so if you are not using those features currently you have nothing to worry about in changing that port, but if something does go wrong you can always change it back. If I’m right, I’d also assume there is a way to disable the MQTT broker, that would likely be even better as you won’t have it running for no reason on your server.

11

Yes, Thingsboard if my other critical part for this system, it’s my IoT platform which receives the uplink messages from CS, processes them to show them on nice panels, and generates the downlink resposes when needed to be sent back to CS and this in charge to the device. So CS+TB is my full setup.
Ok I will try it now…
Thanks

12

This also solves what I was confused about initially, if mosquitto was simply down it would have given you a connection refused: connection failed, error but it gave you a clientidentifiernotvalid, that is because it was communicating with a broker, just the wrong one.

13

But your clients do not connect over MQTT to thingsboard correct? They use the webUI 8080? (which is why that port was taken) If so I’d assume you can disable the whole broker, I’ll look into it and get back to you.

I looked into it, the MQTT broker with Thingsboard is optional, and from what you have explained of your use case you don’t need it.

If changing the port doesn’t help, or you decide you don’t want this extra overhead and potential vulnerability. Try setting tb-mqtt-broker.enabled: false in the thingsboard.yml file. It will disable ThingsBoard from starting its internal MQTT broker (TBMQ) entirely. This setting stops the broker component itself, rather than merely stopping ThingsBoard from trying to communicate with it.

14

Ok I tried it and after restarting both TB and CS services it all works great, I can now see my gateway under CS, I have no more errors under journald, and I still see my thingsboard service running perfectly fine.
So hopefully this thread can serve for future reference to anyone trying to put Thingsboard and Chirpstack together in the same server.
THANKS!!

15

Gotchu, glad you got it working. I would still consider disabling that broker entirely though.

16

Understood, I will leave that option for the future, right now Im in the mindset “if it works, don’t touch it”, just in case I need some new ability in the future that requires MQTT integration.
:+1: :+1: :+1: :+1: :+1:

17

A good mindset :wink: Hope all works well for you and the town doesn’t die of thirst!

1 Like