Login issue with Chirpstack V4

Hi, Thanks for all the work to release Chirpstack v4.
I have been using V3 OK for a while and tried to install V4 on a new VPS running both Debian 11 and Ubuntu 22.04 and tried the migration process with a clone of my running Chirpstack V3 server but I am getting the same issue with the login on all install attempts.

I followed the doc to install and configure OK and completed with no errors but when I try to login it gets a “Incomplete Response” error and makes Chirpstack crash and restart.

If I try to login with an invalid user, the web interface correctly says “invalid username or password” and does not cause the crash and restart but any valid user login attempt (including the default admin/admin) causes the same “incomplete response” error

The log shows

Aug 15 17:55:59 npchirp2 kernel: [ 562.895964] traps: tokio-runtime-w[1029] trap invalid opcode ip:55b7348d9ef0 sp:7f947b9fa040 error:0 in chirpstack[55b733b51000+1064000]
Aug 15 17:55:59 npchirp2 systemd[1]: chirpstack.service: Main process exited, code=killed, status=4/ILL
Aug 15 17:55:59 npchirp2 systemd[1]: chirpstack.service: Failed with result ‘signal’.
Aug 15 17:56:00 npchirp2 systemd[1]: chirpstack.service: Service RestartSec=100ms expired, scheduling restart.
Aug 15 17:56:00 npchirp2 systemd[1]: chirpstack.service: Scheduled restart job, restart counter is at 1.
Aug 15 17:56:00 npchirp2 systemd[1]: Stopped ChirpStack open-source LoRaWAN Network Server.
Aug 15 17:56:00 npchirp2 systemd[1]: Started ChirpStack open-source LoRaWAN Network Server.

Given this does not occur with an invalid user/pwd, I am guessing it is not the DB lookup but something that happens directly afterwards with a valid login

Can you give me any clues what may be going on with this?


That is very odd. Which hosting / cloud provider are you using?

Thanks for the reply
Network Presence which is an Australian company which I am also part of and very standard KVM VPS system which is also where I run the Chirpstack V3 server OK and a wide range of other Linux VPS machines on different distros over many years.
The same issue happens on several clean Debian 11 and Ubuntu22.04 installs and on a clone of the older V3 server I tried to migrate to migrate on Debian 10. The existing V3 server was upgraded OK as well before trying to migrate. I was not able to find much on searching the trap error either

I forgot to add that the V3 server clone work OK with the chirpstack-application-server but when I install V4 and stop the application-server and launch the V4 chirpstack that the issue appears again

The reason why I’m asking is that I can not reproduce this issue locally or on my own testing VM instance. I just setup the latest ChirpStack v4 build on a Debian 11 VM and it runs fine.

Would you be able to provide me with a test VM on Network Presence so that I can reproduce this issue and find out what triggers this? If I understand correctly, the issue already exists before the data migration, when logging in using admin / admin, so it doesn’t require any of your device data on it.

If this is possible, please send me a private message to exchange the hostname & credentials and I’ll look into this asap.

Thanks for reporting!

I am happy to reimage my test instance with debian 11 so you have a clean start and send you credentials but I am unsure how to send a private message on this forum system :slight_smile:

The issue exists from the start after installing V4 and trying to login for the first time. I may be just missing something in the install but did follow the doc for the install.

Please advise how I can send a private message and I will forward you login credentials.

Sorry, I found the private message function but it said I was not allowed to send a message to your user


I just sent you a PM :slight_smile:


Thanks to the VM that @amac provided, I have been able to find what has caused this issue. The error happens at the execution of this line:

The pbkdf2 crate depends on sha2, which if available (runtime check) uses CPU instructions for crypto operations or else uses a software implementation. My assumption is that it thinks the CPU instructions are available, but once these are executed it fails.

I have confirmed that once the software implementation is forced, all works fine :slight_smile:

An other update, this issue happened on a VM using a Qemu CPU. The sha2 library does a runtime check if the avx2 CPU extension is available so that the the library can use these CPU instructions over a software implementation of these instructions:

It looks like the (emulated) CPU does advertise the presence of avx2 as documented here (which is checked by the cpufeatures library):


But it doesn’t actually support the avx2 extension instructions, thus when sha2 starts using these instructions, the execution panics because of the unsupported instructions.

This was a rather deep dive :slight_smile:

Many thanks for all the work to debug this rather esoteric issue.


Actually, I believe it is a bug and I created a pull-request to fix this:

1 Like

hi @brocaar ,

I seem to have ran into this problem on a fresh docker install.
The installation runs fine but the application server crashes and restarts the moment I try to log in.
I think the docker images are still on 4.0.0. Can you update to 4.0.1 or is there another way for me to update?

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.