"context deadline exceeded" error

Hello,

I am starting a new discussion of this topic although it has been discussed previously, mainly because the past discussions are old and are somewhat confusing.

The symptom is that a clean install of ChirpStack on a clean Raspberry Pi OS results in inability to create any components; through the web UI an attempt to e.g. define a network server results in a “Context Deadline Exceeded” error a few seconds after “Add Network Server.” The network server log contains an error like:

Sep 08 22:17:43 raspberrypi chirpstack-network-server[550]:
    time="2020-09-08T22:17:43+01:00" level=error msg="gateway:
    handle gateway stats error" ctx_id=58fc360d-a9d2-4781-90f7-dd5d1933679c
    error="get application-server client error: get application-server client error:
    create application-server api client error: dial application-server api error:
    context deadline exceeded"

My environment is an installation of chirpstack-application-server v3.12.1 and chirpstack-network-server v3.9.0 on Raspberry Pi OS “Buster.” All chirpstack components are installed on the same Raspberry Pi v3 machine, all necessary prerequisites are installed, and the posgres initialization is performed according to the installation instructions.

After a lot of searching and experimenting, guided by hints in some previous posts by @brocaar and others (see e.g. https://forum.chirpstack.io/t/network-server-cant-connect-to-app-server/6953), I discovered that this is due to failure to correctly interpret the hostname “localhost” when used in clauses which define a partner component address (client-side) in chirpstack-application-server.toml and chirpstack-network-server.toml configuration files. I believe this is also true when setting the network server address in the Web UI “network-servers/create” page.

To work around this, I changed all occurrences of “localhost” to “127.0.0.1” in both configuration files as well as in the Web UI, e.g.

  server="http://localhost:8003"

to

  server="http://127.0.0.1:8003"

in chirpstack-network-server.toml and

  public_host="localhost:8001"

to

  public_host="127.0.0.1:8001"

in chirpstack-application-server.toml.

Unfortunately, doing so after the network and application server postgres databases have been used does not solve the issue - I had to shut down the servers and purge the database as follows:

sudo systemctl stop chirpstack-network-server
sudo systemctl stop chirpstack-application-server
sudo -u postgres dropdb -h localhost -U chirpstack_ns -i -e chirpstack_ns
sudo -u postgres psql -c "create database chirpstack_ns with owner chirpstack_ns;"

sudo -u postgres dropdb -h localhost -U chirpstack_as -i -e chirpstack_as
sudo -u postgres psql -c "create database chirpstack_as with owner chirpstack_as;"
sudo -u postgres psql chirpstack_as -c "create extension pg_trgm;"
sudo -u postgres psql chirpstack_as -c "create extension hstore;"
sudo systemctl start chirpstack-network-server
sudo systemctl start chirpstack-application-server

Of course, doing this deletes all system setup contained in the database … but it is unlikely that there is anything useful there anyway on a new install.

This issue does not appear to affect all installations, but may be helpful for those who have been vexed by this phenomenon.

3 Likes