Application Server Connection Refused

randall-au · July 21, 2020, 7:52pm

I am attempting to turn up a new chirpstack instance using docker compose. I’ve done this in the past without issue but now I am hitting a connection refused error from the application server. I’m not even entirely sure where the 10.214.254.133:8080 address is coming from. I’m thinking the 0.0.0.0:8080 is resolving to this somehow? Here is my application config (just the default that comes with compose) followed by the output. Any help to get me pointed in the right direction would be appreciated

(Top portion of config file edited out to get under two link limit)

[application_server.api]
public_host="chirpstack-application-server:8001"

[application_server.external_api]
bind="0.0.0.0:8080"
jwt_secret="verysecret"



Starting chirpstackdocker_chirpstack-geolocation-server_1 ... 
Starting chirpstackdocker_chirpstack-gateway-bridge_1 ... 
Starting chirpstackdocker_chirpstack-geolocation-server_1
Starting chirpstackdocker_mosquitto_1 ... 
Starting chirpstackdocker_chirpstack-gateway-bridge_1
Starting chirpstackdocker_redis_1 ... 
Starting chirpstackdocker_mosquitto_1
Starting chirpstackdocker_redis_1
Starting chirpstackdocker_chirpstack-application-server_1 ... 
Starting chirpstackdocker_chirpstack-network-server_1 ... 
Starting chirpstackdocker_chirpstack-network-server_1
Starting chirpstackdocker_postgresql_1 ... 
Starting chirpstackdocker_chirpstack-application-server_1
Starting chirpstackdocker_chirpstack-gateway-bridge_1 ... done
Attaching to chirpstackdocker_chirpstack-geolocation-server_1, chirpstackdocker_redis_1, chirpstackdocker_chirpstack-network-server_1, chirpstackdocker_chirpstack-application-server_1, chirpstackdocker_mosquitto_1, chirpstackdocker_postgresql_1, chirpstackdocker_chirpstack-gateway-bridge_1
chirpstack-geolocation-server_1  | time="2020-07-21T19:28:09Z" level=info msg="starting ChirpStack Geolocation Server" docs="https://www.chirpstack.io/geolocation-server/" version=3.3.1
chirpstack-geolocation-server_1  | time="2020-07-21T19:28:09Z" level=info msg="starting api server" backend=collos bind="0.0.0.0:8005" ca_cert= tls_cert= tls_key=
redis_1                          | 1:C 21 Jul 2020 19:28:09.399 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
redis_1                          | 1:C 21 Jul 2020 19:28:09.399 # Redis version=5.0.6, bits=64, commit=00000000, modified=0, pid=1, just started
redis_1                          | 1:C 21 Jul 2020 19:28:09.399 # Warning: no config file specified, using the default config. In order to specify a config file use redis-server /path/to/redis.conf
redis_1                          | 1:M 21 Jul 2020 19:28:09.402 * Running mode=standalone, port=6379.
redis_1                          | 1:M 21 Jul 2020 19:28:09.403 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.
redis_1                          | 1:M 21 Jul 2020 19:28:09.403 # Server initialized
redis_1                          | 1:M 21 Jul 2020 19:28:09.403 # WARNING overcommit_memory is set to 0! Background save may fail under low memory condition. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.
redis_1                          | 1:M 21 Jul 2020 19:28:09.403 # WARNING you have Transparent Huge Pages (THP) support enabled in your kernel. This will create latency and memory usage issues with Redis. To fix this issue run the command 'echo never > /sys/kernel/mm/transparent_hugepage/enabled' as root, and add it to your /etc/rc.local in order to retain the setting after a reboot. Redis must be restarted after THP is disabled.
redis_1                          | 1:M 21 Jul 2020 19:28:09.403 * DB loaded from disk: 0.000 seconds
redis_1                          | 1:M 21 Jul 2020 19:28:09.403 * Ready to accept connections
chirpstack-network-server_1      | time="2020-07-21T19:28:10Z" level=info msg="starting ChirpStack Network Server" band=EU868 docs="https://www.chirpstack.io/" net_id=000000 version=3.10.0
chirpstack-network-server_1      | time="2020-07-21T19:28:10Z" level=info msg="storage: setting up storage module"
chirpstack-network-server_1      | time="2020-07-21T19:28:10Z" level=info msg="storage: setting up Redis client"
chirpstack-network-server_1      | time="2020-07-21T19:28:10Z" level=info msg="storage: connecting to PostgreSQL"
chirpstack-network-server_1      | time="2020-07-21T19:28:11Z" level=info msg="storage: applying PostgreSQL data migrations"
chirpstack-network-server_1      | time="2020-07-21T19:28:11Z" level=info msg="storage: PostgreSQL data migrations applied" count=0
chirpstack-network-server_1      | time="2020-07-21T19:28:11Z" level=info msg="gateway/mqtt: connecting to mqtt broker" server="tcp://mosquitto:1883"
chirpstack-network-server_1      | time="2020-07-21T19:28:11Z" level=info msg="configuring join-server client" ca_cert= server="http://chirpstack-application-server:8003" tls_cert= tls_key=
chirpstack-network-server_1      | time="2020-07-21T19:28:11Z" level=info msg="api: starting network-server api server" bind="0.0.0.0:8000" ca-cert= tls-cert= tls-key=
chirpstack-network-server_1      | time="2020-07-21T19:28:11Z" level=info msg="starting downlink device-queue scheduler"
chirpstack-network-server_1      | time="2020-07-21T19:28:11Z" level=info msg="starting multicast scheduler"
chirpstack-network-server_1      | time="2020-07-21T19:28:11Z" level=info msg="backend/gateway: connected to mqtt server"
chirpstack-network-server_1      | time="2020-07-21T19:28:11Z" level=info msg="gateway/mqtt: subscribing to gateway event topic" qos=0 topic=gateway/+/event/+
chirpstack-application-server_1  | time="2020-07-21T19:28:10Z" level=info msg="starting ChirpStack Application Server" docs="https://www.chirpstack.io/" version=3.11.0
chirpstack-application-server_1  | time="2020-07-21T19:28:10Z" level=info msg="storage: setting up storage package"
chirpstack-application-server_1  | time="2020-07-21T19:28:10Z" level=info msg="storage: setup metrics"
chirpstack-application-server_1  | time="2020-07-21T19:28:10Z" level=info msg="storage: setting up Redis client"
chirpstack-application-server_1  | time="2020-07-21T19:28:10Z" level=info msg="storage: connecting to PostgreSQL database"
chirpstack-application-server_1  | time="2020-07-21T19:28:11Z" level=info msg="storage: applying PostgreSQL data migrations"
chirpstack-application-server_1  | time="2020-07-21T19:28:11Z" level=info msg="storage: PostgreSQL data migrations applied" count=0
chirpstack-application-server_1  | time="2020-07-21T19:28:11Z" level=info msg="integration: configuring global integrations"
chirpstack-application-server_1  | time="2020-07-21T19:28:11Z" level=info msg="integration/mqtt: TLS config is empty"
chirpstack-application-server_1  | time="2020-07-21T19:28:11Z" level=info msg="integration/mqtt: connecting to mqtt broker" server="tcp://mosquitto:1883"
chirpstack-application-server_1  | time="2020-07-21T19:28:11Z" level=info msg="api/as: starting application-server api" bind="0.0.0.0:8001" ca_cert= tls_cert= tls_key=
chirpstack-application-server_1  | time="2020-07-21T19:28:11Z" level=info msg="api/external: starting api server" bind="0.0.0.0:8080" tls-cert= tls-key=
chirpstack-application-server_1  | time="2020-07-21T19:28:11Z" level=info msg="integration/mqtt: connected to mqtt broker"
chirpstack-application-server_1  | time="2020-07-21T19:28:11Z" level=info msg="integration/mqtt: subscribing to tx topic" qos=0 topic=application/+/device/+/command/down
chirpstack-application-server_1  | time="2020-07-21T19:28:11Z" level=info msg="api/external: registering rest api handler and documentation endpoint" path=/api
chirpstack-application-server_1  | time="2020-07-21T19:28:11Z" level=info msg="api/js: starting join-server api" bind="0.0.0.0:8003" ca_cert= tls_cert= tls_key=
chirpstack-application-server_1  | time="2020-07-21T19:28:11Z" level=warning msg="grpc: addrConn.createTransport failed to connect to {10.214.254.133:8080  <nil> 0 <nil>}. Err: connection error: desc = \"transport: Error while dialing dial tcp 10.214.254.133:8080: connect: connection refused\". Reconnecting..."

eugenev · July 21, 2020, 8:17pm

is this name is registered in the your name server to resolve it ? I suppose, no.

If you read carefully comments inside the chirpstack-application-server.toml file, you must be read this:
You could generate this by executing ‘openssl rand -base64 32’ for example

brocaar · July 22, 2020, 9:17am

After what action do you see this error?

@eugenev the chirpstack-application-server:8001 is correct, Docker Compose creates these DNS entries.

randall-au · July 22, 2020, 11:53am

I see this immediately after docker-compose up. The web interface does not come up properly. When I access localhost:8080 it redirects me to the applications page and displays a connection error as shown here

randall-au · July 23, 2020, 4:12pm

I turned on debug messages for the application server to see if I could determine what was going on. Looks like it’s using ccResolverWrapper for something and comes up with an address of 10.214.254.133. What would it be resolving here? That is an address on my network but nothing I would expect to see here. I must have a dns issue somewhere but since I might be able to fix it with an entry in hosts but I don’t know what it is looking for.

chirpstack-application-server_1  | time="2020-07-23T15:35:34Z" level=debug msg="sql query executed" args="[3 2020-07-23 15:35:34.325529253 +0$
chirpstack-application-server_1  | time="2020-07-23T15:35:34Z" level=debug msg="parsed scheme: \"\""
chirpstack-application-server_1  | time="2020-07-23T15:35:34Z" level=debug msg="scheme \"\" not registered, fallback to default scheme"
chirpstack-application-server_1  | time="2020-07-23T15:35:34Z" level=debug msg="parsed scheme: \"\""
chirpstack-application-server_1  | time="2020-07-23T15:35:34Z" level=debug msg="scheme \"\" not registered, fallback to default scheme"
chirpstack-application-server_1  | time="2020-07-23T15:35:34Z" level=debug msg="parsed scheme: \"\""
chirpstack-application-server_1  | time="2020-07-23T15:35:34Z" level=debug msg="scheme \"\" not registered, fallback to default scheme"
chirpstack-application-server_1  | time="2020-07-23T15:35:34Z" level=debug msg="parsed scheme: \"\""
chirpstack-application-server_1  | time="2020-07-23T15:35:34Z" level=debug msg="scheme \"\" not registered, fallback to default scheme"
chirpstack-application-server_1  | time="2020-07-23T15:35:34Z" level=debug msg="parsed scheme: \"\""
chirpstack-application-server_1  | time="2020-07-23T15:35:34Z" level=debug msg="scheme \"\" not registered, fallback to default scheme"
chirpstack-application-server_1  | time="2020-07-23T15:35:34Z" level=debug msg="parsed scheme: \"\""
chirpstack-application-server_1  | time="2020-07-23T15:35:34Z" level=debug msg="scheme \"\" not registered, fallback to default scheme"
chirpstack-application-server_1  | time="2020-07-23T15:35:34Z" level=debug msg="parsed scheme: \"\""
chirpstack-application-server_1  | time="2020-07-23T15:35:34Z" level=debug msg="scheme \"\" not registered, fallback to default scheme"
chirpstack-application-server_1  | time="2020-07-23T15:35:34Z" level=debug msg="parsed scheme: \"\""
chirpstack-application-server_1  | time="2020-07-23T15:35:34Z" level=debug msg="scheme \"\" not registered, fallback to default scheme"
chirpstack-application-server_1  | time="2020-07-23T15:35:34Z" level=debug msg="parsed scheme: \"\""
chirpstack-application-server_1  | time="2020-07-23T15:35:34Z" level=debug msg="scheme \"\" not registered, fallback to default scheme"
chirpstack-application-server_1  | time="2020-07-23T15:35:34Z" level=debug msg="parsed scheme: \"\""
chirpstack-application-server_1  | time="2020-07-23T15:35:34Z" level=debug msg="scheme \"\" not registered, fallback to default scheme"
chirpstack-application-server_1  | time="2020-07-23T15:35:34Z" level=debug msg="parsed scheme: \"\""
chirpstack-application-server_1  | time="2020-07-23T15:35:34Z" level=debug msg="scheme \"\" not registered, fallback to default scheme"
chirpstack-application-server_1  | time="2020-07-23T15:35:34Z" level=debug msg="parsed scheme: \"\""
chirpstack-application-server_1  | time="2020-07-23T15:35:34Z" level=debug msg="scheme \"\" not registered, fallback to default scheme"
chirpstack-application-server_1  | time="2020-07-23T15:35:34Z" level=debug msg="parsed scheme: \"\""
chirpstack-application-server_1  | time="2020-07-23T15:35:34Z" level=debug msg="scheme \"\" not registered, fallback to default scheme"
chirpstack-application-server_1  | time="2020-07-23T15:35:34Z" level=info msg="api/external: registering rest api handler and documentation e$
chirpstack-application-server_1  | time="2020-07-23T15:35:34Z" level=info msg="api/js: starting join-server api" bind="0.0.0.0:8003" ca_cert=$
chirpstack-application-server_1  | time="2020-07-23T15:35:34Z" level=debug msg="ccResolverWrapper: sending update to cc: {[{10.214.254.133:80$
chirpstack-application-server_1  | time="2020-07-23T15:35:34Z" level=debug msg="ClientConn switching balancer to \"pick_first\""
chirpstack-application-server_1  | time="2020-07-23T15:35:34Z" level=debug msg="Channel switches to new LB policy \"pick_first\""
chirpstack-application-server_1  | time="2020-07-23T15:35:34Z" level=debug msg="Subchannel Connectivity change to CONNECTING"