Many errors with context deadline exceeded (code: 2)

ChirpStack Application Server
1

Recently this error appears to me, i’ve reinstall everything, read the posts here and nothing work.

When i try to register a new gateway, this happen.

erro1
erro11366×768 217 KB

And also here.

error2
error21366×768 76.1 KB

I can’t really find the error, everything looks set up right.

Output from journalctl -u chirpstack-application-server -f -n 50

mar 18 21:07:19 victorUTF chirpstack-application-server[1124]: time="2020-03-18T21:07:19-03:00" level=info msg="finished unary call with code OK" ctx_id=f949b931-cb4b-428f-986f-b379fc295799 grpc.code=OK grpc.method=List grpc.service=api.NetworkServerService grpc.start_time="2020-03-18T21:07:19-03:00" grpc.time_ms=4.048 peer.address="127.0.0.1:42300" span.kind=server system=grpc
mar 18 21:07:19 victorUTF chirpstack-application-server[1124]: time="2020-03-18T21:07:19-03:00" level=warning msg="creating insecure network-server client" server="localhost:8000"
mar 18 21:07:24 victorUTF chirpstack-application-server[1124]: time="2020-03-18T21:07:24-03:00" level=info msg="finished unary call with code OK" ctx_id=254ff03e-aab9-4f72-a62f-0ef6d08ba861 grpc.code=OK grpc.method=Get grpc.service=api.NetworkServerService grpc.start_time="2020-03-18T21:07:19-03:00" grpc.time_ms=5003.745 peer.address="127.0.0.1:42300" span.kind=server system=grpc
mar 18 21:07:38 victorUTF chirpstack-application-server[1124]: time="2020-03-18T21:07:38-03:00" level=info msg="gateway created" ctx_id=bf05996f-243d-427e-9008-6307f1e57401 id=b827ebffff839157 name=RHF0M301
mar 18 21:07:38 victorUTF chirpstack-application-server[1124]: time="2020-03-18T21:07:38-03:00" level=warning msg="creating insecure network-server client" server="localhost:8000"
mar 18 21:07:43 victorUTF chirpstack-application-server[1124]: time="2020-03-18T21:07:43-03:00" level=error msg="finished unary call with code Unknown" ctx_id=bf05996f-243d-427e-9008-6307f1e57401 error="rpc error: code = Unknown desc = context deadline exceeded" grpc.code=Unknown grpc.method=Create grpc.service=api.GatewayService grpc.start_time="2020-03-18T21:07:38-03:00" grpc.time_ms=5012.103 peer.address="127.0.0.1:42318" span.kind=server system=grpc
mar 18 21:07:43 victorUTF chirpstack-application-server[1124]: time="2020-03-18T21:07:43-03:00" level=info msg="gateway created" ctx_id=254674ce-913b-42a3-8769-837527f950bc id=b827ebffff839157 name=RHF0M301
mar 18 21:07:43 victorUTF chirpstack-application-server[1124]: time="2020-03-18T21:07:43-03:00" level=warning msg="creating insecure network-server client" server="localhost:8000"
mar 18 21:07:48 victorUTF chirpstack-application-server[1124]: time="2020-03-18T21:07:48-03:00" level=error msg="finished unary call with code Unknown" ctx_id=254674ce-913b-42a3-8769-837527f950bc error="rpc error: code = Unknown desc = context deadline exceeded" grpc.code=Unknown grpc.method=Create grpc.service=api.GatewayService grpc.start_time="2020-03-18T21:07:41-03:00" grpc.time_ms=7768.594 peer.address="127.0.0.1:42318" span.kind=server system=grpc

My chirpstack-application-server.toml

# This configuration sets the required settings and configures an integration
# with a MQTT broker. Many options and defaults have been omitted for
# simplicity.
#
# See https://www.chirpstack.io/application-server/install/config/ for a full
# configuration example and documentation.


# PostgreSQL settings.
#
# Please note that PostgreSQL 9.5+ is required.
[postgresql]
# PostgreSQL dsn (e.g.: postgres://user:password@hostname/database?sslmode=disable).
#
# Besides using an URL (e.g. 'postgres://user:password@hostname/database?sslmode=disable')
# it is also possible to use the following format:
# 'user=chirpstack_as dbname=chirpstack_as sslmode=disable'.
#
# The following connection parameters are supported:
#
# * dbname - The name of the database to connect to
# * user - The user to sign in as
# * password - The user's password
# * host - The host to connect to. Values that start with / are for unix domain sockets. (default is localhost)
# * port - The port to bind to. (default is 5432)
# * sslmode - Whether or not to use SSL (default is require, this is not the default for libpq)
# * fallback_application_name - An application_name to fall back to if one isn't provided.
# * connect_timeout - Maximum wait for connection, in seconds. Zero or not specified means wait indefinitely.
# * sslcert - Cert file location. The file must contain PEM encoded data.
# * sslkey - Key file location. The file must contain PEM encoded data.
# * sslrootcert - The location of the root certificate file. The file must contain PEM encoded data.
#
# Valid values for sslmode are:
#
# * disable - No SSL
# * require - Always SSL (skip verification)
# * verify-ca - Always SSL (verify that the certificate presented by the server was signed by a trusted CA)
# * verify-full - Always SSL (verify that the certification presented by the server was signed by a trusted CA and the server host name matches the one in the certificate)
dsn="postgres://victor_as:tcc@localhost/chirpstack_as?sslmode=disable"


# Redis settings
#
# Please note that Redis 2.6.0+ is required.
[redis]
# Redis url (e.g. redis://user:password@hostname/0)
#
# For more information about the Redis URL format, see:
# https://www.iana.org/assignments/uri-schemes/prov/redis
url="redis://localhost:6379"


# Application-server settings.
[application_server]

  # Integration configures the data integration.
  #
  # This is the data integration which is available for all applications,
  # besides the extra integrations that can be added on a per-application
  # basis.
  [application_server.integration]
  # Enabled integrations.
  enabled=["mqtt"]

    # MQTT integration backend.
    [application_server.integration.mqtt]
    # MQTT topic templates for the different MQTT topics.
    #
    # The meaning of these topics are documented at:
    # https://www.chirpstack.io/application-server/integrate/data/
    #
    # The following substitutions can be used:
    # * "{{ .ApplicationID }}" for the application id.
    # * "{{ .DevEUI }}" for the DevEUI of the device.
    #
    # Note: the downlink_topic_template must contain both the application id and
    # DevEUI substitution!
    uplink_topic_template="application/{{ .ApplicationID }}/device/{{ .DevEUI }}/rx"
    downlink_topic_template="application/{{ .ApplicationID }}/device/{{ .DevEUI }}/tx"
    join_topic_template="application/{{ .ApplicationID }}/device/{{ .DevEUI }}/join"
    ack_topic_template="application/{{ .ApplicationID }}/device/{{ .DevEUI }}/ack"
    error_topic_template="application/{{ .ApplicationID }}/device/{{ .DevEUI }}/error"
    status_topic_template="application/{{ .ApplicationID }}/device/{{ .DevEUI }}/status"
    location_topic_template="application/{{ .ApplicationID }}/device/{{ .DevEUI }}/location"

    # MQTT server (e.g. scheme://host:port where scheme is tcp, ssl or ws)
    server="tcp://localhost:1883"

    # Connect with the given username (optional)
    username=""

    # Connect with the given password (optional)
    password=""


    # Settings for the "internal api"
    #
    # This is the API used by ChirpStack Network Server to communicate with ChirpStack Application Server
    # and should not be exposed to the end-user.
    [application_server.api]
    # ip:port to bind the api server
    bind="0.0.0.0:8001"

    # Public ip:port of the application-server API.
    #
    # This is used by ChirpStack Network Server to connect to ChirpStack Application Server. When running
    # ChirpStack Application Server on a different host than ChirpStack Network Server, make sure to set
    # this to the host:ip on which ChirpStack Network Server can reach ChirpStack Application Server.
    # The port must be equal to the port configured by the 'bind' flag
    # above.
    public_host="localhost:8001"


    # Settings for the "external api"
    #
    # This is the API and web-interface exposed to the end-user.
    [application_server.external_api]
    # ip:port to bind the (user facing) http server to (web-interface and REST / gRPC api)
    bind="0.0.0.0:8080"

    # http server TLS certificate (optional)
    tls_cert=""

    # http server TLS key (optional)
    tls_key=""

    # JWT secret used for api authentication / authorization
    # You could generate this by executing 'openssl rand -base64 32' for example
    jwt_secret="G8vPeaQ9qYjKacVgXeIUgr+6ZtQUzBZ5u+XMmXtFfYM="


# Join-server configuration.
#
# ChirpStack Application Server implements a (subset) of the join-api specified by the
# LoRaWAN Backend Interfaces specification. This API is used by ChirpStack Network Server
# to handle join-requests.
[join_server]
# ip:port to bind the join-server api interface to
bind="0.0.0.0:8003"

Output from journalctl -u chirpstack-network-server -f -n 50

-- Reboot --
mar 18 20:47:44 victorUTF systemd[1]: Started ChirpStack Network Server.
mar 18 20:47:45 victorUTF chirpstack-network-server[1128]: time="2020-03-18T20:47:45-03:00" level=info msg="starting ChirpStack Network Server" band=US_902_928 docs="https://www.chirpstack.io/" net_id=000000 version=3.8.0
mar 18 20:47:45 victorUTF chirpstack-network-server[1128]: time="2020-03-18T20:47:45-03:00" level=info msg="disabling all channels"
mar 18 20:47:45 victorUTF chirpstack-network-server[1128]: time="2020-03-18T20:47:45-03:00" level=info msg="enabling channels" channels="[0 1 2 3 4 5 6 7]"
mar 18 20:47:45 victorUTF chirpstack-network-server[1128]: time="2020-03-18T20:47:45-03:00" level=info msg="storage: setting up storage module"
mar 18 20:47:45 victorUTF chirpstack-network-server[1128]: time="2020-03-18T20:47:45-03:00" level=info msg="storage: setting up Redis connection pool"
mar 18 20:47:45 victorUTF chirpstack-network-server[1128]: time="2020-03-18T20:47:45-03:00" level=info msg="storage: connecting to PostgreSQL"
mar 18 20:47:45 victorUTF chirpstack-network-server[1128]: time="2020-03-18T20:47:45-03:00" level=info msg="storage: applying PostgreSQL data migrations"
mar 18 20:47:46 victorUTF chirpstack-network-server[1128]: time="2020-03-18T20:47:46-03:00" level=info msg="storage: PostgreSQL data migrations applied" count=0
mar 18 20:47:46 victorUTF chirpstack-network-server[1128]: time="2020-03-18T20:47:46-03:00" level=info msg="gateway/mqtt: connecting to mqtt broker" server="tcp://localhost:1883"
mar 18 20:47:46 victorUTF chirpstack-network-server[1128]: time="2020-03-18T20:47:46-03:00" level=info msg="no geolocation-server configured"
mar 18 20:47:46 victorUTF chirpstack-network-server[1128]: time="2020-03-18T20:47:46-03:00" level=info msg="configuring join-server client" ca_cert= server="http://localhost:8003" tls_cert= tls_key=
mar 18 20:47:46 victorUTF chirpstack-network-server[1128]: time="2020-03-18T20:47:46-03:00" level=info msg="backend/gateway: connected to mqtt server"
mar 18 20:47:46 victorUTF chirpstack-network-server[1128]: time="2020-03-18T20:47:46-03:00" level=info msg="gateway/mqtt: subscribing to gateway event topic" qos=0 topic=gateway/+/event/+
mar 18 20:47:46 victorUTF chirpstack-network-server[1128]: time="2020-03-18T20:47:46-03:00" level=info msg="api: starting network-server api server" bind="0.0.0.0:8000" ca-cert= tls-cert= tls-key=
mar 18 20:47:46 victorUTF chirpstack-network-server[1128]: time="2020-03-18T20:47:46-03:00" level=info msg="starting downlink device-queue scheduler"
mar 18 20:47:46 victorUTF chirpstack-network-server[1128]: time="2020-03-18T20:47:46-03:00" level=info msg="starting multicast scheduler"

I appreciate any sugestion, thanks for now

2

Victor Experimenta mudar o log Level para um mais alto tipo m Debug nas configurações, ajuda muito nesse tipo de erro.

1 Like
4

Obrigado pela dica, apareceu algo que eu não tinha visto mesmo

After click on Create Gateway, application server with log_level=5 .(i see network-server log too and it hasn’t change)

mar 19 06:59:15 victorUTF chirpstack-application-server[6492]: time="2020-03-19T06:59:15-03:00" level=info msg="gateway created" ctx_id=ce26d13f-83b4-4b25-bd11-c5c98fda5891 id=b827ebffff839157 name=test
mar 19 06:59:15 victorUTF chirpstack-application-server[6492]: time="2020-03-19T06:59:15-03:00" level=debug msg="sql query executed" args="[1]" duration="378.798µs" query="select * from network_server where id = $1"
mar 19 06:59:15 victorUTF chirpstack-application-server[6492]: time="2020-03-19T06:59:15-03:00" level=warning msg="creating insecure network-server client" server="localhost:8000"
mar 19 06:59:15 victorUTF chirpstack-application-server[6492]: time="2020-03-19T06:59:15-03:00" level=debug msg="parsed scheme: \"\""
mar 19 06:59:15 victorUTF chirpstack-application-server[6492]: time="2020-03-19T06:59:15-03:00" level=debug msg="scheme \"\" not registered, fallback to default scheme"
mar 19 06:59:15 victorUTF chirpstack-application-server[6492]: time="2020-03-19T06:59:15-03:00" level=debug msg="sql query executed" args="[]" duration="862.077µs" query="\n\t\tselect\n\t\t\tg.*\n\t\tfrom gateway g\n\t\tinner join network_server ns\n\t\t\ton ns.id = g.network_server_id\n\t\twhere\n\t\t\tns.gateway_discovery_enabled = true\n\t\t\tand g.ping = true\n\t\t\tand (g.last_ping_sent_at is null or g.last_ping_sent_at <= (now() - (interval '24 hours' / ns.gateway_discovery_interval)))\n\t\torder by last_ping_sent_at\n\t\tlimit 1\n\t\tfor update"
mar 19 06:59:15 victorUTF chirpstack-application-server[6492]: time="2020-03-19T06:59:15-03:00" level=debug msg="sql query executed" args="[3 2020-03-19 06:59:15.888902276 -0300 -03 m=+152.841825888 100]" duration=1.464861ms query="\n\t\tselect\n\t\t\tfs.dev_eui,\n\t\t\tfs.frag_index,\n\t\t\tfs.created_at,\n\t\t\tfs.updated_at,\n\t\t\tfs.mc_group_ids,\n\t\t\tfs.nb_frag,\n\t\t\tfs.frag_size,\n\t\t\tfs.fragmentation_matrix,\n\t\t\tfs.block_ack_delay,\n\t\t\tfs.padding,\n\t\t\tfs.descriptor,\n\t\t\tfs.state,\n\t\t\tfs.state_provisioned,\n\t\t\tfs.retry_after,\n\t\t\tfs.retry_count,\n\t\t\tfs.retry_interval\n\t\tfrom\n\t\t\tremote_fragmentation_session fs\n\t\twhere\n\t\t\tfs.state_provisioned = false\n\t\t\tand fs.retry_count < $1\n\t\t\tand fs.retry_after < $2\n\t\t\tand (\n\t\t\t\t-- in case of unicast\n\t\t\t\tarray_length(fs.mc_group_ids, 1) is null\n\n\t\t\t\t-- in case of multicast\n\t\t\t\tor exists (\n\t\t\t\t\tselect\n\t\t\t\t\t\t1\n\t\t\t\t\tfrom\n\t\t\t\t\t\tremote_multicast_setup ms\n\t\t\t\t\twhere\n\t\t\t\t\t\tms.dev_eui = fs.dev_eui\n\t\t\t\t\t\tand ms.state_provisioned = true\n\t\t\t\t\t\tand ms.mc_group_id = any(fs.mc_group_ids)\n\t\t\t\t)\n\t\t\t)\n\t\tlimit $3\n\t\tfor update of fs\n\t\tskip locked"
mar 19 07:01:15 victorUTF chirpstack-application-server[6492]: time="2020-03-19T07:01:15-03:00" level=debug msg="dns: SRV record lookup error: lookup _grpclb._tcp.localhost on 127.0.0.53:53: dial udp 127.0.0.53:53: operation was canceled"
mar 19 07:01:15 victorUTF chirpstack-application-server[6492]: time="2020-03-19T07:01:15-03:00" level=debug msg="ccResolverWrapper: sending update to cc: {[{127.0.0.1:8000  <nil> 0 <nil>}] <nil> <nil>}"
mar 19 07:01:15 victorUTF chirpstack-application-server[6492]: time="2020-03-19T07:01:15-03:00" level=error msg="finished unary call with code Unknown" ctx_id=c4c5fa30-8a45-4c3b-842c-7e4fe95c50c2 error="rpc error: code = Unknown desc = context deadline exceeded" grpc.code=Unknown grpc.method=Create grpc.service=api.GatewayService grpc.start_time="2020-03-19T07:01:10-03:00" grpc.time_ms=5014.316 peer.address="127.0.0.1:48544" span.kind=server system=grpc

This seems to be the problem, but i don’t know how to solve it (i think):
msg="dns: SRV record lookup error: lookup _grpclb._tcp.localhost on 127.0.0.53:53: dial udp 127.0.0.53:53: operation was canceled"

5

This is a strange issue, which only happens in some environments. One thing you could try is replace localhost with 127.0.0.1.

1 Like
6

Thanks for the help, i still don’t know why… but when i tried it again on another network, it worked!
Apparently my home network has something strange.