SenseCAP LoRaWAN Outdoor Gateway offline operation

Gateways
21

You may try 127.0.0.1 instead of localhost.

1 Like
22

Yes the DNS failure is likely the issue here, not being able to resolve localhost would cause all sorts of issues. As Datnus said you can try 127.0.0.1 but assuming the gateway is unix-based you could also add 127.0.0.1 localhost to your /etc/hosts file to have the gateway autoresolve the DNS, that way if there are any other cases where localhost is used they should work too.

23

127.0.0.1 localhost was already in the /etc/hosts file, and network server URL was 127.0.0.1:8005 but didnā€™t work.

update:

We have dnsmasq on the gateway, and we have configured:

cat /etc/resolv.conf
nameserver 127.0.0.1
nameserver ::1

cat /etc/resolv.dnsmasq.conf
nameserver 223.5.5.5
nameserver 8.8.8.8

cat /var/run/connman/resolv.conf
# Generated by Connection Manager
nameserver 192.168.177.1

And from the log, we can find that chirpstack-network-server does dial dnsmasq on port 53, but just fails when there is no Internet:

DEBU[0025] dns: SRV record lookup error: lookup _grpclb._tcp.localhost on [::1]:53: dial udp [::1]:53: operation was canceled

My problem is: what is _grpclb._tcp.localhost for? Can we replace it with local ip?

24

_grpclb._tcp.localhost is a DNS Service record commonly associated with gRPC load balancing. gRPC uses this SRV record to locate load-balancer services or backends dynamically. Itā€™s a feature of gRPC but Iā€™m not sure why it would be enabled here.

Do you know if this section of your chirpstack-application-server.toml is the default?

    # Settings for the "internal api"
    #
    # This is the API used by ChirpStack Network Server to communicate with ChirpStack Application Server
    # and should not be exposed to the end-user.
    [application_server.api]
    # ip:port to bind the api server
    bind="0.0.0.0:8001"

    # Public ip:port of the application-server API.
    #
    # This is used by ChirpStack Network Server to connect to ChirpStack Application Server. When running
    # ChirpStack Application Server on a different host than ChirpStack Network Server, make sure to set
    # this to the host:ip on which ChirpStack Network Server can reach ChirpStack Application Server.
    # The port must be equal to the port configured by the 'bind' flag
    # above.
    public_host="localhost:8001"


    # Settings for the "external api"
    #
    # This is the API and web-interface exposed to the end-user.
    [application_server.external_api]
    # ip:port to bind the (user facing) http server to (web-interface and REST / gRPC api)
    bind="0.0.0.0:8080"
25

yes, this section is the default.

26

Try setting the public host to 127.0.0.1, even if it is in /etc/hosts, perhaps grpc handles lookups weirdly.

If not my only other guess at this point is that it could be IPv6 issue if the gateway doesnā€™t handle that properly internally. As the error:

DEBU[0025] dns: SRV record lookup error: lookup _grpclb._tcp.localhost on [::1]:53: dial udp [::1]:53: operation was canceled

Shows that its reaching out to the IPV6 localhost equivalent [::1]:53:

Perhaps removing IPv6 localhost resolution from /etc/hosts could fix it if it:

127.0.0.1   localhost
# ::1         localhost ipv6-localhost ipv6-loopback

Also going back to this question, ideally yes it would instead reach out to local IP. But where / how would you replace it with local IP? My understanding is that this is a grpc thing, and not a line you guys have in a configuration file somewhere?

EDIT:

The second log shows that currently localhost (as that is what is in the public_host) is being resolved to both the IPv4 127.0.0.1:8001 and IPv6 [::1]:8001.

DEBU[0025] dns: SRV record lookup error: lookup _grpclb._tcp.localhost on [::1]:53: dial udp [::1]:53: operation was canceled
DEBU[0025] ccResolverWrapper: sending update to cc: {[{127.0.0.1:8001  <nil> 0 <nil>} {[::1]:8001  <nil> 0 <nil>}] <nil> <nil>}

My hunch is by explicitly using 127.0.0.1 in public_host and ensuring IPv6 is disabled or correctly configured, gRPC will no longer attempt the SRV lookup for _grpclb._tcp.localhost.

27

I have set public_host="127.0.0.1:8001" in the chirpstack-application-server.toml, but still got the same logs.
And I noticed that in the network-server log it still reports: creating insecure application-server client server="localhost:8001" even if I set public_host="127.0.0.1:8001". Should we also need to change localhost to 127.0.0.1 in the network-server.toml? But I couldnā€™t find localhost:8001 in the default configuration.

28

I believe the Network server should use whatever the public_host is in the application servers config. Therefore it doesnā€™t need its own specification for the API port.

Not sure where to go from here.

29

I tried to change public_host to public_host="127.0.0.1:8022" and bind="0.0.0.0:8022", but Network Server was still trying to create a client at port 8001. Is it 8001 hard-code in Network Server?

How about chirpstack v4, can it be used in a local network without Internet?

30

Yes Chirpstack V4 can definitely be run without internet, theoretically V3 can as well.

But in V4 the application and networks servers are merged so this grpc communication between the platforms would be completely removed as a concern.

Theres also all sorts of improvements in V4, if you were willing to go through the process of upgrading Iā€™m sure all the customers would thank you. The only hesitation in my head is that default support for the REST api is removed in V4 (you would need to install an extra piece to get it to work again).

31

Hi Liam,
I am facing this problem when I try to run chirpstack v4 on our gateway, could you please help me? How can I configure "reset_commands" in toml?

32

Sorry man, this is beyond my knowledge. You should probably reach out to @brocaar himself, from what Iā€™ve seen heā€™s always keen to help manufacturers implement Chirpstack.

1 Like