Passive Roaming between 2 Chirpstack NS

Hi everyone,

I am setting up a stateless passive roaming PoC. I’ve almost succeeded but something still goes wrong.

Here is my scenario:
I own 2 Chirpstack NS on 2 different servers, and I have both configured them in a peer-to-peer roaming agreement. To do so, I have read the backend interface and I have followed the Chirpstack NS configuration guidelines available here: Configuration - ChirpStack open-source LoRaWAN<sup>®</sup> Network Server
Elements of my PoC:

  • A Chirpstack NS (server1) that works as fNS. This one has a NetID. (.toml configuration at the end of the post)
  • A Chirpstack NS (server2) works as hNS and sNS. This one has a NetID. (.toml configuration at the end of the post)
  • An ABP end-device registered in the server2. It has already emitted successful uplinks into the server 2 coverage.

What I have done and what I have seen:
My end-device emits frames into the server1 coverage. I should receive the emitted frames into the “LORAWAN FRAMES” section of my server2… but nothing comes.
When my end-device emit a frame, here are the logs I meet on the server1 (I am well aware of the 3-first lines meaning. I am actually wondering about the last one):

chirpstack-network-server_1      | time="2022-06-13T12:22:09.244006575Z" level=info msg="uplink/data: devaddr does not match netid, assuming roaming device" ctx_id=bbfaeb9e-2b79-4fe0-89e7-4ba672568418 dev_addr=041a2b3c
chirpstack-network-server_1      | time="2022-06-13T12:22:09.245126677Z" level=info msg="uplink/data: starting passive-roaming sessions with matching netids" ctx_id=bbfaeb9e-2b79-4fe0-89e7-4ba672568418 dev_addr=041a2b3c
chirpstack-network-server_1      | time="2022-06-13T12:22:09.245861101Z" level=info msg="uplink/data: starting passive-roaming session" ctx_id=bbfaeb9e-2b79-4fe0-89e7-4ba672568418 dev_addr=041a2b3c net_id=000002
chirpstack-network-server_1      | time="2022-06-13T12:22:09.319217632Z" level=error msg="uplink/data: start passive-roaming error" ctx_id=bbfaeb9e-2b79-4fe0-89e7-4ba672568418 dev_addr=041a2b3c error="request error: http post error: Post \"http://ADDRESS_SEVER2:8000\": net/http: HTTP/1.x transport connection broken: malformed HTTP response \"\\x00\\x00\\x06\\x04\\x00\\x00\\x00\\x00\\x00\\x00\\x05\\x00\\x00@\\x00\"" net_id=000002

And on the server2:

chirpstack-network-server_1      | time="2022-06-13T12:22:09.319169137Z" level=warning msg="[core] grpc: Server.Serve failed to create ServerTransport:  connection error: desc = \"transport: http2Server.HandleStreams received bogus greeting from client: \\\"POST / HTTP/1.1\\\\r\\\\nHost: s\\\"\""

Has anyone ever managed to do a passive roaming scenario like the one I am trying to setup?
Is there something wrong in my configuration?

Really appreciate your attention, thank you!
Regards,
.
.
.
.
.

Here is the chirpstack-network-server.toml configuration of the server1 (fNS):

[network_server]
net_id="C00052"
...
[roaming]
  # Resolve NetID domain suffix.
  resolve_netid_domain_suffix=".netids.lora-alliance.org"
  # Per roaming-agreement server configuration.
  [[roaming.servers]]
    # NetID of the roaming server.
    net_id="000002"
    # Use the async API scheme.
    async=true
    # Async request timeout.
    async_timeout="1s" 
    # Allow passive-roaming.
    passive_roaming=true
    # Passive-roaming session lifetime.
    # When set to 0s, the passive-roaming will be stateless.
    passive_roaming_lifetime="0s"
    # Passive-roaming KEK label (optional).
    # When set, the session-keys will be encrypted using the given KEK when these
    # are exchanged.
    # passive_roaming_kek_label=""
    # Server (optional).
    # When set, this will bypass the DNS resolving of the Network Server.
    server="http://ADDRESS_SEVER2:8000"
    # CA certificate (optional).
    # When configured, this is used to validate the server certificate.
    # ca_cert=""
    # TLS client certificate (optional).
    # When configured, this will be used to authenticate the client.
    # This mist be configured together with the tls_key.
    # tls_cert=""
    # TLS key for client certificate (optional).
    # tls_key=""

.
.

Here is the chirpstack-network-server.toml configuration of the server2 (hNS, sNS):

[network_server]
net_id="000002"
….
[roaming]
  # Resolve NetID domain suffix.
  resolve_netid_domain_suffix=".netids.lora-alliance.org"
  # Per roaming-agreement server configuration.
  [[roaming.servers]]
    # NetID of the roaming server.
    net_id="C00052"
    # Use the async API scheme.
    async=true
    # Async request timeout.
    async_timeout="1s"
    # Allow passive-roaming.
    passive_roaming=true
    # Passive-roaming session lifetime.
    # When set to 0s, the passive-roaming will be stateless.
    passive_roaming_lifetime="0s"
    # Passive-roaming KEK label (optional).
    # When set, the session-keys will be encrypted using the given KEK when these
    # are exchanged.
    # passive_roaming_kek_label=""
    # Server (optional).
    # When set, this will bypass the DNS resolving of the Network Server.
    server="http://ADDRESS_SEVER1:8000"
    # CA certificate (optional).
    # When configured, this is used to validate the server certificate.
    # ca_cert=""
  
    # TLS client certificate (optional).
    # When configured, this will be used to authenticate the client.
    # This mist be configured together with the tls_key.
    # tls_cert=""
    # TLS key for client certificate (optional).
    # tls_key=""

You are using the wrong port, port 8000 is the NS gRPC API endpoint. Please see this configuration section:

  # Roaming API settings.
  [roaming.api]
  # Interface to bind the API to (ip:port).
  bind=""

  # CA certificate (optional).
  #
  # When configured, this is used for client-certificate validation.
  ca_cert=""

  # TLS certificate (optional).
  #
  # When configured, this is used to secure the API interface using TLS.
  # This must be configured together with the tls_key.
  tls_cert=""

  # TLS key (optional).
  tls_key=""

You must configure the bind option, e.g. 0.0.0.0:8090 and then use this as server endpoint in the other NS configuration (e.g. server="http://ADDRESS_SERVER1:8090").