How to debug downlink sending on ChirpStack on Docker?

I am new to ChirpStack. I managed to get it up and running on Docker and have secured it using Traefik. I have access to the various containers using Portainer CE.

Currently I am working through this: I can’t get downlinks to get sent to my device. Details:

  • Class A device, OTAA, eu868
  • ChirpStack v4 on Docker with Traefik and Portainer
  • Uplinks work fine

Downlink details:

  • Two bytes non encrypted. Tried through the web interface and API.
  • The downlink is visible in the queue but marked as “pending=false”. Should it not be pending?

My question: where do I begin to trace what is happening? I can see the log in Portainer from the ChirpStack (main app) container. But I lack enough knowledge to understand what I am seeing. Here is an excerpt:

2024-02-08T19:31:16.491935Z INFO chirpstack::gateway::backend::mqtt: Message received from gateway region_id=“eu868” topic=“eu868/gateway/xxxxxxxxxxxxxxxx/event/up” qos=0 json=false
2024-02-08T19:31:16.694249Z INFO up{deduplication_id=804f1cd9-3a1d-4ec2-aad2-1f240b2bc7ff}: chirpstack::uplink: Uplink received m_type=UnconfirmedDataUp
2024-02-08T19:31:16.695690Z INFO up{deduplication_id=804f1cd9-3a1d-4ec2-aad2-1f240b2bc7ff}:data_up: chirpstack::storage::device_session: Device-session saved dev_eui=xxxxxxxxxxxxxxxx dev_addr=01542784
2024-02-08T19:31:16.699199Z INFO up{deduplication_id=804f1cd9-3a1d-4ec2-aad2-1f240b2bc7ff}:data_up{dev_eui=“xxxxxxxxxxxxxxxx”}: chirpstack::storage::device: Data-rate updated dev_eui=xxxxxxxxxxxxxxxx dr=5
2024-02-08T19:31:16.699596Z INFO up{deduplication_id=804f1cd9-3a1d-4ec2-aad2-1f240b2bc7ff}:data_up{dev_eui=“xxxxxxxxxxxxxxxx”}: chirpstack::storage::device_gateway: Gateway rx-info saved dev_eui=xxxxxxxxxxxxxxxx
2024-02-08T19:31:16.700044Z INFO up{deduplication_id=804f1cd9-3a1d-4ec2-aad2-1f240b2bc7ff}:data_up{dev_eui=“xxxxxxxxxxxxxxxx”}: chirpstack::storage::device_session: Device-session saved dev_eui=xxxxxxxxxxxxxxxx dev_addr=01542784
2024-02-08T19:31:16.701398Z INFO up{deduplication_id=804f1cd9-3a1d-4ec2-aad2-1f240b2bc7ff}:data_up{dev_eui=“xxxxxxxxxxxxxxxx”}: chirpstack::storage::metrics: Metrics saved name=device:xxxxxxxxxxxxxxxx aggregation=HOUR
2024-02-08T19:31:16.701637Z INFO up{deduplication_id=804f1cd9-3a1d-4ec2-aad2-1f240b2bc7ff}:data_up{dev_eui=“xxxxxxxxxxxxxxxx”}: chirpstack::storage::metrics: Metrics saved name=device:xxxxxxxxxxxxxxxx aggregation=DAY
2024-02-08T19:31:16.701852Z INFO up{deduplication_id=804f1cd9-3a1d-4ec2-aad2-1f240b2bc7ff}:data_up{dev_eui=“xxxxxxxxxxxxxxxx”}: chirpstack::storage::metrics: Metrics saved name=device:xxxxxxxxxxxxxxxx aggregation=MONTH
2024-02-08T19:31:16.702633Z INFO chirpstack::integration::http: Posting event event=up url=https://(stuff cut)
2024-02-08T19:31:16.702762Z INFO chirpstack::integration::mqtt: Publishing event topic=application/244f0d9f-00c4-4126-8be4-xxxxxxxxxxxxxxxx/device/xxxxxxxxxxxxxxxx/event/up
2024-02-08T19:31:16.805501Z INFO up{deduplication_id=804f1cd9-3a1d-4ec2-aad2-1f240b2bc7ff}:data_up{dev_eui=“xxxxxxxxxxxxxxxx”}:data_down{downlink_id=2651647479}: chirpstack::storage::device_queue: Device queue-item updated id=c933e436-b32b-4175-b1b0-f84443bb02be dev_eui=xxxxxxxxxxxxxxxx
2024-02-08T19:31:16.805835Z INFO up{deduplication_id=804f1cd9-3a1d-4ec2-aad2-1f240b2bc7ff}:data_up{dev_eui=“xxxxxxxxxxxxxxxx”}:data_down{downlink_id=2651647479}: chirpstack::storage::downlink_frame: Downlink-frame saved downlink_id=2651647479
2024-02-08T19:31:16.806201Z INFO up{deduplication_id=804f1cd9-3a1d-4ec2-aad2-1f240b2bc7ff}:data_up{dev_eui=“xxxxxxxxxxxxxxxx”}:data_down{downlink_id=2651647479}: chirpstack::storage::device_session: Device-session saved dev_eui=xxxxxxxxxxxxxxxx dev_addr=xxxxxxxx
2024-02-08T19:31:16.806281Z INFO up{deduplication_id=804f1cd9-3a1d-4ec2-aad2-1f240b2bc7ff}:data_up{dev_eui=“xxxxxxxxxxxxxxxx”}:data_down{downlink_id=2651647479}: chirpstack::gateway::backend::mqtt: Sending downlink frame region_id=eu868 gateway_id=xxxxxxxxxxxxxxxx topic=eu868/gateway/xxxxxxxxxxxxxxxx/command/down json=false

I see a “data_down”. Is this the downlink? If so where do I go to find out what happens next? Another container? Is there enough logging detail or do I need to change logging details?

Thanks for your time

The next step would be to check the logs on your gateway, if the downlinks aren’t making it to your gateway it could be a firewall issue.

And its normal for the downlinks to say pending = no when in the queue.

FYI using the “preformatted text” option is helpful in sharing logs.

This is the line that shows the downlink was sent from Chirpstack:

2024-02-08T19:31:16.806281Z INFO up{deduplication_id=804f1cd9-3a1d-4ec2-aad2-1f240b2bc7ff}:data_up{dev_eui=“xxxxxxxxxxxxxxxx”}:data_down{downlink_id=2651647479}: chirpstack::gateway::backend::mqtt: Sending downlink frame region_id=eu868 gateway_id=xxxxxxxxxxxxxxxx topic=eu868/gateway/xxxxxxxxxxxxxxxx/command/down json=false
1 Like

Thanks! With “logs on my gateway” do you mean the log on the gateway container? I mean: the actual physical LoRa gateway I have has a web UI but it is very basic and only allows you to set some parameters. Nothing indicating a log. In the container log I see it picking up the msg from the MQTT server but then it’s crickets:

time="2024-02-09T15:37:19.808489195Z" level=info msg="integration/mqtt: downlink frame received" downlink_id=XXXXXX gateway_id=xxxxxxxx

Is this the log that might help me? And if so should I set the log level to debug?

Peter

I had meant the physical LoRa gateway, as debugging the link from Chirpstack → gateway → device cannot be done entirely from Chirpstack, and often people have issues with routing between chirpstack and the gateway.

If I can ask what gateway are you using? I would be shocked if it had no logging function at all, if you cannot find one on the UI it may require SSH’ing into the gateway itself.

Not sure what you mean by the “gateway container” do you mean the gateway bridge? I don’t think the gateway bridge would be the issue but if you haven’t it might be useful to check the logs of everything on the server instead of just one container with: sudo docker-compose logs -f, then trigger another downlink and look for any errors or warnings.

Also setting the log level to debug can’t hurt.

It’s a RisingHF 8 Channels LoRaWAN Home gateway RHF2S024

Link: RisingHF Docs

I played around with the settings on that gateway and set the protocol from Semtech Packet Forwarder v1 to v2. And then I got an error

level:"ERROR"
code:"DOWNLINK_GATEWAY"
description:"TX_POWER"

So I dug deeper and set the downlink_tx_power from -1 to 20 for my region in the toml file. And now the error has gone and it looks like we’re getting somewhere. More to follow.

That sure is a basic web UI… I would try to SSH into it and dig around for some logs, another gateway RisingHF makes says:

RHF2S025B has two log output directories:/var/log and/home/rxhf/rlog

Maybe your gateway has a similar file structure?

Without seeing whether or not your gateway receives the downlink it’s hard to continue debugging.

And downlink power of -1 sounds like it would cause issues, weird that would be the default.

EDIT: My downlink power is set to -1 as well so I doubt that is the issue.

1 Like

I can confirm that setting the gw protocol to v2 and setting the tx_power to 20 solved it. Thanks for your help.

Interestingly enough the -1 is the default in the git repo of the config.

Glad you could fix it.

Interesting though, I guess Chirpstack’s -1 (which claims it sets it to the regional default) is setting it to a value too high for your gateway.