Unable to access payload in codec

Xornix · March 2, 2024, 9:30am

Hello,

I have a setup with a base-os gw version 4.1.1 using concentratord and MQTT forwarder connected to a chirpstack 4.6.0 docker instance.

The MQTT forwarder on the gateway is configured like that:

# For a complete configuration example and documentation, please refer to:
# https://www.chirpstack.io/chirpstack-mqtt-forwarder/configuration.html

[logging]
   level="info"
   log_to_syslog=true

[mqtt]
  topic_prefix="eu868"
  #topic_prefix=""
  #json=true
  server="tcp://X.X.X.X:1883"
  username=""
  password=""
  ca_cert=""
  tls_cert=""
  tls_key=""

[backend]
  enabled="concentratord"

  [backend.concentratord]
    event_url="ipc:///tmp/concentratord_event"
    command_url="ipc:///tmp/concentratord_command"

The chirpstack toml configuration on the server side is like that:



# Logging.
[logging]

  # Log level.
  #
  # Options are: trace, debug, info, warn error.
  level="info"


# PostgreSQL configuration.
[postgresql]

  # PostgreSQL DSN.
  #
  # Format example: postgres://<USERNAME>:<PASSWORD>@<HOSTNAME>/<DATABASE>?sslmode=<SSLMODE>.
  #
  # SSL mode options:
  #  * disable - no SSL
  #  * require - Always SSL (skip verification)
  #  * verify-ca - Always SSL (verify that the certificate presented by the server was signed by a trusted CA)
  #  * verify-full - Always SSL (verify that the certification presented by the server was signed by a trusted CA and the server host name matches the one in the certificate)
  dsn="postgres://chirpstack:chirpstack@$POSTGRESQL_HOST/chirpstack?sslmode=disable"

  # Max open connections.
  #
  # This sets the max. number of open connections that are allowed in the
  # PostgreSQL connection pool.
  max_open_connections=10

  # Min idle connections.
  #
  # This sets the min. number of idle connections in the PostgreSQL connection
  # pool (0 = equal to max_open_connections).
  min_idle_connections=0


# Redis configuration.
[redis]

  # Server address or addresses.
  #
  # Set multiple addresses when connecting to a cluster.
  servers=[
    "redis://$REDIS_HOST/",
  ]

  # TLS enabled.
  tls_enabled=false

  # Redis Cluster.
  #
  # Set this to true when the provided URLs are pointing to a Redis Cluster
  # instance.
  cluster=false


# Network related configuration.
[network]

  # Network identifier (NetID, 3 bytes) encoded as HEX (e.g. 010203).
  net_id="000000"

  # Enabled regions.
  #
  # Multiple regions can be enabled simultaneously. Each region must match
  # the 'name' parameter of the region configuration in '[[regions]]'.
  enabled_regions=[
    "eu868",
  ]


# API interface configuration.
[api]

  # interface:port to bind the API interface to.
  bind="0.0.0.0:8080"

  # Secret.
  #
  # This secret is used for generating login and API tokens, make sure this
  # is never exposed. Changing this secret will invalidate all login and API
  # tokens. The following command can be used to generate a random secret:
  #   openssl rand -base64 32
  secret="you-must-replace-this"


[integration]
  enabled=["mqtt"]

  [integration.mqtt]
    server="tcp://$MQTT_BROKER_HOST:1883/"
    #json=true

The custom codec is programmed like that:

function Decode2(fPort, bytes) {
  var tempObj = new Object();
  tempObj.byte1 = bytes[0];
  tempObj.byte2 = bytes[1];
  tempObj.byte3 = bytes[2];
  tempObj.byte4 = bytes[3];
  tempObj.byte5 = bytes[4];
  tempObj.byte6 = bytes[5];
  tempObj.byte7 = bytes[6];
  tempObj.byte8 = bytes[7];
  tempObj.byte9 = bytes[8];
  tempObj.byte10 = bytes[9];
  tempObj.byte11 = bytes[10];
  tempObj.byte12 = bytes[11];
  tempObj.byte13 = bytes[12];
 
  return tempObj;
}

function decodeUplink(input) {
   // Wrapper function for ChirpStack v4
  //var myData = Decode(input.fPort, input.bytes, input.variables);
  var myData = Decode2(input.fPort, input.bytes);
  myData.node_name = input.variables.node_name;
   return {
      data: myData
   };
}

Now, what I see is extremely strange.

The gw is properly receiving the payload from the IoT device and sends it to the server.

In the server, if I monitor the device, I am seeing very strange things.
When monitoring the frames, I am seeing things like that:

Multiple things to note here:

why is the data displayed in JSON and note base64 ?
I have (manually…) checked the data and they correspond to the right value, the value that the IoT device sent

When I am montiring the events, I am seeing things like that:

So, basically, after testing lots of things on the codec I came to the conclusion that I can’t really access the data in the received packets.

Does anyone know what I’m doing wrong here ?

Thanks in advance

Xornix · March 6, 2024, 1:13am

I think this is rather peculiar while following all the official rules I found in the wild. Has no one ever seen that / is able to explain it ?

Thanks

datnus · March 7, 2024, 10:10am

Is the packet photo uplink or downlink?

The “data” should be base64. It is empty in the second packet photo.
It should not be empty.

Object is decoded from your codec to json file.

Xornix · March 7, 2024, 9:46pm

Hello Datnus,

Both pictures are actually uplink. This is the strange thing: the data is empty in the [Events] tab but in the [LoRaWAN frames] tab, as pictured in my previous messages, all the proper data are here !!!

I know that the codec should decode what comes from data to json and that’s why I get all the « null » because the data is empty. What I don’t get is how the data can be empty if it’s perfectly seen in the frame (it’s in the « frm_payload » in the « Raw » part on the first pic)?

Thanks anyway for taking the time to look into it.

IoTThinks · March 8, 2024, 8:25am

Are you using DIY LoRaWAN node or commercial LoRaWAN node?
Looking at your packets, the fport=0.
For normal uplink, the fPort should be > 0. fPort=0 is usually for MAC commands only.

For your reference and your debug, this is my packet photo.
My fPort is 2. My node puts fPort=2 before uploading.

In LoRaWAN Frame, the payload is still encrypted in frm_payload.

In Events, the frm_payload is decrypted into data.
data is in base64 format.
Then codec will decode data into object.

May be you can try to use Unconfirmed Uplink same as mine and see if you see similar behavior.

Xornix · March 8, 2024, 2:02pm

Thanks for taking the Time to answer and for your feedback.
From prior experience, I don’t think that port 0 would be an issue but I agree that it’s worth testing it. Same for unconfirmed vs confirmed.

I don’t have an easy access to the IoT device but I’ll try to update it ASAP (probably a few weeks) and let you know.

Thanks again for your proposals.

brocaar · March 12, 2024, 3:32pm

fport: 0 means that the frmPayload holds mac-commands. ChirpStack will try to decode these as mac-commands, which fails, which is why you are seeing these as Raw.

Xornix · March 12, 2024, 7:28pm

Thanks a lot. That clears it once and for all.
I won’t have access to the device until the end of march but will confirm shortly afterwards.

Thanks a lot for taking the time to drop by.

Xornix · March 30, 2024, 11:07pm

Confirmed: by changing fPort to 1, issues were solved. Thanks a lot again.