Json format / MQTT subscription / objectJSON

ChirpStack Gateway Bridge
1

Hi everyone,
I publish here because i see there are more active discussion around MQTT .
I’m working with:
chirpstack-application-server version : 3.10.0
chirpstack-network-server version : 3.9.0
chirpstack-gateway-bridge version : 3.8.0

All on a rpi with raspbian buster…

The problem is when i subscribe to my application topic events i receive the “objectJSON” as a string format (double quoted and unescaped ) like if its stringify twice during the transport…

If i do this command on the gateway:
mosquitto_sub -t "application/3/#" -v

application/3/device/12345678910/rx {"applicationID":"3","applicationName":"sondehumtemp","deviceName":"node01","devEUI":"123456789=",
"rxInfo":[{"gatewayID":"3KY123456=","time":"2020-05-09T09:15:24.313107Z","timeSinceGPSEpoch":"1273050941.313s","rssi":-53,"loRaSNR":8,"channel":2,"rfChain":1,"board":0,"antenna":0,"location":{"latitude":0,"longitude":0,"altitude":0,"source":"UNKNOWN","accuracy":0},"fineTimestampType":"NONE","context":"d+GjNA==","uplinkID":"qZLyA4YOTOG6zkNcAp6kBw==","crcStatus":"CRC_OK"}],
"txInfo":{"frequency":868500000,"modulation":"LORA","loRaModulationInfo":{"bandwidth":125,"spreadingFactor":12,"codeRate":"4/5","polarizationInversion":false}},
"adr":true,
"dr":0,
"fCnt":1221,
"fPort":8,
"data":"CAIBbgdoawZzJbcCZwCvBAIDvQ==",
"objectJSON":"{\"barometer\":965.5,\"battery\":3.66,\"gasResistance\":9.57,\"humidity\":53.5,\"temperature\":17.5}","tags":{}}

The induce problem is i cant do JSON.parse(myMQTTmessage).
error unexpected token at position...

I use this script for decode payload data, that you can found here: (i have just remove all the unit for a simpler integration…)

Just to add : i have integrate this “data events” into a postgres data base … all look very good , I could “query” without problems differents elements of the objectJSON.

i have ask on the RAK forum if they have the same problem … they answer that with the same script but an old version of chirpstack the result is different (in their case no problems).

Thanks for the help.

2

This might be an issue with the JSON parsing, I have checked www.jsonlint.com and the JSON payload is valid. Please note that objectJSON must be seen as a string, not an object. That string needs to be parsed separately.

Please see the definition of this message here: chirpstack-api/protobuf/as/integration/integration.proto at master · brocaar/chirpstack-api · GitHub.

3

Thanks for your quick answer!
OK its like this … the ObjectJSON is receive as “string” !!
So i have to use a different way to extract it from the MQTT message… Why not if i have the control on the MQTT client. but that not the case for certain applications. doesn’t matter.
I will try to get the base64 data first and decode them in a second time with an other task.
Its certainly more efficient than to send decoded data …
Efficient but more complicated…

At the end again thanks for your quick answer!

4

When i have started to search why i get this “string object” i ask to RAK if its come from their decode payload script. and we have a conversation about it.
The discussion is here: Scripts to decode RAK node(RAK5205/RAK7205/RAK7200/RAK7204) sensor data for ttn and chirpstack - #12 by kaptepatou - LPWAN Nodes - RAKwireless Forum

They dont have the same result like me.
please have a look on the result of a mosquitto_sub with the RAK’s version:

application/2/device/3739343557376202/rx {“applicationID”:“2”,“applicationName”:“1”,“deviceName”:“3739343557376202”,“devEUI”:“3739343557376202”,“txInfo”:{“frequency”:487100000,“dr”:5},“adr”:true,“fCnt”:6,“fPort”:8,“data”:“CAIBogdoLAZzJW8CZwFHBAIR5QNxAxkAJf2n”,“object”:{“acceleration_x”:0.793,“acceleration_y”:0.037,“acceleration_z”:-0.601,“barometer”:958.3,“battery”:4.18,“gasResistance”:45.81,“humidity”:22,“temperature”:32.7}}

The decoded data named here object and not objectJSON is directly in Json format (keys + fields).

I’m very bad with javascript and Json so maybe i’m totally wrong about this but :

  • Is it possible to have an option to control this format? (into config file) ?
  • I have a mqtt client who wait for data into key + field format… so with the format “string” i cant do anything (i haven’t the possibility to apply a function or something like this). and i suppose i’m not the only one in this position.
5

https://www.json.org/json-en.html

6

It is already there :wink:

  # Integration configures the data integration.
  #
  # This is the data integration which is available for all applications,
  # besides the extra integrations that can be added on a per-application
  # basis.
  [application_server.integration]
  # Payload marshaler.
  #
  # This defines how the MQTT payloads are encoded. Valid options are:
  # * protobuf:  Protobuf encoding
  # * json:      JSON encoding (easier for debugging, but less compact than 'protobuf')
  # * json_v3:   v3 JSON (will be removed in the next major release)
  marshaler="json_v3"

I recommend you to use the json or protobuf however as it is using the Protobuf message format as defined here: chirpstack-api/protobuf/as/integration/integration.proto at master · brocaar/chirpstack-api · GitHub

7

Ok some aspect are clearer. Thanks.
I understand too why Rak have not the same MQTT message with the same codec.
They use the Marsahler Json_v3. And as i use the marshaler JSON , difference come from here.

For me protobuff is (impossible) to integrate with my MQTT client. Json only is supported .
So i could use the Json_v3 Marshaler and it will working perfectly .
But first : it said that this encoding type will be deprecated. So i prefer starting my integration with JSON.

second, My base problem persist. i can’t get a JsonObject with “keys” and “fields” to integrate it easely on my MQTT client. As it’s appear as a String with the JSON marshaler.
Of course i understand that normaly the MQTT message could be parse and modified by the MQTT client. But with my MQTT it’s only Json Key and field. (In the field it wait for the final value )

So i will understand that this MQTT client is poor and i have to look for a Client with better configurations option. But at the end it’s not compatible.

I gonna see with the suplier of my MQTT client if they could do something for a better integration .