Rermove all metadata from json except payload

ChirpStack Gateway Bridge
1

how can I remove all the metadata from json, so that I only get the json with the actual sensor values and perhaps an ID in mosquitto_sub?

2

is there a way to reshape the json coming out from chirpstack and going to the mosquitto broker?
instead of this:

{
“applicationID”: “2”,
“applicationName”: “LHT65”,
“deviceName”: “LHT65-FInnen”,
“devEUI”: “a840412171822755”,
“rxInfo”: [
{
“gatewayID”: “a840411d92904150”,
“uplinkID”: “1ce91da0-dd85-4c97-9992-cf21a5c3ea99”,
“name”: “HCDragino”,
“time”: “2012-01-03T17:40:40.77952Z”,
“rssi”: -33,
“loRaSNR”: 11.8,
“location”: {
“latitude”: 0,
“longitude”: 0,
“altitude”: 77
}
}
],
“txInfo”: {
“frequency”: 867300000,
“dr”: 0
},
“adr”: true,
“fCnt”: 74,
“fPort”: 2,
“data”: “zBsNagFcAX//f/8=”,
“object”: {
“BatV”: 3.099,
“Ext_sensor”: “Temperature Sensor”,
“Hum_SHT”: “34.8”,
“TempC_DS”: “327.67”,
“TempC_SHT”: “34.34”
}
}

I want only this:

{
“BatV”: 3.099,
“Ext_sensor”: “Temperature Sensor”,
“Hum_SHT”: “34.8”,
“TempC_DS”: “327.67”,
“TempC_SHT”: “34.34”
}

3

You could deselect “Add gateway meta-data” in the service profile.

4

Thx, that helped. However there is still some metadata and I need to move the object json one level up:

{
“applicationID”: “3”,
“applicationName”: “HCFassadenWyler”,
“deviceName”: “HCLoraWyler”,
“devEUI”: “a8610a32332e8209”,
“txInfo”: {
“frequency”: 868300000,
“dr”: 0
},
“adr”: true,
“fCnt”: 1053,
“fPort”: 19,
“data”: “ADiSEjiSEjfRj/+UNP+UNP9+iA==”,
“object”: {
“SensorId1”: 1,
“SensorId2”: 2,
“dateTimeSend”: “2020-08-12T11:30:14.619+02:00”,
“ftPitch”: -0.033144,
“ftRoll”: 3.6581270000000004,
“id”: “WB01”,
“period”: 150,
“rPitch”: -0.027596000000000002,
“rRoll”: 3.70741,
“tPitch”: -0.027596000000000002,
“tRoll”: 3.70741
}
}

The reason for this is I am sending this data into the Watson IoT and they can only parse it in this particular format. I would like to avoid using an API or external app,and use the Chirpstack functionality as far as possible

5

Short of an official integration, you will probably need to write a service that massages the data into the desired format.

1 Like
6

using the chirpstack API?

7

Not really. You could do a HTTP integration that publishes to the desired topic or you do a more “general” integration with mqtt.
subscribe to the application topics
Clean up the received Event
Do topic mapping if required
publish cleaned up data to Watson IoT

If you are using just one Broker you will need to do a mapping to separate the data streams, with two brokers you can keep the topics.
https://www.chirpstack.io/application-server/integrations/http/
https://www.chirpstack.io/application-server/integrations/mqtt/

8

Thanks for the reply. Do I need to write a service in e.g. Python? The service listens on a topic which has the ChirpStack signature. Upon event received, it reformats the json and publishes under a new topic which is mapped to Watson IoT in the mosquitto bridge configuration.
Is this what you mean, or is it possible to do an integration directly in Chirpstack using config-settings in the .toml files, without coding?

9

Hi dng;
This is actually a common requirement. Data stream processing is important to message the data into a proper format before storage or additional analytics. You could use Python, or Nodejs or other applications to perform this function. If you want, we also have these applications already running and can assist you if you wish. Drop me a note at clay@penteon.io if you want to take this offline.

10

You’ll have to do a bit of coding.
For quick&dirty I’d said go with node-red.
If you want something more “stable” have a look at https://github.com/wobcom/lorawan-qos
it uses the jsonv3 messages and not the latest “Event-messages”, but you get an idea.
You could use https://www.ibm.com/cloud/functions and implement a HTTP integration that way and then pump it to IoT

11

Thx I will look at this. I wrote a python script which I implemented as a service on the rpi which hosts the mosquitto broker. The python subscribes to a topic filter, then on message, it reformats the json and publishes it again. In the broker config there are topic mappings for the bridge configuration, which relay the newly formatted packets to Watson. Not particularly elegant. I think IBM need to solve this on their side.

1 Like
12

A quick go would be to use node-red for this (hosted for free or on your server), I’m using it every day for this kind of issues, subscribe to topic with MQTT node, link to a function node something like that

var  payload =  { 
  BatV : msg.payload.BatV,
  Ext_sensor : msg.payload.Ext_sensorSensor,
  Hum_SHT: msg.payload.Hum_SHT,
  TempC_DS: msg.payload.TempC_DS,
  TempC_SHT: msg.payload.TempC_SHT
 };
 
msg.payload = payload;
return msg

Then link the output to whatever you like

But python or just JS can do the stuff, but nodered will be shorter code to be done by yourself