MQTT JSON Downlink payload conversion to semtech UDP Downlink format

mightyChamp · September 24, 2021, 8:03am

Hello,

I am using chripstack on my server, My gateway is custom made gateway, I want to use MQTT protocol rather than Semtech UDP Packet Forwarder. I get success in uplinks using MQTT packet structure as mentioned here. But In downlink, json packet, some parameters are missing comapred to semtech udp downlink packet.

Semtech Downlink Json:
{“txpk”:{
“imme”:true,
“freq”:864.123456,
“rfch”:0,
“powe”:14,
“modu”:“LORA”,
“datr”:“SF11BW125”,
“codr”:“4/6”,
“ipol”:false,
“size”:32,
“ncrc”:true,
“data”:“H3P3N2i9qc4yt7rK7ldqoeCVJGBybzPY5h1Dd7P7p8v”
}}

Chripstack MQTT Downlink Json :

{
“phyPayload”:“YGcsBACQAAAAGzv2H29bigkqF3roPD1jQ5tyAJkJqZIn09HH”,
“txInfo”:{
“gatewayID”:“fJ69z///vKA=”,
“frequency”:865062500,
“power”:27,
“modulation”:“LORA”,
“loRaModulationInfo”:{
“bandwidth”:125,
“spreadingFactor”:12,
“codeRate”:“4/5”,
“polarizationInversion”:true
},
“board”:0,
“antenna”:0,
“timing”:“DELAY”,
“delayTimingInfo”:{
“delay”:“1s”
},
“context”:null
},
“token”:14831,
“downlinkID”:“Oe9C1x0dSkKLZO2gMwmpGw==”,
“items”:[
{
“phyPayload”:“YGcsBACQAAAAGzv2H29bigkqF3roPD1jQ5tyAJkJqZIn09HH”,
“txInfo”:{
“gatewayID”:null,
“frequency”:865062500,
“power”:27,
“modulation”:“LORA”,
“loRaModulationInfo”:{
“bandwidth”:125,
“spreadingFactor”:12,
“codeRate”:“4/5”,
“polarizationInversion”:true
},
“board”:0,
“antenna”:0,
“timing”:“DELAY”,
“delayTimingInfo”:{
“delay”:“1s”
},
“context”:null
}
},
{
“phyPayload”:“YGcsBACQAAAAGzv2H29bigkqF3roPD1jQ5tyAJkJqZIn09HH”,
“txInfo”:{
“gatewayID”:null,
“frequency”:866550000,
“power”:27,
“modulation”:“LORA”,
“loRaModulationInfo”:{
“bandwidth”:125,
“spreadingFactor”:10,
“codeRate”:“4/5”,
“polarizationInversion”:true
},
“board”:0,
“antenna”:0,
“timing”:“DELAY”,
“delayTimingInfo”:{
“delay”:“2s”
},
“context”:null
}
}
],
“gatewayID”:“fJ69z///vKA=”
}

Missing Parameter in Chripstack MQTT JSON Data:

timestamp to transmit data
RF Chain to be use
Data Size
CRC

I cannot use chripstack gateway bridge to convert MQTT to semtech UDP packet format, as only want to use MQTT. So, I need help to get missing data information.

How gateway bridge code converts MQTT JSON format to UDP format?
How can I get 4 missing parameter from MQTT JSON data?

Thank You

mightyChamp · September 28, 2021, 10:25am

Hello,
Please help me as soon as possible.
Thanks You

fmgst · September 28, 2021, 2:28pm

This will limit who can help answer your question. You may need to way some weeks…

cstratton · September 28, 2021, 4:01pm

Not really, because it’s a pure protocol question having nothing to do with the ultimate target.

However, the asker could do a lot more to try to figure this out - if one is going to make custom things, that’s pretty routinely necessary.

For example, one of the shown examples is a packet in immediate mode, which by definition doesn’t have a transmit time stamp.

The two main ways to sort this out would be to look at the interface specs (eg in the Semtech repo) and the code, and to watch the provided gateway bridge in operation by running a packet dump on the UDP ports and comparing to the backhaul protocol.

A little research effort would have lead one to this:

github.com

brocaar/chirpstack-api/blob/master/protobuf/gw/gw.proto#L318


      
              DownlinkTiming timing = 12;
          
              oneof timing_info {
                  // Immediately timing information.
                  ImmediatelyTimingInfo immediately_timing_info = 13;
          
                  // Context based delay timing information.
                  DelayTimingInfo delay_timing_info = 14;
          
                  // GPS Epoch timing information.
                  GPSEpochTimingInfo gps_epoch_timing_info = 15;
              }
          
              // Gateway specific context.
              // In case of a Class-A downlink, this contains a copy of the uplink context.
              bytes context = 16;
          }
          
          message ImmediatelyTimingInfo {
              // Not implemented yet.
          }

Where it can be learned that “timing”:“DELAY” makes it relative to a 16-byte gateway context (eg from an uplink) which the gateway or bridge needs to internally know the timing of.

That reveals another problem with the example being examined is that this context is null. To me that sounds like a “should not happen” case (what’s the purpose of these downlinks - gateay to gateway pings? since the polarity is inverted like a conventional downlink, probably not) but if I had to implement something to deal with it and decided they were valid transmit requests, I’d probably just map them back to immediate mode transmissions.

As a guess, one probably could just use the traditional 32-bit uplink timestamps to form contexts, and then do a little simple addition when calculating the corresponding downlink times.

Or it could be a more sophisticated scheme with a dynamic list of usefully recent uplinks that downlinks can be referenced to, which expire once they’re too stale to be of any use.

(sorry about the highlight on the GPS line, that’s not intended, just a good place to center the snipped to show the relevant info for delay timing)

mightyChamp · September 29, 2021, 6:26am

Thanks for your support.
I have full knowledge about Semtech UDP protocol and their uplink, downlink mechanism. but as mentioned I want to gateway to be only MQTT Compatible. for that I need to use chripstack MQTT Json formats to decode the data, but chripstack protocol, json tags are somewhat differ from semtech udp protocol.

I have tried to understand mechanism used by chripstack gateway bridge code which converts Semtech UDP Protocol JSON format to chripstack mqtt protocol json frame, now I got some functional logic for how to convert data. but if there is any protocol document available, it would help me lot. as It would give surety about my code implementation.

I didn’t get any logic for crc to be enable or disable from gateway bridge code, and which RF chain to be used. Help me regarding this.

Thank you

cstratton · September 29, 2021, 2:16pm

This was just linked. Not that you couldn’t easily have found it yourself - it took me about 3 minutes to find it.

Do you truly understand the purpose of those?

If you do, you’ll see where the information is in the linked documentation and possibly in the uplink context.

If you actually understand their purpose, you’ll also know what the settings need to be.

mightyChamp · October 1, 2021, 5:27am

So could you please give me this link which shows how Chripstack MQTT Json is converted into Semtech UDP Json Packet?

Thanks for your support.

cstratton · October 1, 2021, 4:03pm

Again, the link to the chirpstack protocol was already posted in this thread. You’re presumably already aware that the semtech version is documented in their packet forwarder repo.

Speaking as someone who’s done it making a custom gateway that runs custom software is an advanced task that is going to require some real effort in taking time to understand how things work based on the available documentation, and if/when that fails reading the existing source code.

You may want to consider using a configuration where you can run a stock version of the gateway bridge.

mightyChamp · October 2, 2021, 4:55am

Sorry for this but I didn’t get link, which link you taking about , it is this one:

brocaar/chirpstack-api/blob/master/protobuf/gw/gw.proto#L318

Currently My custom gateway works on MQTT and written c language, with my own Server code which works perfect fine, I used directly semtech UDP packet structure over MQTT, but My server side code is not as featured as chripstack code, and Chripstack also supports MQTT. So I decided to switch to Chripstack, that’s why I am asking for packet conversion mechanism code theory, So I can develop in my code.

Thank You