Reusing vendor CODECs from TTN V3 github in ChirpStack Device Profile

fmgst · August 30, 2021, 5:44pm

TTN has a repository of vendor-curated CODECs which is appealing to me, since I don’t have time to reinvent wheels.

I got to ask, are there any plans to allow ChirpStack to use these natively (drop-in)?

Currently, I got TTN code to compile by appending the following to the end (calls Decode function with sample data object - but it’s based on actual data being reported by a Laird RTD sensor, 455-00098):

function Decode(fPort, bytes, variables) {
  return decodeUplink(bytes);
}

sensordata = 
{
  "bytes": [0x0b,0x01,0x00,0x06,0x00,0x13,0x05,0x00,0x00,0x00,0x00], 
  "fPort": 1
};
console.log(sensordata);
console.log(Decode(1, sensordata, ""));

Which outputs:

Output:

[
  11, 1, 0, 3, 0,
  12, 5, 0, 0, 0,
   0
]
{
  data: {
    msgType: 'Laird_RTD',
    options: [ 'Sensor request for server time' ],
    temperature: 12.03,
    batteryCapacity: '80-100%',
    alarmMsgCount: 0,
    backlogMsgCount: 0
  }
}

The Protocol Manual agrees with this decoding and, most importantly, this is using stock TTN V3 CODEC!

However, while it compiles and decodes sample data, I can’t get anything on ChirpStack except an error:

// ChirpStack expects different function name
function Decode(fPort, bytes, variables) {
  //var sensordata = { "bytes": bytes, "fPort": fPort };
  return decodeUplink(bytes); //sensordata);
}

DEVICE DATA error contains:

applicationID:"5"
applicationName:"DSE-App2021"
deviceName:"rs-blue-RTD"
devEUI:"0025ca0a0000f616"
type:"UPLINK_CODEC"
error:"execute js error: js vm error: TypeError: Cannot access member '0' of undefined"
fCnt:1
tags:{} 0 keys
publishedAt:"2021-08-30T20:44:10.160990126Z"

Grr. Does anyone see why this might be?

Note1: My chirpstack is on intranet so I can not use integrations, meaning that my devices need to be decoded inside ChirpStack. Some vendors make this a challenge (I’m looking at you, radiobridge.com), but even for the ones that do release CODECs, there is always the problem of maintaining support for new devices. Thus, it’s always nicer to ‘borrow’ maintained code in these situations, rather than hacking stuff found on the net.

Note2: Don’t know about you, but my knowledge of javascript is very primitive/sucks so THIS online compiler came in very handy when trying to figure out what the somewhat cryptic error message in DEVICE DATA tab meant (e.g., "error:“execute js error: js vm error: TypeError: Cannot access member ‘0’ of undefined”).

fmgst · August 30, 2021, 11:59pm

Bad News: this can’t be done in ChirpStack until otto, which is used to execute JavaScript decoders, is upgraded to support ES6 features such as TypedArrays. So, looks like my plan is DOA.

I looked at the particular decoder and it uses DataView and ArrayBuffer. Then, I found that there may be an “automated” way of converting that to ES5 using Babel and core-js (which emulates ES6 with ES5 code, for the purpose of supporting older browsers). I haven’t figured out yet how to run it - looks like I need to install some packages but it should produce what I’m after. Or NOT - looks like somebody ran into further issues attempting this.

Q: Also, does anyone know of a way of running otto manually, from command line? Currently, I’m hacking code, pasting it into payload-profile, and forcing sensor to send some data - this is slow & tedious/PITA.

A: Yes, it can be done as explained here. However, running THIS codec with command-line version otto I get:

./otto test_decoder.js 
[object Object]

BUT running identical code using on-line compiler gives me a different answer:

Output:
{ E: '150.127', U: 'MWh', ID: 67583954 }

So, looks like Babel is NOT the solution when it’s otto that’s running resulting JS code…

brocaar · September 6, 2021, 3:32pm

It doesn’t solve your issue right now, but I’m working on some changes that will improve the codec functions (in terms of speed and JS compatibility). I know this can be a bottleneck, and I’m looking forward to move away from ES5 only support

fmgst · September 7, 2021, 1:08am

Thank you thank you thank you! This is my only complaint with your otherwise stellar software!

jjanderson · June 8, 2022, 8:57am

Hello usual suspects I thought I would share this quick and dirty little hack I stumbled across whilst trying to decode some dodgy oem’s undocumented trinkets… @fmgst funny, I got the idea from how radiobridge does chirpstack…

Essentially, you start off like this:

function Decode(fPort, bytes, variables) {
    return Generic_Decoder(bytes, fPort);
}

and then you simply take the TTN codec and rename it from Decoder(port, bytes) to Generic_Decoder(port, bytes)

and paste it underneath… bish bash bosh!

Well, at least it seems to be working out ok for what I am trying… hope this helps…

brocaar · June 13, 2022, 11:04am

Please note, that ChirpStack v4 will support this Last week I merged this into the master branch. After cloning the TTN lorawan-devices repo, you can import these devices as “device-profile templates”. You can find the v4 draft documentation here: Device-profile templates - ChirpStack open-source LoRaWAN® Network Server documentation.

See also: Help testing ChirpStack v4 (test releases)!.

jjanderson · June 13, 2022, 12:02pm

@brocaar you legend! This is AWESOME! Thanks so much for your relentless work on this!

jjanderson · June 13, 2022, 12:45pm

Hi, where do you input the command chirpstack -c /etc/chirpstack import-ttn-lorawan-devices -d /opt/lorawan-devices in order to import the TTN device profile templates? Is this in the docker contiainer running the application server? container → [chirpstack/chirpstack:4.0.0-test.5] ?

brocaar · June 13, 2022, 1:05pm

You will find this CLI option in the :4.0.0-test.6 tag which was pushed a few hours ago

jjanderson · June 13, 2022, 1:48pm

ok, I installed test 6, but still battling on where I actually run the command. There is no reference to CLI in the UI, do I go into the Application server CLI and run this?

brocaar · June 14, 2022, 10:35am

Correct, you must run this from the CLI. This might need further documentation (or maybe I can implement this in an automated way for the ChirpStack Docker repo).

E.g.

 docker run --rm -it --entrypoint bash chirpstack/chirpstack:4.0.0-test.6

And inside:

nobody@e24096cd9bf5:/$ chirpstack --help
chirpstack 4.0.0-test.6
Orne Brocaar <info@brocaar.com>
ChirpStack open-source LoRaWAN network-server

USAGE:
    chirpstack --config-dir <DIR> [SUBCOMMAND]

FLAGS:
    -h, --help       Prints help information
    -V, --version    Prints version information

OPTIONS:
    -c, --config-dir <DIR>    Path to configuration directory

SUBCOMMANDS:
    configfile                    Print the configuration template
    help                          Prints this message or the help of the given subcommand(s)
    import-ttn-lorawan-devices    Import TheThingsNetwork LoRaWAN devices repository
    print-ds                      Print the device-session for debugging

Not tested, but you could do the same using Compose (which is what you probably want, as you need the database to be up + the configuration to be present):

docker-compose run --rm --entrypoint bash chirpstack

(again I have not tested the above, but it should give you a starting point)

jjanderson · June 14, 2022, 11:19am

awesome, thanks will give it a go through. Maybe in future releases, these templates can be automatically included as I can see that most people would want to include it

brocaar · July 1, 2022, 9:35am

Yes, I think that is a good idea. In this case “included” would mean that these templates aren’t included within the ChirpStack package itself, but that there would be scripts included to make this as easy as possible (and that these provides can be updated at any time, separate from the ChirpStack version).

jjanderson · July 20, 2022, 4:17pm

awesome all working great!

Anni1 · September 15, 2022, 3:35pm

Hi,
who can help me with the correct decode format for my Global Sat LT-20 device? I have a code that is working on TTN. I tried to decode it like jjanderson suggested but is gives me still errors.
This is the code i tried to put in the Chirpstack decoder.
this is the decoder for the device in TTN: This is a javascript decoder for the GlobalSat LT-20 LORAWAN GPS tracker. The decoder is built to work with TheThingsNetwork/Stack V3 · GitHub

fmgst · September 15, 2022, 4:01pm

You should start a new topic, not piggyback on one that was closed 2 months ago. Basically, Chirpstack calls the function Decode(fPort, bytes, variables), but the codec you’re using starts in function decodeUplink(input), so you will need to modify it a bit.

Anni1 · September 15, 2022, 4:25pm

sorry, I will open a new topic. I know that need I to change the code to function Decode(fPort, bytes, variables). I was using the example from jjanderson and modified it but I was not able to copy in the code into this chat thats why I copied the link.