HTTP integration with Python script

Applications
1

Hi,

I took the python script provided in the following link to get uplink data from application sever with POST request.

But I think we need a protocol buffer file as per this line

“from chirpstack_api.as_pb import integration”

but I am not sure where to find the protocol buffer file as the page above does not tell where it can be located.

When I tun the script I was getting this error (running script from python 3 docker container )

python_chirpstack_adapter_1  | Traceback (most recent call last):
python_chirpstack_adapter_1  |   File "/usr/src/app/./main.py", line 4, in <module>
python_chirpstack_adapter_1  |     from chirpstack_api.as_pb import integration
python_chirpstack_adapter_1  |   File "/usr/local/lib/python3.11/site-packages/chirpstack_api/as_pb/__init__.py", line 1, in <module>
python_chirpstack_adapter_1  |     from .as_pb_pb2_grpc import *
python_chirpstack_adapter_1  |   File "/usr/local/lib/python3.11/site-packages/chirpstack_api/as_pb/as_pb_pb2_grpc.py", line 5, in <module>
python_chirpstack_adapter_1  |     from chirpstack_api.as_pb import as_pb_pb2 as chirpstack__api_dot_as__pb_dot_as__pb__pb2
python_chirpstack_adapter_1  |   File "/usr/local/lib/python3.11/site-packages/chirpstack_api/as_pb/as_pb_pb2.py", line 17, in <module>
python_chirpstack_adapter_1  |     from chirpstack_api.common import common_pb2 as chirpstack__api_dot_common_dot_common__pb2
python_chirpstack_adapter_1  |   File "/usr/local/lib/python3.11/site-packages/chirpstack_api/common/__init__.py", line 1, in <module>
python_chirpstack_adapter_1  |     from .common_pb2 import *
python_chirpstack_adapter_1  |   File "/usr/local/lib/python3.11/site-packages/chirpstack_api/common/common_pb2.py", line 33, in <module>
python_chirpstack_adapter_1  |     _descriptor.EnumValueDescriptor(
python_chirpstack_adapter_1  |   File "/usr/local/lib/python3.11/site-packages/google/protobuf/descriptor.py", line 755, in __new__
python_chirpstack_adapter_1  |     _message.Message._CheckCalledFromGeneratedFile()
python_chirpstack_adapter_1  | TypeError: Descriptors cannot not be created directly.
python_chirpstack_adapter_1  | If this call came from a _pb2.py file, your generated code is out of date and must be regenerated with protoc >= 3.19.0.
python_chirpstack_adapter_1  | If you cannot immediately regenerate your protos, some other possible workarounds are:
python_chirpstack_adapter_1  |  1. Downgrade the protobuf package to 3.20.x or lower.
python_chirpstack_adapter_1  |  2. Set PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=python (but this will use pure-Python parsing and will be much slower).
python_chirpstack_adapter_1  | 
python_chirpstack_adapter_1  | More information: https://developers.google.com/protocol-buffers/docs/news/2022-05-06#python-updates
python_scripts_python_chirpstack_adapter_1 exited with code 1

I resolved the above error by adding environment variable “PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION” at start of the file like this

import os
os.environ["PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION"]='python'

Now when I send an uplink I am getting this error

python_chirpstack_adapter_1  | 103.92.102.92 - - [17/Nov/2022 13:30:32] "POST /?event=up HTTP/1.1" 200 -
python_chirpstack_adapter_1  | ----------------------------------------
python_chirpstack_adapter_1  | Exception occurred during processing of request from ('103.92.102.92', 50898)
python_chirpstack_adapter_1  | Traceback (most recent call last):
python_chirpstack_adapter_1  |   File "/usr/local/lib/python3.11/site-packages/google/protobuf/json_format.py", line 544, in _ConvertFieldValuePair
python_chirpstack_adapter_1  |     raise ParseError(
python_chirpstack_adapter_1  | google.protobuf.json_format.ParseError: Message type "integration.UplinkEvent" has no field named "deduplicationId" at "UplinkEvent".
python_chirpstack_adapter_1  |  Available Fields(except extensions): "['applicationID', 'applicationName', 'deviceName', 'devEUI', 'rxInfo', 'txInfo', 'adr', 'dr', 'fCnt', 'fPort', 'data', 'objectJSON', 'tags', 'confirmedUplink', 'devAddr', 'publishedAt', 'deviceProfileID', 'deviceProfileName']"
python_chirpstack_adapter_1  | 
python_chirpstack_adapter_1  | During handling of the above exception, another exception occurred:
python_chirpstack_adapter_1  | 
python_chirpstack_adapter_1  | Traceback (most recent call last):
python_chirpstack_adapter_1  |   File "/usr/local/lib/python3.11/socketserver.py", line 317, in _handle_request_noblock
python_chirpstack_adapter_1  |     self.process_request(request, client_address)
python_chirpstack_adapter_1  |   File "/usr/local/lib/python3.11/socketserver.py", line 348, in process_request
python_chirpstack_adapter_1  |     self.finish_request(request, client_address)
python_chirpstack_adapter_1  |   File "/usr/local/lib/python3.11/socketserver.py", line 361, in finish_request
python_chirpstack_adapter_1  |     self.RequestHandlerClass(request, client_address, self)
python_chirpstack_adapter_1  |   File "/usr/local/lib/python3.11/socketserver.py", line 755, in __init__
python_chirpstack_adapter_1  |     self.handle()
python_chirpstack_adapter_1  |   File "/usr/local/lib/python3.11/http/server.py", line 431, in handle
python_chirpstack_adapter_1  |     self.handle_one_request()
python_chirpstack_adapter_1  |   File "/usr/local/lib/python3.11/http/server.py", line 419, in handle_one_request
python_chirpstack_adapter_1  |     method()
python_chirpstack_adapter_1  |   File "/usr/src/app/./main.py", line 24, in do_POST
python_chirpstack_adapter_1  |     self.up(body)
python_chirpstack_adapter_1  |   File "/usr/src/app/./main.py", line 33, in up
python_chirpstack_adapter_1  |     up = self.unmarshal(body, integration.UplinkEvent())
python_chirpstack_adapter_1  |          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
python_chirpstack_adapter_1  |   File "/usr/src/app/./main.py", line 42, in unmarshal
python_chirpstack_adapter_1  |     return Parse(body, pl)
python_chirpstack_adapter_1  |            ^^^^^^^^^^^^^^^
python_chirpstack_adapter_1  |   File "/usr/local/lib/python3.11/site-packages/google/protobuf/json_format.py", line 436, in Parse
python_chirpstack_adapter_1  |     return ParseDict(js, message, ignore_unknown_fields, descriptor_pool,
python_chirpstack_adapter_1  |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
python_chirpstack_adapter_1  |   File "/usr/local/lib/python3.11/site-packages/google/protobuf/json_format.py", line 461, in ParseDict
python_chirpstack_adapter_1  |     parser.ConvertMessage(js_dict, message, '')
python_chirpstack_adapter_1  |   File "/usr/local/lib/python3.11/site-packages/google/protobuf/json_format.py", line 502, in ConvertMessage
python_chirpstack_adapter_1  |     self._ConvertFieldValuePair(value, message, path)
python_chirpstack_adapter_1  |   File "/usr/local/lib/python3.11/site-packages/google/protobuf/json_format.py", line 629, in _ConvertFieldValuePair
python_chirpstack_adapter_1  |     raise ParseError(str(e))
python_chirpstack_adapter_1  | google.protobuf.json_format.ParseError: Message type "integration.UplinkEvent" has no field named "deduplicationId" at "UplinkEvent".
python_chirpstack_adapter_1  |  Available Fields(except extensions): "['applicationID', 'applicationName', 'deviceName', 'devEUI', 'rxInfo', 'txInfo', 'adr', 'dr', 'fCnt', 'fPort', 'data', 'objectJSON', 'tags', 'confirmedUplink', 'devAddr', 'publishedAt', 'deviceProfileID', 'deviceProfileName']"
python_chirpstack_adapter_1  | ----------------------------------------

I am using JSON marshaler and I have set “json = True” in the scritpt
I think the protocol buffers are not updated to handle new variables (I am not an expert on protocol buffers and just a beginner).

I have mqtt integration and captured the uplink payload which is appearing on topic

application/7c7746bb-bc82-41b7-9a32-211fdcb72651/device/506f98000000ac63/event/up

{
    "deduplicationId": "2272a61d-41b3-404a-a081-49295df6463f",
    "time": "2022-11-17T13:30:56.476834105+00:00",
    "deviceInfo": {
        "tenantId": "52f14cd4-c6f1-4fbd-8f87-4025e1d49242",
        "tenantName": "ChirpStack",
        "applicationId": "7c7746bb-bc82-41b7-9a32-211fdcb72651",
        "applicationName": "locathost",
        "deviceProfileId": "5fd57ed5-cb76-4e00-ace7-3bdcb223946c",
        "deviceProfileName": "device_profile",
        "deviceName": "Node_506f98000000ac63",
        "devEui": "506f98000000ac63"
    },
    "devAddr": "0000ac63",
    "adr": true,
    "fCnt": 4,
    "fPort": 7,
    "confirmed": true,
    "data": "MA==",
    "rxInfo": [
        {
            "gatewayId": "dabbccffff2a79cb",
            "uplinkId": 51976,
            "rssi": -43,
            "snr": 8.0,
            "channel": 3,
            "location": {},
            "context": "qRlwdw==",
            "metadata": {
                "region_name": "in865",
                "region_common_name": "IN865"
            }
        }
    ],
    "txInfo": {
        "frequency": 865402500,
        "modulation": {
            "lora": {
                "bandwidth": 125000,
                "spreadingFactor": 12,
                "codeRate": "CR_4_5"
            }
        }
    }
}

I have setup the Chirpstack version 4 sever using docker today which I think is the latest build.
I am running a simulator for uplink but the data sent is an actual sample of real data transmitted by a device.

I have the following questions:
a. How do I resolve the above error ?
b. Where do I get protocol buffer file so I dont have to use environment variable
“PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION”
c. The python script does not show how to perform a downlink. Can I get an sample code to do this? (with and without MQTT integration )
d. I would like to implement the server in NodeJS instead of Python. Is there any sample code to do this ?

Thanks

Regards,
Harsha

2

Are you mixing ChirpStack v3 with the chirpstack_api v4 package?

3

How do I say ? I have deployed LNS v4 using docker and using a simulator to push some data to the gateway.

4

I think you installed the chirpstack_api package through pip. Could you check the chirpstack_api package version?

5

I am setting up the script to run in a docker container. This is the command I used in Dockerfile

pip install --no-cache-dir -r requirements.txt

…and this is content of the requirements.txt.

chirpstack-api==3.12.4

So I think it is version 3.
Which version should I use in the requirements.txt ?

I got the requirements.txt from this site

6

This is the v3 documentation, for ChirpStack v4, please make sure to use the latest chirpstack-api=4.x.x version :slight_smile: (chirpstack-api · PyPI)

7

Thank you.

Regards,
Harsha