[release] ChirpStack Application Server v3.6

ChirpStack Application Server v3.6.0

Features

Protobuf based integrations

This feature adds a marshaler configuration option to the integration configuration. The default is backwards compatible with the current JSON format. New options are protobuf (Protobuf binary) and json (Protobuf based JSON).

RPM packaging

This is the first release providing .rpm packages for CentOS and RedHat. (#383

Improvements

gRPC / Protobuf cleanup

All definitions are now imported from github.com/brocaar/chirpstack-api/go. When using the gRPC API, you must update your imports.

Azure Service-Bus integration

Implement re-connect in case of an Azure Service-Bus error. This to work partly around azure-service-bus-go/issues/149.

Support multiple HTTP endpoints

Implement support for configuring multiple endpoints in the HTTP integration.

Internal improvements

Handlink of the received uplinks has been improved to reduce the number of required database transactions.

Bugfixes

  • Payload codec is hidden on application create (it was already on edit).
1 Like

Hello, Orne.
After upgrade to v3.6 ā€œtimeā€ field is gone from rxInfo. on 3.5.1 it works fine. Tell me please is it a bug or I need to make some changes in app server configuration ?

I see no changes to time field here. Are you saying that if you downgrade to 3.5.1 the time field returns?

Do the gateways that are forwarding the traffic using GPS time or fake time?

Gateways using internal RTC clock time, which synced via ntp and using polypacket forwarder.
And I suggest, this is not a gateway side problem. We are using many kinds of gateways i.e Lorix, Tektelic with the same issue after upgrade. For today I can’t rollback the app server.

Let me check this for you.

1 Like

Sorry about that, I made a typo when refactoring the migrations to Protobuf structs. In the conversion to the ā€œoldā€ JSON format (which is currently the default), I was setting the timestamp on err != nil instead of err == nil :man_facepalming: This fixes the issue (including the test):

1 Like

Thanks a lot, Orne. Will be debian/ubuntu package update, or I need to build app-server from source for today?

This will be included in the next bugfix release which I have not yet created, so until then you need to build from source. Alternatively, you could change the marshaler to json (the default is json_v3). This gives you a slightly different structure, but has the advantage that you get the rxInfo and txInfo information as-is as forwarded by the ChirpStack Gateway Bridge.

You can use this Protobuf definition for the Protobuf based json or (binary) protobuf marshalers:

(Internally everything is now implemented using the Protobuf structs, the bug you found was in the Protobuf to ā€œoldā€ / ā€œjson_v3ā€ conversion where I made a typo).

I always use marshaler=ā€œjsonā€ in gateway-bridge. but this typo ā€œbugā€ appears with it too.

This is a new feature of the ChirpStack Application Server, see the changelog:

This feature adds a marshaler configuration option to the integration configuration. The default is backwards compatible with the current JSON format. New options are protobuf (Protobuf binary) and json (Protobuf based JSON).

1 Like

ChirpStack Application Server v3.6.1

Bugfixes

  • Fix setting time in rxInfo for json_v3 (default) integration marshaler.
  • Fix integrations break when codec function returns a NaN value. (#390)
2 Likes

Hi , @brocaar , I cloned the chirpstack-api and made changes in the .proto files there and after compilation the changes reflected in the respective .pb.go file also. But how do I import those changes in chirpstack-application-server's respective .go file? I keep trying to use make api command but it says :

 can't load package: package api/api.go: malformed module path "api/api.go": missing dot in first path element
>         Makefile:46: recipe for target 'api' failed
>         make: *** [api] Error 1

How do I integrate my additional rpc methods, defined in the .proto file, in my chirpstack-application-server’s .go file now?

You need to replace the github.com/brocaar/chirpstack-api/go Go module in the go.mod (chirpstack-application-server) with the path of your modified chirpstack-api/go. You can use the replace directive for this. For documentation see: https://github.com/golang/go/wiki/Modules#when-should-i-use-the-replace-directive.

2 Likes

Hi, @brocaar, I did as you told me, here is my go.mod (chirpstack-application-server):

module github.com/brocaar/chirpstack-application-server

go 1.13

require (
cloud.google.com/go v0.44.3
github.com/Azure/azure-service-bus-go v0.9.1
github.com/NickBall/go-aes-key-wrap v0.0.0-20170929221519-1c3aa3e4dfc5
github.com/aws/aws-sdk-go v1.19.19
github.com/brocaar/chirpstack-api/go v0.0.0-20191119153543-35c59165bf94
github.com/brocaar/lorawan v0.0.0-20191115102621-6095d473cf60
github.com/dgrijalva/jwt-go v3.2.0+incompatible
github.com/eclipse/paho.mqtt.golang v1.2.0
github.com/elazarl/go-bindata-assetfs v1.0.0
github.com/gobuffalo/packr v1.22.0 // indirect
github.com/gofrs/uuid v3.2.0+incompatible
github.com/gogo/protobuf v1.2.1
github.com/golang/protobuf v1.3.2
github.com/gomodule/redigo v2.0.0+incompatible
github.com/gopherjs/gopherjs v0.0.0-20190430165422-3e4dfb77656c // indirect
github.com/goreleaser/goreleaser v0.106.0
github.com/goreleaser/nfpm v0.11.0
github.com/gorilla/mux v1.7.1
github.com/grpc-ecosystem/go-grpc-middleware v1.0.0
github.com/grpc-ecosystem/go-grpc-prometheus v1.2.0
github.com/grpc-ecosystem/grpc-gateway v1.11.3
github.com/jacobsa/crypto v0.0.0-20190317225127-9f44e2d11115 // indirect
github.com/jmoiron/sqlx v1.2.0
github.com/jteeuwen/go-bindata v3.0.8-0.20180305030458-6025e8de665b+incompatible
github.com/konsorten/go-windows-terminal-sequences v1.0.2 // indirect
github.com/lib/pq v1.2.0
github.com/mmcloughlin/geohash v0.9.0
github.com/pkg/errors v0.8.1
github.com/golang/protobuf v1.3.2
github.com/gomodule/redigo v2.0.0+incompatible
github.com/gopherjs/gopherjs v0.0.0-20190430165422-3e4dfb77656c // indirect
github.com/goreleaser/goreleaser v0.106.0
github.com/goreleaser/nfpm v0.11.0
github.com/gorilla/mux v1.7.1
github.com/grpc-ecosystem/go-grpc-middleware v1.0.0
github.com/grpc-ecosystem/go-grpc-prometheus v1.2.0
github.com/grpc-ecosystem/grpc-gateway v1.11.3
github.com/jacobsa/crypto v0.0.0-20190317225127-9f44e2d11115 // indirect
github.com/jmoiron/sqlx v1.2.0
github.com/jteeuwen/go-bindata v3.0.8-0.20180305030458-6025e8de665b+incompatible
github.com/konsorten/go-windows-terminal-sequences v1.0.2 // indirect
github.com/lib/pq v1.2.0
github.com/mmcloughlin/geohash v0.9.0
github.com/pkg/errors v0.8.1
github.com/prometheus/client_golang v1.1.0
github.com/robertkrimen/otto v0.0.0-20180617131154-15f95af6e78d
github.com/rubenv/sql-migrate v0.0.0-20181213081019-5a8808c14925
github.com/sirupsen/logrus v1.4.2
github.com/smartystreets/assertions v1.0.0 // indirect
github.com/smartystreets/goconvey v0.0.0-20190330032615-68dc04aab96a
github.com/spf13/cobra v0.0.5
github.com/spf13/viper v1.4.0
github.com/stretchr/testify v1.4.0
github.com/tmc/grpc-websocket-proxy v0.0.0-20190109142713-0ad062ec5ee5
github.com/ziutek/mymysql v1.5.4 // indirect
golang.org/x/crypto v0.0.0-20190605123033-f99c8df09eb5
golang.org/x/lint v0.0.0-20190409202823-959b441ac422
golang.org/x/net v0.0.0-20190628185345-da137c7871d7
golang.org/x/tools v0.0.0-20190708203411-c8855242db9c
google.golang.org/api v0.9.0
google.golang.org/grpc v1.24.0
gopkg.in/gorp.v1 v1.7.2 // indirect
gopkg.in/sourcemap.v1 v1.0.5 // indirect
pack.ag/amqp v0.12.1 // indirect
)

replace github.com/brocaar/chirpstack-api/go => /home/network-server/src/github.com/brocaar/chirpstack-api/go

After setting this up, I again implemented make api , and got the same result as follows:

root@localsvr:/home/network-server/src/github.com/brocaar/chirpstack-application-server# make api
Generating API code from .proto files
can’t load package: package api/api.go: malformed module path ā€œapi/api.goā€: missing dot in first path element
Makefile:46: recipe for target ā€˜api’ failed
make: *** [api] Error 1

I am guessing, this error is returned because there is no api/ directory which contains any api.go file; unlike previously. As this folder has been moved from the source. So now, how do I generate my additional API endpoint?

You should generate the api definitions in the chirpstack-api repository, not chirpstack-application-server. Anyway, lets not mix the release announcement topic with support on using Go modules. If needed, please create a separate topic :slight_smile:

1 Like