[release] ChirpStack Application Server v3.11

Announcements
#1

ChirpStack Application Server v3.11.0

Features

OpenID Connect authentication

This feature makes it possible to use an OpenID Connect authentication backend together with ChirpStack Application Server, for example Auth0.com. Users are automatically matched based on email address on the first login. If there is a successful match, an external OpenID Connect user identifier is associated with the user, so that even in case of an email address update, it can still be matched (and updated in the ChirpStack Application Server database).

The implementation of this feature required a couple of changes:

Usernames

Usernames no longer exist in ChirpStack Application Server and have been renamed to email. Existing users can still login with their username, but on user update, the username must be set to an email address.

Previously there was an unused email field. As there was no guarantee that this field was unique this field has been deprecated. It is still present in the database as email_old so that the new email field (containing the old username as value) can be updated by hand if needed.

Create users

Creating new users as an organization admin has been removed. However, the OpenID Connect integration provides a registration_enabled option, which automatically creates new users when it does not exist.

Together with the registration_callback_url, it can be used to automatically onboard new users. When this option is set, ChirpStack Application Server will make a HTTP request to configured URL when a new user is created. This makes it possible to implement an onboarding service which (using the API) could:

  • Create an organization for the given user ID
  • Setup one or multiple service-profiles
  • …
Assigning users to organization

Previously there was an auto-complete field to assign an user to an organization. As the username has been replaced by email and to not leak email addresses, the autocomplete function has been removed and the exact email must be given when associating a given user with an organization.

LoRa Cloud integration

Geolocation

This migrates the geolocation integration from ChirpStack Network Server to ChirpStack Application Server and allows a per-application integration configuration. The LoRa Cloud Geolocation integration provides support for TDOA, RSSI, Wifi and LR1110 based GNSS geolocation.

DAS

This implements the integration with the LoRa Cloud Device & Application Services.

Disable device

This feature makes it possible to (temporarily) disable a device.

Kafka integration

This (global) integration makes it possible to forward events to a Kafka broker.

Gateway client-certificates

When also configured in ChirpStack Network Server this makes it possible to generate per-gateway client-certificates which can be used to implement gateway authentication and authorization. For example a MQTT broker can be configured to validate the client-certificate against a pre-configured CA certificate and if valid it can use the CommonName of the certificate (which contains the gateway ID) to authorize publish / subscribe to certain topics.

Support max gateway / device option

This option makes it possible to configure the maximum gateways and devices that can be created per organization.

Integrations

Cloud integrations

It is now possible to configuring per-application integrations for AWS SNS, Azure Service-Bus and GCP Pub/Sub

Web-interface

The interface to configure per-application configurations has been re-implemented.

Single HTTP integration endpoint

This updates the HTTP integration with a single endpoint configuration. Instead of configuring a per-event endpoint, this makes it possible to have a single endpoint to which all events are posted, with the event as query parameter.

This update makes it also possible to configure the marshaler (JSON, Protocol Buffers and legacy JSON v3) per HTTP integration.

Already configured HTTP integrations are not affected by this update but it is recommended to update your configuration so that you will automatically receive new event-types when they are introduced.

MQTT topic configuration

The MQTT integration has been updated with new MQTT topic configuration. Instead of configuring each topic separately, an event_topic_template template can be configured, which is used for all events. The command topic(s) can now be configured through the command_topic_template variable. In case the old configuration is still present, then the old topic configuration will remain active for backwards compatibility.

Important:

For consistency with the other components, the new configuration does introduce a change in MQTT topics (default configuration):

  • application/[ApplicationID]/device/[DevEUI]/rx -> application/[ApplicationID]/device/[DevEUI]/event/up
  • application/[ApplicationID]/device/[DevEUI]/join -> application/[ApplicationID]/device/[DevEUI]/event/join
  • application/[ApplicationID]/device/[DevEUI]/ack -> application/[ApplicationID]/device/[DevEUI]/event/ack
  • application/[ApplicationID]/device/[DevEUI]/error -> application/[ApplicationID]/device/[DevEUI]/event/error
  • application/[ApplicationID]/device/[DevEUI]/status -> application/[ApplicationID]/device/[DevEUI]/event/status
  • application/[ApplicationID]/device/[DevEUI]/txack -> application/[ApplicationID]/device/[DevEUI]/event/txack
  • application/[ApplicationID]/device/[DevEUI]/location -> application/[ApplicationID]/device/[DevEUI]/event/location
  • application/[ApplicationID]/device/[DevEUI]/tx -> application/[ApplicationID]/device/[DevEUI]/command/down

API keys

It is now possible to also generate per-organization API keys in the web-interface for easy integration with the gRPC and REST API.

Improvements

  • Make it possible to use activate API endpoint for OTAA devices (to import an existing activation).
  • Gateway Profile / channel-plan re-configuration documentation in web-interface.
  • Internal cleanup auto-complete select in web-interface.
  • Remove links in web-interface which could cause a logout due to user-permissions.
  • Redirect to device overview after saving root-keys.
  • Add network-server name in gateway list.
  • Expose (un)confirmed + DevAddr in integration payloads. (#453)
  • Highlight rlow when hovering over table rows in web-interface. (#474)
  • Include Network Server name column in Device- and Service Profile list. (#475)
  • Include RSSI and SNR in ThingsBoard integration. (#478)
  • Align Gateway detail overview styling. (#480)
  • Makefile changes to speed up tests. (#493)
  • Support multiple Redis addresses for clustering. (#491)

Bugfixes

  • Fix delete icon in gateway-profile UI (was showing + icon instead of delete).
  • Fix updating organization with global API key. (#487)
  • Add missing error check in AWS SNS integration. (#483)
4 Likes
Documentation restyle and re-organization
#2

been having trouble since the update. Been trying to login, but can’t.

I was able to log onto postgres and this is the info from the user db. When I enter, admin for user and password I get a invalid username or password (code: 16)

Did tried a couple of other hashes suggested in the forum, but none worked

 id |          created_at           |          updated_at           | email |                                                             password_hash                                                              | session_ttl | is_active | is_admin | email_old | note | external_id | email_verified 
----+-------------------------------+-------------------------------+-------+----------------------------------------------------------------------------------------------------------------------------------------+-------------+-----------+----------+-----------+------+-------------+----------------
  1 | 2020-03-11 21:51:17.462185-06 | 2020-07-21 15:23:41.025606-05 | admin | PBKDF2$sha512$100000$4u3hL8krvlMIS0KnCYXeMw==$G7c7tuUYq2zSJaUeruvNL/KF30d3TVDORVD56wzvJYmc3muWjoaozH8bHJ7r8zY8dW6Pts2bWyhFfkb/ubQZsA== |           0 | t         | t        |           |      |             | f
(1 row)
#3

check the dsn credentials in .toml file.

#4

Eugenev

Thanks. You mean for postgres? don’t think that is the problem, since chirp talks to the db (see below). I have the following line for this

dsn=“postgres://chirpstack_as:dbpassword@localhost/chirpstack_as?sslmode=disable”

Did noticed that there are some changes in the toml file (toml.dpkg-dist), so I applied these, but still not working

"starting ChirpStack Application Server" docs="https://www.chirpstack.io/" version=3.11.0
"storage: setting up storage package"
"storage: setup metrics"
"storage: setting up Redis client"
"storage: connecting to PostgreSQL database"
"storage: applying PostgreSQL data migrations"
"storage: PostgreSQL data migrations applied" count=0
"integration: configuring global integrations"
"integration/mqtt: TLS config is empty"
"integration/mqtt: connecting to mqtt broker" server="tcp://localhost:1883"
"integration/mqtt: connected to mqtt broker"
"integration/mqtt: subscribing to tx topic" qos=0 topic=application/+/device/+/command/down
"api/as: starting application-server api" bind="0.0.0.0:8001" ca_cert= tls_cert= tls_key=
"api/external: starting api server" bind="0.0.0.0:8080" tls-cert= tls-key=
"api/external: registering rest api handler and documentation endpoint" path=/api
"api/js: starting join-server api" bind="0.0.0.0:8003" ca_cert= tls_cert= tls_key=
"finished unary call with code Unauthenticated" ctx_id=1d1a6074-7d5a-4729-8bd3-a8c5aedf52f2 error="rpc error: code = Unauthenticated desc = authentication failed: get token from context error: no authorization-data in metadata" grpc.code=Unauthenticated grpc.method=List grpc.service=api.OrganizationService grpc.start_time="2020-07-22T…" grpc.time_ms=0.435 peer.address="127.0.0.1:44136" span.kind=server system=grpc
"finished unary call with code Unauthenticated" ctx_id=87a62add-14dc-4817-b386-b02dde3b1d33 error="rpc error: code = Unauthenticated desc = invalid username or password" grpc.code=Unauthenticated grpc.method=Login grpc.service=api.InternalService grpc.start_time="2020-07-22T…" grpc.time_ms=3.993 peer.address="127.0.0.1:44132" span.kind=server system=grpc
#5

Please try with a clean browser cache. As the username has been renamed to email, it could be that you have the old version of the web-interface cached and the email field is missing in your login request.

1 Like
Login error to app server
#6

brocaar

Thanks that did it. I had tried several browsers, which I use normally, so I didn’t thought of that. Was thinking it was a problem with the data, since there was no username in the Users / admin window. It coincided with my wanting to change the user name from admin.

If anyone has the same issue, the first field should read Username / email . Any chance of putting an alert when something like this happens. You do a great job with chirpstack, but don’t keep up with the change logs :blush:

Thanks so much for your all your effort and great work

1 Like
#7

So to provide backward compatibility, we need to track the chirpstack application server version in order to use /api/internal/login with “username” & “password” fields or “email” & “password” fields. This is not a good practice I think.

#8

This is not a backwards compatibility issue, this is a browser cache issue. The above issue was due to the web-interface cached at the client side, while the fields expected by the backend changed. You can control the cache behavior if you are using a proxy (e.g. NGINX).

The use of the /api/internal/login endpoint was never encouraged because it is for internal usage and the API might change over time. I recommend to use the API tokens which you now can generate in the web-interface :slight_smile:

1 Like
#9

Thank you for your feedback @brocaar . Your time and effort are greatly appreciated!

#10

@brocaar just to mention, on a fresh setup there is an issue, login admin/admin, then unable to change password due to "invalid email’
So we can’t use anymore admin user (because we can’t change the password and it’s not secure)

So now first thing to do, rename admin user, then change password :slight_smile:

#11

The username admin is there for backwards compatibility but indeed, the username field has changed to email and is validated as an email address (so that you can integrate with an OpenID Connect backend).

#12

@brocaar

I am using the latest AS and NS and trying to configure the Gateway Configuration.
On Certificate Tab I have seen the below note.

This certificate must be configured on the gateway

Do we need to install chirpstack-gateway-bridge at the Gateway?

where can I configure gateway certificates in the gateway?

image
image1903×489 28.4 KB

Thanks

#14

If you want to use this feature, then yes as the generated TLS certificate is per gateway id. Please note that you must also configure tls authentication in your mqtt broker (e.g. mosquitto).

#15

ChirpStack Application Server v3.11.1

Improvements

  • Update Kafka configuration parameters so that it is possible to integrate with Azure Event Hub.

Bugfixes

  • Fix setting LoRa Cloud DAS Modem port default value.
1 Like
#16

That helped me. :smiley: Thank you! Hammering CTRL+F5 was enough to fix it.

#17

Hii @brocaar

Gateway Live Frames and Device Data and Live Frame is not showing properly in this version, Some frames are missing due to closing the WebSocket connection issue.

We have tested with the previous version of Chirpstack Application Server and it’s working fine

image
image1920×793 65.9 KB

Chirpstack Application Server Logs

time="2020-08-24T12:00:17Z" level=warning msg="[write] error writing websocket message: write tcp 172.23.0.9:8080->192.168.0.3:52059: i/o timeout"
time="2020-08-24T12:00:17Z" level=info msg="finished streaming call with code Canceled" error="rpc error: code = Canceled desc = context canceled" grpc.code=Canceled grpc.method=StreamFrameLogs grpc.service=api.GatewayService grpc.start_time="2020-08-24T11:59:41Z" grpc.time_ms=36443.703 peer.address="127.0.0.1:43288" span.kind=server system=grpc
2 Likes
#18

@brocaar

Any update on above comment

#19

Same problem Here :slight_smile:

#20

3.12.0 has a fix for the websocket timeout:

1 Like
#21

Hello, i’m sorry but i’m in 3.12.0 and still have the problemCapture|690x59

1 Like