[release] ChirpStack v4 (release candidates)

Announcements
1

ChirpStack v4.0.0-rc.2

After many months of development and testing, we are really excited to share ChirpStack v4.

The aim of ChirpStack v4 is to make it significantly easier to setup and use ChirpStack, compared to the previous version/ One of the major changes that you will notice is that the ChirpStack Network Server and ChirpStack Application Server have been merged into a single component. Over the years we have seen many issues reported on the forum and GitHub, related to setting up and connecting both services.
ChirpStack v4 also provides multi-region support out-of-the-box, including region configuration. No longer it is needed to define your own configuration file or setup multiple ChirpStack Network Server instances to serve multiple regions simultaneously.

A big thank you to the ChirpStack community for supporting and contributing to the ChirpStack project! Please find below a breakdown of all the new features and changes that v4 brings.

Main features and changes

Multi-region support

ChirpStack v4 adds multi-region support, removing the need to setup multiple ChirpStack Network Server instances. Configuration files are included for the common regions (as defined by the LoRa Alliance), which should help getting started with ChirpStack.

Each enabled region has its own gateway backend, making it possible to use one or multiple MQTT brokers for the different gateway pools. In case a single MQTT broker is used, each backend must be configured with its own MQTT topic prefix (e.g. eu868/gateway/...).

ChirpStack v4 also supports multiple configurations of the same region, e.g. to configure a US915 for 8 channels and to configure a US915 band for 16 channels.

TTN device repository support

ChirpStack v4 adds support for importing the TTN LoRaWAN Devices repository as device-profile templates, including codec functions if these are defined in this repository.

Device metrics dashboard

In the device-profile template and / or device-profile, it is possible to define the measurements that are exposed by the device in the decoded payload. Once defined, ChirpStack will automatically aggregate and store this data. These metrics can be viewed in the web-interface on the device dashboard.

Configuration

Directory

Instead of using a single configuration file (e.g. chirpstack-network-server.toml), ChirpStack makes use of a configuration directory such that the configuration can be split in multiple files. By default you will find a single chirpstack.toml configuration file, and many region_...toml configuration files, split by region.

Environment variables

ChirpStack v4 removes the TOML hierarchy to environment variable mapping. Instead it allows you to define the variables like $MY_CONFIGURATION, which will get automatically substituted when an environment variable is found with the name MY_CONFIGURATION.

API

The REST interface that was present in ChirpStack Application Server v3 has been removed, in favor of the gRPC API interface (please see the api/ folder of the repository for the API definitions). However, a gRPC to REST interface bridge component will be provided as a separate service. Please note that
in v3, this bridge component was embedded and REST interface calls were internally translated to gRPC calls. Therefore, gRPC was always recommended interface to use.

The ChirpStack v4 gRPC API does support server reflection, making it possible to use for example gRPC UI or BloomRPC as development interface.

ChirpStack Gateway Bridge v3 compatibility

ChirpStack v4 is fully compatibility with the latest version of ChirpStack Gateway Bridge v3. This should help migrating from v3 to v4. Please note that the ChirpStack Gateway Bridge must be configured with the protobuf marshaler.

UI rewrite

ChirpStack v4 contains a rewrite of the ChirpStack Application Server v3 UI. The new UI aims to be more user-friendly. Under the hood the API interface has been ported to gRPC-web and all code has been ported to Typescript.

Other changes

Passive Roaming improvements

The implementation of Passive Roaming has been improved, adding support for appending /sns and /fns server endpoint suffixes. The usage of this suffix is not specified in the Backend Interfaces specification, but is required by some other network-server implementations.

UUID identifiers

All identifiers that are exposed have been changed to UUID. Previously most identifiers (e.g. users, applications…) were incremental integers. In case ChirpStack is setup as multi-tenant instance, this could expose some information about the number of clients on the network. The migration script (see below)
will migrate these integers by converting these as strings, prefixed with zeros in the UUID format. E.g. ID 123 would be converted to the UUID string 00000000-0000-0000-0000-000000000123.

String identifiers in API

All binary identifiers have been changed to string type in the API. While binary fields are more efficient, these were confusing when encoded as JSON as the Protobuf to JSON mapping uses base64 encoding for binary fields. For example, a Gateway ID 0102030405060708 was encoded as AQIDBAUGBwg=
in JSON.

JavaScript codec engine

The JavaScript codec engine is based on QuickJS, which is an embeddable JavaScript engine which supports the ES2020 specification. In the near future support for the NodeJS Buffer class will be added.

API interface changes

While the structure of API messages is roughly the same as the ChirpStack Application Server API interface, some small changes have been made.

Integration events

The integration event messages have been restructured for better consistency. Each event message has a deviceInfo field which holds device-related information (tenant id & name, application id & name, device-profile id & name, device EUI & name and tags).

Development changes

Single repository

ChirpStack v4 will make it a lot easier to make customizations, especially when API changes are involved, as API definitions are no longer separated from the code. In v3 these definitions were moved to an external repository to avoid cross dependencies.

Rust

For ChirpStack v4, it was decided to use Rust rather than Go. This was not an easy choice and the arguments for this decision are debatable. However, as most code was touched during the ChirpStack Application Server and ChirpStack Network Server merge, it was the only moment to re-consider this. The Rust memory management prevents many memory related pitfalls and helps catching bugs at compile time rather than runtime.

Migrating from v3 to v4

The recommended way to migrate from v3 to ChirpStack v4 is to create a new PostgreSQL and Redis database and to use the ChirpStack v3 to v4
migration script. This script will copy all the data from the “old” into the “new” database. While the script does not make any modifications to the old database, it is always recommended to make a backup first.

In a few minutes the www.chirpstack.io documentation will be updated with the ChirpStack v4 documentation (+ link to ChirpStack v3 documentation).

Note: As no stable v4.0.0 release exists at this point, the release-candidates will be pushed to both the stable and testing v4 respository channels (Debian / Ubuntu). After the first stable v4.0.0 release, only the testing channel will be used for test-releases.

12 Likes
3

Fantastic! Very nice work, well done and thanks! now to run the migration!

1 Like
4

I have been looking v4 documentation and at the moment it does not support AQMP/RabbitMQ backend. Will it be available on the final v4 release? We are currently using it with the v3 version for integration with our backend and without this feature the upgrade will be impossible.

1 Like
5

@iames
v4 documentation says

Support for using AMQP, GCP Pub/Sub or Azure IoT Hub as gateway backend is not supported by the initial ChirpStack v4 release, but will be added in a future version.

6

All my decoding scripts are not working. May I know the data type and example of the bytes parameter in decodeUplink function?

7

@Bormy_Chanthong if u’re using js decodec func, you will see these on the input box

// Decode uplink function.
//
// Input is an object with the following fields:
// - bytes = Byte array containing the uplink payload, e.g. [255, 230, 255, 0]
// - fPort = Uplink fPort.
// - variables = Object containing the configured device variables.
//
// Output must be an object with the following fields:
// - data = Object representing the decoded payload.
function decodeUplink(input) {
  return {
    data: {
      temp: 22.5
    }
  };
}

in V4, u can try:

function decodeUplink(input) {
  var bytes = input.bytes;  // an array; the same as bytes in chirpstack V3
  var fPort = input.fPort;
  var variables = input.variables;
  return {
    data: {
      temp: 22.5
    }
  };
}
8

Yeah, I think so; I have just found out the following:

“object”:{“bytes”:null,“port”:{“fPort”:2.0,“recvTime”:{},“variables”:{},“bytes”:[30.0,15.0,67.0,30.0,92.0,236.0,3.0,0.0,0.0,238.0,0.0,2.0,13.0]}}

many thanks

9

ChirpStack v4.0.0-rc.3

Features

  • Add config option to allow unknown gateways.

Improvements

  • Remove generating certificates on .deb install (this is probably more confusing than helpful, as there are still manual steps needed). This has been documented instead.
  • Replace $POSTGRESQL_HOST, $REDIS_HOST and $MQTT_BROKER_HOST env. variables in configuration by localhost for .deb and .rpm packages. These variables were set in the Systemd chirpstack.service configuration, but this can be confusing if executing the chirpstack binary from the CLI (as you must export these env. variables again by hand).

Bugfixes

  • Fixes setting location in rxInfo of uplink events.
  • Include fix for avx2 CPU instruction check. See also: https://github.com/RustCrypto/utils/pull/792.
  • Fix LinkADRReq channel-mask computation regression, which would result in 3 mac-commands while the same could have been done using 3 mac-commands.
4 Likes
10

@orne you are a machine! Tireless… thanks for all your hard work!

2 Likes
11

Very excited with multi-region support.
Thanks a lot for the effort.

1 Like
12

ChirpStack v4.0.0-rc.4

Improvements

  • Also log txack errors for mac-only payloads.
  • Add user_id filter when listing tenants with global API key. (#34)
  • Cleanup DevEUI from DevAddr > DevEUI set, if DS does not exist.

Bugfixes

  • Misc. fixes to passive-roaming implementation to pass inter-op test with Actility.
  • Fix missing kafka key in integration configuration.
  • Fix missing CN470 region + configuration files.
  • Fix filtering on mac-command conflict and error count.
1 Like
13

With the new standard TS013-1.0.0 Payload Codec API launched by Lora Alliance, what will happen to the TTN device repository?

14

I believe it is already compatible with the Payload Coded API :slight_smile:

15

You are correct the Payload Codec is the same. However, the file structure from TTN (Example: Dragino) is different from the standard. Some files remain the same but others change. Thank you :sweat_smile: