Lora App Server new device registration

As i am working on a project where we will have multiple end node so it will not be possible to add device everytime manually , is there any option to get device registered automatically or some sort of solution available , please help me out with this, Thank you in advance !

1 Like

You have to setup each device before it can be activated on the network. You can do this either by using the web-interface or the provided API.

Is it possible to import multiple devices simultaneously (BULK), for example from the .csv file? I have 100 end devices, the seller sent me all the data in csv and it would be easier to import them from the file!

BR,
Mladen

You could use the gRPC / RESTful API for this :slight_smile:

Hi guys, I think I have the same issue as Mladen, I was able to link my end-device to the server and I can see the data that it is sending, but some minutes after that, I lose the link and I need to make it join again, is there a way of setting a stable join between the end-devices and loraserver?
Thanks in advance!

Importing a list of devices using a csv file containing their name, eui and appkey would definitely be very useful. In the case of ABP (which I have many uses for): name, dev-addr, nwkskey, appskey.

Hi,

Apologies for opening up an old thread…

I am trying to use nodered to bulk upload devices to Chirpstack from a csv file.

Here is my nodered flow

[{"id":"63bc6162.04ceb","type":"tab","label":"Create Device API","disabled":false,"info":"Add new device to Chirpstack"},{"id":"67f0bb04.0764f4","type":"http request","z":"63bc6162.04ceb","name":"api/devices","method":"POST","ret":"txt","paytoqs":"query","url":"http://192.168.1.200:8080/api/devices","tls":"","persist":false,"proxy":"","authType":"basic","x":1310,"y":300,"wires":[["e164a406.fd7698"]]},{"id":"e164a406.fd7698","type":"debug","z":"63bc6162.04ceb","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"true","targetType":"full","statusVal":"","statusType":"auto","x":1510,"y":300,"wires":[]},{"id":"b9651280.2937","type":"function","z":"63bc6162.04ceb","name":"createHeaders","func":"msg.headers = {};\nmsg.headers['Content-Type'] = 'application/json';\nmsg.headers['Accept'] = 'application/json';\nmsg.headers['Grpc-Metadata-Authorization'] = 'Bearer R E D A C T E D'\nreturn msg;","outputs":1,"noerr":0,"initialize":"","finalize":"","libs":[],"x":1080,"y":300,"wires":[["67f0bb04.0764f4"]]},{"id":"7408ce58.9e9f4","type":"template","z":"63bc6162.04ceb","name":"","field":"payload","fieldType":"msg","format":"handlebars","syntax":"mustache","template":"{\n \"device\": {\n \"applicationID\": \"16\",\n \"description\": \"{{desc}}\",\n \"devEUI\": \"{{deveui}}\",\n \"deviceProfileID\": \"82b4a8c3-2ba9-4292-a86d-348866784c77\",\n \"name\": \"{{name}}\",\n \"referenceAltitude\": 0,\n \"skipFCntCheck\": true,\n \"tags\": {},\n \"variables\": {}\n }\n}","output":"str","x":860,"y":300,"wires":[["b9651280.2937"]]},{"id":"e8dc5a4b.7ae528","type":"function","z":"63bc6162.04ceb","name":"","func":"msg.payload = \"\"\n//var myDesc = flow.get(\"desc\");\n//var myeui = flow.get(\"eui\");\n//var myname = flow.get(\"name\")\nmsg.deveui = flow.get(\"eui\")\nmsg.name = flow.get(\"name\")\nmsg.desc = flow.get(\"desc\")\nreturn msg;","outputs":1,"noerr":0,"initialize":"","finalize":"","libs":[],"x":410,"y":240,"wires":[["7408ce58.9e9f4"]]},{"id":"18e7b509.4df6db","type":"file in","z":"63bc6162.04ceb","name":"","filename":"/data/LoRaDevices.csv","format":"utf8","chunk":false,"sendError":true,"encoding":"none","x":380,"y":500,"wires":[["d42d3848.5930e8"]]},{"id":"d42d3848.5930e8","type":"csv","z":"63bc6162.04ceb","name":"","sep":",","hdrin":true,"hdrout":"","multi":"one","ret":"\\n","temp":"","skip":"0","strings":true,"include_empty_strings":false,"include_null_values":false,"x":650,"y":500,"wires":[["2dcb7e26.81d7a2"]]},{"id":"eb318850.0b9528","type":"inject","z":"63bc6162.04ceb","name":"","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"","payloadType":"date","x":140,"y":500,"wires":[["18e7b509.4df6db"]]},{"id":"2dcb7e26.81d7a2","type":"function","z":"63bc6162.04ceb","name":"","func":"msg.name=msg.payload.Name\nmsg.deveui=msg.payload.Devid\nmsg.desc=\"Irrigation\"\nreturn msg;","outputs":1,"noerr":0,"initialize":"","finalize":"","libs":[],"x":780,"y":400,"wires":[["7408ce58.9e9f4"]]}]

I am however getting an API error as follows:

_msgid: "7df155b4.af602c"

payload: "{"error":"lorawan: exactly 8 bytes are expected","code":3,"message":"lorawan: exactly 8 bytes are expected","details":[]}"

topic: ""

filename: "/data/LoRaDevices.csv"

columns: "name,deveui,desc"

I could really do with any advice or pointers as where I am going wrong?

My CSV looks like this:

image

Many many thanks for any help that anyone can provide…

i didnt use .csv import.
But i think that devEUI in your .scv is wrong.
devEUI cann’t contain “g” symbol.
Because devEUI its a hex-number, 8 bytes length.

@Supmik Thanks very much. I have tried to convert to hex, but I am still getting the same errors… Do you know of an decoder I can use? I think the one I am using is not converting the deveui correctly.

Many thanks again

If what you’re after is using a spreadsheet to add devices to an existing application, then you can use THIS go script. You need to edit init() and specify details. Running this script on server will skip existing lines and just add any new lines - Note: you need to create the application with # specified in 2nd column before running script.

modify init()

populate XLSX specified above:

run script:

I previously wrote that I installed Docker & Docker-container, but those were all lies (sorry). Instead, I installed go-lang as follows:

  1. I saved brocaar’s script to “official_go_package.go”
  2. download latest “chirpstack-api-longHEXid.zip”, rename to shorter chirpstack-api & unzip it.
  3. install go-lang using “sudo snap install go --classic”
  4. when I tried running brocaar’s script with “go run official_go_package.go” it complained about missing go.mod so I fixed that by moving brocaar’s script to “go” subfolder under api where go.mod is located.
  5. created devices_spreadsheet XLSX as shown above.
    a. You may need to “sudo snap install libreoffice” first, like I did :slight_smile:
    b. WARNING: be sure to change formatting of columns with device IDs to “text” or Libreoffice will do funky number on any IDs beginning with a number.
  6. edited official_go_package.go and changed init() as shown above.
  7. installed missing packages:
    a. go get github.com/brocaar/lora-app-server/api
    b. go get github.com/pkg/errors
    c. go get github.com/tealeg/xlsx
  8. ran script with “go run official_go_package.go”

The only thing brocaar’s existing script doesn’t support is setting additional parameters for each device (which I have to do manually) such as “Disable frame-counter validation”, etc.

1 Like

@fmgst you legend. Thank you very much for this. VERY much appreciated…

So, I crudely hacked in the column which original script was missing (i.e., SkipFCntValidation), because I’m too lazy to manually update most of my devices (all my RadioBridge sensors need this, but Laird sensors don’t).

// Author: Orne Brocaar. Author of LoRa Server https://www.chirpstack.io
// Script taken from https://forum.chirpstack.io/t/creating-a-terraform-provider-for-loraserver/4081/2
//
// ignore WARNING: proto: file “common.proto” is already registered
//		A future release will panic on registration conflicts. See:
//		https://developers.google.com/protocol-buffers/docs/reference/go/faq#namespace-conflict

// fmg: added column "SkipFCntValidation"; now requires XLSX to have 8 columns; see DeviceImportRecord for type
// col0   col1          col2            col3 col4        col5       col6           col7 
// devEUI applicationID deviceProfileID name description networkKey applicationKey skipFCntValidation

package main

import (
	"context"
	"crypto/tls"
	"flag"
	"log"

	"github.com/pkg/errors"
	"github.com/tealeg/xlsx"
	"google.golang.org/grpc"
	"google.golang.org/grpc/codes"
	"google.golang.org/grpc/credentials"

	"github.com/brocaar/lora-app-server/api"
)

// JWTCredentials provides JWT credentials for gRPC
type JWTCredentials struct {
	token string
}

// GetRequestMetadata returns the meta-data for a request.
func (j *JWTCredentials) GetRequestMetadata(ctx context.Context, url ...string) (map[string]string, error) {
	return map[string]string{
		"authorization": j.token,
	}, nil
}

// RequireTransportSecurity ...
func (j *JWTCredentials) RequireTransportSecurity() bool {
	return false
}

// SetToken sets the JWT token.
func (j *JWTCredentials) SetToken(token string) {
	j.token = token
}

// DeviceImportRecord defines a record for a device to import.
type DeviceImportRecord struct {
	DevEUI          string
	ApplicationID   int64
	DeviceProfileID string
	Name            string
	Description     string
	NetworkKey      string
	ApplicationKey  string
	SkipFCntValidation	bool
}

var (
	username       string
	password       string
	file           string
	apiHost        string
	apiInsecure    bool
	jwtCredentials *JWTCredentials
)

func init() {
	jwtCredentials = &JWTCredentials{}

	//fmg: flag.StringVar(&username, "username", "admin", "LoRa App Server username")
	flag.StringVar(&username, "username", "admin", "LoRa App Server username")
	//fmg: flag.StringVar(&password, "password", "admin", "LoRa App Server password")
	flag.StringVar(&password, "password", "admin", "LoRa App Server password")
	//fmg: flag.StringVar(&file, "file", "", "Path to Excel file")
	flag.StringVar(&file, "file", "/home/ds-admin/Documents/2fmg_new_devices.xlsx", "Path to Excel file")
	//fmg: flag.StringVar(&apiHost, "api", "localhost:8080", "hostname:port to LoRa App Server API")
	flag.StringVar(&apiHost, "api", "10.1.6.90:8080", "hostname:port to LoRa App Server API")
	//fmg: flag.BoolVar(&apiInsecure, "api-insecure", false, "LoRa App Server API does not use TLS")
	flag.BoolVar(&apiInsecure, "api-insecure", true, "LoRa App Server API does not use TLS")
	flag.Parse()
}

func getGRPCConn() (*grpc.ClientConn, error) {
	dialOpts := []grpc.DialOption{
		grpc.WithBlock(),
		grpc.WithPerRPCCredentials(jwtCredentials),
	}

	if apiInsecure {
		log.Println("using insecure api")
		dialOpts = append(dialOpts, grpc.WithInsecure())
	} else {
		dialOpts = append(dialOpts, grpc.WithTransportCredentials(credentials.NewTLS(&tls.Config{
			InsecureSkipVerify: true,
		})))
	}

	conn, err := grpc.Dial(apiHost, dialOpts...)
	if err != nil {
		return nil, errors.Wrap(err, "grpc dial error")
	}

	return conn, nil
}

func login(conn *grpc.ClientConn) error {
	internalClient := api.NewInternalServiceClient(conn)

	resp, err := internalClient.Login(context.Background(), &api.LoginRequest{
		Username: username,
		Password: password,
	})
	if err != nil {
		return errors.Wrap(err, "login error")
	}

	jwtCredentials.SetToken(resp.Jwt)

	return nil
}

func getDeviceImportList() ([]DeviceImportRecord, error) {
	xlFile, err := xlsx.OpenFile(file)
	if err != nil {
		return nil, errors.Wrap(err, "open excel file error")
	}

	var out []DeviceImportRecord

	for _, sheet := range xlFile.Sheets {
		for i, row := range sheet.Rows {
			if i == 0 {
				continue
			}

			if len(row.Cells) != 8 {
				log.Fatalf("expected exactly 8 columns (row %d) but found %d", i+1, len(row.Cells))
			}

			devEUI := row.Cells[0].String()
			applicationID, err := row.Cells[1].Int64()
			if err != nil {
				log.Fatalf("application id parse error (row %d): %s", i+1, err)
			}
			deviceProfileID := row.Cells[2].String()
			name := row.Cells[3].String()
			description := row.Cells[4].String()
			networkKey := row.Cells[5].String()
			applicationKey := row.Cells[6].String()
			skipFCntValidation := row.Cells[7].Bool()
			if err != nil {
				log.Fatalf("SkipFCntValidation parse error (row %d): %s", i+1, err)
			}


			out = append(out, DeviceImportRecord{
				DevEUI:          devEUI,
				ApplicationID:   applicationID,
				DeviceProfileID: deviceProfileID,
				Name:            name,
				Description:     description,
				NetworkKey:      networkKey,
				ApplicationKey:  applicationKey,
				SkipFCntValidation: skipFCntValidation,
			})
		}
	}

	return out, nil
}

func importDevices(conn *grpc.ClientConn, devices []DeviceImportRecord) error {
	deviceClient := api.NewDeviceServiceClient(conn)

	for i, dev := range devices {
		d := api.Device{
			DevEui:          dev.DevEUI,
			Name:            dev.Name,
			ApplicationId:   dev.ApplicationID,
			Description:     dev.Description,
			DeviceProfileId: dev.DeviceProfileID,
//In http://10.1.6.90:8080/static/js/main.ee69ccb2.chunk.js found that variable for "Disable frame-counter validation" is "skipFCntCheck"
// {label:"Disable frame-counter validation",control:i.a.createElement(Ht.a,{id:"skipFCntCheck",checked:
// !!this.state.object.skipFCntCheck,onChange:this.onChange,color:"primary"})})),
// i.a.createElement(Kt.a,null,"Note that disabling the frame-counter validation will compromise security as it enables people to perform replay-attacks.")),
			SkipFCntCheck: dev.SkipFCntValidation,
//Using search term "+site:github.com/brocaar skipFCntCheck", found skipFCntCheck being handled in:
//https://github.com/brocaar/chirpstack-network-server/blob/master/internal/storage/device_session.go
//Used in func deviceSessionToPB() as SkipFCntCheck: d.SkipFCntValidation,
//Used in func deviceSessionFromPB() as SkipFCntValidation: d.SkipFCntCheck,
			//SkipFCntValidation: dev.SkipFCntValidation, //didn't work
			//skip_f_cnt_check: dev.SkipFCntValidation, //didn't work
		}

		dk := api.DeviceKeys{
			DevEui: dev.DevEUI,
			NwkKey: dev.NetworkKey,
			AppKey: dev.ApplicationKey,
		}

		_, err := deviceClient.Create(context.Background(), &api.CreateDeviceRequest{
			Device: &d,
		})
		if err != nil {
			if grpc.Code(err) == codes.AlreadyExists {
				log.Printf("device %s already exists (row %d)", d.DevEui, i+2)
				continue
			}
			log.Fatalf("import error (device %s row %d): %s", d.DevEui, i+2, err)
		}

		_, err = deviceClient.CreateKeys(context.Background(), &api.CreateDeviceKeysRequest{
			DeviceKeys: &dk,
		})
		if err != nil {
			if grpc.Code(err) == codes.AlreadyExists {
				log.Printf("device-keys for device %s already exists (row %d)", d.DevEui, i+2)
				continue
			}
			log.Fatalf("import error (device %s) (row %d): %s", d.DevEui, i+2, err)
		}
	}

	return nil
}

func main() {
	conn, err := getGRPCConn()
	if err != nil {
		log.Fatal("error connecting to api", err)
	}

	if err := login(conn); err != nil {
		log.Fatal("login error", err)
	}

	rows, err := getDeviceImportList()
	if err != nil {
		log.Fatal("get device import records error", err)
	}

	if err := importDevices(conn, rows); err != nil {
		log.Fatal("import error", err)
	}
}

XLSX file needs 8th column:

1 Like

Just a little something, probably irrelevant in the context of the amount of work you have put into this, but in the code you have skipFCntValidation however in the excel you have SkipFCntValidation. I assume the latter is correct?

It doesn’t seem to matter - I think it’s entirely ignoring column headings/first row. Script & screencap are actual that worked.

Can someone help with a subdomain configuration for hostname and port?

flag.StringVar(&apiHost, "api", "canthisbe.subdomain.com", "hostname:port to LoRa App Server API")

I have tried a number of permutations and combinations, but can only get this working with an IP address and port number.

Many thanks

Hey,
Thank you so much for those informations, please i wanna ask you about “chirpstack-api-longHEXid.zip”, i can’t see from where i can download it ?
Best Regards,

I can’t seem to edit old entry (from August 2021) so these are errata/changes to that entry:

For #2, visit https://github.com/brocaar/chirpstack-api and download code as a ZIP file:


That’s all I was trying to say. When I did it under Ubuntu, resulting filename had a long hex number in it, but that doesn’t happen in windows.

For #5, “do funky number on any IDs beginning with a number” just means that if your device information is a huge number (say 80000363788217) then Libreoffice will convert it to a number by rounding which is NOT what you want. You need all those digits exactly so make that column “text” first.

Hope that clarifies things.

Thank you so much for your response.
I did exactly what you suggest, but i have this error message.


Any idea about the reason.

thank you very much once again :wink:

It’s safe to ignore the warning (“a future release will die” :-). I got it too and ignored it.

Your error, and why it exits, is that your excel file does not have exactly 7 columns. Compare your file to the format shown in image just below text “populate XLSX specified above:” to see which one you’re missing; most likely ApplicationID, which is subject of Note: :slight_smile:

Thank you so much, all works perfectly :slight_smile: