Hi, all, this will probably be a feature request, but I do not have the latest version of the software… Can anyone check if this problem still exists on the later versions of GatewayOS? ( my build is sort of special from december) Does the ABP keys get expired when you have no internet? or set the clock wrong on the raspberry?
— long chatty version —
Server run networks - We had some ABP configured nodes in storage for a month and then I discovered that they did not work and I had to re-configure the keys on our chirpstack server ( a very old version, running on real server hardware ). Looking into this I found that chirpstack sets a TTL value on both OTAA and ABP devices to 1 month on the Redis database entires. Thus any ABP devices that does not transmit data o the server for a month gets it’s keys expired, even though it still exists as a device.
There are a lot of issues regarding this Time To Live/Expire thing over the years, and it seems the generic advice by Broocar is to set it to such a high value it will not expire.
Example of bug-report/issue - https://github.com/brocaar/chirpstack-network-server/issues/300
Embedded networks - We use the GatewayOS on raspberries that are powered on and part of a control-system for a few months a year, with hardcoded ABP temperature devices ( feeding data into a fan-controller ). Thus with only power a few months a year, we must be able to change this settng.
I can’t find out how to set the TimeToLive value on the GatewayOS at build-time? How can this be done?
Clock. Redis requires a stable clock that does work.
“Even running instances will always check the computer clock, so for instance if you set a key with a time to live of 1000 seconds, and then set your computer time 2000 seconds in the future, the key will be expired immediately, instead of lasting for 1000 seconds.” - https://redis.io/commands/expire#expires-and-persistence
The raspberry does not have a real-time clock that is battery backup-ed. Thus it can basicly be at any kind of time when a device is configured, and if it does find a time-server or sync it’s clocks it could really expire all encryption keys. I really wonder what happens if it has the real time, and then get a clock that is from 1970… ? (I’m not sure about the time set at startup, but I assume if a sensors sends in a value with a clock that is from 1970 and then after a while it gets a message to decode from 2020 it will just expire the keys)
If the raspberry shutsdown with “sudo shutdown -h now” ( i.e a nice friendly shutdown ). The date is saved in the file-system, and upon boot, this date is loaded from the file-system and used as the current time.
If the raspberry looses power, the date is not written to the file-system. Thus upon boot the time is loaded from the file-system, with the date of the last time the “sudo shudown -h now” was issued.
If the internet cable is plugged into the Raspberry, the date will be set correctly after about half a minute. I assume this is done through synchronizing with a time server on the internet. This correct time is not written to the file-system. So upon a power cycling, the date of the last time “sudo shutdown -h now” is issued, will be reloaded again. Until the next synchronisation takes place.
Some further experiments on the raspberry and GatewayOS shows that a historical date on the raspberry will expire the keys once the correct date is set.
( gateway os version from november 2019 when it was still loraserver.io, but this is most likely valid on the latest REDIS version as I read the source there, the mstime() function is used to get the time in REDIS ).
Run raspberry, let it sync up the clock. Enter the ABP decryption keys. See that it receives data from the unit.
Reboot raspberry and set the date to 1999 with “sudo date 0000000000”. Shut it downwith “sudo shutdown -h now”
Remove all networks from the Pi, ethernet cables, wifi and USB dongles. This is to prevent it from synching the date with internet.
Start the raspberry and let it run so you are sure it has received data from the sensors you are testing with. ( in my test I run it for two hours )
Connect the internet again ( plug in the ethernetcable ) and see that it gets an IP ( “ifconfig” ). Wait for it to synch the time. “date” ( enter + enter ). Wait for it to be sure it receives data. ( it might be enough to access the decryption keys )
Login to the raspberry and check the device. The ABP keys are now lost, and it does not receive messages from the device.
This means that the current version of GatewayOS is dangerous to use off the shelves for autonomous systems that just work part of the year ( we do agriculture automation that just runs a few month a year ).
Other variants of problems with Expire, but not really related to the raspberry pi -
We also had an incident when a plumber nocked down our gateway and did not reconnect the GSM USB dongle. This was during the winter in a remote place supervising 200 tree sensors. Once we got to that place in the spring and fixed the internet connection to that gateway, nothing worked. ( most likely due to the expiry of the encryption keys… )
We also have a lot of outdoor sensors that needs battery replacement, for example public bath temperatures. Nobody really cares if they do not transmit data until this time of the year, which means that the sensors can be without batteries or any REDIS database activity for several months during winter. We see some of these right now when people call us and wonder what the problem is when they have swapped the batteries and nothing works…
Thanks, I haven’t been able to find the device_session_ttl config file in the GatewayOS yocto build environment yet. Do you know were it is? Either in the build environment, or in the final file-system so I can try it. I’m not sure if this value is set on registration of the sensor, or if it is set later on as well?
OTAA requires quite good radio signals to work since it negotiates the encryption keys. We typically operate with very low RSSI values sometimes. Just one out of ten transmissions might reach the gateway, but it is enough to see at least some trend-curves, so we are stuck with ABP, and it never gives us any headaches like we had with OTAA.