Is there any way to perform local persistence on the gateway for cases where the internet connection of the gateway (backhaul) is temporarily lost?
In other words, it would be good if the internet connection of the gateway is lost for example for 1 hour, the gateway can store locally the messages received from the nodes in that period and send them once the connection is reestablished.
Can this be possible?
Thanks!
Have you tested the qos (quality of service) config option in lora-gateway-bridge.toml?
# Quality of service level
#
# 0: at most once
# 1: at least once
# 2: exactly once
#
# Note: an increase of this value will decrease the performance.
# For more information: https://www.hivemq.com/blog/mqtt-essentials-part-6-mqtt-quality-of-service-levels
qos=0
I have not tested this, but I would expect that setting this to (at least) 1 could already solve your issue as then the buffering should happen on MQTT client level.
I’m going to try it and I’ll tell you the results. Anyway, this approach implies that I have to install the Lora Gateway Bridge in the GW, anyway it is not a big problem as long as the GW supports it.
Thank you very much
Has anyone gotten this to work?
I tried setting qos=1, clean_session=false, and set a client_id on both the lora-gateway-bridge and loraserver, however I could not get the gateway to publish uplinks after coming back online after it received uplinks while offline.
After reading the documentation for Paho MQTT Golang ClientOptions, for MQTT data to persist when offline and send when online, the following needs to be set:
Currently in the lora-gateway-bridge MQTT backend file, AutoReconnect is False, and both the MessageChannelDepth and Store are not set.
@brocaar can you please comment if this is the problem with qos 1 and 2 not working?
Thanks.
Thanks for digging into this. I believe a memory store will be created by default, if what my experiences with using the Paho Go library elsewhere (and its logging) lead me to believe are true. That being said, it would be helpful to get this working and documented.
I’ve been playing around with qos=1 and qos=2 and lora-gateway-bridge version 3.0.1 and I’ve found some interesting things:
Qos works as long as your connection doesn’t drop for too long. If it’s just a few seconds(10-40) things almost work as you’d expect.
Qos 2 somewhat behaves like 1. So you get multiple duplicate messages upon restoring the lora-gateway-bridge connection to the broker.
Not all messages are stored from the moment that the bridge loses connection to the broker. Sometimes you get all messages and other times there will be many missing. It’s behavior is erratic but you seems to lose more messages after being down 40 seconds or greater.
The amount of time that the connection is down determines whether messages or stored at all. If I intentionally drop the connection to the broker for 120 seconds no messages are stored. If I drop the connection for 60 seconds, messages are stored but there are some that are lost.
SetOrderMatters seems to be ignored even though this is set to true by default. Messages are to be sent in the order that they are received and not out of order. So frame count 150 should always follow 149 and 151 should follow 150 for instance.
lora-gateway-bridge also crashes after it tries to reconnect once the mqtt broker is down for approximately 1 minute. If your connection goes up and down a few times after messages are stored lora-gateway-bridge never seems to catch up and live locks it seems.
Something horrible is happening on reconnect. It tries to create multiple sessions which many brokers don’t allow without a unique client id for each. Due to this it crashes almost every time that I test qos with mosquitto due to reconnection retries it seems.
The only way that I can keep it from crashing is by using vernemq broker and specifying this option in the vernemq.conf
allow_multiple_sessions = on
I also built the bridge from git and added a few options but that was unnecessary and didn’t actually make lora-gateway-bridge behave any better. It actually crashes faster once I declare other options. @lpitka mentioned that AutoReconnect needed to be true(I read this also) and MessageChannelDepth should be set but as is things work without a known MessageChannelDepth. Evidently there is a default setting for MessageChannelDepth even though the docs don’t say what it is(100?).
I think it would be apropos if we could specify the MessageChannelDepth and the Store. Some devices are better suited for a file store rather than a memory store and vice versus.
This definitely needs more testing.
Has anyone done anything similar and if so what have you seen?
Great investigation! Regarding MessageChannelDepth, there is a feature request here to make it configurable, but obviously more is needed than that for a reliable offline configuration.
We have a business requirement to queue up messages for a short period of time while in a disconnected state, so we will also be looking into this in the next couple months.
Hi to all,
I’m in the same situation as @bconway : we need as a business requirement to have a local storage of messages in case of a lost internet connection (this kind of event normally happens on Saturday nights ahaha 
). By internet connection loss, I’m talking about up to 48hours (think about a factory closed on week-ends…)
Since it seems to be a common request and not a simple one, how about to join our efforts and propose to Mr. Brocaar to work on this specific item through, for example a crowdfunding effort ? I would be ready to finance such a development.
Thanks,
Luc
In the next release of LoRa Gateway Bridge, there will be a fix to make sure the Paho library is actually buffering the messages. It turned out that when the re-connect is handled manually, the Paho library just drops the messages. See:
Hello there. My question is similar. The gateway has sensor data collected. According to this data, the solenoid or sprinkler should be opened and closed. When the Internet is gone, this is disrupted. Can custom rules be run even when there is no internet in the gateway? Things should go on, even without the internet.
What are “rules” in your situation? If you are making decisions based on LoRa sensor data that needs to be available even when your internet connection is down, it sounds like you might consider an on-site deployment/solution.