Redis HA w/o clustering

ChirpStack Application Server
1

I have redis setup with 2 standalone redis instances

servers=[ "10.0.0.1",  "10.0.0.2" ]
password="SECRET"
database=0
cluster=false

but it doesn’t look like chirpstack is mirroring the keys on both instances, so if I lose the “active” instance then I’ve lost the keys/activations until the device reactivates. Is that correct?

Do I need to make sure to externally sync the keys between the two instances or is this something that chirpstack can do? If yes, this would be a great feature addition to mirror all keys in environments that don’t support true clustering.

1 Like
2

Why are you trying to have two redis instances? I think you should try with the amount of databases you need but in postgreSQL not in redis. Services provided by this two are quite different.

PD: Maybe I missunderstood your issue. Hope this helps.

1 Like
3

For HA purposes. If one redis instance goes down I would like the other one to pick up where the first left off. Otherwise mm y devices will have to renegotiate with the server

1 Like
4

Unfortunately, currently Redis Sentinel isn’t supported yet. I believe it would require a wrapper around the Redis connection object, which before performing any action polls which node is the master node. But this hasn’t been implemented yet.

1 Like
5

Unfortunately I’m not using sentinel. Just using two independent Redis instances for basic HA capability should one instance go down, the other could continue to serve but unfortunately the keys aren’t mirrored.

1 Like
6

Hi @brocaar,
is redis sentinel for v4 not supported? We are trying to migrate from v3 to v4 but I’m getting lot of failures like:

ERROR chirpstack::api: Log request error, error: An error was signalled by the server: You can't write against a read only replica.
7

I believe this is not yet available in the Rust redis client, but this is in the work:

8

Hi @brocaar,
thanks for information. I‘m waiting for it very hard :smiley:

1 Like
9

the PR has been merged!!

10

Hi @brocaar,
since the redis client has now basic sentinel functionality, is there any chance to get this integrated for the next release?

11

I believe the connection-pooling is still lacking in the Redis client. Let me look into if this has been added since the last time I looked at it, or if it is simple to propose a pull-request for this.

12

Hi @brocaar,
isn’t this a sentinel modul for the redis rust client:

Regards,
Stefan

13

Yes, but I believe what is still missing is a connection-pool implementation for this part of the code. ChirpStack uses deadpool, e.g. this is the implementation of a regular Redis connection pool:

The same exists for Redis Cluster:

I believe the same is needed for Redis Sentinel.

14

Hi @brocaar! How are you!

Please can you inform if sentinel is already supported?

Many Thanks!

15

It is not supported yet.

16

Hello @brocaar,

Do you still have a shared interest in introducing Redis Sentinel to Chirp v4? I have noticed that Deadpool recently added support for Sentinel mode, and the implementation file you are referencing for Redis Cluster is now defined for Redis Sentinel too.

Sentinel support was added to Deadpool in the 0.17.0 release.

1 Like