Hi everyone. I’m trying to estimate the size for my Postgres and Redis instance, and I was wondering:
For Postgres:
What does ChirpStack store in Postgres?
How much can I expect data to grow? Does this depend on the number of gateways? I’m expecting 150 gateways, 1200 devices and total 40.000 messages per hour.
For Redis:
How much space does one LoRaWAN message use in Redis?
How long are messages stored in Redis? Are they deleted eventually?
Can I configure how long they last?
For reference, I’m using AWS RDS Aurora for Postgres and AWS ElastiCache for Redis. I just need to understand how much data is stored there, then I can determine the appropriate instance size.
I’ve actually been asking this for a while in a couple different threads and haven’t got a definitive answer. I’m not so concerned about Postgres, as there’s not a ton of stuff stored there and storage is cheap, but Redis is volatile and not so cheap, and if we’re scaling to massive volumes (say 100k-1million devices), i really need to know what my Redis deployment will look like. I know you can change the max number of frames for devices and gateways that are stored in Redis cache, but i’m not sure what else is tweakable. I’ll get back to you shortly, as i have about 500 devices and 10-ish gateways and i can give you an idea of what my Redis utilization is. definitely following this thread though!
In general, PostgreSQL is stored for all data that must never expire (tenants, users, applications, devices, device-profiles, device root-keys, etc…), Redis is used for storing data that eventually will expire (e.g. device-sessions, DevAddr > DevEUI mappings, de-duplication, metrics, streams). In most cases you want to make sure that the Redis data is persisted on disk, as flushing the Redis database would mean that all devices need to re-join.
In general, no housekeeping should be needed as the PostgreSQL database does not store data that should be cleaned up over time (unless you want to delete devices, but you should do that through the UI or API). The keys in Redis will auto-expire after a given time.
Per gateway, there are various metrics that are stored in Redis (per hour, day and month). Per device there are also various metrics that are stored. If you are using the measurements feature (in the device-profile), then each measurement will also be stored in Redis (per hour, day and month). But also for these metrics, this data will expire after some time.
@Guilleojeda Just anecdotally, i have 2500 devices or so on my network server, but only 500 have been activated (OTAA). I also have 25 gateways loaded, only half are online and the other half have been offline for > 1 month.
I am currently sitting around 42MB of Redis usage (on Redis Cloud) which seems quite high to me, but i do have HA/replication enabled, which i think doubles the memory usage reported. I also i did have device_frame_log_max_history=6 in my NS config, which i didn’t actually need since so I have since removed that entry, which i assume should set the value to 0 and eventually that data will be evicted.
@brocaar What is the risk if the cache fills up? I’m using volatile-lru as my eviction policy, not sure if volatile-ttl is a better option? i have alerting on low memory regardless so i can scale up if/when necessary.