My data is a classical time series kind with data unrelated between different time series. A lookup for a key in 1 time series does not have to search any other of the time series data in the DB.
The key is different from the time series item timestamp in the time series and I am actually looking for a hybrid crossover store between KV and time series kind…
Segmentation: What is the best advice?
The intention is to optimise performance of lookups/writes/disk-persisting…
In my case I have a pretty high number of keys (in range of 100s of millions) and by the nature of the case I could easily segment into 1000s of segments…
-
Should I open 1 BadgerDB for each time series or rather use some internal mechanism for segmentation?
-
Is there a way to delete efficiently the old/expired data (by bulk), from the BadgerDB periodically (without keeping a TTL on the single KV items). Let’s say, the time series of data arrives constantly… and each hour I wish to efficiently delete the whole hour of data that has expired… (for example, older than X days). I immagine this could be somewhat similar the classical Unix logrotate handling of log files…