Dgraph caches configuration

alexeevaa · April 12, 2023, 9:53pm

I could not find any info on how to configure Dgraph caches.
There is cache size, cache percentages and three caches:

PostingListCache,PstoreBlockCache,PstoreIndexCache

As far as i can tell PostingListCache does not work at all, but worked in 21.12 on dgraph side.
In badger repo there is info on block cache:

github.com

dgraph-io/badger/blob/a89c52c8368658690b1669fdce4ede80cfaa2cbc/table/table.go#L78

    
      
          
          
	// BlockSize is the size of each block inside SSTable in bytes.
          	BlockSize int
          
          
	// DataKey is the key used to decrypt the encrypted text.
          	DataKey *pb.DataKey
          
          
	// Compression indicates the compression algorithm used for block compression.
          	Compression options.CompressionType
          
          
	// Block cache is used to cache decompressed and decrypted blocks.
          	BlockCache *ristretto.Cache
          	IndexCache *ristretto.Cache
          
          
	AllocPool *z.AllocatorPool
          
          
	// ZSTDCompressionLevel is the ZSTD compression level used for compressing blocks.
          	ZSTDCompressionLevel int
          }
          
          
// TableInterface is useful for testing.

Also here we can find something about index cache:

github.com

dgraph-io/badger/blob/a89c52c8368658690b1669fdce4ede80cfaa2cbc/table/table.go#L514

    
      
          			i = oLen - 1
          		}
          		y.AssertTrue(t.offsets(&bo, i))
          		if bytes.HasPrefix(bo.KeyBytes(), prefix) {
          			res = append(res, string(bo.KeyBytes()))
          		}
          	}
          	return res
          }
          
          
func (t *Table) fetchIndex() *fb.TableIndex {
          	if !t.shouldDecrypt() {
          		return t._index
          	}
          
          
	if t.opt.IndexCache == nil {
          		panic("Index Cache must be set for encrypted workloads")
          	}
          	if val, ok := t.opt.IndexCache.Get(t.indexKey()); ok && val != nil {
          		return val.(*fb.TableIndex)
          	}

Seems like block cache stores decompressed parts of badger tables & index cache stores badger table indices. Both help to avoid reading from disc and doing related work. Still i have no clue how to adjust them. Are there any general rules? Like, what percentage of RAM should I use? What should I do to help writes over reads or vice versa?

MichelDiz · April 12, 2023, 10:18pm

Like this

--cache "size-mb=1024; percentage=10,65,25;" 
#PostingListCache 10%, PstoreBlockCache 65%, PstoreIndexCache 25%

In general size-mb should be half of the available RAM.

The defaults are

size-mb=1024; percentage=0,65,35;

Topic		Replies	Views
Block cache might be too small Dgraph kind:bug	0	638	August 31, 2021
Tuning Dgraph / Badger for HDDs instead of SSDs? Dgraph	1	426	February 22, 2025
Does Dgraph supports database side caching? App Development	5	55	August 8, 2024
Releasing BadgerDB v2.0 - Dgraph Blog Blog	3	812	February 4, 2020
Badger LSMT options on huge dataset Dgraph kind:question , badger , dgraph	0	404	May 2, 2023

Dgraph caches configuration

Related topics