# Documentation about how Badger actually stores Dgraph data on disk

**URL:** <https://discuss.dgraph.io/t/documentation-about-how-badger-actually-stores-dgraph-data-on-disk/9184>\
**Category:** Badger\
**Created:** [August 3, 2020, 11:49am UTC](https://discuss.dgraph.io/t/documentation-about-how-badger-actually-stores-dgraph-data-on-disk/9184 "2020-08-03T11:49:55Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![gotjoshua](https://yyz1.discourse-cdn.com/flex007/user_avatar/discuss.dgraph.io/gotjoshua/32/4150_2.png) [@gotjoshua](https://discuss.dgraph.io/u/gotjoshua)\
**Post date:** [August 3, 2020, 11:49am UTC](https://discuss.dgraph.io/t/documentation-about-how-badger-actually-stores-dgraph-data-on-disk/9184/1 "2020-08-03T11:49:55Z")

</div>

I am just starting to do a deep-dive into how badger is storing data. So, I am looking at the vlog files for zero and alpha and trying to make sense of it all.

I want to create an offline-first system that is designed to sync seamlessly with dgraph/badger, thus I want to represent the data in IndexedDB as conveniently and efficiently as possible.

I want to know:

1. How does badger persist its key-value pairs?
2. How does dgraph translate its triads into key-value pairs?
3. What is the relationship between alpha and zero?
4. How can I read the badger files to understand what it is storing where, and why?  
4b. This includes the issues of encoding, compression and encryption - How to disable compression and re-encode the log file to human readable utf8?

I assume that this is documented somewhere, but I didn’t find it in the level of detail I want.  
I did find this basic conceptual description:

> **[Overview - Dgraph](https://docs.hypermode.com/dgraph/overview)**

> Posting Lists get stored in Badger, in a key-value format, like so:  
> `(Predicate, Subject) --> PostingList`

## In Zero:

I see that the vlog has many of these three slightly different sequences:  
`alpha:70800֚Bz1- , alpha:7080 0֚Bz1, alpha:70800Bz1-`  
and they are always followed by a number

Another often repeated sequences is:  
`!badger!txn`

## In Alpha:

I see that the `!badger!txn` sequence is also present, but I don’t find matching sequence numbers

I can see the actual changes, eg. this is how the change of name@en from “michaellll” to “mikeee” is stored:  
`!badger!tx601517j@amemichaellll> \'`hnamemikeee\> '`hqb@!name'\> ~a,S Mikeee *enh v*!badger!txn>82881 `

## Please Help!

In addition to replies here in discuss, I am open to direct links to commits, documentation, blog posts, or whatever you find relevant for this deep dive.

---

<div class="post-metadata">

**Author:** ![gotjoshua](https://yyz1.discourse-cdn.com/flex007/user_avatar/discuss.dgraph.io/gotjoshua/32/4150_2.png) [@gotjoshua](https://discuss.dgraph.io/u/gotjoshua)\
**Post date:** [August 3, 2020, 8:02pm UTC](https://discuss.dgraph.io/t/documentation-about-how-badger-actually-stores-dgraph-data-on-disk/9184/2 "2020-08-03T20:02:50Z")

</div>

I assume that #4b is related to parsing protocol buffers:

> **[Protocol Buffer Basics: Python](https://protobuf.dev/getting-started/pythontutorial/)**
>
> A basic Python programmers introduction to working with protocol buffers.

But i have no idea how to go about reading the vlog and/or sst files and parsing them in order to output something more human readable…

any help?

---

<div class="post-metadata">

**Author:** ![dmai](https://yyz1.discourse-cdn.com/flex007/user_avatar/discuss.dgraph.io/dmai/32/1254_2.png) [@dmai](https://discuss.dgraph.io/u/dmai)\
**Post date:** [August 3, 2020, 10:18pm UTC](https://discuss.dgraph.io/t/documentation-about-how-badger-actually-stores-dgraph-data-on-disk/9184/3 "2020-08-03T22:18:22Z")

</div>

Hi @gotjoshua. You can use the `dgraph debug` tool to inspect the content of the p directory of Dgraph Alpha. Here’s some docs that show some example output: [https://dgraph.io/docs/howto/#debug-tool-output](https://dgraph.io/docs/howto/#debug-tool-output)

You can use Badger to open up a Badger DB (e.g., a Dgraph p directory) and iterate over the keys and values.

You can also check out the [Dgraph paper](https://dgraph.io/paper) to learn about the underlying data format.

---

<div class="post-metadata">

**Author:** ![gotjoshua](https://yyz1.discourse-cdn.com/flex007/user_avatar/discuss.dgraph.io/gotjoshua/32/4150_2.png) [@gotjoshua](https://discuss.dgraph.io/u/gotjoshua)\
**Post date:** [August 7, 2020, 4:14pm UTC](https://discuss.dgraph.io/t/documentation-about-how-badger-actually-stores-dgraph-data-on-disk/9184/4 "2020-08-07T16:14:11Z")

</div>

Thanks for the reply, the debug tool is quite helpful!

If I use a badger command line, can i access a locked folder? or i still need to shut down the alpha in order to have a look?

Also, I’m wondering if there is anyone from the core team that could take time to answer this thread:

> [@Querying for historical values, and creation and modification timestamps as dateTime](http://discuss.hypermode.com/t/querying-for-historical-values-and-creation-and-modification-timestamps-as-datetime/9097):
>
> I read that dgraph is configuring badger to keep all value mutations in the log permanently: [options.NumVersionsToKeep . We set this by default to 1 in Badger, but in Dgraph, we set this to infinity](https://dgraph.io/blog/post/badger-over-rocksdb-in-dgraph/#3d-access-key-value-versions-a-idanchor-3da) So… My question is, can we access that “infinite history” via dgraph queries? I am experimenting with adding explicit created and modified dateTime predicates, but it feels like this could/should be provided via an implicit function of some sort. Intuitively i would like this kind of query: { …

Now (thanks to the debug tool) I can see that the infinite history is there, and I’d love to have easy access to it (without having to create additional dgraph structures for it)

---

<div class="post-metadata">

**Author:** ![ibrahim](https://yyz1.discourse-cdn.com/flex007/user_avatar/discuss.dgraph.io/ibrahim/32/6387_2.png) [@ibrahim](https://discuss.dgraph.io/u/ibrahim)\
**Post date:** [August 9, 2020, 2:06pm UTC](https://discuss.dgraph.io/t/documentation-about-how-badger-actually-stores-dgraph-data-on-disk/9184/5 "2020-08-09T14:06:24Z")

</div>

> [@gotjoshua](#):
>
> If I use a badger command line, can i access a locked folder? or i still need to shut down the alpha in order to have a look?

You can open that directory in a read-only mode with `BypassDirLock` set to true in badger

> <https://github.com/dgraph-io/badger/blob/3e067a54e5964bf7640c7d74fa92cd015f2a3fc8/options.go#L88-L91>

It would be a good idea to just make a copy of the badger directory, delete the lock file and then perform whatever operations you want.
