Having issues with binary backups

Users
21

Here’s the output of a restore command:

root@dgraph-alpha-0:/dgraph#  dgraph restore -p /var/db/dgraph -l s3://s3.us-east-2.amazonaws.com/dgraph-backup-dgraph
[Decoder]: Using assembly version of decoder
Restoring backups from: s3://s3.us-east-2.amazonaws.com/dgraph-backup-dgraph
Writing postings to: /var/db/dgraph
Downloading "dgraph.20191204.092029.643/r21716-g1.backup", 1530 bytes
badger 2019/12/04 09:21:57 INFO: All 1 tables opened in 22ms
badger 2019/12/04 09:21:57 INFO: Replaying file id: 0 at offset: 46009658
badger 2019/12/04 09:21:57 INFO: Replay took: 11.467µs
badger 2019/12/04 09:21:57 DEBUG: Value log discard stats empty
Restoring groupId: 1
badger 2019/12/04 09:21:57 DEBUG: Storing value log head: {Fid:0 Len:80 Offset:46013951}
badger 2019/12/04 09:21:57 INFO: Got compaction priority: {level:0 score:1.73 dropPrefix:[]}
badger 2019/12/04 09:21:57 INFO: Running for level: 0
badger 2019/12/04 09:21:58 DEBUG: LOG Compact. Added 218254 keys. Skipped 0 keys. Iteration took: 453.236153ms
badger 2019/12/04 09:21:58 DEBUG: Discard stats: map[]
badger 2019/12/04 09:21:58 INFO: LOG Compact 0->1, del 2 tables, add 1 tables, took 567.882273ms
badger 2019/12/04 09:21:58 INFO: Compaction for level: 0 DONE
badger 2019/12/04 09:21:58 INFO: Force compaction on level 0 done
Restore version: 21716
Restore: Time elapsed: 1s

Something just came to mind. I’m testing dgraph’s backup by first backing up, then deleting all the data, using the “drop all” button in the ratel dashboard. Is it possible that during the restoration process ,dgraph sort of merges the dropped actions to the backed up actions?. Maybe the “drop all” action takes precedence over the backed up actions because it happens later.

22

If you didn’t made a lot of changes in between the two backups, what you are seeing is expected.

What are you doing after running dgraph restore? I checked the docs and I am realizing they are not very explicit after what to do next so I will work on improving them. In the meantime I will explain below.

The backup is not restored into a live cluster so if all you do is running dgraph restore it’s not going to have an effect on your existing cluster. The backup files are processed and the result is written to the p1, etc folders inside the directory passed to the -p flag. After that, you would copy those folders into whatever machine/container you are using for your new cluster and run dgraph alpha pointed at this folder. This will start a new alpha that loads the restored backup.

If you have multiple replicas,all of them should get a copy of the folder for their group. Let’s say you made a backup to a 2 group cluster with two replicas then you would copy:

  • p1 to first container of first group (most likely you want to rename it to p)
  • p1 to second container of first group.
  • p2 to first container of second group.
  • p2 to first container of first group.

I apologize for the insufficient docs. These are the kind of things that seem obvious to someone that’s worked with p directories before but are not obvious to other people.

23

Thanks i’ll try this and get back to you.

24

Any updates?

Best,
Martin

25

Apologies, i’ve been occupied with other more pressing activities. I’ll hopefully get back to you in a week or so

26

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.