Live loader produces duplicates with upsertPredicate enabled

Issues Dgraph
#1

Report a Dgraph Bug

I’m pushing a graphql schema to dgraph using xid fields on types that map to the xid dgraph predicate. When I run live loader with upsertPredicate ‘xid’ however, subsequent imports result in duplicate records being created.

What version of Dgraph are you using?

Dgraph Version 
$ dgraph version
 
Dgraph version   : v21.03.2
Dgraph codename  : rocket-2
Dgraph SHA-256   : 00a53ef6d874e376d5a53740341be9b822ef1721a4980e6e2fcb60986b3abfbf
Commit SHA-1     : b17395d33
Commit timestamp : 2021-08-26 01:11:38 -0700
Branch           : HEAD
Go version       : go1.16.2
jemalloc enabled : true

Have you tried reproducing the issue with the latest release?

Yes

What is the hardware spec (RAM, OS)?

MacOS / Docker / 16GB (8GB allocated)

Steps to reproduce the issue (command/config used to run Dgraph).

This is the schema I’m pushing to my dgraph instance:
index.graphqls (2.2 KB)

This is the JSON file I’m live loading.
out.json (98.7 KB)

This is the command I’m using: dgraph live --files /path_to/out.json --upsertPredicate "xid"

Expected behaviour and actual result.

Buildings with the same xid value should not be duplicated, but they are.

#2

It’s entirely possible that I’m missing something obvious here, in that case please point me in the right direction :slight_smile:

#3

Duplicate (or rename it) the XID value in the dataset to the key “uid”. In fact, you don’t need the XID field during the live load. This field is necessary for the dataset already in the DB only. For the case you are loading new data, you have to use UID instead of XID.

The upsert feature in live load takes UID in the dataset into consideration instead of XID. And when it loads it, it creates the XID field to compare with the new data during the live load.

BTW, the UID key have to be a blank node. It will throw an error if you don’t prefix it with _:

#4

@martaver Curious if you ever got this working? I’m encountering the same issue with Live Loader using
a graphql schema and json formatted data.

I’m trying to follow the advice of MichelDiz above who suggested mapping the XID value to the key “uid” but not having any luck.

If you got it working, could you please post what your schema and .json file looked like?

Thanks!