# Bulk loader -x option **URL:** \ **Category:** Users\ **Tags:** mutation\ **Created:** [March 11, 2020, 5:08pm UTC](https://discuss.dgraph.io/t/bulk-loader-x-option/6115 "2020-03-11T17:08:39Z")\ **Posts on this page:** 8\ **Page:** 1
**Author:** ![aamrtv](https://yyz1.discourse-cdn.com/flex007/user_avatar/discuss.dgraph.io/aamrtv/32/2519_2.png) [@aamrtv](https://discuss.dgraph.io/u/aamrtv)\ **Post date:** [March 11, 2020, 5:08pm UTC](https://discuss.dgraph.io/t/bulk-loader-x-option/6115/1 "2020-03-11T17:08:39Z")
I’m trying to load initial data into dgraph via the _bulk loader_ and afterwards add new/modify existing nodes with the _live loader_. The problem is that if the _live loader_ uploads nodes, which were uploaded by the _bulk loader_ before, it creates duplicate nodes with new uids. I don’t want duplicate nodes. I need those new nodes either not to be loaded at all (if they bring no new edges for bulk loaded nodes) or to modify already existing nodes (if those new nodes **do** bring new edges for old nodes). If I use **only** _live loader_, then it is trivial: I just add **-x dirname** to all my dgraph live commands and I get a xid directory named **dirname** , so I don’t get duplicate nodes with new uids from the following _live loader_ data. The issue with bulk loader is that **-x** option does not create a folder for xids. Therefore, when I _live load_ nodes with the uids, which were uploaded by the _bulk loader_ before, I get duplicate nodes with new uids. How do I prevent duplicate nodes when I use bulk load first and live load afterwards? ---
**Author:** ![MichelDiz](https://yyz1.discourse-cdn.com/flex007/user_avatar/discuss.dgraph.io/micheldiz/32/11873_2.png) [@MichelDiz](https://discuss.dgraph.io/u/MichelDiz)\ **Post date:** [March 11, 2020, 6:23pm UTC](https://discuss.dgraph.io/t/bulk-loader-x-option/6115/2 "2020-03-11T18:23:47Z")
Thank you for reporting this I have created an issue for this and it is in the backlog now > > > \## Update > Previously it was believed that the --store\_xids flag had the same be…have as \`--xidmap\`. However, it is not true. But the \`--xidmap\` flag is needed in Bulkloader. > > \### What version of Dgraph are you using? > > \`\`\` > Dgraph version : v2.0.0-beta1 > Dgraph SHA-256 : 178663a98a3d59879a3d5c42928c89eb5f83afc2bfc0093272941e7a53515847 > Commit SHA-1 : 6fac5d7c4 > Commit timestamp : 2020-01-30 14:45:54 +1100 > Branch : HEAD > Go version : go1.13.7 > \`\`\` > > \### Have you tried reproducing the issue with the latest release? > Yes. > > \### Steps to reproduce the issue (command/config used to run Dgraph). > > Update: Ignore these steps, go to my last comment below. > > \`\`\` > dgraph bulk --store\_xids -f ./agrovoc\_2019-11-04\_lod.nt --format=rdf -s schema.sch > \`\`\` > > \`\`\` > dgraph bulk -x -f ./agrovoc\_2019-11-04\_lod.nt --format=rdf -s schema.sch > \`\`\` > > \### Expected behavior and actual result. > > It should create an XID folder to be used in later imports via Live loader. > > Origin of this issue: https://discuss.hypermode.com/t/bulk-loader-x-option/6115 ---
**Author:** ![dmai](https://yyz1.discourse-cdn.com/flex007/user_avatar/discuss.dgraph.io/dmai/32/1254_2.png) [@dmai](https://discuss.dgraph.io/u/dmai)\ **Post date:** [March 11, 2020, 7:31pm UTC](https://discuss.dgraph.io/t/bulk-loader-x-option/6115/3 "2020-03-11T19:31:39Z")
The `--store_xids` flag for bulk loader writes `xid` edges into your database. This is different from the `--xidmap` flag for live loader, which writes out the xid-uid mapping to a separate directory. ---
**Author:** ![aamrtv](https://yyz1.discourse-cdn.com/flex007/user_avatar/discuss.dgraph.io/aamrtv/32/2519_2.png) [@aamrtv](https://discuss.dgraph.io/u/aamrtv)\ **Post date:** [March 12, 2020, 5:43am UTC](https://discuss.dgraph.io/t/bulk-loader-x-option/6115/4 "2020-03-12T05:43:28Z")
Okay. How do I use the fact that `xid` edges are stored in the database to avoid duplicate nodes appearing after the bulk load, which is followed by \>= 1 live loads? The docs don’t give much insight into that. What is the purpose of writing `xid` directly to the database? ---
**Author:** ![aamrtv](https://yyz1.discourse-cdn.com/flex007/user_avatar/discuss.dgraph.io/aamrtv/32/2519_2.png) [@aamrtv](https://discuss.dgraph.io/u/aamrtv)\ **Post date:** [March 17, 2020, 3:44pm UTC](https://discuss.dgraph.io/t/bulk-loader-x-option/6115/5 "2020-03-17T15:44:37Z")
I am sorry, still interested in the answer, so I reply to bring the topic to the top. Feel free to How I can use xid edges stored in the dgraph after bulk load to avoid duplicates with the future dgraph live loads? ---
**Author:** ![MichelDiz](https://yyz1.discourse-cdn.com/flex007/user_avatar/discuss.dgraph.io/micheldiz/32/11873_2.png) [@MichelDiz](https://discuss.dgraph.io/u/MichelDiz)\ **Post date:** [March 27, 2020, 4:06am UTC](https://discuss.dgraph.io/t/bulk-loader-x-option/6115/6 "2020-03-27T04:06:44Z")
I have found the answer, take a look at the issue I had opened. The last comment. Cheers. ---
**Author:** ![Anurag](https://yyz1.discourse-cdn.com/flex007/user_avatar/discuss.dgraph.io/anurag/32/2599_2.png) [@Anurag](https://discuss.dgraph.io/u/Anurag)\ **Post date:** [April 9, 2020, 11:00am UTC](https://discuss.dgraph.io/t/bulk-loader-x-option/6115/7 "2020-04-09T11:00:43Z")
@aamrtv thanks for raising this. It has been resolved in the current master. You can use `--xidmap dirname` flag while doing a bulk upload to save xids in a directory named **dirname**. ---
**Author:** ![system](https://canada1.discourse-cdn.com/flex007/uploads/dgraph/original/2X/2/2b38fdece2abe5814f2e51bee69d1e67fc304b0a.png) [@system](https://discuss.dgraph.io/u/system)\ **Post date:** [May 9, 2020, 11:00am UTC](https://discuss.dgraph.io/t/bulk-loader-x-option/6115/8 "2020-05-09T11:00:44Z")
This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.