# Upsert Will duplicate nodes be generated in the process of multithreading data update using the upsert block **URL:** \ **Category:** Dgraph\ **Created:** [December 15, 2020, 3:24am UTC](https://discuss.dgraph.io/t/upsert-will-duplicate-nodes-be-generated-in-the-process-of-multithreading-data-update-using-the-upsert-block/11905 "2020-12-15T03:24:52Z")\ **Posts on this page:** 10\ **Page:** 1
**Author:** ![tss](https://yyz1.discourse-cdn.com/flex007/user_avatar/discuss.dgraph.io/tss/32/4409_2.png) [@tss](https://discuss.dgraph.io/u/tss)\ **Post date:** [December 15, 2020, 3:24am UTC](https://discuss.dgraph.io/t/upsert-will-duplicate-nodes-be-generated-in-the-process-of-multithreading-data-update-using-the-upsert-block/11905/1 "2020-12-15T03:24:52Z")
Will duplicate nodes be generated in the process of multithreading data update using the upsert block？ ---
**Author:** ![anand](https://yyz1.discourse-cdn.com/flex007/user_avatar/discuss.dgraph.io/anand/32/5674_2.png) [@anand](https://discuss.dgraph.io/u/anand)\ **Post date:** [December 15, 2020, 4:18am UTC](https://discuss.dgraph.io/t/upsert-will-duplicate-nodes-be-generated-in-the-process-of-multithreading-data-update-using-the-upsert-block/11905/2 "2020-12-15T04:18:04Z")
Hi @tss, This is a popular question, hence I am trying to explain at length. Dgraph does not have a notion of unique attribute values across nodes. In fact every node in Dgraph is guaranteed to have a distinct **uid** (this attribute is controlled by Dgraph). Thus, when we talk about duplicates, we can only think of them in terms of the **attribute value on the nodes**. Let’s imagine that Dgraph has predicates **fullName** and **accountBalance**. We declare that the **fullName** is unique within our dataset. The simplest way to avoid duplicates is to go with a logic below. **Path A** If a node with value **fullName** exists update node X with value **accountBalance** **Path B** if a node X with value **fullName** does not exist create node X update node X with value **fullName** and **accountBalance** This, of course, is the **[upsert block](https://dgraph.io/docs/mutations/upsert-block/)**, a construct supported by Dgraph. The upsert block can be invoked via Ratel as well as Dgraph clients. **Multi-threading / Concurrency** If our transactions are spaced out, with no concurrency, the upsert block will help in avoiding duplicates. But if transactions happen concurrently, **we could still end up with duplicates**. We need _an additional mechanism_ to help avoid duplicates for this particular concurrent update scenario. This is exactly where @upsert directive in the schema helps. The @upsert directive checks if concurrent transactions are modifying nodes with the same attribute value, and if found, aborts one of the transactions. In our scenario, we can set the @upsert directive on the **fullName** attribute. From the client perspective, all it needs to do in case of an aborted transaction is to do a retry. When duplicate transactions arrive concurrently, the first one will take **Path B** and the one which retries will take **Path A**. Here is a [video](https://www.youtube.com/watch?v=lIUNIR7L7jw&t=4s) on the upsert directive. ---
**Author:** ![tss](https://yyz1.discourse-cdn.com/flex007/user_avatar/discuss.dgraph.io/tss/32/4409_2.png) [@tss](https://discuss.dgraph.io/u/tss)\ **Post date:** [December 15, 2020, 6:27am UTC](https://discuss.dgraph.io/t/upsert-will-duplicate-nodes-be-generated-in-the-process-of-multithreading-data-update-using-the-upsert-block/11905/3 "2020-12-15T06:27:00Z")
The result of my upsert test is different from the result of the test on the video. ---
**Author:** ![tss](https://yyz1.discourse-cdn.com/flex007/user_avatar/discuss.dgraph.io/tss/32/4409_2.png) [@tss](https://discuss.dgraph.io/u/tss)\ **Post date:** [December 15, 2020, 6:29am UTC](https://discuss.dgraph.io/t/upsert-will-duplicate-nodes-be-generated-in-the-process-of-multithreading-data-update-using-the-upsert-block/11905/4 "2020-12-15T06:29:17Z")
![image](https://canada1.discourse-cdn.com/flex007/uploads/dgraph/original/2X/c/cb91ed7e379c8be46df642ef118f73d5a42d2245.png) ![image](https://canada1.discourse-cdn.com/flex007/uploads/dgraph/original/2X/f/f76afd0f660bc6b4f60dac127f537066630966e4.png) ![image](https://canada1.discourse-cdn.com/flex007/uploads/dgraph/original/2X/a/a9120501121e89721442826b22f5939fbc5d2e6b.png) ---
**Author:** ![anand](https://yyz1.discourse-cdn.com/flex007/user_avatar/discuss.dgraph.io/anand/32/5674_2.png) [@anand](https://discuss.dgraph.io/u/anand)\ **Post date:** [December 15, 2020, 6:51am UTC](https://discuss.dgraph.io/t/upsert-will-duplicate-nodes-be-generated-in-the-process-of-multithreading-data-update-using-the-upsert-block/11905/5 "2020-12-15T06:51:45Z")
Hi @tss , Thanks for sharing the schema. Please also share the steps you followed in a bit more detail. If you follow the steps exactly as mentioned in the video, you should see a transaction abort. ---
**Author:** ![tss](https://yyz1.discourse-cdn.com/flex007/user_avatar/discuss.dgraph.io/tss/32/4409_2.png) [@tss](https://discuss.dgraph.io/u/tss)\ **Post date:** [December 15, 2020, 6:56am UTC](https://discuss.dgraph.io/t/upsert-will-duplicate-nodes-be-generated-in-the-process-of-multithreading-data-update-using-the-upsert-block/11905/6 "2020-12-15T06:56:30Z")
I ran the test3 function twice. The first run produced a node, and then the second run, and found that two nodes with the same topic\_type were produced. ---
**Author:** ![anand](https://yyz1.discourse-cdn.com/flex007/user_avatar/discuss.dgraph.io/anand/32/5674_2.png) [@anand](https://discuss.dgraph.io/u/anand)\ **Post date:** [December 15, 2020, 7:01am UTC](https://discuss.dgraph.io/t/upsert-will-duplicate-nodes-be-generated-in-the-process-of-multithreading-data-update-using-the-upsert-block/11905/7 "2020-12-15T07:01:52Z")
In the code, you are committing immediately. When you run this code twice, Dgraph simply processes it as two back-to-back transaction and no aborts are done. Please note that in the video, the transactions were sent to Dgraph and then committed one after another. This was done to illustrate two transactions overlapping with each other (which is likely to happen if you would run your example in a multithreaded mode). ---
**Author:** ![tss](https://yyz1.discourse-cdn.com/flex007/user_avatar/discuss.dgraph.io/tss/32/4409_2.png) [@tss](https://discuss.dgraph.io/u/tss)\ **Post date:** [December 15, 2020, 7:13am UTC](https://discuss.dgraph.io/t/upsert-will-duplicate-nodes-be-generated-in-the-process-of-multithreading-data-update-using-the-upsert-block/11905/8 "2020-12-15T07:13:21Z")
Use 100 threads to insert and find that 5 nodes are successfully inserted！ ---
**Author:** ![anand](https://yyz1.discourse-cdn.com/flex007/user_avatar/discuss.dgraph.io/anand/32/5674_2.png) [@anand](https://discuss.dgraph.io/u/anand)\ **Post date:** [December 15, 2020, 7:25am UTC](https://discuss.dgraph.io/t/upsert-will-duplicate-nodes-be-generated-in-the-process-of-multithreading-data-update-using-the-upsert-block/11905/9 "2020-12-15T07:25:17Z")
I suggest you use the same 100 threads, but write up a [conditional upsert block](https://dgraph.io/docs/mutations/conditional-upsert/#example-of-conditional-upsert) instead of simple mutations. You should see a better result. Pseudo code for conditional upsert: check for node with `topic_type1` if not present create node Please see if it helps. Here is the python documentation for [conditional upsert](https://github.com/dgraph-io/pydgraph#running-a-conditional-upsert). ---
**Author:** ![seanlaff](https://yyz1.discourse-cdn.com/flex007/user_avatar/discuss.dgraph.io/seanlaff/32/2142_2.png) [@seanlaff](https://discuss.dgraph.io/u/seanlaff)\ **Post date:** [March 13, 2021, 1:12am UTC](https://discuss.dgraph.io/t/upsert-will-duplicate-nodes-be-generated-in-the-process-of-multithreading-data-update-using-the-upsert-block/11905/10 "2021-03-13T01:12:57Z")
I’ve been confused about this for over a year, but this comment really clears things up. Thanks!