# Many small mutations vs one large. Best usage patterns

**URL:** <https://discuss.dgraph.io/t/many-small-mutations-vs-one-large-best-usage-patterns/15589>\
**Category:** Users\
**Tags:** kind:question\
**Created:** [September 17, 2021, 9:34am UTC](https://discuss.dgraph.io/t/many-small-mutations-vs-one-large-best-usage-patterns/15589 "2021-09-17T09:34:15Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![igrekun](https://yyz1.discourse-cdn.com/flex007/user_avatar/discuss.dgraph.io/igrekun/32/8545_2.png) [@igrekun](https://discuss.dgraph.io/u/igrekun)\
**Post date:** [September 17, 2021, 9:34am UTC](https://discuss.dgraph.io/t/many-small-mutations-vs-one-large-best-usage-patterns/15589/1 "2021-09-17T09:34:15Z")

</div>

Hi!

Let’s say my app consumes data from an external API which returns up to around 5k entities per request.

General structure of an entity in api response looks like this:

```auto
{
  "uuid": "1234",
  "parent_uuid": "4321",
  "name": "SomeName",
  ...
}

```

Saving those in Dgraph calls for an upsert.

It is trivial to write an upsert mutation for a single entity i.e:

```auto
upsert {
  query {
    t as var(func: eq(xid, $uuid)) {uid}
    p as var(func: eq(xid, $parent_uuid)) {uid}
  }

  mutation {
    set {
      uid(t) <name> "SomeName" .
    }
  }

  mutation @if(gt(len(p),0)) {
    set {
      uid(p) <link> uid(t) .
    }
  }
}

```

Saving 5k of those entities would require running this block 5k times.

The other way (and that’s how I do it right now) is to first run a query and fetch all present target and parent nodes and then generate one huge NQUAD mutation, replacing blank nodes with matches where necessary and adding RDFs for links.

Which approach is better in terms of Dgraph ways of doing things? Which is more performant?

Does Dgraph optimize 5k runs of the upsert above if it happens under single transaction?

---

<div class="post-metadata">

**Author:** ![iluminae](https://yyz1.discourse-cdn.com/flex007/user_avatar/discuss.dgraph.io/iluminae/32/7367_2.png) [@iluminae](https://discuss.dgraph.io/u/iluminae)\
**Post date:** [September 17, 2021, 3:00pm UTC](https://discuss.dgraph.io/t/many-small-mutations-vs-one-large-best-usage-patterns/15589/2 "2021-09-17T15:00:51Z")

</div>

I have found that batching in many-thousands is pretty performant. I have batches of 1000 in my ingestion pipeline. It is a little awkward to make a bunch of unique variables in the query portion and use them below in the nquads, but once that is solved its fine to have a knob on arbitrary sized batches you can play with/optimize for your use case.

---

<div class="post-metadata">

**Author:** ![igrekun](https://yyz1.discourse-cdn.com/flex007/user_avatar/discuss.dgraph.io/igrekun/32/8545_2.png) [@igrekun](https://discuss.dgraph.io/u/igrekun)\
**Post date:** [September 17, 2021, 3:05pm UTC](https://discuss.dgraph.io/t/many-small-mutations-vs-one-large-best-usage-patterns/15589/3 "2021-09-17T15:05:24Z")

</div>

By batching you mean code-generating the query portion of request with thousands of variables and send that as a single transaction to Dgraph?

---

<div class="post-metadata">

**Author:** ![iluminae](https://yyz1.discourse-cdn.com/flex007/user_avatar/discuss.dgraph.io/iluminae/32/7367_2.png) [@iluminae](https://discuss.dgraph.io/u/iluminae)\
**Post date:** [September 17, 2021, 3:07pm UTC](https://discuss.dgraph.io/t/many-small-mutations-vs-one-large-best-usage-patterns/15589/4 "2021-09-17T15:07:06Z")

</div>

yea I currently have 1000 variables being resolved in the query{…} section and using them all in the set{…} section below. I hash the unique id of each thing to make its variable, and prefix the hash since a variable cannot start with a number.

---

<div class="post-metadata">

**Author:** ![igrekun](https://yyz1.discourse-cdn.com/flex007/user_avatar/discuss.dgraph.io/igrekun/32/8545_2.png) [@igrekun](https://discuss.dgraph.io/u/igrekun)\
**Post date:** [September 17, 2021, 3:09pm UTC](https://discuss.dgraph.io/t/many-small-mutations-vs-one-large-best-usage-patterns/15589/5 "2021-09-17T15:09:43Z")

</div>

Thanks will try that!

---

<div class="post-metadata">

**Author:** ![zhaojiangkun](https://yyz1.discourse-cdn.com/flex007/user_avatar/discuss.dgraph.io/zhaojiangkun/32/10393_2.png) [@zhaojiangkun](https://discuss.dgraph.io/u/zhaojiangkun)\
**Post date:** [April 20, 2023, 9:20am UTC](https://discuss.dgraph.io/t/many-small-mutations-vs-one-large-best-usage-patterns/15589/6 "2023-04-20T09:20:50Z")

</div>

Hi，I wonder the max threshold of {…}.  
Or what is {…} associated with?  
Or I wonder how many keys using one transaction in {…}.