# Upper Transaction Mutation Limit - Concurrent Mutation Processing?

**URL:** <https://discuss.dgraph.io/t/upper-transaction-mutation-limit-concurrent-mutation-processing/19139>\
**Category:** Dgraph\
**Created:** [December 22, 2023, 5:12am UTC](https://discuss.dgraph.io/t/upper-transaction-mutation-limit-concurrent-mutation-processing/19139 "2023-12-22T05:12:01Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![rahst12](https://yyz1.discourse-cdn.com/flex007/user_avatar/discuss.dgraph.io/rahst12/32/2416_2.png) [@rahst12](https://discuss.dgraph.io/u/rahst12)\
**Post date:** [December 22, 2023, 5:12am UTC](https://discuss.dgraph.io/t/upper-transaction-mutation-limit-concurrent-mutation-processing/19139/1 "2023-12-22T05:12:01Z")

</div>

The writes to our system are transactional mutations. We see that the `pending_propsoals` exceeded going from 128, 256, 512, even 1024. This would indicate that DGraph can’t keep up with the amount of transactional mutations we’re sending it. When polling the Prometheus endpoint, we regularly see 1200+ mutations with 300-600 queries at the same time. This cluster is deployed on performant hardware (80 cores/256gb ram per container with SSDs. There seems to be an upper transaction mutation limit here.

I understand DGrpah does transactional mutations in serial, which seems to be the limiting factor here. First, how do I confirm this?

Second, I see that pull request introduced concurrent transactions with mutations:

> <https://github.com/dgraph-io/dgraph/pull/7694>
>
> \<!--
> Your title must be in the following format: topic(Area): Feature
> Topic mu…st be one of build|ci|docs|feat|fix|perf|refactor|chore|test
> 
> Sample Titles:
> feat(Enterprise): Backups can now get credentials from IAM
> fix(Query): Skipping floats that cannot be Marshalled in JSON
> perf: \[Breaking\] json encoding is now 35% faster if SIMD is present
> chore: all chores/tests will be excluded from the CHANGELOG
> 
> Please add a description with these things:
> 1. A good description explaining the problem and what you changed.
> 2. If it fixes any GitHub issues, say "Fixes #GitHubIssue".
> 3. If it corresponds to a Jira issue, say "Fixes DGRAPH-###".
> 4. If this is a breaking change, please put "\[Breaking\]" in the title. In the description, please put a note with exactly who these changes are breaking for.
> \--\>
> 
> This PR introduced concurrent mutation processing. It does so in this way:
> \- We receive a txn with mutation at start ts = Ts.
> \- The server is at MaxAssignedTs Tm \< Ts.
> \- Before, we would block proposing until Tm \>= Ts. This PR changes that.
> \- Now, we propose the mutation immediately.
> \- Once the mutation goes through raft, it is executed concurrently, and the "seen" MaxAssignedTs is registered as Tm-seen.
> \- The same mutation is also pushed to applyCh, like we do today.
> \- When applyCh sees the mutation, it checks if any reads the txn incurred, have been written to with a commit ts in the range (Tm-seen, Ts\]. If so, the mutation is re-run.
> \- If no commits have happened for the read key set, we are done.
> \- If multiple mutations happen for the same txn, the sequential mutations are always run serially by applyCh. This is to avoid edge cases.
> 
> This sequence improves the 75-th %-ile latency of mutations from 30ms down to 1ms. The 21M live loader dataset gets loaded on my workstation in 8 mins.
> 
> Given these improvements, I've removed ludicrous mode, which was buggy. This PR also contains a bug fix for how we were handling filterStringFunction.
> 
> 
> \---
> This change is \[\<img src="https://reviewable.io/review\_button.svg" height="34" align="absmiddle" alt="Reviewable"/\>\](https://reviewable.io/reviews/dgraph-io/dgraph/7694)

However, it was part of the 21.12 release and I don’t believe part of the new baseline in 23.1.0

Is there a current plan to support transactional mutations?

Thanks,  
Ryan