Transactions committing/aborting other transaction : Dgraph

Users Dgraph
1

I was uploading data to dgraph server and was committing after every 100 records (Java code provided below ) . During the same time, i tried to index a column which failed initially saying they were pending transactions but after 2 - 3 retries, it succeeded .

But due to this, my upload transaction failed giving the below message :

Exception in thread “main” io.dgraph.TxnFinishedException: Transaction has already been committed or discarded
at io.dgraph.AsyncTransaction.commit(AsyncTransaction.java:197)
at io.dgraph.Transaction.lambda$commit$3(Transaction.java:98)
at io.dgraph.ExceptionUtil.withExceptionUnwrapped(ExceptionUtil.java:27)
at io.dgraph.Transaction.commit(Transaction.java:96)
at com.dgraphj.client.App2.main(App2.java:102)

My upload transaction block is as follows :

Transaction mutationTxn = dgraphClient.newTransaction();
try {
for(double k=0;k<10000;k++){
// Serialize it
String json = SingleRowDataPush1.getData((int)(k+2));
//gson.toJson(b);

  // Run mutation
  Mutation mutation =
      Mutation.newBuilder().setSetJson(ByteString.copyFromUtf8(json.toString())).build();
  
  Response res = mutationTxn.mutate(mutation);
  
System.out.println(k+" "+res.getLatency());

if(k>99 && k%100==0){
  System.out.println("####################### commit"+k+"#################");

  mutationTxn.commit();
  mutationTxn = dgraphClient.newTransaction();

}
}

} finally {
	mutationTxn.commit();
	mutationTxn.discard();
}

}

I am not sure if to different transactions can commit or hinder other one which if being the case, then what should be done ?

  1. A possible solution : Restart from last check point or last commit .
  2. A wait time so that the transaction can resume itself

Am i missing something ?

PS : The data is just for testing.

2

There are currently two approaches i am evaluating :

  1. Using a mq for sending out transactions hence maintaining order
  2. Maintaining a transaction (distributed transaction) fault tolerant system which retries basis on the failure reason hence kind of a thread pool to submit these transactions and then retrying basis on the exception type
  3. For bulk upload, maintaining a smaller window of commit and defining a better strategy for last checkpoint revert and retry . PS: Cannot rely on using Bulk upload since we do not have explicit instructions on what will be the nature of data upload ( PURE INSERT or UPDATE+INSERT)
3

Hi Nishant,

Some pointers:
You could use the request class, which as a addMutation(Collection) option. This will go well with the kind of loop structure you have and you could have a tight code around opening a transaction, passing a collection (100 records each), and committing. The same collection will also help you with the tailing records that might not go into the commit if clause.

Request request = Request.newBuilder()
                              .addAllMutations(values)
                              .build()

The schematic of the ideal transaction is as below. Please note that there is no commit in the finally clause. Please try to follow this. I feel there is a bug in the code, and it will help you debug it out.

try {
    // …
    // Perform any number of queries and mutations
    // …
    // and finally …
    txn.commit()
} catch (TxnConflictException ex) {
    // Retry or handle exception.
} finally {
    // Clean up. Calling this after txn.commit() is a no-op
    // and hence safe.
    txn.discard();
}
1 Like
4

Will try this out to see which of my problems are getting solved .
Can you please help me with the following analogy "If a lot of transactions are submitted, then the transaction (among all submitted) on same key that will be committed will be chosen basis on the smallest timestamp

Assumption : We have n clients firing n transaction at the same time on the same key

PS : I asked this query also but waiting for inputs on this : Transaction related queries for conflicting transactions ( Coming from a Mysql Innodb Transaction mgmt environment)