Using assigning uids - what benefits are there?

makitka · 2018-12-24 11:42:02 UTC

i am inserting data into dgraph in batches of 5000-10000 mutations, where ~2000 of them are blank nodes.
does it make sense to use external assigning uids in this case (/assign?what=uids&num=N)? what benefit will i get here?

did anyone measure memory/cpu consumption or insert rate, how it changes when you replace blank nodes by uids given from zero?

MichelDiz · 2018-12-24 15:34:12 UTC

The only benefit of this is not to run into UID allocation errors. It is safer to allocate UIDs than to force creation, this would generate an error at the moment of the mutation.

makitka · 2018-12-24 21:23:12 UTC

what errors are you talking about? how they can happen?

dmai · 2018-12-24 21:57:36 UTC

It’s an error to use any UID literals that have not yet been leased out from Zero. You would see this error message:

github.com

dgraph-io/dgraph/blob/b843e0479329e0c4d52dee2f9ac5ed1d4eea38bc/query/mutation.go#L118


func verifyUid(ctx context.Context, uid uint64) error {
	if uid <= worker.MaxLeaseId() {
		return nil
	}
	// Even though the uid is above the max lease id, it might just be because
	// the membership state has fallen behind. Update the state and try again.
	if err := worker.UpdateMembershipState(ctx); err != nil {
		return x.Wrapf(err, "updating error state")
	}
	if lease := worker.MaxLeaseId(); uid > lease {
		return x.Errorf("Uid: [%d] cannot be greater than lease: [%d]", uid, lease)
	}
	return nil
}


// AssignUids tries to assign unique ids to each identity in the subjects and objects in the
// format of _:xxx. An identity, e.g. _:a, will only be assigned one uid regardless how many times
// it shows up in the subjects or objects
func AssignUids(ctx context.Context, nquads []*api.NQuad) (map[string]uint64, error) {
	newUids := make(map[string]uint64)
	num := &pb.Num{}

You can see the max leased uid value by looking at the maxLeaseId field in the /state page for Zero.

dmai · 2018-12-24 22:05:19 UTC

If you need to create a new node and re-use its UID across different mutations during your batch, you can use the /assign endpoints for IDs to guarantee a range of new IDs that you can use across your mutations for the same node. Blank nodes across mutations will result in new nodes.

makitka · 2018-12-25 10:03:35 UTC

thank you for an answer.
so, if i understand right, there is no any sense (memory use, speed) to replace blank nodes with assigned uids in that manner. if i have different mutations, i can get newly created uids as result from first mutation and reuse them in a second one.

the only one reason is if i want to send several mutations in parallel using the same newly created nodes