How to handle N to 1 entity relationships

milosgajdos · February 17, 2021, 3:03pm

I Want to Do

I’d like to figure out how to handle/design N to 1 relationship in Dgraph

What I Did

I have the following two types of entities defined in the schema

	type Object {
		xid
		name
		namespace
		resource
		links
	}

	type Resource {
		xid
		name
		group
		version
		kind
		namespaced
	}

	xid: string @index(exact) .
	name: string @index(exact) .
	namespace: string @index(exact) .
	links: [uid] @count @reverse .
	created_at : datetime @index(hour) .
	group: string @index(exact) .
	version: string @index(exact) .
	kind: string @index(exact) .
	namespaced: bool .
	resource: uid .

Now, there can be several Objects pointing to the same Resource. By reading this comment I’ve learned that Dgraph does not support unique UIDs across types, which make me wonder about some questions:

Say I create 2 Objects which share the same Resource i.e. the Resource should be globally unique. I can guarantee the uniqueness on my side via xid field in the schema definition, but that still would not prevent Dgraph from creating new Resource nodes which leads to unnecessary data bloat and a bit tricky handling of R<->O relationship in terms of consistency i…e say I update a Resource with some XID (which I guarantee will be unique) – I now have to update all instances of Resource which have the same xid to make sure the update is consistent across all Objects that share the same Resource. I understand I can create a “map” type i.e. intermediary node but that would not solve the problem with large updates.

Is there any way to somehow deal with this situation in, say, more “performant” less redundant way or this is simply not possible?

Say I want to link two entities, but I don’t know what Dgraph Types they are. All I know is their XIDs (i.e. externallly managed unique IDs). Here I figured I can link to SAME Types of entities like this

	query := `
	{
		from as var(func: eq(xid, "` + from.Value() + `")) {
			fid as uid
		}

		to as var(func: eq(xid, "` + to.Value() + `")) {
			tid as uid
		}
	}
	`
	weight := 1.0
	relation := "Test"

	node := &Object{
		UID:   "uid(fid)",
		DType: []string{"Object"},
		Links: []Object{
			{UID: "uid(tid)", DType: []string{"Object"}, Relation: relation, Weight: weight},
		},
	}

	pb, err := json.Marshal(node)
	if err != nil {
		return fmt.Errorf("link json marshal: %w", err)
	}

	mu := &dgapi.Mutation{
		Cond:    `@if(gt(len(from), 0) AND gt(len(to), 0))`,
		SetJson: pb,
	}

	req := &dgapi.Request{
		Query:     query,
		Mutations: []*dgapi.Mutation{mu},
		CommitNow: true,
	}

	if _, err := s.c.NewTxn().Do(ctx, req); err != nil {
		return fmt.Errorf("txn Link: %w", err)
	}

I wonder how do I do that when I want to link Object and Resource if all I know are their XIDs. There are 3 possible combinations

O -> R
O -> O
R -> O

I suspect I could retrieve DType in a query like so:

query := `
	{
		from as var(func: eq(xid, "` + from.Value() + `")) {
			fid as uid
		}

		to as var(func: eq(xid, "` + to.Value() + `")) {
			tid as uid
		}
	}

       fromUid(func: uid(fid)) {
	  	  uid
		  xid
          dgraph.type
        }
	`

And then maybe based on the returned type I’d hack up Go code so the linking is correct as per the relationship type. This however feels a bit clunky and it seems to me I wouldn’t be able to do Upstert easily from Go code in this situation given the way Objects are linked (many to many) and how Object is linked to Resource (many to 1) i.e. for O<-> link something like this works:

node := &Object{
		UID:   "uid(fid)",
		DType: []string{"Object"},
		Links: []Object{
			{UID: "uid(tid)", DType: []string{"Object"}, Relation: relation, Weight: weight},
		},
	}

However, for O->R I’d need something like this:

node := &Object{
		UID:   "uid(fid)",
		DType: []string{"Object"},
		Resource: Resource{UID: "uid(tid)", DType: []string{"Resource"}}
	}

The problem is you can’t do “both” in an upsert. You’d need to query, get the results back, parse them, create the right mutation object (either of the above) based on the returned dgraph.type and do the mutation.

It seems to me that if the UIDs across Types were unique the linking between Resource and Object entities would be a piece of cake.

That being said, I’m quite new to Graph databases so I might be missing something fundamental here.

Dgraph Metadata

dgraph version

$ dgraph version
[Decoder]: Using assembly version of decoder
Page Size: 4096

Dgraph version   : v20.11.0
Dgraph codename  : tchalla
Dgraph SHA-256   : 8acb886b24556691d7d74929817a4ac7d9db76bb8b77de00f44650931a16b6ac
Commit SHA-1     : c4245ad55
Commit timestamp : 2020-12-16 15:55:40 +0530
Branch           : HEAD
Go version       : go1.15.5
jemalloc enabled : true

For Dgraph official documentation, visit https://dgraph.io/docs/.
For discussions about Dgraph     , visit http://discuss.dgraph.io.

Licensed variously under the Apache Public License 2.0 and Dgraph Community License.
Copyright 2015-2020 Dgraph Labs, Inc.

chewxy · February 19, 2021, 12:21am

About to pop off to the doctor’s but you could checkout Conditional Upserts, to see if they fit your use case.

https://dgraph.io/docs/mutations/conditional-upsert/

I’ll reply more if it doesn’t.

milosgajdos · February 19, 2021, 8:53am

Yeah, so last night I thought of that, too and I ended up doing something like this for Delete to maintain some sort of referential integrity (I couldn’t find any docs if DGraph provide support for that). In case it might help someone in the future here is a snippet of the code I use. The idea here is to avoid deleting R if there are any Os linked to it.

	q := `
		query Entity($xid: string) {
			entity(func: eq(xid, $xid)) {
				uid
			}
		 }
	`

	resp, err := s.c.NewTxn().QueryWithVars(ctx, q, map[string]string{"$xid": uid.Value()})
	if err != nil {
		return  mt.Errorf("query txn: %w", err)
	}

	var r struct {
		Results []struct {
			UID string `json:"uid,omitempty"`
		} `json:"entity"`
	}

	if err = json.Unmarshal(resp.Json, &r); err != nil {
		return fmt.Errorf("query result unmarshal: %w", err)
	}

	res := len(r.Results)

	switch {
	case res == 0:
		return nil
	case res > 1:
		// NOTE: this should never happen
		panic("ErrDuplicateNode")
	}

	node := map[string]string{"uid": r.Results[0].UID}
	pb, err := json.Marshal(e)
	if err != nil {
		return fmt.Errorf("mutation marshal: %w", err)
	}

	mu := &dgapi.Mutation{
		Cond:    `@if(not type(Resource) OR eq(count(~resource), 0))`,
		SetJson: pb,
	}

	...
    ...
}

The code for `Add`ition mutation looks similar-ish. These things are all doable indeed, but I'd love if they were simpler :-) 

Also, the referential integrity support in DGraph would go a long way, too. Unless I missed it in any docs.

milosgajdos · February 20, 2021, 11:22am

I was looking at this yesterday a bit and thinking I could maybe simplify it to something like this:

	q := `
	{
		node(func: eq(xid, "` + uid.Value() + `")) {
			u as uid
		}
	}
	`

	node := map[string]string{"uid": "uid(u)"}

	// Only delete Resource if it has no inbound edges
	cond := `@if(not type(Resource) OR eq(count(~resource), 0))`

	pb, err := json.Marshal(node)
	if err != nil {
		return nil, fmt.Errorf("json marshal: %w", err)
	}

	mu := &dgapi.Mutation{
		Cond:    cond,
		SetJson: pb,
	}

	req := &dgapi.Request{
		Query:     query,
		Mutations: []*dgapi.Mutation{mu},
		CommitNow: true,
	}

	_, err = s.c.NewTxn().Do(context.Background(), req)
	if err != nil {
		return fmt.Errorf("txn Delete: %w", err)
	}

But when I try to do this I get the following error:

2021/02/20 11:08:55 txn Delete: rpc error: code = Unknown desc = Some variables are defined but not used
Defined:[__dgraph__0 u]
Used:[__dgraph__0]

This, I assume, means that u does not seem to be used, however, it actually is?

	node := map[string]string{"uid": "uid(u)"}

	pb, err := json.Marshal(node)
	if err != nil {
		return nil, fmt.Errorf("json marshal: %w", err)
	}

I’m not quite sure what the problem is here, given, for example, the following works like a charm:

	query := `
	{
		resource(func: eq(xid, "` + uid.Value() + `")) {
			u as uid
	        }
	}
	`

	res := &Resource{
		UID:        "uid(u)",
	}

	pb, err := json.Marshal(res)
	if err != nil {
		return nil, fmt.Errorf("json marshal: %w", err)
	}

	mu := &dgapi.Mutation{
		Cond:    cond,
		SetJson: pb,
	}

	req := &dgapi.Request{
		Query:     query,
		Mutations: []*dgapi.Mutation{mu},
		CommitNow: true,
	}

	if _, err := s.c.NewTxn().Do(ctx, req); err != nil {
		return fmt.Errorf("txn Add: %w", err)
	}

This puzzles me a bit, but I’m sure I’m missing something fundamental here.

Topic		Replies	Views
Live Import Upsert using xid creates duplicates Dgraph	0	205	November 23, 2023
How to keep entity unique? Users	2	572	May 31, 2018
Migrate from Dgraph v1.0 - Migration Documentation	0	478	January 28, 2021
Migrate to Dgraph v1.1 - Howto Documentation	0	420	August 29, 2020
Understanding bulk data loads, and bulk updates, with XID in v0.8 Users	2	850	November 1, 2017

How to handle N to 1 entity relationships

I Want to Do

What I Did

Dgraph Metadata

Related Topics