Copying a list of linked objects in a predicate

miko · January 28, 2020, 4:52pm

Hello, I am trying to rename a predicate containing a list of links using upsert in JSON format. Below are my operations. I must be missing something, because new_projects predicate is empty, while I would expect it to be equal to projects for each Company node.

$ curl localhost:8080/alter -X POST -d $'
name: string @index(exact) .
id: string @index(exact) .
projects: [uid] .
new_projects: [uid] .
type Project {
name: string
}
type NewProject {
name: string
}
type Company {
name: string
projects: [Project]
new_projects: [NewProject]
}
'

$  curl -H "Content-Type: application/json" -X POST localhost:8080/mutate?commitNow=true -d '
  { "set":[
    {"uid":"_:p1","name":"Project 1","dgraph.type":"Project"},
    {"uid":"_:p2","name":"Project 2","dgraph.type":"Project"},
    {"uid":"_:p3","name":"Project 3","dgraph.type":"Project"},
    {"uid":"_:c1","name":"Company 1","dgraph.type":"Company"},
    {"uid":"_:c2","name":"Company 2","dgraph.type":"Company"},
    {"uid":"_:c1","projects":{"uid":"_:p1"}},
    {"uid":"_:c1","projects":{"uid":"_:p2"}},
    {"uid":"_:c2","projects":{"uid":"_:p3"}}
  ]}'

$  curl -H "Content-Type: application/json" -X POST localhost:8080/mutate?commitNow=true -d '
{
  "query": "{ v as var(func:type(Project) ) ",
  "set": {
    "uid": "uid(v)",
    "dgraph.type": "NewProject"
  }
}
'

$  curl -H "Content-Type: application/json" -X POST localhost:8080/mutate?commitNow=true -d '
{
  "query": "{ v as var(func:type(Company) ) {p as projects {uid}}}",
  "set": {
    "uid": "uid(v)",
    "new_projects": "val(p)"
  }
}
'

So, why this last mutation does not work? How to make it working?

BTW, using dgraph 1.2.0.

Regards,
miko

MichelDiz · January 28, 2020, 6:55pm

Maybe something like this

{
  "query": "{ v as var(func:type(Company) ) { p as projects } }",
  "set": {
    "uid": "uid(v)",
    "new_projects": [{ "uid": "uid(p)"}]
  }
}

miko · January 28, 2020, 10:21pm

Indeed, thanks! Maybe you can add something similar to the documentation?

miko · January 28, 2020, 10:45pm

OK, it is not exactly what I wanted. Now “new_projects” for each company contains every project.
For the following query:
{ node(func:type(Company)) { expand(_all_) { expand(_all_) } } }
I get now:

    "node": [
      {
        "name": "Company 1",
        "projects": [
          {
            "name": "Project 1"
          },
          {
            "name": "Project 2"
          }
        ],
        "new_projects": [
          {
            "name": "Project 1"
          },
          {
            "name": "Project 2"
          },
          {
            "name": "Project 3"
          }
        ]
      },
      {
        "name": "Company 2",
        "projects": [
          {
            "name": "Project 3"
          }
        ],
        "new_projects": [
          {
            "name": "Project 1"
          },
          {
            "name": "Project 2"
          },
          {
            "name": "Project 3"
          }
        ]
      }
    ]
  }

So how can I get the expected result (no “new_projects” equals “projects” for each company)?

miko · January 28, 2020, 11:29pm

One more example:

  curl -H "Content-Type: application/json" -X POST localhost:8080/mutate?commitNow=true -d '
{
  "query": "{ v as var(func:type(Company) ) {p as projects {uid} n as name}}",
  "set": {
    "uid": "uid(v)",
    "new_projects": "val(p)",
    "new_name": "val(n)"
  }
}

new_name gets set correctly, new_projects - does not. Is this unsupported for preditates of type [uid] ?

MichelDiz · January 29, 2020, 12:22am

The value in that variable is an uid map. The val() func doesn’t support convert the uid map to string. As far as I know.

What is the expected result?

It seemed simple and straightforward to me. You want to define that some projects are “new” and the “projects” edge just points to all projects (new or old). And “new_projects” for projects that you consider new. You need to define how to identify what’s new from what’s not. Maybe using Facets.

If you want to use edges to define what is new or not. Just remove it on one and add it on the other edge.

Cheers.

miko · January 29, 2020, 8:01am

What is the expected result?

As a process of data migration I want to rename “projects” predicate to “new_projects” in all nodes of type Company. I would delete old “projects” predicate as a last step.
Is there any simpler way to achieve my goal?

My broader goal was to check if it is possible to maintain the data migration using only curl / HTTP interface. So far I know that renaming predicates containing list of uids is not possible (as in “projects” case above), as well as deleting preticate from schema.

MichelDiz · January 29, 2020, 2:26pm

Like this?

I’m not sure if you can do this query bellow via HTTP API. Pls, let me know if it doesn’t.

{
  "query": "{ v as var(func:type(Company) ) { p as projects }",
  "set": {
    "uid": "uid(v)",
    "new_projects": [{ "uid": "uid(p)"}]
  },
  "delete": {
    "uid": "uid(v)",
    "projects": [{ "uid": "uid(p)"}] #This deletes the relations only
  },
  "delete": {
    "uid": "uid(p)" # This will clean the projects
  }
}

miko · January 29, 2020, 11:08pm

Thanks for another try, but it still does not work for me. Without delete, I get:

    "node": [
      {
        "name": "Company 1",
        "projects": [
          {"name": "Project 1"},{"name": "Project 2"}
        ],
        "new_projects": [
          {"name": "Project 3"},{"name": "Project 1"},{"name": "Project 2"}
        ]
      },
      {
        "name": "Company 2",
        "projects": [
          {"name": "Project 3"}
        ],
        "new_projects": [
          {"name": "Project 3"},{"name": "Project 1"},{"name": "Project 2"}
        ]
      }
    ]

I would expect “new_projects” to be equal to “projects” for each “Company”, but what I get now is concatenation of all projects of all companies, and this value (list of all projects) is set for each Company. So it does not work as a predicate rename.

Regards,
miko

MichelDiz · January 30, 2020, 1:50am

Hey, sorry but I really don’t understand what we are trying to achieve. All you say seems trivial to me, but the results you get are unlike what you desire. We may have some communication issues (from me especially this week). I believe we can solve this.

Let’s go back one step.

To be simple. No need to explain again what was said. Give me a “before and after” result example (JSONs). So I can “visualize” what you wanna do.

Renaming is a trivial operation using upsert block. If there is something blocking you. We have to find what is it.

Cheers.

miko · January 30, 2020, 8:45am

Hey, sorry but I really don’t understand what we are trying to achieve

OK, I should have made this clear. I want to rename “projects” to “new_projects” for each Company node.

Expected result is:
“Company 1” has “new_projects” predicate containing “Project 1” and “Project 2”.
“Company 2” has “new_projects” predicate containing “Project 3”.

Currently I get:
“Company 1” has “new_projects” predicate containing “Project 1”, “Project 2” and “Project 3”.
“Company 2” has “new_projects” predicate containing “Project 1”, “Project 2” and “Project 3”.

This seems trivial but I cannot get the expected result.

Regards,
miko

MichelDiz · January 30, 2020, 3:40pm

Okay, now I get what is going on.

The Bulk Upsert won’t work for you. We would need a kind of loop in grapqhl+- to be able to do it. But for now, you have to do this in your business logic.

This code bellow, you have to perform a mutation for each existing company. So in this case you gonna use pagination (first and offset) to make it work. So as you have two companies. You have to run

1 - var(func:type(Company), first:1, offset:0 )
2 - var(func:type(Company), first:1, offset:1 )

{
  "query": "{ v as var(func:type(Company), first:1, offset:0 ) { p as projects } }",
  "set": {
    "uid": "uid(v)",
    "new_projects": [{ "uid": "uid(p)"}]
  },
  "delete": {
    "uid": "uid(v)",
    "projects": [{ "uid": "uid(p)"}]
  }
}

However, if you don’t want to use pagination. If it gets too complex. I recommend the approach below using a temporary edge called “migrationdone”. At the end of the day, you can just clean up these edges.

So, if you have thousands of companies to do this operation. You have to run this thousand times. It will skip those who have “migrationdone”.

{
  "query": "{ v as var(func:type(Company), first:1 ) @filter(NOT has(migrationdone)) { p as projects } }",
  "set": {
    "uid": "uid(v)",
    "new_projects": [{ "uid": "uid(p)"}],
    "migrationdone": true
  },
  "delete": {
    "uid": "uid(v)",
    "projects": [{ "uid": "uid(p)"}]
  }
}

Let me know if that works for you.

PS. Maybe it can be done in Bulk Upsert using Conditional Upsert. Need to explore this idea.

Cheers.

system · February 29, 2020, 3:40pm

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.

Topic		Replies	Views
FOREACH func in DQL (loops in Bulk Upsert) Dev proposal , discussion	8	3349	July 9, 2020
Predicate in a Go struct and uid is now string? Users	4	839	January 14, 2018
Filters not working when using them with variables in an upsert Dgraph kind:question , dgraph	2	364	September 16, 2020
Modifying predicate lists Users	2	1597	June 27, 2018
Combine different predicates Dgraph	8	1794	May 24, 2018

Copying a list of linked objects in a predicate

Related Topics