Problem with lists in queries response

diggy · December 27, 2017, 4:45pm

Use case:

I build a mutation as following:

Mutation.newBuilder().setSetJson(ByteString.copyFromUtf8(json.toString())).build()

Where json.toString() is:

{
  "uri":"http://skos.um.es/unescothes/C00001",
  "term":"Abandoned children",
  "jurisdiction":["global1","global2"],
  "lastUpdate":"2017-12-27T16:36:02.861"
}

After that, I did a query to the database to retrieve the inserted object as following:

Response res = dgraphClient.newTransaction().queryWithVars(builder.build(), vars);

And res.getJson().toStringUtf8() was:

{“concepts”:[ {
“uri”:“http://skos.um.es/unescothes/C00001”,
“term”:“Abandoned children”,
“jurisdiction”:“global2”,
“lastUpdate”:“2017-12-27T16:36:02.861”
} ] }

As you can see, jurisdiction was inserted in the database as a list, and when you retrieve it from the database it comes as a String.

diggy · January 2, 2018, 9:55am

deepakjois commented :

I believe @pawanrawal was looking at fixing this on the server side.

diggy · January 2, 2018, 11:18pm

pawanrawal commented :

I have merged a fix for this on the server which would be part of the next release.

diggy · January 3, 2018, 3:54am

deepakjois commented :

Closing this for now. Please re-open if you continue to see this problem.

diggy · January 18, 2018, 2:13pm

gfsilva92 commented :

I continue to have the same behaviour using the version 1.1.0 of dgraph4j.

Is not supposed to be fixed?

diggy · January 22, 2018, 3:30am

deepakjois commented :

@gfsilva92 @pawanrawal was talking about a change in the server. Are you using the latest version of dgraph server?

diggy · January 22, 2018, 9:21am

gfsilva92 commented :

No. I will try with the last server version and give you feedback.

Thanks.

diggy · January 26, 2018, 11:46am

gfsilva92 commented :

I still with the error. I insert a list of strings and when I retrieve the response comes:

{“concepts”:[…,“jurisdiction”:“DZA”,…}]}

instead of

{“concepts”:[…,“jurisdiction”:[“DZA”],…}]}

diggy · January 28, 2018, 11:30pm

pawanrawal commented :

Could you please share the version of dgraph you are using and maybe a small reproducible example which illustrates the problem?

diggy · January 29, 2018, 5:52pm

gfsilva92 commented :

Dgraph server version: 1.1.0
Dgraph client version: 1.1.0

There are two different moments.

Creation moment, where I create a concept with a list of multiple jurisdictions:

String json = gson.toJson(concept);
Mutation mu = Mutation.newBuilder().setSetJson(ByteString.copyFromUtf8(json.toString())).build();
Assigned mutate = txn.mutate(mu);
txn.commit();

If you watch json variable you can see that you insert something like:
{
“uri”:“http://skos.um.es/unescothes/C00001”,
“term”:“Abandoned children”,
"jurisdiction":[“global1”,“global2”],
“lastUpdate”:“2017-12-27T16:36:02.861”
}

Retrieval moment, where I try to get the inserted concept. In this moment I execute a dgraph query to get that concept and return the jurisdiction field. The problem is that jurisdiction field always have only the last jurisdiction as a string instead of a list. This is what I get:

{
“uri”:“http://skos.um.es/unescothes/C00001”,
“term”:“Abandoned children”,
"jurisdiction":“global2”,
“lastUpdate”:“2017-12-27T16:36:02.861”
}

Hope it helps,
Gonçalo.

diggy · January 30, 2018, 12:27am

pawanrawal commented :

Tried reproducing but couldn’t. Please share a complete reproducible example if you have one. Here is a go test that I created and verified with v1.0.2.

package main

import (
	"context"
	"log"
	"testing"

	"google.golang.org/grpc"

	"github.com/dgraph-io/dgraph/client"
	"github.com/dgraph-io/dgraph/protos/api"
	"github.com/dgraph-io/dgraph/x"
	"github.com/stretchr/testify/require"
)

func TestList(t *testing.T) {
	d, err := grpc.Dial("localhost:9080", grpc.WithInsecure())
	if err != nil {
		log.Fatal(err)
	}

	c := client.NewDgraphClient(
		api.NewDgraphClient(d),
	)

	ctx := context.Background()
	x.Check(c.Alter(ctx, &api.Operation{DropAll: true}))
	x.Check(c.Alter(ctx, &api.Operation{Schema: `jurisdiction: [string] .`}))

	out := []byte(`{
		"uri":"http://skos.um.es/unescothes/C00001",
		"term":"Abandoned children",
		"jurisdiction":["global1","global2"],
		"lastUpdate":"2017-12-27T16:36:02.861"
	}`)
	uids, err := c.NewTxn().Mutate(context.Background(), &api.Mutation{SetJson: out, CommitNow: true})
	x.Check(err)
	uid := uids.Uids["blank-0"]

	q := `
	{
	  me(func: uid(` + uid + `)) {
	    uri
	    term
	    jurisdiction
	    lastUpdate
	  }
	}
	`

	resp, err := c.NewTxn().Query(ctx, q)
	x.Check(err)
	require.JSONEq(t, `{"me":[{"uri":"http://skos.um.es/unescothes/C00001","term":"Abandoned children","jurisdiction":["global2","global1"],"lastUpdate":"2017-12-27T16:36:02.861"}]}`, string(resp.Json))
}

diggy · January 30, 2018, 11:25am

gfsilva92 commented :

Is difficult to give you a reproducible example. I could do a unit test but we are using Mockito so behaviours as database retrievals are simulated.

But I’ve been doing some tests and I found out that the problem is not when getting what was inserted but on the insert. Maybe I’m doing something wrong.

I have this code to create a concept:
`Transaction txn = dgraphClient.newTransaction();
try {
String json = gson.toJson(concept);
LOGGER.info(“json to insert: \n” + json);
Mutation mu = Mutation.newBuilder().setSetJson(ByteString.copyFromUtf8(json.toString())).build();

Assigned mutate = txn.mutate(mu);
txn.commit();

String databaseId = (String) mutate.getUidsMap().values().toArray()[0];
dbNodeIds.put(concept.getId(), databaseId);
} catch (Exception e) {
LOGGER.error("An error occur when trying to create the concept " + concept.getId(), e);
} finally {
txn.discard();
}`

When trying to insert, the LOGGER.info shows the following:
2018-01-30 11:19:34 INFO DGraphConnector:74 - json to insert: {"id":"c3acc43c-c635-475d-8bed-7c3b4f2af37d","type":"concept","term":"myTerm1","jurisdiction":["jur1","jur2"],"definition":"myDef1","lastUpdate":"2018-01-30T11:19:34.150"}

After that, I went to Dgraph Ratel Dashboard and I did the following query:

{ concepts(func:eq(id, "54088896-d9e8-4cf5-90f3-9c41ff342c62"))@recurse (depth: 2){ term definition jurisdiction } }

And the query returned the following json:

{ "data": { "concepts": [ { "term": "myTerm1", "definition": "myDef1", "jurisdiction": "jur2" } ] }, "extensions": { "server_latency": { "processing_ns": 1003600 }, "txn": { "start_ts": 564, "lin_read": { "ids": { "1": 556 } } } } }

So the question is, Im I doing something wrong on the mutation to insert that content? Or is a bug from you guys?

Thanks a lot,
Gonçalo.

diggy · February 1, 2018, 1:23am

pawanrawal commented :

I strongly suspect that you don’t have the correct schema. jurisdiction is not of list type in your schema. Can you share the output of doing the schema {} query? Here is some more code to illustrate that this works.

package main

import (
	"context"
	"fmt"
	"log"
	"testing"

	"google.golang.org/grpc"

	"github.com/dgraph-io/dgraph/client"
	"github.com/dgraph-io/dgraph/protos/api"
	"github.com/dgraph-io/dgraph/x"
)

func TestList(t *testing.T) {
	d, err := grpc.Dial("localhost:9080", grpc.WithInsecure())
	if err != nil {
		log.Fatal(err)
	}

	c := client.NewDgraphClient(
		api.NewDgraphClient(d),
	)

	ctx := context.Background()
	x.Check(c.Alter(ctx, &api.Operation{DropAll: true}))
	x.Check(c.Alter(ctx, &api.Operation{Schema: `
		jurisdiction: [string] .
		id: string @index(hash) .
	`}))

	out := []byte(`{"id":"c3acc43c-c635-475d-8bed-7c3b4f2af37d","type":"concept","term":"myTerm1","jurisdiction":["jur1","jur2"],"definition":"myDef1","lastUpdate":"2018-01-30T11:19:34.150"}`)
	_, err = c.NewTxn().Mutate(context.Background(), &api.Mutation{SetJson: out, CommitNow: true})
	x.Check(err)

	q := `
	{
		concepts(func:eq(id, "c3acc43c-c635-475d-8bed-7c3b4f2af37d"))@recurse (depth: 2){
			term
			definition
			jurisdiction
		}
	}
	`

	resp, err := c.NewTxn().Query(ctx, q)
	x.Check(err)
	fmt.Println(string(resp.Json))
}

diggy · February 1, 2018, 3:26pm

gfsilva92 commented :

I’m using io.leangen.graphql.spqr maven dependency to build the schema. Maybe that’s the problem? Before I had an explicit schema, but in order to not have to update schema and pojos everytime something changed I started using the maven dependency.

diggy · February 2, 2018, 7:45am

pawanrawal commented :

I am not sure about the dependency that you are using. Could you check them schema by querying for schema{} in Ratel dashboard or using the client?

diggy · February 2, 2018, 2:46pm

gfsilva92 commented :

You are right. The jurisdiction is defined as string…

Is there a way to force it to list? For example with Operation.newBuilder().setSchema(schema)?

diggy · February 5, 2018, 3:08am

pawanrawal commented :

Yeah, there is. You set the schema as jurisdiction: [string] .

diggy · February 5, 2018, 10:59am

gfsilva92 commented :

Thanks, this worked! Problem solved.

Topic		Replies	Views
Schema query not working with .getJson() Dgraph Clients untagged , dgraph-js	3	440	July 11, 2020
Weird bug when trying to store an array with repeated values Dgraph Cloud / Slash GraphQL kind:bug	3	719	October 26, 2021
String array out of order Dgraph kind:question , dgraph	3	892	September 16, 2020
Basic Schema, Data and Query Problem Dgraph	3	382	November 1, 2019
Support lists in query variables (Dgraph's GraphQL Variable) Dgraph dgraph , status:accepted , kind:feature , area:querylang , area:querylang:vars	18	1876	May 20, 2022

Problem with lists in queries response

Related Topics