Upsert with multiple UIDs

Thanks for the help. You were right about the need to remove those first two lines.
Now when I run the upsert, it throws the exception:
Some variables are used but not defined

What does that mean?
I’m wondering if there’s some sort of conflict between:
productsUid : uid
and productsUid as uid in the two parts of my query; however, in the bottom part of the query (the getVals part), I’m not sure how to get the UID of the products node in a way that will allow me to get it back out of the query in a uidMap. (I need to pass the UIDs to another function that will perform additional Dgraph operations after completing the upsert.)

Regarding @normalize, here’s what I get with it:

And, here’s what I get without it:

You can see that the output is identical and is not flat in either.

One question, why your query is like that and not like this?

{
  getVals(func: has(products)) @normalize {
    productsUid : uid
  	products @filter(eq(productId, 19610626)) {
      productUid : uid
      options @filter(eq(optionId, 32661491)) {
        optionUid : uid
      }
    }
  }
}

It must have a list of defined variables and used but undefined.

UPDATE:

 Map<String, String> map = new HashMap<String, String>();
            map.put("$productID", inputMessage.getProductId().toString());
            map.put("$optionID", inputMessage.getOptionId().toString());

            for (Map.Entry<String, String> entry : map.entrySet()) {
                query = query.replace(entry.getKey(), entry.getValue());
            }

I don’t know Java, but I think something is wrong here. Why are you mapping this and doing a loop?

I doesn’t looks like this GitHub - dgraph-io/dgraph4j: Official Dgraph Java client

Also, you don’t need to use normalize in an upsert block. There’s no reason for this. You will not receive responses from this transaction. Only logs as response.

(I thought this part was about issue with normalize, ignore it) Yeah, it is. When you use aliases with equal naming you gonna have bad time, as you have a single root entity. Set each one unique for a moment. It flats but also try to “smash”.

But actually it should return a list tho. e.g:

 {
        "productsUid": [
          "0x4b928",
          "0x29f",
          "0x26cbf",
          "0x9d358",
          "0xe"
        ]
      }

It works fine in https://play.dgraph.io

This query:

{
  TEST as var(func: uid("0x4b928") )
  
  director(func: uid(TEST)) @normalize  {

    productsUid : uid
    director.film {
      productsUid : uid
      initial_release_date
      starring(first: 2) {
        performance.actor {
          productsUid : uid
        }
        performance.character {
          productsUid : uid
        }
      }
      country {
        productsUid : uid
      }
    }
  }
}
@filter (eq (optionId, $ optionID))

Now I see that you are trying to use GraphQL Variables with Upsert Block. Unfortunately, this is not currently supported. I realize that you’re troubleshooting the issue by testing with various queries.

There are several factors involved here in this case.

  1. GraphQL Variables with Upsert Block is not supported.
  2. Normalize has no effect within an upsert transaction.
  3. Two blocks would indeed be the right way to use GraphQL Variables with Upsert Block, but it is not supported at this time. But you are on the right track, but you should use Value Variables instead. In fact, we haven’t even defined what this procedure would be like yet.
  4. You are using aliases as if they were variables. This does not work in Dgraph. But I believe you are doing this to try injecting through GraphQL Variables. Which wouldn’t work.

Please, feel free to open an issue requesting the GraphQL Variables with Upsert Block as a feature/enhancement.

Cheers.

1 Like

Regarding:

Map<String, String> map = new HashMap<String, String>();
            map.put("$productID", inputMessage.getProductId().toString());
            map.put("$optionID", inputMessage.getOptionId().toString());

            for (Map.Entry<String, String> entry : map.entrySet()) {
                query = query.replace(entry.getKey(), entry.getValue());
            }

This is just a compact way to replace the placeholders (e.g. $productID and $optionID) with actual values.
The loop is actually replacing the string for each value.
It’s equivalent to replacing the string with itself after substituting $productID for an actual value and then repeating for $optionID.
In Go, it would be equivalent to this:
myText = strings.Replace(myText, "$productID", "28734602", -1)
myText = strings.Replace(myText, "$optionID", "327865", -1)

Since I’m replacing the variables with specific values in the string before executing the query, I don’t think I’m using GraphQL variables because the $ character won’t actually appear in the string when it’s submitted to Dgraph.

With this in mind, the code would look like this at runtime:

String query =      "    {\n" +
                    "        var(func: has(products)){\n" +
                    "            productsUid as uid\n" +
                    "            productNode as products @filter(eq(productId, \"28734602\")){\n" +
                    "                optionNode as options @filter(eq(optionId, \"327865\")){\n" +
                    "                uid\n" +
                    "                }\n" +
                    "            }\n" +
                    "        }\n" +
                    "        getVals(func: uid(productsUid)) @normalize {\n" +
                    "            productsUid : uid\n" +
                    "                products @filter(uid(productNode)){\n" +
                    "                    productUid : uid\n" +
                    "                    options @filter(uid(optionNode)){\n" +
                    "                        optionUid : uid\n" +
                    "                    }\n" +
                    "                }\n" +
                    "            }\n" +
                    "        }";

            DgraphProto.Mutation mu =
                    DgraphProto.Mutation.newBuilder()
                            .setSetNquads(ByteString.copyFromUtf8("uid(productsUid) <products> uid(productUid) ."))
                            .setSetNquads(ByteString.copyFromUtf8("uid(productsUid) <dgraph.type> \"Products\" ."))
                            .setSetNquads(ByteString.copyFromUtf8("uid(productUid) <productId> \"" + inputMessage.getProductId().toString() + "\" ."))
                            .setSetNquads(ByteString.copyFromUtf8("uid(productUid) <options> uid(optionUid) ."))
                            .setSetNquads(ByteString.copyFromUtf8("uid(productUid) <dgraph.type> \"Product\" ."))
                            .setSetNquads(ByteString.copyFromUtf8("uid(optionUid) <dgraph.type> \"Option\" ."))
                            .setSetNquads(ByteString.copyFromUtf8("uid(optionUid) <color> \"" + inputMessage.getColor().toString() + "\" ."))
                            .setSetNquads(ByteString.copyFromUtf8("uid(optionUid) <optionId> \"" + inputMessage.getOptionId().toString() + "\" ."))
                            .build();

Hmm, okay. This syntax is very similar to graphql variables. But I’m not sure about this code either. Don’t feels like a thing that we do in the clients. Why not just put the values rather than doing a loop to do so? It may have a reason, but it is odd to me.

But what about the other point?

  1. normalize - no need.
  2. two blocks - no need.
  3. aliases- no need.

The reason that I can’t just put the values directly in the code is because they’re being provided by an incoming message. (This is a stream function using Apache Pulsar.) I’m sure the code to substitute the values is working fine. It’s been sufficiently tested. I think I actually got the idea of using a map to replace the values in one of the Dgraph repos, but I don’t remember exactly where.

Regarding:

{
  getVals(func: has(products)) {
    productsUid : uid
  	products @filter(eq(productId, 19610626)) {
      productUid : uid
      options @filter(eq(optionId, 32661491)) {
        optionUid : uid
      }
    }
  }
}

I like this approach, but when I use it in the upsert, I get the exception:

io.grpc.StatusRuntimeException: UNKNOWN: while parsing query: “{\n getVals(func: has(products)) {\n productsUid : uid\n \tproducts @filter(eq(productId, 19610626)) {\n productUid : uid\n options @filter(eq(optionId, 32661491)) {\n optionUid : uid\n }\n }\n }\n}”: Some variables are used but not defined

As you’re using the Upsert Block. It means you’re using the variables in the mutation part of that block. e.g: .setSetNquads(ByteString.copyFromUtf8("uid(productsUid) <products> uid(productUid) ."))
So you need to define those variables in the query tho. Tha variables are “productsUid, productUid, optionUid”. That’s why you got this error. So you query should look like:

{
  getVals(func: has(products)) {
    productsUid as uid
  	products @filter(eq(productId, 19610626)) {
      productUid  as uid
      options @filter(eq(optionId, 32661491)) {
        optionUid as uid
      }
    }
  }
}
1 Like

When I use:

{
  getVals(func: has(products)) {
    productsUid as uid
  	products @filter(eq(productId, 19610626)) {
      productUid  as uid
      options @filter(eq(optionId, 32661491)) {
        optionUid as uid
      }
    }
  }
}

I still get:

io.grpc.StatusRuntimeException: UNKNOWN: while parsing query: “{\n getVals(func: has(products)) {\n productsUid as uid\n \tproducts @filter(eq(productId, 19610626)) {\n productUid as uid\n options @filter(eq(optionId, 32661491)) {\n optionUid as uid\n }\n }\n }\n}”: Some variables are defined but not used

Actually, the error is slightly different because it’s saying “Some variables are defined but not used” rather than “Some variables are used but not defined”.

Can you share an updated Gist from this context?

I’m not sure how to setup a Gist for a Dgraph query, but you can reproduce it with these steps:

First, run this alter on the schema:

type Products { 
    products: [Product] 
} 
type Product { 
    productId: string 
    options: [Option] 
} 
type Option { 
    optionId: string 
    color: string 
}
<collectionId>: int @index(int) .
<color>: string .
<optionId>: int @index(int) .
<options>: [uid] .
<productId>: int @index(int) .
<products>: [uid] .

Then, run this mutate:

{
  "set":[ {
    "uid": "_:products",
    "dgraph.type": "Products",
    "collectionId": 1,
    "products": [
      {
        "dgraph.type": "Product",
        "uid": "_:product",
        "productId": 19610626,
        "options": [
          {
            "dgraph.type": "Option",
            "uid": "_:option",
            "optionId": 32661491,
            "color": "red"
          }
        ]
      }
    ]
}]
}

Then, run this Java code:

String query = "{\n" +
            "  getVals(func: has(products)) {\n" +
            "    productsUid as uid\n" +
            "    products @filter(eq(productId, 19610626)) {\n" +
            "      productUid  as uid\n" +
            "      options @filter(eq(optionId, 32661491)) {\n" +
            "        optionUid as uid\n" +
            "      }\n" +
            "    }\n" +
            "  }\n" +
            "}";

DgraphProto.Mutation mu =
            DgraphProto.Mutation.newBuilder()
                    .setSetNquads(ByteString.copyFromUtf8("uid(productsUid) <products> uid(productUid) ."))
                    .setSetNquads(ByteString.copyFromUtf8("uid(productsUid) <dgraph.type> \"Products\" ."))
                    .setSetNquads(ByteString.copyFromUtf8("uid(productUid) <productId> \"19610626\" ."))
                    .setSetNquads(ByteString.copyFromUtf8("uid(productUid) <options> uid(optionUid) ."))
                    .setSetNquads(ByteString.copyFromUtf8("uid(productUid) <dgraph.type> \"Product\" ."))
                    .setSetNquads(ByteString.copyFromUtf8("uid(optionUid) <color> \"blue\" ."))
                    .setSetNquads(ByteString.copyFromUtf8("uid(optionUid) <dgraph.type> \"Option\" ."))
                    .setSetNquads(ByteString.copyFromUtf8("uid(optionUid) <optionId> \"32661491\" ."))
                    .build();
    Map uidsMap;
    try(DgraphConnection dgraphConnection = DGraphQueryHelper.createDgraphClient(false, context)){
        Transaction txn = dgraphConnection.getDgraphClient().newTransaction();
        try{
            DgraphProto.Request request = DgraphProto.Request.newBuilder()
                    .setQuery(query)
                    .addMutations(mu)
                    .setCommitNow(true)
                    .build();
            DgraphProto.Response res = txn.doRequest(request);
            uidsMap = res.getUidsMap();
            String exactOutput = res.getJson().toStringUtf8();
        } catch (Exception e){
            throw e;
        }
        finally {
            txn.discard();
        }

    } catch (Exception e){
        logger.error("The message on the exception is: " + e.getMessage());
        StringWriter sw = new StringWriter();
        PrintWriter pw = new PrintWriter(sw);
        e.printStackTrace(pw);
        logger.error("Showing stack trace: " + sw.toString());
        throw e;
    }

The spacing got messed up a little when I pasted the code, but I don’t think it has any syntax errors.

I meant a github gist like so upsert test · GitHub
So, this upsert block is working great. No errors. It might mean that some typo in your Java Code is happening. As I don’t know Java I can’t tell what is it.

1 Like

Could there be a bug in the Java Dgraph client?

I just noticed that there was more information in the exception that was thrown. It stated:

io.grpc.StatusRuntimeException: UNKNOWN: while parsing query: . . . Some variables are defined but not used
Defined:[optionUid productUid productsUid]
Used:[optionUid]

Why would it be saying that only optionUid was used? That seems odd to me.
I wonder if @amanmangal would have more ideas about this because he’s one of the Java client maintainers.

You should only call .setSetNquads once in the Mutation builder. As you currently have it, only the last setSetNquads call is ultimately applied, which is why the error says only optionUid is used.

2 Likes

That’s the issue! Thanks for the help!

Now, is there a way for me to be able to get the UIDs back out of the upsert? Or, will I need to run a second query to do that?
(I’m referring to this line: uidsMap = res.getUidsMap(), which is currently empty.)

it is empty cuz, for now, there’s no response for existent entities. Only returns UIDs when you create new entities/nodes.

1 Like

You guys are awesome. Thanks for the help.

2 Likes

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.