Live loader - Option to skip unauthorized predicates

Users Dgraph Cloud
#1

I am migrating from one shared to another shared instance of DGraph Cloud using the live loader as documented here https://dgraph.io/docs/cloud/admin/import-export/#importing-data-with-live-loader

I did create an export (json) using the DGraph Cloud interface and downloaded it to mu computer, but when using the live loader – towards the end I get multiple errors which stop the process from completing

Error while mutating: unauthorized to mutate acl predicates: dgraph.xid
Error while mutating: unauthorized to mutate acl predicates: dgraph.user.group
Error while mutating: unauthorized to mutate acl predicates: dgraph.password

I didn’t find an option in the live loader documentation how to skip certain predicates, so my solution was to ungzip the JSON , remove those predicates, rezip it and run the live loader

Am I missing something?

P.S. the API key I use for importing is of type Admin

Potential Sharing Issue - query sometimes empty sometimes returns data
#2

Yeah, I know. There was a request to remove those predicates from export and exibing it in the schema. What version are you using?

#3

export from v21.03.0-78-ge4ad0b113 import into v21.03.0-78-ge4ad0b113 used dgraph live from docker image dgraph/dgraph:v21.03-slash

#4
1 Like
#5

@jcsrb can you share the command you are using to live load?

#6 
docker run -it --rm -v /Users/jakob/work/dgraph/g01.json.gz:/tmp/g01.json.gz dgraph/dgraph:v21.03-slash \
  dgraph live --slash_grpc_endpoint=<server>.grpc.eu-central-1.aws.cloud.dgraph.io:443 -f /tmp/g01.json.gz -t <api-key>
1 Like
#7

I see. It seems that this procedure is on the right track. However, the schema and RDF appear to have namespaced data. Can you check? Below is the example.

My conclusion is that since it is an instance that has ACL enabled, it imagines that it should export the ACL context along. It seems to be a lack of foresight of the situation. It shouldn’t export with Namespaces. We may have to add an option to disable(filter out) the ACL context when it is Shared instances.

So, In order that your load was completed successfully. You should log in as groot. But it is not possible as it is not a dedicated instance.

Schema

[0x0] <dgraph.rule.predicate>:string @index(exact) @upsert . 
[0x0] <dgraph.graphql.p_query>:string @index(sha256) . 
[0x0] <dgraph.rule.permission>:int . 
[0x0] type <Review> {
	Review.about
	Review.by
	Review.comment
	Review.rating
}

RDF

<0x1> <dgraph.xid> "guardians"^^<xs:string> <0x0> .
<0x2> <dgraph.user.group> <0x1> <0x0> .
<0x8> <dgraph.user.group> <0x1> <0x0> .
<0xb> <dgraph.rule.predicate> "Review.about"^^<xs:string> <0x0> .
<0xc> <dgraph.rule.predicate> "Review.by"^^<xs:string> <0x0> .
<0xd> <dgraph.rule.predicate> "Review.rating"^^<xs:string> <0x0> .
<0x1> <dgraph.type> "dgraph.type.Group"^^<xs:string> <0x0> .
<0x2> <dgraph.type> "dgraph.type.User"^^<xs:string> <0x0> .
<0x1> <dgraph.acl.rule> <0xb> <0x0> .
<0x1> <dgraph.acl.rule> <0xc> <0x0> .
<0x1> <dgraph.acl.rule> <0xd> <0x0> .
<0x1> <dgraph.acl.rule> <0xe> <0x0> .
<0x1> <dgraph.acl.rule> <0xf> <0x0> .
<0x2> <dgraph.password> "$2a$10$KE4j5XZhjQ/Ojj8DgToDQT32vAC"^^<xs:password> <0x0> .
<0x8> <dgraph.password> "$2a$10XiIhDnttPde8KfHJieONqDMOg3v2"^^<xs:password> <0x0> .
<0x18> <dgraph.rule.permission> "7"^^<xs:int> <0x0> .

New issue

#8

In the exported schema, I have the following dgraph related lines

[0x57e99] <dgraph.xid>:string @index(exact) @upsert . 
[0x57e99] <dgraph.type>:[string] @index(exact) . 
[0x57e99] <dgraph.drop.op>:string . 
[0x57e99] <dgraph.acl.rule>:[uid] . 
[0x57e99] <dgraph.password>:password . 
[0x57e99] <dgraph.user.group>:[uid] @reverse . 
[0x57e99] <dgraph.graphql.xid>:string @index(exact) @upsert . 
[0x57e99] <dgraph.graphql.schema>:string . 
[0x57e99] <dgraph.rule.predicate>:string @index(exact) @upsert . 
[0x57e99] <dgraph.graphql.p_query>:string @index(sha256) . 
[0x57e99] <dgraph.rule.permission>:int . 

[0x57e99] type <dgraph.graphql> {
	dgraph.graphql.schema
	dgraph.graphql.xid
}
[0x57e99] type <dgraph.type.Rule> {
	dgraph.rule.predicate
	dgraph.rule.permission
}
[0x57e99] type <dgraph.type.User> {
	dgraph.xid
	dgraph.password
	dgraph.user.group
}
[0x57e99] type <dgraph.type.Group> {
	dgraph.xid
	dgraph.acl.rule
}
[0x57e99] type <dgraph.graphql.persisted_query> {
	dgraph.graphql.p_query
}

in the RDF I don’t hav eany dgraph.acl.rule
but dgraph.user.group and dgraph.password and dgraph.xid

<0x1a48a00750> <dgraph.xid> "guardians"^^<xs:string> <0x57e99> .
<0x1a48a00751> <dgraph.xid> "groot"^^<xs:string> <0x57e99> .
<0x1a48a00751> <dgraph.user.group> <0x1a48a00750> <0x57e99> .
<0x1a48a00751> <dgraph.password> "encrypted password"^^<xs:password> <0x57e99> .
#9

Yeah, this is expected. Theory hits. We should export without ACL. The value [0x57e99] represents your namespace on the shared cluster.

1 Like