jcsrb
(Jakob)
November 1, 2022, 10:44am
1
I am migrating from one shared to another shared instance of DGraph Cloud using the live loader as documented here https://dgraph.io/docs/cloud/admin/import-export/#importing-data-with-live-loader
I did create an export (json) using the DGraph Cloud interface and downloaded it to mu computer, but when using the live loader – towards the end I get multiple errors which stop the process from completing
Error while mutating: unauthorized to mutate acl predicates: dgraph.xid
Error while mutating: unauthorized to mutate acl predicates: dgraph.user.group
Error while mutating: unauthorized to mutate acl predicates: dgraph.password
I didn’t find an option in the live loader documentation how to skip certain predicates, so my solution was to ungzip the JSON , remove those predicates, rezip it and run the live loader
Am I missing something?
P.S. the API key I use for importing is of type Admin
MichelDiz
(Michel Diz)
November 1, 2022, 3:06pm
2
Yeah, I know. There was a request to remove those predicates from export and exibing it in the schema. What version are you using?
jcsrb
(Jakob)
November 1, 2022, 3:12pm
3
export from v21.03.0-78-ge4ad0b113
import into v21.03.0-78-ge4ad0b113
used dgraph live from docker image dgraph/dgraph:v21.03-slash
MichelDiz
(Michel Diz)
November 1, 2022, 9:25pm
5
@jcsrb can you share the command you are using to live load?
jcsrb
(Jakob)
November 1, 2022, 9:31pm
6
docker run -it --rm -v /Users/jakob/work/dgraph/g01.json.gz:/tmp/g01.json.gz dgraph/dgraph:v21.03-slash \
dgraph live --slash_grpc_endpoint=<server>.grpc.eu-central-1.aws.cloud.dgraph.io:443 -f /tmp/g01.json.gz -t <api-key>
1 Like
MichelDiz
(Michel Diz)
November 1, 2022, 9:40pm
7
I see. It seems that this procedure is on the right track. However, the schema and RDF appear to have namespaced data. Can you check? Below is the example.
My conclusion is that since it is an instance that has ACL enabled, it imagines that it should export the ACL context along. It seems to be a lack of foresight of the situation. It shouldn’t export with Namespaces. We may have to add an option to disable(filter out) the ACL context when it is Shared instances.
So, In order that your load was completed successfully. You should log in as groot. But it is not possible as it is not a dedicated instance.
Schema
[0x0] <dgraph.rule.predicate>:string @index(exact) @upsert .
[0x0] <dgraph.graphql.p_query>:string @index(sha256) .
[0x0] <dgraph.rule.permission>:int .
[0x0] type <Review> {
Review.about
Review.by
Review.comment
Review.rating
}
RDF
<0x1> <dgraph.xid> "guardians"^^<xs:string> <0x0> .
<0x2> <dgraph.user.group> <0x1> <0x0> .
<0x8> <dgraph.user.group> <0x1> <0x0> .
<0xb> <dgraph.rule.predicate> "Review.about"^^<xs:string> <0x0> .
<0xc> <dgraph.rule.predicate> "Review.by"^^<xs:string> <0x0> .
<0xd> <dgraph.rule.predicate> "Review.rating"^^<xs:string> <0x0> .
<0x1> <dgraph.type> "dgraph.type.Group"^^<xs:string> <0x0> .
<0x2> <dgraph.type> "dgraph.type.User"^^<xs:string> <0x0> .
<0x1> <dgraph.acl.rule> <0xb> <0x0> .
<0x1> <dgraph.acl.rule> <0xc> <0x0> .
<0x1> <dgraph.acl.rule> <0xd> <0x0> .
<0x1> <dgraph.acl.rule> <0xe> <0x0> .
<0x1> <dgraph.acl.rule> <0xf> <0x0> .
<0x2> <dgraph.password> "$2a$10$KE4j5XZhjQ/Ojj8DgToDQT32vAC"^^<xs:password> <0x0> .
<0x8> <dgraph.password> "$2a$10XiIhDnttPde8KfHJieONqDMOg3v2"^^<xs:password> <0x0> .
<0x18> <dgraph.rule.permission> "7"^^<xs:int> <0x0> .
New issue
opened 09:45PM - 01 Nov 22 UTC
kind/bug
area/enterprise
area/import-export
area/enterprise/acl
dgraph
dgraph/cloud
### What version of Dgraph are you using?
latest
### Tell us a little more… about your go-environment?
N/A
### Have you tried reproducing the issue with the latest release?
_No response_
### What is the hardware spec (RAM, CPU, OS)?
N/A
### What steps will reproduce the bug?
Create a shared instance in the Cloud.
Then add data. Then export them.
Then open RDF and Schema in a text editor and you will see that the data comes with namespace context.
### Expected behavior and actual result.
The expected result was that Dgraph exported the Schema and RDF cleanly. No ACL context.
In short, as it is a shared instance. The namespace context is irrelevant. Because if the user is exporting it, he will certainly load it into a new instance. And if it loads into a new shared instance, it can't choose which namespace it will have. Because he is not general admin.
### We have two options.
Or add a way to ignore namespaces (and predicates for namespaces during mutation) in liveload and bulkload.
Or do not export any namespace and ACL data. Have an option to export clean.
### Additional information
https://discuss.dgraph.io/t/live-loader-option-to-skip-unauthorized-predicates/17929/7?u=micheldiz
jcsrb
(Jakob)
November 1, 2022, 9:55pm
8
In the exported schema, I have the following dgraph related lines
[0x57e99] <dgraph.xid>:string @index(exact) @upsert .
[0x57e99] <dgraph.type>:[string] @index(exact) .
[0x57e99] <dgraph.drop.op>:string .
[0x57e99] <dgraph.acl.rule>:[uid] .
[0x57e99] <dgraph.password>:password .
[0x57e99] <dgraph.user.group>:[uid] @reverse .
[0x57e99] <dgraph.graphql.xid>:string @index(exact) @upsert .
[0x57e99] <dgraph.graphql.schema>:string .
[0x57e99] <dgraph.rule.predicate>:string @index(exact) @upsert .
[0x57e99] <dgraph.graphql.p_query>:string @index(sha256) .
[0x57e99] <dgraph.rule.permission>:int .
[0x57e99] type <dgraph.graphql> {
dgraph.graphql.schema
dgraph.graphql.xid
}
[0x57e99] type <dgraph.type.Rule> {
dgraph.rule.predicate
dgraph.rule.permission
}
[0x57e99] type <dgraph.type.User> {
dgraph.xid
dgraph.password
dgraph.user.group
}
[0x57e99] type <dgraph.type.Group> {
dgraph.xid
dgraph.acl.rule
}
[0x57e99] type <dgraph.graphql.persisted_query> {
dgraph.graphql.p_query
}
in the RDF I don’t hav eany dgraph.acl.rule
but dgraph.user.group
and dgraph.password
and dgraph.xid
<0x1a48a00750> <dgraph.xid> "guardians"^^<xs:string> <0x57e99> .
<0x1a48a00751> <dgraph.xid> "groot"^^<xs:string> <0x57e99> .
<0x1a48a00751> <dgraph.user.group> <0x1a48a00750> <0x57e99> .
<0x1a48a00751> <dgraph.password> "encrypted password"^^<xs:password> <0x57e99> .
MichelDiz
(Michel Diz)
November 1, 2022, 9:57pm
9
Yeah, this is expected. Theory hits. We should export without ACL. The value [0x57e99] represents your namespace on the shared cluster.
1 Like