Bulk load with graphql schema

Users GraphQL
, ,
1

I use Dgraph v20.07.1 and trying to bulk load a json file with a graphql schema file like:

type User {
    id: String! @id
    username: String @search(by: [hash, regexp])
    followees: [User]
}

I use the following command:

dgraph bulk -f users.json -s schema.graphql --map_shards=9 --reduce_shards=3 --http localhost:8000 --zero=localhost:5080

but i get this error:

2020/09/29 17:43:54 line 2 column 16: Expected new line after field declaration. Got @
github.com/dgraph-io/dgraph/lex.Item.Errorf
	/ext-go/1/src/github.com/dgraph-io/dgraph/lex/lexer.go:54
github.com/dgraph-io/dgraph/schema.parseTypeField
	/ext-go/1/src/github.com/dgraph-io/dgraph/schema/parse.go:401
github.com/dgraph-io/dgraph/schema.parseTypeDeclaration
	/ext-go/1/src/github.com/dgraph-io/dgraph/schema/parse.go:342
github.com/dgraph-io/dgraph/schema.Parse
	/ext-go/1/src/github.com/dgraph-io/dgraph/schema/parse.go:458
github.com/dgraph-io/dgraph/dgraph/cmd/bulk.readSchema
	/ext-go/1/src/github.com/dgraph-io/dgraph/dgraph/cmd/bulk/loader.go:169
github.com/dgraph-io/dgraph/dgraph/cmd/bulk.newLoader
	/ext-go/1/src/github.com/dgraph-io/dgraph/dgraph/cmd/bulk/loader.go:123
github.com/dgraph-io/dgraph/dgraph/cmd/bulk.run
	/ext-go/1/src/github.com/dgraph-io/dgraph/dgraph/cmd/bulk/run.go:228
github.com/dgraph-io/dgraph/dgraph/cmd/bulk.init.0.func1
	/ext-go/1/src/github.com/dgraph-io/dgraph/dgraph/cmd/bulk/run.go:48
github.com/spf13/cobra.(*Command).execute
	/go/pkg/mod/github.com/spf13/cobra@v0.0.5/command.go:830
github.com/spf13/cobra.(*Command).ExecuteC
	/go/pkg/mod/github.com/spf13/cobra@v0.0.5/command.go:914
github.com/spf13/cobra.(*Command).Execute
	/go/pkg/mod/github.com/spf13/cobra@v0.0.5/command.go:864
github.com/dgraph-io/dgraph/dgraph/cmd.Execute
	/ext-go/1/src/github.com/dgraph-io/dgraph/dgraph/cmd/root.go:70
main.main
	/ext-go/1/src/github.com/dgraph-io/dgraph/dgraph/main.go:78
runtime.main
	/usr/local/go/src/runtime/proc.go:203
runtime.goexit
	/usr/local/go/src/runtime/asm_amd64.s:1373

github.com/dgraph-io/dgraph/x.Check
	/ext-go/1/src/github.com/dgraph-io/dgraph/x/error.go:42
github.com/dgraph-io/dgraph/dgraph/cmd/bulk.readSchema
	/ext-go/1/src/github.com/dgraph-io/dgraph/dgraph/cmd/bulk/loader.go:170
github.com/dgraph-io/dgraph/dgraph/cmd/bulk.newLoader
	/ext-go/1/src/github.com/dgraph-io/dgraph/dgraph/cmd/bulk/loader.go:123
github.com/dgraph-io/dgraph/dgraph/cmd/bulk.run
	/ext-go/1/src/github.com/dgraph-io/dgraph/dgraph/cmd/bulk/run.go:228
github.com/dgraph-io/dgraph/dgraph/cmd/bulk.init.0.func1
	/ext-go/1/src/github.com/dgraph-io/dgraph/dgraph/cmd/bulk/run.go:48
github.com/spf13/cobra.(*Command).execute
	/go/pkg/mod/github.com/spf13/cobra@v0.0.5/command.go:830
github.com/spf13/cobra.(*Command).ExecuteC
	/go/pkg/mod/github.com/spf13/cobra@v0.0.5/command.go:914
github.com/spf13/cobra.(*Command).Execute
	/go/pkg/mod/github.com/spf13/cobra@v0.0.5/command.go:864
github.com/dgraph-io/dgraph/dgraph/cmd.Execute
	/ext-go/1/src/github.com/dgraph-io/dgraph/dgraph/cmd/root.go:70
main.main
	/ext-go/1/src/github.com/dgraph-io/dgraph/dgraph/main.go:78
runtime.main
	/usr/local/go/src/runtime/proc.go:203
runtime.goexit
	/usr/local/go/src/runtime/asm_amd64.s:1373

it seems that bulk loader is not compatible with graphql schema, should i convert the schema file to something else?

2

I think you should use

dgraph bulk -h | grep graphql
  -g, --graphql_schema string            Location of the GraphQL schema file.

Instead of the config option -s.

You can use an empty schema file in the -s option.

PS. This option is on the master. Not sure if your version has it.

Cheers.

3

@MichelDiz yes, there is -g option in my version. but when i replace -s with -g, i get :
Schema file must be specified.
anyone tried -g option successfully?

4

You can just add an empty schema file.

5

thank you @MichelDiz your workaround just worked.
however, i think it should be considered as bug, because -s and -g should be exclusive

6

Hum, @martinmr what do you think?

7

It’s not really exclusive. You can have both if you are adding predicates other than the ones created by the graphql layer.

The empty schema should work fine.

8

@martinmr at least, -s should not be mandatory when -g is provided.

9

Yeah, that’s fair. We’ll open a ticket to do this.

10

I use dgraph v21.03.1 bulk to import data. I specify that dgraph schema is empty and use graphql schema to import data. However, in the end, I can’t create index for predicate and Type`

11

What do you mean? Not sure what indexing you are talking about.