Import error, dgraph live reports "Attr: [xxx] should have @lang directive in schema to mutate edge"

Issues
1

I want to take a performance testing on dgraph v20.07. I founded a huge dataset at:

When I execute command:
dgraph live -f 21million.rdf.gz
It always shows me a lot of error as below:
…
Error while mutating: Attr: [film.performance.character_note] should have @lang directive in schema to mutate edge: [entity:1081204 attr:“film.performance.character_note” value:“Danny at 10” lang:“en” ] s.Code Unknown

Error while mutating: Attr: [film.performance.character_note] should have @lang directive in schema to mutate edge: [entity:185125 attr:“film.performance.character_note” value:“Audiotraffic Member #2” lang:“en” ] s.Code Unknown

Error while mutating: Attr: [film.film.tagline] should have @lang directive in schema to mutate edge: [entity:242618 attr:“film.film.tagline” value:“Three different women married Frankie Lymon. Now as they fight to get a piece of his millions, They will have to prove who married him first and who loved him most.” lang:“en” ] s.Code Unknown

I unzipped the data file and review it line by line. I found the data item that reported error has @lang specified. For example:

<m.0v9m6tc> <film.performance.character_note> “Danny at 10”@en .
<m.0w2yvwk> <film.performance.character_note> “Audiotraffic Member #2”@en .
<m.087kv6> <film.film.tagline> “Three different women married Frankie Lymon. Now as they fight to get a piece of his millions, They will have to prove who married him first and who loved him most.”@en .

All of data item has been specified language with “@en”. Unfortunately, dgraph reports error line by line. It seems work not as designed.

2

Hi @micli, welcome to the Dgraph community.
The following error could be because of the wrong schema definition. Can you please help with the schema you have defined. Just to define the context, " A @lang directive must be specified in the schema to query or mutate predicates with language tags." You can find more about lang directive here. Looking at the dataset, predicates which needed the @lang directive are type.object.name, film.performance.character_note, film.film_cut.note, film.film_cut.note.

3

Hi:
Thanks for your great help. I’ve made a schema file as below:

film.director.film: [uid] @reverse .
film.film.genre: [uid] @reverse .
film.film.initial_release_date: dateTime @index(day) .
film.film.rating: [uid] @reverse .
film.performance.character_note: string @index(exact, fulltext) @lang .
film.film_cut.note: string @index(exact, fulltext) @lang .
film.film.tagline: string @index(exact, fulltext) @lang .
loc: geo @index(geo) .
type.object.name: string @index(exact, fulltext) @lang .
type.object.name.en: string @index(exact, fulltext) .
type.object.name.hi: string @index(exact, fulltext) .
type.object.name.ta: string @index(exact, fulltext) .

It seems start to import.

micl@dgraph:~/dgraph$ dgraph live  -f 21million.rdf.gz -s film.schema
[Decoder]: Using assembly version of decoder
I0903 01:13:06.129177   25166 init.go:102] 

Dgraph version   : v20.07.0
Dgraph codename  : shuri
Dgraph SHA-256   : 4cd320fc6eab163ab68602a5122a6c82c8467c2ed5ac93478d5f40d44eec71c4
Commit SHA-1     : d65e20530
Commit timestamp : 2020-07-28 15:31:37 -0700
Branch           : HEAD
Go version       : go1.14.4

For Dgraph official documentation, visit https://dgraph.io/docs/.
For discussions about Dgraph     , visit http://discuss.dgraph.io.
To say hi to the community       , visit https://dgraph.slack.com.

Licensed variously under the Apache Public License 2.0 and Dgraph Community License.
Copyright 2015-2020 Dgraph Labs, Inc.


I0903 01:13:06.129446   25166 util_ee.go:126] KeyReader instantiated of type <nil>

Running transaction with dgraph endpoint: 127.0.0.1:9080

Processing schema file "film.schema"
Processed schema file "film.schema"

Found 1 data file(s) to process
Processing data file "21million.rdf.gz"
[01:13:31Z] Elapsed: 05s Txns: 334 N-Quads: 334000 N-Quads/s [last 5s]: 66800 Aborts: 0
[01:13:36Z] Elapsed: 10s Txns: 703 N-Quads: 703000 N-Quads/s [last 5s]: 73800 Aborts: 0
[01:13:41Z] Elapsed: 15s Txns: 1021 N-Quads: 1021000 N-Quads/s [last 5s]: 63600 Aborts: 0
[01:13:46Z] Elapsed: 20s Txns: 1309 N-Quads: 1309000 N-Quads/s [last 5s]: 57600 Aborts: 0
[01:13:51Z] Elapsed: 25s Txns: 1583 N-Quads: 1583000 N-Quads/s [last 5s]: 54800 Aborts: 0
[01:13:56Z] Elapsed: 30s Txns: 1893 N-Quads: 1893000 N-Quads/s [last 5s]: 62000 Aborts: 0
[01:14:01Z] Elapsed: 35s Txns: 2262 N-Quads: 2262000 N-Quads/s [last 5s]: 73800 Aborts: 0
[01:14:06Z] Elapsed: 40s Txns: 2636 N-Quads: 2636000 N-Quads/s [last 5s]: 74800 Aborts: 0
[01:14:11Z] Elapsed: 45s Txns: 3032 N-Quads: 3032000 N-Quads/s [last 5s]: 79200 Aborts: 0
[01:14:16Z] Elapsed: 50s Txns: 3361 N-Quads: 3361000 N-Quads/s [last 5s]: 65800 Aborts: 0
[01:14:21Z] Elapsed: 55s Txns: 3665 N-Quads: 3665000 N-Quads/s [last 5s]: 60800 Aborts: 0
[01:14:26Z] Elapsed: 01m00s Txns: 3938 N-Quads: 3938000 N-Quads/s [last 5s]: 54600 Aborts: 0
[01:14:31Z] Elapsed: 01m05s Txns: 4265 N-Quads: 4265000 N-Quads/s [last 5s]: 65400 Aborts: 0
[01:14:36Z] Elapsed: 01m10s Txns: 4537 N-Quads: 4537000 N-Quads/s [last 5s]: 54400 Aborts: 0
[01:14:41Z] Elapsed: 01m15s Txns: 4791 N-Quads: 4791000 N-Quads/s [last 5s]: 50800 Aborts: 0
[01:14:46Z] Elapsed: 01m20s Txns: 5074 N-Quads: 5074000 N-Quads/s [last 5s]: 56600 Aborts: 0
[01:14:51Z] Elapsed: 01m25s Txns: 5433 N-Quads: 5433000 N-Quads/s [last 5s]: 71800 Aborts: 0
[01:14:56Z] Elapsed: 01m30s Txns: 5778 N-Quads: 5778000 N-Quads/s [last 5s]: 69000 Aborts: 0
......
4

Hi:

It seems there is still an issue. I got below error at 4m30s.

[01:18:01Z] Elapsed: 04m35s Txns: 14724 N-Quads: 14724000 N-Quads/s [last 5s]:     0 Aborts: 0
Error while mutating: Server overloaded with pending proposals. Please retry later s.Code Unknown
Error while mutating: Server overloaded with pending proposals. Please retry later s.Code Unknown

The imported number suspended at 1472400 till now.

[01:18:06Z] Elapsed: 04m40s Txns: 14724 N-Quads: 14724000 N-Quads/s [last 5s]:     0 Aborts: 2
[01:18:11Z] Elapsed: 04m45s Txns: 14724 N-Quads: 14724000 N-Quads/s [last 5s]:     0 Aborts: 2
......
[01:23:31Z] Elapsed: 10m05s Txns: 14724 N-Quads: 14724000 N-Quads/s [last 5s]:     0 Aborts: 2
[01:23:36Z] Elapsed: 10m10s Txns: 14724 N-Quads: 14724000 N-Quads/s [last 5s]:     0 Aborts: 2
[01:23:41Z] Elapsed: 10m15s Txns: 14724 N-Quads: 14724000 N-Quads/s [last 5s]:     0 Aborts: 2
......

How can I fix it?
Thanks!

5

Hi:

Due to import process has been disturb, how can I drop all data in dgraph?
I did not find any way of command line in document.

6

You can go to Ratel UI and in the Schema panel, click on “Bulk edit” then “Drop Data” write “DROP DATA” and hit “Drop All”.