Bulk-Import does not really work but live successful

Users Dgraph
,
1

When I use bulk to load data, the log does not report an error, but an error is reported during query, indicating that the data is not imported correctly.

ubuntu@ip-172-30-2-14:/new_disk/data/for_dgraph_run$ dgraph bulk -f /new_disk/data/sf1/json_data/df1/ -s /new_disk/data/schema --xidmap xidmap
[Decoder]: Using assembly version of decoder
I1004 05:40:24.087268  128842 init.go:102] 

Dgraph version   : v20.07.1
Dgraph codename  : shuri-1
Dgraph SHA-256   : 29d6804d57da4b9ddcc2173c5fe1631b97031d1b364b8b75e52cfa180ec36de6
Commit SHA-1     : d7123d93e
Commit timestamp : 2020-09-17 14:57:43 -0700
Branch           : HEAD
Go version       : go1.14.4

For Dgraph official documentation, visit https://dgraph.io/docs/.
For discussions about Dgraph     , visit http://discuss.dgraph.io.
To say hi to the community       , visit https://dgraph.slack.com.

Licensed variously under the Apache Public License 2.0 and Dgraph Community License.
Copyright 2015-2020 Dgraph Labs, Inc.


I1004 05:40:24.087432  128842 util_ee.go:126] KeyReader instantiated of type <nil>

The bulk loader needs to open many files at once. This number depends on the size of the data set loaded, the map file output size, and the level of indexing. 100,000 is adequate for most data set sizes. See `man ulimit` for details of how to change the limit.
Current max open files limit: 8192

Connecting to zero at localhost:5080
Predicate "dgraph.graphql.schema" already exists in schema
Predicate "dgraph.graphql.xid" already exists in schema
badger 2020/10/04 05:40:24 INFO: All 0 tables opened in 0s
I1004 05:40:24.106809  128842 xidmap.go:111] Loaded up 0 xid to uid mappings
Processing file (1 out of 31): /new_disk/data/sf1/json_data/df1/comment_0_0.json
Processing file (2 out of 31): /new_disk/data/sf1/json_data/df1/comment_hasCreator_person_0_0.json
Processin

...
[05:42:24Z] REDUCE 02m00s 98.81% edge_count:65.69M edge_speed:1.365M/sec plist_count:35.89M plist_speed:746.0k/sec. Num Encoding: 1
[05:42:25Z] REDUCE 02m01s 99.60% edge_count:66.22M edge_speed:1.348M/sec plist_count:36.42M plist_speed:741.5k/sec. Num Encoding: 1
[05:42:26Z] REDUCE 02m02s 100.00% edge_count:66.48M edge_speed:1.327M/sec plist_count:36.68M plist_speed:731.9k/sec. Num Encoding: 0
[05:42:27Z] REDUCE 02m03s 100.00% edge_count:66.48M edge_speed:1.301M/sec plist_count:36.68M plist_speed:717.6k/sec. Num Encoding: 0
writeCh: Time taken to write req: 1.566s
[05:42:28Z] REDUCE 02m04s 100.00% edge_count:66.48M edge_speed:1.276M/sec plist_count:36.68M plist_speed:703.7k/sec. Num Encoding: 0
[05:42:29Z] REDUCE 02m05s 100.00% edge_count:66.48M edge_speed:1.252M/sec plist_count:36.68M plist_speed:690.5k/sec. Num Encoding: 0
[05:42:30Z] REDUCE 02m06s 100.00% edge_count:66.48M edge_speed:1.231M/sec plist_count:36.68M plist_speed:679.2k/sec. Num Encoding: 0
Total: 02m06s

But after I changed bulk to live, the data was imported correctly because the query ran correctly.
Why is this?

2

Did you bulk load when the clusters are already running?

3

@chewxy

clusters?? live load was worked, I guess clusters are running…right?

4

Yeah, so you should not run Bulk Load when your Dgraph clusters are running already. You run the Bulk Loader before you start up your Dgraph clusters.

5

I 'm not run alpha, just zero and ratel

What do you mean by cluster？

6

Dgraph needs Zero and Alpha to run at the same time. A cluster is at least one Zero + one Alpha grouped together.