Unable to write to value log file

Users Dgraph
1

I’m trying to bulk load around a million triples into DGraph.

I’m not sure why this would be failing in this way. I have a very large amount of free disk space.

MAP 46s nquad_count:10.14M err_count:0.000 nquad_speed:219.7k/sec edge_count:28.30M edge_speed:613.2k/sec
Shard tmp/shards/000 -> Reduce tmp/shards/shard_0/000
REDUCE 47s [5.38%] edge_count:1.524M edge_speed:1.524M/sec plist_count:159.4k plist_speed:159.4k/sec
REDUCE 48s [15.45%] edge_count:4.372M edge_speed:4.370M/sec plist_count:457.3k plist_speed:457.2k/sec
badger2019/02/16 23:05:09 ERROR: writeRequests: Unable to write to value log file: "out/0/p/000000.vlog": write out/0/p/000000.vlog: file too large
badger2019/02/16 23:05:09 ERROR: writeRequests: Unable to write to value log file: "out/0/p/000000.vlog": write out/0/p/000000.vlog: file too large
badger2019/02/16 23:05:09 ERROR: writeRequests: Unable to write to value log file: "out/0/p/000000.vlog": write out/0/p/000000.vlog: file too large
2019/02/16 23:05:09 write out/0/p/000000.vlog: file too large
Unable to write to value log file: "out/0/p/000000.vlog"
github.com/dgraph-io/dgraph/vendor/github.com/dgraph-io/badger.(*valueLog).write.func1
        /ext-go/1/src/github.com/dgraph-io/dgraph/vendor/github.com/dgraph-io/badger/value.go:896
github.com/dgraph-io/dgraph/vendor/github.com/dgraph-io/badger.(*valueLog).write
        /ext-go/1/src/github.com/dgraph-io/dgraph/vendor/github.com/dgraph-io/badger/value.go:951
github.com/dgraph-io/dgraph/vendor/github.com/dgraph-io/badger.(*DB).writeRequests
        /ext-go/1/src/github.com/dgraph-io/dgraph/vendor/github.com/dgraph-io/badger/db.go:594
github.com/dgraph-io/dgraph/vendor/github.com/dgraph-io/badger.(*DB).doWrites.func1
        /ext-go/1/src/github.com/dgraph-io/dgraph/vendor/github.com/dgraph-io/badger/db.go:663
runtime.goexit
        /usr/local/go/src/runtime/asm_amd64.s:1333

github.com/dgraph-io/dgraph/x.Wrap
        /ext-go/1/src/github.com/dgraph-io/dgraph/x/error.go:91
github.com/dgraph-io/dgraph/x.Check
        /ext-go/1/src/github.com/dgraph-io/dgraph/x/error.go:41
github.com/dgraph-io/dgraph/dgraph/cmd/bulk.(*reducer).reduce.func2
        /ext-go/1/src/github.com/dgraph-io/dgraph/dgraph/cmd/bulk/reduce.go:100
github.com/dgraph-io/dgraph/vendor/github.com/dgraph-io/badger.runTxnCallback
        /ext-go/1/src/github.com/dgraph-io/dgraph/vendor/github.com/dgraph-io/badger/txn.go:615
runtime.goexit
        /usr/local/go/src/runtime/asm_amd64.s:1333
badger2019/02/16 23:05:09 ERROR: writeRequests: Unable to write to value log file: "out/0/p/000000.vlog": write out/0/p/000000.vlog: file too large```
2

Can you share what version and the configs used to start Dgraph? Also machine config.

3

Smells like some network file system, or some old file system…

4

i have the same problem…
there still have enough space, but bulk load failed with the msg: write out/0/p/000000.vlog: file too large

i test on my ubuntu18.04 , dgraph version: 1.0.13

dgraph zero -o 10000
dgraph bulk -r data.rdf -s g01.schema --map_shards=2 --reduce_shards=1 --http localhost:18000 --zero=localhost:15080

follow is my bulk log:

MAP 37s nquad_count:1.006M err_count:0.000 nquad_speed:27.17k/sec edge_count:3.018M edge_speed:81.51k/sec
MAP 38s nquad_count:1.006M err_count:0.000 nquad_speed:26.45k/sec edge_count:3.018M edge_speed:79.36k/sec
MAP 39s nquad_count:1.006M err_count:0.000 nquad_speed:25.78k/sec edge_count:3.018M edge_speed:77.33k/sec
badger 2019/03/21 15:42:43 INFO: Force compaction on level 0 done
Shard tmp/shards/000 -> Reduce tmp/shards/shard_0/000
Shard tmp/shards/001 -> Reduce tmp/shards/shard_0/001
badger 2019/03/21 15:42:43 INFO: All 0 tables opened in 0s
REDUCE 40s [19.58%] edge_count:590.9k edge_speed:590.9k/sec plist_count:13.73k plist_speed:13.73k/sec
badger 2019/03/21 15:42:45 ERROR: writeRequests: Unable to write to value log file: "out/0/p/000000.vlog": write out/0/p/000000.vlog: file too large
2019/03/21 15:42:45 write out/0/p/000000.vlog: file too large
Unable to write to value log file: "out/0/p/000000.vlog"
github.com/dgraph-io/dgraph/vendor/github.com/dgraph-io/badger.(*valueLog).write.func1
	/ext-go/1/src/github.com/dgraph-io/dgraph/vendor/github.com/dgraph-io/badger/value.go:936
github.com/dgraph-io/dgraph/vendor/github.com/dgraph-io/badger.(*valueLog).write
	/ext-go/1/src/github.com/dgraph-io/dgraph/vendor/github.com/dgraph-io/badger/value.go:991
github.com/dgraph-io/dgraph/vendor/github.com/dgraph-io/badger.(*DB).writeRequests
	/ext-go/1/src/github.com/dgraph-io/dgraph/vendor/github.com/dgraph-io/badger/db.go:581
github.com/dgraph-io/dgraph/vendor/github.com/dgraph-io/badger.(*DB).doWrites.func1
	/ext-go/1/src/github.com/dgraph-io/dgraph/vendor/github.com/dgraph-io/badger/db.go:650
...
5

Can you share what file system you’re using? and if you have swap enabled.

6

I still have this error. Unable to restart the alpha as it keeps crashing. @MichelDiz please help.

7

@quillen I’m not sure about what. Can you please open a new topic with details? so I can help.