Error while trying to load bulk rdf data [gzip: invalid header]

Hello all,
I am trying to bulk load data into Dgraph using bulk and live options but I always receive this error.

My Command :

1. dgraph bulk -f 1million.rdf.gz -s 1million.schema --map_shards=4 --reduce_shards=2 --http localhost:8000 --zero=localhost:5080

2. dgraph live -f 1million.rdf.gz

I received the same error in both the cases:

Running transaction with dgraph endpoint:

Found 1 data file(s) to process

Processing data file "1million.rdf.gz"

2019/09/30 20:09:45 **gzip: invalid header**







Version details of my Dgraph:

[Decoder]: Using assembly version of decoder

Dgraph version   : v1.1.0
Dgraph SHA-256   : 98db2956f6dd8b7b9b88e02962d2036845b057fe5fe953190eaafac0a83dfcce
Commit SHA-1     : ef7cdb28
Commit timestamp : 2019-09-04 00:12:51 -0700
Branch           : HEAD
Go version       : go1.12.7

Machine details:

MacBook Pro(2017) v10.14.6

Can you share a sample of your dataset?
This could be some gz mistake or corrupted.

This is where I got the dataset.( 1million.rdf.gz, 1million.schema )

well, why are you loading the same dataset twice?

I’ll check this later. If I can reproduce it.

I am not trying to load twice, since one method didn’t work, I tried to load in a different way.

Sorry Sairam, but I can’t reproduce it in any way.

PS. Tested it using iMac Pro (2017)

1 - Download the 1mi rdf from benchmarks/1million.rdf.gz at master · dgraph-io/benchmarks · GitHub
2 - Copy this schema benchmarks/1million.schema at master · dgraph-io/benchmarks · GitHub
3 - Download the v1.1.0 binary from releases Releases · dgraph-io/dgraph · GitHub
4 - Create a simple cluster.
5 - Start live loader

result = Works

6 - Delete all files related to that test and star only Zero
7 - Start Bulk loader

result = Works

I don’t know what the problem is, but .gz seems to be the issue here.

dgraph live -f ./1million.rdf.gz -s ./1million.schema -a localhost:9080

Didn’t work(It was throwing the same error as listed above). But

dgraph live -f ./1million.rdf -s ./1million.schema -a localhost:9080

Works as intended.

Without being able to reproduce I can’t do anything to help. What OS are you using? can you normally ungzip it? There is a gz compression option, are you using it?

When I try to unzip usually, it says contents of 1million.rdf.gz cannot be extracted. (both with archive utility and The unarchiver)

But when I downloaded the file directly, Macos by default extracted the file for me. That extracted file was executed properly as I mentioned above.

You can try this command on Mac OS:

gzip -d 1million.rdf.gz

and that should give you the uncompressed file.

I just tried it and it worked for me.



You can try this command on Mac OS:

gzip -d

and that should give you the uncompressed file.

I just tried it and it worked for me.



This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.