Error running dgraph-bulk-loader on Windows

Users
1

It seems dgraph-bulk-loader depends on “ulimit” as it is starting up. It seems this is not compatible with Windows - please see error below. Maybe it can be defaulted to something reasonable, or entered manually, for running it on Windows. Let me know if you want me to create an issue on github and I’ll create one there.

C:\dgraph\dgraph83>dgraph-bulk-loader.exe -r=data.rdf.gz -s=data.schema -map_shards=1 -reduce_shards=1
{
        "RDFDir": "data.rdf.gz",
        "SchemaFile": "data.schema",
        "DgraphsDir": "out",
        "LeaseFile": "LEASE",
        "TmpDir": "tmp",
        "NumGoroutines": 4,
        "MapBufSize": 67108864,
        "ExpandEdges": true,
        "BlockRate": 0,
        "SkipMapPhase": false,
        "CleanupTmp": true,
        "NumShufflers": 1,
        "Version": false,
        "StoreXids": false,
        "MapShards": 1,
        "ReduceShards": 1
}
e[31mNonfatal error: max open file limit could not be detected: cannot detect max open files on this platform
e[0mThe bulk loader needs to open many files at once. This number depends on the size of the data set loaded, the map file output size, and the level of indexing. 100,000 is adequate for most data set sizes. See `man ulimit` for details of how to change the limit.
2017/10/23 11:33:03 open data.schema: The system cannot find the file specified.
2

Hi @tamethecomplex, the ulimit message you’re seeing is non-fatal, and the bulk loader should continue fine even in the absence of ulimit.

The error that is stopping bulk loader from running in your particular case is 017/10/23 11:33:03 open data.schema: The system cannot find the file specified. The bulkloader can’t open the schema file. Can you double check to make sure the supplied schema file path is correct?

3

Whoops - can’t believe I didn’t see that. Loader finished without error this time, thanks!

1 Like
4

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.