# Loading 50GB CSV data

**URL:** <https://discuss.dgraph.io/t/loading-50gb-csv-data/4992>\
**Category:** Dgraph\
**Created:** [September 2, 2019, 5:47pm UTC](https://discuss.dgraph.io/t/loading-50gb-csv-data/4992 "2019-09-02T17:47:20Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![shoukat](https://avatars.discourse-cdn.com/v4/letter/s/e36b37/32.png) [@shoukat](https://discuss.dgraph.io/u/shoukat)\
**Post date:** [September 2, 2019, 5:47pm UTC](https://discuss.dgraph.io/t/loading-50gb-csv-data/4992/1 "2019-09-02T17:47:20Z")

</div>

Hi, We are exploring dgraph and need to load 50GB csv file to dgraph.  
Is there any utility to load csv directly? Any inputs?

---

<div class="post-metadata">

**Author:** ![MichelDiz](https://yyz1.discourse-cdn.com/flex007/user_avatar/discuss.dgraph.io/micheldiz/32/11873_2.png) [@MichelDiz](https://discuss.dgraph.io/u/MichelDiz)\
**Post date:** [September 2, 2019, 7:13pm UTC](https://discuss.dgraph.io/t/loading-50gb-csv-data/4992/2 "2019-09-02T19:13:34Z")

</div>

> [@shoukat](#):
>
> Is there any utility to load csv directly?

nope

here a way to do it [Get started with Dgraph](https://docs.dgraph.io/howto/#loading-csv-data)

you could also export it to JSON and import via bulkload or liveloader

[![](https://canada1.discourse-cdn.com/flex007/uploads/dgraph/original/2X/b/b75bd3524d4ce2ddc6aa3ee185b1f37f2947c3e8.jpeg "Parse CSV Data Using The Go Programming Language") ](https://www.youtube.com/watch?v=fIof0yCMMX0)

---

<div class="post-metadata">

**Author:** ![yyyguy](https://yyz1.discourse-cdn.com/flex007/user_avatar/discuss.dgraph.io/yyyguy/32/2020_2.png) [@yyyguy](https://discuss.dgraph.io/u/yyyguy)\
**Post date:** [September 3, 2019, 3:45pm UTC](https://discuss.dgraph.io/t/loading-50gb-csv-data/4992/3 "2019-09-03T15:45:16Z")

</div>

The ability to load CSV data is one of the basic pieces of functionality I look for when I have been researching graph database solutions. I was a bit surprised when I couldn’t find something to do that “out of the box”.

I overcame this by using NiFi and building a CSV to JSON FlowFile, and was able to load my dgraph instance with a 25GB CSV file. It worked great.

From my perspective, as Dgraph matures, and to make it more accessible/approachable, (like Neo4J) pre-built functions will be a necessity.

---

<div class="post-metadata">

**Author:** ![graphpivot](https://yyz1.discourse-cdn.com/flex007/user_avatar/discuss.dgraph.io/graphpivot/32/1909_2.png) [@graphpivot](https://discuss.dgraph.io/u/graphpivot)\
**Post date:** [September 5, 2019, 3:28pm UTC](https://discuss.dgraph.io/t/loading-50gb-csv-data/4992/4 "2019-09-05T15:28:46Z")

</div>

@yyyguy: You have to be careful importing csv data from relational database, because there can be often “NULL” strings for missing values. I have not tried the csv parser mentioned above, but I assume that “NULL” would be interpreted as a string and thus imported later in the graph. If you take a look at the demo set of neo4j [here](https://github.com/neo4j-contrib/northwind-neo4j/blob/master/data/customers.csv) you can see these NULL values.

In my opinion missing values should not be imported - I think this is one of the strengths of graph data over relational tables. For example when a customer does not have a fax number (it’s another question who uses today a fax number…🙂 ) then there is no predicate for this customer.

---

<div class="post-metadata">

**Author:** ![yyyguy](https://yyz1.discourse-cdn.com/flex007/user_avatar/discuss.dgraph.io/yyyguy/32/2020_2.png) [@yyyguy](https://discuss.dgraph.io/u/yyyguy)\
**Post date:** [September 6, 2019, 4:28am UTC](https://discuss.dgraph.io/t/loading-50gb-csv-data/4992/5 "2019-09-06T04:28:25Z")

</div>

@graphpivot I agree with your thoughts. I have dealt with the issue of not loading NULL values at the NiFi level. That way I can take a database-agnostic approach. I can then focus on the data load.

Again, thanks for your thoughts.

---

<div class="post-metadata">

**Author:** ![MattH](https://yyz1.discourse-cdn.com/flex007/user_avatar/discuss.dgraph.io/matth/32/8007_2.png) [@MattH](https://discuss.dgraph.io/u/MattH)\
**Post date:** [February 25, 2022, 11:15am UTC](https://discuss.dgraph.io/t/loading-50gb-csv-data/4992/6 "2022-02-25T11:15:55Z")

</div>

@yyyguy I know this was awhile ago, but I’m looking at doing something similar with Nifi and was curious about the details of how you loaded the Json file into Dgraph - e.g. did you invoke liveloader or use a Post step?
