Bulk JSON loading

mutation
(Alex Kotovsky) #1

Hello everybody!

I have 10 json files with 1.300.000 elements looks like:

{
  accounts: [
   { "id": 1, "name": "Alex", "likes" : [{ "id": 5 }  .... ] }   <- 1.300.000 these elements, where likes array contains
  ...                                                                             <- users which got likes
  ]
}

What way to create all nodes within “like” edge between them in the shortest possible time (max 16 min)?

Docs contains too little information about loading large json :roll_eyes:

Thanks :raised_hands:

(Michel Conrado) #2

The JSON Bulk loading is too new and we’re still working on it.
There are other things to be done to be tuned. Like #2848 and 2923.

For now you will have to generate a compatible JSON. In this case your JSON would need to enter only Blank Nodes. e.g:

Just add a key like: { "uid": "_:someIdentifier" }

{
  accounts: [
   { "uid": "_:id1", "name": "Alex", "likes" : [{ "uid": "_:id5" }  .... ] }  
  ...                                                                           
  ]
}

Anyway, if you do a large, already hierarchically related JSON. It’s okay to make Bulk right away. However, if you have several unstructured but related values. You will need to enter your respective Blank Nodes to make them unique.

Cheers.