Live Loader from M1 and GitHub Actions

Users Dgraph
1

Context

  1. I am developing against Dgraph Cloud.

  2. I am running node and python code locally (on my Mac M1 machine) that transform CSV files to ~100 000 triples that need to be loaded into the Dgraph cloud a few times per day.

  3. I also run some of the same ETL pipelines (node and python) via Github actions (containerized).

How could I best tap into the live loader on my local machine when node and python have generated their 100 000s triples in JSON or Quad format? (It seems like a hassle to run this inside Docker locally and I’d be very happy for any alternative options.)

2

Dgraph binary has support for Linux only. So you have to use Docker or K8s. About JSON or NQuad, it is up to you to choose the best for your case.

You can try to build your own binary by cloning the repo.