Embedded Dgraph in Golang Application?

Users Dgraph
1

Hello,

I am working on a LibP2P project where I will have many distributed nodes and would like to embed Dgraph in each node such that as the nodes come online then the Dgraph cluster gets larger.

Of course, there will need to be some sharding and replication to ensure that if a nodes drops out then the data is still available, but this is the general idea.

I saw some previous discussions (2017) about interest in providing an embeddable Dgraph version and was wondering if that ever happened or if some other solution became available.

Any information on this would be greatly appreciated.
Thanks in advance

2

Hey Lonnie,

No embeddable Dgraph is available, and I think this won’t happen as the whole concept of Dgraph is set up in a Cluster with a distributed context.

Cheers.

3

Hi Michel,

Thanks for the quick response on this and I completely understand. Normally, I would have external databases running as well, but this is more of an embedded project for this one so I will keep searching for an embedded Golang solution.

Appreciate your input and have a great day.

4

Well, BadgerDB is embeddable tho. Not a Graph DB, but it is a Golang embeddable DB.

5

you may need https://github.com/etcd-io/etcd/tree/main/raft to solve your problem

6

Hi All,

Actually been researching this a bit an thinking about another possible direction.

As I understand things, for a Dgraph cluster we generally have:

  1. Zero Node — One or more, but seems to be a standard of at least three, so that the cluster can be balanced as needed.

  2. Alpha Node — These are the actual storage nodes and the cluster can have many of them. Perhaps thousands ??? (not sure on this part) that can dynamically join and leave the P2P mesh.

  3. Ratel Node — These are the user UI nodes but seem to be optional and may not be needed for a P2P setup since other client code can make the API calls to the “Alpha” nodes.

If this is correct, then I think that the main P2P design will be to take the Alpha node and develop from there as the base such that the existing “main” function can be renames and spawned off from another Golang main function as a goroutine (thread) to run independently.

The objective is to have a few “servers” shelled off from the main wrapper as goroutines but to also have all of the Go servers compiled into a single application. (i.e. no independent Dgraph Alpha, IPFS node, etc…) so that everything is encapsulated as a single startup background application that starts and stops all of the servers.

That is the goal at least and if I am to use Golang, which has some very powerful features over C/C++ already built-in, then maybe this is the way to integrate all of these Golang servers into a single encapsulating application.

Any thoughts or suggestions would be greatly appreciated.
Thanks in advance

7

Yep, zero is the “orchestrator” of the Cluster. It leases uids(mean, he is responsible for generating them), it is responsible for the timestamps, to balance the data in the cluster, and so on.

Infinity.

Sort of. It is a distributed system. Managed by the Zero Group.