Is it possible to request and reserve an ID prior to indexing?

Users Dgraph
1

I’m using Badger+Kafka as my primary data store and I’m planning to use DGraph just for graph lookups. I’m currently using bitmaps for indexing, so each node produces its own uint64 secondary identifier with a GUID as the primary id.

Given that my system is built around eventual concurrency, I didn’t really see the need to have uint64 that weren’t isolated to the node until I got to dgraph.

I could solve the problem of managing the allocation and distribution of the secondary IDs myself but it seems that DGraph could solve this for me.

Given that I don’t always need to index all data types and for those that I do, I can do it in the background, is it possible to simply request and reserve an ID without indexing the document?

edit:
I could just stand up an identifier microservice but will DGraph allow me to assign the IDs?

2

Not a dev - but, iirc -
If you want to track the UIDs yourself, you can - you just have to tell the Zero to allocate maxInt64 uids (via the grpc call AssignUids()). Then they are all allocated and you can place them however you want. If you dont tell the zero to assign all the IDs you need, you will get errors like “this is above the max allocated UID”

Any dgraph dev feel free to correct me.

1 Like
3

Thanks. I think this is probably the decent solution for me. I wasn’t aware of Zero (just started looking into dgraph).

I guess I could alternatively, request a small batch of IDs per node but I think I’ll end up losing a lot of IDs along the way which isn’t ideal for roaring.

4

Yup, Zero manages UID assignment. Each UID is a uint64. Typically Zero automatically assigns a unique UID to a new node as you load data into Dgraph, but if you want to manually assign UIDs you can request a range of UIDs from Zero that you can use yourself.

See the More about Dgraph Zero docs. As @iluminae said, there’s a gRPC call for it and in the docs you’ll see the HTTP call for /assign on Zero.

You can always find a UID in a query using the uid func in DQL. e.g.,
{ q(func: uid(0x123)) { ... } }

Indexing happens synchronously and automatically in Dgraph, so I’m not sure what you mean about applying an index later in the background with your mutations.

5

You can always find a UID in a query using the uid func in DQL. e.g.,
{ q(func: uid(0x123)) { … } }

What do you mean by that? As in, find aa free and available ID?

Indexing happens synchronously and automatically in Dgraph, so I’m not sure what you mean about applying an index later in the background with your mutations.

Understood. I just meant that in my normal application flow, I don’t need to index the data in dgraph but I do need a uint64 ID for indexing on my end. I’d rather avoid the round trip + indexing time and handle the dgraph indexing later.

Thank you and @iluminae .

6

I think getting your own range of UIDs from Dgraph to use for your app is exactly what you’re looking for. That’s already covered in this topic.

You can get a range of UIDs, use them in your app, and then use the same UIDs in mutations to Dgraph.