Subscriptions to Dgraph changes

Users
,
1

Hi!
I use dgo and need to add subscriptions to changes in my app. I don’t use GraphQL, so I would like to use Badger’s subscriptions.
There is an example of my code based on https://github.com/dgraph-io/badger/blob/f50343ff404d8198df6dc83755ec2eab863d5ff2/db_test.go#L1939 · GitHub
But changes aren’t handled, since there is no message about this in the logs

Is that right to subscribe to changes via Badger and how to do it correctly?

2

Hi @poketulhu, you can follow How to subscribe to dgraph changes? thread. Also, here is the working code.

Feel free to reach out for followups.

4

Another question is about a connection to Badger.
I need to open the database and then add a subscription, but when trying to open it I get the error:
Cannot acquire directory lock on "/dgraph/p". Another process is using this Badger database.: resource temporarily unavailable
As far as I understand it’s because Dgraph has already opened the base.
What is the right way to connect to Badger in this case?

5

As for subscription, I guess you would want to use badger in read-only mode. You can set the badger open options to ReadOnly=true and then open it.

6

I tried to open the same db twice and found that it works only if both databases are opened with WithReadOnly(true)

And WithReadOnly(true) doesn’t help if dgraph is running
Maybe I am doing something wrong?

7

Sorry, I misunderstood the problem. Badger acquires an exclusive lock over the directory to prevent multiple processes from using it.
I don’t think there is a way to use the subscription of badger to subscribe over changes made by dgraph as both will be separate processes. This will essentially require some kind of publisher-subscriber model which the team/community wants to have but is not on the immediate roadmap.
Tagging @ibrahim, if he knows of some low-level hacks.

8

This should be relatively easy to build though. We could consider adding it to the sprint.

1 Like
9

@Naman, just wanted to give a +1 and my 2 cents on publish-subscribe.
IMO, the custom mutation feature should be extended in some way, so that clients can register their callbacks. This allows a pub-sub mechanism, and hides badger implementation detail. This can then be used by a wider community for a large number of use cases (such as distribution of master data, or notification bridges between microservice domains).

2 Likes
10

Hey @poketulhu,
The script you have on based on https://github.com/dgraph-io/badger/blob/f50343ff404d8198df6dc83755ec2eab863d5ff2/db_test.go#L1939 · GitHub wouldn’t work because the prefix []byte("status") isn’t what dgraph will store for
[]byte( _:foo <status> "pending" .) mutation. The key would be <some prefix> predicate. We can add away to support this via dgraph but I don’t you can figure out the badger keys for your data directly.

There is a non-recommended way of doing this. You can open the p directory using a second badger instance and set the following options BypassLockGuard and ReadOnly to true.

Please note that this option isn’t thoroughly tested and could issues with dgraph when two badger instances are using the same directory (that’s why we have the lock).