Motivation
Users of Dgraph’s auto-generated GraphQL CRUD API, use the auto-generated mutations from the client-side to mutate(add/update/delete) data in the DB. Once the data has been mutated in the DB, they often want to listen to the changes that were done by the mutation and do some actions based on those changes. For example:
- Sending those changes to a 3rd-party tool like DataDog.
- Maintaining a changelog/audit-log on their own.
- Auto-filling some data: When a user adds/updates an address, a function is triggered to verify the address against another API and fill out extra data with DQL such as linking Geo data from a 3rd party API.
Recently we have introduced CDC, but that sends the changes at a predicate level. While in GraphQL, the basic entities are types. So, with webhooks on mutations, we want to send all the changes that were done on a type in a single mutation as one event to the webhook. The user will have the ability to write some user-defined code in JavaScript to handle the events.
We aim for this to be part of the 21.03 release.
User Impact
Webhooks will greatly enhance the experience for users of the GraphQL API. Slash GraphQL users will be able to seamlessly enjoy this feature without any extra efforts.
Implementation
We plan to implement it in a similar manner to @lambda. At present, for @lambda following JSON HTTP payload is sent to the lambda server URL configured with Alpha:
{
"resolver": "<Typename.FieldName>",
"X-Dgraph-AccessToken": "<ACL-JWT-Token>",
"authHeader": {
"key": "<auth-header-name>",
"value": "<auth-header-value == JWT>"
},
"parents": [{...}, {...}, ...],
"args": {...},
}
where parents are sent for fields and args are sent for queries/mutations.
Similarly, after a mutation has been executed and committed to the DB, we will send the following JSON HTTP payload to the pre-configured lambda server URL:
{
"resolver": "$webhook",
"X-Dgraph-AccessToken": "<ACL-JWT-Token>",
"authHeader": {
"key": "<auth-header-name>",
"value": "<auth-header-value == JWT>"
},
"event": {
"__typename": "<Typename>",
"operation": "<one-of: add/update/delete>",
"commitTs": <uint64, the commitTs of the mutation>
"add": {
"rootUIDs": [<list-of-UIDs-that-were-created-for-root-nodes-in-this-mutation>],
"input": [<AddTypeInput: i.e. all the data that was received as part of the `input` argument>]
},
"update": {
"rootUIDs": [<list-of-UIDs-of-root-nodes-for-which-somethig-was-set/removed-in-this-mutation>],
"setPatch": <TypePatch: the object that was received as the patch for set>,
"removePatch": <TypePatch: the object that was received as the patch for remove>
},
"delete": {
"rootUIDs": [<list-of-UIDs-of-root-nodes-which-were-deleted-in-this-mutation>]
}
}
}
Where:
- only one of add/update/delete would be present inside the
eventkey. -
$webhookis chosen as the value forresolveras no type in GraphQL can start with a$. - Initially, we will start by sending the
rootUIDsfor any kind of operation, along with sending outinputfor add mutations andpatchfor update mutations as shown above. Then later we may extend the capability, wherein the case of add mutations, theinputcan itself contain the UIDs for the root input nodes mapped correctly to each node. - The order of
rootUIDsisn’t the same as the order ofinputfor add mutations. - The initial guarantee will be that any event would be delivered at most once to the webhook. It might be improved in the future.
- Also, note that there is no guarantee as to the order in which the events will be delivered to the webhook, i.e., an update mutation that was committed after an add mutation may be delivered before the add mutation. One can use the
commitTsin such cases to figure out if they have seen a highercommitTsfor the samerootUID.
These events could be exposed to the user by adding lambda function like hooks in their lambda JS code, like this:
// If there is a handler registered for any specific operation on a type,
// the incoming webhook payloads for that type+operation pair will be forwarded to that handler
self.addWebHook({
"Type1.add": type1AddHandler,
"Type1.update": type1UpdateHandler,
"Type2.delete": type2DeleteHandler,
...
})
And, the signature of a webhook handler will look like:
function webHookHandler({event, dql, graphql, authHeader}) {
// user-defined code here
}
The choice of for which operations to send updates to webhook can be made by using the following new directive on a type:
"""
Type User has been configured to receive webhook events for add and delete mutations.
"""
type User @lambdaOnMutate(
add: true,
update: false,
delete: true
) {
id: ID!
name: String
# ... other fields
}
By default, no webhook events will be sent unless this directive overrides that behavior.
GOTCHA
A point to note is that doing DQL mutations won’t trigger these webhooks, these will be triggered only when someone does GraphQL mutations. That means, if the user is making GraphQL calls inside their webhook handlers, that would generate webhook events too, leading to recursion. For handling this scenario, we might consider the following approaches in a future release:
- Send a particular header in the GraphQL request to disable the webhook trigger. The GraphQL API currently exposed in lambda can do this by exposing a third boolean argument in the
graphql(query: String, variables: Object, disableWebhook: Boolean)call. The header may need to be like a JWT so that not anyone can send that header. Or, actually, this could just be part ofextensionsin the GraphQL request instead of being a header, but still like a JWT so that we are sure it hasn’t been meddled with. We can take the JWT signing information as webhook configuration in the schema. - The auto-generated mutations can have a flag to do the same. But, that would enable any client to disable webhook, which may not be intended.
The first approach seems suitable for this, open to discussions.
Note: At present, the user of webhooks is expected to handle this recursion scenario.
References
- None yet