# Processed Queries constantly increasing **URL:** \ **Category:** Dgraph\ **Tags:** kind:question\ **Created:** [April 11, 2021, 2:39pm UTC](https://discuss.dgraph.io/t/processed-queries-constantly-increasing/13638 "2021-04-11T14:39:20Z")\ **Posts on this page:** 20\ **Page:** 1
**Author:** ![marcown](https://yyz1.discourse-cdn.com/flex007/user_avatar/discuss.dgraph.io/marcown/32/6069_2.png) [@marcown](https://discuss.dgraph.io/u/marcown)\ **Post date:** [April 11, 2021, 2:39pm UTC](https://discuss.dgraph.io/t/processed-queries-constantly-increasing/13638/1 "2021-04-11T14:39:21Z")
Hey there, I installed dgraph + prometheus + grafana in a docker-compose stack and posted a schema. The “processed queries” metrics is constantly increasing (see picture below) without doing anything (I do no queries and no mutations). Is that a normal behavior? If yes, what is it doing in the background? ![queries](https://canada1.discourse-cdn.com/flex007/uploads/dgraph/original/2X/e/e8eca31cd4ffee35c93710ec50514a2a0dfc8616.png) ---
**Author:** ![chewxy](https://yyz1.discourse-cdn.com/flex007/user_avatar/discuss.dgraph.io/chewxy/32/4339_2.png) [@chewxy](https://discuss.dgraph.io/u/chewxy)\ **Post date:** [April 11, 2021, 11:56pm UTC](https://discuss.dgraph.io/t/processed-queries-constantly-increasing/13638/2 "2021-04-11T23:56:23Z")
Hey @marcown What version of dgraph are you using? It seems you are doing a lot of queries. ---
**Author:** ![marcown](https://yyz1.discourse-cdn.com/flex007/user_avatar/discuss.dgraph.io/marcown/32/6069_2.png) [@marcown](https://discuss.dgraph.io/u/marcown)\ **Post date:** [April 12, 2021, 5:10am UTC](https://discuss.dgraph.io/t/processed-queries-constantly-increasing/13638/3 "2021-04-12T05:10:12Z")
Hey @chewxy I am using 20.11.2. I am doing nothing with the setup above (no queries / mutations initiated by me). There are some custom queries and mutations in the schema connecting to another graphql server. I will try to setup an example later today ---
**Author:** ![marcown](https://yyz1.discourse-cdn.com/flex007/user_avatar/discuss.dgraph.io/marcown/32/6069_2.png) [@marcown](https://discuss.dgraph.io/u/marcown)\ **Post date:** [April 13, 2021, 8:58am UTC](https://discuss.dgraph.io/t/processed-queries-constantly-increasing/13638/4 "2021-04-13T08:58:30Z")
@chewxy I think the reason is the following: we use a lot of subscriptions e.g. we use dgraph like this docker-compose.yml ```auto version: '3.7' volumes: dgraph_data: prometheus_data: grafana_data: services: zero: image: dgraph/dgraph:v20.11.2 volumes: - dgraph_data:/dgraph ports: - 5080:5080 - 6080:6080 restart: on-failure command: dgraph zero --my=zero:5080 alpha: image: dgraph/dgraph:v20.11.2 volumes: - dgraph_data:/dgraph ports: - 8080:8080 - 9080:9080 restart: on-failure command: dgraph alpha --whitelist 0.0.0.0/0 --my=alpha:7080 --zero=zero:5080 ``` dgraph.graphql ```auto type Product @withSubscription { productID: ID! name: String @search(by: [term]) reviews: [Review] @hasInverse(field: about) } type Customer @withSubscription { username: String! @id @search(by: [hash, regexp]) reviews: [Review] @hasInverse(field: by) } type Review @withSubscription { id: ID! about: Product! by: Customer! comment: String @search(by: [fulltext]) rating: Int @search } ``` post with ```auto curl -X POST localhost:8080/admin/schema --data-binary '@dgraph.graphql' ``` subscribe to ws://localhost:8080/graphql (e.g. using altair client): ```graphql subscription { queryProduct { name } } ``` and check total queries: ```auto watch -n 1 curl 'localhost:8080/debug/prometheus_metrics |grep "dgraph_num_queries_total"' ``` example grafana graph: ![processed2](https://canada1.discourse-cdn.com/flex007/uploads/dgraph/original/2X/d/d1ace14ce0deba51cea3f7fd1d078a34e3ae37e8.png) shouldn’t subscriptions do the opposite? less queries and only push changes? For big databases this is really bad ---
**Author:** ![maaft](https://avatars.discourse-cdn.com/v4/letter/m/4af34b/32.png) [@maaft](https://discuss.dgraph.io/u/maaft)\ **Post date:** [April 13, 2021, 9:17am UTC](https://discuss.dgraph.io/t/processed-queries-constantly-increasing/13638/5 "2021-04-13T09:17:52Z")
@chewxy any chance that GraphQL subscriptions are translated into DQL polling? When I think about a chat application with 10k users and everyone is subscribing to a chat, this would mean that the advertised 10k QPS of Dgraph is maxed out easily. Probably with a lot less users, when “standard” queries and mutations are added to that. If that is indeed true (DQL polling), are event-driven subscriptions on the roadmap yet? This would be very important for medium- to large-scale applications in my opinion. ---
**Author:** ![chewxy](https://yyz1.discourse-cdn.com/flex007/user_avatar/discuss.dgraph.io/chewxy/32/4339_2.png) [@chewxy](https://discuss.dgraph.io/u/chewxy)\ **Post date:** [April 13, 2021, 11:35pm UTC](https://discuss.dgraph.io/t/processed-queries-constantly-increasing/13638/6 "2021-04-13T23:35:29Z")
I think @pawan was working on something adjacent to this ---
**Author:** ![marcown](https://yyz1.discourse-cdn.com/flex007/user_avatar/discuss.dgraph.io/marcown/32/6069_2.png) [@marcown](https://discuss.dgraph.io/u/marcown)\ **Post date:** [April 14, 2021, 8:49am UTC](https://discuss.dgraph.io/t/processed-queries-constantly-increasing/13638/7 "2021-04-14T08:49:01Z")
@chewxy So, at the moment the polling is expected behavior for subscriptions? ---
**Author:** ![pawan](https://yyz1.discourse-cdn.com/flex007/user_avatar/discuss.dgraph.io/pawan/32/1946_2.png) [@pawan](https://discuss.dgraph.io/u/pawan)\ **Post date:** [April 14, 2021, 11:48pm UTC](https://discuss.dgraph.io/t/processed-queries-constantly-increasing/13638/8 "2021-04-14T23:48:30Z")
> [@marcown](#): > > So, at the moment the polling is expected behavior for subscriptions? Yes, that is expected behavior for subscriptions. We also have event-driven lambda webhooks that you can set up. These get triggered anytime there is an add/update or delete mutation that happens. Have you had a look at [Lambda Webhooks - GraphQL](https://dgraph.io/docs/graphql/lambda/webhook/) yet? ---
**Author:** ![chewxy](https://yyz1.discourse-cdn.com/flex007/user_avatar/discuss.dgraph.io/chewxy/32/4339_2.png) [@chewxy](https://discuss.dgraph.io/u/chewxy)\ **Post date:** [April 15, 2021, 12:06am UTC](https://discuss.dgraph.io/t/processed-queries-constantly-increasing/13638/9 "2021-04-15T00:06:11Z")
I think @marcown was talking about DQL polling - if that is exposed as an API to the end user. It’d be very useful for “subscriptions” in the DQL world, and I recalled you were working on something like that ---
**Author:** ![pawan](https://yyz1.discourse-cdn.com/flex007/user_avatar/discuss.dgraph.io/pawan/32/1946_2.png) [@pawan](https://discuss.dgraph.io/u/pawan)\ **Post date:** [April 15, 2021, 1:06am UTC](https://discuss.dgraph.io/t/processed-queries-constantly-increasing/13638/10 "2021-04-15T01:06:38Z")
Yeah, you can subscribe to custom DQL queries as well. Here is an example on how to do that [https://dgraph.io/docs/graphql/subscriptions/#subscriptions-to-custom-dql](https://dgraph.io/docs/graphql/subscriptions/#subscriptions-to-custom-dql) ---
**Author:** ![marcown](https://yyz1.discourse-cdn.com/flex007/user_avatar/discuss.dgraph.io/marcown/32/6069_2.png) [@marcown](https://discuss.dgraph.io/u/marcown)\ **Post date:** [April 15, 2021, 7:19am UTC](https://discuss.dgraph.io/t/processed-queries-constantly-increasing/13638/11 "2021-04-15T07:19:37Z")
@pawan My problem is the following: As the number of subscriptions is increasing the number of internal DQL Polling requests per second are increasing. It would be better to have subscriptions only event driven, this would significantly reduce the total number of queries per second → less overhead as @maaft wrote Is this on the roadmap yet? ---
**Author:** ![pawan](https://yyz1.discourse-cdn.com/flex007/user_avatar/discuss.dgraph.io/pawan/32/1946_2.png) [@pawan](https://discuss.dgraph.io/u/pawan)\ **Post date:** [April 15, 2021, 7:48am UTC](https://discuss.dgraph.io/t/processed-queries-constantly-increasing/13638/12 "2021-04-15T07:48:49Z")
I understand, that’s not on the roadmap for now. You could also look into increasing the polling interval if the number of queries per second is a concern or use the lambda webhook which are event driven. ---
**Author:** ![marcown](https://yyz1.discourse-cdn.com/flex007/user_avatar/discuss.dgraph.io/marcown/32/6069_2.png) [@marcown](https://discuss.dgraph.io/u/marcown)\ **Post date:** [April 15, 2021, 8:39am UTC](https://discuss.dgraph.io/t/processed-queries-constantly-increasing/13638/13 "2021-04-15T08:39:04Z")
Is there an easy way to reduce the polling interval? EDIT: I found --graphql\_poll\_interval 1 on alpha nodes, is possible to set it per subscription? ---
**Author:** ![maaft](https://avatars.discourse-cdn.com/v4/letter/m/4af34b/32.png) [@maaft](https://discuss.dgraph.io/u/maaft)\ **Post date:** [April 18, 2021, 5:44pm UTC](https://discuss.dgraph.io/t/processed-queries-constantly-increasing/13638/14 "2021-04-18T17:44:34Z")
@pawan what about my chat example with 10k users? > When I think about a chat application with 10k users and everyone is subscribing to a chat, this would mean that the advertised 10k QPS of Dgraph is maxed out easily. Probably with a lot less users, when “standard” queries and mutations are added to that. Am I correct with this assumption? ---
**Author:** ![mrjn](https://yyz1.discourse-cdn.com/flex007/user_avatar/discuss.dgraph.io/mrjn/32/775_2.png) [@mrjn](https://discuss.dgraph.io/u/mrjn)\ **Post date:** [April 18, 2021, 9:05pm UTC](https://discuss.dgraph.io/t/processed-queries-constantly-increasing/13638/15 "2021-04-18T21:05:29Z")
Yeah. We need to build an event driven system here. That was on our todo list. ---
**Author:** ![maaft](https://avatars.discourse-cdn.com/v4/letter/m/4af34b/32.png) [@maaft](https://discuss.dgraph.io/u/maaft)\ **Post date:** [April 19, 2021, 7:21am UTC](https://discuss.dgraph.io/t/processed-queries-constantly-increasing/13638/16 "2021-04-19T07:21:53Z")
Great to hear! In the meantime I guess we could increase throughput by deploying more alphas? ---
**Author:** ![mrjn](https://yyz1.discourse-cdn.com/flex007/user_avatar/discuss.dgraph.io/mrjn/32/775_2.png) [@mrjn](https://discuss.dgraph.io/u/mrjn)\ **Post date:** [April 19, 2021, 3:56pm UTC](https://discuss.dgraph.io/t/processed-queries-constantly-increasing/13638/17 "2021-04-19T15:56:00Z")
Yup, you could do that. Also, not sure if we’re doing best effort queries or not for the streaming. That’s the other thing we should do. CC: @hardik ---
**Author:** ![maaft](https://avatars.discourse-cdn.com/v4/letter/m/4af34b/32.png) [@maaft](https://discuss.dgraph.io/u/maaft)\ **Post date:** [April 20, 2021, 10:57am UTC](https://discuss.dgraph.io/t/processed-queries-constantly-increasing/13638/18 "2021-04-20T10:57:09Z")
Out of curiosity: What happens when this happens in-between the DQL polling interval? The initial state is: `Foo.name` = “Bob” ---- polling paused ---- 1. Set `Foo.name` = “Alice” 2. Set `Foo.name` = “Bob” ---- polling continued ---- Are you using a diff to check if you need to trigger a subscription event? If yes, one need to be very careful setting the DQL pollling interval to large values. ---
**Author:** ![mrjn](https://yyz1.discourse-cdn.com/flex007/user_avatar/discuss.dgraph.io/mrjn/32/775_2.png) [@mrjn](https://discuss.dgraph.io/u/mrjn)\ **Post date:** [April 20, 2021, 2:11pm UTC](https://discuss.dgraph.io/t/processed-queries-constantly-increasing/13638/19 "2021-04-20T14:11:17Z")
The way subscriptions are working right now – they’d run the query, and compare the results with a cached version (or a hash I think) of the previous result. If there’s a difference, that result would be sent out. Otherwise, nothing gets sent. ---
**Author:** ![maaft](https://avatars.discourse-cdn.com/v4/letter/m/4af34b/32.png) [@maaft](https://discuss.dgraph.io/u/maaft)\ **Post date:** [April 20, 2021, 2:16pm UTC](https://discuss.dgraph.io/t/processed-queries-constantly-increasing/13638/20 "2021-04-20T14:16:45Z")
Okay, thanks for clarification. We definitely need event driven subscriptions then or dgraph users will run into performance issues when they scale their apps. [Next page](https://discuss.dgraph.io/t/processed-queries-constantly-increasing/13638.md?page=2)