We want to
- Measure GraphQL query/mutation performance and see if there are bits of code that we can improve. We would also like to see what is the overhead being added by query rewriting and processing apart from the time it takes to process the query by Dgraph.
- Load testing and getting P50, P95, P99 latencies for query/mutation performance.
Measure individual query/mutation performance
As a first step, we come up with 10-20 queries/mutations that would form the most common use case. We can use the suggestions by Harshil. These are the kind of queries we can look at
- getType/queryType - single level
- getType/queryType - multi level
- getType/queryType - single and multi-level for interfaces
- getType/queryType with auth - single and multi-level
- getType/queryType along with custom fields
- add/update mutations - single level with id/xid
- add/update mutations - deep with id/xid
These queries/mutations can be part of a benchmarking suite (we can use Go benchmarks for this) where each query/mutation would be run for 10-60 seconds.
- We can look at the memory/CPU profiles of Dgraph during these runs and figure out which bits are taking the most memory/time. Ideally these should be bits within dgraph that are required to process the GraphQL± query. If not, then we should look at the slow parts and optimize them.
- These benchmarking tests should also measure and tell us the ratio of time taken to pre and post-process the query against the total time to process the query. This would give us an idea of the overhead added to the query. We should aim to minimise this.
- auth queries should also check the overhead added by auth rules over normal queries.
https://github.com/tsenart/vegeta can be used for doing this part of the testing.
- We should do load testing using concurrent clients to measure P50, P95, P99 latencies of the queries mentioned above. For this, we can choose 10, 100, 1000 concurrent clients and see how the latency changes in response.
- Along with the queries above, we should also test subscriptions and see how many concurrent subscriptions are supported and how do they depend on the number of open file descriptors.
Once we get some initial latency/throughput numbers from our load testing, then we should find a way to store them and compare them across releases.