# Graphql Benchmark with Locust

**URL:** <https://discuss.dgraph.io/t/graphql-benchmark-with-locust/10817>\
**Category:** GraphQL\
**Tags:** benchmark\
**Created:** [October 7, 2020, 7:43am UTC](https://discuss.dgraph.io/t/graphql-benchmark-with-locust/10817 "2020-10-07T07:43:51Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![aman-bansal](https://yyz1.discourse-cdn.com/flex007/user_avatar/discuss.dgraph.io/aman-bansal/32/4475_2.png) [@aman-bansal](https://discuss.dgraph.io/u/aman-bansal)\
**Post date:** [October 7, 2020, 7:43am UTC](https://discuss.dgraph.io/t/graphql-benchmark-with-locust/10817/1 "2020-10-07T07:43:51Z")

</div>

Locust is a load testing framework designed to define user behaviors. More about locust can be found [here](https://docs.locust.io/en/stable/).

# Test Suite Setup

The test is performed from [this](https://github.com/dgraph-io/dgraph/commit/0f2807b7003f0a4983c4f5fa6be129c33c56070c) commit id.

The test repository is available [here](https://github.com/dgraph-io/dgraphWithLocust).

Data for restaurants was used for testing purposes. Here is the approximate data size initially:  
Restaurant: 8422  
Dishes: 4158700  
Cuisine: 116

The load test was done on a single server setup of dgraph with locust running for 100 users at a spawn rate of 10 users per sec. We can simulate the load in a distributed manner too.

For simulating queries, initial data was built with 8422 restaurants, 116 cuisines, and 10K dishes. After that random combinations are used to perform all the queries.

The total simulation time is 30 minutes. Memory, CPU, and goroutines have been profiled in a period of 30 seconds. The complete profiles list can be found here.

The tests were run after performing a set of random queries to initialize the random state of the server.

**During testing, I didn’t get any major memory spikes or CPU spikes during the simulation.**

# Simulation Without Auth

Schema can be found [here](https://github.com/dgraph-io/dgraphWithLocust/blob/main/schema.graphql).  
Queries and mutations are simulated separately.

## Query Simulation

#### Queries Used for searching restaurants:

```auto
query($name: String) {
  queryRestaurant(
    filter: {
      name: {
        allofterms: $name
      }
    }) {
    id
    name
    cuisines(first: 100) {
      name
      id
      dishes(first: 100) {
        id 
        name
      }
    }
  }
}

```

#### Queries used for Searching dishes

```auto
query ($name: String, $dishName: String, $restName: String) {
  queryDish(
    filter: {
      name: {
        allofterms: $dishName
      }
    }
  ) {
    id 
    name
    cuisine(filter:{
      name: { eq: $name }
    }) {
      name
      restaurants(filter: {
        name: { allofterms: $restName }
      }) {
        id
        name
      }
    }
  }
}

```

#### Queries to search cuisine

```auto
query($name: String, $dishName: String, $restName: String) {
  queryCuisine(
    filter: {
      name: {
        eq:$name
      }
    }
  ) {
    id
    name
    restaurants(filter:{
      name: {
        allofterms: $restName
      }
    }) {
      name
    }
    dishes (
      filter: {
        name: {
          allofterms: $dishName
        }
      },
      first: 100) {
      name
    }
  }
}

```

### Results

 ![Screenshot from 2020-10-07 00-53-53](https://canada1.discourse-cdn.com/flex007/uploads/dgraph/original/2X/8/8f58b957b23dda64138c9f5260e5044e34765b1e.png)

### Observations

Cpu Profile

 ![Screenshot 2020-10-07 at 1.17.33 AM](https://canada1.discourse-cdn.com/flex007/uploads/dgraph/original/2X/e/e3455779ee53f566951a23007c830cbd15fb9775.png)

[Graph](https://drive.google.com/file/d/1YbIXbmkgoApCKGKBuHmys58nVCh0WUsY/view?usp=sharing)

Memory Profile

 ![32c1ce8f27b2a3850e500f9d011d437b1ec183f8](https://canada1.discourse-cdn.com/flex007/uploads/dgraph/original/2X/0/02b0df9eb0d5860ec3cd1160e7a6770aedde173a.png)

[Graph](https://drive.google.com/file/d/1cztJH3E-vxtOl9nzY7Wot1Fl8DgWnlmT/view?usp=sharing)

## Mutation simulation

#### mutation to add restaurant

```auto
mutation(
  $lat: Float!
  $long: Float!
  $address: String!
  $locality: String!
  $cityId: String!
  $countryId: String!
  $countryname: String!
  $restName: String!
  $rating: Float!
  $costFor2: Float!
  $currency: String!
  $cuisineName: String!
  
  $dishName: String!
  $dishPic: String!
  $dishPrice: Float!
  $description: String!
  $isVeg: Boolean!
  $zipcode: Int!
  
) {
  addRestaurant(
    input: [
      {
        dishes: [{
          name: $dishName
          pic: $dishPic
          price: $dishPrice
          description: $description
          isVeg: $isVeg
          
        }]
        addr: {
          lat: $lat
          long: $long
          address: $address
          locality: $locality
          city: {
          	id: $cityId
            name: $cityId
            country: {
              id: $countryId
              name: $countryname
            }
          }
          zipcode: $zipcode
          
        }
        name: $restName
        xid: $restName
        createdAt: 1234567890
        rating: $rating
        costFor2: $costFor2
        currency: $currency
        cuisines: [{
          name: $cuisineName
        }]
      }
    ]
  ) {
    restaurant {
      id
    }
  }
}

```

#### mutation to add dish

```auto
mutation($dishName: String!, $price: Float!, $isVeg: Boolean!, $restId: ID!) {
  addDish(
    input: [
      {
        name: $dishName
        price: $price
        isVeg: $isVeg
        servedBy: {
          id: $restId
        }
      }
    ]
  ) {
    dish {
      id
    }
  }
}

```

#### mutation to add cuisine

```auto
mutation($cuisineName: String!) {
  addCuisine(
    input: [
      {
        name: $cuisineName
      }
    ]
  ) {
    cuisine {
      id
    }
  }
}

```

### Results

 ![Screenshot from 2020-10-06 21-46-58](https://canada1.discourse-cdn.com/flex007/uploads/dgraph/original/2X/1/1d25ada0255e474d4bc703f77720b14e5b4744fb.png)

### Observations

Cpu Profile

 ![Screenshot 2020-10-07 at 1.38.30 AM](https://canada1.discourse-cdn.com/flex007/uploads/dgraph/original/2X/d/d89e44d03af35bde48425a5e807e1b2594c712d2.png)

[Graph](https://drive.google.com/file/d/12txzg1EUsFnnL-180yuHDT-Pwr86XgVu/view?usp=sharing)

Memory Profile

 ![cba24286448ab13379abfe5df784830e68c1cf13](https://canada1.discourse-cdn.com/flex007/uploads/dgraph/original/2X/c/cb0c55c9e5996dfd0c6d112fa572ac47584e78a0.png)

[Graph](https://drive.google.com/file/d/1Ux3fNqjdBLv_e8nak-LUycGaDC7y4Uys/view?usp=sharing)

# Simulation with Auth

Schema can be found [here](https://github.com/dgraph-io/dgraphWithLocust/blob/main/schema_auth.graphql). Queries and mutations are simulated separately.

## Query Simulation

Six queries are used. 3 with admin role are same as the one mentioned in without auth and 3 with restaurant scope.

#### Queries with restaurant scope

Query to search restaurant

```auto
query {
 queryRestaurant{
   id
   name
   cuisines(first: 10) {
     name
     id
   }
 }
}

```

Query to search Dish

```auto
query ($name: String, $dishName: String) {
  queryDish(
    filter: {
      name: {
        allofterms: $dishName
      }
    }
  ) {
    id 
    name
    cuisine(filter:{
      name: { eq: $name }
    }) {
      name
      restaurants {
        id
        name
      }
    }
  }
}

```

Query to search Cuisine

```auto
query($name: String, $dishName: String) {
  queryCuisine(
    filter: {
      name: {
        eq:$name
      }
    }
  ) {
    id
    name
    restaurants {
      id
      name
    }
    dishes (
      filter: {
        name: {
          allofterms: $dishName
        }
      },
      first: 100) {
      name
    }
  }
}

```

### Results

 ![Screenshot from 2020-10-07 00-54-38](https://canada1.discourse-cdn.com/flex007/uploads/dgraph/original/2X/5/51b64728e5c80c985aaa4ca75939bbf87c3a003d.png)

### Observations

Cpu Profile

 ![Screenshot 2020-10-07 at 1.42.18 AM](https://canada1.discourse-cdn.com/flex007/uploads/dgraph/original/2X/a/ae7551eef39d208a37ed39283acdadded6bbbacd.png)

[Graph](https://drive.google.com/file/d/1tcWdGleN_7b-2vOYaWrldY1ID2rOZlGl/view?usp=sharing)

Memory Profile

 ![Screenshot 2020-10-07 at 11.55.07 AM](https://canada1.discourse-cdn.com/flex007/uploads/dgraph/original/2X/9/9bf8863e87ec76546faa63fcfa2dce96bfd73d91.png)

[Graph](https://drive.google.com/file/d/1izq5-uDhjvEbBdJ29b5V24puUTgHKTMd/view?usp=sharing)

## Mutations with Auth

mutations are performed with the admin role. mutations queries are the same as the one used without the auth scope.

### Results

 ![Screenshot from 2020-10-06 21-44-07](https://canada1.discourse-cdn.com/flex007/uploads/dgraph/original/2X/3/36c34aa9489f9e0a1c974cb6825a588fb01f414f.png)

### Observations

Cpu Profile

 ![Screenshot 2020-10-07 at 1.44.12 AM](https://canada1.discourse-cdn.com/flex007/uploads/dgraph/original/2X/6/663c267e4efe95802d9c0c0cb793ab609b555610.png)

[Graph](https://drive.google.com/file/d/1Ev7GT9N9Zro_5wpF6Iq1HdTA-CFDY9FW/view?usp=sharing)

Memory Profile

 ![Screenshot 2020-10-07 at 1.00.43 PM](https://canada1.discourse-cdn.com/flex007/uploads/dgraph/original/2X/3/3d3b1e9af8f67819aa6b511d86987849790ca254.png)

[Graph](https://drive.google.com/file/d/167wspiU2Y0GPUFKgkPPe4u7tPtUD2zFX/view?usp=sharing)

@pawan @abhimanyusinghgaur let’s review this and see if we can add more test cases to this.

---

<div class="post-metadata">

**Author:** ![vvbalaji](https://yyz1.discourse-cdn.com/flex007/user_avatar/discuss.dgraph.io/vvbalaji/32/3091_2.png) [@vvbalaji](https://discuss.dgraph.io/u/vvbalaji)\
**Post date:** [October 15, 2020, 1:52pm UTC](https://discuss.dgraph.io/t/graphql-benchmark-with-locust/10817/2 "2020-10-15T13:52:26Z")

</div>

How often do we plan to run this? How do we plan to use this on a ongoing basis?

---

<div class="post-metadata">

**Author:** ![aman-bansal](https://yyz1.discourse-cdn.com/flex007/user_avatar/discuss.dgraph.io/aman-bansal/32/4475_2.png) [@aman-bansal](https://discuss.dgraph.io/u/aman-bansal)\
**Post date:** [October 15, 2020, 3:22pm UTC](https://discuss.dgraph.io/t/graphql-benchmark-with-locust/10817/3 "2020-10-15T15:22:51Z")

</div>

Nothing is planned as of yet. But one suggestion is we could make it part of release testing. Though would require few changes to automate it.

@pawan do we have anything planned to make it part of our regular testing?

---

<div class="post-metadata">

**Author:** ![aman-bansal](https://yyz1.discourse-cdn.com/flex007/user_avatar/discuss.dgraph.io/aman-bansal/32/4475_2.png) [@aman-bansal](https://discuss.dgraph.io/u/aman-bansal)\
**Post date:** [October 19, 2020, 7:04am UTC](https://discuss.dgraph.io/t/graphql-benchmark-with-locust/10817/4 "2020-10-19T07:04:24Z")

</div>

# Test Scenario: Trends with different number of users

For this simulation number of users has been varied from 10, 20, 50 and 100. Results and memory/cpu profiling has been added below. It can observed from the results that there is no unexpected surge in response time or memory/cpu usage.

## Number of Users = 10

### Query Simulation without Auth

#### Results:

 ![Screenshot 2020-10-19 at 11.31.54 AM](https://canada1.discourse-cdn.com/flex007/uploads/dgraph/original/2X/b/be8d6505ab9bc66bc5e239969a0ac8d7943fe9a4.png)

#### Observations:

CPU Profile

 ![Screenshot 2020-10-19 at 12.05.00 PM](https://canada1.discourse-cdn.com/flex007/uploads/dgraph/original/2X/f/fdb6556c1b9fb049e79503cc87eff6fd953b397b.png)  
[[Graph](https://drive.google.com/file/d/182MTK1HvGC2tOAddnHuAr1zeYchHIkFq/view?usp=sharing)]

Memory Profile

 ![Screenshot 2020-10-19 at 12.06.41 PM](https://canada1.discourse-cdn.com/flex007/uploads/dgraph/original/2X/5/538137c4cf59faa5e78a9af887407a5d198418a9.png)  
[[Graph](https://drive.google.com/file/d/1YJD2WDoOFPUy0GXFPBwkqLTjsmQdwDLg/view?usp=sharing)]

## Number of Users = 20

### Query Simulation without Auth

#### Results

 ![Screenshot 2020-10-19 at 12.09.56 PM](https://canada1.discourse-cdn.com/flex007/uploads/dgraph/original/2X/5/588f0a5096f8487cc92b6a84d581d4c5264fda88.png)

#### Observations

CPU Profile

 ![Screenshot 2020-10-19 at 12.13.15 PM](https://canada1.discourse-cdn.com/flex007/uploads/dgraph/original/2X/3/3d88bb3cd85e94156aced4935035fa94cd9e27bd.png)  
[[Graph](https://drive.google.com/file/d/10MNkGyJEmvcuTU_uB4OjiMGO1_DAhAy9/view?usp=sharing)]

Memory Profile

 ![Screenshot 2020-10-19 at 12.08.05 PM](https://canada1.discourse-cdn.com/flex007/uploads/dgraph/original/2X/3/37666750c13c92dfdf67ff00f7afe29bb7753bdc.png)  
[[Graph](https://drive.google.com/file/d/1MyRoDwEQSiCKytHQrGTmPnrnvgfmCBaj/view?usp=sharing)]

## Number of Users = 50

### Query Simulation without Auth

#### Results

 ![Screenshot 2020-10-19 at 12.33.21 PM](https://canada1.discourse-cdn.com/flex007/uploads/dgraph/original/2X/f/ffd7dae5675ef52a66fca5943ca77e7a2061648a.png)

CPU Profile

 ![Screenshot 2020-10-19 at 12.16.58 PM](https://canada1.discourse-cdn.com/flex007/uploads/dgraph/original/2X/8/826102c32c3ebf6c38e62836512d4fcc844e7b00.png)  
[[Graph](https://drive.google.com/file/d/1rUnrgjQOtbxPy-4vbiJQohQqWkr3P_yn/view?usp=sharing)]

Memory Profile

 ![Screenshot 2020-10-19 at 12.14.51 PM](https://canada1.discourse-cdn.com/flex007/uploads/dgraph/original/2X/3/32461627d259e431196bf11227ceb25473977ac6.png)  
[[Graph](https://drive.google.com/file/d/1ZvDDI1KyjJwxrzAC0edrZUqqhSDBx670/view?usp=sharing)]

---

<div class="post-metadata">

**Author:** ![aman-bansal](https://yyz1.discourse-cdn.com/flex007/user_avatar/discuss.dgraph.io/aman-bansal/32/4475_2.png) [@aman-bansal](https://discuss.dgraph.io/u/aman-bansal)\
**Post date:** [October 27, 2020, 3:45pm UTC](https://discuss.dgraph.io/t/graphql-benchmark-with-locust/10817/5 "2020-10-27T15:45:26Z")

</div>

I talked to @pawan , and we believe that it should be part of DQL layer first. We would like to monitor the latency patterns and then extend it to the GraphQL layer.

@vvbalaji how should we proceed with this?