Auth directives cripling database speed

Here is some usage statistics about how auth directives are slowing down our quries sometimes by a factor of 4,500% Yikes!

I have a query I run for an admin vs. a standard user. For admin it takes 197 ms which is really great! I have been super happy until I ran it as a user and the same query took 9,970 ms.

When initially comparing Dgraph to Neo4j, Neo4j devs discussed how that adding authentication and rules actually made their queries faster because it reduced the graph size. However I think that is done through ACL and not as I am doing with the auth directive.

I am trying to refigure how I am applying these “security” rules and ways to simplify them. Here is an example of the current rule:

type Contact @auth(
  query: { or: [
    # allows admins to see all contacts
    { rule: "{$USERROLE: { eq: \"ADMINISTRATOR\"}}"},
    # allows visibility to public churches to the world
    { rule: "query { queryContact (filter: {isPublic: true and: {isType: {eq: church}}}) { id } }" },
    # allows visibility to public businesses to the world
    { rule: "query { queryContact (filter: {isPublic: true and: {isType: {eq: business}}}) { id } }" },
    # allows visibility to public boards to user
    { and: [
      { rule: "{$USERROLE: { eq: \"USER\"}}"},
      { rule: "query { queryContact (filter: {isPublic: true and: {isType: {eq: church}}}) { id } }" }
    ]},
    # allows visibility to public schools to user
    { and: [
      { rule: "{$USERROLE: { eq: \"USER\"}}"},
      { rule: "query { queryContact (filter: {isPublic: true and: {isType: {eq: school}}}) { id } }" }
    ]},
    # allows visibility to public colleges to user
    { and: [
      { rule: "{$USERROLE: { eq: \"USER\"}}"},
      { rule: "query { queryContact (filter: {isPublic: true and: {isType: {eq: college}}}) { id } }" }
    ]},
    # allows visibility to public groups to user
    { and: [
      { rule: "{$USERROLE: { eq: \"USER\"}}"},
      { rule: "query { queryContact (filter: {isPublic: true and: {isType: {eq: group}}}) { id } }" }
    ]},
    # TODO allows visibility to public contacts if staff of a contact that is visible
    # { rule: "{$USERROLE: { eq: \"USER\"}}"},
    # allows access to user contacts
    { and: [
      { rule: "{$USERROLE: { eq: \"USER\"}}"},
      { rule: "query { queryContact (filter: {isPublic: true and: {isType: {eq: group}}}) { id } }" }
    ]},
    # allow users to see contact where they are a owner
    { rule: "query ($USERNAME: String!) { queryContact { hasOwners(filter: {isType: {eq: person}}) { isUser(filter: {username: {eq: $USERNAME}}) { username } } } }" },
    # allow users to see contact where they are a moderator
    { rule: "query ($USERNAME: String!) { queryContact { hasModerators(filter: {isType: {eq: person}}) { isUser(filter: {username: {eq: $USERNAME}}) { username } } } }" },
    # allow users to see contact where they are a viewer
    { rule: "query ($USERNAME: String!) { queryContact { hasViewers(filter: {isType: {eq: person}}) { isUser(filter: {username: {eq: $USERNAME}}) { username } } } }" },
    # allow users to see contact-church where they have group access to own
    { rule: "query ($USERNAME: String!) { queryContact(filter: { isType: { eq: church } }) { hasOwners { isGroup { hasGrantedAccess { hasRights( filter: { isTrue: true, and: { name: { eq: canViewChurch } } } ) { id } user(filter: { username: { eq: $USERNAME } }) { username } } } } } }" },
    # allow users to see contact-person where they have group access to own
    { rule: "query ($USERNAME: String!) { queryContact(filter: { isType: { eq: person } }) { hasOwners { isGroup { hasGrantedAccess { hasRights( filter: { isTrue: true, and: { name: { eq: canViewPerson } } } ) { id } user(filter: { username: { eq: $USERNAME } }) { username } } } } } }" },
    # allow users to see contact-board where they have group access to own
    { rule: "query ($USERNAME: String!) { queryContact(filter: { isType: { eq: board } }) { hasOwners { isGroup { hasGrantedAccess { hasRights( filter: { isTrue: true, and: { name: { eq: canViewBoard } } } ) { id } user(filter: { username: { eq: $USERNAME } }) { username } } } } } }" },
    # allow users to see contact-business where they have group access to own
    { rule: "query ($USERNAME: String!) { queryContact(filter: { isType: { eq: business } }) { hasOwners { isGroup { hasGrantedAccess { hasRights( filter: { isTrue: true, and: { name: { eq: canViewBusiness } } } ) { id } user(filter: { username: { eq: $USERNAME } }) { username } } } } } }" },
    # allow users to see contact-class where they have group access to own
    { rule: "query ($USERNAME: String!) { queryContact(filter: { isType: { eq: class } }) { hasOwners { isGroup { hasGrantedAccess { hasRights( filter: { isTrue: true, and: { name: { eq: canViewClass } } } ) { id } user(filter: { username: { eq: $USERNAME } }) { username } } } } } }" },
    # allow users to see contact-school where they have group access to own
    { rule: "query ($USERNAME: String!) { queryContact(filter: { isType: { eq: school } }) { hasOwners { isGroup { hasGrantedAccess { hasRights( filter: { isTrue: true, and: { name: { eq: canViewSchool } } } ) { id } user(filter: { username: { eq: $USERNAME } }) { username } } } } } }" },
    # allow users to see contact-college where they have group access to own
    { rule: "query ($USERNAME: String!) { queryContact(filter: { isType: { eq: college } }) { hasOwners { isGroup { hasGrantedAccess { hasRights( filter: { isTrue: true, and: { name: { eq: canViewCollege } } } ) { id } user(filter: { username: { eq: $USERNAME } }) { username } } } } } }" },
    # allow users to see contact-group where they have group access to own
    { rule: "query ($USERNAME: String!) { queryContact(filter: { isType: { eq: group } }) { hasOwners { isGroup { hasGrantedAccess { hasRights( filter: { isTrue: true, and: { name: { eq: canViewGroup } } } ) { id } user(filter: { username: { eq: $USERNAME } }) { username } } } } } }" },
    # allow users to see contact-route where they have group access to own
    { rule: "query ($USERNAME: String!) { queryContact(filter: { isType: { eq: route } }) { hasOwners { isGroup { hasGrantedAccess { hasRights( filter: { isTrue: true, and: { name: { eq: canViewRoute } } } ) { id } user(filter: { username: { eq: $USERNAME } }) { username } } } } } }" },
    # allow users to see contact-church where they have group access to moderate
    { rule: "query ($USERNAME: String!) { queryContact(filter: { isType: { eq: church } }) { hasModerators { isGroup { hasGrantedAccess { hasRights( filter: { isTrue: true, and: { name: { eq: canViewChurch } } } ) { id } user(filter: { username: { eq: $USERNAME } }) { username } } } } } }" },
    # allow users to see contact-person where they have group access to moderate
    { rule: "query ($USERNAME: String!) { queryContact(filter: { isType: { eq: person } }) { hasModerators { isGroup { hasGrantedAccess { hasRights( filter: { isTrue: true, and: { name: { eq: canViewPerson } } } ) { id } user(filter: { username: { eq: $USERNAME } }) { username } } } } } }" },
    # allow users to see contact-board where they have group access to moderate
    { rule: "query ($USERNAME: String!) { queryContact(filter: { isType: { eq: board } }) { hasModerators { isGroup { hasGrantedAccess { hasRights( filter: { isTrue: true, and: { name: { eq: canViewBoard } } } ) { id } user(filter: { username: { eq: $USERNAME } }) { username } } } } } }" },
    # allow users to see contact-business where they have group access to moderate
    { rule: "query ($USERNAME: String!) { queryContact(filter: { isType: { eq: business } }) { hasModerators { isGroup { hasGrantedAccess { hasRights( filter: { isTrue: true, and: { name: { eq: canViewBusiness } } } ) { id } user(filter: { username: { eq: $USERNAME } }) { username } } } } } }" },
    # allow users to see contact-class where they have group access to moderate
    { rule: "query ($USERNAME: String!) { queryContact(filter: { isType: { eq: class } }) { hasModerators { isGroup { hasGrantedAccess { hasRights( filter: { isTrue: true, and: { name: { eq: canViewClass } } } ) { id } user(filter: { username: { eq: $USERNAME } }) { username } } } } } }" },
    # allow users to see contact-school where they have group access to moderate
    { rule: "query ($USERNAME: String!) { queryContact(filter: { isType: { eq: school } }) { hasModerators { isGroup { hasGrantedAccess { hasRights( filter: { isTrue: true, and: { name: { eq: canViewSchool } } } ) { id } user(filter: { username: { eq: $USERNAME } }) { username } } } } } }" },
    # allow users to see contact-college where they have group access to moderate
    { rule: "query ($USERNAME: String!) { queryContact(filter: { isType: { eq: college } }) { hasModerators { isGroup { hasGrantedAccess { hasRights( filter: { isTrue: true, and: { name: { eq: canViewCollege } } } ) { id } user(filter: { username: { eq: $USERNAME } }) { username } } } } } }" },
    # allow users to see contact-group where they have group access to moderate
    { rule: "query ($USERNAME: String!) { queryContact(filter: { isType: { eq: group } }) { hasModerators { isGroup { hasGrantedAccess { hasRights( filter: { isTrue: true, and: { name: { eq: canViewGroup } } } ) { id } user(filter: { username: { eq: $USERNAME } }) { username } } } } } }" },
    # allow users to see contact-route where they have group access to moderate
    { rule: "query ($USERNAME: String!) { queryContact(filter: { isType: { eq: route } }) { hasModerators { isGroup { hasGrantedAccess { hasRights( filter: { isTrue: true, and: { name: { eq: canViewRoute } } } ) { id } user(filter: { username: { eq: $USERNAME } }) { username } } } } } }" },
    # allow users to see contact-church where they have group access to view
    { rule: "query ($USERNAME: String!) { queryContact(filter: { isType: { eq: church } }) { hasViewers { isGroup { hasGrantedAccess { hasRights( filter: { isTrue: true, and: { name: { eq: canViewChurch } } } ) { id } user(filter: { username: { eq: $USERNAME } }) { username } } } } } }" },
    # allow users to see contact-person where they have group access to view
    { rule: "query ($USERNAME: String!) { queryContact(filter: { isType: { eq: person } }) { hasViewers { isGroup { hasGrantedAccess { hasRights( filter: { isTrue: true, and: { name: { eq: canViewPerson } } } ) { id } user(filter: { username: { eq: $USERNAME } }) { username } } } } } }" },
    # allow users to see contact-board where they have group access to view
    { rule: "query ($USERNAME: String!) { queryContact(filter: { isType: { eq: board } }) { hasViewers { isGroup { hasGrantedAccess { hasRights( filter: { isTrue: true, and: { name: { eq: canViewBoard } } } ) { id } user(filter: { username: { eq: $USERNAME } }) { username } } } } } }" },
    # allow users to see contact-business where they have group access to view
    { rule: "query ($USERNAME: String!) { queryContact(filter: { isType: { eq: business } }) { hasViewers { isGroup { hasGrantedAccess { hasRights( filter: { isTrue: true, and: { name: { eq: canViewBusiness } } } ) { id } user(filter: { username: { eq: $USERNAME } }) { username } } } } } }" },
    # allow users to see contact-class where they have group access to view
    { rule: "query ($USERNAME: String!) { queryContact(filter: { isType: { eq: class } }) { hasViewers { isGroup { hasGrantedAccess { hasRights( filter: { isTrue: true, and: { name: { eq: canViewClass } } } ) { id } user(filter: { username: { eq: $USERNAME } }) { username } } } } } }" },
    # allow users to see contact-school where they have group access to view
    { rule: "query ($USERNAME: String!) { queryContact(filter: { isType: { eq: school } }) { hasViewers { isGroup { hasGrantedAccess { hasRights( filter: { isTrue: true, and: { name: { eq: canViewSchool } } } ) { id } user(filter: { username: { eq: $USERNAME } }) { username } } } } } }" },
    # allow users to see contact-college where they have group access to view
    { rule: "query ($USERNAME: String!) { queryContact(filter: { isType: { eq: college } }) { hasViewers { isGroup { hasGrantedAccess { hasRights( filter: { isTrue: true, and: { name: { eq: canViewCollege } } } ) { id } user(filter: { username: { eq: $USERNAME } }) { username } } } } } }" },
    # allow users to see contact-group where they have group access to view
    { rule: "query ($USERNAME: String!) { queryContact(filter: { isType: { eq: group } }) { hasViewers { isGroup { hasGrantedAccess { hasRights( filter: { isTrue: true, and: { name: { eq: canViewGroup } } } ) { id } user(filter: { username: { eq: $USERNAME } }) { username } } } } } }" },
    # allow users to see contact-route where they have group access to view
    { rule: "query ($USERNAME: String!) { queryContact(filter: { isType: { eq: route } }) { hasViewers { isGroup { hasGrantedAccess { hasRights( filter: { isTrue: true, and: { name: { eq: canViewRoute } } } ) { id } user(filter: { username: { eq: $USERNAME } }) { username } } } } } }" },
  ]}
) {
  ...
}

Just thought I would post about this as I work to clean this up and combine as many of these into as few queries as possible. The access control I’ll admit is quite complex right now. We have different roles of users, different types of data, multiple groups that users can be part of which again can have different roles and the user in that group can have different rights… Anyways, this is where I am at.

We are currently in the process of benchmarking performance for the auth queries. Also, recently we have added a PR to optimize the performance of auth. Can you share the query which is causing performance issues?

{
  queryContact(first: 1) {
    id
    guid
    hasOwners {
      id
      firstName
      lastName
      organization
      gravatar
    }
    hasModerators {
      id
      firstName
      lastName
      organization
      gravatar
    }
    creator {
      id
      firstName
      lastName
      gravatar
    }
    isType
    isGroup {
      slug
      name
      isContact {
        id
        organization
        abbreviation
        gravatar
      }
    }
    isUser {
      username
    }
    prefix
    firstName
    nickName
    middleName
    lastName
    suffix
    preferredName
    organization
    abbreviation
    gender
    field
    hasLocation {
      id
      lat
      lon
    }
    hasAddresses {
      id
    }
    hasPhones {
      id
    }
    hasEmails {
      id
    }
    hasNotes {
      id
    }
    hasEvents {
      id
    }
    attendedEvents {
      id
    }
    hasTasks {
      id
    }
    hasToDo {
      id
    }
    hasLinks {
      id
    }
    hasStaff {
      id
    }
    staffOf {
      id
    }
    hasHours {
      id
    }
    hasMadeCommitments {
      id
    }
    hasReceivedCommitments {
      id
    }
    hasCustomIds {
      id
    }
    recommends {
      id
      for {
        id
      }
    }
    recommended {
      id
      by {
        id
      }
    }
    friends {
      id
    }
    edits {
      id
    }
    parents {
      id
    }
    children {
      id
    }
    members {
      id
      member {
        id
      }
    }
    memberOf {
      id
      memberOf {
        id
      }
    }
    sent {
      id
    }
    sentBy {
      id
    }
  }
}

Yeah, it is a really big query, but the fact that it is so fast for the admin but so much slower for users where the auth queries rules have to run it really drags down performance. Thanks to @michaelcompton who helped explain auth rule processing which gave the insight to understand this better.

I think the optmization PR might fix your issue since you have nested query. Can you try the query on latest 20.07 or master branch?

I am actually running docker image dgraph/dgraph:master which currently is shown in ratel as v2.0.0-rc1-448-gd5892dc0c

Response today running this again:

as Admin: 81.24ms
as User: 9.32s (SECONDS)

I am unable to find if your docker image has the optimization PR. Can you rebuild latest master or 20.07 branch and try the query on it?

Will do.

oh wow! Now running version: v2.0.0-rc1-518-g3c710183a

With admin token the query is 17ms and with the user token it is 869ms. That is quite a difference than before.

4 Likes

Glad that it worked for you. How much time did the query take without the Auth rules?

sorry, I don’t have that data, and that would be going backwards in productivity for me to get it.

1 Like