Product Roadmap 2020

Moved from GitHub dgraph/4724

Posted by manishrjain:

Here’s the product roadmap for 2020.

  • Official GraphQL spec compliance
    • Queries (Q1)
    • Mutations (Q1)
    • Subscriptions (Q2)
    • Live Queries (Q2)
  • SaaS (Q2)
    • Launch Slash GraphQL (Q3)
    • Hosted Dgraph (Q3)
  • Scalability, performance, and reliability
    • Distributed bulk loader (TBD)
    • Rolling upgrades (Q4)
    • Ludicrous mode (Q1)
    • Single predicate sharded across groups (Postponed)
    • Query Planner (Postponed)
  • Enterprise features
    • Change Data Capture (Postponed)
    • Multi-tenancy (Q4)
    • Point-in-time recovery (TBD)
    • Audit logs (Q4)
    • ACL integration with AD, LDAP (stretch goal)
  • Integrations
    • Kubernetes
      • Operator (TBD)
      • Bulk Loader (TBD)
    • Kafka (TBD)
    • Others. Let us know in comments.

We have mentioned the features we are planning to focus on in Q1 and Q2 (first half of 2020). For the rest, we’ll assess them for their ETA as reach mid-year. Tell us what more you’d like to see happen in 2020!

1 Like

manishrjain commented :

I know a bunch of folks have been looking for Gremlin support. We’re currently focused on GraphQL, but if you need Gremlin to work with Dgraph, show your support by upvoting this comment and we’ll consider prioritizing it in a few months.

smkhalsa commented :

@manishrjain Thanks for the update and for the wonderful work you and the dgraph team are doing.

I’m glad to see that a fully managed SaaS option is on the roadmap.

One question: I don’t see any specific mention of exposing full GraphQL± functionality in spec-compliant GraphQL queries (something like what neo4j-graphql-js offers). There was an indication from the dgraph team shortly after the graphql api was launched that this was coming soon. If that’s the case, can you give a preview of how that might work?

manishrjain commented :

exposing full GraphQL± functionality in spec-compliant GraphQL queries

We’re considering multiple ways of doing this:

  1. Figuring out ways in which we can port over ± functionality in GraphQL spec compliant way, so a GraphQL user can just construct these queries directly (fuzzy matching, full-text search, has function).

  2. Adding automatically generated GraphQL functions for the complex ± functionality (like, say email: string @index(exact) @upsert could have upsertEmail func automatically generated).

  3. If a feature can’t be translated into GraphQL, then having a way for a user to specify a function name along with the ± query it maps to. This is the closest to the resolver pattern exposed via Apollo. This would be done at schema level, instead of involving any particular programming language (Apollo’s resolvers are typically javascript based) for maximizing compatibility among various languages.

For now, we’re going with easy wins, that is 1 and 2. Once we have covered and exhausted ways to port ± into 1 or 2, we can look into 3.

vikram-ma commented :

@manishrjain
Where can I find more details to understand what these goals really mean?

  • For example, what does “Ludicrous mode (TBD)” do? and what’s the plan to achieve this is.
  • Also can you elaborate Query Planner (TBD)?
    Do you guys have any checks in place to control GraphQL (and graphql±) query complexity? To prevent friendly fire scenario? Will this be part of Query Planner?

19 posts were split to a new topic: Make Multi-Tenancy open source

manishrjain commented :

All the enterprise features (including multi-tenancy) would be automatically included in the SaaS offering. That should allow for flexible, pay as you go sort of pricing.

Ludicrous mode – Idea is to allow a mode of Dgraph, which gives up on some “correctness” things to achieve maximum performance. For a lot of people, if Dgraph doesn’t give them the needed speed, they revert to a NoSQL database, which provide you very little in terms of consistency / transactional guarantees. This mode would allow Dgraph to run with lower guarantees, but at a faster speed.

Query Planner: Dgraph doesn’t do much query planning right now. It executes the queries in the same order they’re given. Of course, we could do a better job by having an internal query planner which can alter the ordering of tasks to achieve better performance.

Do you guys have any checks in place to control GraphQL (and graphql±) query complexity

Not sure what you mean.

vikram-ma commented :

Clients can do arbitrarily complex GraphQL queries.i.e clients control it the query complexity.
What it means is that clients can issue very very complex queries i.e queries could take minutes/hours to complete, and impact dgraph’s ability to serve other requests.

A valid client is making very complex query, that significantly slows down or impacts dgraph’s ability to serve other client requests.

Are there checks in place to detect/prevent/handle this?

manishrjain commented :

Are there checks in place to detect/prevent/handle this?

A client can specify a context with timeout, which can shut the query down once if it runs too long. But, apart from that, nothing avoids that right now. Once we have a query planner and can calculate the cost of running a query, we can do a better job of rejecting expensive queries.

nmabhinandan commented :

  • A startup program offering evaluation license of enterprise edition. (or better yet, SaaS credits).
  • A reactive Spring Data client (Along the lines of SDN/RX)
  • GeoJson support

manishrjain commented :

A startup program offering evaluation license of enterprise edition. (or better yet, SaaS credits).

Already there. Every Dgraph instance comes with a month of free enterprise trial.

GeoJson support

Already there. Dgraph has been supporting geo queries since the early days. In fact, some users have said that Dgraph’s geo support is better than PostGis (we haven’t verified).

Relaxe111 commented :

Saas including multi-tenancy I think is not what majority of developers will really searching for. I mean by more flexible pricing plan to make gradual licencing. For example a company could choose and pay only for Enterprise feature in which is interested, if I need only one feature I would not be happy to pay full license but using only one feature.

jdeal-mediamath commented :

My company is in the position of exploring different graph DBs to modernize our stack, and Dgraph is the best fit in every category (especially looking at this 2020 roadmap), except for lack of Gremlin support (we would like to port our existing queries, and eventually switch). Excited to see how this roadmap progresses!

manishrjain commented :

Generally asking the community here, for Gremlin support, I’m curious if that’s really a deal breaker. The port of queries from Gremlin to GraphQL is probably a one-time effort – and you get the benefit of new, easy to use tech, JSON support, with a growing ecosystem of tools and editors to support creating queries, exploring data, etc. (GraphQL has so many editors).

jdeal-mediamath commented :

Oh porting is definitely something we would be opening to do. Its that we are moving under time pressures and we know that what we have works, and that a few of our members are experienced with gremlin. Not to mention we can switch backends as needed with the support. So that is very specific to us, but it would help teams who are exploring dgraph decide early on if it is a good fit for them by allowing them to use what they have.

bronzels commented :

Hope Gremlin support in Q1 pls.

thefliik commented :

Generally asking the community here, for Gremlin support, I’m curious if that’s really a deal breaker. The port of queries from Gremlin to GraphQL is probably a one-time effort – and you get the benefit of new, easy to use tech, JSON support, with a growing ecosystem of tools and editors to support creating queries, exploring data, etc. (GraphQL has so many editors).

@manishrjain I imagine most people currently investigating dgraph are people with existing graph db needs and, historically, many existing graph dbs use gremlin. Personally, having used Cypher, Gremlin, and SQL (GraphQL too, though I’ve never seen it used for directly querying a db), as well as some proprietary APIs like Firestore, I’d say that Gremlin is by far the worst (and one of the reasons why graph dbs are a nich product). I can appreciate someone asking for support because porting an app over to a new language can be a huge undertaking, but, long term, I really hope Gremlin dies in favor of other languages (e.g. upcoming GQL standard). Providing tooling to help port existing Gremlin apps to a newer query language might be a compromise.

I’m speculating, but I think one challenge for dgraph could be that, historically, graph database usage is mostly confined to backend engineers. It seems likely that most of the current dgraph users are backend folks as well. This would contrast with GraphQL which is mostly a frontend query language (tho obviously it can be used server side as well). From my perspective, one of the most exciting aspects of dgraph is the idea that maybe in the future, I can use Apollo Client to query the backend directly from the frontend, eliminating a huge chunk of work in building out an API server (similar to what Firestore or Hasura can accomplish). This is probably not something that has any appeal to backend folks though.

shekarm commented :

On the issue of GPU acceleration, there is a separate issue opened by @marvin-hansen and it is being tracked separately here.

seanlaff commented :

@shekarm Thanks for your consideration- I think it maps to the elasticsearch pattern of supporting multiple indices in the free tier, and then supporting document (and field) level security in the enterprise tier.

marvin-hansen commented :

@manishrjain @shekarm

Please consider an in-memory mode to boost performance, as reported in issue https://github.com/dgraph-io/dgraph/issues/4813

GPU acceleration is hard and complex to implement, but an in-memory mode gives about the same performance, requires less complexity, and scales much cheaper because adding a few more 100GM memory cost way less than adding a few more high-end GPU’s.

Ludicrous mode – Idea is to allow a mode of Dgraph, which gives up on some “correctness” things > to achieve maximum performance.

Do not sacrifice “correctness” for performance, otherwise, Dgraph ends-up being no different.
Use a proper in-memory mode like Redis-graph, but actually useful.