What is Dgraph lacking?

@BenW In my opinion, this would not be wise. There is still so much that can be done within the specification at the moment. And just the fact alone that there is so much tooling built to the specification that breaking it would most likely break who knows what tooling, would not be wise.

1 Like

Agree that this should not be immediate priority.

But if you’re only making it available to users who specifically enable it, and make it clear that it could break things… then I don’t see the problem. Before Dgraph I was building a GraphQL API by hand with complete disregard for the spec, I just made it do whatever I needed it to do. Never ran into problems. What I would like is this amount of freedom and control over my API.

Following Anthony, GraphQL is like that for consistency/predictability and security. We cannot require an API language to do the work of a DB language. I think the custom query solves part of this problem. It’s basically what other DBs do when they add GraphQL to their code base in some way. However, there is still a lot that can be done under the spec.

Nope, GraphQL foundation has total control of the Spec now.

There are profound changes to ask. That’s too much pretentious for us to request. We should work for now with what we have.

The logic behind GraphQL is to build your own GraphQL server by doing the whole business logic in the backend. I can’t imagine Dgraph predicting every possible means of business logic and supporting them all out of the blue. Or do something general. I think a pretty good part of the logic will stay with the DQL, lambdas and so on. Which it doesn’t mean we won’t fix or add new things to prevent the user from having so much work with the business logic.

I think you were covered by the spec, cuz no one can do anything out of the spec. In your backend, to resolve the query you can do anything you want. In Dgraph is different cuz everything is written in stone. The only logical freedom to solve a query is using Custom and so on.

1 Like

This doesn’t sound very ambitious Michel! There’s no reason that Dgraph couldn’t be the most influential force on the GraphQL landscape. From my point of view Dgraph has a responsibility to put forward suggestions to the GraphQL Foundation, to ensure that the spec doesn’t prevent Dgraph from being able to achieve everything that’s technically possible. By the looks of it, adding feature requests is as simple as creating Github issues on their repo: Issues · graphql/graphql-spec · GitHub.

Seems like this is similar to how things work with the HTML and CSS specs:

  • Browser vendors come up with new features
  • They submit feature requests to the W3C
  • A JS or CSS feature that is only available in xyz browser becomes popular with developers
  • Developers build polyfills so they can start using the feature in their projects
  • W3C recognises that the feature is required, adds it to the spec
  • Other browsers implement the feature because it’s now in the spec

That being said, I don’t know enough about what the spec is actually preventing to be able to make informed comments about this.

EDIT: In fact this could be incredible marketing opportunity for Dgraph—to implement experimental features that show what is really possible with GraphQL if you put a v12 engine inside it. Experimental features can be only be used with a special build, or with ENV flags. Build some demos of the feature, create feature request on the Spec repo with links to the demos. Get the GraphQL community talking about it. Good way to demonstrate what’s really possible with GraphQL, and that Dgraph is what makes it possible.

4 Likes

This is probably bringing Dgraph full circle for those who have been around for a while. Be careful with what you wish for, just saying that if Dgraph is the only one that implement X feature in GraphQL then it probably won’t get much attention from the foundation because it is specifically language and database agnostic.

https://github.com/dgraph-io/dgraph/issues/933#issue-228222037

This leads to one of those situations where it seems very deja vu to me and probably others.

Created a separate topic for this:

Can you point me to where the bug fix for this one is:

I tried this on Tuesday and it still wasn’t working. I have a table A, with an array relationship to B. I need to get every row of A with their B and perform an update for each one. So if A1 has B1, B2, B3, I need to do an update with [A1,B1], [A1, B2], [A1,B3], and I need to also include A2…A500 and it’s relationships.

AFAIK This is still not possible.

Update Many: Fail in GQL, pass in DQL

I gave this a fail because of the node limit error, it makes it almost impossible to use effectively because in some cases the node limit happens updating 200 records, and other times 500, also the node limit is enforced across multiple mutations in the same request. There is absolutely 0 reason to use this over DQL upsert and should probably be just removed from the graphql API if they aren’t going to improve it.

Update Many Conditionally: Fail everywhere. (See bugs below)

I can’t remember why I failed this one, so yeah, this is okay.

Update Many using data across relationships: Fail everywhere (See bugs below)
Update many to many relationships: Fail (See bugs below)
Read Many Nested Loops: Fail everywhere (See bugs below)

These bug still exists, and I’m referencing what I described at the top of this post. It essence these all boil down to the ability to permutate across relationships. In SQL, if I do a join, I will get a row for every unique combination of A and B that are joined via foreign keys. How do we do that here and get A1+B1, A1+B2, A1+B3,A2+B5,A3+B88, A3+B89 and so on and be able to use those in filters and upserts.

Otherwise yea, I think this conversation as been good for me as I have realized that that if they fixed this, added nested filtering and foreign keys then it would probably address 95-99% of my current biggest issues.
I understand that some things are not possible in Graphql due to the spec. Some/many of those issues could be addressed with tooling as they are mostly used by developers and not really needed.
I’m allowed to make mistakes, and when I do it can be punishing for a new user to have to figure out how to fix them, especially if they started using just the graphql API.

1 Like

So obviously DGraph (DQL and GQL) does not handle every single scenario. I am not sure it is even possible to do so in GraphQL the way the specs are.

However, I do believe there are inventive ways we can do things without breaking the GraphQL specs, so everything could be possible. You could probably do another table to compare what other GraphQL clients can handle; nested filters and @reference directive would probably bring DGraph GraphQL up to par with them.

Here you can see every scenario I could think of:

Update Type GraphQL (Link) DQL
- Flat Relationship - - -
All Fail Pass
One Pass Pass
Many (Conditional) Pass Pass
Many (Nested Conditional) Fail Pass
Many (Specific) Fail * Pass
- Nested Relationship - - -
All to Nested One Fail Pass
One to Nested One Fail (Pass) Pass
One to Nested Many Fail Pass
All to Nested Many Fail Fail
Many (Conditional) to Nested Many Fail Fail
Many (Conditional) to Nested One Fail (Pass) Pass
Many (Nested Conditional) to Nested Many Fail Fail
Many (Nested Conditional) to Nested One Fail Pass
Many (Specific) to Nested One Fail (Fail *) Pass
Many (Specific) to Nested Many Fail Pass

* - Can be done with multiple blocks in one mutation
() - Whether or not the nested link can be updated

Notes:

  • As far as I know, you can’t update all nodes in GraphQL without a filter, let me know if I’m wrong
  • @MichelDiz - Are these many-to-many cases possible in DQL like in this thread?
  • All is referring to no filters
  • Specific is referring to multiple ID or @id inputs

Let me know if I made a mistake somewhere, as it is a lot.

Now we know what to strive to do.

J

5 Likes

I don’t remember, but I think it was 90% fixed. The problem is that you have to have an extra block(that’s why I mentioned the “loop solution”). And also I think you have to do it 1root:1root - You can have many nested, but it can break(I think, I don’t remember) the query. Need to check. But is partially fixed as far as I remember.

2 Likes

As a side note: DARPA projects are probably a good way to get funding to expand DGraph, as the technologies really apply to a lot of emerging hard problems.

3 Likes

Hello there! As a newcomer, my 5 cents:

  1. Documentation / quick start are OK but not perfect yet! It took me long time to understand basic stuff like the difference between GQL and DQL. The differences and usage of the apis to control data and control backend also are not really clear, at least got me confused. Another point that I’m still completely clueless is lambdas. I would love to have some field as “calculated” values that depend on other fields, but I can’t find a way to do this.

  2. Dgraph cloud: I tend to skip every initial tutorial of every platform/tool, I find them really annoying. The first time i got into the Dgraph cloud, tho, i would have loved having somebody explaining to me what are each one of the elements in the menu. I’m still a bit confused after 2-3 days

  3. Feature request: This is my personal wishlist so far

  • native timestaps and timerange
  • Nested filters and updates
  • cascade operations
  • @auth at field level in addition to having them at a type level
  • pre hooks & post hooks: not sure if this is doable with lambdas, as I still did not get how they work :sweat_smile:
  1. Bugs: This one is driving me crazy. In chrome (W10), each time that i want to paste something into the schema or the GraphQL editor a menu pops in… CTRL + V just opens the menu with the types. If i already set up the type, then it just does not do anything. So each time i have to write a first type (f.i. query), use the mouse to paste all the data, and clean the file. The only point of real frustration i’ve faced so far!

  2. Marketing: I agree with most of the comments about your com. It is not clear. You may need some help from a marketing guy, but my intuition says you have to market yourselves as the firebase killers or a supabase with technology of this century. So answering if dgraph must be a DB or not… first step in my mind is continue being an amazing DB and fix some key missing features. The second step is to become a full BaaS to help devs build apps and MVPs faster.

  3. Dgraph is amazing, I’ve fallen here by chance and it seems i will use it for as many projects as I can :slight_smile:

5 Likes

Hi, Any plan to include some authorization/authentication system into dgraph?..I’m using supabase and it works fantastic, this provides a sweet authentication system and I don’t need use external (and expensive) services or run my own auth server

I love graphs db’s, dgraph was love at first sight (I come from using neo4j and more experimental db like titan db…I think it was discontinued) but I really find overcomplicated to use an auth services when supabase gives me a whole package with the serverless philosophy

thank you so much for create Dgraph

1 Like

Dgraph already have the ACL and Auth feature(this one in the GraphQL part). There’s no plan to add Auth in the DQL part for now.

Firebase Auth is free and pretty easy to get going:

Scroll down to see the standard claims version, which is much easier to implement.

J

1 Like

Dgraph uses JWT auth. I’m using https://magic.link to log users in, then I generate a JWT serverside and hand it back to the user. The JWT authorises the GraphQL requests coming from their client (browser). I posted some example code in Discord you can search for.

Agree that Dgraph ideally needs a Supabase-like UI for quickly configuring and verifying auth rules. Would speed up Dev time a lot

1 Like

As a followup to my suggestions mostly around the topic of providing much more robust search capabilities much like ElasticSearch I recently discovered https://github.com/prabhatsharma/zinc

Zinc is a search engine that does full text indexing. It is a lightweight alternative to elasticsearch and runs in less than 100 MB of RAM. It uses bluge as the underlying indexing library.

Seems like a great fit for Dgraph!

5 Likes

In my opinion, the entire bet on graphql is cool but if it doesn’t take over the API standard then pretty soon most people would have a high entry barrier for using dgraph and its goodness like cluster transactions and performance scaling and the smooth smooth modelling may remain unutilized. I am personally graphql biased but still in projects where there is no graphql at all it feels like bringing a cannon to a knife fight to get dgraph (I do it anyway cause I love modeling data on dgraph schema, lol).

I should probably not suggest solutions but I would like the graphql and dgraph database to be separated (somehow). And I don’t mean just the graphql serving but also to support another language (like gremlin) or ur own custom SQLesque graph DB language. So that people who don’t buy into graphql can be comfortable with dgraph.
And on the other hand, I do like the graphql serving features and it’s so good to just give schema and have such a good API ready with the knowledge that it’s backed by a graph db with infinite flexibility.
But it would be even better if this was separated into like nodejs or go or rust server(embeddable runtime? library?) so that it can be hosted separately, modified as we like, and more generally have an ecosystem rising on it so that we can be even lazier and pull plugins to, let’s say, add events to subscriptions from kafka or nats source, or maybe to do the whole jwt thing but with infinite flexibility or it could be file uploads streaming to S3. But still have the super tight integration with dgraph. Maybe a graphql to dql compiler/code gen so we get the code and just modify the code as we like. without learning the dgraph specific things.
Essentially it’s all about building something on which an ecosystem can thrive. the current solution with lambda works but has some learning involved and an ecosystem wont start on it.

  1. Also collections probably? more json focus ? multi model like feel?
  2. Schemas are good but currently, dgraph is for rapid iterations so more focus on schemaless dev?
  3. expand to work without schema
  4. tools to gen txn that move data when schema is modified
  5. multiple databases in single backend?
  6. typescript binding for dql client as well so responses are typed?
  7. customisable full text
  8. ability to add custom indexes
  9. pregel like compute system (with some common targets already implemented)
  10. rewrite in rust to brag on performance (just a rust fan boi pls don’t kill me)
  11. OLAP mode / integration with olap tools
1 Like

I find this ironic. Dgraph brought GraphQL into the core because of the users demand for it. Go back and read old github issues, etc. This was done by bringing Michael Compton (and others) into Dgraph who had built exactly what you are referring to now outside of Dgraph. And now 2 years later users ask for this to go away back into the host it yourself graphql that connects to DQL.

I don’t believe this is the right direction but there is value here to your point. With other databases you can swap out the engine. What if dgraph handled GraphQL and other future languages such as Gremlin like extensions that could be added into the core?

This way a user who doesn’t want GraphQL doesn’t even get any of those endpoint because they didn’t install the GraphQL extension but they want Gremlin so they install that extension. And this I believe would open it up for more and more extensions to be added such as Graph Algorithms, or Cypher, or GQL (looks like it is becoming official spec)

Dgraph supports namespaces now.

I don’t think GraphQL is going anywhere. I just think it is for advanced real users. That being said, the trend has always been, and will always be, quick development with the least amount of effort (so we can concentrate on the product).

I believe that is why things like Supabase and Firebase are popular (and why they got backers). They work out-of-the-box with little to no knowledge and configuration. This is always the future.

That being said, they work on SQL and noSQL databases, not Graph. Whether or not it is DGraph, Graph Databases will catch on as the most powerful and intuitive, as soon as someone makes them easy and stream-lined.

DGraph is 80% of the way there, but needs to tighten up some things like the @auth directive, search features, and nested data relationships (filters, updates, and deletes).

That being said, we don’t know anything about the direction of DGraph, because there is unequivocally no communication from anyone about this. Hopefully it is the right direction for both our frontend developers, and our backend primary graph database users.

J

3 Likes

I do remember the time when it was added. You are right about that users asked for it but at least I wasn’t the one asking.
Maybe what users didn’t expect is that it with such a tight integration it would become claustrophobic.
I guess we should not listen to solutions proposed by users but only the problems they are facing.
That’s why I said I shouldn’t be giving out solutions but it was just faster that way. I could list the problems instead of solutions if you want.

@jdgamble555
Graphql may stick, and I would like it to but it’s certainly not fit in every situation and not the majority of developer. It’s a team thing so there’s high barrier of entry for graphql into stacks.
As you said graphql is loved only by advanced users and advanced users are much more likely to trade full freedom in customisation over magic that does things for them.
So tight integration of graphql serving might be hampering acceptance of dgraph. I don’t really know. You probably need a survey or something.