6 years ago, I set out to build an application database backed by graph engine. But, we haven’t reached there yet – Dgraph is still not an application DB. I’m curious to hear people’s thoughts about what is lacking for Dgraph to become an application database.
Is it lacking performance?
Do you really need transactions?
Should Dgraph not be a database? Should it be a serving system instead?
Should Dgraph interface with other databases better?
Does Dgraph not work with GraphQL properly? or React?
Let us know what would make you run Dgraph in production, and pay for it?
Maybe I have the wrong impression, but I think the community is overall very satisfied with DGraph and I don’t see reasons for such a hard change in direction.
I think none of the questions are real issues, or at least I don’t see many posts about them.
What I see is that people need better GraphQL support with the most important missing features being:
Nested Filters
Cascade triggers
More sofisticated auth (update-after, easier definition of auth rules)
Generally I and others would like
More educational material
Some docs that allow us to get into the DGraph code easier, so we can contribute
Transparent roadmap, open issues and bugs, so we don’t get surprised by missing features or minor bugs (I realised too late that there are no nested filters)
In the end I would like to thank you for your transparency. Like many others I love DGraph and you give me more confidence to trust in your product.
Edit: In short:
Dgraph should stay a database. It’s amazing but needs more stability. At the same time I love GraphQL as my main access point and DQL/Lambda as secondary.
Not at the moment that I see. Memory issues were fixed. Really wish I could run it on a smaller footprint like you can SQL, but the thing is this is not SQL, and it’s competition graph databases like neptune can’t run on a small footprint either.
I don’t need control over them, but I need to know for sure that mutations are committed.
Not necessary. We can write scripts to sync/import data as needed. It would help new users transition, but mute points for new projects.
Just some missing feature/capabilities, and a few gotchas like “lists are sets” that are odd.
We run it in production and have over 16 billion edges on prod.
Bad news is I just had to rebuild it because of corruption issues.
I am going to be candid here, not meaning to offend: I am 0% interested in a GraphQL solution. I am only interested in a high performance infinitely scalable graph database - which dgraph almost is.
We need to focus on the goal of making the database itself solid as a rock without any question. We need to make it the best, easiest to run graph database. Having an ok GraphQL experience on top of a good graph database with some scale issues is not my desired target.
Overall we have done things with dgraph you cannot with any other database. But I have hung my hat on dgraph being solid and every time I get an alert on slack I am terrified.
Edit: you know what a great step in the right direction is: topics like this!
Yes!! Just being transparent and talking about what issues you are having and where we can help. We (at least me) cannot pay you millions of dollars, but we can love you and Dgraph like the billion dollar industry it really is. We can spread the word and knock on the door of EVERY GraphQL and graph db topic on the internet and point them to Dgraph. But when they get here, they need to know what the direction of the company is.
The fact that you wrote these two messages is the most important thing I have read on Dgraph in months! I believe this message can be the catalyst to a good relationship with your users!
We can also give you great feed back along with critique. This product is absolutely amazing, and the users on your forms can remind you of that. But you have to communicate with your users to do so!
@MichelDiz mentioned that you guys have been busy with the Cloud. You guys need to remember your cloud users use your forum too!
I know we could spend hours talking about feature requests, but I think someone said it in the Discord server perfectly. I believe you guys don’t know what you have.
You guys have the perfect Firebase / Firestore competitor. I know you think of yourself as more of a backend service, but the GraphQL service is unique in every sense.
Market to them! If they knew the relational power and simplicity, you would double your user base over night. There is actually a large Firebase userbase even for dedicated server payers. Firestore has not updated their database in over a year (been working on Firebase Product — sound Familiar?), and they lost and are losing many users because of it. I know, I am very active still in the userbase and one of them. You can see hundreds (thousands across all videos) of people just on youtube down-voting videos where they talk about new Firebase features, but not Firestore.
The fact that it is GraphQL (using triples under the hood) is the cherry on top that is awesome, but not the product. Full functional Cloud Hosting is freaking amazing. Here are your only real competitors (I have done my research):
Firebase / Firestore
Supabase
Fauna
Hasura
There are absolutely dozens more, but these particular (in no apparent order) don’t require you to spin up a server or docker in order to get Subscriptions, Security, and Full Front-end database access form your framework. All other options I have looked at are missing one or more of those 3 things. IMHO Auth and Storage are just a plus.
Dgraph does all of this, in GraphQL, and can handle dijkstra algorithm’s like postgressql can’t imagine being horizontally scaling outside of the box! You want some damn marketing, we are here to help!
I realize dedicated payers, pay the bills. But a crap load of newbies paying for 10 small projects a piece is equal to one dedicated user… and one of their projects taking off can grow DGraph!
Here is what we need:
Communicate with your users and get us involved! One post from you (the freaking founder!) a week giving simple product updates and plans can go a very long way. Is 21.12 on track? If you have to push it back, let us know that week. We understand! But let us know, not guessing what is happening. I will promise to keep this place positive if you just keep in touch!
When we tag your current team members, make them respond to the post… even if they only login once a month. There are seriously only 2 posts a day that are not responded to. That is not asking for a lot of their time.
GraphQL features to most of your form users is just plain being put to the bottom of the list. This is your product, as I mentioned above. This is your progress. You also never responded to my request for pre-hooks, which would probably be the easiest thing to fix and solve most security problems. I think in order to use your current resources wisely, you need to think about what is going to get you NEW users, not just keep your current ones. GraphQL will IMHO. It is so close to being better than Hasura, but not quite there.
I think everyone on here should agree that fixing bugs should come first before any feature request. I have no idea the backlog of bugs that have NOT been fixed. If @iluminae is using DGraph Cloud, you guys should work with him to pin down the problem before any new feature request is worked on. But don’t hold off on committing the fixes until a new version.
Whether or not some people are interested in GraphQL is besides the point. Where are you going to get NEW money from… GraphQL users, or Graph DB users. If you have a lot of paying customers that do not use GraphQL, then maybe you need to market in the OTHER direction. But the market will be slimmer IMHO. There are many other Graph databases (Redis Graph Enterprise is cloud hosted i.e.), there is only one DGraph Cloud.
Everyone will agree to this statement.
IN SUM
What is DGraph’s identity? I told you what I think it should be…
Communicate once a week with simple updates
Bugs, then Security then GraphQL then Cloud Features (unless you chose a different identity, but chose one)
This is freaking a great opening, and we appreciate it!
J
EDIT: - I don’t see a problem with DGraph having more than one identity, but I think you can get more users and possibly more VC money if you realize the Frontend product you have.
EDIT2: - It is not the GraphQL that developers care about, but the fact they can easily access the data securely and directly from the framework (react, svelte, angular, vue, flutter, whatever) to get their app going. GraphQL just happens to be what DGraph uses. Unlike noSQL, it can do complex relations, easily.
I freaking LOVE dgraph dgraph is the best thing that happened to me in the last 2 years its an amazing product that gives aaaaaaa loooot of possibilities and freedom to develop which saves a lot of development costs + i can build awesome apps on my own. Dgraph is a democratization of the internet, making tech more awesome, more performant, more easier, more cheap. Dgraph is what the world needs!
what drgaph really needs is:
marketing + storytelling. Please watch this video from sinek explaining why apple is so successful and why big innovations like bluray players failed https://youtu.be/qp0HIF3SfI4 - as @BenW said: dgraph needs a simple website rebranding. a better website which takes a different approach with a big title on the main site “Firebase & Spanner killer” catches every eye
marketing + storytelling + onboarding/learning. I think I am the one who is most qualified here for that question, because everyone here like @amaster507@MichelDiz@BenW@jdgamble555@iluminae and all other awesome people I meet here on dgraph: are awesome great devs with much knowledge and experience. But I - I am a 20 year old 0815 beginner noob shit SaaS developer that doesn’t even know how things like docker work. I am not smart, I am actually really dumb, but the only one good power I have: is patience, endurance, and big motivation for learning because I put so much passion into my charity project I’ve been working the last years. dgraph became in the last weeks crucial to me because it gives me so much power. But we all humans are individual and different, so 90% won’t or don’t want to spend much brain nerves capacity stress on learning something. they just want to swallow a pill and thats it. not getting big surgery. Professionals are permanently trapped in their pro-bubble; every tech they see they can understand awesome quickly and easy; so it is a goal to try to understand the perspective of noobs who are in a noob-bubble and need time and explanations and so on to understand things.
Most people here have much technical experience and come from cool tech products that they’ve used. I come from Firebase. The easiest thing on the world. Firebase does not need patience endurance or big motivation. The community is big, the docs are 11/10, it’s straight forward, there are awesome community tutorials, there are awesome easy-to-understand good helpful tutorials from google firebase tech-team themselves (ofc i gotta admit that firebase products are very simple and not that awesome as dgraph. tbh firestore is not that cool actually, its the same as mongodb, just with that bad expensive annoying read/write pricing model)
I just had 10 more things in my head, I could write a whole big XXL roman, but I rather just write here a to-do list that we (dgraph team + community) should do, else I could still write the next 6 hours here
we want and need any kind of users: senior devs, and junior devs. senior devs who are in their 40s probably dont have much time to spend, and won’t find out about dgraph randomly. Also many dont want to learn new things since tech world is getting bloated with 99999 new techs & frameworks being released every day. we need to attract young unskilled devs: they will do the marketing, they will spread word, and they work in big tech companies tomorrow. (ofc seniors do marketing too)
website rebranding. this is a very cheap and effective measure. we have to rebrand dgraph as a spanner firebase killer, as a cheap awesome powerful open-source graphql SaaS. if you explain dgraph to people like you explained to me personally on reddit, many more would fall in love with dgraph Reddit - Dive into anything
better docs. people like the one I mentioned above don’t have a problem with the actual docs. But I am not smart. I had a problem - thanks to the big help here + the dgraph graphlql&dql Tour and my endurance i was able to learn & understand dgraph. but people dont want to spend time frustration and nerves. I was quite frustrated. the Dgraph graphql/DQL tour is 8/10, it could be much more well explained (I even spotted in the graphql tour a mistake from the DQL tour migration). More examples & explanations = more users & less spam in the forum (i really spammed the forum with dumb questions. And I am a very open person. Other kids my age would just drop dgraph instead of asking questions and continue using firebase(firestore) or mysql). The docs are 5/10. They are very very slim and a frustrating experience. The docs are great for active dgraph developers who just quickly want to refresh some knowledge, or senior devs who know how things work. But devs like me, can do not much with the docs. Please split up that whole learning and docs thing into 3 segments: 1. Docs as they are now. They are perfect for a quick refresh. Nothing has to be added, else they will overblow. The only thing that shall be added: quick 2-5 minutes tutorials on every subject. 2. DQL/graphQL tour & learn, please overblow them and explain every single dumb thing every single line. It is very frustrating to read something that I e.g as a noob does not understand, but for you as a pro is normal knowledge. it is very frustrating to open a new tab and try to understand that through docs forum and so on. its a bad disturbing experience. It’s OK to overblow things. Explain things while imagining you are explaining them to a 5 year old kid “Explain me like I am 5”. 3. Second Docs - yes you read correctly. Leave the current docs as they are, and create second Docs specially for beginner, and overblow overflow them. Put 999999 lines of text in them, no problem. put 999999 different explanations examples in there. It’s OK to overblow things. Explain things while imagining you are explaining them to a 5 year old kid “Explain me like I am 5”. create here 20-30 minutes tutorials on every subject. If things are too slow for someone, he can just jump over to the professional docs.
This is what people want to hear, this is what has to stand on the front page (discaimer: the things that stand at the moment on the front page, are AWESOME. you guys do very awesome good work, I feel like I am speaking that bad now. the things that stay now atm at the front page should stay there. Instead I want that more things like the following ones are added. my whole text here is just a critics I dont want to hate anything), your words from reddit:
Yeah, Dgraph is built on the same Spanner principles – horizontal scalability, distributed ACID transactions, consistent replication, geographical distribution, etc. There’s even two Jepsen reports about the consistency guarantees of Dgraph. There’re no scalability limitations. Dgraph supports full-text as you mentioned, also regular expressions, fuzzy matching, geolocation, etc.
In fact, it’s a mix of 3 different techs. Spanner, Search and Graph. Internally, Dgraph uses posting list format / roaring bitmaps, which give it the same speed as search engines. And joins are first class citizens, allowing it provide very powerful graph queries.
The cloud pricing is similar to MongoDB / SQL. [1]
Yes, you can scale horizontally via the interface, and Dgraph would rebalance shards internally, automatically.
Dgraph is perfect for building real-time apps. It has a pretty strong support for datetime type, so can work well with the live incoming IOT / GPS data.
If you have the geolocation for the businesses, you can just store that, and use Dgraph’s geo-indexing system. It can then find all the businesses nearby to a specific geolocation, or within polygons like towns / cities / states / countries, etc.
Hope it helps! If you’re in the cloud interface, you can also ask questions to our helpdesk via the “Support” button.
these words are AWESOME and convinced me to start learning dgraph. And now 2 months later I am here! i fell in LOVE with dgraph
Your reddit comments at my post were like a shotgun and the bullets were reasons to use dgraph. You just shot at me one bullet after another bam bam bam bam bam one more reason bam bam bam one more explanation bam bam bam bam bam
BTW Reddit - Dive into anything it would be awesome to have a page on the website where you just have a list of apps you could build using dgraph. Just a list of company logos like facebook twitter tumblr pinterest youtube whatsapp discord, and so on. this are shotgun bullets too
Thank you very much!!!
PS: i dont know if I maybe wrote things to harsh, we germans are always considered harsh and rude and too direct (._.) sorry, please dont feel attacked by my text, its just my feedback
PS2: regarding your questions: dgraph is for me already perfect as it is, there are just some features that maybe would be cool like better fulltext search (maybe team up with typesense), fulltextsearch over multiple fields, nested filters, and so on. But NOW dgraph is already very very cool. it just really lacks documentation + popularity. We need to spread word about dgraph around the tech world
PS3: I NEARLY FORGOT!!! there is one feature that is very very crucial and missing!!! Increment/decrement fields. Building a basic Like counter, or warehouse stock counter of products. I think nearly EVERY app nowadays needs that increment/decrement thing. No matter if its a like feature or a crucial warehouse item stock counter. This is really the only one feature that I miss the most. Because I don’t want to use additionally to dgraph something like redis or another DB just to count numbers. We really need that feature. Feature Request: Increment/Decrement field values like Firestore - #5 by jdgamble555 Dgraph needs to support to incremtent fields 10000 times per millisecond, so that even if a justin bieber tweets something, so that 10k users can like that tweet within a second without a problem. this is really a very important feature that I really miss.
ok i’ve been writing that text now for 2 damn hours, that’s it! thanks a lot! ok i want to control read it now again but then i would add again 999 lines of text. i hope I have no dumb mistakes
#edit I dont want to hate https://supabase.io/ or something, but how can supabase have more attention than dgraph, more VCs pumping money into it, i really cant understand that lol i know these are 2 different products. We really have to make a better website and much better docs and some simply tutorial youtube videos. - VCs want to invest into the next big thing. I dont think supabase is the next big thing because there is honestly nothing special about it even though its a neat good product I’d also use. it just looks like the next thing. that’s what VCs care about. whole wall street VC & crypto community, they are all gamblers who want to pump things up to the moon. They dont really care what is behind the walls. How else could you imagine that there were billion dollar pyramide schemes in the USA, it’s really just great marketing. So dgraph has to look also the same as supabase. It’s really only the website, and some presence on youtube with tutorials. that’s it. nothing more. really. If I am a VC, and I don’t know dgraph or superbase, and I have 10 million USD, I would pump that into Supabase just because of the website. even though in reality dgraph is 999999999x much more awesome and more sophisticated than supabase which is basically just a CRUD app
#edit2: summarised: we need a new/better website, and much better documentation/explanations/tutorials. that’s it. really. The comments you wrote to me on reddit amazed me and gave me motivation to give dgraph a try. and here i am now.
I think the other thing is that there’s a large number of Devs who aren’t looking for specific solutions, i.e.they aren’t specifically looking for a relational database, graph database, NoSQL database. Most of the time Devs just want something that is easy to use and lets them ship faster and don’t care how it works. Devs who use Firebase don’t sign up to Firebase because it’s a document store, they sign up because of what Firebase promises which is that you can use it to ship fast and it’s reliable. The fact that it happens to be a document store isn’t even mentioned on the homepage because it’s an implementation detail. If Dgraph markets itself as a GraphQL generator it limits it’s market to developers who know they want a GraphQL API. However, if they market themselves as the ultimate backend tool (that happens to have a GraphQL API and a graph db persistence layer) then becoming the next Firebase in terms of popularity and profits becomes feasible
#edit4: UX brother UX. I was always an android fan. since I am 12 I was the biggest android fan and the biggest apple hater ever. Because iphones since jobs death are not the same anymore, they are nothing special anymore… I thought. Do you know which smartphone I have now? an iphone 11 pro max. Just because I started to dont care about cool technical stuff (since in 2021 every camera is already good). Just the UX the user experience of using an iphone makes me happy. I really love my iphone. I wouldnt exchange it for that new awesome galaxy fold phone. Even though i desire that galaxy phone, i still want to keep my iphone. i just like it. When I was a kid I always asked class comrades with an iphone 5 iphone6 “why tf do you guys use an iphone, get an android its much better better camera better battery better…” and they always responded to me “es ist einfach geil digga ein iphone is einfach geil. Ist mir doch scheiss egal wie gut dein android ist. mein iphone ist einfach geiler” - ‘geil’ means in english something like ‘■■■■■■■ awesome cool’. so the sentance translated “its just ■■■■■■■ awesome cool brotha an iphone is just much awesome cooler. i dont fking care how better your android is. my iphone is just more awesome cool”
this is the one marketing thing that apple mastered, and still helps them to make 'till today big profit with average products. Please watch that simon sinek video I linked at the start of my roman
#edit5: i just see that ■■■■■■■ gets censored. ■■■■■■■ = fuc*king
sorry for the bad words bro ._.
#edit6: supabase has big competition. dgraph has no competitoin (i wouldnt compare it to other graph solutions) . dgraph is unique. dgraph has a monopol. karl marx would hate dgraph
#edit7: marketing idea: libraries with some offline caching functionality for react native, flutter, angular, vue, react… every front end developer uses one of these. and if we offer for every of that framework a library, many will be happy to try out dgraph. Then we can put the react native flutter vue… logos on our start page dgraph.io this creates more trust credibility etc. Really, if I open a SaaS and I see they have a React Native library, I am automatically 999999x more interested. dunno why.
I think Dgraph is on the right track to be come a very popular solution. From what I can see (front end dev) there are a few things I can see that would make a difference:
Provide a generous free tier. From experience most adoption of new technology comes from developers trying things out in their spare time / side projects, loving the tech, spreading the word, and then finally influencing their work to adopt the tech. If casual users have to pay a premium to explore the product then it makes the barrier to entry higher.
Focus on creating a product for everyone, then improve / adjust it for businesses/enterprise. I understand that businesses using the product is where you will make more money, but it’ll be a harder sell to developers if it hasn’t already got a strong presence in the community. In the context of Dgraph this would be making the GraphQL features and Dgraph Cloud the best products they can be.
Ensure the GraphQL features are equivalent (ideally better) than competitors. This includes making sure they have a similar (or better) way of working and syntax. Eg nested filtering, nesting aggregate query inside regular query etc.
Invest in building a community, docs and industry related content. I think this going to the hardest thing to do as it needs to happen organically. Building a community takes time, engagement and an emphasis on the end users. I think Dgraph has a bit of work to make it more accessible. If I personally don’t have a good time using the product, I’m not going to suggest to my work (or other people) to use it and object if other people suggest it. In fact, I know we have made decisions at my work because the team behind the product was hard to work with, didn’t respond etc. It didn’t make a difference how good the product was.
Have a strong product vision (this is very important). I think its ok to even have a couple. And this has to be clearly communicate publicly, ideally including roadmaps (or some other form of transparency). I would also make sure that the product visions have a clearly identified audience.
A few things on my wishlist (all for GraphQL):
An out of the box local (offline) development experience (aka without me having to learn / do much). I have achieved this but it took a while, and was harder than it should have been. I had to spend time trying difference parts of the docs to get something working (ie with importing data, exporting data, updating my schema etc). A great example of what to strive for is Firebase’s local emulator.
Nested filtering. This is very important to many parts of my projects.
Be able to simply replace a list in a mutation. Currently I have to remove the items in the array and then fill it back up with what I want. Very annoying and not how any other garphql framework I have used works.
I personally use Dgraph as my database. And I feel most people wanting to explore with it will do the same. However, I know this isn’t the case if I was to use it at my work. We would want to use as an interface for some of our databases, and if we liked it then maybe look at using it more as a database (if it made sense too).
I have also not really explored the DQL aspect of Dgraph. Mostly due to it (from what I can tell) not a transferable skill. If I decide to move away from Dgraph then I can take my GraphQL schema and all of my queries/mutations along with me (with some exceptions naturally).
All that aside, you have done a wonderful job @mrjn (and the entire team) in creating a very compelling product. Excited to see what’s ahead!
Thanks for giving us the opportunity to give direct feedback like this, it’s really valuable. You really should join the Discord, because there’s a lot of important conversations going on (mostly around how Dgraphers can help Dgraph succeed) and it’d be great to have your input! Click the link below:
In response to your questions:
No, the performance is insane. So good that I have to look at logs to verify that the request actually went through because there was no discernible latency. Currently not working with a lot of data though, although compared with literally anything else I’ve ever used, Dgraph completely knocks it out of the park.
GraphQL transactions are not critical for me right now. I can live with DQL transactions. But what I hope to see over time is gradual improvements in the GraphQL API so I need DQL less and less. GraphQL is preferable because my client can go direct to the server rather than via an endpoint in my app. Plus it has guardrails that prevent me from screwing up my data.
This is the thing I have the biggest opinions about. Essay incoming…
Some of what I’m saying here echoes what others have already said, but it bears repeating.
In short, one of my biggest concerns about Dgraph is that I think it positions itself wrongly. Although the native GraphQL graph database is an absolutely phenomenal achievement—it is what Dgraph’s database makes possible that is most exciting. Which is to become the ultimate backend solution. This is the direction I hope Dgraph decides to take, because:
It’s what I would invest in if I had the option to
It’s what will make Dgraph most valuable to my business
There’s obvious opportunity for growth here
When I look at Dgraph, what I see is a company that has all the pieces in place to become the no.1 backend as a service tool. Currently Dgraph markets itself as a GraphQL generator, however, I think this limits it’s market to developers who know they want a GraphQL API. However, if you market Dgraph as a Firebase killer that can provide all the benefits of Firebase (iteration speed, ease of use) minus the costs (runaway costs and complicated pricing, terrible query performance, vendor lock-in) then becoming the next Firebase in terms of popularity and profits becomes feasible.
Marketing yourself as a GraphQL generator is IMO an example of marketing the features instead of the value. Most developers, myself included, don’t care how something works internally, they just care that a given tool can help them build and ship apps faster and without headaches. If you look at Firebase’s marketing page it doesn’t mention the fact that it uses a NoSQL document store, and it’s because it’s not directly relevant to the problem it’s solving—creating a powerful backend with minimal effort.
As others have mentioned, Firebase is losing customers due to the problems mentioned above. I think a good example of a startup that has suffered as a direct result of being built on top of Firebase is the bidirectional note-taking tool https://roamresearch.com. They chose Firebase presumably so they could iterate fast. In bidirectional note-taking tools, you notes are stored as a giant interlinked graph. Roam has always had big. performance. issues, and as a result it has led to a lot of outrage from customers, which led to 15 or so different companies seeing an opportunity to create competing products. Roam have acknowledged that Firebase is at the heart of their performance issues, and it’s easy to see why; querying graph-structured data from a document store is always gonna be slow.
This has created an opportunity for other Firebase-competitive startups like https://supabase.com. However, none of the Firebase competitors I’ve looked at come close to what’s possible with Dgraph. Supabase for example uses Postgres instead of NoSQL. However, as a Supabase user, although you now have a nice ORM for Postgres, you’re still stuck with the same old query-performance issues that seem to plague every app I’ve build no matter how simple. Plus, you still have to construct your queries and endpoints, and similarly to Firebase your client code is closely intertwined with your backend implementation due to the custom API. Because Dgraph generates a spec-compliant GraphQL API I don’t have these issues, because I can technically swap Dgraph out for another GraphQL API (that can return the same data) and not have to rework my client code.
The performance of Dgraph is a game-changing feature because even if you’re building a simple MVP it means that you never have to worry about having to waste precious time optimising queries. Of course, you know this because you built it! But I don’t think it’s clear from how Dgraph describes itself, or its marketing.
Reading through Supabase’s (an open-source Firebase challenger) Github Discussions, I found some comments about Dgraph as a possible integration, and there’s confusion about whether Dgraph is just a database, or a Supabase/Firebase competitor. This was my experience learning about Dgraph—it wasn’t immediately obvious why this product is so powerful. I initially looked into Dgraph because I have a specific use-case where a graph database makes sense, however, once I realised that Dgraph is basically a BaaS that can be used for any use-case it blew my mind. My thinking has now flipped, Dgraph is the default go-to, and from the conversations I’ve had with other Dgraphers, I see a lot of agreement on all of the above. I also wonder if a lot of those devs who are using Supabase now knew what was possible with Dgraph would still be using Supabase, my guess is that Dgraph if positioned correctly could win over a large proportion of this market.
So basically, I’d like to see a simplified vision for Dgraph, and laser focussed approach to achieving it. Right now, Dgraph isn’t super clear about what problem it’s solving then it’s not clear what products it’s competing with. It’s kind of competing with 20 different products at once. It’s kind of competing with other graph databases, kind of competing with Hasura and other GraphQL generators, kind of competing with other BaaS systems like Firebase. It just needs a clear message about what exactly problem it is solving for the customer and what its mission is.
As for what Dgraph needs to do to achieve this, in no particular order:
Make it possible to configure Dgraph entirely via a few clicks in the Cloud admin UI. Down to things like configuring auth rules.
Build a migration tool that makes migrating your data after a schema change as easy as using the Ruby on Rails migration tool
Directly integrate other tools into Dgraph like https://magic.link, Auth0, and make it configurable with a few clicks
Invest heavily in education (the best kind of marketing for a dev product). Primarily polished YouTube content.
Simplify the messaging around the product and what value it creates for the user.
Detailed and accurate GraphQL errors
Focus really hard on making it as easy as possible for devs to build side-projects and hobby-projects free/cheap Dgraph instances
Involve the Dgraph community more in the success of Dgraph, by ensuring there is two-way communication and feedback. At the moment it often feels kinda one-way, we submit tickets in this forum and wait for replies. Dgraph and the community working together is key to the success of Dgraph.
In summary, no-code/low-code tools are one of the most investable areas of tech right now, because web development is getting insanely complex. Dgraph has all the pieces in place to build the ultimate low-code tool. The simpler you can make it for users, the more users you’re gonna get.
I don’t see this needing to be a priority. From my point of view, it’d be a distraction from achieving the goal of making Dgraph into the easiest to use most powerful BaaS on the market.
I would second on the thought that this is the best pitch Dgraph could have. If it’s good for Google - it’s enough for our puny workloads (:
As for the points:
Is it lacking performance? No. The only tool that handles the data I work with.
Do you really need transactions? Yes. Dgraph is our main database serving each and every requests. We have a second one, but that is just logs. Business logic is complex and being able to do complex rollbacks in the database layer vs having to write code for it is great.
Should Dgraph not be a database? Should it be a serving system instead? No. Dgraph is the database for me. Evaluated lots of stuff including TiDB, Arango, cloud offerings (snowflake etc). If you need more details I can elaborate on what my thought process was, will try to keep it short here.
Should Dgraph interface with other databases better?
Now that’s interesting.
I don’t see any drawback to rolling Dgraph as the main DB (stability, but I will talk about that later).
Given Dgraph’s performance I bet some corporate clients could be very tempted to integrate it into their zoo and do some nasty aggregates across Spark / Ancient GreenPlum setups and shiny new Dgraph. Writing that layer yourself is probably not a priority, but you could get cube.dev guys to support Dgraph as part of their tesseract.
Does Dgraph not work with GraphQL properly? or React? No. Just lacking some things. I use DQL exclusively and played with graphql today just to write this answer.
Now on to what Dgraph is for me. There is this post on ycombinator which illustrates the idea
It is the unique combination of being an open source Spanner with stellar performance and development simplicity. Setting up a Dgraph cluster took me less that an hour.
I don’t have to care about migrations. Product iteration speed is easily three times faster than with relational DB. I can’t stress enough how important this is for startups.
Being able to iterate quickly and not being performance limited is what makes Dgraph unique. Nothing else on the market fits the bill. You either sacrifice performance for speed (Hasura / ORMs / whatever). And I stand by that. Anything we tried came to a halt when we pulled in production data and added a few new features.
Dgraph lets you mutate the schema in unimaginable ways without much of a performance hit if any at all.
And DQL is amazing. What previously had to reside in application layer is now done in a single query.
I would like to coin in some arguments on why Dgraph is a database in the first place. Fast forward a few month and say Dgraph has achieved parity with BaaS services in having auth service and lambdas which I believe is all it is to BaaS.
Auth is database agnostic and has nothing to do with Dgraph directly.
Lambdas can be run anywhere and have nothing to do with Dgraph directly.
Now, lets answer a few questions:
How much will I pay to run MySQL serving those workloads?
How much will I spend combating migrations and schemas?
How fast my queries will be?
How easy will it be to setup?
That’s why I think in the end it all boils down to the core technology serving the workload and why Dgraph is a database.
Given all options on the market have auth and lambdas built in I see no way any competing techology win against Dgraph in the long run.
More performant DB will always result in lower cost of ownership, faster queries and most importantly faster iteration time.
My closing thoughts on the future trajectory: flesh out the barebones BaaS requirements and focus on the database.
Having your own lambdas is great but that’s not the deal breaker. Having insane GraphQL options is nice but not a deal breaker. I can live with DQL and tolerate a lot for what Dgraph offers, even occasional instability and corrupted state.
If the DB is the fastest there is - no competitor could offer cloud options at the lower price, and that’s what can force other to adapt and support Dgraph and not your team trying to please everyone else.
Better performance is always nice, but Dgraph is already very fast.
Do you really need transactions?
Yes
Should Dgraph not be a database? Should it be a serving system instead?
Dgraph should primarily a database.
Should Dgraph interface with other databases better?
No need for me.
Does Dgraph not work with GraphQL properly? or React?
GraphQL is not a priority for me. I only focus on DQL as it has far more features and allows me to offload computation to Dgraph.
What I would like to see is more focus on the core product: the graph database. Also, more graph algorithms directly integrated in Dgraph (like shortest path) would be nice.
I use DGraph for multiple huge databases - millions of nodes and edges, with gigs of data. To analyze this data, I use the DGraph → Spark connector. For me, the main competition to DGraph is Neo4j, which is VERY expensive. I use DQL exclusively and not GraphQL.
However, I have had multiple show-stopping crash/corrupt bugs which have impeded my progress. These are frustrating. (There are other threads on these)
There really isn’t another option OTHER than DGraph and Apache SPARK when it comes to open source graph analytics, which is a pretty huge market. Focusing DGraph on the analytics might mean integrating some graph algorithms (community detection is the primary one!), which is not a bad idea, and integrating and extending the DGraph->SPARK connector as a primary feature.
Yes. We’ve been impressed with dgraph’s performance at scale- but things get stressful once the cluster grows large enough. Certainly don’t take my tone as disingenuous, dgraph is young and engineering tradeoffs have to be made, but here’s what we run up against:
No sharding of predicates
This is a big concern for us- as it means if a predicate becomes large enough, the only way to deal with it is vertically scaling. (We have predicates like xid and name that have over 100 million entries, and are over 100gb in size each).
The shard-balancer wrecks so much havoc on these large predicates that we had to completely disable it. It moves things too frequently, and its only criteria is size (instead of “hotness”). Any time one of these large predicates is moved (or indexed), the cluster grinds to a halt because incoming data gets stuck behind the operation- so much so that if we have to move a predicate, we have to pause all of our ingestion pipelines and wait for dgraph to catch up.
No query planner
Dgraph makes naive decisions on how a query gets executed- for instance, if filtering by two predicate A and B in a query,
A with 1 billion unique UIDs, with an index
B with 10 unique UIDs, without an index,
dgraph will choose A over be if it sees that A is indexed, but B would have bounded the query faster. Its frustrating when end users of our system have to do things like write their query in reverse to get better performance.
Related, cascade is one of dgraph’s most powerful features, but without a smarter query planner, deep/wide cascades can be unusably slow. This is frustrating since the whole reason you chose a graph db is because you want cheap inner-joins
Im excited for the roaring bitmaps work, as I could see that improving some of these issues.
Upsert by xid painful for ingest-heavy workloads
All of our inserts happen with deterministic ids- but since dgraph creates its own uid for everything, we are forced to have every insert be an upsert. (Query by xid, if it doesn’t exist, add it, otherwise update it). This puts pressure on the cluster that I wish we had a way to avoid. We want to be able to do blind-writes.
We’d be happy to hash our ids in a way that they became dgraph uint64 uids (that we could insert directly), but it feels like dgraph is not intended to work this way (the way the zeros lease blocks of uids, them being contiguous and sequential, etc)
Feature Wishlist
All of the data we store in dgraph is temporal, so we’ve had to do some gnarly gymnastics to allow users to filter nodes/edges by spans of time (we have dangling “timerange” nodes that hang off of every node in the graph… which we use as a filtering mechanism via cascade).
I would be over the moon is dgraph had a native timerange type that was just [starttime, endtime]. This would allow us to put a list of timeranges directly on a node, and then a query like intersects(100,200) return us a node that had a timerange predicate like [0,110], [180, 300]. This would reduce the stress we put on the cluster across the board.
Dgraph has a strong foundation, and I know the team has ideas about the issues I’ve brought up. To echo some of the other commenters, I am more interested in dgraph as a graph databse than a gql platform, given dgraph is the only modern, cloud-native, graphdb in town.
Right now it seems like we have a messaging problem. I do not seriously know if this is an app-platform you are building or a graph database. If the target of dgraph is just a means to support running GraphQL like hasura did with postgresql then I did not understand that from the get-go.
What is the future of dgraph? A Hasura competitor or a Tigergraph/Neo4j/Neptune competitor? If it is both, should they live under the same namesake? Can they coexist and coevolve?
Thank you for stepping in @mrjn and sorry to hear it’s been a tough year. Fingers crossed the VCs will come to their senses. I can only echo some of what was said above:
Important with a more transparent roadmap even if it takes twists and turns and comes to a halt sometimes due to unforeseen events etc. The product is already great in many aspects and I think the community would be more patient if we had more insight and better ways of contributing. Personally I think you could take more inspiration from Chakra-js and Next-js, two very vibrant communities that use Github issues, projects and discussions. https://github.com/chakra-ui/chakra-ui, https://github.com/vercel/next.js. I think this would really benefit the community and make for more in-depth and organized conversations. Building open source without having a tight integration between discussions and source code, prevents a natural flow of ideas and contributions. I understand that ‘policing’ Github issues could be a lot of work, but that’s what this community is for. We can help close issues and move them to discussions as needed. That way we can much better separate discussions and ideas from actual issues that need proper tracking and assignment … and it will look more cutting-edge in the eyes of VCs …
In terms of lacking features, I am leaning more towards the DQL crowd (with a rocksolid core and rich DQL layer) but I totally see the value of GraphQL as well. @seanlaff and @illuminae had many good points re. that core – such as anything that facilitates imports/upserts with XID, query planner, safe sharding of large predicate collections etc. Basically the things that keeps the system performant and makes I/O pleasant. I think better to improve the DQL documentation first (quite a lot is crammed into “Functions”: https://dgraph.io/docs/query-language/functions/) rather than spreading thin across DQL and GraphQL. Maybe document the GraphQL (and the desired roadmap and input you have) in a way that the community can take it on. Seems like less of rocket-science compared to the other parts, and it also requires more collaborative work to keep up with 3rd party specs and tools.
Overall I think more customers (both smaller and larger) will come as a result of more of us getting into production and spreading the word. So “customer success engineer” role is continuously important along with good Cloud Features that are self-explanatory. But a success engineer that is not working directly with Github issues in a transparent way, quickly leads to frustration. Here I think it’s important to draw the line more clearly so that customer success engineers can focus on issues that directly enable the customers work linked to customer payment.
I think your website and docs are actually quite good. But it depends on who you’re targeting obviously … The branding is good. I think the problem is a lack of understanding of ‘graph’ overall but as more developers see the benefits, the time will come. I think it does make sense to explain and emphasize graph/RDF upstream to GraphQL. For being a graph database, you seem to interface relatively little with the graph community. That might lead to the following dilemma: Frontenders are not yet fully familiar with RDF/graph data and its benefits. ‘Semantic web people’ are not yet fully familiar with GraphQL and its benefits. Also, because you’re neither fully RDF compliant nor fully GraphQL compliant (I might be wrong here), there will always be some naysayers. But then, take a look at what TypeDB is doing … They don’t take that No for an answer. They instead redefine the rulebook, much like you are doing. I think the key is to redefine it only so much that you can offer a natural segue (or enough bonus features) that a leap of faith is worth taking. For me personally, I’m willing to make the leap from “real semantics/real RDF” to Dgraph if, and only if, the performance and ease-of-use (especially of the query language) trumps over the downsides of no longer having a link to the rest of the LOD (linked open data) space. For people coming from the GraphQL side of things, I think that segue is more about delivering the GraphQL that they know and love coupled with demonstrations of performance, while gradually teaching them concepts from the world of graph logics.
I still think that every system should allow for maintenance windows, but I understand not wanting them. Facebook, Google, etc. don’t do maintenance windows, and Dgraph is pitched as a backend solution that should have been used by Google, but they passed on it.
I think the big kicker still is that the data files are part of the upgrade as well and cannot just be dumped between versions. I would think that eventually the data files would not need to be changed between versions just upgrading the algorithms to query/mutate the data in them more efficiently and with more tools/functions.
From my point of view there are lot of possibility for dgraph it can be a great database for apps or apps that need complex relations ; dgraph would be better in such cases
as @BenW mentioned https://roamresearch.com.
Another Issue i find is the pricing of dgraph cloud or hosting it
It would be nice if we have one click hosting on platfroms like digital ocean for open source
or maybe have a hobby plan for it like 2$ or 5$ per month on dgraph cloud
the current with 1mb limit is just not enough and the enterprise and shared plans are just out of budget for hobbits developers like me
I myself am a very big fan of dgraph due to performance ,graphql and the many possibility of it as a general purpose database and the simplicity it provides over other databases
but kept away from it due to its plans
and same for so many hobbyist developers like me
For that graphql with typescript with Apollo becomes complicated and not easy to start as compared to supabase or firebase
Are you using codegen? It is an absolute must if using GraphQL, Apollo, and Typescript
With Dgraph, Apollo, Codegen, React, and Typescript you have ONE source of truth for your types that are strongly typed from the top to the bottom of your app. Need to change the types anywhere, you just change them in one place, the Dgraph GraphQL schema, and then deploy/rerun codegen and you have the same schema updated at the database (DQL schema) and the frontend (Typescript). Gamechanging!!
