@amaster507 – I’m not going to talk about the team here. As I mentioned, we had to do layoffs, and we got slammed on Glassdoor after that for various reasons – many of them were politically motivated to tell a version of the story. Also, remember that Glassdoor is an anonymous platform, anyone can write pretending to be anyone – there’s zero verification. So, I’ll take all that with a cup of salt.
Sorry, Anthony. I’ll have to delete your post. Let’s keep this to the technical side of things.
But, honestly – are they? No GraphQL developer I have met has mentioned that they want a graph DB. Most full stack devs just want a Postgres. Most don’t even think that graph DBs are worth being a primary DB.
Dgraph has been extremely unique in its approach. It is not really a category, because other graph DBs don’t behave the same way (even if they have a GraphQL layer on top).
And IMO that’s the information overload that DQL creates. If Dgraph only supported GraphQL APIs, sure you might not be able to do “everything”, but it would be more approachable to most developers who just want to get their job done and go home on time. And if you look at the GraphQL spec, it has the same intentions – they kept it deliberately simplistic and approachable.
And that’s achievable without exposing DQL. I think if you want simplicity, you have to cut somewhere.
Docs are hard because there’re two different languages to tackle and two different deployment models (on-prem / cloud). If there was only one language and only one deployment model (like Firebase), docs would be more straightforward. Tutorials would all start the same way. Onboarding would be simpler and so on. You get the idea.
The more surface area is exposed, the more words, pictures, videos, tutorials need to be written to explain them.
My apologies sir.
Sure thing, so from your latest blog:
There are two major things I see here outside of the amazing 
performance boosts.
In this change, we also got rid of ludicrous mode, which was riddled with bugs. Concurrent transactions provide the same performance as ludicrous mode, while also providing strict transactional guarantees.
In v21.12, we have added a flag to forbid any key which has greater than 1000 splits by marking it forbidden in Badger. Once a key is forbidden, that key would continue to drop data and would always return an empty result.
I believe that types are stored as actual predicates so what happens when the predicate dgraph.type overflows? Do I stop having indexing on my types or maybe just a specific type? I assume this overflow would vary by the length of the value indexed and the size of the uids actually stored assuming that posting lists does not reserve 64 bits for every uid but rather just the size needed on disk.
Could this have been a problem recently discovered here:
They don’t need to do anything. The flag is removed, IIRC, so perhaps config changes. All the writes should be faster by default, because we changed how transactions are run altogether.
There’s a flag which you can use to change the number. We set it by default to 1000. But, you could modify it as you need to. I strongly believe that any single key which has that many UIDs should NOT be served.
If one type has billions of nodes, then instead of using type(X), would be better to retrieve a field which only belongs to that type of node. Then check the existence of that to determine if the type is legit or not.
This really concerns me and I think is a fundamental flaw with how types are handled/indexed.
Are types stored and indexed just like every other predicate and the type(Foo) is really only equivalent to eq(dgraph.type,"Foo")?
Are you up for discussions on how to make type indexing better?
It is possible that there is not any common field shared between all nodes of a type besides the uid, and I can’t get the whole galaxy with has(uid). So if all my nodes don’t share a common required field (required fields in DQL are mute points anyways) then how else could I get all nodes of any one type.
Will type function work after the galaxy is reduced? This is a BIG question. If I do happen to be able to reduce my nodes down to some other kind of has common predicate field, but that field is also shared across other types, would the type function work then, or would it always return nil from then on out? So if I grabbed a single node by uid, could I do a type check or not??
Boring. This has been done before, and frankly has a huge relational problem. Don’t get Anthony started on the n+1 problem. They don’t know what they want.
Again, most people don’t care about the database, they care about the interface and development time.
Uh… exactly!
So cut the development of DQL, and focus on the GraphQL, but don’t cut DQL out! I believe a lot of the missing features of DGraph’s GraphQL SHOULD be solved in DQL, not the top level (cascade delete, nested updates, constraints, auths, etc).
That being said, we are where we are… so close to a product that beats neo4j’s GraphQL out of the park (again, doesn’t work out-of-the-box), and Hasura’s GraphQL. The @auth directive is so powerful in DGraph… with a graph database under the hood, the complex filter problem is a thing of the past… just match Hasura’s functionality as GraphQL (90% there to Hasura… 80% the way there to compete with postgres’s full functionality) and you would knock ALL GraphQL out of the park… Hasura, Prisma, neo4j… all graphql api’s.
So, it seems you want to look in that direction, I just don’t think you need to fork.
Focus on GraphQL. I obviously have no issues with that (not everyone will agree). But you can have both over time, just don’t give up on both.
But, where is the current paying customers mostly? Where could they be? Those are still my questions…
Not related, but again, when should we expect v21.12 on cloud… about time if you’re not sure… since this is the question forum.
EDIT
This is really bothering me. Why would just get rid of more functionality. You SHOULD add custom dql mutations as well to the schema.
Even neo4j allows cypher, since GraphQL will ALWAYS be limited in and of itself. If you add certain GraphQL features, you should be able to build an app WITHOUT DQL, but don’t take away the option.
That is backwards!
EDIT 2
Firebase queries the collection, and can filter it with very little features. Not much of a language.
To be clear:
J
I have read in detail your other latest blog post:
It was so mesmerizing. Your engineering mind is exceptional!! 
The optimization improvements are terrific for a database that already touts ingestion speeds keeping up with major databases.
And then I saw the type problem, and you confirmed the problem with indexes. So, what if, types weren’t indexes, they were edges??
I can hear Chewxy correcting me already, that everything is already an edge, it is turtles all the way down. But please hear me out.
So sroar comes along and fixes the posting list problem. Not only with memory consumption but storage on disk, encoding/decoding, transmitting between nodes (Dgraph really uses the same terms for multiple meanings way too often) of the cluster.
Sroar has been optimized for the various intersection and union operations expected from RBs.
So let’s store types as a posting list. Let there be special edges that are types that store a posting list of uids that point to all of the nodes of the type.
Maybe under the hood even beneath DQL this edge would be named something that would be scoped into dgraph only like maybe even dotting notation the dgraph.type out further:
dgraph.type.User: [uid]
dgraph.type.Post: [uid]
And then using these bitmaps quickly retrieve all nodes of a type very very easily without hitting any indexes anywhere. No need to go through a big index tree of all the types to get a single type, just graph the single edge. This would in theory be the opposite of:
which uses some kind of reverse index I assume somehow with the has function.
Instead this proposal to fix types would be just like grabbing an edge. You could even internally in the datastore have a root node that then fans out to the types that fans out to the nodes. In DQL pseudo code:
{
var(uid("0x1")) { # 0x1 is already reserved for GraphQL stuff AFAIK
User as dgraph.type.User
}
queryUser(func: uid(User)) {
id: uid
username: User.username
}
}
Fundamentally, these triples:
<0x2a> <dgraph.type> "User" .
<0x2b> <dgraph.type> "Post" .
would be rewritten to:
<0x1> <dgraph.type.User> <0x2a> .
<0x1> <dgraph.type.Post> <0x2b> .
Type indexing does need to get better. But, more importantly, queries should be able to figure out the best way to check the type. This is not something users should have to think about.
I’ll skip the technical details for how it can be improved for now. I think this all falls in the general domain of query planning.
Thank you Anthony. You made my evening 
. @ahsan did a lot of this work with me.
Agree.
Well, in my experience, most devs aren’t really attached to Postgres. They just use Postgres because:
This doesn’t mean that people don’t run into problems (like I did) and then look for alternatives. Ideally Dgraph should be educating devs on what these problems are—so devs for whom Dgraph is the right choice can avoid going through the process of building on Postgres only to realise that I should have just been using Dgraph from the start.
When devs say they don’t want to use a graph db that’s just due to misconceptions about problems that IMO Dgraph has actually solved. This is something I wrote about here:
I initially had these misconceptions and reservations about graph databases when I started looking at Dgraph. The only reason I didn’t bounce was that I had a strong incentive to dig deeper and figure out what this Dgraph is because using a relational db just wasn’t working. So I don’t think Dgraph necessarily needs to be something different, it just needs to emphasise the right things, and educate devs on how Dgraph will make your life easier, not just in technical terms but in terms of what life is like once you switch to Dgraph.
In practical terms that probably means deemphasising the graph db piece, and emphasising that Dgraph is a Backend as a Service, and the business benefits that come with using a BaaS. Dgraph should be more upfront about what problems it’s solving for users—right now most of Dgraph’s messaging is focussed on how it solves problems which draws attention to a lot of the scary complexity that devs are actually trying to avoid.
Another point to make here is that if devs truly cared about how the underlying persistence layer arranges data on disk, then the Firebase wouldn’t be a huge business. Because Firebase’s whole value proposition is “don’t worry about the data stuff, just build your app and we’ll give you an easy API”. Turns out that Firebase stores data in a proprietary document store format that leads to all kinds of problems. It’s only when devs start bumping up against the scalability limits of the database they are using when they really start thinking about this stuff.
Firebase has been around long enough that there are a bunch of mature startups running on Firebase who are bumping up against Firebase’s limits, which is the likely reason why there’s a growing number of ex-Firebase users looking for alternatives and talking about their frustrations. This is a fantastic opportunity for Dgraph to educate 1000s of frustrated Firebase devs on how Dgraph fixes all these problems they are having.
I’m optimistic that Dgraph is actually just really early in a new category of Firebase alternatives.
A category is more of an abstract marketing thing, it just how you present yourself to prospective customers. There’s all kinds of ways that Dgraph can categorise itself. Some are better than others, but the ideal way to categorise yourself is to find a way to create a new category. The category is just a mental framing that helps people understand how the product is differentiated.
For example, if Dgraph merely categorised itself as “a database” that puts it in the most undifferentiated category possible. If Dgraph categorised itself as “Super-scalable Backend as a Service” then it’s prompting the prospective user to compare it to any other products that feasibly fit into that category. The best category is one that is easy to grok, makes it clear what kind of product it is, but most importantly communicates that the product is a category that very few products have been able to achieve. It’s the difference between Usain Bolt categorising himself as “someone who is good at running” and “one of the only 156 humans in history who has been able to run 100m in less than 10 seconds”. I think currently Dgraph puts itself in some impressive categories, but it’s not spelled out for people—it’s only after doing a lot of reading that you realise that oh… all this technical stuff they are talking about actually equates to “Dgraph is gonna make it possible for me to achieve huge scale without having to hire a team of $300k/yr engineers / Dgraph is gonna make it possible for me to ship product 10x faster / Dgraph is gonna allow me to side-step all the query optimisation problems I’m having”.
Let’s compare Dgraph’s homepage to Firebase (please don’t take offence at any of these criticisms!)
These are the first things people look at on the retrospective homepages. There’s a stark difference. Dgraph describes what the product does, the main feature (it’s a GraphQL API). There’s a vague value proposition (build it right first time), not sure what this means.
Firebase by comparison does not reference the technical implementation at all. The key message on the homepage is a description of a benefit (Firebase helps you be successful as a web developer). This is what devs care about, being successful. This is what Dgraph needs to be talking about—how Dgraph will make you successful.
It’s possible that our platform could help with the funding issues. MIcroventures.com is a democratized angel investment platform with exposure to accredited investors.
We’re moving our entire platform away from MySQL into Dgraph. We’re in the early stages of doing this and hearing there are issues for the future of the platform has me hesitating in a big way.
What I see in Dgraph is the ability to unify my engineering team under Javascript and Graphql. All my core API functionality is handled by dgraph’s API replacement under the hood. That’s code I don’t have to write.
When my engineers are asked to add a feature, Say, “Present a list of the user’s last 5 posts”; In the old stack, either they have to query an existing API endpoint that would probably present far too much data and be inefficient, possibly prompting more work to overcome the inefficiencies like caching… OR, they’d have to make a request to the backend team and get on their schedule, to get a custom endpoint.
With Dgraph, the power of the backend team is transferred to the frontend team. My engineering team can focus on React and the presentation of the data. They can easily query and interact with the data without delays when that structure needs to be slightly different.
It also means that I can pivot off the dgraph types generated by Graphql Codegen and be confident that my application is functional across the board even when the schema changes.
But all of these advancements have come with a development cost.
IMHO, Dgraph has a ton of performance power for our engineering team. It has not been without its challenges to get our stack working with the platform, however we’ve solved most of those issues.
What I would love to see in the future:
Dgraph offers something that’s hard to find, efficiency, and cost savings in the engineering department. Play on that angle and re-double efforts to support the community and the engineers will evangelize the product.
I sincerely hope Dgraph is successful.
Agreed that it’s a hard problem to solve. I’m not a marketing expert, but I believe if you try to win at web-dev then you need an angle like “firebase, but better because you can actually query stuff”. But then you have to build the one-stop-shop - auth, data, functions, object storage, remote config/feature flags, analytics, app performance, etc. I agree that web devs don’t care (and sometimes don’t know) about the difference between different database engines - document, relational, graph, SQL, NoSQL, NewSQL… just give me data fast, reliably, and don’t make me write too much code. The database engine that drives it is just an implementation detail. Dgraph is the fastest, most queriable, most scalable data engine you’ll ever use! Maybe there is an opportunity to partner with other web companies whose missions are “a faster web” like Vercel or Cloudflare or Fastly. If Vercel includes Dgraph in their tutorials, you’ve got another stream of curious devs.
On the other hand, going after the graph DB market seems like a longer-term play. Again, outsiders view, but graph analytics and algorithms that people/companies in data science seem to shop for. Probably bigger contracts but also long sales cycles meeting the needs of big companies. Or maybe there is an ML niche to slot into.
One thing I’ll agree with that someone else commented on - don’t remove DQL or on-premise functionality and documentation. If it’s not the golden path you’re aiming for, just bury the documentation under an “Advanced” section. Let the cloud, graphql approach be featured in the documentation. Everything else should be harder to find and therefore less confusing.
As for questions, I have none. I’m a happy user.
About what VCs are we all actually talking about, what kind of VC was the one that did the last funding?
If we are talking about the kind of VCs who also ‘invest’ big cash into companies like Uber and WeWork, they MUST also invest big money into dgraph because dgraph is a much more better and more legit product, than Uber or WeWork. i dont understand why they dont want to.
The only thing that dgraph lacks in my opinion, besides of the docs who could be better and more noob-dev friendly, is the presentation/marketing
Please look at this video manish https://youtu.be/Hv6EMd8dlQk
VCs seeks popularity and opportunity for capital gain. Kind of usual and it is their money they are putting in risk. Even in the case of UBER which is deficitary for years, they do invest on it cuz it is popular. The factual value of a product isn’t how good it is. Is how popular it gets.
Just like the most famous(my personal opinion) Graph DB out there. It is famous cuz they put a lot of money and effort in the popularity of the product. Not the quality.
This is so good. In fact, that’s my biggest learning – The best product doesn’t win. Best marketing does.
Great marketing is a skill that’s really hard to find. Much harder than finding great engineers.
True.
First of all thanks for taking the time to do this AMA. I really think you and your team upped your game again in terms of communication and transparency.
Of course there are still many more questions and we now have a more clear understanding why transparency and communicated progress was lacking. You are currently at a problematic spot and trying to find a way to transition.
I would like to ask some questions in terms of Open Source.
How come you switched from Github Issues to Discuss? I think it’s currently very hard to track which issues and bugs are open within Discuss.
Do you plan on using any alternative tracking software?
Was your plan always to keep development inside the company and just put the code out there?
I may not be as professional as you are, but I’m sure there are small things that could be contributed by the community.
Overall I’m very happy with DGraph and I think you are very close to a product that could explode with the right marketing.
The idea was to consolidate the GitHub issues from the dozens of repositories over to one place – Discourse. But, I also acknowledge that many people have complained about that for a while.
No. That was never the idea. But, given we were VC backed, we had the money to hire engineers. The PRs from the engineers kept us plenty busy. Contributions from community took a back step. This isn’t ideal, but also isn’t unusual for a VC backed open source project.
I think community plays a HUGE role in any project. And we’d be supportive of contributions.
Here’s a poll.
this!!! +1
this also makes my heart cry - there are so many talented engineers out there with brilliant ideas, which will never get executed…
What is Dgraph lacking? - #8 by Juri I wrote here many things, I break them down now:
here’s a quick to-do what can be done imo:
this are really simple quick things you can get done within december! sum up:
that’s it. these are easy measures you can start today. this can polish dgraph extremely up imo
took me now an hour to write this (._.) gud night
I don’t believe that is 100% accurate. It’s true that great products with bad marketing don’t get traction, it’s also true that bad products with great marketing do get traction, however, the ultimate winners are always those who have both great product and great marketing. Dgraph is already a great product, it’s marketing/messaging just isn’t right—everyone says this. Fix the marketing and you’ll get the traction.
Apple users are persuaded to purchase Apple products with the excellent marketing, but they choose not to switch away to competing products because they are happy with the actual product—longterm customer satisfaction can only be achieved with great product.*
I can come with endless examples of this. I think where the confusion is when people think about this is that they measure product quality in terms of features, but ultimately the reason why people continue to use products is because the product solves a painful problem. Successful companies understand exactly what the pain is that customers are experiencing, build great products that remove that pain and market the product by educating customers about what the pain is and how their product fixes it.
In the case of Dgraph:
Customer: Developer building a business, or working for a business
Customer pain: Stressed and bogged down by backend scaling issues, frustrated because they can’t iterate fast enough
Solution: A tool that delivers backend scalability with 100x less effort
Marketing: Educate customers about what this frustration is that they are experiencing, how it is preventing them from being successful. Educate customers on how Dgraph as a product removes this pain and enables them to experience success.
I think the real mismatch is what product creators consider great and what customers think is great. Product creators (designers, developers) recognise quality of implementation, but that’s not directly relevant to customers pay attention to, they quantify a product as being good when it solves a problem. Dgraph is a great product to me not because I’ve pored through the source-code, (although I do have great appreciation for the elegance of the implementation, that’s just not the reason I’m a customer) but because it perfectly solves multiple frustrating problems.
There are all kinds of ways to measure the ‘quality’ of a product, but the product’s ability to remove customer pain is the only metric that really means anything in the context of business.
*A note about Apple, because a lot of people seem to think Apple is an example of bad products with great marketing… the fact that Apple has millions of happy customers is evidence that they make great products in the eyes of their target customers. Saying Apple makes bad products because you don’t like them is like saying that makeup is a bad product because you don’t wear makeup.
Thanks @CosmicPangolin1 for your detailed thoughts. I agree that there’s a lot of money floating around and it seems strange that we aren’t able to reach out and grab it. But, there’re more factors at play in a fundraise than the excellence of the product or clarity of the vision. In fact, none of those matter (i’ll elaborate on that in a later post).
@BenW This is why products with best marketing win – they get traction, get to raise funds and live another day. And with enough funding, with a decent team, you can fix up the product over time. With enough runway, bad products with great marketing transform into decent products with great marketing. And over time, they just become good products with great marketing.
Meanwhile, great products with bad marketing die, without getting the chance to fix up their marketing (there are caveats to this, but that’s another topic).
Moreover, whether a product is great or not is subjective. Whether a product has traction is very objective. There’s a separate post I’ll write later about why VCs invest and why some companies get a lot and some don’t. But, rest assured, product is not a factor in these decisions.
I have read all those reviews. There is a lot of crap in there that is not important. What is important is it seems you guys are in a good place, potentially in the black. You have a board to report to, and for the moment can make some smart daily decisions for the company without too much push back from the VCs and board until you get some more money, whether it be a bridge loan, a complete buy out, or the same VCs educate themselves on what Dgraph really is. You can’t really talk about that, nor can you share the P&Ls with anyone, which is expected. That is all fine and dandy, but also means growth is limited, and focus is important.
Others have spoken about marketing and getting more money, which you know is something to be strived for.
What is important, is getting back the trust of your users. One person paying $40 a month is not going to change anything for the company. However, one person reading 10 different users’ posts about the frustrations of Dgraph, may could lose hundreds of thousands a year for Dgraph. That person would recommend Dgraph for other projects, and the cycle continues. One out of say 20 $40 a month user, upgrades to enterprise, making Dgraph money, and 20 $40 a month projects (which could be one user), could be equal to 1 or 2 enterprise users. Every user counts. We know this, but it needs to be said.
I also read that it seems you have been disorganized in choosing when and which features to implement. This is great for innovation, bad for focused progress. While I can’t speak for the team issues, I can confirm this is the second biggest issue for your users. We want discuss for problem solving, but you NEED github to not only track issues, but to track features. Obviously, everyone agrees.
So, again, focus and gaining back user trust…
I think by now you know where you want Dgraph to go. This thread as been extremely important in that. Write out your plan for the next 6 months. Stop ignoring posts from your users, and start building trust back. I have asked 3 times when v21.12 will be on the cloud since you created this post, and I have been ignored. I have dozens of unanswered posts (not even related to feature requests), and I have had at least 3 broken promises from marketers, managers, and programmers. I think it is hard for me, Anthony, and others to ignore the temptation to write about frustrations. We want Dgraph to succeed, we don’t want a trace of frustration in the forums, but we also want to be real with everyone, especially so management can DO SOMETHING about it, instead of creating more traces of problems.
Bringing the critique sandwich back around, I believe in this product. Despite 2021 (Dgraph’s 2020), 2022 should be a good year. It truly does seem Manish wants to head in a good direction, due to the layoffs the company is very still-alive, and while the future seems slow, it seems like there will be a future. However, Manish, you need to always keep your users happy going forward, or there will be easily preventable problems. This is what you need to do IMHO:
That’s basically it. We need to know things are moving forward, despite the ups-and-downs with internal problems.
That is how you can lead this company going forward, and gain more money and followers.
J
