In February, we had the opportunity to join hundreds of other enthusiasts and professionals in attendance this February at the GraphQL Asia 2020 conference. It's been five years since Dgraph started, and we felt this was the perfect time to look back at our journey and share the Dgraph story.
If you're interested in the talk itself instead of reading about it, you can watch it below.
The Inception
Today, Dgraph is the world's most advanced native GraphQL database. It's open-source, extremely efficient at scale, and responds in JSON. But it all started as an idea when Manish felt that the world was missing an open-source graph database.
Manish had worked on a graph serving system at Google (also called Dgraph, read his blog here), and after leaving had decided that he would build a graph database outside Google. He even e-mailed his then-manager who led Google's Knowledge Graph efforts for his opinion on which query language Manish's database should support.
His answer:
After ensuring that there would be no problems from Google concerning the name of the software, Dgraph was officially born.
Picking Up Steam
By the end of 2016, after about a year of sweat, blood, and tears, Dgraph had become quite popular in the GraphQL community as we led the charge on GraphQL powered databases. In October of 2016, react-etc.net published an article with the headline “Dgraph has the potential to become a defacto standard for GraphQL powered DBs.”
We knew we had a great product, and we were working tirelessly to make it the gold standard of GraphQL databases. Listening to others agree with us and laud our progress early on was an essential step in our development as it promised the support of our community.
But this support also came with a lot of expectations. At the time, GraphQL's popularity was increasing, and a large userbase was waiting to get their hands on this technology. On top of this, we had created a big challenge for the team by trying to build a GraphQL-first database that hadn't been done before (the react-etc.net article wasn't exaggerating when it said we were “ambitious”). In fact, another article published in 2016 said:
Dgraph is a new project that essentially aims to create a high-performance Graph Database that uses GraphQL natively as an internal query language. This will make GraphQL a direct competitor to graph databases query languages like Open Cypher and Gremlin.
When The Going Gets Tough, The Tough Get Going
The main problem that we were facing in 2016 was that GraphQL, at that time, was built more as a REST API replacement rather than a graph query language. It lacked features that were necessary for any database query language to succeed. Therefore, in 2016, with the growing demand for more significant support and functionality in Dgraph, we started working on modifications to GraphQL. Changes that could make it a complete graph query language on its own, that can seamlessly talk to Dgraph while staying close to original GraphQL spec.
And with that, GraphQL+- was born.
Dgraph's query language is based on Facebook's GraphQL, but it's not the same. Dgraph's query language has some additional features on top of GraphQL and leaves out some unnecessary artifacts from GraphQL spec. That's why the “+-” is in the name. The plus is for the things we added, and the minus is for the things we removed. We wrote more in detail about our journey building a native GraphQL database in this blog.
So what modification did we make?
GraphQL+- features on top of GraphQL GraphQL features missing from GraphQL+- Transactions Enums Upserts Interfaces Variables (used across query blocks) Unions Math Functions (pow, ceil, exp, ln, etc.) Strong Types Aggregations (min, max, count, sum, avg, etc.) Introspection Group By Facets Geo, DateTime, Human Languages (Chinese, Hindi, etc.) The added features opened a whole new world of possibilities in terms of what you could do with GraphQL+-, including a lot of things that were just not possible in standard GraphQL.The Community Spoke, and We Listened
We created something new in the form of GraphQL+- to allow our users to innovate without being slowed down by technological limitations. In doing so, a different part of our community started facing problems - the part that needed the standardization of GraphQL. So now, we buckled up and started working on reconciling GraphQL+- with GraphQL. On the side, our community also began coming up with their own solutions, some commendable, and others just quick-fixes. We could've come up with a quick and dirty solution, but we knew how important it was for our users, making it imperative for us to get it right.
New Milestones
Between these criticisms and these lows, we also had our highs. In August 2018, Dgraph became the first GraphQL database to pass Jepsen tests. This means that Dgraph can preserve financial-level data transactional correctness under:
- Machine failures
- Process crashes
- Network partitions
- Network interruptions
- Clock skews
It was a huge milestone in our journey, but we had no time for celebration. We were still working hard on reconciling GraphQL+- and GraphQL, and we would soon have some excellent news for our community.
Let There Be Dgraph GraphQL
After a soft release in 2019, we launched Dgraph GraphQL in 2020, making Dgraph the first and only open-source database to natively support GraphQL—fully spec-compliant GraphQL. The new version brings vast improvements over existing solutions that require extra layers, tables, and schema mapping. With Dgraph GraphQL, you can initialize a database just with a GraphQL schema.
Furthermore, you can now make changes to your application by simply updating your GraphQL schema to modify your data types, indices, entities, and their properties. This means you can skip table migrations, data denormalization, and, most importantly, avoid downtime.
So to wrap up, four years ago, Dgraph was expected to become the standard in GraphQL based databases, and in 2020, we don't just meet those expectations but exceed them. Dgraph today has native GraphQL interpretation, storage, and execution capabilities. It's faster than its alternatives (no iteration over results to generate more queries), and has impressive scalability.
What's Next?
Dgraph and GraphQL are still evolving, and today we have achieved another milestone. We are excited to announce the private beta launch of our managed GraphQL service: Slash GraphQL. Powered by Dgraph, it uses GraphQL for all interactions. As the name suggests, we provide /graphql as the home endpoint for developers’ application data on the internet. Try Slash GraphQL today and experience faster, simpler, and more scalable application development.
And as always, if you're interested in being a part of what we're doing here at Dgraph Labs, give us a shout because we're hiring!
References Daniel Mai, Author Daniel is a DevOps Partner Engineer at Dgraph Labs helping customers use Dgraph in production. Previously, he was responsible for 24/7 production operations and monitoring at a cloud security startup. In his free time (away from his dubbed duties as the office Hand of the King), he enjoys hanging out with friends and family, reading books, and cycling whenever the chance allows, which is pretty often given Bay Area weather.This is a companion discussion topic for the original entry at https://dgraph.io/blog/post/graphql-asia-2020-talk/