larvinloy commented :
@thefliik Could you give some examples as to why
Gremlin is by far the worst
thefliik commented :
@larvinloy are you asking as a dgraph employee? Or are you just curious? The difference being that the first is “on topic” for this thread, the second is probably “off topic.”
larvinloy commented :
@larvinloy are you asking as a dgraph employee? Or are you just curious? The difference being that the first is “on topic” for this thread, the second is probably “off topic.”
Just curious. Feel free to not respond if it’s off topic.
marvin-hansen commented :
@larvinloy @thefliik
Gremlin can be used to perform any arbitrary graph query, but it lacks much of the intuitive and clean syntax made available by SPARQL.
As the DGraph engineers aleady have figured out, GrapQL alone doesn’t do the trick of querying an RDF graph effectively that’s why they came up with the +/- extension.
GraphQL and it’s extension still have some way to go but it’s certainly a very welcome addition to have a native GraphQL endpoint in DGraph.
I wouldn’t call Gremlin the worst, but I’m still left wondering why DGraph never even considered SPARQL as it’s specifically made for RDF graphs and is one of the very few mature query languages that can uniformly query graphs, relational data, XML, and JSON. Due to it’s strict predicate namespace, cross origin queries are a piece of cake in SPARQL and thus it’s pretty useful in complex system integration. At least you don’t have the foreign entity mess you have to deal with in an Apollo federation.
However, I haven’t seen anything about SPARQL so am I correct to assume that DGraph isn’t going into that direction?
iluminae commented :
hey guys - excellent work so far.
I would like to say that rudimentary data isolation is an absolute must for me to start using dgraph. For my use case, I would currently have to spin up a dgraph instance for every customer - which is not possible operationally. Customers will have conflicting schemas, which is not something an ACL can fix. Call this multi-tenancy if you wish - but I am actually not interested in ACLs. I need data isolation as far as other databases give me (postgresql, mysql, elasticsearch, etc. - all just have another directory on disk segmented by “database”). If each GQL call to dgraph selected one and only one “database”, represented by it’s own directory on disk, without any cross-talk, it would fit my need exactly.
larvinloy commented :
@marvin-hansen What’s intuitive is subjective. From my experience of using Neptune, I’m yet to see a query language as powerful as Gremlin. Two of the big features that I miss in every other graph query language are the ability to set query timeout on individual hops (inside the same query), and the ability to do recursive queries until a certain condition is met (i.e. without having to specify depth).
SparkQL might feel cleaner to a lot of folks because of it’s similarity with SQL, but I’m yet to see a language for Graph Dbs that is as rich and mature as Gremlin.
marvin-hansen commented :
@manishrjain @shekarm @MichelDiz
Please support CSV data import in Dgraph.
Details in ticket [Feature request] Support data import from CSV file · Issue #4920 · dgraph-io/dgraph · GitHub
emregency commented :
Hey all,
-
Would NATS Streaming integration be something interesting in relation with Kubernetes integrations? Event-driven k8s is gaining traction with serverless k8s frameworks e.g. OpenFAAS , Fission
-
Would a
dgraph-graphql-jsclient (as mentioned by @smkhalsa) not be interesting? Gateway+federated services is a more common architectural pattern for enterprises and sophisticated applications than JAMStack-ish Server Side Rendered WebApps connecting directly to a dGraph/graphqlendpoint.
1 Like
Willem520 commented :
it would be great for dgraph to support ingesting data from hive or hdfs and other similar big data stores. now, bulk loader or live loader only support local files
dmitryyankowski commented :
Most interested in “Single predicate sharded across groups” out of all of the things planned! I think it would be great for scalability and performance 
1 Like
liveforeverx commented :
@larvinloy Wouldn’t it be better to suggest enhancements for GraphQL± with not supported use cases and challenge the enhancements for GraphQL± to support this cases?
I think, it would be great to add some features to DGraph, which makes Gremlin not a requirement for any new greenfield project, so that GraphQL± just covers this.
abhijit-kar commented :
@manishrjain,
With my limited knowledge of the back-end, some fantasy requirements for the website I am making & a few pages of High-scalability blog: I present to you my humble wishlist.
- Rate limiting
- Archiving or making older data read-only (Read only data can be compressed with higher compression ratio: e.g. using Z Standard)
- Input validation for each predicate
- NATS Streaming as buffer
- Integration with anything from CNCF
- LZ4 for compression instead of Snappy for writable data, as it is faster than Z Standard in both read & write, and uses less CPU power.
- ApolloGraphQL Plugin to support GraphQL±
- Subscriptions & Live Queries
1 Like
mac2000 commented :
Wondering if there is any chance to see apollo federation support in future? e.g. ability to add new services backed by dgraph into existing ecosystem with other services and beeing able to extend existing entities and so on.
From implementation perspective there is really few things need to be made:
- Add
Query._service.sdl- which will return current schema as a string with user defined directives (@key, @extends, etc) - Add
Query._entity- aka generic resolver which will take an array of resolve references ({__typename, id}) as input and resolve desired entitites
StevenACoffman commented :
@mac2000 gqlgen (which dgraph uses) supports Apollo federation (i.e. being federated ) out of the box as described here. However, graphql-go (which dgraph also uses) does not support being federated, see graphql-go/graphql#492 so it’s not clear whether DGraph already has this capability or not.
Have you tested whether _service and _entity already work?
Clarification for others, Apollo Federation is made up of two parts:
- Federated services, which are standalone parts of the graph (DGraph could support being federated)
- A gateway which composes the overall schema and executes federated queries (out of scope for DGraph)
To be part of a federated graph, a microservice implements the Apollo Federation spec which exposes its capabilities to tooling and the gateway. The federated service can extend types from other services and add its own types that can be extended.
Collectively, federated services form a composed graph. This composition is done by a gateway which knows how to take an incoming operation and turn it into a plan of fetches to downstream services. The gateway orchestrates requests, merges the data and errors together, and forms the overall result to send back to the client.
mac2000 commented :
@StevenACoffman indeed, seems like gqlgen supports it but if understand everything correct it also means that I gonna need to recompile dgraph itself and somehow enable this feature.
Did check given docker images and there is not _service, _enitity query fields.
Wondering if federation might be shipped by default? Reasoning: federation itself wont break any existing servicess because it does not do anything specific to service only I as a user can write down some directives and still until being connected to gateway it does not affect graph in any way
PS: probably I should create separate issue for this one, to get it up and running and add some docs around this topic
PPS: also I’m not sure how it will work then in slash
MichaelJCompton commented :
Hi @mac2000 and @StevenACoffman,
Thanks for the interest in Dgraph GraphQL and for the ideas around Apollo Federation. This is a bit of an intricate one, so sorry for the long reply.
A few technical points first. We don’t take a dependency on graphql-go. Nor do we really take a dependency on gqlgen - I think we use one bit to help respond to GraphQL introspection. So supporting Apollo Federation would mean us implementing it. It’s not as simple as flicking it on.
That said, we did cut a branch at one stage to test building it out, which worked fine, but we weren’t convinced it should be built directly into Dgraph for these reasons:
- It’s not part of the GraphQL spec, it’s a cool feature built by Apollo. Which is nice, but meant it wasn’t on our core roadmap for our spec compliant GraphQL support. It’s also not the only suggestion around for federation: for example, at GraphQL java, they are building a federation engine nadel that works the other way around by the federation saying what parts of the services it is federating, rather than the services telling the federation how to federate them. Which is a cool idea because it makes the federated services independent of the federation.
- Because it’s part of another product, rather than the GraphQL spec, it’s likely to make it more of a moving target for us to support.
- It’s able to be done already by putting an Apollo server as a proxy in front of your Dgraph instance. That’ll take a small amount of boilerplate, but I currently feel that if you are building an Apollo federation, it’s likely that you have the capability for that, and it’s a more Apollo native solution than us baking it in. (we have a blog coming soon around this)
- There are other things arriving around federating GraphQL that I expect are likely to be more flexible than us building in one solution for Apollo federation. I’ve mentioned Nadel, but also I think (but haven’t yet tried a working example) that GraphQL Mesh (https://graphql-mesh.com/) could be a really cool way to get federations (including Apollo Federation) without it having to be hard coded into our service.
So … yes it’s a cool idea; no it’s not on the roadmap for the next quarter; yes please put up a feature request if you really want it built in (that’ll help us gauge interest).
We are also watching this space with interest and will build out some examples around the products mentioned above over the next few months so we can give some guidance on how to use what’s already available, and work out if we need to take something on, say Q4 2020 or beyond.
StevenACoffman commented :
In the Go community, it might also be worth noting these other two approaches:
Nautilus Gateway
Nautilus is based around the assumption that all your services are GraphQL Services. It uses the Node interface (Relay Gloabl Object Identification) to automatically federate between multiple services. This approach is quite similar to the approach Apollo Federation took. So all services have to comply to the Relay Global Object spec and you’re ready to go. Nautilus gateway will analyze your services via introspection at startup time and generate the final gateway schema.
Tyk’s Graphql Gateway
In contrast Tyk’s Graphql Gateway (and I believe this tyk gateway as well? ) goes with a complete different approach. The basic assumption is that your public gateway schema should not be an artifact. The gateway schema is the contract between gateway and clients. The gateway doesn’t make any assumptions on your services other than complying to some protocol and some spec. It’s a lot more manual work at the beginning but this gives a lot of advantages. Because the gateway is the single source of truth regarding the schema you cannot easily break the contract. With federation an upstream service can directly break the contract. Additionally you’re not limited to GraphQL upstreams. While GraphQL upstreams are supported it’s only a matter of implementing another DataSource interface to support more upstream protocols, e.g. SOAP, MQTT, KAFKA, Redis etc… On top of that, because the gateway schema is the single source of truth you’ll get two additional benefits. First, you can swap DataSources for a given schema without changing the contract. E.g. you could replace a REST User-Service with a GraphQL User-Service without changing the contract between client and gateway. This could help to easily transition from legacy to more modern architectures. Second, because the gateway owns the schema you can apply features like rate limiting, authorization etc. at the gateway level.
mac2000 commented :
@MichaelJCompton thank you for such explanation I have few thoughts if you will:
- as about graphql spec - it also does not have DateTime scalar for example, but dgraph has, so the question then why DateTime is ok to add and _Service is not - what I’m trying to say here is that it does not correlate with spec or its change in any way it is just about few additional prebuild helpers in api
- totally agree about third party, but it is preatty easy to implement and can be just turned off at any time because it is just extension and it is pretty generic but after all you are right here and there is no way i can convice on that in any way
- indeed dgraph can be put behind gateway as is but it wont achieve anything from federation
- proxies like mesh can be used (I still wait for PR on underlying library to make it more usable) but it will be hacky because we will be forced to deploy two pieces also graph manager wont work nice with such setup, e.g. I wish to be able to check schema changes right inside continuous integration process
Did go thru this in dotnet which also have no federation support, we were forced to implement it - it requires some time but at the end it becomes clear that it is just few more pieces which can be added/removed at any given time, so yes I agree with your point that it will be hard to keep track on that but might be if it will be some kind of pluggable turn on/off piece with support from community might be there is a chance it will work (but really not sure whether it is doable in golang world)
In either case dgraph seems to be pretty cool project hope will be able to give it a try
pranaypratyush commented :
Subscriptions and live queries are marked Q2. What’s the status?
pawanrawal commented :
Hey @pranaypratyush
Subscriptions have been implemented and are available in master. They allow you to subscribe to changes in a query result i.e. they are implemented as live queries. They would also be part of the upcoming 20.07.0 release which should be out soon. We are currently working on getting the docs in place on how to use them. The docs should be available on https://graphql.dgraph.io/ in a week. We’ll mark them as done once we release 20.07.0.
If you want to give it a try right now, you can try the dgraph master docker image. You can use the @withSubscription directive on any type that you want to generate subscription queries for. Here you can see an example of an end to end test for a subscription query. Let us know what do you think about it.
1 Like