Dgraph DQL data not visible from in Slash GraphQL?

App Development
1

Hello everyone,

I am hitting a wall with Slash GraphQL that I am hoping somebody with a little bit more experience can help me find a way around.

I am developing a series of importers from a python jupyter notebook using the pydgraph library. My importer gets a data point from a csv file does some processing and augmentation and then stores it in Dgraph using DQL (I believe that’s what the python API implements).

In my Slash > Settings > Advanced I have set the Backend Mode to be Flexible because according to this documentation (https://dgraph.io/docs/slash-graphql/admin/backend-modes/) it would allow me to run DQL queries without having any schema restrictions. No schema restrictions is what made me love graph-based databases to begin with.

My importer sends mutations in the form of so-called blank nodes that look like this:

{'uid': '_:deposit80031',
 'dgraph.type': 'Deposit',
 'type': 'book',
 'subtype': 'monograph',
 'title': 'Krzysztof Wodiczko - Public address',
 'code': 'wodi-k-2',
 'institution': {'uid': '_:institution48025',
  'dgraph.type': 'Institution',
  'name': 'Some name'}}

Each blank node has a dgraph.type attribute. My Slash backend seems to take this without a problem and there are no errors at all during the import. After the import is complete I don’t see my data anywhere in the Data Visualizer, nothing at all. However if I run a DQL query in Slash I see that my data is there. I just can’t seem to be able to use it from the GraphQL interface to DGraph.

One of the most important selling points of dgraph for my use case is the ability to work schema free and to have nodes that are not necessarily uniform in k-v attributes. By this I mean for example that I can have a Book node that looks like this:

{
 'uid': '_:book',
 'dgraph.type': 'Book',
 'title': 'Some title',
}

and another second node that looks like this:

{
 'uid': '_:book',
 'dgraph.type': 'Book',
 'title': 'Some other title',
 'isbn':'1234567890ABC',
 'isbn13': '1234567890ABC',
 'published' : 1992
}

This flexibility is just what I need in my use case.

And one of the best selling points for my front-enders is the ability to use GraphQL for data-fetching. But I can’t seem to find a way to reconcile these workflows within dgraph Slash. Once I corset my data into a schema, I lose the flexibility of defining odd-ball nodes with attributes that I need there, having another node called metadata seems to defeat the purpose of a graph database and brings traumatic memories of using attribute tables in RDBMs.

Has anybody here found a way to make the dgraph.type node types visible from Slash’s GraphQL schema or data visualization tools? Is the workflow I am attempting even possible in dgraph? Does anybody have any advice on how I could go about this differently?

Thanks in advance.

2

The @dgraph directive in the GraphQL schema should help you here: https://dgraph.io/docs/graphql/dgraph/

Dgraph’s DQL data model doesn’t rely on a type system that’s strongly typed or structured like GraphQL schemas are. You can definitely connect these two together using the @dgraph directive in GraphQL schemas (here’s the docs again). That way, you can load your data and then expose it via GraphQL for your frontend folks.

For example, given your mutation example your GraphQL schema can look like this:

type Deposit {
  id: ID!
  type: String @dgraph(pred: "type")
  subtype: String @dgraph(pred: "subtype")
  title: @search(by: [hash]) String @dgraph(pred: "title")
  code: String @dgraph(pred: "code")
  institution: Institution @dgraph(pred: "institution")
}

type Institution {
  id: ID!
  name: String @dgraph(pred: "name")
}

The @dgraph(pred: "...") directives are needed so that you can map the predicate name in your actual data (e.g., "type", "subtype") to the GraphQL schema fields. Otherwise, Dgraph will create a new predicate following the naming convention Type.field e.g., Deposit.type. Putting type: String @dgraph(pred: "type") specifies that you already have a predicate called type that will be used for the field.

Another thing to remember when using @dgraph is that your @search directives in the GraphQL schema must match the corresponding indexes that you want to set in the Dgraph schema. e.g., if the Dgraph schema for title is title: string @index(hash) ., then the GraphQL field schema should be title: String @search(by: [hash]) @dgraph(pred: "title"). That way the index is set as you expect it to be.

1 Like
3

Hi Daniel,

Thanks a lot for your suggestion! This looks like it could be the thing that bridges both workflows, thanks a lot, that really helps. I’ll try it out.

If I understand correctly then as far as GraphQL’s interface goes, I must always corset my data into a schema and all my nodes must be uniform? Is there perhaps an * attribute or something that would allow me to express every other attribute of this node type that is not strictly defined in this schema?

4

Yup, the @dgraph directive should be able to give you the best of both worlds. :slight_smile:

GraphQL has stricter schema rules where you need to be specific about the structure of your graph. There’s no “wildcard” field that you could add. The specificity enables, say, the schema introspection and auto-completion in GraphQL queries that all the GraphQL tooling takes advantage of.

In any case, for the dgraph.type types like Deposit and Institution, Dgraph needs to know which fields are part of this type for you to do *-like queries using expand(_all_): https://dgraph.io/docs/query-language/type-system/#expand-queries-and-types

On the note about your “Book” type, you can specify that a field is not mandatory so they can be all part of the same Book type. The GraphQL type can look like this (omitting the @dgraph pred for simplicity):

type Book {
  id: ID!
  title: String! # String! (with the exclamation point) is a mandatory field
  isbn: String # No "!" means it's optional and can be null
  isbn13: String
  published: Int
}
1 Like
5

Thanks again Daniel! I’ll try this out and see if I can wrap my head around it.

1 Like
6

No problem. Do check out the “Book” type example in my previous post (I just edited it), which should handle the flexibility you need in terms of the various kinds of fields a book can have.

1 Like
7

Yes, nullable attributes make sense in this context.

I would however rather have my front-end colleagues go through a potentially empty list of other attributes, than null-check for everything that is nullable in a possible returned object. I see potential there to ruin perfectly happy lives with over-optimistic data fetching practices. :slight_smile:

8

The GraphQL spec doesn’t allow returning fields that are not explicitly asked for a in a query. From GraphQL

All GraphQL operations must specify their selections down to fields which return scalar values to ensure an unambiguously shaped response.

If you wanted a blob of data back, then you could store it in a String (this String itself could have a JSON object as well)

9

You could add a field that gets lambda resolved and in that lambda script you can do a DQL request with expand(_all_) and then stringify that and return it.

That would be a way to allow to keep current DQL support while also allowing data to be resolved through GraphQL without knowing all of the predicates.