Can we have support for file-uploads and downloads?

Continuing the discussion from: Can we have @remote scalars? - #10 by maaft

When building applications that use databases, often binary data has to be stored somewhere. This is often done by storing the location of the binary-blob in the database and the blob itself somewhere else on the backend. Usually this requires custom business logic for storing and retrieving this data.

File-Uploads and Downloads with GraphQL are often implemented using the GraphQL Multipart Request Spec

This is a commonly used feature in other GraphQL servers like e.g. gqlgen or Apollo.

Dgraph GraphQL is currently missing this Multipart-Scalar and it is therefore impossible to use a @custom mutation that relays the upload request to the custom graphql business logic. Currently the client has to:

  1. upload the file to custom-business-logic-server:4000/upload
  2. store the file ID to the database dgraph:8080/graphql

Similar, the client has to take these two steps to retrieve the binary files from the backend.

Using two different URLs for database access and binary data upload/download is cumbersome and error-prone. It would be great if dgraph-graphql would support the upload-scalar using multipart spec referenced above.

Thinking this one step further

When the upload-scalar is implemented, the next step would be to add native support for binary uploads/downloads. binary data could be either stored on disk or on a 3rd party s3 solution like minio (which is already used by dgraph for backups so I guess it could be integrated without much effort).

This would be super convenient for clients as dgraph would take care of all the nasty stuff that comes with CRUD operations (data-storing, -fetching, -deleting, … ) and I think it would give dgraph-graphql even more of an edge over competitors

Looking forward to discuss this! :slight_smile:

8 Likes

Been thinking about this when I was getting my hair cut.

One step required for this is to have a byteslice as a scalar type that is supported. This way any []byte would not need to be interpreted as any other datatype, leaving the decoding of the byteslice to the receiver of the data.

Thoughts on this @pawan ?

3 Likes

Getting a haircut during corona? You have it good! :wink: Barbershops are closed here.

@pawan have you thought about this?

2 Likes

Are there any news on this? It would be very helpful to parse a binary blob from the backend through dgraph to the client

S3 / minio integration to dgraph would be nice, that way you dont have to implement your own datastorage

@pawan is currently in quarantine. I’ll bring this up when I see him later in the month

1 Like

@hardik let’s accept this as a feature request for cloud.

3 Likes

@mrjn self managed would be nice, too. That way you can use your own existing dgraph + minio

2 Likes

What does “cloud” mean in this context? Slash-GraphQL?

As far as I understand, self-managed dgraph uses the same code-base, so the feature would naturally also be available there, right?

And I really like the idea of integrating with s3/minio!

Attaching binary blobs to the graph would really make dgraph stand out among competitors.

Dgraph Cloud includes Slash GraphQL.

1 Like

Hi guys, any news on this?

3 Likes

Hi @akon @Raphael, still desperate for this feature. Is this in the backlog?