I’d like to route upload mutations to my business-logic server using the @custom directive.
On the remote server I have defined an Upload scalar.
How can I use this scalar in the dgraph schema?
Things that (currently) don’t work:
scalar Upload @remote → Can’t use @remote on scalarsscalar Upload → Not allowed to define scalarsWhat are my options? I don’t want to talk to the second server directly.
Proposal
Allow @remote also for scalars.
Any opinions?
Hey @maaft
Could you share some more details about how do you plan to use this remote scalar? Are you thinking of using it from within a remote type? Can you use a string instead within the type?
Hi @pawan
I plan to use this specific scalar to send multipart/form-data uploads to my business-logic-server.
Yes, I would use it within a remote type but not necessarily.
Unfortunately I don’t know enough about file uploads so can’t answer your last question, but my feeling is that strings would not be sufficient.
Thanks, I need more details of how this is used in your remote server and how would this work with our GraphQL schema? Can you share some details of what problem you are trying to solve and why whatever we have currently is not enough?
Sure. Here is the schema that I’d like to use ideally.
scalar Upload @remote
type Mutation {
uploadFile(file: Upload!): Boolean!
@custom(
http: {
url: "${GQL_URL}"
method: POST
graphql: "mutation($file: Upload!) {uploadFile(file: $file)}"
forwardHeaders: ["Auth"]
}
)
}
Here is my server pseudo-code:
type Upload struct {
File io.Reader // So basic strings are probably not enough? Idk
Filename string
Size int64
ContentType string
}
func (r * mutationResolver) UploadFile(ctx context.Context, file Upload) (bool, error)
// read file stream
content, err := ioutil.ReadAll(file.File)
if err != nil {
result.Message = err.Error()
return &result, nil
}
}
The reason is that I already have a server for business logic (token generation, login etc…) and therefore use @custom mutations/queries. Which is great because I only need to expose one endpoint to the client. Ideally I want to implement the uploadFiles mutation also in this server.
However, I could imagine that this is not only useful for the Upload scalar but rather for any scalar types.
Hi @maaft,
I believe that having Upload as a scalar is not the ideal way to go about solving your problem.
And, then I don’t think you can do file uploads directly through GraphQL as it is just not possible to convey binary data through GraphQL. All the GraphQL travels as JSON over HTTP, and JSON doesn’t have any scalar that can represent binary file data.
So, instead what you can do is read the file at the client-side, base64 encode its contents, and send them over as a GraphQL string. Here is my interpretation of this solution:
type Upload @remote {
file: String! # this will contain the base64 encoded contents of the file
filename: String!
size: Int64!
contentType: String!
}
type Mutation {
uploadFile(file: Upload!): Boolean!
@custom(
http: {
url: "${GQL_URL}"
method: POST
graphql: "mutation($file: Upload!) {uploadFile(file: $file)}"
forwardHeaders: ["Auth"]
}
)
}
And then this would be your server-side code:
type Upload struct {
File string
Filename string
Size int64
ContentType string
}
func (r * mutationResolver) UploadFile(ctx context.Context, file Upload) (bool, error)
// decode file contents
content, err := base64.StdEncoding.DecodeString(file.File)
if err != nil {
result.Message = err.Error()
return &result, nil
}
}
File-Uploads using GraphQL are a common thing (see File Uploads with Apollo Server 2.0 - Apollo Blog).
They are also specified e.g. here: GitHub - jaydenseric/graphql-multipart-request-spec: A spec for GraphQL multipart form requests (file uploads).
The go-graphql-server I’m using has this scalar built in and I want to use it also in my dgraph schema (only on remote types). I understand that dgraph cannot (and probably won’t ever) handle file uploads. But I don’t like that I need to talk to different servers from my client and need to decide on a per-use-case basis when I need to talk to dgraph or when I need to talk to my business-logic-servers.
What are the problems (if any) that you see for supporting @remote scalars?
Thanks, I see.
On reading those docs I found that the GraphQL server is also handling multipart requests, but at present Dgraph only handles the standard application/json and application/graphql content types which come in as one single HTTP request and not as an HTTP request spread over multiple parts.
And, then the GraphQL multipart request specification that you shared has also defined a standard way of sending those request, and that is what enables them to be able to stream the file contents while the server is resolving it at the same time. This is very interesting.
In general, I don’t see any problems supporting @remote scalars, as long as they can be represented in JSON. Because, JSON is what is the standard way of transporting GraphQL, and that is what we support.
But, for this use-case, it requires supporting multipart requests, which is not the standard GraphQL way, and that is the reason we didn’t support it yet. And, then the moment you lose a pre-defined structure (which JSON provides), there can be innumerous ways to represent something, and the server may not know in advance how to parse it. Like, this particular scalar expects file to be a stream and the GraphQL server which doesn’t already know about this scalar, won’t be able to handle it. So, I don’t see it working as a remote scalar.
Instead, a better ask would be to have Dgraph support the GraphQL multipart request specification, which in turn would make it support the Upload scalar as well. Let me discuss it internally, and update you if this is something we would be able to work on in the near future.
EDIT: Linking a previous discussion about file uploads: Upload file mutation
Thank you @abhimanyusinghgaur for your kind thought-out responses. I think I understand this issue now and you might be right in saying that the real question to ask is: Can dgraph-graphql support file up-/downloads?
And if the answer to that will be yes, this would be totally awesome because we could ditch all this cumbersome custom code for attaching (and retrieving) binary data to/from the graph.
Should we maybe open a new issue regarding this? What do you think?
Hey @maaft, sorry for the very late response, was OOO.
Yes, please open a new feature request asking to support the GraphQL multipart request specification, and we will be happy to add that to Dgraph later this year.
Thanks