The first thing I thought when I saw this illustration was “amazing! how they managed to make data validation so easy”.
But then I was reviewing the issue of validations very thoroughly and I see that they don’t exist.
In a realworld application. How would it be possible to mutate data directly from your application to the server without validating simple data like an email?
1 Like
Welcome @miguel to the Dgraph community!
Regarding validation, you could write your validation business logic in a service and use a custom mutation in graphql to integrate it. Dgraph will invoke the validation logic when the corresponding mutation is being performed. Here is the documentation for this feature.
2 Likes
Hey @anand,
If eventually Dgraph decides to introduce validation in GraphQL Schema.
I would like to propose that the team comes up with an API similar to Yup.
I am using it in the front end to send proper data to the backend. See:
YupHey, I’d more recommend validation like it’s implemented in Firestore.
Also a directive like @validate(…) would be more fitting into the Dgraph theme.
(Btw. is it possible to make an alias from e.g. “String @validate(…)” a directive so you could create a type called “Email” or something to have validation already implemented?)
1 Like
IMHO validation should be done at the application level. Because it reduces the processing load on the back-end and validating values such as email is simple to do with regex patterns. Not to mention that besides having to create a whole new logic, it is necessary to add an error handling in GraphQL for that. Which is an extra code to maintain.
At the server level, I would just check if the email already exists(the user register).
If you notice formik is done for integration with the UI. I personally find it a mistake to want the database to do all the work. We should recommend practices and not force different functions like typo checking. I think it fragments and disrupts the main function of the DB. Which is to be a DB.
Cheers.
@MichelDiz - I 100% disagree with this statement, and I believe most programmers will too. You should have BOTH frontend and backend validation.
You’re basically saying backend security is not important, except user roles authentication @auth
Let me give you some use cases of why this is imperative, and IMHO what is the only missing real feature of dgraph.
- Check if a user’s email is verified
- Check if a post has the minimum / maximum character requirement
- Only allow someone to like a post once
- Stop invalid timestamps (not necessarily just createdAt or updatedAt)
- Stop a game user from hacking levels, resources paid for, etc
- Lock certain fields (types), banned users…
- A hacker could easily overload your database with posts greater than the max size etc…
- This list can go on, and on…
I have personal experience with people hacking my apps because my Firestore rules were not implemented. It is not difficult to hack a front-end app.
I know there are many references to why backend field validation security is important just on these forms, a google search will immediately point you to cyber attacks etc.
I am not sure how that works, but it sounds like this just fix the speed problem if it were implemented this way. Either way, it is way more important to have fast queries than to have fast mutations. Every single database front-end service that I know of provides this. It is a must. I hope the Dgraph team takes this seriously, as IMHO this is the only real feature missing for a commercial app, and it is a security related.
The work-around, custom mutations, is a pain.
Thanks,
J
4 Likes
Do you have a fundamental reason? Why two typo checks for the same value? don’t you believe in the machine to correct the user?
Typo isn’t security. Why an attacker would force a typo if he gonna have to force it over and over? Unless the typo generates a glitch out something else in the business logic - somehow. If you concern, instead of making a machine to “double-check” should fix the issue at the root of the cause. Right?
This is part of a complex service. You should do it in a Lambda server following some standard out there.
Another business logic rule. You can also do it via Lambda.
Another business logic rule.
Really out of scope.
No DB is safe from a DDOS. No matter what we do, no matter how many typo checks we add. If a hacker wanna DDOS you, he will. Unless you contract some anti-DDOS service like Cloudflare.
It is just a side server that runs JS and you can attach it to your GraphQL Schema.
So, data validation isn’t about security. Is about data consistency, about the correct format. Putting it in a DB Is to protect your data model from lazy front-end devs from breaking the rules. But isn’t mandatory and doesn’t protects you from hacking.
If you need validation, just create a lambda server validating it. But be aware that this increases overall latency.
Yes. It is very easy to hack the front end. It is not on the backend.
I am not sure what you mean here, but it is definitely not just for the purposes of typo checking.
Not necessarily. You may have a field that needs to be checked, isUserValidated, that could easily be check that way.
Lambdas solve all these problems, but are a pain and way slower than say boolean checks.
Not really out of scope at all. I don’t want a user to be able to add 100,000 points, instead of 10 points, just by a simple javascript hack. Again, I imagine there are way faster ways to than lambdas to secure this.
Agreed. However, there are great security, and there are poor security practices.
Data validation is not just about data consistency. I do not want my app to get hacked. If people pay for features, they should not be able to hack them. This is just one example. I would also argue that data consistency could be a security risk in and of itself.
https://betterprogramming.pub/back-end-data-validations-73ea9004c6d7
@MichelDiz I think this is something that is a necessity for any commercial products, but not in all cases. There is an argument about this:
I definitely don’t think you need to validate everything on the back end, but it is a must for certain things. Again, I have direct experience with this.
Sure, lambdas can handle this, but why not make it easy and faster. Lambdas are a pain. Simple boolean checks would be wonderful. Here is an example of a firestore rule:
let isValidLength = username.size() >= 3 && username.size() <= 15;
All I would have to do is use Chrome dev tools to hack a form. If my page is based on a username, I could permanantly have my site indexed with a username like so:
I do not want this to even be an option.
I am pro this post: http://discuss.dgraph.io/t/moving-graphql-authorization-to-admin-api/13304
Fauna DB (as new as Slash DGraph) currently does not have a solution to this either, and uses something similar to a pre-hook: https://forums.fauna.com/t/data-validation-in-fql-with-abac/329/2
I think a pre-hook may be an option I could be okay with. The problem with lambdas is that I have to basically disable my mutation and create a new one. This is the majority of the painstaking process.
#add-prehook
It would be interesting to hear from other people on this.
Thanks,
J
3 Likes
User validation requires a validation procedure external to the database. For example, in some cases, it is necessary to send an email to the user so that the user then clicks on a link and validates the email. This is not a job for a database. Perhaps a specific platform (e.g. e-commerce platform or a CRM can offer this).
Anyway, at some point, you would need to develop your own business logic. Lambdas are there to help.
It is out of scope cuz Dgraph isn’t a Game Framework. It does not make much sense to put specific features for each user’s case. We would have to incorporate thousands of framework features. This is really not a job for a DB. Right?
Then you add those in the Back-end. We can’t support each piece of a feature in your game in the DB. That’s a job for a back-end server. Dgraph is just a repository for data.
We also support Apollo Federation. You can have a nice GraphQL server with any complex feature your game has. And Dgraph will be in sync with Apollo Federation.
At some level, GraphQL spec covers the minimum rules on this. For example, a user can’t send an INT in a String. GraphQL validates the objects. It is very strict to the structure you have. So, front-end devs won’t break anything. An end-user can’t exploit a GraphQL server in general. In the end, only typo validation is left. A user who forces a wrong email won’t hack your server.
I see in part your point. You can have a validation back-end with Dgraph. But Dgraph itself is a DB, we can’t do any unusual validation for a DB. If we make an exception, soon we will be the largest framework as DB in Latin America.
In my opinion (remembering that everything I wrote is just my opinion). You should have more control over your backend and not expect a DB to do herculean work for you and your team(I say any DB, not the Dgraph team - we gonna often offer some unusual feature tho). If you depend on a third party for every bit of your structure in production, you will have a lot of trouble growing and expanding. As a developer, I don’t like the idea of relying on a single point. It is a point of failure. I like to have mastery of my production Be it part open-source or partly closed. Just like a maestro - And in an orchestra, everything has its role. We don’t have a drummer in an orchestra for example - if you know what I mean.
If you expect the DB to do the job of an API + a game framework + a data validator + etc. You will wait years to finish your own project.
Try Apollo Federation. It can mix services in GraphQL servers. It might be clear and fast for you. As you can spin up a GraphQL server in Go, Elixir, or Rust. If you wanna something better than JS.
Cheers.
1 Like
I am not creating a game, that was just one example. Any product with features that a user should not be able to hack are examples. I can’t send an INT as a string, but I could change something I buy to 10,000 quantity instead of 1. This could be game points, hours left, anything really. This is not specific but general. You should not be able to hack this.
Yes, it is. But Slash DGraph, or Cloud Dgraph is not. It is a “full backend” according to the website.
Apollo Federation is way too powerful for basic use cases. I believe Slash Dgraph was created for small startups. I am not talking about “unusual validation.”
I think the best way to solve this without a new feature specific to validation is with a pre-hook. Yes, you need business logic for most cases, not specific. Of course that pre-hook would have to have the ability to cancel the transaction if certain conditions were not met.
J
4 Likes
It is not full in this sense, it is about the cluster administration. it says “Scale your business, not your DevOps budget || Build and scale with your backend managed by us. Operate thousands of clusters at once without the DevOps headache.”
This is in the DB scope, we can’t manage your backend. If you wanna introduce an image repository, for example, you should use a bucket or similar and make a custom business logic to attach it(or just point out URLs). If you wanna add a video streaming system, we can’t manage it for you. if you take it literally, Dgraph Cloud should do everything indeed. But this is just a matter of publicity. You can’t be very specific in an advertisement. There’s no space for big texts in ads hehehe But the “basic” backend is possible tho.
@MichelDiz Slash Dgraph is made to be called from the frontend in a secure way or @auth would not exist. I don’t think anyone is asking Dgraph to do everything. Dgraph should add the ability the manage your backend easier than @lambdas when it comes to field validation. That is all I am saying.
Image Repository, Video Logic, etc would be an external source. Field Validation, whether using pre-hooks or some sort of field rules is not custom, but a regularly used needed feature for ALL users.
I would also argue that right now I am literally paying DGraph to manage my backend so I don’t have to.
J
5 Likes
So, I would agree with @MichelDiz if Dgraph was only a database and was not meant for direct connection from front end services. If it was this it would have a feature set similar to Neptune or Mongo or SQL, but Dgraph is not just another database. It is a complete Backend in a single service. This was made clear with the rollout of 20.03. The GraphQL API is meant for direct front end connectivity not only for another backend layer to connect. @auth is a great example of the reasoning in this post.
This does change a lot for Dgraph and while engineers at Dgraph may think of what they are building as a database only, it is not just that at the GraphQL endpoint.
Saying this… there needs to be a way to add in middleware to the mutations. @auth is an example of one form of middleware, but users may still need other forms.
The original idea of JS hooks was to have this pre/post syntax to do just this. It changed a lot from rfc to implementation and is still lacking from the original rfc. There should be a way to do this middleware in the generated mutations.
6 Likes
I agree that this would be pretty useful to have. A validation directive would be pretty awesome. Currently, this is possible using lambda mutations and anonymous access rules (cloud only)
6 Likes
I would recommend you to check out type-graphql. Powerful custom validation that runs on the graphQL layer (which can also be serverless). Another big advantage: your validation will be independent from whatever database you use.
1 Like
Hi @MentalGear. Welcome!
The GraphQL types are already created within Dgraph. There is already a GraphQL API. We don’t need middleware, as this will greatly slow down the code. The problem here is backend validation. As I talked about in this post, we can actually already do more validation than we realize, but having an update-after logic would greatly solve a lot of them (there no point in securing what you can add if you can’t secure what you can update).
J
1 Like