Wondering how this translates to situations where I have multiple nodes that I want to increment/decrement in a batch. It does not fire off a set mutation for each node because DQL does not process foreach loops… proof:
Here is an example schema and data set:(yes I am combining graphql with DQL)
type counter {
id: ID
val: Int
}
mutation {
addcounter(input:[{val:1},{val:2},{val:3}]) {
numUids
counter {
id
val
}
}
}
If I run your upsert and then the following query:
upsert {
query {
counter(func: has(counter.val)) {
old: A as counter.val
new: B as math(A+1)
}
}
mutation @if(gt(len(A), 0)) {
set {
uid(A) <counter.val> val(B) .
}
}
}
I think that issue doesn’t apply here cuz the value is coming from math. That means it is out of the var mapping - which is the problem that causes issues in wide queries like this using has func. This increment approach should work fine with several nodes.
BTW, this gives an idea about the loop issue… If we had a “string” function as the “math” one, I think the issue with the mapping vars would be fixed. Like this Custom Block - Discussion
Sorry, copy/paste/edit error. Fixed it in my question.
I was just copying @dmai upsert example exactly as provided. I don’t know why he used it.
Nonetheless, the results stayed the same as the original and the upsert did not increment the count.val as the @dmai stated it should. Not sure what I missed here. I can retry again after a little while to see if I had another typo somewhere in my code.
hmm, I probably had a typo in my test then. I only use GraphQL when doing a DQL test, because it is faster to get a test environment up and running in Slash and test, and kill it, then it is to startup Dgraph locally for me. And it ensures my test environment is the same as my production environment excluding schema/data.
This solution will still have problems if 999 user like something at the same time. Right? (because 998 mutations/transactions would fail)
Or am I missing something?
this is the final code. right? This is an upsert mutation, and therefore a transaction (since everything in dgraph is a transaction). if 999 users run that at the same time, 998 will fail. or not?
Well, in theory it could happen but with some really huge value of transactions. Even tho you could try to use the upsert directive(this is a different topic from upsert block). BUT, I don’t see any reason to bother with it. As you would never in a real scenario have a thousand users giving a like to the same post at the same time. You have to have a very singular situation. And the likes should be at the nanosecond level, There should be a perfect astronomical junction. If Michael Jackson were alive and one of those users, maybe
If you wanna a “perfect” counter I recommend using edges instead. It will always be a single/unique count from one user to another and will never ever have such a possibility you have mentioned.
An example of this would be if Justin Bieber (who has 114 million followers) posted a new tweet on twitter. Everyone would see that tweet at once and it could easily get 1000 likes in one second (particularly in the first few hours).
If you have a live event on your website where many people will be active at once, there is another use case.
A game could have this.
I would definitely NOT say a real world scenario would NEVER have this. If Dgraph scales like it claims it does (although I cannot find any soft limits here), real world apps could definitely have a use case for this. Of course, Dgraph needs to be secure as well etc…
I think a Dgraph cluster(well configured) will handle 1k requests per second just fine for a case like this(this context of increment with Upsert Block).
A second is a lot of time in terms of the internet. You can’t be sure about this measurement, cuz the time you are seeing the 1k number for sure already passed a few seconds.
Also, several services (e.g: Youtube) no matter their size in resources, they will kind “hold” the stats in a line/queue and give to the user the values consolidated later. Just like YT had the 300 views limit, back then.
For sure Twitter has its own way of dealing with this. Their stack is a mix of several DBs, cache and so on. They even use FlockDB(a Graph DB) in the end of the tail.
PS. Also think about geographical differences.
Games in general uses UDP (UDP doesn’t give packet confirmations) and other type of logic in storing data on demand. Really different from a web application.
I bet Dgraph does it, but I can’t say for sure without real-world usage. We can just theorize. Would be nice to have a big tech pushing hard on Dgraph. Or some one trying to simulate the chaos.
We have some nice tests of reliability, but not for really specific things. Like incr/Decr upsert block.
thank you very much!!! As @jdgamble555 said, it’s possible, instagram for example shows new post of justin bieber directly to a large set of followers because all his post get much range, so they don’t have to test the post with small range groups like they do with other accounts. Also because I am planning to not build a like feature like Instagram, instead I want to build a hand clap feature like medium has. At medium.com you can clap 50 times,
And every single clap (if you dont keep it pressed) is a single request to the server.
The counter doesnt have to be perfect, latency and delay is no problem at all since it isn’t a crucial feature.
is the edge solution a huge waste of resources? or is it OK to count/aggregate the likes much frequently? I have no experience with that .-. I dont want to overuse my dgraph server unnecessary to save money.
What do you mean with upsert block? https://dgraph.io/docs/mutations/upsert-block/ I found that article, but that’s the same solution that was discussed in this thread here or not? Can you please explain that solution option further and enlighten me? (._.)
For me edges are simpler than INT with increment and decrement. And thousand times reliable. It uses UIDs, which has no difference in storage usage from a INT as it is just an address. The problem for me with INT in this case is the complexity of the query.
An edge(relational) is a source of truth more reliable you can have in Dgraph.
Today’s games on the app store can use the same database scenarios and protocols as any web app. A lot of them are really just complicated web apps. In fact, I would say most games will not need strong GUIs etc.
However, I get your point.
True. If you really get to that point you probably will have some kind of memcache etc. But, it would be interesting to see those tests. However, until DGraph GraphQL gets the updates it deserves for features and security, I don’t see any big tech using Dgraph for a while.
1000% agree that increment and decrement are terrible. Use Firestore for half a second and you learn this terrible lesson.
Edges or Facets, however, are not available yet in Dgraph Graphql, which means creating another node will have to do for now. Dealing with two schemas and all the other horrors of mixing DQL and GraphQL are never a good idea IMHO.
dgraph has security issues?? which are they? i wanted to use dgraph in production now. are there things I have to fear about?
which crucial features are actually still missing? (so that I know what I have to think through to compensate these features)
because 'till now, I am quite happy with dgraph, it has everything I need (i believe), but maybe I’m forgetting something important to handle. Which important features are missing? (telling the names of the features is enough for me I will then google them through and think about if they are a must have I can’t renounce. please educate me about features that are missing)
I would say there are some security holes, but there are also some work arounds if you don’t mind creating custom mutations for every mutation you have.
There are some giant holes in features that Dgraph is missing (particularly in Graphql, not necessarily DQL), but there seems to be some work-arounds to most of them (although it can be a lot of extra work).
The core team will rarely communicate with you, but @MichelDiz checks these forms multiple times a day.
There has been no new versions since March, but the next version is supposedly a few weeks away (although it has been a few weeks away for 5 months).
The next version seems to only include major bug fixes, and a few new features that are arguably bug fixes in and of themselves, but they seem to take bugs seriously and do fix them asap - 21.03.X
There has only been a few new features confirmed for the future, which won’t be in the next version but the version after that, but should be big fixes to only SOME of these problems (pre-hooks anyone?).
There is an average of 12 or so posts a day, and only @MichelDiz responds to any of them on a daily basis. The team seems too busy to read these posts, or give simple updates. I believe the company has some jobs they don’t need, and they obviously don’t have enough programmers. However, this is just my suspicion. That being said, the product is amazing.
I tried to leave a positive with a negative, as I am a firm believer in this product.
I do not recommend that you use another product, but I do recommend that you map out your application to see what features you do and do not use before you start down the long path of creating it. Just like anything else, there is a point of no return. I would recommend this to anyone with any app.
You need to do your research, as there are many posts on some of these topics. I cannot not tell you what you need. Keep in mind, every product has their pros and cons. I am not going to get into any specific feature details etc, as there are already many posts on all of that.
thank you very much!!! yesss Michel is a sweetheart an awesome guy he helps everywhere 247 in the dgraph forum, along with some other people that reply very often like amaster, or also you, the dgraph community is really great and ai am very happy to have so many kind helpful souls here
I want to do my research, but i dunno honestly where to start (._.) is there a list or anything with all threads that I should go through? or should I go all forum posts through? because there are sooo many forum posts it would take months going through every thread
BTW, with security holes, you mean security holes if users use the dgraph auth system and directly access dgraph, or? I am using client → cloudflare worker / cloud function → dgraph
So I won’t have to fear about any of these security problems since only my cloud functions access dgraph directly, and I dont give any possibilities to anyone to access my dgraph instance. right? or?
this is what I want to do:
With that setup, there are no security holes anymore. or?
I assume you mean for mutations and not for queries, but generally, yes. This is also assuming your cloud functions are written securely, the GraphQL security problems would not be an issue.
Side note, I would suggest you learn DQL in the cloud function (don’t give anyone direct access to it of course), as it will be much faster since GraphQL compiles to DQL. This could be more code and would still be an optional enhancement.
Definitely don’t do that. That may take a while - I would map out your app in a flow chart or knowledge graph and think about how you would handle vulnerabilities etc like in this pic (thanks google images)… Search the forums for just features you are unsure about.
which are they? i wanted to use dgraph in production now. are there things I have to fear about?
I will then google them through and think about if they are a must have I can’t renounce. please educate me about features that are missing)
- I would map out your app in a flow chart or knowledge graph and think about how you would handle vulnerabilities etc like in this pic (thanks google images)… Search the forums for just features you are unsure about.