Sharing some numbers from the ludicrous mode

Well, I got insane numbers. That’s what I can state for you. I gonna share with you some tests that I made 24(?) hours ago.

Resume

I did the tests focusing on the maximum use of the machine’s resources. Some stats:

4GHz - 8 cores, 16 threads
32GB DDR4 DUAL Channel.
NVMe WD 500GB with 3GB/s of write.
Single Alpha and Zero (means, only two nodes)

I’ve used the 21 million dataset benchmarks/21million.rdf.gz at master · dgraph-io/benchmarks · GitHub
and benchmarks/release.schema at master · dgraph-io/benchmarks · GitHub

To get the most out of this machine I had to run Dgraph via binaries. And not through containerization or virtualization. Also, abuse the settings to push it beyond.

OBS: I forgot to mention that you have to use master to test it. You can do by building it from the repo or by using the master tag in docker.

The test

I did 4 types of tests (all in ludicrous mode, except the first one).

Normal values: --conc=10 --batch =1000

1 - Liveload in normal configurations and Nodes also normal.
2 - Liveload in normal configurations and Nodes using --badger.tables=ram and abusing other flags.
3 - Liveload in configurations with the flags --conc=30 --batch=10000 and Nodes using --badger.tables=ram and abusing other flags.
4 - Liveload in configurations with the flags --conc=100 --batch=10000 and Nodes using --badger.tables=ram and abusing other flags.

I also pushed other values, but they were not consistent. The best and safest were number 3.

To activate the lud mode

./dgraph zero --ludicrous_mode
./dgraph alpha --badger.tables=ram --ludicrous_mode

To abuse of its configs (not safe for day to day)

./dgraph zero --ludicrous_mode --rebalance_interval=40m0s
./dgraph alpha --badger.tables=ram --ludicrous_mode --abort_older_than=10m --snapshot_after=20000000

The numbers (Results)

1 - ./dgraph live -f /21million.rdf.gz -s /release.schema
First, without ludicrous mode the load takes 24 minutes more or less
I got decent results, but not amazing.
With a single spike of 61.200 N-Quads/s

Number of TXs run            : 21240                                                                
Number of N-Quads processed  : 21239870
Time spent                   : 24m35.126823599s
N-Quads processed per second : 14399

From my tests with ludicrous mode:

2 - ./dgraph live -f /21million.rdf.gz -s /release.schema

The load takes 6 minutes more or less - this is quite surprising!
I had a consistent load of higher values for 1 minute and 25 secs.
With 16 spikes of between 169.600 and 253.400 N-Quads/s! and consistent load until the end.

This was really impressive, and I was just starting.

Number of TXs run            : 21240                                                                
Number of N-Quads processed  : 21239870
Time spent                   : 6m22.860696732s
N-Quads processed per second : 55601

3 - ./dgraph live -f /21million.rdf.gz -s /release.schema --conc=30 --batch=10000

The load takes 1 minute 43 secs!!! this is LUDICROUS!
I had a consistent load of even higher values until the end!
With 16 spikes of between 230.000 and 302.100 N-Quads/s! and consistent load until the end.

Number of TXs run            : 2124                                                                 
Number of N-Quads processed  : 21239870
Time spent                   : 1m43.322019717s
N-Quads processed per second : 206212

4 - ./dgraph live -f /21million.rdf.gz -s /release.schema --conc=100 --batch=10000

The load takes 1 minute 39 secs!!! A new record! hehehe
I had a consistent load of even higher values until the end!
With all spikes of between 200.000 and 398.000 N-Quads/s! and consistent load until the end.

[23:42:09-0300] Elapsed: 05s Txns: 100 N-Quads: 1000000 N-Quads/s [last 5s]: 200000 Aborts: 0
[23:42:14-0300] Elapsed: 10s Txns: 200 N-Quads: 2000000 N-Quads/s [last 5s]: 200000 Aborts: 0
[23:42:19-0300] Elapsed: 15s Txns: 283 N-Quads: 2830000 N-Quads/s [last 5s]: 166000 Aborts: 0
[23:42:24-0300] Elapsed: 20s Txns: 400 N-Quads: 4000000 N-Quads/s [last 5s]: 234000 Aborts: 0
[23:42:29-0300] Elapsed: 25s Txns: 500 N-Quads: 5000000 N-Quads/s [last 5s]: 200000 Aborts: 0
[23:42:34-0300] Elapsed: 30s Txns: 600 N-Quads: 6000000 N-Quads/s [last 5s]: 200000 Aborts: 0
[23:42:39-0300] Elapsed: 35s Txns: 700 N-Quads: 7000000 N-Quads/s [last 5s]: 200000 Aborts: 0
[23:42:44-0300] Elapsed: 40s Txns: 800 N-Quads: 8000000 N-Quads/s [last 5s]: 200000 Aborts: 0
[23:42:49-0300] Elapsed: 45s Txns: 900 N-Quads: 9000000 N-Quads/s [last 5s]: 200000 Aborts: 0
[23:42:54-0300] Elapsed: 50s Txns: 1000 N-Quads: 10000000 N-Quads/s [last 5s]: 200000 Aborts: 0
[23:42:59-0300] Elapsed: 55s Txns: 1100 N-Quads: 11000000 N-Quads/s [last 5s]: 200000 Aborts: 0
[23:43:04-0300] Elapsed: 01m00s Txns: 1200 N-Quads: 12000000 N-Quads/s [last 5s]: 200000 Aborts: 0
[23:43:09-0300] Elapsed: 01m05s Txns: 1300 N-Quads: 13000000 N-Quads/s [last 5s]: 200000 Aborts: 0
[23:43:14-0300] Elapsed: 01m10s Txns: 1400 N-Quads: 14000000 N-Quads/s [last 5s]: 200000 Aborts: 0
[23:43:19-0300] Elapsed: 01m15s Txns: 1500 N-Quads: 15000000 N-Quads/s [last 5s]: 200000 Aborts: 0
[23:43:24-0300] Elapsed: 01m20s Txns: 1600 N-Quads: 16000000 N-Quads/s [last 5s]: 200000 Aborts: 0
[23:43:29-0300] Elapsed: 01m25s Txns: 1701 N-Quads: 17010000 N-Quads/s [last 5s]: 202000 Aborts: 0
[23:43:34-0300] Elapsed: 01m30s Txns: 1801 N-Quads: 18010000 N-Quads/s [last 5s]: 200000 Aborts: 0
[23:43:39-0300] Elapsed: 01m35s Txns: 2000 N-Quads: 20000000 N-Quads/s [last 5s]: 398000 Aborts: 0

Number of TXs run            : 2124                                                                 
Number of N-Quads processed  : 21239870
Time spent                   : 1m39.544146028s
N-Quads processed per second : 214544

Final considerations.

It’s not all flowers, Ludicrous mode makes mutations/queries non-transactional. And it is eventually consistent writes. Reads may not see the absolute latest data. It’s not the strictest guarantees that’s provided by Dgraph by default (paraphrasing Dmai). And it is not recommended for financial applications.

Ludicrous mode is still being polished on the master branch. Which means you can only use it for testing right now. It will officially come in version 20.03.1 (if so).

I am particularly super excited about this mode. It will be very promising for those who load huge datasets. BTW, If you can help by testing and sharing logs. We appreciate it.

Cheers!

Just adding tags for future readers using the search box:
[performance], [speed], [benchmark], [optimization], [tuning], [fast]

5 Likes

I forgot to mention that you have to use master to test it. You can do by building it from the repo or by using the master tag in docker.

Pasting the same comment I had done in other conversation.

Try the new feature (you gonna need more memory on docker, something like 14GB+ to run 21 million RDFs) called ludicrous mode (eventually consistent writes). It hasn’t been released yet, use it sparingly. And disable it after you have loaded the dataset.

This mode is not recommended for financial systems.

docker pull dgraph/dgraph:master
docker run -d -p 5080:5080 -p 6080:6080 -p 8080:8080 -p 9080:9080 -p 8000:8000 -v ~/dgraph:/dgraph --name dgraph dgraph/dgraph:master dgraph zero --ludicrous_mode
docker exec -d dgraph dgraph alpha --zero localhost:5080 --ludicrous_mode
docker exec -it dgraph sh

AND

curl --progress-bar -LS -o 21million.rdf.gz "https://github.com/dgraph-io/benchmarks/blob/master/data/release/21million.rdf.gz?raw=true"
curl --progress-bar -LS -o release.schema "https://github.com/dgraph-io/benchmarks/blob/master/data/release/release.schema?raw=true"

ls

Then

dgraph live -f 21million.rdf.gz -s release.schema --conc=100 --batch=10000

Decrease the values of conc and batch according to your machine (do some testing of parameters to find out which is the best)

1 Like

Hi @MichelDiz thanks for the awesome write up about ludicrous mode! What kind of data do you see being being imported/utilized with ludicrous mode? Even for data that isn’t related to the financial sector… wouldn’t it be useful that you guarantee you have all of the data, with consistent imports?

For example… let’s even take a forum for example… you load posts and comments…and comments of comments right? If you’re missing even 1 comment, and it shows a comment with no context to a missing comment, wouldn’t that mess stuff up?

Curious on your thoughts! Thanks :slight_smile:

In theory any type of data. It is just not recommended to use lud in applications that require ACID for example. That is a requirement in some types of applications, such as the financial ones.

In general, any application you would make with MongoDB, Cassandra or any other NoSQL DB that is “eventually consistent” you can use Dgraph’s ludicrous mode.

Ludicrous mode comes with reduced guarantees. And for example, there is no “commit” in ludicrous mode. It skips the “line” of guarantees and is therefore much faster.

2 Likes