How does --config + config.json work?

chris-carrington · December 19, 2019, 5:44pm

Hi,

I love dgraph, thank you for creating an A+ product!

I have this config.json file

{
  "my": "localhost:7080",
  "zero": "localhost:5080",
  "lru_mb": 4096,
  "postings": "/postings",
  "wal": "/wal"
}

How do I ask dgraph to use the config.json? I’ve tried each of these commands but they don’t compile

dgraph --config config.json
dgraph zero --config config.json
dgraph alpha --config config.json
dgraph-ratel --config config.json

Thank you!

MichelDiz · December 19, 2019, 5:51pm

This config file is just to Alphas and Zeros. And each instance type has its own config aspects. You can’t use the same for all. Ratel doesn’t use a config file. And there’s no “dgraph --config config.json” binary.

In overall config file is working correctly as far as I know. And about your JSON config, make sure you are pointing “postings” and “wal” to correct paths and with required permissions.

Cheers.

chris-carrington · December 19, 2019, 6:00pm

Hi MichelDiz!

Thank you for helping me! I really appreciate your swift reply! What’s an “instance type” please?

So I have this config.json

{
  "my": "localhost:7080",
  "zero": "localhost:5080",
  "lru_mb": 4096,
  "postings": "/postings",
  "wal": "/wal"
}

& in my terminal I run

dgraph alpha

And the output is

[Decoder]: Using assembly version of decoder
2019/12/19 12:54:15 LRU memory (--lru_mb) must be at least 1024 MB. Currently set to: -1.000000
For more information on --lru_mb please read https://docs.dgraph.io/deploy/#config

If my “config file is working correctly” & it contains “lru_mb”: 4096 what’s causing the error?

MichelDiz · December 19, 2019, 6:09pm

Alpha, Zero, Ratel are “instances type”. (But that is just my way of speaking - this “instances type”)

Cuz you’re not using any flag. Use dgraph alpha --config config.json

chris-carrington · December 19, 2019, 6:24pm

You’re a wizard! I’m picking up what you’re putting down! Thank you for explaining this to me!

So when I run dgraph alpha --config config.json I get the error 2019/12/19 13:20:22 mkdir /wal: permission denied

I already have a /wal folder created @ the same tree level as config.json. Are you aware of how I may solve this permission denied error & why it’s performing a mkdir when the directory exists? I’m on a macbook pro btw.

Thank you so much!

MichelDiz · December 19, 2019, 6:30pm

I do not recommend put the files in the root /, so move it to ~/.

{
  "my": "localhost:7080",
  "zero": "localhost:5080",
  "lru_mb": 4096,
  "postings": "~/dgraph/alpha0/postings",
  "wal": "~/dgraph/alpha0/wal"
}

To solve any issue with the permissions. Just right click to “get info” and change the permissions. Or chmod -R 777 ./dgraph (or ~/dgraph if you’re not in the same path in the terminal).

chris-carrington · December 21, 2019, 2:42pm

Thank you @MichelDiz! I appreciate all your guidance here & throughout the forums!

chris-carrington · December 22, 2019, 12:28am

Hi @MichelDiz!

I have the config.json as you described, thank you! When I run dgraph alpha --config config.json I get the error Error while creating badger KV WAL store full output below

$ dgraph alpha --config config.json
[Decoder]: Using assembly version of decoder
I1221 19:12:21.975798    2666 init.go:98] 

Dgraph version   : v1.1.1
Dgraph SHA-256   : 02f468aaea43acc1b0a1c88f6d465a2d329cad4b69bdff3fe599f910eb284a90
Commit SHA-1     : 8994a5740
Commit timestamp : 2019-12-16 18:24:50 -0800
Branch           : HEAD
Go version       : go1.13.5

For Dgraph official documentation, visit https://docs.dgraph.io.
For discussions about Dgraph     , visit http://discuss.dgraph.io.
To say hi to the community       , visit https://dgraph.slack.com.

Licensed variously under the Apache Public License 2.0 and Dgraph Community License.
Copyright 2015-2018 Dgraph Labs, Inc.


I1221 19:12:21.976289    2666 run.go:511] x.Config: {PortOffset:0 QueryEdgeLimit:1000000 NormalizeNodeLimit:10000}
I1221 19:12:21.976313    2666 run.go:512] x.WorkerConfig: {ExportPath:export NumPendingProposals:256 Tracing:1 MyAddr:localhost:7080 ZeroAddr:localhost:5080 RaftId:0 WhiteListedIPRanges:[] MaxRetries:-1 StrictMutations:false AclEnabled:false AbortOlderThan:5m0s SnapshotAfter:10000 ProposedGroupId:0}
I1221 19:12:21.976344    2666 run.go:513] worker.Config: {PostingDir:~/dgraph/whystle/alpha0/postings BadgerTables:mmap BadgerVlog:mmap WALDir:~/dgraph/whystle/alpha0/wal MutationsMode:0 AuthToken: AllottedMemory:4096.0MB AccessJwtTtl:0s RefreshJwtTtl:0s AclRefreshInterval:0s}
I1221 19:12:21.976399    2666 server_state.go:101] Setting Badger table load option: mmap
I1221 19:12:21.976403    2666 server_state.go:113] Setting Badger value log load option: mmap
I1221 19:12:21.976408    2666 server_state.go:158] Opening write-ahead log BadgerDB with options: {Dir:~/dgraph/whystle/alpha0/wal ValueDir:~/dgraph/whystle/alpha0/wal SyncWrites:false TableLoadingMode:1 ValueLogLoadingMode:2 NumVersionsToKeep:1 ReadOnly:false Truncate:true Logger:0x2a35d58 Compression:1 EventLogging:true InMemory:false MaxTableSize:67108864 LevelSizeMultiplier:10 MaxLevels:7 ValueThreshold:1048576 NumMemtables:5 BlockSize:4096 BloomFalsePositive:0.01 KeepL0InMemory:true MaxCacheSize:10485760 NumLevelZeroTables:5 NumLevelZeroTablesStall:10 LevelOneSize:268435456 ValueLogFileSize:1073741823 ValueLogMaxEntries:10000 NumCompactors:2 CompactL0OnClose:true LogRotatesToFlush:2 ZSTDCompressionLevel:15 VerifyValueChecksum:false EncryptionKey:[] EncryptionKeyRotationDuration:240h0m0s ChecksumVerificationMode:0 managedTxns:false maxBatchCount:0 maxBatchSize:0}
2019/12/21 19:12:21 resource temporarily unavailable
Cannot acquire directory lock on "~/dgraph/whystle/alpha0/wal".  Another process is using this Badger database.
github.com/dgraph-io/badger/v2.acquireDirectoryLock
        /tmp/go/pkg/mod/github.com/dgraph-io/badger/v2@v2.0.1-0.20191210152157-7c5e6d7f6e8b/dir_unix.go:66
github.com/dgraph-io/badger/v2.Open
        /tmp/go/pkg/mod/github.com/dgraph-io/badger/v2@v2.0.1-0.20191210152157-7c5e6d7f6e8b/db.go:236
github.com/dgraph-io/dgraph/worker.(*ServerState).initStorage
        /tmp/go/src/github.com/dgraph-io/dgraph/worker/server_state.go:161
github.com/dgraph-io/dgraph/worker.InitServerState
        /tmp/go/src/github.com/dgraph-io/dgraph/worker/server_state.go:58
github.com/dgraph-io/dgraph/dgraph/cmd/alpha.run
        /tmp/go/src/github.com/dgraph-io/dgraph/dgraph/cmd/alpha/run.go:515
github.com/dgraph-io/dgraph/dgraph/cmd/alpha.init.2.func1
        /tmp/go/src/github.com/dgraph-io/dgraph/dgraph/cmd/alpha/run.go:88
github.com/spf13/cobra.(*Command).execute
        /tmp/go/pkg/mod/github.com/spf13/cobra@v0.0.5/command.go:830
github.com/spf13/cobra.(*Command).ExecuteC
        /tmp/go/pkg/mod/github.com/spf13/cobra@v0.0.5/command.go:914
github.com/spf13/cobra.(*Command).Execute
        /tmp/go/pkg/mod/github.com/spf13/cobra@v0.0.5/command.go:864
github.com/dgraph-io/dgraph/dgraph/cmd.Execute
        /tmp/go/src/github.com/dgraph-io/dgraph/dgraph/cmd/root.go:68
main.main
        /tmp/go/src/github.com/dgraph-io/dgraph/dgraph/main.go:73
runtime.main
        /usr/local/go/src/runtime/proc.go:203
runtime.goexit
        /usr/local/go/src/runtime/asm_amd64.s:1357
Error while creating badger KV WAL store
github.com/dgraph-io/dgraph/x.Checkf
        /tmp/go/src/github.com/dgraph-io/dgraph/x/error.go:49
github.com/dgraph-io/dgraph/worker.(*ServerState).initStorage
        /tmp/go/src/github.com/dgraph-io/dgraph/worker/server_state.go:162
github.com/dgraph-io/dgraph/worker.InitServerState
        /tmp/go/src/github.com/dgraph-io/dgraph/worker/server_state.go:58
github.com/dgraph-io/dgraph/dgraph/cmd/alpha.run
        /tmp/go/src/github.com/dgraph-io/dgraph/dgraph/cmd/alpha/run.go:515
github.com/dgraph-io/dgraph/dgraph/cmd/alpha.init.2.func1
        /tmp/go/src/github.com/dgraph-io/dgraph/dgraph/cmd/alpha/run.go:88
github.com/spf13/cobra.(*Command).execute
        /tmp/go/pkg/mod/github.com/spf13/cobra@v0.0.5/command.go:830
github.com/spf13/cobra.(*Command).ExecuteC
        /tmp/go/pkg/mod/github.com/spf13/cobra@v0.0.5/command.go:914
github.com/spf13/cobra.(*Command).Execute
        /tmp/go/pkg/mod/github.com/spf13/cobra@v0.0.5/command.go:864
github.com/dgraph-io/dgraph/dgraph/cmd.Execute
        /tmp/go/src/github.com/dgraph-io/dgraph/dgraph/cmd/root.go:68
main.main
        /tmp/go/src/github.com/dgraph-io/dgraph/dgraph/main.go:73
runtime.main
        /usr/local/go/src/runtime/proc.go:203
runtime.goexit
        /usr/local/go/src/runtime/asm_amd64.s:1357

And with your requested addition of "postings": "~/dgraph/alpha0/postings" dgraph is adding the folders @ the config.json root is this desired?

Thank you!

MichelDiz · December 22, 2019, 12:52am

This log usually appears when you have another instance using the same path. Or an old instance that was not properly shut down.

Open HTOP (if you have it installed), press F4 type “dgraph” then press F9 and SIGTERM or SIGKILL. Then restart your cluster from scratch. It is not good to keep previous data when problems like this happen. You can try tho.

Are you using a single instance configuration? (1 Alpha + 1 Zero)

system · January 21, 2020, 12:52am

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.

Topic		Replies	Views
Config - Deploy Documentation	0	447	August 28, 2020
Add JSON config file for Ratel Ratel kind:enhancement , ratel	3	478	April 16, 2020
Initial setup help? Ratel help-wanted , ratel	23	2239	August 5, 2020
How to configure port for alpha , zero and ratel Dgraph	8	1874	February 18, 2021
Some confusion about paramaters in Cluster Config Dgraph	4	677	July 5, 2019

How does --config + config.json work?

Related topics