Errors running dgraph/dgraph:master with Docker Compose

forstmeier · December 18, 2020, 7:39pm

Hi!

I’m attempting to take advantage of some of the features that are available in the master branch but I’m encountering errors trying to run this docker-compose.yaml (525 Bytes) file.

The error appears to be related to an unsupported Badger version but I’m not entirely sure. If there’s a recommended tag that contains “cutting edge” features (e.g. @auth support on interfaces or the improved @auth array/regex/etc support) that’s also something I’d be interested in. Thanks!

Best,
John

dmai · December 18, 2020, 9:58pm

Hey, John. The @auth support for interfaces is available in Dgraph v20.11 (dgraph/dgraph:v20.11.0). The auth token support for regex and arrays was merged this week and is available in master (dgraph/dgraph:master).

If you’re coming from a previous version of Dgraph you may get the error message about the “unsupported Badger version”. In that case, you’ll want to run that version, export your data, and re-import that data into the new version: https://dgraph.io/docs/deploy/dgraph-administration/#upgrading-database

forstmeier · December 18, 2020, 10:44pm

Hi @dmai!

Thanks for the info on the features/tags - super helpful!

When you say “coming from a previous version” what do you mean? Previously I had been running with dgraph/dgraph:latest and now wanted to shift up to the master (or other tags you mentioned). This was done via the images themselves. Do I need to clear out the volumes or something? Also, I wasn’t aware the exporting data was available to non-enterprise users.

Here’s the error for reference:

alpha_1   | I1218 22:40:49.219021      16 run.go:696] x.Config: {PortOffset:0 QueryEdgeLimit:1000000 NormalizeNodeLimit:10000 MutationsNQuadLimit:1000000 PollInterval:1s GraphqlExtension:true GraphqlDebug:false GraphqlLambdaUrl:}
alpha_1   | I1218 22:40:49.219036      16 run.go:697] x.WorkerConfig: {TmpDir:t ExportPath:export NumPendingProposals:256 Tracing:0.01 MyAddr:alpha:7080 ZeroAddr:[zero:5080] TLSClientConfig:<nil> TLSServerConfig:<nil> RaftId:0 WhiteListedIPRanges:[{Lower:172.20.0.1 Upper:172.20.0.1}] MaxRetries:-1 StrictMutations:false AclEnabled:false AbortOlderThan:5m0s SnapshotAfter:10000 ProposedGroupId:0 StartTime:2020-12-18 22:40:48.906295342 +0000 UTC m=+0.008309327 LudicrousMode:false LudicrousConcurrency:2000 EncryptionKey:**** LogRequest:0 HardSync:false}
alpha_1   | I1218 22:40:49.219063      16 run.go:698] worker.Config: {PostingDir:p PostingDirCompression:1 PostingDirCompressionLevel:0 WALDir:w MutationsMode:0 AuthToken: PBlockCacheSize:697932185 PIndexCacheSize:375809638 WalCache:0 HmacSecret:**** AccessJwtTtl:0s RefreshJwtTtl:0s CachePercentage:0,65,35,0 CacheMb:0}
alpha_1   | I1218 22:40:49.219215      16 log.go:295] Found file: 9 First Index: 0
alpha_1   | I1218 22:40:49.220228      16 storage.go:132] Init Raft Storage with snap: 0, first: 1, last: 0
alpha_1   | I1218 22:40:49.220246      16 server_state.go:76] Setting Posting Dir Compression Level: 0
alpha_1   | I1218 22:40:49.220254      16 server_state.go:120] Opening postings BadgerDB with options: {Dir:p ValueDir:p SyncWrites:false NumVersionsToKeep:2147483647 ReadOnly:false Logger:0x2e04cb8 Compression:1 InMemory:false MemTableSize:67108864 BaseTableSize:2097152 BaseLevelSize:10485760 LevelSizeMultiplier:10 TableSizeMultiplier:2 MaxLevels:7 ValueThreshold:1024 NumMemtables:5 BlockSize:4096 BloomFalsePositive:0.01 BlockCacheSize:697932185 IndexCacheSize:375809638 NumLevelZeroTables:5 NumLevelZeroTablesStall:15 ValueLogFileSize:1073741823 ValueLogMaxEntries:1000000 NumCompactors:4 CompactL0OnClose:false ZSTDCompressionLevel:0 VerifyValueChecksum:false EncryptionKey:[] EncryptionKeyRotationDuration:240h0m0s BypassLockGuard:false ChecksumVerificationMode:0 DetectConflicts:false managedTxns:false maxBatchCount:0 maxBatchSize:0}
alpha_1   | [Sentry] 2020/12/18 22:40:49 Sending fatal event [bd58a02027a24df5a48dc82654e39ae8] to o318308.ingest.sentry.io project: 5208688
alpha_1   | 2020/12/18 22:40:49 manifest has unsupported version: 7 (we support 8).
alpha_1   | Please see https://github.com/dgraph-io/badger/blob/master/README.md#i-see-manifest-has-unsupported-version-x-we-support-y-error on how to fix this.
alpha_1   | Error while creating badger KV posting store
alpha_1   | github.com/dgraph-io/dgraph/x.Checkf
alpha_1   | 	/ext-go/1/src/github.com/dgraph-io/dgraph/x/error.go:51
alpha_1   | github.com/dgraph-io/dgraph/worker.(*ServerState).initStorage
alpha_1   | 	/ext-go/1/src/github.com/dgraph-io/dgraph/worker/server_state.go:124
alpha_1   | github.com/dgraph-io/dgraph/worker.InitServerState
alpha_1   | 	/ext-go/1/src/github.com/dgraph-io/dgraph/worker/server_state.go:54
alpha_1   | github.com/dgraph-io/dgraph/dgraph/cmd/alpha.run
alpha_1   | 	/ext-go/1/src/github.com/dgraph-io/dgraph/dgraph/cmd/alpha/run.go:700
alpha_1   | github.com/dgraph-io/dgraph/dgraph/cmd/alpha.init.2.func1
alpha_1   | 	/ext-go/1/src/github.com/dgraph-io/dgraph/dgraph/cmd/alpha/run.go:94
alpha_1   | github.com/spf13/cobra.(*Command).execute
alpha_1   | 	/go/pkg/mod/github.com/spf13/[email protected]/command.go:830
alpha_1   | github.com/spf13/cobra.(*Command).ExecuteC
alpha_1   | 	/go/pkg/mod/github.com/spf13/[email protected]/command.go:914
alpha_1   | github.com/spf13/cobra.(*Command).Execute
alpha_1   | 	/go/pkg/mod/github.com/spf13/[email protected]/command.go:864
alpha_1   | github.com/dgraph-io/dgraph/dgraph/cmd.Execute
alpha_1   | 	/ext-go/1/src/github.com/dgraph-io/dgraph/dgraph/cmd/root.go:71
alpha_1   | main.main
alpha_1   | 	/ext-go/1/src/github.com/dgraph-io/dgraph/dgraph/main.go:102
alpha_1   | runtime.main
alpha_1   | 	/usr/local/go/src/runtime/proc.go:204
alpha_1   | runtime.goexit
alpha_1   | 	/usr/local/go/src/runtime/asm_amd64.s:1374

Best,
John

dmai · December 18, 2020, 10:50pm

If you were running :latest before it could be from a Dgraph v20.07 version. So, a version before the current master or v20.11.

If you have data in Dgraph you want to keep, you can run with that previous version and export your data. Otherwise, if you want to start fresh, you can delete your existing Docker volumes (docker volume ls and then docker volume rm <volumename>) and run with the newer image.

forstmeier · December 18, 2020, 11:20pm

Hi @dmai,

Hmmm, I purged everything - images, containers, and volumes - but I’m continuing to see the same error. I’m using Docker Compose (essentially like this piece here but with the master tag) - could that have anything to do with it?

Best,
John

dmai · December 18, 2020, 11:23pm

Can you share the full cluster logs?

forstmeier · December 18, 2020, 11:25pm

@dmai, is this what you’re looking for? Output from running docker-compose up:

Building custom
Step 1/12 : FROM golang:1.15
1.15: Pulling from library/golang
6c33745f49b4: Pull complete
ef072fc32a84: Pull complete
c0afb8e68e0b: Pull complete
d599c07d28e6: Pull complete
c616e0dda35f: Pull complete
2e68d0e3d3c7: Pull complete
4dea2301be07: Pull complete
Digest: sha256:c47d6c4ee25e34c97f3b1a92923a913e266a2b4b95731e8f5aa9065ca9e7c7fc
Status: Downloaded newer image for golang:1.15
 ---> 5f9d35ce5cfe
Step 2/12 : ENV GO111MODULE=on
 ---> Running in 6c1115dfeea5
Removing intermediate container 6c1115dfeea5
 ---> 152e78e42424
Step 3/12 : ENV CGO_ENABLED=0
 ---> Running in 478c189c7300
Removing intermediate container 478c189c7300
 ---> ad34173cadc6
Step 4/12 : ENV GOOS=linux
 ---> Running in 96620cf1d6d1
Removing intermediate container 96620cf1d6d1
 ---> 875ce55f4f57
Step 5/12 : ENV GOARCH=amd64
 ---> Running in b985932c4b4a
Removing intermediate container b985932c4b4a
 ---> 2a3eb9ccc1f0
Step 6/12 : RUN mkdir /app
 ---> Running in 39cba8a45e71
Removing intermediate container 39cba8a45e71
 ---> 7850b103a3eb
Step 7/12 : ADD . /app
 ---> 45050b0fb479
Step 8/12 : WORKDIR /app
 ---> Running in dfee164a27cf
Removing intermediate container dfee164a27cf
 ---> e830f84814e2
Step 9/12 : RUN go mod download
 ---> Running in 80d73ffe32ad
Removing intermediate container 80d73ffe32ad
 ---> 9dda4ca6deaf
Step 10/12 : RUN cd cmd/custom && go build .
 ---> Running in 7361501a68d2
Removing intermediate container 7361501a68d2
 ---> 0f27afb0d3c6
Step 11/12 : WORKDIR cmd/custom
 ---> Running in 678998317de6
Removing intermediate container 678998317de6
 ---> c1fff5a4adff
Step 12/12 : CMD ["./custom"]
 ---> Running in 0feb68c496e2
Removing intermediate container 0feb68c496e2
 ---> e8480f4799f6

Successfully built e8480f4799f6
Successfully tagged custom:latest
WARNING: Image for service custom was built because it did not already exist. To rebuild this image you must use `docker-compose build` or `docker-compose up --build`.
Pulling zero (dgraph/dgraph:master)...
master: Pulling from dgraph/dgraph
da7391352a9b: Pull complete
14428a6d4bcd: Pull complete
2c2d948710f2: Pull complete
ae5b3b9a1611: Pull complete
fac34419d0b3: Pull complete
8a9e4dc6e7e2: Pull complete
Digest: sha256:6fa89cd4c1344d3353eac4da0eaaab6bcdb1dd46b1664c62877abac142c18636
Status: Downloaded newer image for dgraph/dgraph:master
Creating api_custom_1 ... done
Creating api_alpha_1  ... done
Creating api_zero_1   ... done
Attaching to api_zero_1, api_alpha_1, api_custom_1
alpha_1   | [Decoder]: Using assembly version of decoder
zero_1    | [Decoder]: Using assembly version of decoder
zero_1    | Page Size: 4096
zero_1    | [Sentry] 2020/12/18 23:20:02 Integration installed: ContextifyFrames
zero_1    | [Sentry] 2020/12/18 23:20:02 Integration installed: Environment
zero_1    | [Sentry] 2020/12/18 23:20:02 Integration installed: Modules
zero_1    | [Sentry] 2020/12/18 23:20:02 Integration installed: IgnoreErrors
alpha_1   | Page Size: 4096
alpha_1   | [Sentry] 2020/12/18 23:20:02 Integration installed: ContextifyFrames
alpha_1   | [Sentry] 2020/12/18 23:20:02 Integration installed: Environment
alpha_1   | [Sentry] 2020/12/18 23:20:02 Integration installed: Modules
alpha_1   | [Sentry] 2020/12/18 23:20:02 Integration installed: IgnoreErrors
zero_1    | [Decoder]: Using assembly version of decoder
alpha_1   | [Decoder]: Using assembly version of decoder
zero_1    | Page Size: 4096
zero_1    | [Sentry] 2020/12/18 23:20:02 Integration installed: ContextifyFrames
zero_1    | [Sentry] 2020/12/18 23:20:02 Integration installed: Environment
zero_1    | [Sentry] 2020/12/18 23:20:02 Integration installed: Modules
zero_1    | [Sentry] 2020/12/18 23:20:02 Integration installed: IgnoreErrors
alpha_1   | Page Size: 4096
alpha_1   | [Sentry] 2020/12/18 23:20:02 Integration installed: ContextifyFrames
alpha_1   | [Sentry] 2020/12/18 23:20:02 Integration installed: Environment
alpha_1   | [Sentry] 2020/12/18 23:20:02 Integration installed: Modules
alpha_1   | [Sentry] 2020/12/18 23:20:02 Integration installed: IgnoreErrors
zero_1    | I1218 23:20:02.680270      17 sentry_integration.go:48] This instance of Dgraph will send anonymous reports of panics back to Dgraph Labs via Sentry. No confidential information is sent. These reports help improve Dgraph. To opt-out, restart your instance with the --enable_sentry=false flag. For more info, see https://dgraph.io/docs/howto/#data-handling.
alpha_1   | I1218 23:20:02.682779      17 sentry_integration.go:48] This instance of Dgraph will send anonymous reports of panics back to Dgraph Labs via Sentry. No confidential information is sent. These reports help improve Dgraph. To opt-out, restart your instance with the --enable_sentry=false flag. For more info, see https://dgraph.io/docs/howto/#data-handling.
alpha_1   | I1218 23:20:02.683297      17 util_ee.go:126] KeyReader instantiated of type <nil>
alpha_1   | I1218 23:20:02.784172      17 init.go:107] 
alpha_1   | 
alpha_1   | Dgraph version   : v20.11.0-rc1-209-geece404e
alpha_1   | Dgraph codename  : unnamed
alpha_1   | Dgraph SHA-256   : 273c9556bbde8be6bd9d2e449fb7b04aae3b9ad891be2a62fc52c1599df657c6
alpha_1   | Commit SHA-1     : eece404e
alpha_1   | Commit timestamp : 2020-12-18 18:42:55 -0300
alpha_1   | Branch           : master
alpha_1   | Go version       : go1.15.5
alpha_1   | jemalloc enabled : true
alpha_1   | 
alpha_1   | For Dgraph official documentation, visit https://dgraph.io/docs/.
alpha_1   | For discussions about Dgraph     , visit https://discuss.dgraph.io.
alpha_1   | 
alpha_1   | Licensed variously under the Apache Public License 2.0 and Dgraph Community License.
alpha_1   | Copyright 2015-2020 Dgraph Labs, Inc.
alpha_1   | 
alpha_1   | 
alpha_1   | I1218 23:20:02.784184      17 run.go:696] x.Config: {PortOffset:0 QueryEdgeLimit:1000000 NormalizeNodeLimit:10000 MutationsNQuadLimit:1000000 PollInterval:1s GraphqlExtension:true GraphqlDebug:false GraphqlLambdaUrl:}
alpha_1   | I1218 23:20:02.784200      17 run.go:697] x.WorkerConfig: {TmpDir:t ExportPath:export NumPendingProposals:256 Tracing:0.01 MyAddr:alpha:7080 ZeroAddr:[zero:5080] TLSClientConfig:<nil> TLSServerConfig:<nil> RaftId:0 WhiteListedIPRanges:[{Lower:172.20.0.1 Upper:172.20.0.1}] MaxRetries:-1 StrictMutations:false AclEnabled:false AbortOlderThan:5m0s SnapshotAfter:10000 ProposedGroupId:0 StartTime:2020-12-18 23:20:02.475378011 +0000 UTC m=+0.008047826 LudicrousMode:false LudicrousConcurrency:2000 EncryptionKey:**** LogRequest:0 HardSync:false}
alpha_1   | I1218 23:20:02.784226      17 run.go:698] worker.Config: {PostingDir:p PostingDirCompression:1 PostingDirCompressionLevel:0 WALDir:w MutationsMode:0 AuthToken: PBlockCacheSize:697932185 PIndexCacheSize:375809638 WalCache:0 HmacSecret:**** AccessJwtTtl:0s RefreshJwtTtl:0s CachePercentage:0,65,35,0 CacheMb:0}
zero_1    | I1218 23:20:02.784176      17 init.go:107] 
zero_1    | 
zero_1    | Dgraph version   : v20.11.0-rc1-209-geece404e
zero_1    | Dgraph codename  : unnamed
zero_1    | Dgraph SHA-256   : 273c9556bbde8be6bd9d2e449fb7b04aae3b9ad891be2a62fc52c1599df657c6
zero_1    | Commit SHA-1     : eece404e
zero_1    | Commit timestamp : 2020-12-18 18:42:55 -0300
zero_1    | Branch           : master
zero_1    | Go version       : go1.15.5
zero_1    | jemalloc enabled : true
zero_1    | 
zero_1    | For Dgraph official documentation, visit https://dgraph.io/docs/.
zero_1    | For discussions about Dgraph     , visit https://discuss.dgraph.io.
zero_1    | 
zero_1    | Licensed variously under the Apache Public License 2.0 and Dgraph Community License.
zero_1    | Copyright 2015-2020 Dgraph Labs, Inc.
zero_1    | 
zero_1    | 
zero_1    | I1218 23:20:02.784218      17 run.go:185] Setting Config to: {bindall:true portOffset:0 nodeId:1 numReplicas:1 peer: w:zw rebalanceInterval:480000000000 tlsClientConfig:<nil>}
zero_1    | I1218 23:20:02.784252      17 run.go:98] Setting up grpc listener at: 0.0.0.0:5080
alpha_1   | I1218 23:20:02.784462      17 log.go:295] Found file: 18 First Index: 0
alpha_1   | I1218 23:20:02.785601      17 storage.go:132] Init Raft Storage with snap: 0, first: 1, last: 0
alpha_1   | I1218 23:20:02.785632      17 server_state.go:76] Setting Posting Dir Compression Level: 0
zero_1    | I1218 23:20:02.784601      17 run.go:98] Setting up http listener at: 0.0.0.0:6080
alpha_1   | I1218 23:20:02.785640      17 server_state.go:120] Opening postings BadgerDB with options: {Dir:p ValueDir:p SyncWrites:false NumVersionsToKeep:2147483647 ReadOnly:false Logger:0x2e04cb8 Compression:1 InMemory:false MemTableSize:67108864 BaseTableSize:2097152 BaseLevelSize:10485760 LevelSizeMultiplier:10 TableSizeMultiplier:2 MaxLevels:7 ValueThreshold:1024 NumMemtables:5 BlockSize:4096 BloomFalsePositive:0.01 BlockCacheSize:697932185 IndexCacheSize:375809638 NumLevelZeroTables:5 NumLevelZeroTablesStall:15 ValueLogFileSize:1073741823 ValueLogMaxEntries:1000000 NumCompactors:4 CompactL0OnClose:false ZSTDCompressionLevel:0 VerifyValueChecksum:false EncryptionKey:[] EncryptionKeyRotationDuration:240h0m0s BypassLockGuard:false ChecksumVerificationMode:0 DetectConflicts:false managedTxns:false maxBatchCount:0 maxBatchSize:0}
zero_1    | I1218 23:20:02.784738      17 log.go:295] Found file: 1 First Index: 1
zero_1    | I1218 23:20:02.784788      17 storage.go:132] Init Raft Storage with snap: 0, first: 1, last: 6
zero_1    | I1218 23:20:02.784940      17 node.go:152] Setting raft.Config to: &{ID:1 peers:[] learners:[] ElectionTick:20 HeartbeatTick:1 Storage:0xc00012c4b0 Applied:0 MaxSizePerMsg:262144 MaxCommittedSizePerReady:67108864 MaxUncommittedEntriesSize:0 MaxInflightMsgs:256 CheckQuorum:false PreVote:true ReadOnlyOption:0 Logger:0x2e04cb8 DisableProposalForwarding:false}
zero_1    | I1218 23:20:02.785082      17 node.go:321] Found hardstate: {Term:4 Vote:1 Commit:6 XXX_unrecognized:[]}
zero_1    | I1218 23:20:02.785113      17 node.go:326] Group 0 found 6 entries
zero_1    | I1218 23:20:02.785121      17 raft.go:541] Restarting node for dgraphzero
zero_1    | I1218 23:20:02.785137      17 log.go:34] 1 became follower at term 4
zero_1    | I1218 23:20:02.785145      17 log.go:34] newRaft 1 [peers: [], term: 4, commit: 6, applied: 0, lastindex: 6, lastterm: 4]
zero_1    | I1218 23:20:02.785245      17 run.go:303] Running Dgraph Zero...
zero_1    | I1218 23:20:02.785646      17 node.go:189] Setting conf state to nodes:1 
zero_1    | I1218 23:20:02.785690      17 raft.go:877] Done applying conf change at 0x1
alpha_1   | [Sentry] 2020/12/18 23:20:02 Sending fatal event [a2072939a05c4f80bfb7ff5c3cab0e94] to o318308.ingest.sentry.io project: 5208688
alpha_1   | 2020/12/18 23:20:02 manifest has unsupported version: 7 (we support 8).
alpha_1   | Please see https://github.com/dgraph-io/badger/blob/master/README.md#i-see-manifest-has-unsupported-version-x-we-support-y-error on how to fix this.
alpha_1   | Error while creating badger KV posting store
alpha_1   | github.com/dgraph-io/dgraph/x.Checkf
alpha_1   | 	/ext-go/1/src/github.com/dgraph-io/dgraph/x/error.go:51
alpha_1   | github.com/dgraph-io/dgraph/worker.(*ServerState).initStorage
alpha_1   | 	/ext-go/1/src/github.com/dgraph-io/dgraph/worker/server_state.go:124
alpha_1   | github.com/dgraph-io/dgraph/worker.InitServerState
alpha_1   | 	/ext-go/1/src/github.com/dgraph-io/dgraph/worker/server_state.go:54
alpha_1   | github.com/dgraph-io/dgraph/dgraph/cmd/alpha.run
alpha_1   | 	/ext-go/1/src/github.com/dgraph-io/dgraph/dgraph/cmd/alpha/run.go:700
alpha_1   | github.com/dgraph-io/dgraph/dgraph/cmd/alpha.init.2.func1
alpha_1   | 	/ext-go/1/src/github.com/dgraph-io/dgraph/dgraph/cmd/alpha/run.go:94
alpha_1   | github.com/spf13/cobra.(*Command).execute
alpha_1   | 	/go/pkg/mod/github.com/spf13/[email protected]/command.go:830
alpha_1   | github.com/spf13/cobra.(*Command).ExecuteC
alpha_1   | 	/go/pkg/mod/github.com/spf13/[email protected]/command.go:914
alpha_1   | github.com/spf13/cobra.(*Command).Execute
alpha_1   | 	/go/pkg/mod/github.com/spf13/[email protected]/command.go:864
alpha_1   | github.com/dgraph-io/dgraph/dgraph/cmd.Execute
alpha_1   | 	/ext-go/1/src/github.com/dgraph-io/dgraph/dgraph/cmd/root.go:71
alpha_1   | main.main
alpha_1   | 	/ext-go/1/src/github.com/dgraph-io/dgraph/dgraph/main.go:102
alpha_1   | runtime.main
alpha_1   | 	/usr/local/go/src/runtime/proc.go:204
alpha_1   | runtime.goexit
alpha_1   | 	/usr/local/go/src/runtime/asm_amd64.s:1374
alpha_1   | Page Size: 4096
alpha_1   | [Sentry] 2020/12/18 23:20:03 Integration installed: ContextifyFrames
alpha_1   | [Sentry] 2020/12/18 23:20:03 Integration installed: Environment
alpha_1   | [Sentry] 2020/12/18 23:20:03 Integration installed: Modules
alpha_1   | [Sentry] 2020/12/18 23:20:03 Integration installed: IgnoreErrors
alpha_1   | [Decoder]: Using assembly version of decoder
alpha_1   | Page Size: 4096
alpha_1   | [Sentry] 2020/12/18 23:20:03 Integration installed: ContextifyFrames
alpha_1   | [Sentry] 2020/12/18 23:20:03 Integration installed: Environment
alpha_1   | [Sentry] 2020/12/18 23:20:03 Integration installed: Modules
alpha_1   | [Sentry] 2020/12/18 23:20:03 Integration installed: IgnoreErrors
alpha_1   | I1218 23:20:03.992677      18 sentry_integration.go:48] This instance of Dgraph will send anonymous reports of panics back to Dgraph Labs via Sentry. No confidential information is sent. These reports help improve Dgraph. To opt-out, restart your instance with the --enable_sentry=false flag. For more info, see https://dgraph.io/docs/howto/#data-handling.
alpha_1   | I1218 23:20:03.993373      18 util_ee.go:126] KeyReader instantiated of type <nil>
alpha_1   | I1218 23:20:04.095064      18 init.go:107] 
alpha_1   | 
alpha_1   | Dgraph version   : v20.11.0-rc1-209-geece404e
alpha_1   | Dgraph codename  : unnamed
alpha_1   | Dgraph SHA-256   : 273c9556bbde8be6bd9d2e449fb7b04aae3b9ad891be2a62fc52c1599df657c6
alpha_1   | Commit SHA-1     : eece404e
alpha_1   | Commit timestamp : 2020-12-18 18:42:55 -0300
alpha_1   | Branch           : master
alpha_1   | Go version       : go1.15.5
alpha_1   | jemalloc enabled : true
alpha_1   | 
alpha_1   | For Dgraph official documentation, visit https://dgraph.io/docs/.
alpha_1   | For discussions about Dgraph     , visit https://discuss.dgraph.io.
alpha_1   | 
alpha_1   | Licensed variously under the Apache Public License 2.0 and Dgraph Community License.
alpha_1   | Copyright 2015-2020 Dgraph Labs, Inc.
alpha_1   | 
alpha_1   | 
alpha_1   | I1218 23:20:04.095080      18 run.go:696] x.Config: {PortOffset:0 QueryEdgeLimit:1000000 NormalizeNodeLimit:10000 MutationsNQuadLimit:1000000 PollInterval:1s GraphqlExtension:true GraphqlDebug:false GraphqlLambdaUrl:}
alpha_1   | I1218 23:20:04.095099      18 run.go:697] x.WorkerConfig: {TmpDir:t ExportPath:export NumPendingProposals:256 Tracing:0.01 MyAddr:alpha:7080 ZeroAddr:[zero:5080] TLSClientConfig:<nil> TLSServerConfig:<nil> RaftId:0 WhiteListedIPRanges:[{Lower:172.20.0.1 Upper:172.20.0.1}] MaxRetries:-1 StrictMutations:false AclEnabled:false AbortOlderThan:5m0s SnapshotAfter:10000 ProposedGroupId:0 StartTime:2020-12-18 23:20:03.787664131 +0000 UTC m=+0.008412087 LudicrousMode:false LudicrousConcurrency:2000 EncryptionKey:**** LogRequest:0 HardSync:false}
alpha_1   | I1218 23:20:04.095126      18 run.go:698] worker.Config: {PostingDir:p PostingDirCompression:1 PostingDirCompressionLevel:0 WALDir:w MutationsMode:0 AuthToken: PBlockCacheSize:697932185 PIndexCacheSize:375809638 WalCache:0 HmacSecret:**** AccessJwtTtl:0s RefreshJwtTtl:0s CachePercentage:0,65,35,0 CacheMb:0}
alpha_1   | I1218 23:20:04.095314      18 log.go:295] Found file: 19 First Index: 0
alpha_1   | I1218 23:20:04.096660      18 storage.go:132] Init Raft Storage with snap: 0, first: 1, last: 0
alpha_1   | I1218 23:20:04.096695      18 server_state.go:76] Setting Posting Dir Compression Level: 0
alpha_1   | I1218 23:20:04.096702      18 server_state.go:120] Opening postings BadgerDB with options: {Dir:p ValueDir:p SyncWrites:false NumVersionsToKeep:2147483647 ReadOnly:false Logger:0x2e04cb8 Compression:1 InMemory:false MemTableSize:67108864 BaseTableSize:2097152 BaseLevelSize:10485760 LevelSizeMultiplier:10 TableSizeMultiplier:2 MaxLevels:7 ValueThreshold:1024 NumMemtables:5 BlockSize:4096 BloomFalsePositive:0.01 BlockCacheSize:697932185 IndexCacheSize:375809638 NumLevelZeroTables:5 NumLevelZeroTablesStall:15 ValueLogFileSize:1073741823 ValueLogMaxEntries:1000000 NumCompactors:4 CompactL0OnClose:false ZSTDCompressionLevel:0 VerifyValueChecksum:false EncryptionKey:[] EncryptionKeyRotationDuration:240h0m0s BypassLockGuard:false ChecksumVerificationMode:0 DetectConflicts:false managedTxns:false maxBatchCount:0 maxBatchSize:0}
alpha_1   | [Sentry] 2020/12/18 23:20:04 Sending fatal event [ba992816742246109c551b5c4e996a34] to o318308.ingest.sentry.io project: 5208688
alpha_1   | 2020/12/18 23:20:04 manifest has unsupported version: 7 (we support 8).
alpha_1   | Please see https://github.com/dgraph-io/badger/blob/master/README.md#i-see-manifest-has-unsupported-version-x-we-support-y-error on how to fix this.
alpha_1   | Error while creating badger KV posting store
alpha_1   | github.com/dgraph-io/dgraph/x.Checkf
alpha_1   | 	/ext-go/1/src/github.com/dgraph-io/dgraph/x/error.go:51
alpha_1   | github.com/dgraph-io/dgraph/worker.(*ServerState).initStorage
alpha_1   | 	/ext-go/1/src/github.com/dgraph-io/dgraph/worker/server_state.go:124
alpha_1   | github.com/dgraph-io/dgraph/worker.InitServerState
alpha_1   | 	/ext-go/1/src/github.com/dgraph-io/dgraph/worker/server_state.go:54
alpha_1   | github.com/dgraph-io/dgraph/dgraph/cmd/alpha.run
alpha_1   | 	/ext-go/1/src/github.com/dgraph-io/dgraph/dgraph/cmd/alpha/run.go:700
alpha_1   | github.com/dgraph-io/dgraph/dgraph/cmd/alpha.init.2.func1
alpha_1   | 	/ext-go/1/src/github.com/dgraph-io/dgraph/dgraph/cmd/alpha/run.go:94
alpha_1   | github.com/spf13/cobra.(*Command).execute
alpha_1   | 	/go/pkg/mod/github.com/spf13/[email protected]/command.go:830
alpha_1   | github.com/spf13/cobra.(*Command).ExecuteC
alpha_1   | 	/go/pkg/mod/github.com/spf13/[email protected]/command.go:914
alpha_1   | github.com/spf13/cobra.(*Command).Execute
alpha_1   | 	/go/pkg/mod/github.com/spf13/[email protected]/command.go:864
alpha_1   | github.com/dgraph-io/dgraph/dgraph/cmd.Execute
alpha_1   | 	/ext-go/1/src/github.com/dgraph-io/dgraph/dgraph/cmd/root.go:71
alpha_1   | main.main
alpha_1   | 	/ext-go/1/src/github.com/dgraph-io/dgraph/dgraph/main.go:102
alpha_1   | runtime.main
alpha_1   | 	/usr/local/go/src/runtime/proc.go:204
alpha_1   | runtime.goexit
alpha_1   | 	/usr/local/go/src/runtime/asm_amd64.s:1374
api_alpha_1 exited with code 1
alpha_1   | Page Size: 4096
alpha_1   | [Sentry] 2020/12/18 23:20:04 Integration installed: ContextifyFrames
alpha_1   | [Sentry] 2020/12/18 23:20:04 Integration installed: Environment
alpha_1   | [Sentry] 2020/12/18 23:20:04 Integration installed: Modules
alpha_1   | [Sentry] 2020/12/18 23:20:04 Integration installed: IgnoreErrors
alpha_1   | [Decoder]: Using assembly version of decoder
alpha_1   | Page Size: 4096
alpha_1   | [Sentry] 2020/12/18 23:20:05 Integration installed: ContextifyFrames
alpha_1   | [Sentry] 2020/12/18 23:20:05 Integration installed: Environment
alpha_1   | [Sentry] 2020/12/18 23:20:05 Integration installed: Modules
alpha_1   | [Sentry] 2020/12/18 23:20:05 Integration installed: IgnoreErrors
alpha_1   | I1218 23:20:05.222075      18 sentry_integration.go:48] This instance of Dgraph will send anonymous reports of panics back to Dgraph Labs via Sentry. No confidential information is sent. These reports help improve Dgraph. To opt-out, restart your instance with the --enable_sentry=false flag. For more info, see https://dgraph.io/docs/howto/#data-handling.
alpha_1   | I1218 23:20:05.222793      18 util_ee.go:126] KeyReader instantiated of type <nil>
^CGracefully stopping... (press Ctrl+C again to force)
Stopping api_zero_1   ... done
Stopping api_custom_1 ... done
Stopping api_alpha_1  ... done

dmai · December 18, 2020, 11:35pm

Yup, this helps. Looks like there’s still a volume with existing data still being used. For example, the logs say “Restarting node for dgraphzero”, so it’s a restart of a node (not a fresh one). You’ll want to clean that up to start fresh.

You can share your docker-compose.yml file which should list the volumes. If they’re bind mounts, then you can delete the directories on your host machine. If they’re Docker volumes, then you can remove the volume using docker volume rm.

forstmeier · December 18, 2020, 11:40pm

Brilliant! You’re right it was the line:

    volumes:
      - /tmp/data:/dgraph

in the docker-compose.yaml (linked in my first message) that was causing the issue - I deleted that data and the docker-compose up worked like a charm! Is there documentation on exporting data I could reference for the future? Right now it’s not a big deal to just delete the data since I’m still in the development stage.

dmai · December 18, 2020, 11:44pm

Glad you got it working. Thanks for reminding me where your docker-compose.yml file is I need to read more carefully.

Here’s the docs on exporting data: https://dgraph.io/docs/deploy/dgraph-administration/#exporting-database

You can use the GraphQL admin mutation to export in RDF or JSON format (localhost:8080/admin):

mutation {
  export(input: {format: "rdf"}) {
    response {
      message
      code
    }
  }
}

Or you can use the REST endpoint (localhost:8080/admin/export):

curl "localhost:8080/admin/export"
curl "localhost:8080/admin/export?format=json"

forstmeier · December 18, 2020, 11:48pm

Awesome, thank you so much! I’ll mark this issue as solved.

forstmeier · December 21, 2020, 7:28pm

On a related note, I’m having issues connecting to the @custom handler which is a Go server living in the same repo. I’m almost certain it’s just something I’m doing incorrectly with my docker-compose definition but here’s the error I’m getting back.

"Evaluation of custom field failed because external request returned an error: Post \"http://localhost:4080/users\": dial tcp 127.0.0.1:4080: connect: connection refused for field: customAddUser within type: Mutation."

dmai · December 21, 2020, 8:49pm

@forstmeier Looking at your Docker Compose config, I think your @custom handler should connect via custom:4080, not 127.0.0.1:4080. The former is what’s accessible from Dgraph Alpha.

forstmeier · December 23, 2020, 2:17pm

@dmai that was it exactly! Thanks again!