# Dgraph start to panic out of nowhere on Darwin **URL:** \ **Category:** Dgraph\ **Tags:** graphql, dgraph, kind:bug\ **Created:** [December 30, 2020, 2:04am UTC](https://discuss.dgraph.io/t/dgraph-start-to-panic-out-of-nowhere-on-darwin/12115 "2020-12-30T02:04:58Z")\ **Posts on this page:** 8\ **Page:** 1
**Author:** ![MichelDiz](https://yyz1.discourse-cdn.com/flex007/user_avatar/discuss.dgraph.io/micheldiz/32/11873_2.png) [@MichelDiz](https://discuss.dgraph.io/u/MichelDiz)\ **Post date:** [December 30, 2020, 2:04am UTC](https://discuss.dgraph.io/t/dgraph-start-to-panic-out-of-nowhere-on-darwin/12115/1 "2020-12-30T02:04:58Z")
## Report a Dgraph Bug ### What version of Dgraph are you using? ```auto Dgraph version : v20.11.0 Dgraph codename : tchalla Dgraph SHA-256 : 8591d39e46caf64c1f8b9d8c8dcdb1afe7a61bbe5d9e08cd597faa4978717544 Commit SHA-1 : c4245ad55 Commit timestamp : 2020-12-16 15:55:40 +0530 Branch : HEAD Go version : go1.15.5 jemalloc enabled : false ``` ### Have you tried reproducing the issue with the latest release? Just tchalla ### What is the hardware spec (RAM, OS)? Darwin - 11.1 (20C69) - Bigsur 32 GB DDR4 ### Steps to reproduce the issue (command/config used to run Dgraph). Just start Dgraph. First, it was working fine for days - I was using GraphQL. Now even removing the directories(`rm -fr p w zw`) it panics. Kind of solved by stopping all related dev envs. Stopping the actual Dgraph instances, removing the folders, stopping the React dev, stopping the JS server, stopping the GraphiQL software. And it looks like it was something related to the Subscription. Not sure. ```auto { "errors": [ { "message": "Internal Server Error - a panic was trapped. This indicates a bug in the GraphQL server. A stack trace was logged. Please let us know by filing an issue with the stack trace." } ] } ``` ```auto I1229 22:58:18.615857 90384 log.go:34] Writes flushed. Stopping compactions now... I1229 22:58:18.625141 90384 log.go:34] DropPrefix done I1229 22:58:18.625160 90384 log.go:34] Resuming writes I1229 22:58:18.625226 90384 index.go:942] Rebuilding index for attr dgraph.graphql.p_sha256hash and tokenizers [exact] I1229 22:58:18.626944 90384 groups.go:159] Server is ready I1229 22:58:18.626962 90384 access_ee.go:389] ResetAcl closed I1229 22:58:18.626976 90384 access_ee.go:311] RefreshAcls closed I1229 22:58:18.695223 90384 log.go:34] Rebuilding index for predicate dgraph.graphql.p_sha256hash (1/2): Sent data of size 0 B I1229 22:58:18.712526 90384 mutation.go:188] Done schema update predicate:"dgraph.graphql.p_sha256hash" value_type:STRING directive:INDEX tokenizer:"exact" I1229 22:58:18.712570 90384 draft.go:124] Operation completed with id: opIndexing I1229 22:58:18.829679 90384 graphql.go:39] ResetCors closed I1229 22:58:18.830431 90384 run.go:819] Updating cors from subscription. I1229 22:58:18.830447 90384 run.go:843] Updating cors origins: [*] panic: runtime error: invalid memory address or nil pointer dereference [signal SIGSEGV: segmentation violation code=0x1 addr=0x20 pc=0x4a05c8d] goroutine 239 [running]: github.com/dgraph-io/dgraph/graphql/schema.(*field).GetObjectName(...) /ext-go/1/src/github.com/dgraph-io/dgraph/graphql/schema/wrappers.go:1117 github.com/dgraph-io/dgraph/graphql/schema.(*field).HasCustomDirective(0xc0000d1320, 0x8, 0x10) /ext-go/1/src/github.com/dgraph-io/dgraph/graphql/schema/wrappers.go:965 +0x4d github.com/dgraph-io/dgraph/graphql/resolve.validateCustomFieldsRecursively(0x5522080, 0xc0000d1320, 0x9, 0x10) /ext-go/1/src/github.com/dgraph-io/dgraph/graphql/resolve/resolver.go:555 +0x4c github.com/dgraph-io/dgraph/graphql/resolve.(*RequestResolver).ValidateSubscription(0xc00038cf00, 0xc000724000, 0x2, 0x0) /ext-go/1/src/github.com/dgraph-io/dgraph/graphql/resolve/resolver.go:544 +0x129 github.com/dgraph-io/dgraph/graphql/subscription.(*Poller).AddSubscriber(0xc0004d9050, 0xc000724000, 0xc006403b88, 0x0, 0x0, 0x0) /ext-go/1/src/github.com/dgraph-io/dgraph/graphql/subscription/poller.go:71 +0x87 github.com/dgraph-io/dgraph/graphql/web.(*graphqlSubscription).Subscribe(0xc000010350, 0x5507b00, 0xc0000d0ab0, 0xc00130c000, 0xbc, 0xc00061a610, 0x7, 0xc0000d0a50, 0x8, 0x203000, ...) /ext-go/1/src/github.com/dgraph-io/dgraph/graphql/web/http.go:173 +0x1a7 github.com/dgraph-io/graphql-transport-ws/graphqlws/internal/connection.(*connection).readLoop(0xc0000d0810, 0x5507a40, 0xc000110c80, 0xc00000f900) /go/pkg/mod/github.com/dgraph-io/graphql-transport-ws@v0.0.0-20200916064635-48589439591b/graphqlws/internal/connection/connection.go:191 +0x71d github.com/dgraph-io/graphql-transport-ws/graphqlws/internal/connection.Connect(0x550a0c0, 0xc0000ea2c0, 0x54d6fe0, 0xc000010350, 0x0, 0x0, 0x0, 0xced0263f) /go/pkg/mod/github.com/dgraph-io/graphql-transport-ws@v0.0.0-20200916064635-48589439591b/graphqlws/internal/connection/connection.go:97 +0x209 created by github.com/dgraph-io/graphql-transport-ws/graphqlws.NewHandlerFunc.func1 /go/pkg/mod/github.com/dgraph-io/graphql-transport-ws@v0.0.0-20200916064635-48589439591b/graphqlws/http.go:33 +0x1bc W1229 22:58:21.109191 90378 sentry_integration.go:140] unable to read CID from file /var/folders/sw/r5b83mfx5xsgz5rxczw519y40000gn/T//dgraph-alpha-cid-sentry open /var/folders/sw/r5b83mfx5xsgz5rxczw519y40000gn/T//dgraph-alpha-cid-sentry: no such file or directory. Skip [Sentry] 2020/12/29 22:58:21 Sending fatal event [eea86ca9a8764dbab8de18730b0b64a1] to o318308.ingest.sentry.io project: 1805390 [Sentry] 2020/12/29 22:58:22 Buffer flushed successfully. ➜ ~ ``` ### Error in the React application ![Screen Shot 2020-12-29 at 23.21.50](https://canada1.discourse-cdn.com/flex007/uploads/dgraph/original/2X/3/3ce74b84e339a135f1e4a032c7015b529e75ca1e.png) * * * ## ALSO Dgraph is creating a new folder, which is weird. ![Screen Shot 2020-12-29 at 23.08.38](https://canada1.discourse-cdn.com/flex007/uploads/dgraph/original/2X/6/6c2b71e89c18dd8aabfe264cd86e3a00d01d6480.png) ---
**Author:** ![medrivia](https://yyz1.discourse-cdn.com/flex007/user_avatar/discuss.dgraph.io/medrivia/32/4353_2.png) [@medrivia](https://discuss.dgraph.io/u/medrivia)\ **Post date:** [December 30, 2020, 3:20am UTC](https://discuss.dgraph.io/t/dgraph-start-to-panic-out-of-nowhere-on-darwin/12115/2 "2020-12-30T03:20:35Z")
> [@MichelDiz](#): > > ## ALSO > > Dgraph is creating a new folder, which is weird. > > ![Screen Shot 2020-12-29 at 23.08.38](https://canada1.discourse-cdn.com/flex007/uploads/dgraph/original/2X/6/6c2b71e89c18dd8aabfe264cd86e3a00d01d6480.png) Lol, here I thought that mysterious `t` folder was intentional. Should have mentioned it in [my last report](http://discuss.hypermode.com/t/dgraph-v20-11-0-for-windows-failed-to-run-the-second-time-aborted-itself/11985). ---
**Author:** ![rajas](https://yyz1.discourse-cdn.com/flex007/user_avatar/discuss.dgraph.io/rajas/32/4393_2.png) [@rajas](https://discuss.dgraph.io/u/rajas)\ **Post date:** [December 30, 2020, 3:36am UTC](https://discuss.dgraph.io/t/dgraph-start-to-panic-out-of-nowhere-on-darwin/12115/3 "2020-12-30T03:36:24Z")
The t folder is intentional. It was introduced recently to store temporary buffers. Reference: [dgraph/run.go at a57470e06e88530897e651c064ea70e42dbee6c1 · dgraph-io/dgraph · GitHub](https://github.com/dgraph-io/dgraph/blob/a57470e06e88530897e651c064ea70e42dbee6c1/dgraph/cmd/alpha/run.go#L106) ---
**Author:** ![MichelDiz](https://yyz1.discourse-cdn.com/flex007/user_avatar/discuss.dgraph.io/micheldiz/32/11873_2.png) [@MichelDiz](https://discuss.dgraph.io/u/MichelDiz)\ **Post date:** [December 30, 2020, 3:40am UTC](https://discuss.dgraph.io/t/dgraph-start-to-panic-out-of-nowhere-on-darwin/12115/4 "2020-12-30T03:40:00Z")
> [@rajas](#): > > It was introduced recently to store temporary buffers. Nice, but folders with a single character are really confusing. Why we don’t use the whole name? postings\*, wall, zwall, tmp. It is more human friendly. ---
**Author:** ![rajas](https://yyz1.discourse-cdn.com/flex007/user_avatar/discuss.dgraph.io/rajas/32/4393_2.png) [@rajas](https://discuss.dgraph.io/u/rajas)\ **Post date:** [December 30, 2020, 3:51am UTC](https://discuss.dgraph.io/t/dgraph-start-to-panic-out-of-nowhere-on-darwin/12115/5 "2020-12-30T03:51:10Z")
> [@MichelDiz](#): > > Nice, but folders with a single character are really confusing. Why we don’t use the whole name? > postings\*, wall, zwall, tmp Yes, Even I feel that will be nicer. Tagging @ahsan as he introduced this folder. > [@MichelDiz](#): > > Kind of solved by stopping all related dev envs. Stopping the actual Dgraph instances, removing the folders, stopping the React dev, stopping the JS server, stopping the GraphiQL software. And it looks like it was something related to the Subscription. Not sure. The panic seems to have happened due to some ongoing subscription query. The subscription query did not exist in the newly started dgraph instance and so, it panicked. Can you try shutting down any ongoing subscription queries and then restarting dgraph ? The subscription queries could again be restarted once schema has been provided to new dgraph intsance. ---
**Author:** ![MichelDiz](https://yyz1.discourse-cdn.com/flex007/user_avatar/discuss.dgraph.io/micheldiz/32/11873_2.png) [@MichelDiz](https://discuss.dgraph.io/u/MichelDiz)\ **Post date:** [December 30, 2020, 3:55am UTC](https://discuss.dgraph.io/t/dgraph-start-to-panic-out-of-nowhere-on-darwin/12115/6 "2020-12-30T03:55:54Z")
> [@rajas](#): > > The panic seems to have happened due to some ongoing subscription query. Yeah, what is funny is that the panic starts using a normal query. I have some subscriptions but only that one triggers the panic easily. Its weird. And sometimes is from nothing. > [@rajas](#): > > Can you try shutting down any ongoing subscription queries and then restarting dgraph ? I’ve done this. > [@rajas](#): > > The subscription queries could again be restarted once schema has been provided to new dgraph intsance. It first happened in a fully filled DB. ---
**Author:** ![rajas](https://yyz1.discourse-cdn.com/flex007/user_avatar/discuss.dgraph.io/rajas/32/4393_2.png) [@rajas](https://discuss.dgraph.io/u/rajas)\ **Post date:** [December 31, 2020, 1:32pm UTC](https://discuss.dgraph.io/t/dgraph-start-to-panic-out-of-nowhere-on-darwin/12115/7 "2020-12-31T13:32:32Z")
What are the steps to reproduce this ? ---
**Author:** ![MichelDiz](https://yyz1.discourse-cdn.com/flex007/user_avatar/discuss.dgraph.io/micheldiz/32/11873_2.png) [@MichelDiz](https://discuss.dgraph.io/u/MichelDiz)\ **Post date:** [December 31, 2020, 2:50pm UTC](https://discuss.dgraph.io/t/dgraph-start-to-panic-out-of-nowhere-on-darwin/12115/8 "2020-12-31T14:50:58Z")
I can reproduce via [GitHub - MichelDiz/Grabber: The intention of this application is to "grab" comments from live chat in live streams. And save them all in Dgraph to later use.](https://github.com/MichelDiz/Grabber) It happens in two forms. With a full filled DB and a DB started from scratch. With the empty DB I can reproduce via [Grabber/index.jsx at 49a57e585eb8d588a51d81269e5f94014193e236 · MichelDiz/Grabber · GitHub](https://github.com/MichelDiz/Grabber/blob/49a57e585eb8d588a51d81269e5f94014193e236/googlelogin/src/pages/Settings/components/Settings/index.jsx#L23) - Just start the Deno server and the React one. Go to the Settings page and it happens. With a full filled DB is a bit different to reproduce. It happened when I tried to add this mutation [Grabber/addLiveBroadcast.graphql at main · MichelDiz/Grabber · GitHub](https://github.com/MichelDiz/Grabber/blob/main/src/graphql/addLiveBroadcast.graphql) But only when the input is malformed. So instead of a malformed error, it throws a panic. And when it happened in the first time I couldn’t recover the cluster.