Error in recording data times, unable to find log file


(Lvzhihong) #1

Hi,everyone
This error occurred when I typed data into the dgraph using a Java program
Error in recording data times, unable to find log file

  • Anybody know what the problem is? thank you

unable to find log file
(nyadav) #2

facing exact same problem on Windows Dgraph_v1.0.8_rc1. After some heavy writes i stopped dgraph server neatly Ctrl-C on Windows 10 but on restart most of Read queries throwing same error as above through Java & also ratel. Please look into it on urgent basis and suggest possible ways to repair this data without loss.


(liuluhao) #3


(Alex Miłowski) #4

I have been having the same problem with v1.0.8 …


(Daniel Mai) #5

Is this only happening with the Java client?


(Alex Miłowski) #6

No. My client is python …direct… I can get the same behavior via curl.

I current belief is that the volumes attached to the dgraph-server instances are somehow in an inconsistent state. I think this behavior has happened because the server instances ran out of memory during inserts.

I change the backing nodes of the k8s cluster to have more memory and increased the requested memory for each pod. While things were more stable, the server still degraded over time.

I am currently recreating the cluster and database contents from scratch to see if this can be reproduced.

It would be great to understand how to debug the source of “Unable to find log file” errors. I saw nothing in the server or zero logs that were traceable to that response to the client. In fact, the log files were silent.


(Alex Miłowski) #7

…and now, with a clean start, after quite a few inserts, the response:

{“code”: “ErrorInvalidRequest”, “message”: “rpc error: code = Unknown desc = Unable to find log file. Please retry”}

What conditions cause this and how do I debug the issue?

There have been no pod restarts (crashes).


(Alex Miłowski) #8

In the logs, I see this strange error:

WARNING: This entry should have been caught. {Key: […giant array of integers…

…and another shorter one:

WARNING: This entry should have been caught. {Key:[0 0 14 115 101 110 100 101 114 95 98 97 108 97 110 99 101 2 6 1 0 0 0 0 0 0 118 127 255 255 255 255 255 255 249 155] Value:[0 0 0 5 128 235 198 18 209 163 24 153 134 38 184 188 38 172 193 38 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] UserMeta:9 ExpiresAt:0 meta:68 offset:67024041}


#9

when I insert into or query dgraph,it would occur this exception:Unable to find log file. Please retry. I guess it‘s becuase to when insert data the posting list lost,but I don’t have any ways to solve it.(I use java client insert data)

insert exception:
2018-09-12 19:20:42,750 ERROR DgraphDataHandlerList:183 - UNKNOWN: Unable to find log file. Please retry
io.grpc.StatusRuntimeException: UNKNOWN: Unable to find log file. Please retry
at io.grpc.Status.asRuntimeException(Status.java:526)
at io.grpc.stub.ClientCalls$StreamObserverToCallListenerAdapter.onClose(ClientCalls.java:419)
at io.grpc.ForwardingClientCallListener.onClose(ForwardingClientCallListener.java:41)
at io.grpc.internal.CensusStatsModule$StatsClientInterceptor$1$1.onClose(CensusStatsModule.java:684)
at io.grpc.ForwardingClientCallListener.onClose(ForwardingClientCallListener.java:41)
at io.grpc.internal.CensusTracingModule$TracingClientInterceptor$1$1.onClose(CensusTracingModule.java:391)
at io.grpc.internal.ClientCallImpl.closeObserver(ClientCallImpl.java:475)
at io.grpc.internal.ClientCallImpl.access$300(ClientCallImpl.java:63)
at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl.close(ClientCallImpl.java:557)
at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl.access$600(ClientCallImpl.java:478)
at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1StreamClosed.runInContext(ClientCallImpl.java:590)
at io.grpc.internal.ContextRunnable.run(ContextRunnable.java:37)
at io.grpc.internal.SerializingExecutor.run(SerializingExecutor.java:123)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)

query exception:
%E5%BE%AE%E4%BF%A1%E5%9B%BE%E7%89%87%E7%BC%96%E8%BE%91_20180917104423


#10

retry no effect.Is it a bug in dgraph internal?


(Daniel Mai) #11

Yes, this is a bug we are investigating.


#13

When will it be settled? Or, what solutions do we have to enable us to input data smoothly?


(Daniel Mai) #14

We’re going to push a new release candidate that addresses the “unable to find log file” problem.


(Daniel Mai) #15

Dgraph v1.0.9-rc3 is released, which addresses the log file issue. It’s available via Docker Hub.


(Lvzhihong) #16

When will Windows and Linux versions be updated?thank you


(Daniel Mai) #17

Standalone binaries for v1.0.9-rc3 can be found here:

Linux: https://storage.googleapis.com/dgraph-bin/dgraph-linux-amd64.tar.gz
Windows: https://storage.googleapis.com/dgraph-bin/dgraph-windows-amd64.tar.gz


(Lvzhihong) #18

thank you!It helped me a lot