ResourceExhausted error in grpc

hyunsooda · March 4, 2020, 11:21am

Hi, I’m using golang driver and getting error when load bulk data.

2020/03/04 20:17:17 rpc error: code = ResourceExhausted desc = grpc: received message larger than max (4729171 vs. 4194304)

4194304 = 1024 * 1024 * 4 = 4MB
but, I pushed more than that.
How can I change default max size? There is nothing documentation that mentions about it.

hyunsooda · March 4, 2020, 11:42am

[self-answer]

grpc.WithMaxMsgSize() function can adjust max size and I used it and then solved the problem.

See : change max message size functions name · grpc/grpc-go@d19bbe8 · GitHub

MichelDiz · March 4, 2020, 5:04pm

If it is possible, please fill up an issue. So we can add a flag for that.

MichelDiz · March 5, 2020, 10:27pm

minor update:

Dgraph’s Max msg is 4GB see the links bellow.

github.com

dgraph-io/dgraph/blob/a06178b02193c036a0668e95ae2b1379e5407ced/dgraph/cmd/alpha/run.go#L376


	return net.Listen("tcp", fmt.Sprintf("%s:%d", addr, port))
}

func serveGRPC(l net.Listener, tlsCfg *tls.Config, wg *sync.WaitGroup) {
	defer wg.Done()

	x.RegisterExporters(Alpha.Conf, "dgraph.alpha")

	opt := []grpc.ServerOption{
		grpc.MaxRecvMsgSize(x.GrpcMaxSize),
		grpc.MaxSendMsgSize(x.GrpcMaxSize),
		grpc.MaxConcurrentStreams(1000),
		grpc.StatsHandler(&ocgrpc.ServerHandler{}),
	}
	if tlsCfg != nil {
		opt = append(opt, grpc.Creds(credentials.NewTLS(tlsCfg)))
	}

	s := grpc.NewServer(opt...)
	api.RegisterDgraphServer(s, &edgraph.Server{})
	hapi.RegisterHealthServer(s, health.NewServer())

github.com

dgraph-io/dgraph/blob/a06178b02193c036a0668e95ae2b1379e5407ced/x/x.go#L85


	Error = "Error"
	// ErrorNoData is an error returned when the requested data cannot be returned.
	ErrorNoData = "ErrorNoData"
	// ValidHostnameRegex is a regex that accepts our expected hostname format.
	ValidHostnameRegex = "^(([a-zA-Z0-9]|[a-zA-Z0-9][a-zA-Z0-9\\-]*[a-zA-Z0-9])\\.)*([A-Za-z0-9]" +
		"|[A-Za-z0-9][A-Za-z0-9\\-]*[A-Za-z0-9])$"
	// Star is equivalent to using * in a mutation.
	// When changing this value also remember to change in in client/client.go:DeleteEdges.
	Star = "_STAR_ALL"

	// GrpcMaxSize is the maximum possible size for a gRPC message.
	// Dgraph uses the maximum size for the most flexibility (4GB - equal
	// to the max grpc frame size). Users will still need to set the max
	// message sizes allowable on the client size when dialing.
	GrpcMaxSize = 4 << 30

	// PortZeroGrpc is the default gRPC port for zero.
	PortZeroGrpc = 5080
	// PortZeroHTTP is the default HTTP port for zero.
	PortZeroHTTP = 6080
	// PortInternal is the default port for internal use.

Topic		Replies	Views
gRPC message exceeds Dgraph dgraph	4	2843	December 8, 2020
Error: Received message larger than max Dgraph Clients untagged , dgraph-js	2	3370	February 4, 2018
Error: 8 RESOURCE_EXHAUSTED Dgraph	2	1303	November 14, 2018
How to increase message size in gprc for fetching all nodes of dgraph? Dgraph Clients untagged , pydgraph	2	808	March 31, 2020
Is it possible to set grpc.max_receive_message_length larger than 102410241024? Users	3	1823	October 22, 2019

ResourceExhausted error in grpc

Related topics