Not reusing HTTP2 connections results in significant waits/blocks

Issues Dgraph Clients
1

Report a Dgraph Client Bug

What Dgraph client (and version) are you using?

(put “x” in the box to select)

  • Dgo
  • PyDgraph
  • Dgraph4J
  • Dgraph-js
  • Dgraph-js-http
  • Dgraph.NET

Version:

What version of Dgraph are you using?

23.1.1

Have you tried reproducing the issue with the latest release?

yes

What is the hardware spec (RAM, OS)?

32G, Ubuntu

Steps to reproduce the issue (command/config used to run Dgraph).

Run (with a blocking profiler) a relatively complex app with multiple threads and many queries and mutations to Dgraph. In my case calls to transport.NewHTTP2Client become over 60% of the app waits, with over 90K reconnects in a couple of minutes runtime. A simpler way to test this is to e.g. fire off 5 queries and see that NewHTTP2Client will be called 5 times.

Understanding why this is occuring is confusing due to the complex structure of the grpc library. I think its happening in the call to c.cc.Invoke(ctx, “/api.Dgraph/Query”, in, out, opts…) on the *dgraphClient Query method, as that calls grpc.invoke which calls newClientStream. But there i get a bit lost.

Expected behaviour and actual result.

HTTP2 connections are reused across queries

1 Like
2

Additional info, I am reusing a single client across all threads and connecting in the typical way according to the docs, it just appears that the dgrah client generates a new connection per query

1 Like
3

Which port are you using and are you using the gRPC protocol rather than the REST protocol?

4

yes grpc on the 9080 endpoint. For this test i used snappy compression, grpc.MaxCallRecvMsgSize(500mb), grpc.MaxCallSendMsgSize(500mb), and no TLS. Ill check with all these options disabled today but looking at the code I think this happens in a purely default scenario. Thanks.

1 Like
5

My working hypothesis is its a quirk of DGO using gogoprotobuf which was deprecated in 2022 due to some nasty incompatibities with standard grpc (https://twitter.com/awalterschulze/status/1584553056100057088). I tried regenerating the protos in a fork but i think it no longer runs?

1 Like