Manish Jain - High Performance Manual Memory Management in Go

Go Systems Conf
#1

Manish Jain
Manish Jain1920×1080 226 KB

High Performance Manual Memory Management in Go

Check out the Manual Memory Management in Go using jemalloc blog!

About Manish

Manish is the Chief decision maker at Dgraph. He got thrust into distributed systems right out of college, working on real time web indexing system at Google. He then lead various projects to consolidate and serve knowledge graph right behind web search. Implementing GTD and Zero Waste practices, Manish is into efficient and minimalist living. He loves cycling, swimming and ultra light travelling.

Feel free to follow and connect with Manish on Twitter, LinkedIn, and GitHub!

Have questions for Manish?

Please submit them in this thread. Manish would love to answer them!

Haven’t signed up for the free conference yet?

Grab your free tickets here :slight_smile:

Go Systems Conf - Agenda
#2

The code that @mrjn demoed during the talk is available here: https://play.golang.org/p/oual48HFP9S

1 Like
#3

Bonus points for folks who figure out the memory leak there, using the ‘leak’ build flag.

#4

Thank you for your talk!

I have a question about the benefits of byte-slices over structs from the GC perspective. You said that using only byte slices is more performant and I don’t understand why. Seems like struct in go which does not have pointers should not have any GC-overhead against similar size byte slice, or am I missing something?

#5

The idea is that you laydown structs into byte-slices (using one of the 3 different methods mentioned in the talk), and then lay down many of these bytes slices onto bigger slices. So, from Go’s perspective, you took 64MB byte slice, but then laid down milllions of structs on that one allocation. Now, GC does not need to do a lot of scan, because you only allocated that one byte slice.

Becomes even better when you allocate even that byte slice from jemalloc – I’ve found jemalloc memory management to be a lot more predictable and work better with the system than Go’s.

Of course, if your struct was not allocated on heap to begin with, then there won’t be much value in converting it to a byte slice.

1 Like