Spent CPU time normal?

Issues Ristretto
,
1

Hi,

So I have a net.http handler that functions as a proxy and needs to consume every hostname, parameter and URL and store it into a database.
I use ristretto to cache the stored items so I do not perform unnecessary DB lookups.

Example code:

	hostCache, err := ristretto.NewCache(&ristretto.Config{
		NumCounters: 15 * 1000000,
		MaxCost:     10 * 1000000, // 10mb
		BufferItems: 64,
	})

		valid, found := hostCache.Get(host)
		if found && valid.(bool) == false {
			logger.Debug("denying (cached) access to private IP")
			return req, newHttpResponse(req, http.StatusForbidden, "Forbidden", "invalid target")
		}
		if !found {
			ips, err := net.LookupIP(host)
			if err == nil {
				for _, ip := range ips {
					if !pkg.IsPrivateIP(ip) {
						continue
					}

					go func() {
						hostCacheMutex.Lock()
						hostCache.SetWithTTL(host, false, 0, time.Hour)
						hostCacheMutex.Unlock()
					}()
....

However when profiling my go app during testing, I see that a significant portion of CPU time is spent in processItems. Is this to be expected?

image
image3200×220 34.6 KB

2

@martinmr and @ibrahim does this look normal?

3

This looks normal to me. @hazcod do you see CPU usage even when there was no activity in the cache? If so, it could be because of the clean up ticker that we use to remove the discarded entries.

We can make this ticker configurable so that you can increase it to a very high value if you don’t use TTL. This will prevent some unnecessary CPU cycles.