Filtered counts implementation

Example: We want to count the number of friends satisfying condition A & B & C.

Can we work with just the counts from task.Result? No, you need the UIDs in order to do the filtering.

When filtering, can we just ask the worker to return the count after the filtering? Yes and no.

If it is just a simple filter with one condition, then yes. However, if there are ANDs and ORs, the current code will do the intersection and merging in the coordinator. This also seems unavoidable. Imagine the above condition A & B & C. There are three filters to be applied. You will need the UIDs at each intermediate step. On the last intersection (we’re doing this serially), you can ask the worker to return just the count, but this is only possible for the last condition C.

Now, if you are doing A | B | C, this is even harder. We currently do it in parallel. Yes, we can do it serially, but again, only the worker processing the last condition can return counts but not the UIDs.

1 Like

I see what you mean. Yeah, we’ll have to do the merging ourselves at the coordinator server, and then rewrite the resultBuf flatbuffer. I think that’s the only way to do filtering with count. It’s inconvenient given the nature of FB apis, but if we have good libraries to help with this, those libs can do all the heavy lifting, and we can have clean APIs to work with.

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.