Accumulators/Top-k heap for large collections

[JMMackenzie]

The current process for generating the top-k involves a SIMD-based block-access approach over the accumulator vector after the processing is completed.

We might also want to implement the JASS approach which maintains, during query processing, a top-k heap over the accumulator table (essentially, pointers to the accumulator table). That way, no linear scan is required, and the heap is maintained on-the-fly.

I suspect this will be more efficient for larger document collections.

[JMMackenzie]

Another idea here: Can we do some hybrid in between what JASS does and what we currently do?

EG, store a bit-vector corresponding to each CHUNK_SIZE contiguous elements in the accumulator table. Once an element gets "hit" in the table, set the bit to 1. [That is, we maintain the bitvector on-the-fly as we process the query.]

At top-k generation time, do the current process, but skip directly to blocks with a '1' bit (not naively looking at all blocks).

This also helps us when we need to zero the accumulator table; we simply zero just the CHUNK_SIZE elements within each 1 zone, and then we know the whole table is initialized (this is what JASS does). Then we zero the BV and we're good to go.