- rename BUCKET_(LOWER|UPPER)_LIMIT to HASH_(MIN|MAX)_LOAD
as this value is usually called the hash table's minimum/maximum load factor.
- remove MAX_BUCKET_SIZE (not used)
- regroup/reorder definitions
this was also the loop contents of hashindex_merge, but we also need it callable from Cython/Python code.
this saves some cycles, esp. if the key is already present in the index.
BUCKET_UPPER_LIMIT: 90% load degrades hash table performance severely,
so I lowered that to 75% (which is a usual value - java uses 75%, python uses 66%).
I chose the higher value of both because we also should not consume too much
memory, considering the RAM usage already is rather high.
MIN_BUCKETS: I can't explain why, but benchmarks showed that choosing 2^N as
table size severely degrades performance (by 3 orders of magnitude!). So a prime
start value improves this a lot, even if we later still use the grow-by-2x algorithm.
hashindex_resize: removed the hashindex_get() call as we already know that the values
come at key + key_size address.
hashindex_init: do not calloc X*Y elements of size 1, but rather X elements of size Y.
Makes the code simpler, not sure if it affects performance.
The tests needed fixing as the resulting hashtable blob is now of course different due
to the above changes, so its sha hash changed.
This fixes a infrequent problem when (refcount * chunksize) overflowed a int32_t.
chunksize is always <= 8MiB and usually rather ~64KiB (with default chunker params).
Thus, this happened only for high refcounts and/or unusually big chunks.