ned Productions Consulting


Technology musings by Niall Douglas
ned Productions Consulting
(an expert advice and services company based in Ireland)


Sunday 23rd August 2015 9.19pm

Yay just got a direct-from-mmap dense hash map implementation working, so this hash table works straight from a memory mapped file which means it can scale out to really huge hash tables - like billions of entries. On modification, it rehashes the entries into a new hash table which can also go straight into a mmap, though I suspect the linear complexity would make modifying billion entry hash tables rather slow. Anyway, that's the showstopper which was holding me up on the final tutorial example in proposed Boost.AFIO current before peer review, so I should make much better progress tomorrow now I have the dense hash map working. It was quite a bit trickier than I expected actually, I had to throw away my first attempt because I wasn't being disciplined with offset management and correctly partitioning the mmapped region during writes. Also, I made the new solution all policy class based, so you can have custom key semantics which lets you store string keys and so on.