DISQUS

DISQUS Hello! Phil Dawes' Stuff is using DISQUS, a powerful comment system, to manage its comments. Learn more.

Community Page

Jump to original thread »
Author

Searching arrays in X86 assembler with a bloom filter

Started by phildawes · 9 months ago

I’m currently writing what amounts to an olap database in factor in my spare time, which is a pretty interesting project. Actually spare time is limited now that I have a child but factor is affording me a pretty good productivity compared to python and scheme so it’s proba ... Continue reading »

4 comments

  • How much memory have you got to play with? It's a primitive approach, but holding your set of ints in a 512mb bitset would allow very fast lookup. =)
  • Not sure if you have ruled out C libraries completely but have you looked at Judy? The JudyL functions or some of the related counting ones might get you very close to an optimal solution.

    http://judy.sourceforge.net/
  • David W: the large bitset approach sounds good, but with bitset that size you are going to encounter cache misses on almost every lookup, which is going to end up being far slower than a naive approach. You can execute a *lot* of instructions in the time it takes to load an L2 cache line.
  • did you try a judy array?

    http://judy.sourceforge.net/

Add New Comment

Returning? Login