Phil Dawes' Stuff - Latest Comments in Searching arrays in X86 assembler with a bloom filter

Re: Searching arrays in X86 assembler with a bloom filter

jsh — Thu, 11 Sep 2008 20:45:00 -0000

did you try a judy array?

http://judy.sourceforge.net/

Re: Searching arrays in X86 assembler with a bloom filter

Michael S — Thu, 11 Sep 2008 07:25:42 -0000

David W: the large bitset approach sounds good, but with bitset that size you are going to encounter cache misses on almost every lookup, which is going to end up being far slower than a naive approach. You can execute a *lot* of instructions in the time it takes to load an L2 cache line.

Re: Searching arrays in X86 assembler with a bloom filter

Rs — Wed, 10 Sep 2008 20:41:08 -0000

Not sure if you have ruled out C libraries completely but have you looked at Judy? The JudyL functions or some of the related counting ones might get you very close to an optimal solution.

http://judy.sourceforge.net/

Re: Searching arrays in X86 assembler with a bloom filter

davemw — Wed, 10 Sep 2008 19:45:40 -0000

How much memory have you got to play with? It's a primitive approach, but holding your set of ints in a 512mb bitset would allow very fast lookup. =)