A hierarchical network of pattern engines needs a lot of memory. The lowest layers, however, may be frozen after some initial phase. During this early phase, new output symbols and feedback produce complicated locally constant functions. Later on, modified thresholds or increased veto will stabilize the domain borders. Change and adaptation will stop.
Such a layer may be stored in a very compact form.
Recent advances in computer science have produced a family of hash functions that are a perfect fit for this kind of table. There are now algorithms that can construct a minimal perfect hash function for a very large set of keys in linear time. The resulting hash function is perfect, it has no collisions. It is minimal, function evaluation is just a single array lookup. There is no overhead for a fill factor less than 100% and no overhead for chaining or other forms of collision handling. Storing such a function needs at least 1/log(2)=1.44 bits per key, and this limit is reached up to a small constant factor.
But look for yourself at sourceforge .
The authors have built a complete, ready to use library under the LGPL. Thank you very much.
P.S. I’m trying to understand the algorithms. A minimal perfect hash function X ⟶ Y is just a universal map from X to Y. The graph-theoretic background may have implications for the pattern engine itself. Can you spot the hypergraph lurking in the logo? The image of a point of X under (hk) is an “edge” in a uniform hypergraph. The engine is trying to solve a coloring problem.