Learning by Glueing

Of course, no one will build such a pattern engine from real memory components, at least for now. Since the time of Braitenberg and Marr, universal digital computers have become ubiquitous and incredibly large and fast memory units are cheap.
A software implementation is still very fast.
I have written a simplistic OCR engine along these lines. It is simplistic in the sense that it does not do layout analysis. It surrounds each dark blot on the page by a bounding box and tries to recognize its content. Screenshots will appear soon.Promised. Next: OCR

This code snippet is just bait and hook.
As much as I still like programming after all these years, a blog isn’t the right place for development.

The maximum size of the output space (256) that is hard-coded into class CogWheel might seem too small. In fact, OCR with an input space of 8×8 pixels and receptive fields of 4×4 pixels that are arranged as a 5×5 matrix of overlapping 4×4 patches works well with output spaces of size three. That’s right, just three different output values!

New!

Some more source code. Requirements: QT4, g++ .
source code with images and a 32 MByte dat-file

// This is the long-sought cognitive wheel, a basic building block for an artificial brain.
// Multiple Adressing ( Axon Arbors ) and voting  ( The Winner Takes All ) are well-known
// aspects of  cortical column computation. Less well-known is their relationship
// with sheaves of functions and the construction of spaces by glueing.

#include <stdio.h>
#include <stdint.h>

class Cogwheel
{
    uint32_t  _mask ; // (1<<_bits) - 1
    uint32_t  _bits ;  // number of address bits
    uint32_t  _out ;  // number of different output states
    uint8_t   *_mem ; // uint8_t:  _out < 256
    uint32_t  _count[256] ; // keep track of the size of the fibres
    Cogwheel( int n , int s)
    //  create a pattern engine with 'n' address bits and 's' output states.
    //  memory is initialized with random numbers from [0::states]
    //  the get() method with feedback == true shifts the fibres
    //  around until some kind of equilibrium is reached.
    //  Please note that in equilibrium  there will still be fluctuations
    //  and change. This is not a fault. It is a feature!
    /*

    ~Cogwheel() ;

    uint32_t  count( uint32_t y ) const ;
    //  return the size of the fibre over y

   double   pressure( uint32_t y )
        const { return (y < 256)? 1/(1+_count[y])  : 0.0 ; } ;
    //  as fibres are squeezed, they show increasing resistance against
    //  further compression

    uint32_t  get( const uint64_t x[] , int n , bool feedback ) ;
    //  return the majority decision. The votes are weighted, small fibres
    //  have a high pressure, large fibres become flabby.
    //  if feedback is true, the majority decision is written back
    //  into the memory cells addressed by x[] and count[] is updated.

    uint32_t  get( const uint64_t x[] , int n ,
          double majority_threshold, double veto_threshold, bool create ) ;
    //  another variant of get(). if create is true, get() will never return 0.
    //  it will create a new symbol instead.

    void  set( const uint64_t x[] , int n , uint32_t y ) ;
    //  not used in unsupervised mode

    bool  load( FILE * ) ;
    bool  save( FILE * ) ;

} ;

uint32_t Cogwheel::get( const uint64_t x[], int n , bool feedback ,
              double &m, double &v )
{
    struct
    {
        uint32_t y ;
        double   w ;
    }  rk[n] ;

    int ny = 0 ;
    uint32_t y ;

    for ( int i = 0 ; i < n ; i++ )
    {
        y = _mem[ x[i] & _mask ] ;
        int j ;
        for ( j = 0 ; j < ny ; j++ )
            if ( rk[j].y == y )
                break ; // fast on average !
        if ( j == ny ) // new
        {
            rk[ny].y = y ;
            rk[ny].w = 0 ;
            ny++ ;
        }
        rk[j].w += pressure(y) ;
    }

    double w = 0 ;
    double wtot = 0 ;
    y = 0 ;
    for ( int j = 0 ; j < ny ; j++ )  // search for the y that maximizes w
    {
        wtot += rk[j].w ;
        if ( rk[j].w > w )
        {
            w = rk[j].w ;
            y = rk[j].y ;
        }
    }

    m = w/wtot ;

    w = 0 ;
    for ( int j = 0 ; j < ny ; j++ )
    {
        if ( rk[j].y != y &&  rk[j].w > w )
            w = rk[j].w ;
    }
    v = w/wtot ;

    if ( feedback ) // write back the majority result
    {
        for ( int i = 0 ; i < n ; i++ )
        {
            uint32_t o = _mem[x[i]&_mask] ;
            if ( o != y )
            {
                _mem[x[i]&_mask] = y ;
                _count[o] -- ;
                _count[y] ++ ;  // keep track of the size of the fibres
            }
        }
    }
    return y ;
}

Post a Comment