- What does a single neuron do?
- It computes some map from its large input space to a much smaller output space.
There are many model neurons, from McCullough-Pitts and simple threshold units of perceptron fame, linear functions followed by a sigmoidal transfer function that became widespread with the rediscovery of backpropagation ( do not call them neurons! ), the more realistic spiking models with an ordinary or even partial differential equation at their core, and finally detailed models of synaptic transmission, diffusion of neurotransmitters, location of ion channels and proteins modifying gene activity.
The Blue Brain project will build ( or have they already succeeded? ) a detailed model of a cortical column from detailed model neurons in an heroic attempt to understand cortical columns.
- What does a cortical column do?
- It embeds the neural spaces of individual neurons into a product space.
- It glues the individual functions and extends them into continuous functions on this space. The notion of continuity depends on the output space, so guess why there is so much feedback.
- Only the activity of output axons will influence other brain areas. Thus, in the end, we pass to a quotient of the product space. We have glued the local spaces of our neurons into a more global object. A cortical column constructs product spaces, quotient spaces and continuous maps between them.
- The complex dendritic wiring has computational significance, too. Each neuron gets – almost – the same input as its neighbors. Each input space is different.
- It is a dynamical system whose state space is the set of sections in the sheaf of continuous functions. This space is quite strange and unfamiliar from a dynamical systems point of view. Germs of functions arise from spontaneous spiking. The combination of input patterns and dynamical rules will cause them to grow rapidly, until their growth is inhibited by emerging singularities, e.g. discontinuities. Soon there will be large open subsets where our functions are continuous. The details of the dynamics, however, have very little explanatory content.
- Implementation detail aside: The space of sections of continuous functions seems quite complicated, even over such a ’simple‘ object as ordinary three– space.
Storing sections over finite spaces, however, is quite simple: Conventional digital memory does the trick. It is a pliable map from a finite address space into a finite data space.
- These sections are the subjects of neural darwinism ( Gerard Edelman ). Time-varying input will grow them, push them around, cause them to nibble at their neighbors. Shimon Edelman (Computing the Mind, OUP 2008) emphasizes the crucial role of representation spaces, dimensionality reduction, generic smoothness and hyperacuity through overlapping receptive fields. The ideas presented on this site fit well into his framework. His choice of mathematical tools, however, is quite conventional.
- There is also a slower, long-term dynamical rule. It cleaves the space apart along the singular subset. It may be re-glued in a different fashion, extending the space with new axonal input from elsewhere. This input may arrive from lower areas, from nearby neurons in a topographical map, or it may arrive as feedback from somewhere up in the hierarchy. These processes are a dynamical analogue of the well known processes of blowing–up and resolution of singularities.
- Cortical maps move around and compete with their neighbors, too. Watching them is like watching shadows on a wall.
We can find much of this in the logo. The left down arrows of the diagram correspond to the wiring of input connections. Each cell in a cortical column sees a different projection of total input into rather small neural spaces. At least initially, before the slow dynamics start rebuilding it, there is a large amount of randomness. The lower arrow represents fast memory lookup. The up arrow to the right describes derivation of a result by some kind of voting, threshold operation or averaging. The result is fed back into synaptic storage. Δ and μ become inverse to each other in the fibre addressed by (h_{i}).
- What does this setup buy us?
- individual spaces X_{i} may be removed. The column will still work.
- individual spaces may be added at whim. The feedback path will re-glue them.
- the encoding of the input may change. The feedback path will re-learn it.
- if an X_{i} turns out to be too large, we will split it into factors.
- if an X_{i} is too small. we will replace it by a product space. We may use several small X_{j}, or we may use a product X_{i} ⨉ Y. Feedback will produce covering spaces. The sections try to avoid borders by growing into a covering space.(example)
- in many cases, this extension of X_{i} may be restricted to a single fiber. Some glyphs are easily separated, others are difficult ( eg letters „long s“ and f of the „Schwabacher“ ). We may extend individual fibers over Y while leaving others unchanged. This process generalizes well-known constructions of decision trees.
- And finally: We can easily build hierarchical systems. The function f may appear as coordinate function h_{i} in another cortical column.
Although there is little similarity between a Pattern Engine and a small sheet of cortical tissue, between brain and computer memory, I want to revive the strong impression of similarity that was so prevalent in the early days of AI.
A note on the loose term cortical column:
This article The cortical column: a structure without a function
discusses the state of the notion, fifty years after Mountcastle’s hypothesis.
Half a century ago, Mountcastle et al. (1955) made a seminal observation while recording from cat somatosensory cortex. They noted that all cells in a given vertical electrode penetration responded either to superficial (skin, hair) or deep (joint, fascia) stimulation.
It appeared that for a common receptive field location (e.g. the cat’s foreleg), cells were segregated into domains representing different sensory modalities. This discovery led Mountcastle (1957, p. 430) to hypothesize ‘there is an elementary unit of organization in the somatic cortex made up of a vertical group of cells extending through all the cellular layers’. He termed this unit a ‘column’.
Further research, however, did not result in clear confirmation.
Discrete Categories and Continuous Change