COGNITION AS COMPRESSION
STOP PRESS: New book
about the SP theory and its applications.
From as long ago as the 19th century it has been recognised that information processing in
brains and nervous systems (perception, learning, thinking, neuro-muscular control etc) may be governed by
principles of economy or compression. Similar principles appear in the
design of some of the current generation of artificial neural nets.
Although some researchers continued to develop these ideas, they went out of fashion for many years and
have only recently begun to attract renewed attention. Part of the reason for preparing this page
is to encourage more interest in 'cognitive economy' and principles of information compression which
have been recognised by neurophysiologists for some time, especially
the importance of inhibition in nervous tissue as a means of extracting
redundancy from information. G Von Békésy's book on Sensory Inhibition (Princeton, 1967) deserves a wider audience.
Concepts of economical coding in the nervous system have been a theme
in Horace Barlow's work for many years.
Also good value is Zipf's Human Behaviour and the Principle of Least
Effort (1949), despite the passage of years and despite the fact that it was written before seminal concepts of
'information theory', introduced by Hartley and Shannon, became widely known.
Some relevant work is reviewed in Computing, cognition and information compression and Computing as compression: an overview of
the SP theory and system.
Nice reviews of some of the issues and many pointers to relevant sources
can be found in two articles by Nick Chater: Reconciling simplicity and likelihood principles
in perceptual organization (Psychological Review 103
(3), pp. 566-581, 1996) and The search for simplicity: a
fundamental cognitive principle? (The Quarterly Journal of Experimental Psychology A 52 (2), 273-302, 1999).
Principles of Minimum Length Encoding, and their possible role in explaining how a child may learn
his or her first language, have been examined in a program of research on Language Learning as Compression.
There is solid evidence that first language learning by children may,
to a large extent, be understood as information compression (see the publications that may be obtained from that page, especially Learning syntax and meanings through optimization and distributional analysis).
'Cognition as compression' has provided the inspiration for a
programme of research on Computing as Compression, developing the
conjecture that all kinds of computing and formal reasoning may usefully be
understood as information compression by pattern matching, unification
and search - and developing a 'new generation'
computing system based on the theory. Much of this research is relevant to
understanding human cognition as described in two 'overview' articles (Multiple
alignment, cognition and computing (short) and Information
compression by multiple alignment, unification and search as a unifying
principle in computing and cognition (long)) and in several
'Cognition and computing as compression' is related to work on
Algorithmic Information Theory (AIT)
and Minimum Length Encoding (MLE) (references in the sources cited above)
but there are important differences.
John Winston Bush,
"Fascinating idea! All that mental work I've done over the years, and
what have I got to show for it? A goddamned zipfile!
Well, why not, after all?"