David Marr, Cognitive Science and the Middle Path

David Marr published Vision in 1982, and the work continues to influence research in cognitive science today. So much so in fact that Topics in Cognitive Science has published a special edition ‘Thirty Years after Marr’s Vision’ including articles on the applications and relevance of Marr’s work to the modern cognitive scientist.


The Tri-Level Hypothesis

David Marr (1982) proposed in his ‘Tri-Level Hypothesis’ that when seeking to explain the behaviour of any information-processing system such as the brain or its parts, there are three distinct levels of analysis, each of which must be understood: the computational, the algorithmic and the implementation levels. The computational level represents what the system does (the problems it solves e.g. producing colour vision). The algorithmic level includes the representations and processes used to solve these problems and the MarrTriLevelimplementation level is the way in which the system is physically implemented (e.g. in the brain, the specific wiring and connections between neurons in the system). The Tri-Level hypothesis has been reformulated several times in the subsequent 30 years (e.g. Anderson, 1990; Newell, 1982; Pylyshyn, 1984) and remains a core tenet of cognitive science.


Reductionism and Vagueness

In an introduction to Topics in Cognitive Science’s special issue, Peebles and Cooper (2015) contend that the middle level, the algorithmic, is too often being ignored in modern times, with ‘reductionist neuroscience approaches’ focusing entirely at the implementation level and vague ‘Bayesian approaches’ focusing overly at the computational level. While this latter approach may indeed succeed in solving a problem which the brain solves, little or nothing is learned of how the brain itself actually solves the problem. Noting Marr’s insistence on the necessity of understanding at all three levels, the authors therefore urge greater focus on theories of cognitive architecture, which operate at the middle algorithmic level and decompose and explain the system through the interaction of functional cognitive components.


The Encroachment of Neuroscience

However, Bickle (2015), in the same issue, argues against this view. Peebles and Cooper’s attack, particularly on reductionist neuroscience, echoes Marr’s original attack on the inability of reductionists of his time to explain vision using electrophysiological cellular readings (e.g. Barlow, 1972). Bickle argues that while Marr’s original attack on reductionism was justified, it (and by extension, Peebles and Cooper’s) is no longer tenable. A swathe of new techniques and tools such as cortical microstimulation have allowed neuroscientists to begin constructing causal-mechanistic explanations of the brain including the dynamic interaction of parts and their organization as well as explanations of how these interactions ‘solve the problem’ of interest. While reductionist approaches in Marr’s time were merely descriptive (and clearly operated only at the implementation level) modern neuroscience theories are therefore genuinely explanatory and appear to encroach on the algorithmic level. A causal-mechanistic neuroscientific explanation of a system is indeed different from the kind of explanation given by cognitive science and advocated by Peebles and Cooper, but is not clearly inferior, Bickle contends. Further, the interaction between, or equivalence of, these ‘higher level’ neuroscientific explanations of the brain system and the more traditional cognitive explanations at the algorithmic level, is not fully understood and will need further work. Marr did MarrQuotenot anticipate this encroachment of neuroscience on the algorithmic level, Bickle states, and it is not clear what he would have made of it.


Synergistic Working

In the same issue, Love (2015) argues for greater cooperation between those working at different levels and proposes that findings at one level might be used to test theories at another. In fact there is already at least one good example of this synergistic working between neuroscience and cognitive psychology, and can be found in Smith, Kosslyn and Barselou (2007, p16). It began before and ended after, Marr’s 1982 work, and was in fact also within his own field of vision. In the 1970s there were two competing theories of how mental images (e.g. imagining a square) were represented in the mind. Pylyshyn (1973) claimed they were represented conceptually, similar to language (a mental image of a square would be represented simply as the concept ‘square’). However Kosslyn and Pomerantz (1977) believed that such images were actually ‘depicted’ in the mind, geometrically mapping point for point with a real image (a mental image of a square would literally be represented by four joined equal-length lines with right angles between them). For over a decade this debate continued with neither side able to disprove the other on cognitive evidence alone. However in the late 90s’ advances in neuroscience allowed careful examination of the area of the brain underpinning mental imagery. They were found to be represented in the primary visual cortex of the brain ‘topologically’. Mental imagery literally produced a ‘picture’ of activation on the surface of the cortex, which, while it would not be recognizable to a naïve viewer as the original image, corresponded to the size and orientation of the imagined image (see figure 2, below), and which mapped one to one with the mental experience of imagining the object (Klein et al, 2004; Kosslyn et al, 1995; Kosslyn & Thompson, 2003). This provided strong evidence for the ‘Depiction’ theory, and demonstrated the potential value that multi-level working could provide.


Figure 2. ‘A picture on the brain’: two sets of fMRI images from Klein et al (2004) demonstrating (a) the two stimuli used (b) the unique cortical activation for the horizontal image and (c) the unique cortical activation for the vertical image. Activation is shown for both direct perception of the images and for subsequent ‘mental imagery’.



It has been over 30 years since the publication of David Marr’s Vision and the work still remains central to cognitive science. In this time the work has been revised and the clear distinction between the three levels apparent in Marr’s time has become somewhat blurred by the encroachment of neuroscience on the algorithmic level previously monopolised by cognitive science. Further, in developments that would have surely pleased Marr, synergistic working between levels has produced advancements in understanding of the function of the brain. Finally, the Tri-Level Hypothesis still shows the capacity to provoke debate, even within a single publication, and it is perhaps this capacity which will ensure its centrality is maintained for the next thirty years.



Anderson, J. R. (1990). The adaptive character of thought. Hillsdale, NJ: Lawrence Erlbaum Associates.

Barlow, H. B. (1972). Single units and sensation: A neuron doctrine for perceptual psychology. Perception, 1, 371–394.

Bickle, J. (2015), Marr and Reductionism. Topics in Cognitive Science, 7: 299–311. doi: 10.1111/tops.12134

Klein, I., Dubois, J., Mangin, J. F., Kherif, F., Flandin, G., Poline, J. B., … & Le Bihan, D. (2004). Retinotopic organization of visual mental images as revealed by functional magnetic resonance imaging. Cognitive Brain Research, 22(1), 26-31.

Kosslyn, S. M., & Thompson, W. L. (2003). When is early visual cortex activated during visual mental imagery?. Psychological bulletin, 129(5), 723.

Kosslyn, S. M., Thompson, W. L., Kim, I. J., & Alpert, N. M. (1995). Topographical representations of mental images in primary visual cortex. Nature, 378(6556), 496-498.

Kosslyn, S. M., & Pomerantz, J. R. (1977). Imagery, propositions, and the form of internal representations. Cognitive Psychology, 9(1), 52-76.

Love, B. C. (2015), The Algorithmic Level Is the Bridge Between Computation and Brain. Topics in Cognitive Science, 7: 230–242. doi: 10.1111/tops.12131

Marr, D. (1982). Vision: A computational investigation into the human representation and processing of visual information. New York, NY: Henry Holt and Co.

Newell, A. (1982). The knowledge level. Artificial Intelligence , 18(1), 87–127.

Peebles, D. and Cooper, R. P. (2015), Thirty Years After Marr’s Vision: Levels of Analysis in Cognitive Science. Topics in Cognitive Science, 7: 187–190. doi: 10.1111/tops.12137

Pylyshyn, Z. W. (1973). What the mind’s eye tells the mind’s brain: A critique of mental imagery. Psychological bulletin, 80(1), 1.

Pylyshyn, Z. W. (1984). Computation and cognition: Toward and foundation for cognitive science. Cambridge, MA: MIT Press.

Smith, E. E., Kosslyn, S. M., & Barsalou, L. W. (2007). Cognitive psychology: Mind and brain. Upper Saddle River, NJ: Pearson Prentice Hall.















2 thoughts on “David Marr, Cognitive Science and the Middle Path

Leave a Reply

Your email address will not be published. Required fields are marked *