Amodal Perception

Continuing from Chapter 2: The Schema As A Mental Image

Amodal perception was first discovered as a property of perception, where it occurs so unconsciously and effortlessly that we tend not to notice it at all. For example if we view two pencils crossed, one pencil occluding the other, the occluded pencil has a great gap of missing data, and yet we perceive that pencil as complete and continuous through the occlusion. Furthermore, we only see the exposed side of each pencil as a colored hemi-cylindrical surface, and yet we perceive each pencil as a whole cylindrical form, complete through its invisible volume all the way to its hidden rear surfaces. The modal percept appears as a colored surface, whereas the amodal percept appears as a solid volume occupying a specific region of space, where it is perceived, but remains completely invisible.

When I see a box on the floor before me, I generally see only three exposed surfaces, whose straight edges are usually tilted by perspective so as not to appear at right angles from my viewpoint, and yet I perceive the box as a right-angled rectangular volume, complete with an interior, which can be perceived as hollow or solid, and with hidden rear surfaces. I can easily reach back behind the box and demonstrate by morphomimesis the exact location and orientation of those hidden surfaces as if I were seeing them transparently through the box. I do not decide to form this mental image, or to make it rectangular. The amodal image forms immediately and automatically based only on the appearance of the visible surfaces, assisted by my past experience with boxes. It is my mind’s way of constructing the simplest three-dimensional explanation for the two-dimensional stimulus on my retina. The invisible image has exactly the shape that I perceive the object to have.

One reason why this amodal percept is so easily overlooked, is that it is not really a visual experience as such, even though it is often informed by a visual stimulus. When I encounter a box in pitch darkness, or with eyes closed, and feel it with my palms, I get the same amodal experience of a rectangular volume in a specific location in my space, and I perceive the whole box, through to its rear surfaces, even though I palm only selected faces or corners of the box at a time. And when I turn on the lights, or open my eyes, the tactile texture of the box felt by my palms is experienced on the very same rectangular volume as are its color and brightness that I perceive visually. The hardness or flatness of the ground that we feel underfoot even with eyes closed, are experienced as properties of the very same amodally perceived ground that carries the color and brightness and texture that we perceive visually with eyes open. In other words, the spatial structure that is our amodal experience of the world is the common ground, or lingua franca, that unites all sensory experience in a modality-independent structural representation of the world, and that amodal structure represents our perceptual and cognitive understanding of  the world.

 Furthermore, our picture of the world is incomplete without an amodal experience of our own body at the center of our space, which we also experience, even with eyes closed, as a three-dimensional spatial structure with four limbs and a head, attached to a trunk. And we can feel our own pose or posture even without looking using the sense of proprioception, which is experienced as specific patterns of articulation of the explicit spatial structure of our body.  Close your eyes and observe the experience of proprioception. The information content of that experience is equal to the information content of a wooden marionette, whose limbs and torso are bent into whatever is the current posture of your body. That is the shape of your experience of posture.

The amodal experience of our own body by proprioception, and the amodal experience of the hidden rear faces of perceived objects, are expressed in the same pure geometrical form, like an outline drawing hanging in three dimensional space, printed in invisible ink that we can magically see or feel. And it is that invisible structure that supports the modal colored surfaces that we experience visually on the exposed surfaces of colored objects, and that same amodal structure supports the sensory experience of tactile contact.

When I hold a baseball in my hand, the round shape that I feel in the cup of my hand, is perceived as part of the same amodal sphere that supports the white leathery surface that I perceive visually. The amodal percept is the lingua franca, or common ground that unites all sensory experience in a modality-independent structural representation of the world, that is our cognitive understanding of our self and our world.

The existence of amodal perception compels us to take stock anew of the nature of visual experience. From the earliest days of childhood we have been working under the naive realist assumption that the world we see in our experience is the world itself, viewed directly out there where it lies around us. The fact of amodal perception requires us to revise that view, to acknowledge that the amodal percept is not the object itself, as suggested by naive perception, but rather, the amodal percept is a data structure constructed by our mind based on sensory evidence. And that data structure is expressed in explicit spatial form: It is a solid three-dimensional structure that occupies specific volumes in the space of our experience, and the very existence and shape of that invisible structured experience is identically equal to our understanding of the structure of the world.

Amodal perception is the connecting link between perception and cognition. It has the automatic and instantaneous nature of perceptual recognition, as in the perception of the volume of a box, and yet it is also susceptible to cognitive manipulation – it is possible to modify the shape of your amodal percept by an act of will, for example by imagining (or believing) a sealed box to be full or empty, or by imagining its hidden rear faces to be missing, or punctured, or dented, if we choose to imagine them so. Amodal perception is the earliest, most basic form of mental imagery, one that we certainly share with almost all visual creatures. But amodal perception is also the door that opened perception to cognition and free-wheeling mental imagery, and true human intelligence, connecting the world of direct experience to the world of imagination.

Continued: The Language of the Mind