Our personal experience of this world is something like the experience of waking up in a strange place and momentarily not knowing where we are, or how we got here, or perhaps even who we are. Except that in life, it takes decades to piece together what takes only seconds in awakening. Furthermore, during the mental awakening in our infancy, we do not have the guidance of memory to make sense of the world around us, except indirectly, through the cultural memory of the past as passed on through our cultural history. As we begin to piece together a coherent picture of the world around us in our early childhood, we quickly forget the initial confusion of the first stages of awareness. For we discover through experience that the world in which we live is a stable structured place, in that the places we explore have an existential permanence. If we leave one place and go to another, we find that we can always return, because the places we leave behind seem to continue to exist even in our absence. And when we observe changes in this world, they always seem to have reasonable causes. We no longer experience the unaccountable appearances and disappearances that were common in the magical world of our infancy. We begin to trust that the sun will rise again every new day just as it has every day before, and that the world will continue to run according to the causal laws that it has always seemed to follow. For although we cannot always predict the exact course of events in the future, we can usually piece together how and why things occurred after the fact, which can sometimes give us a fair guess as to how events will unfold in the future. But as our mental picture of the world around us comes into sharper, more stable focus, we experience a progressive amnesia for those decades of bewildered confusion that marked our initial entrance into this world.
Knowledge is a structured affair, individual facts and observations fit neatly into place amongst other facts and observations. The sky arches over the earth, and our personal environment is nestled between the two. The road to the center of town runs this way, and the road to school runs that way. Then we discover that we can travel from the town straight to school, without having to come back home first, because town, school, and home maintain a structured existence in our map of the world. As our knowledge expands into a scientific understanding of the world, we learn that the earth underfoot is a sphere of enormous dimensions, and that the blue sky extends only some few miles overhead under the boundless void of outer space. We build up our understanding of the world from the center outwards, gradually expanding the world of our immediate experience between earth and sky, into a much larger space of the globe of the world suspended in the void of space. And the expansion of knowledge occurs also inwards, from the large to the small, as we learn of the microscopic world of germs and viruses, and the sub-microscopic world of molecules, atoms, and sub-atomic particles, all the way to the quantum fluctuations of space-time. Science seems to offer an impartial objective view of the world, unlike the egocentric perspective of our infantile experience. We begin to develop the impression that the world revealed by science is a world of objective reality that represents a kind of absolute truth. The physical world revealed by science is what is real.
But there is trouble in the world of knowledge. For although our factual understanding of the world hangs together as an integrated whole, with our knowledge of physics and astronomy and geography each taking their appointed place in the structured tree of knowledge, the tree itself hangs unsupported by anything substantial. Although we know that empty space extends upward and outward in all directions, exactly what it is that exists beyond the farthest stars and galaxies, nobody can really say. Modern cosmology offers glib reassurances of the concept of a finite but boundless universe, like the surface of a sphere, which can be traversed endlessly in any direction, without ever leaving the finite surface. In a three-dimensional space with this property we could set out in any direction, and if we travel far enough in a straight line, we would eventually arrive back at our initial starting point from the opposite direction. This explanation seems far more satisfactory than the idea of an infinite universe, a mental image that simply doesn’t seem to fit into our head, or the idea of a finite and bounded universe, which begs the question of what might be beyond the boundary. But the idea of a finite but boundless universe is also somewhat unsatisfactory, because it is difficult to make a mental image of the absolute nothingness that supposedly surrounds the finite universe. The best I can picture nothingness, i.e. the absence even of empty space, is when it is surrounded by somethingness. For example there is nothing between my thumb and forefinger when I pinch them together tightly. But what kind of nothingness can it be that surrounds the four-dimensional sphere of the finite boundless universe? The best we can do with this kind of mystery is to attach some kind of label to the quandary, and file it for future reference, to be dealt with later, preferably by someone else, who can perhaps couch the solution in enough obsfucatory mathematical formalism as to convince us of its veracity.
But the limits of our knowledge of the outer universe is not the only problem of this nature. We have exactly the same kind of fundamental problem at the other end of the size spectrum, in the realm of the very small. Mathematicians comfort us with the notion of space as a continuum, like the line that represents real numbers, any interval of which is infinitely divisible into still smaller intervals. But can we have any confidence that there is any kind of ultimate truth to this as a property of space itself? As far as we know, the quantum fluctuations of space-time are about the smallest thing we know about (what little we can know about such things). But is it at all meaningful to say that distance itself exists to much smaller scales than that? Is there any such thing as distance below a certain ultra-tiny scale? For my part, I can say that it hurts my brain just as much to think about “infinitessimility” as it does to think of infinity. And yet the alternative concept, of a minimum scale to reality below which distance is no longer divisible, seems as absurd as the finite but bounded universe at the other end of the size spectrum. Again, this is a problem to be filed for future reference, or to be papered over with an official- sounding label to indicate that we have been there, and pondered that, and marked it as another well known frontier of the terra incognita that surrounds our knowledge of the world.
The boundaries of our knowledge pertain not only to the universe’s spatial extent, but also to its essence. What is space, or time, or space-time? Or, as philosophers like to phrase it, what is the ontology (is-ness, ultimate nature) of space-time? Ontology is one of those comforting words that we use to paste over ultimate mysteries to shield us from their frightening glare. What kind of explanation could possibly be satisfactory to account for the ultimate nature of space and time? And a similar existential quandary hangs over the ultimate nature of matter, and energy. In physics we describe these things with mathematical equations and formulae that predict how matter and energy behave. But matter and energy are more than just equations, they have material existence, something that an equation does not have. As for time, astronomers assure us that it has a finite beginning at the moment of the big bang, which they assure us, has been reconstructed to the minutest fraction of a second after its spontaneous coming into existence. But as to what, if anything, occurred before the big bang, or whether it is even meaningful to think of anything occurring then, or what the ultimate fate of the universe might be at the other end of the time line, these too are mysteries too great to be grasped in any meaningful way, so we label them neatly and file them for future reference.
With all this profound mystery suffused throughout our world of knowledge, we might well ask what it is that we really know with any kind of confidence? What do we know with any certainty to actually be true? The answer to that question can only be found back at the trunk, or the heart of our tree of knowledge, the ultimate basis on which all of our observations of the world are founded, and that is our own conscious experience. Consciousness is known to be somewhat unreliable as a representation of external reality, because we are easily fooled by illusions, and occasionally by outright hallucinations. But when the subject of our scrutiny is conscious experience itself, our knowledge of that particular entity is the most certain truth that we can ever possibly know. So although I see a table before me, I cannot be absolutely certain that it is not a hallucination, there may or may not be a real table present before me. But I do know for a fact the properties of that spatial experience, that it is a volumetric spatial structure bounded by a colored surface. I see a table, so I am certain that a table has appeared in my consciousness.
Descartes argued that the epistemological tree is rooted on the deduction “I think therefore I am”. It is hard to argue with that logic. But there is something that we know with equal certainty as the knowledge of our existence, and that is our sensory experience. Whether in the form of perception or hallucination, we know for a fact when we are experiencing something. In fact, experience is more basic and primal than abstract cogitation, and therefore raw experience has a stronger claim to the root of knowledge, because thought and cognition are themselves based on experience. It is hard to have a thought of any significance and information content without that thought referring directly or indirectly to some kind of experience. Experience, on the other hand, requires no cogitation, as is demonstrated by the practice of meditation, whose objective is to experience the raw experience of being, in the absence of any explicit cogitation. It is more correct therefore for the foundational statement of epistemology to be “I experience, therefore I am.” or perhaps, “There exists an experience” (in my awareness) whether or not there is a ‘me’ or ‘I’ that observes that experience.
In his book Perception, H. H. Price (1933) presents an insightful analysis of the relation between knowledge and experience.
“When I see a tomato there is much I can doubt. I can doubt whether it is a tomato that I am seeing, and not a cleverly painted piece of wax. I can doubt whether there is any material thing there at all. Perhaps what I took for a tomato was really a reflection, perhaps I am even the victim of some hallucination. One thing however I cannot doubt; that there exists a red patch of a round and somewhat bulgy shape, standing out from a background of other color patches, and having a certain visual depth, and that this whole field of colour is directly present to my consciousness.”
The Critical Realist movement in philosophy (Sellars 1916, Russell 1921, Broad 1925, Drake et al. 1920) made a significant advance in addressing the epistemological question with the introduction of the notion of sense data, or sensa. The sense data are the raw material of experience before cognition has stepped in to analyze and interpret that experience. In the case of the tomato, the sense data are exactly that red patch of round and bulgy shape. It is the aspect of experience of which we can be absolutely certain that it exists, at least in the form of an experience. It is the sense data, therefore, that are the real foundation of all of human knowledge, both of the individual and of humanity at large, so it is with the sense data of experience that we must begin the investigation of what we can know.
All sensory experience is knowledge, although it is direct knowledge only of the internal state of our own mind, and only indirect knowledge of the world beyond mind. We cannot in principle distinguish between the world and our experience of it with any real certainty, because the only way we can see the external world at all is by its effects on the phenomenal world. Nevertheless, like the wiggly picture we see of the bottom of a swimming pool through its surface waves, we can tell that experience is composed of a wiggly wobbly picture of an enduring unchanging pattern seen through the disturbance. The unchanging or invariant feature is likely to be a manifestation of objective external reality seen through our ever-changing and unstable perceptual representation of it. The perceptual representation sets the limits on the possible range of experience, while the configuration of that representation, the state that it is in at any one time, carries the information content of the experience. Like an alphabet of 26 letters, or a color gamut composed of 256 color values, the brain speaks a language of representation that is spelled out in a finite alphabet of experiential tokens. But the code of perceptual experience is analogical, an explicitly spatial language of representation. We see the world as a colored spatial structure, embedded in a spherical space we experience to surround us. The spatial configuration of solid volumes with colored surfaces that we perceive embedded in the sphere of our spatial experience is the knowledge that we have of our perceptual experience, and indirectly, we commonly assume it to also be knowledge of our local environment. We learn to recognize the telltale signs of illusions or hallucinations masquerading as external objects. When perceived objects appear or disappear suddenly and unaccountably, or exhibit other unusual behavior such as floating away or changing shape or color, we quickly learn to question their reality as objectively real external objects. Invariance is the key characteristic of the objectively real seen through the transient unreal.
There is more to perception therefore than the mere recording of sensory experience. There is also a spatial understanding, an extraction of deeper knowledge from the sensory scene. The sensation presents configurations of colored surfaces that we experience, while the perception presents whole objects with characteristic properties. There is a great deal of interpolation and extrapolation that takes place in the perceptual process. Although we only experience the visible front face of the tomato directly, unless we have specific information to the contrary, we generally suppose the tomato to be a whole fruit, whose hidden rear surfaces are presumed to be similar to the visible front face of the fruit that we see. If we have actually seen that hidden surface at some time, for example if we handle the fruit, turning it over in our hand before putting it down before us, then our knowledge of its hidden surfaces becomes much more certain. But in any case our experience of the fruit always has this dual character, split between a visible front surface, the red and bulgy sensation, and our perceptual experience of the whole fruit as a solid volumetric object. We can almost “see” its hidden rear face, in the sense that we “know” with a fairly high degree of certainty the exact shape and spatial extent of the fruit as a quasi-spherical object. And if we have any experience with tomatoes, we can even imagine its internal structure, the thin outer membrane backed by a fleshy crust, and a soft wet core of seeds embedded in a greenish gelatinous mass. This knowledge takes the form of a spatial picture that we experience, like an engineering drawing of a building with semi-transparent walls that reveal its internal structures. In fact, the engineering drawing is a pretty good depiction of our spatial understanding when viewing an object like a building, even when we see only its exterior. That is why engineers find this style of presentation so useful for communicating spatial ideas. Michotte (...) has named this dichotomy between the sensory experience of visible colored surfaces, and the invisible spatial structures that are inferred by perceptual processes. Sensory experience is always expressed in some specific sensory modality, whether it be color, brightness, motion, or binocular disparity, and this experience is therefore known as modal perception. The invisible spatial structure inferred from this experience on the other hand is known as amodal perception, because it is not expressed in any particular sensory modality.
Now of course not all knowledge appears in the form of a transparent mental picture. For example the knowledge of the price of a tomato, or where it can be obtained, or how this particular tomato came to be where we now find it, these are knowledge of a higher more abstract form which does not appear in the transparent pictorial way that we experience its hidden structure. There is however a continuum between that kind of abstract symbolic knowledge and the transparent spatial structure presented by the amodal spatial experience, which in turn is intimately coupled to the spatial experience of the modal surfaces from which that structure is inferred. All knowledge is ultimately founded on direct sensory experience, or at least on a memory or imagination of such an experience, and therefore our investigation of what we can know must begin with an analysis of our experience.
How are we to quantify something as vague and all-encompassing as the concept of experience? What is the appropriate level of analysis? Experience ranges from the low level sensory qualia, the raw material of sensory experience, to the more abstract but still vividly sensory experience of colored surfaces on volumetric objects, to the still more abstracted experience of the objects themselves as volumetric structures, to the still more abstract concept of the identity of those objects, and the hierarchy of categories to which different objects belong. All of these are aspects of experience, ranging from lower, more immediate sensory experience, to higher, more abstracted, non-sensory or cognitive experience. What is the appropriate level of description, and what are the fundamental units?
As in physics, I propose that we begin from the bottom upwards, beginning with a description of the most immediate modal experience, its dimensions of variability, and information content. The units are those of the artist: color, space, and light, with properties as we perceive them to have. So, for example, the experience of a table is defined as a table-shaped volumetric region of phenomenal space that is “painted” in the color that the table is perceived to have. Actually, only the nearer exposed surfaces of the table are experienced with modal color, the hidden rear surfaces, and sometimes the volumetric interior are generally assumed to be of the same color, although that color is not part of the modal experience of the table. The purpose of this almost tautological quantification of the structure of experience is to highlight an aspect of that experience which is often overlooked, that is, its vividly spatial nature. Whatever the neurophysiological coding scheme by which that experience might be implemented in the brain, the experience that is generated by the brain, the product of all that perceptual processing, is an explicitly spatial structure in experience.
There is a curious split, or schism, running right through the perceived object, that splits it into sensory stimulus versus perceptual interpretation; given evidence and inferred conclusions; the vivid modal sensation, and the invisible, amodal, knowledge. Although we generally consider knowledge to be an abstracted nonspatial kind of affair, at this lowest-level sensory interface, knowledge reveals itself to be a vivid spatial structure, even if it is only an amodal one. We see the world as volumetric objects. And those volumetric objects are inferred from their visible exposed surfaces by a process of symmetry completion or spatial extrapolation/ interpolation. The mechanism by which the brain extracts epistemic knowledge from sensory experience involves the same kind of symmetry detection and completion process that is used by the artist, sculptor, poet, and composer, to flesh out the rich symmetrical and periodic patterns of the world as they appear in the exaggerated and cartoonized world of aesthetic synthesis. The processes of artistic creation, musical composition, and mathematical analysis, involve very similar principles of pattern recognition and completion, analysis and synthesis, abstraction and reification, that shed light on the fundamental principles behind perceptual knowledge. These issues are discussed in chapter 8.
Models of conscious experience often go astray by defining experience in such vague or abstracted terms that the explanation is no clearer than the aspect of consciousness that it supposedly explains. I intend to ground my discussion of consciousness by beginning at the lowest, most concrete level, the part of experience that is probably very similar across individuals, and even across related species. I further ground my speculations by expressing them in terms that could actually be implemented, at least in principle, in some physical machine, although the kinds of machines I propose are very different than those generally favored for computation today. In effect what I describe is the functional architecture, or how the perceptual system operates functionally, based on its observed properties in experience. I do not have a complete answer to how perception completes volumetric forms, but I do establish that this is what perception is doing, as a computation. In Chapter 7 and 8 I explore the basic laws of perceptual processing based on an introspective analysis of perceptual completion as I experience it in everyday vision, and in Chapter 9 I propose a neurophysiological theory of how the brain computes these spatial symmetries, by way of harmonic resonance, or patterns of standing waves in the neural substrate to express the spatial structures of our experience.
But first we begin with an analysis of the dimensions of visual experience, its capacities and limitations. In Chapter 3 I discuss the distinction between the experience of self and of non-self, where the boundaries of our self really are. In chapters 4 and 5 I discuss the historical debate between realism and idealism, and show that much of the confusion has been due to lack of clarity on the central epistemological issue, of what in our experience is “inside” our self, and what is “outside”. The realist is right, that we can observe the external world (as if) directly. The idealist is also right, that all we can ever experience is the inside of our own head. These two apparently contradictory views, each of which can be shown to be true, and to be false, can both be true simultaneously within the larger framework of representationalism; our experience of the world is indeed indirect, so we cannot ever experience external reality directly, which is consistent with the idealist claim. At the same time however, certain aspects of our internal experience do correlate reliably with certain essential aspects of our environment, such that we can get reliable information about the configuration of the world around us, as the realist claims, although that information is acquired indirectly, through the medium of the sense data, which are themselves properties of experience. But the qualia of conscious experience, the raw colors and feelings with which it is painted, the interface between our brain and its experience, are themselves intrinsic properties of the brain, at the same time as being part of our experience. The internal mechanism of the physical brain, or certain very special parts of it, are the only physical entity which we can experience directly. We know what it is like to “be” the patterns of electrochemical activity in our own physical brain. But by the very fact that our mind is conscious of its own spatial structure, and given that our mind is a physical process taking place in the physical mechanism of our brain, that is already direct and incontrovertible evidence that a physical process can, under certain circumstances, become conscious of its own spatial structure. This insight has profound implications for the place of science and physical knowledge with respect to conscious experience. In fact, this insight inverts the traditional relationship between science and experience, because it shows that experience is a primary and foundational aspect of all physical matter, and therefore consciousness is not some mysterious entity that serves no functional purpose and leaves no detectable mark in physical reality, as many philosophers propose, but rather, consciousness is a fundamental aspect of all physical existence, of which human consciousness is only a tiny subset. Consciousness is what it feels like for physical matter to exist, and human consciousness is what it feels like to be a certain physical process in a living human brain. It turns out that this one epistemological inversion resolves a number of the profound paradoxes that have plagued discussion of consciousness for centuries.