Sorry!

David Chalmers requested that I not publish his email on the internet. Instead I provide a summary of our exchange.

Summary

Chalmers began by saying that there was perhaps a non-terminological issue on the question of whether the representation in the brain corresponding to spatial experience, is itself spatially structured, but he is doubtful that this has anything to do with the debate between direct perception and representationalism.

As to the issue of whether the brain representation is analogical, Chalmers said it may, or may not be.

I argued that if the representation was not analogical, then there was a problem with information content, because the representation would encode less explicit information than that observed in the content of conscious experience.

Chalmers argued that analog information can be encoded in a variety of non-analog forms without loss of information. And he raised Dennet's argument about the "vehicles v.s. contents".

To this I replied:

Consider Dennett's claim that the *vehicles* of neural representation need have *no resemblance whatsoever* to the subjective *contents* of those vehicles. Think about this. Normally this means, for example, that a non-spatial, or perhaps 2-D pattern of cortical activation could correspond to a 3-D volumetric experience. Right there there is a problem with information content. But the *principle* of the vehicle/content argument is even more problematic, because it suggests that the experience of, say, a sunset by the ocean, could in theory be encoded by just 10 neurons! Or even one! Or a even a single electron! There is a serious problem with that whole concept.

A more reasonable modification of that principle would be to say that the information content of the experience cannot be any greater than the information explicitly encoded in the representation, independent of the *format* in which the information is encoded. Now we don't have the problem of information that is experienced, but not encoded anywhere. But that now means that there must be 3-D volumetric *information* encoded in the brain in some format, usually assumed to be some abstract analytical or symbolic format, as opposed to analogical.

But once we accept that *all* the information in experience must be encoded in the brain, that means there is a 1:1 correspondance between every "voxel" of our experience and the state of some entity in the brain. In simplistic "neuron doctrine" terms, for example, the color of *this* point in my experience is encoded by *that* neuron, and the *next* point must be encoded by a *different* neuron. Whatever the mapping between the neurons and the experience, the entire volumetric scene is encoded in some scrambled form. If we un-scramble the information content of the brain, we get a volumetric "voxel" image of colored volumes embedded in a spatial void. In a very real sense, then, we are viewing, in experience, the physical state of our physical brain.

But we do not experience the world as a set of disconnected values; our experience is of an integrated spatial picture, every point of which is spatially related to every other point, and all of the points in a perceived surface are perceived as a spatial continuum to some finite resolution. In other words, we see not just the individual voxel values, but we see the spatial relationship between them. If you disassembled a volumetric voxel image and scattered it in some non-spatial pattern in the brain, you would have to provide "neighborhood links" between these voxels so that each one knows where it fits in the big picture. This turns out to be more problematic than it might first seem, as shown in this argument-cartoon...

http://cns-alumni.bu.edu/~slehar/spatial/spatial.html

But it is possible to set aside the question of the information coding scheme used in the brain, and discuss perception purely in phenomenological terms, as a transformation from a 2-D sensory stimulus to a 3-D volumetric experience, agnostic to how that information might be encoded by the brain, as I do in this section of my Gestalt Isomorphism paper...

http://cns-alumni.bu.edu/~slehar/webstuff/bubw3/bubw3.html#compmech

All I am asking is for people to recognize that the information of *experience* is volumetric and three-dimensional, as a necessary pre-requisite to our understanding how that information is encoded in the brain.

It is no accident that proponents of *direct perception* universally, and without a single exception (to my knowledge), *deny* that there are spatial representations in the brain, because their theory of direct perception is itself (perhaps unconsciously) an attempt to escape from the thorny problem of the brain having to encode the vast quantity of spatial information clearly present in our experience. Their "solution" is to say that that iformation need not be encoded in the brain, because spatial experience is not an image in your brain, it is the world itself, viewed directly "out there" where it lies, beyond the sensory surface! It is a trade-off of one thorny problem for another. A desperate attempt to rescue naive realism from its obvious logical inconsistencies. Gibson himself demonstrated the absurdities that arise when you take that concept to its logical conclusion.

Chalmers re-iterated that a spatial encoding is only one way to represent spatial information.

In response I made this argument:

Say the brain used a Fourier code, where each "pixel value" (active neuron?) encoded not a pixel's worth of information, but a whole sinusoidal waveform across the visual field. A number of these values active simultaneously summate to represent a complete spatial scene. That way a constellation of individual activations collectively represent a whole image, while not being themselves in image form. In essence, this is like an image compression scheme, where a few values encode a whole scene.

But then the key question arises: Do those activations just light up like a Christmas tree, or do they, by lighting up, automatically regenerate the spatial image somewhere in the brain, like an inverse Fourier transform? Is the image represented exclusively in the abstract compressed featural form, or is there an "image decompression" algorithm that transforms it back to an explicit spatial image? I argue that there *must* be a decompression algorithm that "paints in" a full spatial scene, otherwise there is *no reason* why we would experience those activations as a spatial scene, instead of what they are, just individual points of activation.

In response to my Gestalt Isomorphism paper Max Velmans argued that the representation might be like a hologram, in which the information of a volumetric scene is encoded in two-dimensional patterns on a plate, but there is no sense in which the three-dimensional image is itself "in the plate".

http://cns-alumni.bu.edu/~slehar/webstuff/bubw3/commentaries.html#Velmans

To which I replied:

"Velmans' holographic analogy is very apt. There is indeed no "picture" as such on a holographic plate, just a fine-grained pattern of interference lines. But in order for the picture to be experienced by a viewer, or to be available for data access in an artificial brain, that picture must first be reified out of that pattern of interference lines into an actual image again, i.e. the holograph must be illuminated by a beam of coherent light. After passing through the holographic plate, that beam of light generates a volumetric array of patterned light, every point of which is determined by the sum of all of the light rays passing through that point, and it is that volumetric pattern of light in space that is observed when viewing a hologram. So if holography is to serve as a metaphor for consciousness, the key question is whether the metaphorical hologram is illuminated by coherent light to produce a volumetric spatial pattern of light, or whether the hologram in experience is like a holographic plate in the dark. If it is the former, then conscious experience in this metaphor is the pattern of light waves interfering in three-dimensional space. It is a spatial image that occupies a very specific portion of physical space, and it requires energy to maintain it in that space. This is exactly the kind of mechanism we should be looking for in the brain. If it were the latter, as Velmans suggests, then why would the shape of our experience not be that of the interference patterns etched on the holographic plate, rather than the volumetric image that they encode? What magical substance or process in conscious experience performs the volumetric reconstruction that in the real universe requires an actual light beam and some complicated interference process to reconstruct? If it is a spatial structure that we observe in consciousness, then it is a spatial structure that we must seek out in the brain, not a potentially spatial structure that remains stillborn in a non-spatial form. Otherwise the spatial image-like nature that is so salient a property of subjective experience must remain a magical mystical entity forever in principle beyond the reach of science."

http://cns-alumni.bu.edu/~slehar/webstuff/bubw3/response.html#Velmans

To this Chalmers replied that this is just an instance of the explanatory gap: there is no reason why we should experience brain processes as conscious experience at all, and no reason why brain processes should be associated with the sort of conscious experiences that they are associated with. That point arises whether or not the brain processes have a spatial form that's isomorphic to the structure of experience.

To this I replied:

It *seems* like there's no reason why we should experience brain processes as conscious experience. But it is an observational fact that we *do*. So the sooner we formulate a theory of existence whereby we *necessarily must* experience brain processes as experience, the faster we will close that explanatory gap. I thought that was the whole point of your "Hard Problem" paper, where you posit that (a simple primal form of) consciousness is necessarily a property of physical matter. Otherwise the gap is guaranteed to remain pemanently un-closed.

And the sooner we identify brain processes which are at least dimensionally matched with the dimensions of conscious experience, the sooner we will close the explanatory gap. Otherwise the gap is guaranteed to remain permanently un-closed.

We will never close the explanatory gap by simply proclaiming it to be profoundly un-explainable. That, it seems to me, is the strategy chosen by direct perception. The profound paradoxical mystery is built right in to the theory! It seems to me that we should actively seek out the moving volumetric images in the brain that we know to be in there in some form. That is the only way we can ever hope to close the explanatory gap. We must formulate a theory of existence whereby instead of being a profound paradoxical mystery, spatial experience becomes a *necessary and inevitable* consequence of certain processes in the brain. That is what representationalism is all about.

I concluded with the following proposal:

In the meantime, until we *do* identify the neurophysiology behind spatial experience, is it interesting, significant, or relevant, to model the spatial *experience* itself directly, to quantify its information content and bizarre geometry, and to model the computational tranformation from the 2-dimensional retinal stimulus to the 3-dimensional experience?

http://cns-alumni.bu.edu/~slehar/epist/epist6.html

http://cns-alumni.bu.edu/~slehar/webstuff/bubw3/bubw3.html#compmech

Or is that just a diversion of no particular consequence or significance?

Chalmers conceded that it was interesting to model experience, but with little conviction or enthusiasm.