This is a summarized version of the full
debate that I had with Myrddin Emrys on the epistemology of visual
experience, based largely on my cartoon
epistemology. Certain points have been highlighted [Emphasis added, S. L.] to indicate points that become pivotal in the subsequent
exchange.
To: Steven Lehar
date Dec 5, 2006 8:56 PM
subject Epistemology of Observation
I read through your cartoons
explaining your view, and while I have no particular disagreement with your
overall interpretation, I strongly disbelieve the physicality of your argument.
Apologies if my arguments duplicate some of those you linked... I deliberately
did not read them, so that I wouldn't be infected by their ideas. :-) I'll read
them after I send this.
Specifically, I disagree that any coherent,
physical, volumetric physical spot within my mind hosts a virtual
representation of the world around me. I do agree that all of my perceptions
are really my perceiving the workings of my senses... I do agree that it is
indirect, as you argue. But I believe that the symbols within my mind are so
abstract that it's impossible to point to any one location in my mind and say
'there's the representation of a house!'.
[Emphasis added, S. L.] There are
layers upon layers of abstraction within the mind, that go so deep it's
impossible to even understand it. Even a computer genius of the highest caliber
would have an impossible time understanding the purpose of a computer program
by examining the physical transistors switching off and on inside a processor,
assuming they had a way to observe that directly at all. And processors were
designed to be logical and orderly... the mind is an evolved instrument, with
no requirement to be transparent to our understanding, so I have no belief that
there will be a coherent, recognizable representation of reality that we can
see within it by examining the low level details of the brain.
Second, there is a strong argument that the more perfect a representation
becomes of the world, the more perfectly it simulates reality, the less
important the distinction between the representation and reality becomes. For
example, look out a window. The photons from the world beyond pass through that
window and reach you. When you see a flower, you can reasonably state that you
are looking at a flower. Now replace that window with a photosensitive
surface that absorbs photons and re-emits them identically on the other side
(ie, an invisibility suit). You are no longer seeing the photons that bounced
off the flower; instead, you are viewing the emitted copy of those photons,
deliberately designed to simulate the light from the flower in as fine a detail
as you are able to observe. The 'window' is no longer transparent, but it
appears to be as far as you can tell. Are you looking at a flower, or are you
looking at a representation of the flower? Does it matter?
…[ see full text ]…
The question becomes, 'What value is there in acknowledging the indirectness of
our senses when these indirect senses so closely model reality?'
"What value is there in acknowledging that this window is really an invisible
plate of metal, since it looks just like a pane of glass"
What insights has this view of reality led you to?
To: Myrddin Emrys
date Dec 6, 2006 10:19 AM
subject Re: Epistemology of Observation
Hi Myrddin,
Thank you for your interesting email. Your key point is summarized in your opening statement:
>>
I
disagree that any coherent, physical, volumetric physical spot within my mind
hosts a virtual representation of the world around me. I do agree that all of
my perceptions are really my perceiving the workings of my senses... I do agree
that it is indirect, as you argue. But I believe that the symbols within my
mind are so abstract that it's impossible to point to any one location in my
mind and say 'there's the representation of a house!'. ... Even a computer
genius of the highest caliber would have an impossible time understanding the
purpose of a computer program by examining the physical transistors switching
off and on inside a processor
<<
Yes the computer genius would have trouble locating the representation of a
house in the computer. Nevertheless, the house is in there, whether he can find
it or not. Furthermore, if it is an *image* of a house we are talking about
(and I hope you agree that our *experience* of the world appears in the form of
a spatial "image") then every pixels worth of that image must be
explicitly present in the computer at some location. And that information must
be retrievable by the computer, for example when you ask it to print the image,
in which case that representation must be accessed and unpacked or decoded in
order to restore each pixel value literally in its proper location in the
image.
Yes
a computer often uses abstract symbolic codes that bear little resemblance
to the image itself. For example image compression schemes, like those used in
.gif and .jpg images, store image information in a very non-pictorial form.
But the computer also has access to the appropriate image
*decompression* algorithms required to unpack that compressed code back into a
spatial image when required.
The
question is: Does the brain's abstract coding need to be unpacked or
decompressed into an explicit spatial representatation when I am having a visuospatial
experience? Or is it sufficient for that experience to remain in abstract
symbolic form? My argument is that when I have a spatial experience of a
square, for example, my immediate
(pre-attentive, pre-cognitive) experience of that square contains a separate
and distinct experience of color throughout the surface of that square, and all
of those points are experienced together simultaneously as a spatial continuum
of a certain location, shape, and spatial extent. The *information content* of
immediate experience is identical to the information content of a model of a
square like a painted cardboard model in a museum diorama. That is
*not* the same as the information content of a symbolic code like square(Color: white, Size: 100, Location: (xLoc,
yLoc)). That is NOT the information I am experiencing when I
view a white square. That symbolic information would have to be reified with a
decompression algorithm something like:
// Paint that square!
for(y=0; y<Size; y++){
for(x=0; x<Size; x++){
MyExperience[x+xLoc, y+yLoc] =
white;
}
}
to paint out a square as it appears in my experience. This means that there must be an "image buffer" of some sort somewhere in my brain where this scene is explicitly painted out or registered, otherwise there would be parts of my experience (specific points in that square) which are experienced, without ever being represented in my brain.
So there are two issues that must be disconfounded. First: Is the representation an *explicit* representation, in the sense that every pixel (or voxel) value is explicitly stored somewhere in the brain? I believe the *information content* of immediate experience argues to the affirmative. If so, then Second: Is that explicit representation necessarily spatially organized in the brain? In other words, is the representation not only *explicit*, but also *explicitly spatial*? The second issue is more thorny.
We cannot tell from the structure of visual experience whether it is encoded in an explicitly spatial representation. However it is not only the structure, but also the *function* of experience that appears to involve spatial field-like operations, or spatial algorithms. For example when I dream or hallucinate a rotating cube, there is a cube-like structure in my experience that undergoes coherent spatial transformations such as translation, rotation, and spatial scaling by perspective. If these are not computed in an explicitly spatial representation, then they must at least be computed in a representation that is *functionally equivalent* to an explicitly spatial representation, so that the experienced cube jumps seamlessly from pixel to pixel *as if* they were spatially contiguous, even if they are not.
This argument is made more explicitly in this link:
http://cns-alumni.bu.edu/
And some of the other objections you raise are addressed in this link:
http://cns-alumni.bu.edu/
Your analogy of the "invisibility window" is interesting and apt. Yes it seem hard for us to distinguish the percept from the reality if the representation is sufficiently vivid. But in fact there *is* a simple way to distinguish the percept from reality, and that is to simply close your eyes! The part that continues to exist unchanged is the objective external world, whereas the part that vanishes without a trace is the part that is subjective experience. Like when you pull the plug on your "invisibility window" and the screen goes black, while the world beyond the screen continues to exist unchanged.
Furthermore, notice how your "invisibility window" is itself spatially organized in an image-like array, an explicitly spatial organization, otherwise it would be a very poor picture of the world it represents.
You might want to check out this PowerPoint presentation of the above arguments in one animated audio-visual presentation from my talk at the Tucson 2006 consciousness conference:
http://cns-alumni.bu.edu/
Steve
To: Steven Lehar
date Dec 6, 2006 4:51 PM
subject Re: Epistemology of Observation
You discuss your mental manipulation of an object... such as
a cube. And you describe your mental processes when thinking about it... how you
rotate and transform it mentally, a very exacting mental manipulation that
clearly must use voxels.
I submit that your mental picture is an order of magnitude more precise and
exacting than mine... in fact, I would suspect your mental imagery is vastly
more detailed than most people [Emphasis
added, S. L.] (except those who routinely imagine three dimensional
constructs in their heads... I presume in the field of sculpting and
engineering this skill is relatively common). You say that you spent months of
introspection trying to figure out how your mental model functions... I propose
that due to these months of mental exercise, your mental machinery is no longer
typical.
I am of the opinion that different people have vastly different ways of
perceiving the world. I perceive it very abstractly... [Emphasis added, S. L.] even in my dreams,
spaces have little detail unless I focus on it. People have amorphous faces,
perspective is malleable. Only in locations where I have a strong visual
reference (such as the view from my porch) is there significant visual detail
in my dreams.
Distance is similarly ill defined in my dreams... in fact, it is common to hear
descriptions of dreams have extremely ill defined spatial relationships. People
suddenly being beside you, or being unable to ever reach an objective despite
walking forever. I don't think these are anomalies... I believe this is due to
the ill-defined way in which we perceive distances. In most of my interaction
with the world, objects are clumped into five categories of distance relative
to my focus: far behind, behind, next to, in front of, and very near. These
definitions are fuzzy, but unless I'm looking at a room and attempting to
measure it in my mind, there is no precise spatial entity in my mind. A room
could be a sphere for all I notice, with me standing on the bottom, all points
equidistant. Only if I look at and focus on a corner does that corner become
'farther away' than the walls, or 'behind' the chair.
…[See full debate]…
I think that our visual perception is purely two dimensional, and only our
mental focus (attention, stream of consciousness, active symbols) changes
that two dimensional video stream into a volumetric mental model. [Emphasis added, S. L.] When not in my
mental focus, the illusion of a triangle is not present (referring to the three
circles/triangle illusion you use frequently), it is merely three black blobs.
Even if I focus on something IN the triangle... for example, a picture of a
bicycle... the triangle is not present unless I focus on the cutouts on the
circle. That triangle is a creation of my attention.
…[ see full debate ]…
I don't doubt that we CAN observe the world and create a volumetric representation in our minds, to a certain extent. I doubt that we DO, as a normal matter of course. Perhaps you do, having practiced it for months trying to figure out how you perceive the world... but I don't think that is the normal course of events for most people.
To: Myrddin Emrys
date Dec 7, 2006 6:20 AM
subject Re: Epistemology of Observation
>>
I think
that our visual perception is purely two dimensional, and only our mental focus
(attention, stream of consciousness, active symbols) changes that two
dimensional video stream into a volumetric mental model.
<<
Wow!!! You really believe this? You see the desk in front of you, your hands on the keyboard, your body on the chair below you, all as a 2-D projection?
And I suppose when you view a "kinetic depth effect" stimulus, all you see is a bunch of dots moving on the flat screen?
http://www.lifesci.sussex.ac
And all you see in this stimulus is a bunch of flat ellipses in rotation?
http://www.michaelbach.de/ot
And in this stimulus all you see is a bunch of shaded patches on a flat screen?
http://www.owlnet.rice.edu/~psyc351/Images/DepthFromShading.jpg
I'm sorry, but I am forced to conclude that your observations of your experience are fatally tainted by your theoretical preconceptions. Three-dimensional spatial perception is so basic and primary, to deny it is like denying that the world itself is three dimensional!
How would you even *know* that the world had three dimensions if your brain did not automatically and instinctively see the world as such? Why would you not believe that the world itself was 2-D?
I'm sorry, but if you tell me that your visual experience of the world is of a flat two-dimensional projection, then we simply have no common basis from which to debate. You and I inhabit profoundly different universes!
Steve
To: Steven Lehar
date Dec 7, 2006 12:25 PM
subject Re: Epistemology of Observation
>>>>
>>
I think that our visual perception is
purely two dimensional, and only our mental focus (attention, stream of
consciousness, active symbols) changes that two dimensional video stream into a
volumetric mental model.
<<
Wow!!! You really believe this? You see
the desk in front of you, your hands on the keyboard, your body on the chair
below you, all as a 2-D projection?
>>
And I suppose when you view a
"kinetic depth effect" stimulus, all you see is a bunch of dots
moving on the flat screen?
http://www.lifesci.sussex.ac
<<
<<<<
You mistake me... I said that only when I focus on an object, look at it directly,
does it obtain a three dimensional aspect. In other words, when I don't pay any
attention to those dots, and move them from my focal point to just off, and pay
attention to the words on that page... they have no dimensionality. They are a
wash of moving dots in the near periphery of my vision, and have no inherent
dimensionality.
Only when I LOOK at them, or focus my attention on them (even if they are not
in the focal point of my eye) does the wash of pixels obtain a three
dimensional aspect.
>>
And all you see in this stimulus is a
bunch of flat ellipses in rotation?
http://www.michaelbach.de/ot
And in this stimulus all you see is a
bunch of shaded patches on a flat screen?
http://www.owlnet.rice.edu/~psyc351/Images/DepthFromShading.jpg
<<
Of course I see these visual illusions, and experience the false sense of depth they
engender. But only when I focus on them. [Emphasis
added, S. L.] Try
this... with the middle illusion you chose, look off to the side, do NOT focus
your attention on the moving circles (that means not just a lack of eye focus,
but a lack of mental attention) and tell me if there is any sense of
dimensionality to the moving blob. For me, there is none at all.
From
my experience of the world, dimensionality is a result of attention. [Emphasis added, S. L.] Not necessarily of focusing
my eye on something, but merely of my attention, the movement of my thread of
consciousness to examine sensory input. Now, that thread of thought flits
around my visual input constantly... I see movement out of the corner of my
eye, and I think about it briefly, identify it as the head of a coworker
walking past my row of cubes, and the dimensionality of the input is
identified. This flit of thought ricochets around constantly, as fast as... er,
thought. But input to which I give no thought is mere input, flat shapes, until
it is identified at least semi-consciously.
…[ see full text ]…
And I completely agree that my experience of the world is different from
yours. I think that you have yet to admit that yours is different from MANY
people, and perhaps your insight into cognition is not applicable to everyone.
[Emphasis added, S. L.] We do not all think alike,
at least, not with our conscious minds. Our reflexive thought is mostly
instinctual, so in that we are probably all nearly identical.
Myrddin
To: Myrddin Emrys
date Dec 8, 2006 2:15 PM
subject Re: Epistemology of Observation
>>
For me, the moving dots become nothing but
a moving blur when I'm forcing my attention to ignore them.
<<
Yes,
me too! And for me, the moving dots disappear altogether when I close my eyes.
I don't care how vision works when you *don't* pay attention to it. That is as interesting
as how a computer works when the power is shut off! What is interesting is how
spatial vision works when you DO attend to it, with eyes wide open, and you see
a volumetric spatial experience for all the world like a 3-D model.
If
you *don't* see a volumetric spatial experience from the moving dots in
the kinetic depth effect stimulus, then I have nothing more to say to you.
My theory only addresses how vision works in people who DO see a spatial world in their visual
experience.
Steve
To: Steven Lehar
date Dec 9, 2006 9:36 AM
subject Re: Epistemology of Observation
I guess my point is that I don't 'see' a volumetric
whole. My mind processes several visual stimuli a second (unless I'm
daydreaming of course) which are held in memory, per my own introspection, in a
very similar manner to one of the theories you dismissed in your powerpoint
presentation. [Emphasis added, S. L.]
You discuss an end
table in a corner, and how we do not perceive it as some kind of semantic
network but as a volumetric whole.
I DO perceive it as a symbolic web... overlaid on a
two dimensional video stream (my eyes). [Emphasis
added, S. L.]
The persistent dimensionality of the world is an illusion that my mind creates,
in the same manner that my mind helps me not notice my optic blind spot.
Dimensionality is gained via attention, conscious thought parsing my optic
input. This dimensionality is only there as long as I pay attention to or
remember it... my mind does not 'retain' dimensionality in any kind of
volumetric way. The volume I perceive is almost purely logical... ie, 'that is
in front of that, I know that balls are round' rather than voxel (a mental
model of the object(s) held within my awareness in its entirety), and though I
maintain a visual reference to items that I don't pay attention to, their
dimensionality is lost.
In other words, in my experience my vision is processed in a massively parallel,
streaming, reflexive stream... but the dimensionality of symbolic content of my
world is created by the linear, single tasking process of my consciousness.
I wanted to email because your theory does not explain how I perceive the
world... its descriptions are not accurate in that respect. Much of what
you describe makes perfect sense, but the mental voxel model is completely
alien to my way of perceiving the world. [Emphasis
added, S. L.] I cannot vouch for anyone else... perhaps
other people do use mental voxels. I'm certain I do not. My world is not a
variable scale bubble, it's a video (and audio, and tactile, etc) stream with
symbolic content overlaid. My vision is processed in parallel... three
dimensional edges have a near 'luminance' to them that attracts the attention
(as my vision uses parallax and stereoscopic cues to identify edges), but they
are still two dimensional until my conscious mind notes them and places them
within the world. Dimensionality is symbolc, not volumetric, for me.
>>
Yes, me too! And for me, the moving dots
disappear altogether when I close my eyes. I don't care how vision works when
you *don't* pay attention to it. That is as interesting as how a computer works
when the power is shut off! What is interesting is how spatial vision works
when you DO attend to it, with eyes wide open, and you see a volumetric spatial
experience for all the world like a 3-D model.
<<
That doesn't make sense. On one hand you describe your perceptions of the world
as a kind of volumetric mental model processed from our senses. On the other
hand, my vision is processed primarily in a two dimensional way with a
symbolic dimensionality. This seems to be to be a relatively clear data point outside your theory... or at least
a type of visual processing you don't discuss. If anything that does not fit
within your theory is uninteresting to you, then doesn't that make you out to
be a kook like others have labeled you? [Emphasis
added, S. L.]
>>
If you *don't* see a volumetric spatial experience
from the moving dots in the kinetic depth effect stimulus, then I have
nothing more to say to you. My theory only addresses how vision works in people
who DO see a spatial world in
their visual experience.
<<
Except that you claim your model of vision describes how humans process vision.
So far, all you have possibly proven is that it is how you process vision. Data
to the contrary should be valuable, so you can refine your ideas.
In fact, I've never read anything that gives a serious discussion to the
possibility that different people have vastly different ways of cognating about
the world. I think there is a wider variety of 'thought' than most AI
researchers believe.
Myrddin
To: Myrddin Emrys
date Dec 9, 2006 12:40 PM
subject Re: Epistemology of Observation
Phenomenolgy
is often criticized as being too subjective to be scientific, and this
disagreement of ours, about how we experience vision, would appear to support
that criticism. But scientific phenomenology is always contingent on the reader
confirming the phenomenology of the author. If you don't see the world as a
volumetric spatial structure, then go ahead and reject my conclusions that
follow from that initial observation. In a debate between opponents like you
and I, let the reader make their own judgment whether they see the world
as a volumetric spatial structure, or as a series of brief and fleeting
two-dimensional impressions.
This is not at all a "clear data point outside your theory" it is a difference in the initial observation on which any theory can be based. And it is bare-faced nonsense to suggest that *my* visual system operates by such different principles than the average person's that I see the world in more dimensions than they do! If I were the only one to see things in 3-D, then why is there so much interest in those Gestalt illusions where a vivid 3-D percept pops out mysteriously from what is clearly a 2-D stimulus? If *I* were the only one to see these things, how do you explain the popularity of these illusions in the literature and across the internet, and in theoretical papers about vision? They are popular because it is nothing short of *magical* how a vivid 3-D experience pops out of a 2-D stimulus, and it is that magical emergence, and how it is computed in the brain, that I wish to investigate.
You, on the other hand, seem to be dedicated to denying the very existence of the single most prominent and significant aspect of visual experience, which is that it is spatially structured. Well, even if you were right, and visual experience really is "a symbolic web... overlaid on a two dimensional video stream", then 1: How come that symbolic web overlaid on a 2-D image promotes the “false” impression of a vivid volumetric structure that is so striking in its volumetric realism that it is replicated in books and papers and web sites across the internet? How does the experience, which is supposedly experienced as a symbolic web, *seem* (at the same time, but only “falsely”) to be a vivid spatial structure? And 2: Even if you are right, and the experience *is* actually a symbolic web overlaid on a 2-D image, then what I am interested in modeling in the brain is how that symbolic-2D experience is made to *SEEM* (“falsely”) like a vivid 3-D structure. It is the magical process of that “false” SEEMing function that I wish to explain.
I don't deny that visual experience *can* be analyzed into components. I
too can separate out the 2-D component of my 3-D experience, and I can
analyze my experience into edges and colors and shapes and motions, as
individual impressions. In fact under meditation, or under the influence of
psychedelic substances, I have even seen the visual world fragment into a
chaotic jumble of disconnected pieces, like the shards of a broken mirror. There
is a condition known as visual agnosia ( http://en.wikipedia.org/wiki
It is
*easy* to explain fragmented vision, given the fragmented nature of the visual
cortex, and it is *easy* to explain 2-D vision given the 2-D nature of the
retina and the visual cortex. The real mystery of vision is how, given that 2-D
fragmented architecture, we get to have a vivid integrated 3-D experience. THAT
is the great challenge of understanding vision, and we can either face up to
the challenge, or we can give up, and simply deny that there is a 3-D
experience that requires explanation.
If vision
were as you propose, a series of brief visual impressions, then we would all
suffer from visual agnosia, and like agnosics, we would stumble around
half-blind, because…
Visual
agnosia is the absence, or failure, of the visual function whose existence you
so confidently deny.
It is not your vision, but your theory of vision that suffers from visual agnosia!
Steve
