Visual illusions and perceptual grouping phenomena offer an invaluable tool for probing the computational mechanism of low-level visual processing. Some illusions, like the Kanizsa figure, reveal illusory contours that form collinear with the inducing stimulus edges. This kind of illusory contour has been modeled by neural network models by way of cells equipped with elongated spatial receptive fields designed to detect and complete the collinear alignment. There are however other illusory grouping which are not so easy to account for in neural network terms. The Ehrenstein illusion exhibits an illusory contour that forms orthogonal to the stimulus edge instead of collinear with it. Other perceptual grouping effects reveal illusory contours that exhibit a sharp corner or vertex, and still others take the form of vertices defined by the intersection of three, four, or more illusory contours that meet at a point. A direct extension of the collinear completion models to account for these phenomena tends toward a combinatorial explosion, because it would suggest cells with specialized receptive fields configured to perform each of those completion types, each of which would have to be replicated at every location and every orientation across the visual field. These phenomena therefore challenge the adequacy of the neural network approach to account for these diverse perceptual phenomena.
I have proposed elsewhere an alternative paradigm of neurocomputation in the Harmonic Resonance theory (Lehar 1999), whereby pattern recognition and completion are performed by spatial standing waves across the neural substrate. The standing waves perform a computational function analogous to that of the spatial receptive fields of the neural network approach, except that unlike that paradigm, a single resonance mechanism performs a function equivalent to a whole array of spatial receptive fields of different spatial configuration and of different orientations, and thereby avoids the combinatorial explosion inherent in the older paradigm. The present paper presents the Directional Harmonic model, a more specific development of the Harmonic Resonance theory designed to account for specific perceptual grouping phenomena. Computer simulations of the Directional Harmonic model show that it can account for both collinear contours as observed in the Kanizsa figure, orthogonal contours as seen in the Ehrenstein illusion, and a number of illusory vertex percepts composed of two, three, or more illusory contours that meet in a variety of configurations.
Visual illusions and perceptual grouping phenomena offer an invaluable tool for probing the computational mechanism of low-level visual processing. Some illusions, like the Kanizsa figure shown in figure 1 A, reveal a process of collinear illusory contour formation, whereby the illusory contour spans the gap between stimulus edges that are both parallel, and spatially aligned. This kind of illusory contour formation has been modeled by neural network models that make use of "cooperative cells" equipped with elongated spatial receptive fields designed to detect, and by top-down feedback, to complete the collinear alignment with a line of neural activation along the perceived illusory contour (Grossberg & Mingolla 1985, Zucker et al. 1989). There are however other illusory grouping which are not so easy to account for in neural network terms. For example the Ehrenstein illusion, shown in figure 1 B, exhibits an illusory contour that forms orthogonal to the stimulus line segments instead of collinear with them. Grossberg & Mingolla (1985) account for this kind of illusory contour with a competitive interaction between cooperative cells at any particular spatial location whose orientations differ by about 90 degrees. In other words, the system is hard-wired to perform both collinear and orthogonal completion. But even if this model successfully accounts for these phenomena (an assertion which is not entirely un-problematic) it is difficult to imagine how this concept could possibly generalize to perceptual completion through sharp angles or vertices, as seen for example in the dot-triangle of figure 1 C, for this case is exactly intermediate between collinear and orthogonal completion. But the problem is still more serious, because there are a number of illusory grouping phenomena which involve three, four, or more illusory contours that meet at a perceived vertex, as discussed below. Although these different forms of illusory contour formation exhibit distinct characteristics, there is compelling evidence that they are nevertheless different manifestations of the same underlying mechanism. For example Kanizsa (1987) observes that the collinear grouping percept due to a circle of dots gives way to a polygonal percept when the number of dots in the circle is reduced, as shown in figure 1 D. In other words, the collinear grouping contour passing through each dot gives way to a vertex grouping percept, as if the collinear contour kinks like a drinking straw that is bent beyond its elastic limit. The key factor here concerns the angular deviation of the illusory contour through each dot rather than the size of the circle of dots, the size of the circle being varied merely to maintain the same spacing between dots. This phenomenon appears to be related to a similar abrupt transition observed in the perception of curvature (Wilson & Richards 1989). This calls into question the adequacy of neural network models to account for these various perceptual grouping phenomena, because a neural network model to account for all of these diverse phenomena would suggest specialized receptive field configurations specialized to perform collinear, orthogonal, and sharp vertex completion, as well as for illusory vertices composed of three, four, or more illusory contours that meet at a point, each of which would require a set of hard-wired receptive fields of the appropriate configuration. In fact the spatial pattern of the neural receptive field is no different in principle from a template model, a concept whose limitations are well known.
(a) The Kanizsa illusory triangle demonstrates illusory contour formation collinear with the inducing stimulus edges. (b) The Ehrenstein illusion demonstrates illusory contour formation orthogonal to the inducing stimulus edges. (c) The dot-triangle demonstrates illusory contours that meet at a sharp corner or vertex. (d) A circle of dots becomes, with increased curvature, a polygon of dots with an illusory vertex at each dot location.
Elsewhere I have proposed a Harmonic Resonance Theory (Lehar 1994a, 1994b, 1999, 2002) as an alternative to the neural network paradigm. I propose that the spatial patterns of illusory grouping phenomena are mediated not by hard-wired spatial receptive fields, but by standing waves of electrochemical oscillations in the neural substrate. A standing wave offers a much more flexible and adaptive computational mechanism than the spatial receptive fields of neural network theory. Unlike a neural receptive field, a standing wave pattern is not hard-wired to a particular spatial configuration, but can adapt flexibly to the input stimulus like an elastic template that can kink and fold into a variety of different configurations, all by way of a single computational mechanism. The merits of the harmonic resonance paradigm as a principle of perceptual computation are particularly clear in the case of the more complex illusory grouping phenomena presented below.
Figure 2 A shows a pattern of dots that are grouped in pairs, i.e. an illusory grouping line is observed to connect the two dots of the pair as suggested schematically by the gray shading in the magnified depiction on the right of the figure. In other words each dot projects a single illusory contour extending out in one direction only. A different pattern is observed in figure 2 B which shows a collinear grouping of dots in columns, each dot being connected by an illusory grouping contour that extends out from that dot in opposite directions, as shown schematically to the right of the figure. Figure 2 C shows a hexagonal grouping pattern in which each dot defines the center of a three-way vertex, as suggested schematically to the right in the figure. Finally figure 2 D depicts a grid-like percept in which each dot defines the center of a four-way vertex, as suggested schematically to the right in the figure.
Amodal illusory contours, or grouping percepts, define vertices composed of (a) one, (b) two, (c) three, and (d) four intersecting illusory contours, as suggested schematically to the right, where the gray lines represent the perceived contours in the figures to the left.
What is interesting in these perceptual phenomena is not so much the perceived grouping that occurs between neighboring dots, i.e. a grouping by the Gestalt law of proximity, but there is a more subtle and complex inhibitory effect whereby a nearer grouping is seen to suppress a more distant grouping. For example the horizontal separation between columns of dots in figure 2 B is the same as that in the grid pattern of figure 2 D. But the closer vertical spacing within each column in figure 2 B appears to suppress the horizontal grouping between those dots. Similarly, the vertical and horizontal grid grouping percept of figure 2 D appears to suppress an equally valid diagonal dot grouping, because each dot is located at the intersection of two diagonal rows of dots as well as on vertical and horizontal columns and rows of dots. But since the vertical and horizontal grouping has a closer spacing than the diagonal grouping, the diagonal grouping percept is entirely suppressed in this dot pattern. Similarly the hexagonal grouping percept shown in figure 2 C suppresses an equally present vertical and horizontal grouping percept, because each dot is located on the intersection of a vertical column and a horizontal row of dots in the stimulus, although this pattern is not apparent in the grouping percept. These complex spatial interactions between different grouping patterns offer a detailed manifestation of the specific computational interactions in perception, that goes well beyond the simplistic collinear and orthogonal grouping phenomena which are the usual focus of psychophysical studies. It is this secondary subtle pattern of inhibitory effects which provide the principal evidence for the Directional Harmonic Model.
A similar parametric variation between different perceptual grouping patterns can be seen in patterns composed of line segments as shown in figure 3. For example the lines in figure 3 A group into columns by collinearity, i.e. the illusory contour forms parallel to the inducing line segments, as suggested schematically to the right in figure 3 A. With a closer horizontal spacing however the percept becomes one of an orthogonal grouping, as shown in figure 3 B, and as suggested schematically to the right in the figure. This orthogonal grouping is similar in principle to the Ehrenstein illusion of figure 3 A. Again it is interesting that the closer horizontal spacing seems to suppress the alternative vertical grouping percept, and vice-versa. A third diagonal grouping percept can also be obtained with the proper arrangement of line segments, as shown in figure 3 C, and as suggested schematically to the right in the figure. This percept is considerably less salient than the collinear and orthogonal grouping percepts, and is complicated by the fact that it is not entirely clear whether the grouping lines connect adjacent line endings directly, as suggested schematically to the right in the figure, or whether diagonal rows of line segments form intersecting diagonal "streets", i.e. with longer grouping lines that extend from the top of one line segment to the top of the next and on to the top of the next, rather than from the top of one line ending to the bottom of the next. This illusion is further complicated by the fact that the percept is somewhat bistable or rivalrous between a percept of parallel diagonal streets from lower left to upper right, in competition with diagonal streets from upper left to lower right. However there is clearly a diagonal component to the percept that is clearly distinct from the collinear and orthogonal percepts of figure 3 A and B, and this percept appears to involve a completion by illusory vertex formation with a "Y" vertex at the tip of each line segment.
Amodal illusory contours that form in (a) collinear, (b) orthogonal, and (c) diagonal configurations with respect to their inducing line segment stimuli, as suggested schematically to the right, where the gray lines represent the perceived contours in the figures to the left.
The grouping percepts in figures 2 and 3 are primarily of an amodal nature, although there is perhaps also a faint modal or surface brightness component to them. But the principal focus of the present analysis is on the pattern of amodal grouping observed in these stimuli, regardless of whether or not those amodal contours also promote a corresponding surface brightness percept. The shaded grouping lines shown schematically to the right in figures 3 and 6 therefore represent the amodal component of the grouping percept, as was the case in figure 1 D, and therefore these patterns of gray lines represent the amodal output image that should be produced by an adequate computational model of these perceptual grouping phenomena.
The most remarkable property of harmonic resonance is the sheer number of different unique patterns that can be obtained in even the simplest resonating system. A pioneering study of more complex standing wave patterns was presented by Chladni (1787) who demonstrated the resonant patterns produced by a vibrating steel plate. The technique introduced by Chladni was to sprinkle sand on top of the plate, and then to set the plate into vibration by bowing with a violin bow. The vibration of the plate causes the sand to dance about randomly except at the nodes of vibration where the sand accumulates, thereby revealing the spatial pattern of nodes. This technique was refined by Waller (1961) using a piece of dry ice pressed against the plate, where the escaping gas due to the sublimation of the ice sets the plate into resonance, resulting in a high pitched squeal as the plate vibrates. Figure 4 (adapted from Waller 1961 P. 69) shows some of the patterns that can be obtained by vibrating a square steel plate clamped at its midpoint. The lines in the figure represent the patterns of nodes obtained by vibration at various harmonic modes of the plate, each node forming the boundary between portions of the plate moving in opposite directions, i.e. during the first half-cycle, alternate segments deflect upwards while neighboring segments deflect downwards, and these motions reverse during the second half-cycle of the oscillation. The different patterns seen in Figure 4 can be obtained by touching the plate at a selected point while bowing at the periphery of the plate, which forms a node of oscillation at the damped location, as well as at the clamped center point of the plate. The plate emits an acoustical tone when bowed in this manner, and each of the patterns shown in figure 4 corresponds to a unique temporal frequency, or musical pitch, the lowest tones being produced by the patterns with fewer large segments shown at the upper-left of figure 4, while higher tones are produced by the higher harmonics depicted towards the lower right in the figure. The higher harmonics represent higher energies of vibration, and are achieved by damping closer to the central clamp point, as well as by more vigorous bowing.
Chladni figures for a square steel plate (adapted from Waller 1961) demonstrates the fantastic variety of standing wave patterns that can arise from a simple resonating system. A square steel plate is clamped at its midpoint and sprinkled with sand. It is then set into vibration either by bowing with a violin bow, or by pressing dry ice against it. The resultant standing wave patterns are revealed by the sand, that collects at the nodes of the oscillation where the vibration is minimal.
The utility of standing wave patterns as a representation of spatial form is demonstrated by the fact that nature makes use of a resonance representation in another unrelated aspect of biological function, that of embryological morphogenesis, or the development of spatial structure in the embryo. After the initial cell divisions following fertilization, the embryo develops into an ellipsoid of essentially undifferentiated tissue. Then, at some critical point a periodic banded pattern is seen to emerge as revealed by appropriate staining techniques, shown in figure 5 A. This pattern indicates an alternating pattern of concentration of morphogens, i.e. chemicals that permanently mark the underlying tissue for future development. This pattern is sustained despite the fact that the morphogens are free to diffuse through the embryo. The mechanism behind the emergence of this periodic pattern is a chemical harmonic resonance known as reaction diffusion (Turing 1952, Prigogine & Nicolis 1967, Winfree 1974, Welsh et al. 1983) in which a continuous circular chemical reaction produces periodic patterns of chemical concentration in a manner that is analogous to the periodic patterns of a resonating steel plate. The chemical harmonic resonance in the embryo can thereby define a spatial addressing scheme that identifies local cells in the embryonic tissue as belonging to one or another part of the global pattern in the embryo by way of the relative concentration of certain morphogens. The fact that nature employs a standing wave representation in this other unrelated biological function offers an existence proof that harmonic resonance both can and does serve as a spatial representation in biological systems, and that representation happens to exhibits the same holistic Gestalt properties that have been identified as prominent properties of perception and behavior.
(a) A periodic banded pattern revealed by chemical staining emerges in a developing embryo, due to a chemical harmonic resonance whose standing waves mark the embryonic tissue for future growth. (b) This chemical harmonic resonance has been identified as the mechanism behind the formation of patterns in animal skins, as well as for the periodicity of the vertibrae of vertibrates, the bilateral symmetry of the body plan, as well as the periodicity of the bones in the limbs and fingers. (c) Murray shows the connection between chemical and vibrational standing waves by replicating the patterns of leopard spots and zebra stripes in the standing wave resonances in a vibrating steel sheet cut in the form of an animal skin.
Oscillations and temporal resonances are familiar enough in neural systems and are observed at every scale, from long period circadian rhythms, to the medium period rhythmic movements of limbs, all the way to the very rapid rhythmic spiking of the single cell, or the synchronized spiking of groups of cells. Harmonic resonance is also observed in single-celled organisms like the paramecium in the rhythmic beating of flagella in synchronized travelling waves. Similar waves are observed in multicellular invertebrates, such as the synchronized wave-like swimming movements of the hydra and the jellyfish, whose decentralized nervous systems consist of a distributed network of largely undifferentiated cells. The muscle of the heart provides perhaps the clearest example of synchronized oscillation, for the individual cells of the cardiac muscle are each independent oscillators that pulse at their own rhythm when separated from the rest of the tissue in vitro. However when connected to other cells they synchronize with each other to define a single coupled oscillator. The fact that such unstructured neural architectures can give rise to such structured behavior suggests a level of computational organization below that of the switching and gating functions of the chemical synapse. The idea of oscillations in neural systems is not new. However the proposal advanced here is that nature makes use of such natural resonances not only to define rhythmic patterns in space and time, but also to define static spatial patterns in the form of electrical standing waves, for the purpose that is commonly ascribed to spatial receptive fields. There is plenty of neurophysiological evidence which has accumulated over the last few decades suggestive of harmonic resonances in the brain (Gerard & Libet 1940, Bremer 1953, Eckhorn et al. 1988, Nicolelis et al. 1995, Murthy & Fetz 1992, Sompolinsky et al. 1990, Hashemiyoon & Chapin 1993). However it has been hard to interpret the significance of that evidence in the absence of a paradigmatic framework to suggest what function that resonance might serve in perception. I will show that as a paradigm for defining spatial pattern, the standing wave offers a great deal more flexibility and adaptiveness to local conditions than the alternative receptive field model, and that a single resonating system can replace a whole array of hard-wired receptive fields in a conventional neural model.
One of the most interesting aspects of harmonic resonance as a representational principle in the brain is that it exhibits certain invariances which are also characteristic of perception (Lehar 1994a, 1994b, 1999, 2002). Figure 6 shows the Chladni figures for a circular steel plate. This system exhibits two kinds of periodicity, a radial periodicity in the form of concentric rings, and a circumferential or directional periodicity in the form of radial lines, and these two types of periodicity appear in a variety of combinations. However due to the circular symmetry of the plate, each of these patterns can actually appear on the plate at any orientation. This is a very powerful feature, because if a standing wave pattern does indeed function as a spatial template in the brain, then any one of these patterns of standing waves corresponds not only to a single template in an equivalent neural network model, but to a whole array of them, i.e. with each pattern replicated at every possible orientation. Given that all of the different patterns in figure 6 are produced by a single mechanism, this one circular plate, and its various standing wave patterns represents the computational equivalent of a whole array of different spatial templates in a neural network model, each one replicated at every possible orientation. It is this invariance feature of a harmonic resonance representation that offers an escape from the combinatorial problem inherent in the neural network paradigm. Furthermore, not only does the circular Chladni plate represent a whole array of equivalent neural receptive fields, but also the cooperative or competitive interactions between them, because the various harmonics of the plate interact with one another in lawful ways, and these interactions make specific predictions about the behavior of a harmonic resonance model in response to certain patterns of input.
Chladni figures for circular plate, sorted by number of [diameters, circles] in each pattern. These patterns can appear at any orientation on the plate. Each distinct pattern has a unique vibration frequency. The vibration frequency therefore offers a rotation invariant representation of the pattern present on the plate.
The phenomenon of harmonic resonance is immensely complex, involving parallel interactions in all directions simultaneously through a homogeneous continuum in a manner that defies complete mathematical characterization or accurate numerical simulation in all but its simplest aspects. That very complexity however is exactly why harmonic resonance holds such great potential as a principle of computation and representation in the brain. The focus of this paper will be restricted to a single aspect of harmonic resonance, i.e. the tendency for standing waves to form patterns of circumferential, or directional periodicity, like the patterns of radial node lines seen in figure 6, as suggested originally by Lehar (1994a, 1994b). For the dot grouping patterns presented in figure 2 suggest a periodic basis set of different vertex types, expressed in terms of directional periodicity, which are suggestive of these patterns of standing waves. As in the case of a Fourier representation, any pattern of vertices can be represented in a directional harmonic code to arbitrary precision, by the appropriate combinations of harmonic coefficients. However in a physical system, the higher order terms require higher vibrational energies, as is the case for the Chladni figures. A physical harmonic resonance representation would therefore necessarily be band-limited to the lower harmonics, with a cut-off at some highest harmonic of directional periodicity. This low-pass cut-off introduces a certain granularity or quantization in the representation, limiting the complexity of the kind of vertex completion patterns to some finite set of low-order primitives. In fact it is this granularity in the directional harmonic code which accounts for the geometric regularity of the illusory grouping percepts observed in the different dot grouping patterns, as will be shown below.
The dot grouping patterns observed in figure 2 can be explained by a system that promotes local standing wave patterns at every dot location in the feedback layer, in response to the pattern of influence felt from neighboring dots in adjacent regions. Initially, each dot stimulates a point of activation at the corresponding location in the feedback layer, and that activation propagates radially outward by passive diffusion in all directions from each point. The diffusing activation from neighboring points of activation in turn impinges back on the original point from different directions, and the reciprocal exchange of energy back and forth between these active points across the feedback layer promotes the emergence of a pattern of standing waves of directional harmonic resonance at each active point as described below. Although harmonic resonance is a dynamic process that proceeds to equilibrium, a simplified static model of the process is sufficient to account for many of the observed grouping effects. This is analogous to the heat equation which describes the dynamic propagation of heat along a conductor from a localized source. Given a regular rate of heat loss along the conductor, the heat equation can be solved at equilibrium to produce a declining temperature gradient along the conductor with distance from the localized source, as a static model of the equilibrium state of a dynamic process. Similarly, the pattern of activation in the feedback layer of the directional harmonic model due to the presence of stimulus dots is assumed to produce at equilibrium a static gradient of activation, declining outward from each stimulus point with a Gaussian profile, as a static approximation to the equilibrium state of a dynamic diffusion process. The patterns of standing waves of directional periodicity are then computed at each stimulus dot location in response to this static input field from adjacent dots as described below.
For clarity this calculation is divided in two stages, an input stage, and a resonance stage. The input signal at each dot location is a circular signal, somewhat like a trace on the scope of a radar that scans the horizon in a circular sweep, producing peaks in every direction in which other dots are detected from that location. For example figure 7 A shows a pattern of dots around a central dot (circled), and figure 7 B shows the circular input signal Iq at that central dot for every direction q from that dot, expressed in degrees clockwise from the vertical. The neighboring dot in the 12 o'clock direction in figure 7 A produces a peak in the input response at 12 o'clock, or 360 degrees, as shown in figure 7 B, and the other dots produce similar peaks in the corresponding directions. The magnitude of the input signal fades as a Gaussian function of radial distance r between points, due to the passive diffusion, and this fading is modeled by the Gaussian function
|
(EQ 1) |
where sr is the standard deviation of the radial Gaussian function. This spatial decay explains why the dot at the 9 o'clock direction in figure 7 A produces a smaller peak in the input signal in the 9 o'clock direction in figure 7 B, because of the larger distance to that dot. The larger dashed circle shown in figure 7 A depicts the radius corresponding to two standard deviations (2sr) of the radial Gaussian input function used in these simulations. If the activation due to a dot were confined to a singular point, the input peak due to that dot would actually appear as an impulse function, i.e. a singularity in orientation. But the passive diffusion through the feedback layer would be expected to spread that singular peak somewhat across neighboring directions. In the simulations therefore the input signal Iq was modulated by an angular Gaussian function of the form
|
(EQ 2) |
i.e. this is the same Gaussian function as used in the radial Gaussian term, except that it operates in the angular dimension, with a standard deviation sq of 2p x .05 or 18 degrees, in order to spread that impulse response into a more manageable peak of finite size, as seen in this plot. The equation for the input signal therefore due to one neighboring dot at a bearing of a degrees from the central dot is given by
|
(EQ 3) |
(EQ 2)
The input signal is additive in each direction, so the three dots shown in the 3 o'clock direction in figure 7 A together produce a stronger peak in figure 7 B than any one of them would produce by themselves. The three dots near the 6 o'clock direction on the other hand are each in slightly different directions, and therefore they produce three individual input peaks through the 6 o'clock direction as seen in figure 7 B. The full equation therefore for the input signal Iq due to a set of n neighboring dots di i=1...n, at distances ri, and at angular bearings of ai, is given by
|
(EQ 4) |
Although the input signal is plotted only for the central dot in figure 7 A, a similar input signal is computed at every dot location in the simulations presented below. This circular input signal is then used to compute the circular harmonic resonance response at each dot location, as described below.
Computer simulation of the circular input signal at a central dot location due to the presence of adjacent dots. (a) A pattern of dots around a central dot (circled). The larger dotted circle indicates a radial distance which is twice the standard deviation of the radial Gaussian term. This pattern of input dots produces (b) a circular input function at the central dot location, showing how each adjacent dot produces a positive peak in the corresponding direction, the magnitude of the peaks being modulated by the distance from the central dot.
Figure 8 A depicts a circular harmonic series of directional periodicity, whose nodes, or stationary points (depicted as radial lines) represent the various edges that meet at the vertex. The first harmonic of directional periodicity exhibits a single node extending outward from the center in one direction. This harmonic corresponds to an end-stop feature, or unilateral vertex, as seen in the dot grouping pattern of figure 2 A. The second harmonic exhibits two nodes separated 180 degrees, which corresponds to a collinear vertex, or collinear grouping percept, as seen in figure 2 B. The third harmonic represents a three-way or "Y" vertex composed of three edges that meet at 120 degrees as seen in figure 2 C, and the fourth harmonic represents a "+" or "X" vertex with edges separated by 90 degrees as seen in figure 2 D. There is also a zeroth harmonic, like the DC term in a Fourier code, which represents the energy across all directional frequencies simultaneously. The zeroth harmonic corresponds to a vertex composed of edges extending in all directions simultaneously, which, in the limit, is essentially equivalent to no edges at all. Figure 8 B plots the amplitude function Aq of each of these harmonics as a pattern of nodes and anti-nodes around the circle from q = 0 to 2p, i.e. the height of the plot represents the amplitude of the vibration as a function of angle through the circle, which is given by
|
(EQ 5) |
for harmonics h = 1 to 4. Actually this figure shows a double plot, with upper and lower traces showing the positive and negative of the amplitude, representing a vibration alternately upward and downward from zero, like the pattern of vibration of a guitar string, to emphasize that the phase of the vibration is irrelevant, what is significant is the pattern of nodes and anti-nodes. Since it is the nodes of the vibration which represent perceived edges or grouping percepts in the model, a more convenient form to express these waveforms mathematically is as nodal functions Nq which are given by
|
(EQ 6) |
as shown in figure 8 C. In other words these functions are computed by subtracting the wave forms in figure 8 B from unity, because this encodes the features, i.e. the edges, as positive values rather than as the absence of positive values.The positive peaks in these waveforms now represent the patterns of perceived edges or grouping percepts as shown in figure 8 A. An offset value c was added to these nodal functions in order to shift them half way into the negative region as shown in the plot, with the offset value chosen so as to make the nodal functions sum to zero, i.e to produce equal areas under positive and negative regions of the curve. This was done so that when used as convolution filters they do not impose a bias on the output. The normalized nodal functions are given by
|
(EQ 7) |
with c = 1.63662. Figure 8 D plots these nodal waveforms on a circular plot, whose outer ring represents the value +1, the inner ring represents the value -1, and the middle ring represents the value zero.
Directional Harmonic representation. (a) Various patterns of nodes on a circular plate corresponding to the different harmonics of directional periodicity of the plate. The black lines represent the nodes, or stationary points of the standing wave, which in turn correspond to various configurations of edges that meet at the center, to define a sequence of vertex types. (b) The amplitude function, or variation of the amplitude of vibration as a function of angle around the circular plate. (c) The nodal pattern, or ones- complement of the amplitude function, to produce positive peaks in place of the nodes seen in the amplitude function. (d) A circular plot of the nodal pattern, where the inner circle represents the value -1, the outer circle represents +1, and the middle circle represents zero.
The circular harmonic response Rhq to the input signal at each dot location is then computed by a circular convolution of the circular input signal Iq with each of these circular harmonic nodal filters Nhq in turn, for each harmonic h = 1 to 4, as given by
|
(EQ 8) |
where (q + r) is computed modulo 2p to wrap around the full circle. This produces a set of harmonic responses Rhq to the input, one for each harmonic h, each response being a circular function through q = 0 to 2p, which represents the response to the input for that particular harmonic. As is the case with any convolution, the magnitude of each directional harmonic response is a function of the similarity, or degree of match between the functional form of that particular harmonic and the pattern of the input. For example an input function I with a single peak extending in one direction will produce a strong response R1 to the first harmonic filter N1, and the peak of that response function will be aligned with the peak in the input, while an input function with two peaks separated by 180 degrees will produce a strong response R2 to the second harmonic filter N2, etc. The four harmonic responses interact with each other by constructive and destructive interference as calculated by summation, producing a total harmonic response Rq across all four harmonics which is computed as
|
(EQ 9) |
for h = 1 to 4. In other words, wherever positive peaks from different harmonics coincide, they summate by constructive interference to produce a larger positive peak, whereas positive and negative peaks from different harmonic responses cancel each other by destructive interference to produce the total or resultant harmonic response. It is this total response to all harmonics of directional periodicity which corresponds to the predicted perceptual grouping at each dot location in response to the presence of adjacent dots.
I will now present computer simulations of very simple dot stimuli composed of only two or three dots, to demonstrate how the harmonic response is calculated for these simple cases, before proceeding to more interesting cases involving more complex patterns of stimulus dots. Figure 9 demonstrates the computation of the circular harmonic response at a central dot location in response to a single neighboring dot in the 12 o'clock direction, as shown in figure 9 A. The input signal Iq due to that dot exhibits a single peak in the 12 o'clock direction, as shown in figure 9 B. Each of the filters Nhq of figure 8 D is convolved with the input signal Iq to produce the set of responses Rhq shown in figure 9 C as defined in equation 8. The first harmonic R1q (solid line plot) produces a positive response in the 12 o'clock direction, with a sharp positive peak at 0 degrees, and a broad negative trough through 180 degrees. The second harmonic R2q (dashed line plot) produces a double peaked response, with sharp positive peaks at 0 and 180 degrees, and broad negative troughs through 90 and 270 degrees. The third harmonic R3q (dash-dot line plot) produces a three-peaked response, with positive peaks at 0, 120, and 240 degrees, and negative troughs in between, and the forth harmonic R4q (dotted line plot) produces four positive peaks at 0, 90, 180, and 270 degrees, with negative troughs at the diagonals. The harmonic response to this single peaked input signal is similar to the impulse response, or point-spread function of the system, which is why the response of each harmonic to this particular input is virtually identical to the shape of the waveform of the harmonic itself. The total harmonic response to this input Rq is then calculated by summing all of the four harmonic responses at every direction as defined in equation 9, which results in the combined harmonic response shown in figure 9 D. The positive portion of this response (shaded) points upwards in the direction of the input dot, and this response in turn represents a grouping percept extending upward from the lower dot towards the upper dot. By symmetry, the harmonic response at the upper dot (not plotted) would be identical except rotated by 180 degrees, i.e. with a grouping percept extending downward towards the first dot. If the calculation were extended to include higher harmonics, the irregular response profile of figure 9 D would become progressively smoother in the negative portion and sharper in the positive peak, eventually matching the shape of the input function itself. In other words the irregular pattern of peaks in the negative range in figure 9 D reflect the quantization due to the fact that only the lowest four harmonics were computed in the simulation.
Harmonic response to a single vertically adjacent dot (a) is computed by a circular convolution of the circular input signal (b) with a set of circular nodal functions. This produces a set of circular harmonic response functions (c). The final perceptual grouping is computed as a sum of these response functions, as shown in (d), where the positive portion (shaded) represents the actual grouping percept.
Figure 10 introduces a spatial plotting convention to give a more intuitive depiction of the perceptual grouping predicted by the model, with input and harmonic response functions displayed for every dot in the stimulus rather than only for one central dot. Figure 10 A shows two vertically adjacent dots, as before. Figure 10 B shows the input signal at each dot, plotted as before, but this time overlaid on the spatial plot of that same input signal. In the spatial plot, the magnitude of the circular input signal at each dot is depicted as a grey shading extending radially outward from the dot, the darkness of the gray shading representing the magnitude of the input signal in each direction. The shading fades with distance from the dot by the same Gaussian function as that used in computing the input signal for that dot. For example the lower dot in figure 10 B has a strong peak in its input function at the 12 o'clock direction, and this is depicted in the spatial shading convention as a region of dark shading projecting from the dot in the 12 o'clock direction, and fading with distance from the dot. The upper dot exhibits a similar input signal projecting downwards in the 6 o'clock direction. Figure 10 D through F plot the same data as in figure 10 A through C, except this time showing only the spatial shaded plot without the circular plot overlay. The gray shading in the input plot of figure 10 B and E suggest the pattern of perceptual grouping which would be predicted by a grouping-by-proximity model, in which the strength of grouping between any pair of dots is a simple Gaussian function of the distance between them. Figure 10 C and F show the harmonic response function computed for each dot as above, and displayed with the spatial shading convention. Since it is only the positive values of the harmonic response function which correspond to predicted perceptual grouping, only positive values are plotted in the spatial plot. It is in this plot that the subtle and interesting predictions of the Directional Harmonic model manifest themselves, although in this simple case there is no significant difference between the prediction of the harmonic resonance model and a simple grouping by proximity model.
A spatial plotting convention to give a more intuitive depiction of the depicted perceptual grouping, showing the harmonic responses for all dots in the stimulus simultaneously. (a) A pattern of two adjacent dots. (b) The input signal at each dot location due to the presence of the other dot, produces a peak in the input plot in the direction of the other dot, and that peak is displayed both as a circular plot, and as a gray radial shading. (c) The harmonic response plotted for each dot in the stimulus, again plotted both as a circular plot, and as a gray shading in the direction of the positive peaks of the plots. In this simple case the harmonic response is very similar to the input signal. (d, e, and f) The same as plots (a, b, and c) except this time showing only the gray shading, without the circular plot overlay. The harmonic response shown in (f) represents the predicted grouping percept for this configuration of dots, in this case predicting a first harmonic, or "end stop" feature grouping at each dot location.
Figure 11 depicts a slightly more complex stimulus, with three dots in a vertical line. Figure 11 A depicts the stimulus dot pattern relative to the central dot (circled), and figure 11 B depicts the input response at the central dot. Figure 11 C depicts the response of the first four harmonics of directional periodicity to this input pattern. In this case the response is dominated by the second harmonic, with positive peaks in the 6 and 12 o'clock directions. There is a weaker response of the fourth harmonic, with positive peaks at 6 and 12 o'clock, but the absence of dots in the 3 and 9 o'clock directions keeps this response weaker than the second harmonic response. The third harmonic produces only a very weak response, because its positive and negative peaks are separated by 180 degrees, so when the positive lobe of the filter is aligned with one input peak, the negative lobe is aligned with the other input peak. The same is true also for the first harmonic, which also produces a very weak response. Figure 11 D depicts the total harmonic response, which produces positive peaks in the 6 and 12 o'clock directions due to both the second and fourth harmonics, and small peaks in the 3 and 9 o'clock directions due to the fourth harmonic, but since these peaks are opposed by negative peaks in the second harmonic, the total harmonic response remains negative in those directions. The grouping percept predicted by the directional harmonic model in response to this stimulus therefore is a second harmonic or collinear grouping, with grouping lines projecting upward and downward toward the adjacent dots. Figure 11 E, F, and G depict the spatial plot for this same stimulus. Note that the harmonic responses at the upper and lower dots are dominated by the first harmonic, i.e. with an illusory grouping line projecting downward and upward respectively toward the central dot, similar to the grouping seen in figure 10. Again, in this simple case the prediction of the directional harmonic model is not very different from the input signal itself, which is the kind of grouping which would be predicted by a simple grouping-by- proximity model.
(a through d) Computer simulation of the harmonic response of a dot flanked by two neighboring dots in a straight line. (a) the pattern of dots in the stimulus. (b) The input response at the central dot, showing peaks at 6 and 12 o'clock. (c) The individual harmonic responses to this input at the central dot location, showing a strong second harmonic response, and a weaker fourth harmonic response, and still weaker first and third harmonic responses. (d) The total harmonic response for this stimulus at the central dot location, showing positive peaks at 6 and 12 o'clock, dominated by the second harmonic or collinear grouping percept at the central dot. (e through g) A computer simulation of the grouping between three dots in a vertical column, showing (e) the input dot stimulus, (f) the input function and (g) the total harmonic response at each dot location using the spatial plotting convention. This simulation therefore predicts a collinear grouping through the middle of this column of dots, with end-stop groupings at the top and bottom dots.
Figure 12 shows the computer simulations for all of the dot grouping patterns of figure 2. The three columns in figure 12 represent the dot pattern used in the simulation, the input signal computed for each dot, and the directional harmonic response due to that input signal computed at every dot location. Figure 12 A shows the grouping between pairs of dots, which produces primarily a first harmonic response, as seen in figure 2 A. Figure 12 B shows the collinear grouping along vertical lines of dots, as seen in figure 2 B. This grouping is due to a second harmonic response at each dot location, although a lateral or fourth harmonic response is also in evidence. The terminal dots at the top and bottom of each column of dots exhibits a first harmonic response. Figure 12 C simulates the hexagonal grouping percept observed in figure 2 C, due to a third harmonic response at each dot location. It is in this more complex stimulus that the directional harmonic model demonstrates its predictive power. Prominently absent from the harmonic response are the vertical, horizontal, and diagonal grouping percepts that are in evidence in the input response at each dot location. Figure 12 D shows the four-way or orthogonal grouping percept corresponding to figure 2 D, due to a fourth harmonic response at each dot location. Again, prominently absent from the harmonic response is the diagonal grouping which is in evidence in the input signal for this stimulus.
Computer simulations of the four dot grouping phenomena shown in figure 2, showing for each dot pattern the stimulus configuration, the input signal at each dot location, and the harmonic response at each dot location using the spatial plotting convention. (a) The pairs of dots form end-stop grouping percepts to the adjacent dot. (b) The columns of dots promote a vertical collinear grouping along the columns. (c) The hexagonal dot grouping dominated by a third harmonic response at each dot location. (d) A grid-like percept due to dominance of the fourth harmonic grouping.
When a vertical column dot stimulus like that in figure 2 B is varied parametrically by shifting alternate rows of dots to the right, the perceptual experience due to that grid of dots exhibits characteristic transitions, sometimes abrupt, between different perceptual grouping patterns. Figure 13 shows these transitions as a spatial rather than a temporal sequence. The alternate rows of dots in figure 13 have been shifted by a shift value of zero on the left side of the figure (i.e. no shift at all) to a value of 0.5 on the right side of the figure, expressed in units of the horizontal dot spacing, i.e. a shift of 0.5 means that the alternate rows of dots have been shifted half way to the next adjacent column. The resulting perceptual experience can be categorized as a vertical column or linear grouping percept, as seen in figure 13 A, where the shift value is very small. This then gives way to a zig-zag grouping as seen in figure 13 B, in which each column is perceived to be composed of a series of sharp angles. As the shift value is further increased the percept becomes somewhat ambiguous, before finally settling into a more stable diagonal grouping, or cross-hatch pattern, as seen in figure 13 C. Figure 14 shows computer simulations of these various grouping patterns. The abrupt transitions between distinct grouping percepts seen in this figure reveal the influence of the directional harmonic resonance, because these transitions appear only in the harmonic response image, whereas the input signal exhibits only a gradual or continuous transition between these arrangements of the stimulus. The abruptness of these perceptual transitions therefore mark a significant difference between the predictions of the Directional Harmonic model and a simple grouping-by-proximity model.
rectangular dot grid pattern in which alternate rows are shifted to the right, where the amount of shift varies continuously from the left to the right side of the figure. Perceptually, this shifting segments the percept into three distinct regions, that exhibit a (a) vertical linear, (b) zig-zag, and (c) cross-hatch grouping percept as a function of shift.
Computer simulation of the three perceptual groupings observed in figure 15. (a) Collinear grouping of columns, dominated by the second harmonic grouping. (b) Wavy line grouping where each dot marks the center of a two-armed vertex. (c) Cross-hatch grouping in which every dot marks the center of a diagonal fourth harmonic or "X" grouping percept.
The different grouping patterns seen in figure 13 can be explained by the directional harmonic model as the successive dominance of different harmonics of the grouping mechanism at different shift values. Figure 15 shows a computer simulation of the directional harmonics at a central dot location between two adjacent dots, as the flanking dots are progressively deflected from a collinear configuration. In figure 15 A the flanking dots are deflected 15 degrees downwards from the horizontal, i.e. the three dots form an internal angle of 150 degrees. At this small angle of deflection the harmonic response is still dominated by the second harmonic, i.e. the dots are perceived to be in a collinear alignment. In figure 15 B the deflection has been increased to 30 degrees, so the internal angle between the dots is now 120 degrees. In this configuration the response is dominated by the third harmonic, i.e. the dots are perceived to form an obtuse angle. The third harmonic response has a third branch, besides the two aligned with the flanking dots, which suggests a tendency for a perceptual grouping line to emerge in that direction. However this tendency is balanced by the first harmonic, which exhibits a positive peak downwards, and a negative trough upwards in figure 15 B, as well as by a negative trough in the fourth harmonic, which is why the combined harmonic response shows no positive peak in the 12 o'clock direction. Figure 15 C shows the angle of deflection now increased to 45 degrees, so that the dots now define a right angled corner. This in turn promotes the fourth harmonic as the dominant response of the system. The fourth harmonic response exhibits two additional peaks besides the two aligned with the neighboring dots, which are almost positive in the combined harmonic response shown in figure 15 C. The presence of adjacent dots in exactly those directions in figure 14 C is enough to boost those peaks to positive values, resulting in a fourth harmonic or four-way grouping percept at each dot.
Simulation of kinking of a perceived line of dots with increased curvature, as seen in figure 2 c and d. (a) Three dots in a line with a slight downward deflection (dashed lines) promotes a predominantly second harmonic or collinear groupin gpercept. (b) three dots with a greater deflection produce harmonic responses in which the third harmonic response dominates. (c) When the angle between the dots approaches 90 degrees, the fourth harmonic dominates, with a tendency to form illusory grouping lines in all four directions of the fourth harmonic vertex.
The competition between different harmonics in response to various dot configurations also offers an explanation for the abrupt kinking of a line or circle of dots, as seen in figure 1 D, which is observed to occur just as the angle between three adjacent dots approaches 120 degrees, the angle which favors the third harmonic response. Again these quantized, or abrupt perceptual transitions in response to a continuous parametric variation of the stimulus are due to the loss of the higher harmonics due to impedance, which in turn results in the perceptual characterization of these stimuli in terms of various combinations of the lower order terms, which serve as a basis set of geometrical primitives for encoding the perceived forms.
rectangular dot grid in which alternate pairs of rows are shifted horizontally to produce (a) linear, (b) wavy line, and (c) hexagonal grouping percepts.
Figure 16 demonstrates a different parametric variation of a dot grid pattern, this time obtained by shifting alternate pairs of rows by a variable amount. This leads to distinct perceptual grouping patterns which can be categorized as linear, wavy, and hexagonal grouping percepts with progressively increasing shift value. Figure 17 shows how these patterns too can be explained by the directional harmonic model. The wavy pattern gives way to the hexagonal pattern at the point where the third leg of the third harmonic grouping percept at each apex (see in figure 17 B) comes into alignment with those on the adjacent column of dots, forming a bridge that spans the gap between the columns, resulting in the hexagonal grouping percept of figure 17 C.
Computer simulation of (a) the collinear grouping, (b) the wavy line percept, and (c ) the hexagonal grouping percept seen in figure 18.
In the dot grouping simulations presented above, the input to the system at each dot location was assumed to energize the directional harmonic resonance at that location with equal energy at all orientations. The final pattern of resonance therefore results exclusively from the configuration of neighboring dots, as communicated through the angular and radial Gaussian input functions. A line segment stimulus on the other hand provides an oriented input signal along the line segment, corresponding to a second harmonic or collinear grouping at every point along that line. This raw input is expected to overwhelm the resonance at every point along that line, resulting in a strong second harmonic response along the stimulus lines, corresponding to a collinear "grouping" percept, or the veridical percept of the stimulus line as a line of the appropriate orientation. The interesting grouping effects observed in the line segment stimuli are observed at the line endings, where each line ending behaves somewhat like a dot stimulus, except with an oriented bias, or strong oriented input in the direction of the line segment. The line pattern grouping simulations of the directional harmonic model are therefore performed similar to the dot pattern groupings, in that the harmonic resonance is computed only at the points where the line segments terminate, i.e. at the line endings, as shown in figure 18 A. The resonance at the line ending is therefore biased by a fixed oriented input signal from the line of which it is the terminus. For example the point at the top end of a vertical line segment is assumed to have a permanent input signal from the 6 o'clock direction, in addition to any other influences from adjacent line endings, whereas the bottom end of a vertical line segment has a permanent input from the 12 o'clock direction, as shown in figure 18 A. The model can also be used to simulate vertices, as shown in figure 18 B, in which case the input signal at the vertex is assumed to have permanent inputs from the directions of the component line segments of that vertex, as shown in figure 18 B. With this simple addition, the directional harmonic model now also accounts for the perceptual line segment grouping phenomena shown in figure 3.
(a) The directional harmonic simulations of line segment stimuli are performed similar to those of the dot patterns, with the harmonic response being calculated only at the location of the line endings. The input signal at each line ending is presumed to have a permanent input signal from the direction of the line of which that point is the terminus. For example the top end of a vertical line segment is presumed to have a permanent input signal from the six o'clock direction, while the bottom end would have a permanent input signal from the 12 o'clock direction as shown here. (b) In the case of multiple line segments, the harmonics are computed at each vertex, where a permanent input signal is presumed in the direction of each of the component line segments as shown here.
Figure 19 A through C shows the directional harmonic simulation for collinear, orthogonal, and diagonal grouping respectively of the line segment stimuli, as observed perceptually in figure 3 A through C. The collinear grouping percept shown in figure 3 A is explained by a second harmonic response at each line ending, as shown in figure 19 A. The significant difference between the resonance response and the input signal (equivalent to the prediction of a grouping-by-proximity model) is not so much the predominantly vertical grouping, which is observed in both responses, but in the suppression of the alternative horizontal and diagonal groupings observed in the input signal. The orthogonal grouping percept of figure 3 B is explained by a fourth harmonic grouping at each line ending in figure 19 B, with attenuated grouping percepts in the collinear and diagonal directions, whereas the diagonal grouping percept of figure 3 C is explained by a third harmonic or "Y"-vertex response at each line ending, as seen in figure 19 C, with a suppression of the horizontal, and extraneous diagonal groupings observed in the input signal for that stimulus.
Computer simulations of perceptual grouping between line segment stimuli, showing (a) collinear grouping, (b) orthogonal grouping, and (c) diagonal grouping percepts at each line ending.
Ever since Santiago Ramon y Cajal discovered the cellular basis of the nervous system, the idea of the nervous system as an assembly of quasi-independent processors, sometimes called the Neuron Doctrine (Barlow 1972, 1995), has become firmly established as the dominant paradigm of neurocomputation. Many different arrangements of these simple integrate-and-threshold elements have been tried in computer simulations in an attempt to coax some kind of interesting or suggestive behavior from them as an explanation for perceptual processing. But certain aspects of perceptual function have remained so elusive as to cast doubt on the entire enterprise. Particularly problematic have been the enigmatic properties of perception identified by Gestalt theory. The principle of invariance is ubiquitous in many aspects of perception, including invariance in the recognition of simple shapes to rotation, translation, and scale. And yet this kind of invariance is very difficult to achieve in conventional neural network models, for it leads to a combinatorial explosion in the required number of receptive fields. The principle of emergence, identified by Gestalt theory, is also problematic for neural network models, because it suggests a relaxation to equilibrium of a massively parallel dynamic system through many iterations, a concept which is difficult for the Neuron Doctrine given the time delays inherent in the chemical synapse. There is also a dimensional mismatch between the continuous field-like nature of perceptual experience identified by Gestalt theory, and the constellation of discrete activations of individual neurons in the Neuron Doctrine (Lehar 2002). I propose therefore to return all the way to the debate between Golgi and Cajal, and to reconsider the question of whether the essential principles of neurocomputation are best described as an atomistic system of simple quasi-independent local processors as suggested by neural network theory, or as a holistic continuous field- like principle as suggested by Gestalt theory. I don't propose to question the neurophysiological and histological evidence for the cellular basis of the nervous system, or for the properties of the chemical synapse. However I do contest the significance of that evidence for the essential principles of neurocomputation.
In fact the problems with the Neuron Doctrine date as far back as the development of the first brain recording technique, the electroencephalogram (EEG). From the outset the EEG detected a global oscillation across the whole brain, and that global synchrony was found to correlate with particular functional states. However the significance of this global resonance remains to this day as mysterious as it was when it was first discovered. Now with the introduction of multiple-unit recordings, this global resonance is even beginning to manifest itself in electrophysiological data. Although it is not much discussed in the contemporary literature, this kind of global resonance is problematic for the Neuron Doctrine. For the phasic spiking of a neuron is unlikely to survive across the chemical synapse, due to the multiple parallel paths in each axonal and dendritic arborization, each with variable path lengths and dendritic diameters which would introduce random time delays across all those parallel paths, not to mention the temporal averaging of the signal in each of the 10,000 or so parallel chemical synapses that connect a typical pair of neurons. After the third or fourth such transmission along a line of cells, the phasic pulses of the neural signal would surely diffuse into a featureless blur, which would preclude the kind of global synchrony observed in the EEG signal across the cortex as a whole.
Another Achilles heel in the Neuron Doctrine that receives little mention today, is the fact that the cell membrane, although capable of insulating electrical charge inside the cell, does not insulate alternating currents. For like the insulating dielectric in a capacitor, the cell wall transmits alternating current signals perfectly well, and blocks only direct current. That is why single-cell electrode recording can be performed extra-cellularly, just as well as intra-cellularly. This in turn suggests that the alternating current spiking signal of the neuron is likely to propagate directly from cell to cell across the extracellular matrix, unhindered by the insulation of the cell membrane. Pribram (1971) proposes that the overt spiking of the neuron is not the causal origin, but merely a secondary manifestation or response of the cell to a graded potential oscillation that pervades the body of neural tissue, because that oscillation has been observed to continue even after the spiking cell falls below threshold and ceases to fire. Like whitecaps on ocean waves, the spiking neuron merely reveals the presence of a more subtle underlying resonance that pervades the bulk neural tissue. The reason that this graded potential oscillation has received so little attention in electrophysiological circles has been for lack of a theoretical framework to give meaning to those resonances, or to explain how harmonic resonance can perform a computational function in the brain.
I propose that the global synchrony observed in the EEG recordings, and now in the synchronous activity of cortical neurons, is exactly what it appears to be, i.e. it is a manifestation of a global harmonic resonance that pervades the entire cortex, and that this resonance subserves a computational function which is central to the principle of operation of the brain (Lehar 1994a, 1994b, 1999, 2002). The purpose of this resonance is to set up a pattern of standing waves, and these standing waves serve a function which is normally ascribed to spatial receptive fields in neural network models, i.e. as a mechanism for encoding spatial patterns in the brain both for recognition and for perceptual completion.
The computational function performed by these standing waves is highlighted in the Directional Harmonic model by comparison with the functionally equivalent neural network models that it replaces. The second harmonic standing wave of directional periodicity mirrors the functionality of a collinear completion model as proposed by Grossberg & Mingolla (1985, 1987), Walters (1987), and Zucker et al. (1989), except that functionality no longer requires separate spatial receptive fields replicated at every location and every orientation across the visual field, but rather, the second harmonic resonance emerges spontaneously at the location and orientation determined by the stimulus. Furthermore, this same dynamic resonance also performs perceptual completion through a range of different vertex types corresponding to the various harmonics of directional periodicity, such as "Y" and "+" or "X" vertices, with no additional hardware required. But even that does not exhaust the computational repertoire of the Directional Harmonic model, for as in a Fourier representation, these harmonics are also used in combination to produce compound standing wave patterns corresponding to "L" and "T" and "V" vertices. This is functionally equivalent to a neural network model capable of merging different receptive field profiles into new compound receptive fields on the fly, and using those compound fields to perform completion of spatial patterns.
Finally, there is one further aspect of a harmonic resonance theory that deserves mention, and that is its relation to emergence and reification in Gestalt illusions. The neural network models of Grossberg & Mingolla (1985, 1987), and Zucker et al. (1989) incorporate an explicit top-down feedback from what I have called the "feedback layer" back down to the layer of local oriented edge detectors, so that the higher level groupings computed in the feedback layer are expressed back at the oriented edge layer as explicit edge percepts. This is done in order to account for the perceptual experience of a sharp high resolution contour in figures like the camo and the Kanizsa triangles, at a location where there is no edge present in the stimulus. Although this issue was not addressed explicitly in the Directional Harmonic model, it is in the very nature of harmonic resonances in adjacent sub-systems to tend to couple with each other by mutual reciprocal feedback to produce a single combined resonance. A resonance model therefore does not have to add top-down feedback of this sort as an explicit mechanism of the model, because it is already a natural property of the resonance itself. (Lehar 1999) In fact this natural tendency of individual local resonances to couple into a single global resonance is, I propose, the central operational principle behind the holistic global nature of Gestalt phenomena. In other words harmonic resonance offers a computational principle that exhibits the holistic global aspects of perception identified by Gestalt theory, not as specialized mechanisms or architectures contrived to achieve those properties, but as natural properties of the resonance itself. The principal value of the Directional Harmonic Theory therefore is not so much as evidence for some kind of resonance in the brain, for that evidence already exists from a wide variety of diverse sources. But rather the Directional Harmonic Theory offers a specific and detailed model of exactly how standing waves might perform a computational function in perception normally ascribed to spatial receptive fields, and demonstrates how that function circumvents some fundamental limitations inherent in the template-like concept of the neural receptive field, by an emergent holistic process consistent with Gestalt theory.
Barlow H. B. (1972) Single Neurons and Sensation: A neuron doctrine for perceptual psychology. Perception. Perception 1, 371-394.
Barlow H. B. (1995) The Neuron Doctrine in Perception. In M. Gazzaniga (Ed.), The Cognitive Neurosciences. Cambridge: MIT Press.
Bremer F. (1953) "Some Problems in Neurophysiology". London: Athlone Press.
Chladni, E. E. F. (1787) "Entdeckungen über die Theorie des Klanges". Leipzig: Breitkopf und Härtel.
Eckhorn R., Bauer R., Jordan W., Brosch M., Kruse W., Munk M., Reitboeck J. (1988) "Coherent Oscillations: A Mechanism of Feature Linking in the Visual Cortex?" Biol. Cybern. 60 121-130.
Gerard, R. W. & Libet B. (1940) "The Control of Normal and `Convulsive' Brain Potentials". Amer. J. Psychiat., 96, 1125-1153.
Grossberg, Stephen, & Mingolla, Ennio (1985). Neural Dynamics of Form Perception: Boundary Completion, Illusory Figures, and Neon Color Spreading. Psychological Review, 92, 173-211.
Hashemiyoon R. & Chapin J. (1993) "Retinally derived fast oscillations coding for global stimulus properties synchronise multiple visual system structures". Soc. Neurosci. Abstr. 19 528.
Kanizsa, Gaetano (1987). Quasi-Perceptual Margins in Homogeneously Stimulated Fields.. In The Perception of Illusory Contours, Petry S. & Meyer, G. E. (eds) Springer Verlag, New York, 40-49.
Lehar S. (1994a) "Directed Diffusion and Orientational Harmonics: Neural Network Models of Long-Range Boundary Completion through Short-Range Interactions". Ph.D. Thesis, Boston University. Available at http://cns-alumni.bu.edu/~slehar/webstuff/thesis/thesis.html
Lehar S. (1994b) "Harmonic Resonance in Visual Perception Suggests a Novel Form of Neural Communication. rejected Perception & Psychophysics 1995. Available at http://cns-alumni.bu.edu/~slehar/webstuff/pnp/pnp.html
Lehar S. (1999) "Harmonic Resonance Theory: an Alternative to the `Neuron Doctrine' Paradigm of Neurocom putation to Address Gestalt properties of perception." Rejected Psychological Review November 1999. Available at http://cns-alumni.bu.edu/~slehar/webstuff/hr1/hr1.html
Lehar S. (2002) "TheWorld In Your Head: A Gestalt view of the mechanism of conscious experience." Mahwah NJ: Erlbaum (in press). Information available at http://cns-alumni.bu.edu/~slehar/webstuff/book/WIYH.html
Murthy V., & Fetz E. (1992) "Coherent 25- to 35-Hz oscillations in the sensorimotor cortex of awake behaving monkeys". Proc. Natl. Acad. Sci. USA 89, 5670-5674.
Nicolelis M., Baccala L., Lin R., & Chapin J. (1995) "Sensorimotor encoding by synchronous neural ensemble activity at multiple levels of the somatosensory system". Science 268, 1353-1358.
Prigogine I. & Nicolis G. (1967) "On symmetry-breaking instabilities in dissipative systems". J. Chem. Phys. 46, 3542-3550/
Sompolinsky H., Golomb D., & Kleinfeld D., (1990) "Global processing of visual stimuli in a neural network of coupled oscillators". Proc. Natl. Acad. Sci. USA 87, 7200-7204.
Turing A. M. (1952) "The chemical basis of morphogenesis". Philos. Trans. R. Soc. London. B 237, 37-72.
Waller M. D. (1961) "Chladni Figures, A Study in Symmetry". London, G. Bell & Sons.
Welsh B. J., Gomatam J., & Burgess A. E. (1983) "Three-Dimensional Chemical Waves in the Belousov-Zhabotinski Reaction". Nature 304, 611-614.
Wilson, Hugh R. & Richards, Whitman A. (1989) Mechanism of Contour Curvature Discrimination. J. Opt. Soc. Am. A/ 6, 1.
Winfree A. T. (1974) "Rotating Chemical Reactions". Scientific American 230 (6) 82-95.
Zucker, Steven W., Dobbins, Allan, & Iverson, Lee (1989). Two Stages Of Curve Detection Suggest Two Styles Of Visual Computation. neural computation, 1, 68-81