« Back to Table of Content || Next »

Chapter III. The Neurological Basis of Conscious Experience >> Summary of Chapter III

Summary of Chapter III

In this chapter, we have painted the big picture of neuronal activity underlying the workings of the mind. We have defined the fundamental concept of a neuronal ensemble. The neuronal ensemble is a group of neurons with enhanced connections that enable them to self-organize into synchronous activity. Neuronal ensembles represent physical objects in the cerebral cortex. They consist of tens of thousands of neurons. A neuronal ensemble can be visualized as a pyramid. The majority of neurons (the base of the pyramid) are non object-specific. These neurons are physically located in the less specific cortical areas such as the primary visual cortex (V1). Each of these neurons can be part of many neuronal ensembles. A minority of neurons (the tip of the pyramid) are object-specific. These neurons are physically located in more specific cortical areas such as the temporal lobe (TL). Neurons in the tip of the pyramid belong to only a few neuronal ensembles. Neurons in the middle of the pyramid are more specific than neurons in the base and less specific than neurons at the tip of the pyramid. These neurons are physically located in the extrastriate visual cortex such as V2 and V4 and can represent simple geometric shapes.

A neuronal ensemble can be visualized as a pyramid.

Because neurons at the tip of the pyramid are specific, stimulation of a few of these neurons can trigger activity of a complete neuronal ensemble. On the contrary, many more neurons have to be stimulated at the base of the pyramid to trigger activity of the complete neuronal ensemble. The former type of stimulation likely occurs during mental imagery (one can remember a leopard). The latter type of stimulation occurs during visual recognition of physical objects (one can recognize a leopard). The end result of both types of stimulation is activation of the complete neuronal ensemble representing the leopard.

A neuronal ensemble can represent a small element of an object (an eye or a tail) or a complete frame of a conscious experience (a leopard attacking a tribesman). Neuronal ensembles representing small elements of objects are more characteristic for the left analytical hemisphere. Neuronal ensembles representing complete frames of a conscious experience are more characteristic for the right holistic hemisphere.

Neuronal ensembles are not limited to representing visual information; they can represent information coming from all senses. Sometimes an olfactory stimulation can trigger a clear memory of grandma in an old apartment talking to you in a soft voice. In this case, neuronal ensembles synchronously firing in your brain include visual neurons, olfactory neurons, and auditory neurons; the neurons bound together by an experience that occurred many years ago have been re-activated to let you re-experience that moment. This multimodal experience can be visualized as a 3-dimensional pyramid, each face of the pyramid corresponding to one of the senses.

Schematic representation of a neuronal ensemble encoding information coming from several senses: each face of the pyramid corresponds to one of the senses.

Activation of a neuronal ensemble is essential for recognition and recall of an object or an event. In fact, recognition and recall can be defined as self-organization of a neuronal ensemble associated with the object or event into a synchronously firing unit. When the self-organization is triggered from the base of the pyramid by a few visual cues, a sound, or a familiar smell, we call the process recognition (bottom-up recognition initiated by firing of neurons at the bottom, or base of the pyramid). When the self-organization is triggered from the top (tip of the pyramid) by associative thinking or active planning executed by the prefrontal cortex, we call the process recall (top-down recall initiated by firing of neurons at the tip of the pyramid). As expected, damage to the prefrontal cortex disproportionately impairs visual recall (subjects are asked What they remember?) relative to recognition memory (subjects are asked What is shown on the picture?) (Petrides, 1994; Lee, 2000). Further, in normal subjects, the prefrontal cortex is activated significantly more in tasks that involve visual recall relative to tasks that involve only visual recognition (Christoff, 2000).

Visual recognition versus visual recall. Visual recognition can be schematically represented as bottom-up driven self-organization of the neuronal ensemble pyramid. Visual recall can be schematically represented as top-down driven self-organization of the neuronal ensemble pyramid.

As this book was compiled for publication, scientists reported a direct observation of the thesis put forth above. Researchers led by Hagar Gelbard-Sagiv and Itzhak Fried recorded from hundreds of neurons in and around the hippocampus, a group of specialized neurons located deep inside the temporal lobe, of 13 epilepsy patients (Gelbard-Sagiv, 2008). (The patients were undergoing operations to treat epilepsy. In the course of the operation, surgeons had to introduce electrodes into the brain in order to locate the source of the seizures.) The patients were shown a number of short (10 second) television episodes consisting of audiovisual sequences. The first time the patients saw the episodes, some of the neurons were selectively activated only by certain episodes and not by others. For example, some neurons were activated by a clip from The Simpsons but not by a clip of Michael Jordan. In other words, the neuronal ensemble representing The Simpsons was different than the one representing Michael Jordan and the scientists were able to record from the single neurons that were part of one neuronal ensemble pyramid and not the other.

Note that during these operations, only local anesthesia was used. No general anesthesia is needed because the brain does not have pain receptors. Subjects remain conscious, completely aware of the environment. Accordingly, most patients were able to easily remember nearly all of the clips and were able to verbally describe the content of the clips. Several minutes after showing the episodes, the researchers asked the patients to recall the clips as they came to mind. The sequence of recalled episodes was random, freely chosen by the patient. Scientists continued to record the neuronal activity during the free recall. The neuronal activity in the temporal lobe was able to predict which clip the patient would talk about next. The neurons active during The Simpsons clip were reactivated a second or two before the verbal recall of The Simpsons clip. The neurons active during the Michael Jordan episode were reactivated a second or two before the verbal recall of the Michael Jordan episode. In other words, the scientists observed the formation of a mental image represented by a neuronal ensemble corresponding to an episode and a few seconds later listened to the patient describing the content of that mental image. Note that scientists recorded from only a few neurons located in one site in the temporal lobe. However, I am confident to predict that video clip specific neurons exist at multiple cortical locations (neurons that are part of The Simpsons neural ensemble pyramid or part of the Michael Jordan pyramid, but not part of both pyramids, as in the illustration below). In the future, scientists will undoubtedly locate and record from these neurons.

The patients were shown a number of short television episodes consisting of audiovisual sequences. The first time the patients saw the episodes, some of the neurons in the hippocampus were selectively activated only by certain episodes and not by others. For example, some neurons were activated by a clip from The Simpsons but not by a clip of Michael Jordan. In other words, the neuronal ensemble representing The Simpsons was different than the one representing Michael Jordan and the scientists were able to record from the single neurons that were part of one neuronal ensemble pyramid and not the other. Several minutes after showing the episodes, the researchers asked the patients to recall the clips as they came to mind. The sequence of recalled episodes was random, freely chosen by the patient. The neuronal activity in the temporal lobe was able to predict which clip the patient would talk about next. The neurons active during The Simpsons clip were reactivated a second or two before the verbal recall of The Simpsons clip. The neurons active during the Michael Jordan episode were reactivated a second or two before the verbal recall of the Michael Jordan episode. In other words, the scientists observed the formation of a mental image represented by a neuronal ensemble corresponding to an episode and a few seconds later listened to the patient describing the content of that mental image.

One interesting question is whether recall is possible without the activation of a complete neuronal ensemble. Specifically, when one is asked to remember a scene and to describe it verbally, does one always activate a complete neuronal ensemble pyramid comprised of temporal lobe, V4, V2, and V1 neurons? We have discussed that the tip of the pyramid is absolutely essential for recall of a long-term memory. We have also presented experimental evidence that neurons in V1 (at the base of the pyramid) are often activated in visual recall. The question is: are they always activated? Can subjects describe a recalled image without activating neurons in V1? It is possible that they can. In other words, it is possible that the base of the pyramid is not always activated in the recall process. It is likely that during recall, each pyramid is activated only as much as is needed for a complete representation of an experience stored in memory. Consider the following analogy to the construction of a building. A building can be built from bricks, one brick at a time, or it can be constructed from large blocks. The building corresponds to a neural ensemble representing a conscious experience. The large blocks correspond to neurons in V2 and V4 that represent simple shapes, while bricks correspond to neurons in V1. As long as the goal of a complete functional building is achieved, it does not matter how the building was assembled. Therefore, it is possible to faithfully represent the experience with only the tip (TL) and the middle part of the pyramid (simple shapes, V2 and V4). However, when one needs to mentally assess distances (for example, to determine if the sofa in a store will fit between the TV and the fireplace in the living room), one must activate the complete pyramid (including neurons in V1). It should be noted that the visual system may be unique in its reliance on the primary cortical area. Other sensory systems may not activate the corresponding primary cortical areas during recall, saving energy (after all, every action potential is associated with a significant energy expenditure) and allowing the primary cortical areas to concentrate on processing incoming sensory information. For example, when subjects recall a sound stimulus in their mind, little activation is observed in the primary audio cortex. Thus, a more realistic (albeit more complex) representation of a neural ensemble activated during recall can be provided by a pyramid with a floating base, which descends to include a greater number of less specific neurons when the mind focuses on finer details of the mental image.

During memory recall, a neuronal ensemble pyramid may not activate completely. It is reasonable to assume that partial neuronal ensemble activation (limited to the temporal lobe and some extrastriate cortical neurons - solid lines) is sufficient for a faithful representation of the experience. When the mind focuses on finer details of the mental image, the pyramid can include a greater number of less specific neurons (complete neuronal ensemble pyramid - dashed lines.

Finally, perceptional modalities encoded by a neuronal ensemble are not limited to visual, auditory, and olfactory senses. Some events may include tactile stimulation, burning sensation, physical pain, strong taste, or loss of balance. A memory of an event can be charged emotionally with fear, novelty, or euphoria. All the multimodal information regarding this past event is stored in memory encoded by connections between neurons that are located in distinct regions of the brain. When the event is recalled, the neurons encoding multiple modalities must activate together in a synchronous firing to “play back” this past experience. Therefore the neuronal ensemble encoding a multimodal event can be better represented not by a pyramid, but by a polyhedron, each face of which represents one perceptual modality.

Neuronal ensembles firing in-phase with the attention rhythm are experienced consciously. However, many neuronal ensembles are activated out-of-phase with the attention rhythm and therefore are not experienced consciously. In fact, the greater part of brain activity occurs automatically, hidden from conscious awareness. We talk about these processes as being subconscious or non-conscious. Driving on a familiar road, typing, knitting, playing tennis: all of these activities can be performed automatically. While driving, the red light will be recognized and the foot will step on the brakes. When the process occurs automatically, the mind does not register the event. This means that the neuronal ensemble representing the red light was activated. The synchronous firing of that ensemble activated the automated procedure (encoded by neurons in the motor cortex, subcortical regions, and cerebellum) of stepping on the brakes. When these processes occur out-of-phase with the attention rhythm they are not experienced consciously. However, the mind can, at any moment, direct its attention to any part of the experience by bringing the neural ensemble representing that experience in-phase with the attention rhythm. This creates an illusion of a continuous awareness of all experiences, but this is just an illusion. Try to attend to every move while you are typing or playing tennis: your typing will become slower and you will lose the tennis match. Many automated actions are performed better when left alone and not attended to by the consciousness. In fact, most actions, even complex ones such as responding verbally to a question and reading aloud can be conducted automatically (as discussed earlier). The only process that cannot be conducted automatically is that of mental synthesis.

Mental synthesis is the process of synchronizing two or more neuronal ensembles with the attention rhythm. Mental synthesis enables humans to synthesize new, never-before-seen objects and experiences. It forms the basis for planning, solving problems in the mind, and actively teaching others. It is a uniquely human trait acquired in several steps over the last five million years of hominid evolution. All living animals, including nonhuman primates, likely lack the neuronal mechanism for voluntary synchronization of multiple independent neuronal ensembles with the attention rhythm.

In the beginning of this book you imagined your favorite cup standing on top of your favorite keyboard. Let us examine the formation of the cup on top of the keyboard image in your mind:
(1) Conscious frame 1: the neuronal ensemble representing the cup is activated and synchronized with the attention rhythm. The mental image of the cup is experienced.
(2) Conscious frame 2 (60 milliseconds later): the neuronal ensemble representing the keyboard is activated and synchronized with the attention rhythm. The mental image of the keyboard is experienced.
(3) Conscious frame 3 (120 milliseconds later): the neuronal ensembles representing the cup and the keyboard are synchronized with the attention rhythm. The mental image of the cup standing on top of the keyboard is experienced. Neurons in the prefrontal cortex (frontal lobe, abbreviated FL) execute the synchronization.
(4) The connections between the activated neurons are strengthened and, thus, the new neuronal ensemble representing the cup on top of the keyboard is formed (black pyramid). This ensemble will be stored in memory and can be activated at a later time as one unit.

It is likely that, in humans, the prefrontal cortex executes mental synthesis by activating neurons in the temporal lobe that, in turn, trigger self-activation of ensembles of neurons distributed throughout the cerebral cortex. (You can visualize the frontal cortex as a puppeteer using strings to control its puppets - neuronal ensembles.) Although both hemispheres have the neural machinery for execution of mental synthesis, normally the process is under control of the dominant left hemisphere.

In every conscious frame, our conscious experience (visual, auditory, etc.) is defined by the cortical neuronal ensemble activated in that frame. The subjective reality expressed in the activity of a neuronal ensemble can be as strong for imagined objects as for perception of physical objects. Thus the neuronal ensemble concept transcends the boundary of the physical world. It connects the imaginary and the physical worlds. In other words, mental images are not ephemeral, but rather they are physical objects represented by a synchronous firing of neuronal ensembles that can be measured and studied scientifically. Moreover, neuronal ensembles processed subconsciously out-of-phase with the attention rhythm are also real even though they cannot be described by the subject. They too can be studied scientifically.