Feature detection is a process by which the nervous system sorts or filters complex natural stimuli in order to extract behaviorally relevant cues that have a high probability of being associated with important objects or organisms in their environment, as opposed to irrelevant background or noise.

Feature detectors are individual neurons—or groups of neurons—in the brain which code for perceptually significant stimuli. Early in the sensory pathway feature detectors tend to have simple properties; later they become more and more complex as the features to which they respond become more and more specific.

For example, simple cells in the visual cortex of the domestic cat (Felis catus), respond to edges—a feature which is more likely to occur in objects and organisms in the environment.^[1] By contrast, the background of a natural visual environment tends to be noisy—emphasizing high spatial frequencies but lacking in extended edges. Responding selectively to an extended edge—either a bright line on a dark background, or the reverse—highlights objects that are near or very large. Edge detectors are useful to a cat, because edges do not occur often in the background "noise" of the visual environment, which is of little consequence to the animal.

Early in the history of sensory neurobiology, physiologists favored the idea that the nervous system detected specific features of stimuli, rather than faithfully copying the sensory world onto a sensory map in the brain. For example, they favored the idea that the visual system detects specific features of the visual world. This view contrasted with the metaphor that the retina acts like a camera and the brain acts like film that preserves all elements without making assumptions about what is important in the environment. It wasn't until the late 1950s that the feature detector hypothesis fully developed, and over the last fifty years, it has been the driving force behind most work on sensory systems.^[2]

Horace B. Barlow was one of the first investigators to use the concept of the feature detector to relate the receptive field of a neuron to a specific animal behavior. In 1953, H.B. Barlow's electrophysiological recordings from excised retina of the frog provided the first evidence for the presence of an inhibitory surround in the receptive field of a frog's retinal ganglion cell. In reference to "on-off" ganglion cells—which respond to both the transition from light to dark and the transition from dark to light—and also had very restricted receptive fields of visual angle (about the size of a fly at the distance that the frog could strike), Barlow stated, "It is difficult to avoid the conclusion that the 'on-off' units are matched to the stimulus and act as fly detectors".^[3] In the same year, Stephen Kuffler published in vivo evidence for an excitatory center, inhibitory surround architecture in the ganglion cells of the mammalian retina which further supported Barlow's suggestion that on-off units can code for behaviorally relevant events.^[4]

Barlow's idea that certain cells in the retina could act as "feature detectors" was influenced by E.D. Adrian and Nikolaas Tinbergen.^[2] E.D. Adrian, Barlow's advisor, was the discoverer of the frequency code—the observation that sensory nerves convey signal intensity though the frequency of their firing.^[5] On the other hand, during Barlow's career, Nikolaas Tinbergen was introducing the concept of the innate release mechanism (IRM) and sign stimulus. IRMs are hard wired mechanisms that give an animal the innate ability to recognize complex stimuli. The sign stimulus is a simple, reduced stimulus including only the necessary features of the stimulus capable of evoking a behavioral response. Tinbergen's examination of the pecking behavior in herring gull chicks illustrated that the pecking response could be evoked by any bill-shaped long rod with a red spot near the end. In his own paper, Barlow later compared a sign stimulus to a password which was either accepted or rejected by a feature detector. Accepted passwords would contain the features necessary to trigger specific behavioral responses in an animal.^[6]

In the late 1950s, Jerome Lettvin and his colleagues began to expand the feature detection hypothesis and clarify the relationship between single neurons and sensory perception.^[1] In their paper "What the Frog's Eye Tells the Frog's Brain", Lettvin et al. (1959) looked beyond the mechanisms for signal-noise discrimination in the frog's retina and were able to identify four classes of ganglion cells in the frog retina: sustained contrast detectors, net convexity detectors (or bug detectors), moving edge detectors, and net dimming detectors.

In the same year, David Hubel and Torsten Wiesel began investigating properties of neurons in the visual cortex of cats, processing in the mammalian visual system. In their first paper in 1959,^[7] Hubel and Wiesel took recordings from single cells in the striate cortex of lightly anesthetized cats. The retinas of the cats were stimulated either individually or simultaneously with spots of light of various sizes and shapes. From the analysis of these recordings, Hubel and Wiesel identified orientation-selective cells in the cat's visual cortex and generated a map of the receptive field of cortical cells. At the time, circular spots of light were used as stimuli in studies of the visual cortex.^[4] However, Hubel and Wiesel noticed that rectangular bars of light were more effective stimuli (i.e. more natural stimuli) than circular spots of light, as long as the orientation was adjusted to the correct angle appropriate for each ganglion cell. These so-called simple cells were later called bar detectors or edge detectors. While comparing the receptive fields of neurons in the cat striate cortex with the concentric "on" and "off" receptive fields identified in cat ganglion cells by Kuffler et al., Hubel and Wiesel noticed that, although "on" and "off" regions were present in the striate cortex, they were not arranged in concentric circles. From their discovery of these uniquely orienting receptive fields, Hubel and Wiesel concluded that orientation-selective cells exist within the cat's visual cortex.

In their second major paper,^[8] Hubel and Wiesel extended their technique to more complex regions in the visual cortex in an effort to understand the difference between cortical receptive fields and lateral geniculate fields. They observed that the cat striate cortex contained more cells than the lateral geniculate, and they reasoned that the cortex needs a large number of neurons to digest the large amount of information it receives. Through experimentation, they found that each neuron in the cortex is responsible for a small region of the visual field and also has its own orientation specificity. From the results of these single cell readings in the striate cortex and lateral geniculate, Hubel and Wiesel postulated that simple cortical receptive fields gain complexity and an intricate spatial arrangement through the patterned convergence of multiple "on" or "off" projections from lateral geniculate cells onto single cortical cells.

Hubel and Wiesel's investigation of the cat visual cortex sparked interest in the feature detection hypothesis and its relevance to other sensory systems.^[9] In fact, T.H. Bullock contended in 1961 that the vestibular system was being ignored by most of the contemporary sensory system research, and he suggested that the equivalent stimulation of vestibular organs may yield similarly intriguing results.^[10] Hubel and Wiesel's work also raised the question: How far does the hierarchy of visual processing go? In one answer to this question, Lettvin coined the term grandmother cells in 1969 to describe hypothetical cells that are so specific that they only fire when your grandmother's face is viewed.^[11]

Jörg-Peter Ewert pioneered the study of feature detection in toad vision. He made significant progress by taking advantage of the common toad's natural prey catching behavior.^[12] To study toad behavior, he placed the toad in a cylindrical glass container at a fixed distance from the stimulus. Ewert then rotated a rectangular moving bar around the container in an effort to mimic a worm-like prey object; see video. The toad's rate of turning was used to quantify the toad's orienting behavior.

Ewert showed, by using spots, bars, and square stimuli of different sizes, that toads snapped at a moving bar which was moving in a direction parallel to its long axis, whereas the same bar oriented perpendicularly to the direction of movement (anti-worm configuration) was ignored as prey. Another experimental setup allowed worm or anti-worm stimuli to traverse the toad's visual field in different direction in the x-y co-ordinates, demonstrating that the worm vs anti-worm discrimination is invariant under changes in the direction of movement.^[13] He also showed that the toad would crouch and go immobile in response to a large rectangle. Using worm and anti-worm stimuli like these, Ewert identified that the prey-recognition system in the visual pathway of the toad is based on a number of innate release mechanisms. In response to a worm-like moving stimulus, the toad exhibited the following behaviors: orienting, snapping, or mouth wiping. On the other hand, an anti-worm stimulus evoked a different set of avoidance behaviors: planting down or crouching. After determining the sensory recognition elements of each behavior with this experimental setup, Ewert and co-workers examined the neural mechanisms governing the toad's prey-recognition system and found a number of feature detectors.

Having already used electrical point stimulation to identify the optic tectum as the region responsible for the prey-catching behaviors,^[14] Ewert and colleagues located and recorded from individual prey-selective neurons of the optic tectum in freely moving toads.^[15] These T5.2 neurons would increase in discharge frequency prior to a snapping or orienting behavior response. In the case of the snapping behavior, the neurons would stop firing for the duration of the snap. Evidently, these neurons exhibit a preference in responses to the worm configuration of moving bar stimuli and can therefore be considered feature detectors. To get a general idea of their properties, in successive experiments various rectangular dark objects of different edge lengths traverse a toad's visual field against a bright background at constant velocity; then the discharge frequency of a T5.2 neuron towards such an object is correlated with the toad's promptness of responding with prey-capture, expressed by the response latency. Thus, prey feature detection is not an all-or-nothing condition, but rather a matter of degree: the greater an object's releasing value as a prey stimulus, the stronger is prey-selective T5.2 neuron's discharge frequency, the shorter is toad's prey-catching response latency, and the higher is the number of prey-catching responses during a period of time (prey-catching activity)—as well as vice versa.

Multiple unit recordings showed that a prey object activates several adjacent prey-selective neurons whose receptive fields partly overlap. This suggests that more than one prey detecting cell—i.e., an ensemble of cells—will fire in the releasing system of prey-capture.

Further comparisons between the receptive fields of tectal neurons and retinal ganglion cells, classes R2 and R3, recorded in free-moving toads, revealed that size-sensitive (T5.1) and prey-selective (T5.2) tectal neurons were able to estimate the absolute size of a moving stimulus while retinal ganglion cells were only able to determine the visual angular size of the stimulus. Other selective neurons observed in the optic tectum include widefield arousal neurons, binocular neurons, and approach-sensitive neurons.

As explained above, the optic tectum is responsible for the orienting and snapping responses in prey-catching behavior; however, Ewert's research also revealed that focal electrical stimulation of the thalamic and pretectal brain regions evoked different kinds of avoidance behaviors in the toad.^[16] Feature detectors were also observed in these brain regions. Neurons were observed to be either directionally-sensitive to looming large objects, size-selective or perceptive to stationary obstacles. Certain pretectal thalamic neurons, type TH3, showed a preference for big moving objects and the anti-worm configuration of moving bar stimuli.^[17] It is suggested that the selectivity of tectal prey feature detectors, type T5.2, is determined by inhibitory influences of pretectal anti-worm detectors of the type TH3. Pretectal lesions impaired the prey-selectivity.^[15]

Axons from the feature sensitive/selective neurons of the optic tectum and thalamic-pretectal region then contact motor structures in the medulla oblongata,^[18][19] thus forming a sensorimotor interface. According to Ewert, this sensorimotor interface may serve as the "releaser" which recognizes sensory signals with assemblies of complex feature detectors and executes the corresponding motor responses.^[20] Having analyzed neuronal processing streams in brainstem structures that translate visual sign stimuli into behavioral responses, Ewert and coworkers discovered neural loops that—in connection with different forebrain structures—modulate, e.g. modify or specify, this translation.^[21]

Weakly electric fish generate electric fields for the purpose of object location and communication. They have a specialized electric sense made up of tuberous and ampullary electroreceptors located over the skin surface and innervated by the electrosensory lateral line. Just like in the visual system of toads, the electrosensory system of weakly electric fish extracts features from behaviorally relevant stimuli and uses these representations to perform further processing.^[22]

In the gymnotiform fish, Eigenmannia illustrated here, the primary sensory neurons in the electroreceptor system are simple feature detectors, and they include the ampullary receptors, probability coders (P units), and phase coders (T units). P units and T units are meant to acquire information about the amplitude and phase of the stimulus, respectively, with very little processing. The P and T units differ in tuning and in threshold for evoking a single spike in response to a sinusoidal stimulus. P units have high threshold and are broadly tuned; T units have low thresholds and narrow tuning. The separate processing of information continues passed the primary sensory neurons into the electrosensory lateral-line lobe (ELL) where spherical cells relay phase or time information to higher centers and pyramidal cells code for amplitude information. As a result, we also consider the class of spherical and pyramidal cells located in the ELL to be feature detectors. More specifically, pyramidal cells are considered feature detectors that respond to the amplitude of the stimulus. One class of pyramidal cell, E-cells, respond to increases; a second, I-cells, respond to decreases in stimulus amplitude whereas all peripheral receptors are E-units.

Beyond pyramidal cells and spherical cells, a more complex feature detector exists in the dorsal torus semicurcularis of the midbrain because the separate streams of amplitude and phase information converge on higher order sign-selective neurons in this region of the midbrain.^[23] These sign-selective neurons are considered feature detectors because they fire only upon the recognition of either a positive frequency difference between a jamming signal and the fish's own signal or a negative frequency difference. Afferents from these two types of sign-selective neurons then converge at the top of the neuronal hierarchy—the pre-pacemaker nucleus, which helps to regulate the discharge frequency of the electric organ in the jamming avoidance response.

In the auditory system of bats, like in auditory systems of other vertebrates, primary sensory afferent neurons, which receive inputs from hair cells from a restricted region of the organ of Corti in the cochlea, are the simple feature detectors. These structures are sensitive to a restricted range of frequencies and therefore function as tuned filters. Experimentally, Nobuo Suga and his colleagues (1990) noted that various constant frequency (CF) and frequency modulated (FM) harmonics excited different parts of the basilar membrane because of the frequency difference in the call.^[24] Auditory nerve fibers take this slightly-processed sensory information to the cochlear nucleus where information either converges or diverges into parallel pathways.^[25] In Pteronotus parnellii, a CF-FM bat, these parallel pathways process CF and FM harmonics separately and contain neurons that exhibit amplitude, frequency, and harmonic selectivity. These pathways converge in the medial geniculate body—giving rise to more complex feature detectors that respond to specific combinations of CF and FM signals.

In FM-FM regions of the auditory cortex, Suga et al. (1993) identified combination-sensitive neurons which receive inputs from multiple sources. Suga observed that the FM-FM region selectively responded to an FM component (feature) in the call and in the echo.^[26] More specifically, an individual FM1-FM2 unit requires an input from a unit tuned to the FM1 frequency range and a second unit tuned to the FM2 frequency range in order to fire. These FM-FM neurons can be considered complex feature detectors because they are sensitive to a particular frequency combination and a specific time delay between the echo and call. An accurate determination of the time delay between the call and echo is critical because it allows the bat to measure the distance between itself and its prey. This FM-FM sensitive region is only one example of a feature detector in the bat auditory cortex. A CF-CF sensitive region also exists in the auditory cortex, which in combination with FM-FM regions allows the bat to create maps for relative target velocity and target distance. Tracing the responses of these combination-sensitive neurons to higher order regions of the auditory pathway reveals that there are neurons with even higher levels of frequency and amplitude selectivity.

• Neuroethology
• Pattern recognition (psychology)
• Vision in toads

• Video: "Image Processing in the Visual System of the Common Toad – Behavior, Brain Function, Artificial Neuronal Net"