### abstract ###
A prevailing theory proposes that the brain's two visual pathways, the ventral and dorsal, lead to differing visual processing and world representations for conscious perception than those for action.
Others have claimed that perception and action share much of their visual processing.
But which of these two neural architectures is favored by evolution?
Successful visual search is life-critical and here we investigate the evolution and optimality of neural mechanisms mediating perception and eye movement actions for visual search in natural images.
We implement an approximation to the ideal Bayesian searcher with two separate processing streams, one controlling the eye movements and the other stream determining the perceptual search decisions.
We virtually evolved the neural mechanisms of the searchers' two separate pathways built from linear combinations of primary visual cortex receptive fields by making the simulated individuals' probability of survival depend on the perceptual accuracy finding targets in cluttered backgrounds.
We find that for a variety of targets, backgrounds, and dependence of target detectability on retinal eccentricity, the mechanisms of the searchers' two processing streams converge to similar representations showing that mismatches in the mechanisms for perception and eye movements lead to suboptimal search.
Three exceptions which resulted in partial or no convergence were a case of an organism for which the targets are equally detectable across the retina, an organism with sufficient time to foveate all possible target locations, and a strict two-pathway model with no interconnections and differential pre-filtering based on parvocellular and magnocellular lateral geniculate cell properties.
Thus, similar neural mechanisms for perception and eye movement actions during search are optimal and should be expected from the effects of natural selection on an organism with limited time to search for food that is not equi-detectable across its retina and interconnected perception and action neural pathways.
### introduction ###
Neurophysiology studies of the macaque monkey CITATION CITATION support the existence of two functionally distinct neural pathways in the brain mediating the processing of visual information.
The behavior of patients with brain damage has led to the proposal that perception is mediated by the ventral stream projecting from the primary visual cortex to the inferior temporal cortex, and that action is mediated by the dorsal stream projecting from the primary visual cortex to the posterior parietal cortex CITATION CITATION.
Although there has been debate about whether this separation into ventral/dorsal streams implies that the brain contains two distinct neural representations of the visual world CITATION CITATION, there has been no formal theoretical analysis about the functional consequences of the two different neural architectures on an animal's survival.
Visual search requires animals to move their eyes to point the high-resolution region of the eye, the fovea, to potentially interesting regions of the scene to sub-serve perceptual decisions such as localizing food or a predator.
What is the impact of having similar versus different neural mechanisms guiding eye movements and mediating perceptual decisions on visual search performance for an organism with a foveated visual system?
We consider two leading computational models of multiple-fixation human visual search, the Bayesian ideal searcher CITATION CITATION and the ideal saccadic targeting model for a search task of a target in one of eight locations equidistant from initial fixation.
The ideal searcher uses knowledge of how the detectability of a target varies with retinal eccentricity and statistics of the scenes to move the fovea to spatial locations which maximize the accuracy of the perceptual decision at the end of search CITATION.
The saccadic targeting model makes eye movements to the most probable target location CITATION, CITATION which is optimal if the goal was to saccade to the target rather than collect information to optimize a subsequent perceptual decision CITATION.
Depending on the spatial layout of the possible target locations and the visibility map, the IS and MAP strategies lead to similar or diverging eye-fixations.
For example for a steeply varying visibility map both models make eye movements to the possible target locations while for a broader visibility map the ideal searcher tends to make eye movements in between the possible target locations attempting to obtain simultaneous close-to-fovea processing for more than one location.
Covert attention allows both models to select possible target locations and ignore locations that are unlikely to contain the target when deciding on saccade endpoints and making perceptual search decisions CITATION, CITATION.
Perceptual target localization decisions for both models are based on visual information collected in parallel over the whole retina, temporally integrated across saccades, and based on the location with highest sensory evidence for the presence of the target.
Critically, we implemented the models to have two processing pathways, one determining where to move the fovea and the other stream processing visual information to reach a final perceptual decision about the target location.
Rather than having a single linear mechanism or perceptual template, each pathway in the model had its own neural mechanism which is compared to the incoming visual data at each possible target location.
Likelihood ratios CITATION of the observed responses for each of the mechanisms under the hypothesis that the target is present or absent at that location are used to make decisions about where to move the eyes and perceptual decisions .
We used a genetic algorithm as a method to find near-optimal solutions for perception and action mechanisms but also to simulate the effects of the evolutionary process of natural selection on the neural mechanisms driving saccadic eye movements and perceptual decisions during search.
The computational complexity of the ideal Bayesian searcher makes it difficult to virtually evolve the model and thus we used a recently proposed approximation to the ideal searcher that is computationally faster.
The ELM model chooses the fixation location that minimizes the uncertainty of posterior probabilities over the potential target locations.
The decision rule can be simplified to choose the fixation location with the maximum sum of likelihood ratios across potential target locations, each weighted by its squared detectability given the fixation location CITATION.
The ELM model can be shown to approximate the fixation patterns of the ideal searcher CITATION and capture the main characteristics of the fixation patterns of the IS for our task and visibility maps.
The process of virtual evolution started with the creation of one thousand simulated individuals with separate linear mechanisms for perception and eye movement programming.
Each pathway's template for each individual was created from independent random combinations of the receptive fields of twenty four V1 simple cells.
Each simulated individual was allowed two eye movements before making a final perceptual search decision about the location of the target.
Performance finding the target in one of eight locations for five thousand test-images was evaluated and the probability of survival of an individual was proportional to its performance accuracy.
A new generation was then created from the surviving individuals through the process of reproduction, mutation and cross-over.
The process was repeated for up to 500 generations.
