Towards Computational Models and Applications of Insect Visual Systems for Motion Perception: A Review
---
paper_title: Fly visual course control: behaviour, algorithms and circuits

paper_content:

Understanding how the brain controls behaviour is undisputedly one of the grand goals of neuroscience research, and the pursuit of this goal has a long tradition in insect neuroscience. However, appropriate techniques were lacking for a long time. Recent advances in genetic and recording techniques now allow the participation of identified neurons in the execution of specific behaviours to be interrogated. By focusing on fly visual course control, I highlight what has been learned about the neuronal circuit modules that control visual guidance in Drosophila melanogaster through the use of these techniques.
---
paper_title: Towards Computational Models of Insect Motion Detectors for Robot Vision

paper_content:

In this essay, we provide a brief survey of computational models of insect motion detectors, and bio-robotic solutions to build fast and reliable motion-sensing systems for robot vision. Vision is an important sensing modality for autonomous robots, since it can extract abundant useful features from visually cluttered and dynamic environments. Fast development of computer vision technology facilitates the modeling of dynamic vision systems for mobile robots.
---
paper_title: Small Brains, Smart Machines: From Fly Vision to Robot Vision and Back Again

paper_content:

Neurobiological and neuroethological findings on insects can be used to design and construct small robots controlling their navigation on the basis of bio-inspired visual strategies and circuits. Animals' visual guidance is partly mediated by motion-sensitive neurons, which are responsible for gauging the optic flow. Although neurons of this kind were discovered in vertebrates' and invertebrates' visual systems more than 50 years ago, the principles and neural mechanisms involved have not yet been completely elucidated. Here, first, I propose to outline some of the findings we made during the last few decades by performing electrophysiological recordings on identified neurons in the housefly's eye while applying optical stimulation to identified photoreceptors. Whereas these findings shed light on the inner processing structure of an elementary motion detector (EMD), recent studies in which the latest genetic and neuroanatomical methods were applied to the fruitfly's visual system have identified some of the neurons in the visual chain which are possibly involved in the neural circuitry underlying a given EMD. Then, I will describe some of the proof-of-concept robots that we have developed on the basis of our biological findings. The 100-g robot OCTAVE, for example, is able to avoid the ground, react to wind, and land autonomously on a flat terrain without ever having to measure any state variables such as distances or speeds. The 100-g robots OSCAR 1 and OSCAR 2 inspired by the microscanner we discovered in the housefly's eye are able to stabilize their body using mainly visual means and track a moving edge with hyperacuity. These robots react to the optic flow, which is sensed by miniature optic flow sensors inspired by the housefly's EMDs. Constructing a “biorobot” gives us a unique opportunity of checking the soundness and robustness of a principle that is initially thought to be understood by bringing it face to face with the real physical world. Bio-inspired robotics not only help neurobiologists and neuroethologists to identify and investigate worthwhile problems in animals' sensory-motor systems, but they also provide engineers with ideas for developing novel devices and machines with promising future applications, in the field of smart autonomous vehicles and microvehicles, for example.
---
paper_title: Optic flow-based collision-free strategies: From insects to robots

paper_content:

Flying insects are able to fly smartly in an unpredictable environment. It has been found that flying insects have smart neurons inside their tiny brains that are sensitive to visual motion also called optic flow. Consequently, flying insects rely mainly on visual motion during their flight maneuvers such as: takeoff or landing, terrain following, tunnel crossing, lateral and frontal obstacle avoidance, and adjusting flight speed in a cluttered environment. Optic flow can be defined as the vector field of the apparent motion of objects, surfaces, and edges in a visual scene generated by the relative motion between an observer (an eye or a camera) and the scene. Translational optic flow is particularly interesting for short-range navigation because it depends on the ratio between (i) the relative linear speed of the visual scene with respect to the observer and (ii) the distance of the observer from obstacles in the surrounding environment without any direct measurement of either speed or distance. In flying insects, roll stabilization reflex and yaw saccades attenuate any rotation at the eye level in roll and yaw respectively (i.e. to cancel any rotational optic flow) in order to ensure pure translational optic flow between two successive saccades. Our survey focuses on feedback-loops which use the translational optic flow that insects employ for collision-free navigation. Optic flow is likely, over the next decade to be one of the most important visual cues that can explain flying insects' behaviors for short-range navigation maneuvers in complex tunnels. Conversely, the biorobotic approach can therefore help to develop innovative flight control systems for flying robots with the aim of mimicking flying insects' abilities and better understanding their flight.
---
paper_title: Seeing Things in Motion: Models, Circuits, and Mechanisms

paper_content:

Motion vision provides essential cues for navigation and course control as well as for mate, prey, or predator detection. Consequently, neurons responding to visual motion in a direction-selective way are found in almost all species that see. However, directional information is not explicitly encoded at the level of a single photoreceptor. Rather, it has to be computed from the spatio-temporal excitation level of at least two photoreceptors. How this computation is done and how this computation is implemented in terms of neural circuitry and membrane biophysics have remained the focus of intense research over many decades. Here, we review recent progress made in this area with an emphasis on insects and the vertebrate retina.
---
paper_title: Common circuit design in fly and mammalian motion vision

paper_content:

Motion-sensitive neurons have long been studied in both the mammalian retina and the insect optic lobe, yet striking similarities have become obvious only recently. Detailed studies at the circuit level revealed that, in both systems, (i) motion information is extracted from primary visual information in parallel ON and OFF pathways; (ii) in each pathway, the process of elementary motion detection involves the correlation of signals with different temporal dynamics; and (iii) primary motion information from both pathways converges at the next synapse, resulting in four groups of ON-OFF neurons, selective for the four cardinal directions. Given that the last common ancestor of insects and mammals lived about 550 million years ago, this general strategy seems to be a robust solution for how to compute the direction of visual motion with neural hardware.
---
paper_title: Synaptic Connections of First-Stage Visual Neurons in the Locust Schistocerca gregaria Extend Evolution of Tetrad Synapses Back 200 Million Years

paper_content:

The small size of some insects, and the crystalline regularity of their eyes, have made them ideal for large-scale reconstructions of visual circuits. In phylogenetically recent muscomorph flies, like Drosophila, precisely coordinated output to different motion-processing pathways is delivered by photoreceptors (R cells), targeting four different postsynaptic cells at each synapse (tetrad). Tetrads were linked to the evolution of aerial agility. To reconstruct circuits for vision in the larger brain of a locust, a phylogenetically old, flying insect, we adapted serial block-face scanning electron microscopy (SBEM). Locust lamina monopolar cells, L1 and L2, were the main targets of the R cell pathway, L1 and L2 each fed a different circuit, only L1 providing feedback onto R cells. Unexpectedly, 40% of all locust R cell synapses onto both L1 and L2 were tetrads, revealing the emergence of tetrads in an arthropod group present 200 million years before muscomorph flies appeared, coinciding with the early evolution of flight. J. Comp. Neurol. 523:298–312, 2015. © 2014 Wiley Periodicals, Inc.
---
paper_title: Escapes with and without preparation: the neuroethology of visual startle in locusts.

paper_content:

Locusts respond to the images of approaching (looming) objects with responses that include gliding while in flight and jumping while standing. For both of these responses there is good evidence that the DCMD neuron (descending contralateral movement detector), which carries spike trains from the brain to the thoracic ganglia, is involved. Sudden glides during flight, which cause a rapid loss of height, are last-chance manoeuvres without prior preparation. Jumps from standing require preparation over several tens of milliseconds because of the need to store muscle-derived energy in a catapult-like mechanism. Locusts' DCMD neurons respond selectively to looming stimuli, and make connections with some motor neurons and interneurons known to be involved in flying and jumping. For glides, a burst of high-frequency DCMD spikes is a key trigger. For jumping, a similar burst can influence timing, but neither the DCMD nor any other single interneuron has been shown to be essential for triggering any stage in preparation or take-off. Responses by the DCMD to looming stimuli can alter in different behavioural contexts: in a flying locust, arousal ensures a high level of both DCMD responsiveness and glide occurrence; and there are significant differences in DCMD activity between locusts in the gregarious and the solitarious phase.
---
paper_title: Visual processing in the central bee brain.

paper_content:

Visual scenes comprise enormous amounts of information from which nervous systems extract behaviorally relevant cues. In most model systems, little is known about the transformation of visual information as it occurs along visual pathways. We examined how visual information is transformed physiologically as it is communicated from the eye to higher-order brain centers using bumblebees, which are known for their visual capabilities. We recorded intracellularly in vivo from 30 neurons in the central bumblebee brain (the lateral protocerebrum) and compared these neurons to 132 neurons from more distal areas along the visual pathway, namely the medulla and the lobula. In these three brain regions (medulla, lobula, and central brain), we examined correlations between the neurons' branching patterns and their responses primarily to color, but also to motion stimuli. Visual neurons projecting to the anterior central brain were generally color sensitive, while neurons projecting to the posterior central brain were predominantly motion sensitive. The temporal response properties differed significantly between these areas, with an increase in spike time precision across trials and a decrease in average reliable spiking as visual information processing progressed from the periphery to the central brain. These data suggest that neurons along the visual pathway to the central brain not only are segregated with regard to the physical features of the stimuli (e.g., color and motion), but also differ in the way they encode stimuli, possibly to allow for efficient parallel processing to occur.
---
paper_title: Fly visual system inspired artificial neural network for collision detection

paper_content:

This work investigates one bio-inspired collision detection system based on fly visual neural structures, in which collision alarm is triggered if an approaching object in a direct collision course appears in the field of view of a camera or a robot, together with the relevant time region of collision. One such artificial system consists of one artificial fly visual neural network model and one collision detection mechanism. The former one is a computational model to capture membrane potentials produced by neurons. The latter one takes the outputs of the former one as its inputs, and executes three detection schemes: (i) identifying when a spike takes place through the membrane potentials and one threshold scheme; (ii) deciding the motion direction of a moving object by the Reichardt detector model; and (iii) sending collision alarms and collision regions. Experimentally, relying upon a series of video image sequences with different scenes, numerical results illustrated that the artificial system with some striking characteristics is a potentially alternative tool for collision detection.
---
paper_title: Fly visual course control: behaviour, algorithms and circuits

paper_content:

Understanding how the brain controls behaviour is undisputedly one of the grand goals of neuroscience research, and the pursuit of this goal has a long tradition in insect neuroscience. However, appropriate techniques were lacking for a long time. Recent advances in genetic and recording techniques now allow the participation of identified neurons in the execution of specific behaviours to be interrogated. By focusing on fly visual course control, I highlight what has been learned about the neuronal circuit modules that control visual guidance in Drosophila melanogaster through the use of these techniques.
---
paper_title: Spatial organization of visuomotor reflexes in Drosophila

paper_content:

In most animals, the visual system plays a central role in locomotor guidance. Here, we examined the functional organization of visuomotor reflexes in the fruit fly, Drosophila, using an electronic flight simulator. Flies exhibit powerful avoidance responses to visual expansion centered laterally. The amplitude of these expansion responses is three times larger than those generated by image rotation. Avoidance of a laterally positioned focus of expansion emerges from an inversion of the optomotor response when motion is restricted to the rear visual hemisphere. Furthermore, motion restricted to rear quarter-fields elicits turning responses that are independent of the direction of image motion about the animal's yaw axis. The spatial heterogeneity of visuomotor responses explains a seemingly peculiar behavior in which flies robustly fixate the contracting pole of a translating flow field.
---
paper_title: Predator versus Prey: Locust Looming-Detector Neuron and Behavioural Responses to Stimuli Representing Attacking Bird Predators

paper_content:

Many arthropods possess escape-triggering neural mechanisms that help them evade predators. These mechanisms are important neuroethological models, but they are rarely investigated using predator-like stimuli because there is often insufficient information on real predator attacks. Locusts possess uniquely identifiable visual neurons (the descending contralateral movement detectors, DCMDs) that are well-studied looming motion detectors. The DCMDs trigger ‘glides’ in flying locusts, which are hypothesised to be appropriate last-ditch responses to the looms of avian predators. To date it has not been possible to study glides in response to stimuli simulating bird attacks because such attacks have not been characterised. We analyse video of wild black kites attacking flying locusts, and estimate kite attack speeds of 10.8±1.4 m/s. We estimate that the loom of a kite’s thorax towards a locust at these speeds should be characterised by a relatively low ratio of half size to speed (l/|v|) in the range 4–17 ms. Peak DCMD spike rate and gliding response occurrence are known to increase as l/|v| decreases for simple looming shapes. Using simulated looming discs, we investigate these trends and show that both DCMD and behavioural responses are strong to stimuli with kite-like l/|v| ratios. Adding wings to looming discs to produce a more realistic stimulus shape did not disrupt the overall relationships of DCMD and gliding occurrence to stimulus l/|v|. However, adding wings to looming discs did slightly reduce high frequency DCMD spike rates in the final stages of object approach, and slightly delay glide initiation. Looming discs with or without wings triggered glides closer to the time of collision as l/|v| declined, and relatively infrequently before collision at very low l/|v|. However, the performance of this system is in line with expectations for a last-ditch escape response.
---
paper_title: Escape behaviors in insects

paper_content:

Escape behaviors are, by necessity, fast and robust, making them excellent systems with which to study the neural basis of behavior. This is especially true in insects, which have comparatively tractable nervous systems and members who are amenable to manipulation with genetic tools. Recent technical developments in high-speed video reveal that, despite their short duration, insect escape behaviors are more complex than previously appreciated. For example, before initiating an escape jump, a fly performs sophisticated posture and stimulus-dependent preparatory leg movements that enable it to jump away from a looming threat. This newfound flexibility raises the question of how the nervous system generates a behavior that is both rapid and flexible. Recordings from the cricket nervous system suggest that synchrony between the activity of specific interneuron pairs may provide a rapid cue for the cricket to detect the direction of an approaching predator and thus which direction it should run. Technical advances make possible wireless recording from neurons while locusts escape from a looming threat, enabling, for the first time, a direct correlation between the activity of multiple neurons and the time-course of an insect escape behavior.
---
paper_title: Precise Subcellular Input Retinotopy and Its Computational Consequences in an Identified Visual Interneuron

paper_content:

The Lobula Giant Movement Detector (LGMD) is a higher-order visual interneuron of Orthopteran insects that responds preferentially to objects approaching on a collision course. It receives excitatory input from an entire visual hemifield that anatomical evidence suggests is retinotopic. We show that this excitatory projection activates calcium-permeable nicotinic acetylcholine receptors. In vivo calcium imaging reveals that the excitatory projection preserves retinotopy down to the level of a single ommatidium. Examining the impact of retinotopy on the LGMD's computational properties, we show that sublinear synaptic summation can explain orientation preference in this cell. Exploring retinotopy's impact on directional selectivity leads us to infer that the excitatory input to the LGMD is intrinsically directionally selective. Our results show that precise retinotopy has implications for the dendritic integration of visual information in a single neuron.
---
paper_title: A visual motion detection circuit suggested by Drosophila connectomics

paper_content:

Animal behaviour arises from computations in neuronal circuits, but our understanding of these computations has been frustrated by the lack of detailed synaptic connection maps, or connectomes. For example, despite intensive investigations over half a century, the neuronal implementation of local motion detection in the insect visual system remains elusive. Here we develop a semi-automated pipeline using electron microscopy to reconstruct a connectome, containing 379 neurons and 8,637 chemical synaptic contacts, within the Drosophila optic medulla. By matching reconstructed neurons to examples from light microscopy, we assigned neurons to cell types and assembled a connectome of the repeating module of the medulla. Within this module, we identified cell types constituting a motion detection circuit, and showed that the connections onto individual motion-sensitive neurons in this circuit were consistent with their direction selectivity. Our results identify cellular targets for future functional investigations, and demonstrate that connectomes can provide key insights into neuronal computations.
---
paper_title: A universal strategy for visually guided landing

paper_content:

Landing is a challenging aspect of flight because, to land safely, speed must be decreased to a value close to zero at touchdown. The mechanisms by which animals achieve this remain unclear. When landing on horizontal surfaces, honey bees control their speed by holding constant the rate of front-to-back image motion (optic flow) generated by the surface as they reduce altitude. As inclination increases, however, this simple pattern of optic flow becomes increasingly complex. How do honey bees control speed when landing on surfaces that have different orientations? To answer this, we analyze the trajectories of honey bees landing on a vertical surface that produces various patterns of motion. We find that landing honey bees control their speed by holding the rate of expansion of the image constant. We then test and confirm this hypothesis rigorously by analyzing landings when the apparent rate of expansion generated by the surface is manipulated artificially. This strategy ensures that speed is reduced, gradually and automatically, as the surface is approached. We then develop a mathematical model of this strategy and show that it can effectively be used to guide smooth landings on surfaces of any orientation, including horizontal surfaces. This biological strategy for guiding landings does not require knowledge about either the distance to the surface or the speed at which it is approached. The simplicity and generality of this landing strategy suggests that it is likely to be exploited by other flying animals and makes it ideal for implementation in the guidance systems of flying robots.
---
paper_title: Background visual motion affects responses of an insect motion‐sensitive neuron to objects deviating from a collision course

paper_content:

Stimulus complexity affects the response of looming sensitive neurons in a variety of animal taxa. The Lobula Giant Movement Detector/Descending Contralateral Movement Detector (LGMD/DCMD) pathway is well‐characterized in the locust visual system. It responds to simple objects approaching on a direct collision course (i.e., looming) as well as complex motion defined by changes in stimulus velocity, trajectory, and transitions, all of which are affected by the presence or absence of background visual motion. In this study, we focused on DCMD responses to objects transitioning away from a collision course, which emulates a successful locust avoidance behavior. We presented each of 20 locusts with a sequence of complex three‐dimensional visual stimuli in simple, scattered, and progressive flow field backgrounds while simultaneously recording DCMD activity extracellularly. DCMD responses to looming stimuli were generally characteristic irrespective of stimulus background. However, changing background complexity affected, peak firing rates, peak time, and caused changes in peak rise and fall phases. The DCMD response to complex object motion also varied with the azimuthal approach angle and the dynamics of object edge expansion. These data fit with an existing correlational model that relates expansion properties to firing rate modulation during trajectory changes.
---
paper_title: Visual Motion: Cellular Implementation of a Hybrid Motion Detector

paper_content:

Visual motion detection in insects is mediated by three-input detectors that compare inputs of different spatiotemporal properties. A new modeling study shows that only a small subset of possible arrangements of the input elements provides high direction-selectivity.
---
paper_title: Defence behaviours of the praying mantis Tenodera aridifolia in response to looming objects

paper_content:

Defence responses to approaching objects were observed in the mantis Tenodera aridifolia. The mantis showed three kinds of behaviour, fixation, evasion and cryptic reaction. The cryptic reaction consisted of rapid retraction of the forelegs under the prothorax or rapid extending of the forelegs in the forward direction. Obstructing the mantis’ sight decreased its response rates, suggesting that the visual stimuli generated by an approaching object elicited the cryptic reaction. The response rate of the cryptic reactions was highest for objects that approached on a direct collision course. Deviation in a horizontal direction from the direct collision course resulted in a reduced response. The response rate of the cryptic reaction increased as the approaching velocity of the object increased, and the rate decreased as the object ceased its approach at a greater distance from the mantis. These results suggest that the function of the observed cryptic reactions is defence against impending collisions. The possible role of the looming-sensitive neuron in the cryptic reaction is also discussed.
---
paper_title: Neural networks in the cockpit of the fly

paper_content:

Flies have been buzzing around on earth for over 300 million years. During this time they have radiated into more than 125,000 different species (Yeates and Wiegmann 1999), so that, by now, roughly every tenth described species is a fly. They thus represent one of the most successful animal groups on our planet. This evolutionary success might, at least in part, be a result of their acrobatic maneuverability, which enables them, for example, to chase mates at turning velocities of more than 3000° s–1 with delay times of less than 30 ms (Land and Collett 1974; Wagner 1986). It is this fantastic behavior, which has initiated much research during the last decades, both on its sensory control and the biophysical and aerodynamic principles of the flight output (Dickinson et al. 1999, 2000). Here, we review the current state of knowledge about the neural processing of visual motion, which represents one sensory component intimately involved in flight control. Other reviews on this topic have been published with a similar (Hausen 1981, 1984; Hausen and Egelhaaf 1989; Borst 1996) or different emphasis (Frye and Dickinson 2001; Borst and Dickinson 2002). Because of space limitations, we do not review the extensive work that has been done on fly motion-sensitive neurons to advance our understanding of neural coding (Bialek et al. 1991; Rieke et al. 1997; de Ruyter et al. 1997, 2000; Haag and Borst 1997, 1998; Borst and Haag 2001). Unless stated otherwise, all data presented in the following were obtained on the blowfly Calliphora vicina which we will often casually refer to as 'the fly'.
---
paper_title: A neural network for pursuit tracking inspired by the fly visual system

paper_content:

Abstract This paper presents an artificial neural network that detects and tracks an object moving within its field of view. This novel network is inspired by processing functions observed in the fly visual system. The network detects changes in input light intensities, determines motion on both the local and the wide-field levels, and outputs displacement information necessary to control pursuit tracking. Software simulations demonstrate the current prototype successfully follows a moving target within specified radiance and motion constraints. The paper reviews these limiting constraints and suggests future network augmentations to remove them. Despite its current limitations, the existing prototype serves as a solid foundation for a future network that promises to provide machines with the improved abilities to do high-speed pursuit tracking, interception, and collision avoidance.
---
paper_title: Fundamental mechanisms of visual motion detection: models, cells and functions

paper_content:

Taking a comparative approach, data from a range of visual species are discussed in the context of ideas about mechanisms of motion detection. The cellular basis of motion detection in the vertebrate retina, sub-cortical structures and visual cortex is reviewed alongside that of the insect optic lobes. Special care is taken to relate concepts from theoretical models to the neural circuitry in biological systems. Motion detection involves spatiotemporal pre-filters, temporal delay filters and non-linear interactions. A number of different types of non-linear mechanism such as facilitation, inhibition and division have been proposed to underlie direction selectivity. The resulting direction-selective mechanisms can be combined to produce speed-tuned motion detectors. Motion detection is a dynamic process with adaptation as a fundamental property. The behavior of adaptive mechanisms in motion detection is discussed, focusing on the informational basis of motion adaptation, its phenomenology in human vision, and its cellular basis. The question of whether motion adaptation serves a function or is simply the result of neural fatigue is critically addressed.
---
paper_title: Object tracking in motion-blind flies

paper_content:

In response to the movement of its visual world, Drosophila is capable of optomotor response in head and body turning, as well as a visual fixation response. This study shows that blocking the visual pathway activity responsible for optokinetic response in flies does not affect the visual fixation response, suggesting two distinct pathways for processing each set of information. By doing so, the authors also devised a neural and behavioral hierarchy in fly visual system where fixation behavior and the neurons mediating fixation response are upstream of optokinetic response as performed by lobula plate neurons.
---
paper_title: A look into the cockpit of the fly: visual orientation, algorithms, and identified neurons

paper_content:

The top-down approach to understanding brain function seeks to account for the behavior of an animal in terms of biophysical properties of nerve cells and synaptic interactions via a series of progressively reductive levels of explanation. Using the fly as a model system, this approach was pioneered by Werner Reichardt and his colleagues in the late 1950s. Quantitative input-output analyses led them to formal algorithms that related the input of the fly's eye to the orientation behavior of the animal. But it has been possible only recently to track down the implementation of part of these algorithms to the computations performed by individual neurons and small neuronal ensembles. Thus, the visually guided flight maneuvers of the fly have turned out to be one of the few cases in which it has been feasible to reach an understanding of the mechanisms underlying a complex behavioral performance at successively reductive levels of analysis. These recent findings illuminate some of the fundamental questions that are being debated in computational neuroscience (Marr and Poggio, 1977; Sejnowski et al., 1988; Churchland and Sejnowski, 1992): (1) Are some brain functions emergent properties present only at the systems level? (2) Does an understanding of brain function at the systems level help in understanding function at the cellular and subcellular level? (3) Can different levels of organization be understood independently of each other? In this review we concentrate on two basic computational tasks that have to be solved by the fly, as well as by many other moving animals: (1) stabilization of an intended course against disturbances and (2) intended deviations from a straight course in order to orient toward salient objecs. Performing these tasks depends on the extraction of motion information from the changing distribution of light intensity received by the eyes.
---
paper_title: Behavioral Models of the Praying Mantis as a Basis for Robotic Behavior

paper_content:

Abstract Formal models of animal sensorimotor behavior can provide effective methods for generating robotic intelligence. In this article we describe how schema-theoretic models of the praying mantis derived from behavioral and neuroscientific data can be implemented on a hexapod robot equipped with a real time color vision system. This implementation incorporates a wide range of behaviors, including obstacle avoidance, prey acquisition, predator avoidance, mating, and chantlitaxia behaviors that can provide guidance to neuroscientists, ethologists, and roboticists alike. The goals of this study are threefold: to provide an understanding and means by which fielded robotic systems are not competing with other agents that are more effective at their designated task; to permit them to be successful competitors within the ecological system and capable of displacing less efficient agents; and that they are ecologically sensitive so that agent–environment dynamics are well-modeled and as predictable as possible whenever new robotic technology is introduced.
---
paper_title: A look into the cockpit of the developing locust: Looming detectors and predator avoidance

paper_content:

For many animals, the visual detection of looming stimuli is crucial at any stage of their lives. For example, human babies of only 6 days old display evasive responses to looming stimuli (Bower et al. [1971]: Percept Psychophys 9: 193-196). This means the neuronal pathways involved in looming detection should mature early in life. Locusts have been used extensively to examine the neural circuits and mechanisms involved in sensing looming stimuli and triggering visually evoked evasive actions, making them ideal subjects in which to investigate the development of looming sensitivity. Two lobula giant movement detectors (LGMD) neurons have been identified in the lobula region of the locust visual system: the LGMD1 neuron responds selectively to looming stimuli and provides information that contributes to evasive responses such as jumping and emergency glides. The LGMD2 responds to looming stimuli and shares many response properties with the LGMD1. Both neurons have only been described in the adult. In this study, we describe a practical method combining classical staining techniques and 3D neuronal reconstructions that can be used, even in small insects, to reveal detailed anatomy of individual neurons. We have used it to analyze the anatomy of the fan-shaped dendritic tree of the LGMD1 and the LGMD2 neurons in all stages of the post-embryonic development of Locusta migratoria. We also analyze changes seen during the ontogeny of escape behaviors triggered by looming stimuli, specially the hiding response.
---
paper_title: Higher-Order Figure Discrimination in Fly and Human Vision

paper_content:

Visually-guided animals rely on their ability to stabilize the panorama and simultaneously track salient objects, or figures, that are distinct from the background in order to avoid predators, pursue food resources and mates, and navigate spatially. Visual figures are distinguished by luminance signals that produce coherent motion cues as well as more enigmatic 'higher-order' statistical features. Figure discrimination is thus a complex form of motion vision requiring specialized neural processing. In this minireview, we will highlight recent advances in understanding the perceptual, behavioral, and neurophysiological basis of higher-order figure detection in flies, much of which is grounded in the historical perspective and mechanistic underpinnings of human psychophysics.
---
paper_title: Feedback Network Controls Photoreceptor Output at the Layer of First Visual Synapses in Drosophila

paper_content:

At the layer of first visual synapses, information from photoreceptors is processed and transmitted towards the brain. In fly compound eye, output from photoreceptors (R1-R6) that share the same visual field is pooled and transmitted via histaminergic synapses to two classes of interneuron, large monopolar cells (LMCs) and amacrine cells (ACs). The interneurons also feed back to photoreceptor terminals via numerous ligand-gated synapses, yet the significance of these connections has remained a mystery. We investigated the role of feedback synapses by comparing intracellular responses of photoreceptors and LMCs in wild-type Drosophila and in synaptic mutants, to light and current pulses and to naturalistic light stimuli. The recordings were further subjected to rigorous statistical and information-theoretical analysis. We show that the feedback synapses form a negative feedback loop that controls the speed and amplitude of photoreceptor responses and hence the quality of the transmitted signals. These results highlight the benefits of feedback synapses for neural information processing, and suggest that similar coding strategies could be used in other nervous systems.
---
paper_title: Processing properties of ON and OFF pathways for Drosophila motion detection

paper_content:

Four medulla neurons implement two critical processing steps to incoming signals in Drosophila motion detection.
---
paper_title: The Emergence of Directional Selectivity in the Visual Motion Pathway of Drosophila

paper_content:

Summary The perception of visual motion is critical for animal navigation, and flies are a prominent model system for exploring this neural computation. In Drosophila , the T4 cells of the medulla are directionally selective and necessary for ON motion behavioral responses. To examine the emergence of directional selectivity, we developed genetic driver lines for the neuron types with the most synapses onto T4 cells. Using calcium imaging, we found that these neuron types are not directionally selective and that selectivity arises in the T4 dendrites. By silencing each input neuron type, we identified which neurons are necessary for T4 directional selectivity and ON motion behavioral responses. We then determined the sign of the connections between these neurons and T4 cells using neuronal photoactivation. Our results indicate a computational architecture for motion detection that is a hybrid of classic theoretical models.
---
paper_title: Seeing what is coming: building collision-sensitive neurones

paper_content:

Abstract The image of a rapidly approaching object has to elicit a quick response. An animal needs to know that the object is approaching on a collision course and how imminent a collision is. The relevant information can be computed from the way that the image of the object grows on the retina of one eye. Firm data about the types of neurones that react to such looming stimuli and trigger avoidance reactions come from recent studies on the pigeon and the locust. The neurones responsible are tightly tuned to detect objects that are approaching on a direct collision course. In the pigeon these neurones signal the time remaining before collision whereas in the locust they have a crucial role in the simple strategy this animal uses to detect an object approaching on a collision course.
---
paper_title: Principles of visual motion detection

paper_content:

Motion information is required for the solution of many complex tasks of the visual system such as depth perception by motion parallax and figure/ground discrimination by relative motion. However, motion information is not explicitly encoded at the level of the retinal input. Instead, it has to be computed from the time-dependent brightness patterns of the retinal image as sensed by the two-dimensional array of photoreceptors. Different models have been proposed which describe the neural computations underlying motion detection in various ways. To what extent do biological motion detectors approximate any of these models? As will be argued here, there is increasing evidence from the different disciplines studying biological motion vision, that, throughout the animal kingdom ranging from invertebrates to vertebrates including man, the mechanisms underlying motion detection can be attributed to only a few, essentially equivalent computational principles. Motion detection may, therefore, be one of the first examples in computational neurosciences where common principles can be found not only at the cellular level (e.g. dendritic integration, spike propagation, synaptic transmission) but also at the level of computations performed by small neural networks.
---
paper_title: Impact and sources of neuronal variability in the fly’s motion vision pathway

paper_content:

Abstract Nervous systems encode information about dynamically changing sensory input by changes in neuronal activity. Neuronal activity changes, however, also arise from noise sources within and outside the nervous system or from changes of the animal’s behavioral state. The resulting variability of neuronal responses in representing sensory stimuli limits the reliability with which animals can respond to stimuli and may thus even affect the chances for survival in certain situations. Relevant sources of noise arising at different stages along the motion vision pathway have been investigated from the sensory input to the initiation of behavioral reactions. Here, we concentrate on the reliability of processing visual motion information in flies. Flies rely on visual motion information to guide their locomotion. They are among the best established model systems for the processing of visual motion information allowing us to bridge the gap between behavioral performance and underlying neuronal computations. It has been possible to directly assess the consequences of noise at major stages of the fly’s visual motion processing system on the reliability of neuronal signals. Responses of motion sensitive neurons and their variability have been related to optomotor movements as indicators for the overall performance of visual motion computation. We address whether and how noise already inherent in the stimulus, e.g. photon noise for the visual system, influences later processing stages and to what extent variability at the output level of the sensory system limits behavioral performance. Recent advances in circuit analysis and the progress in monitoring neuronal activity in behaving animals should now be applied to understand how the animal meets the requirements of fast and reliable manoeuvres in naturalistic situations.
---
paper_title: Minimum viewing angle for visually guided ground speed control in bumblebees.

paper_content:

SUMMARY To control flight, flying insects extract information from the pattern of visual motion generated during flight, known as optic flow. To regulate their ground speed, insects such as honeybees and Drosophila hold the rate of optic flow in the axial direction (front-to-back) constant. A consequence of this strategy is that its performance varies with the minimum viewing angle (the deviation from the frontal direction of the longitudinal axis of the insect) at which changes in axial optic flow are detected. The greater this angle, the later changes in the rate of optic flow, caused by changes in the density of the environment, will be detected. The aim of the present study is to examine the mechanisms of ground speed control in bumblebees and to identify the extent of the visual range over which optic flow for ground speed control is measured. Bumblebees were trained to fly through an experimental tunnel consisting of parallel vertical walls. Flights were recorded when (1) the distance between the tunnel walls was either 15 or 30 cm, (2) the visual texture on the tunnel walls provided either strong or weak optic flow cues and (3) the distance between the walls changed abruptly halfway along the tunnel9s length. The results reveal that bumblebees regulate ground speed using optic flow cues and that changes in the rate of optic flow are detected at a minimum viewing angle of 23–30 deg., with a visual field that extends to approximately 155 deg. By measuring optic flow over a visual field that has a low minimum viewing angle, bumblebees are able to detect and respond to changes in the proximity of the environment well before they are encountered.
---
paper_title: Direction Selectivity in Drosophila Emerges from Preferred-Direction Enhancement and Null-Direction Suppression

paper_content:

Across animal phyla, motion vision relies on neurons that respond preferentially to stimuli moving in one, preferred direction over the opposite, null direction. In the elementary motion detector of Drosophila , direction selectivity emerges in two neuron types, T4 and T5, but the computational algorithm underlying this selectivity remains unknown. We find that the receptive fields of both T4 and T5 exhibit spatiotemporally offset light-preferring and dark-preferring subfields, each obliquely oriented in spacetime. In a linear-nonlinear modeling framework, the spatiotemporal organization of the T5 receptive field predicts the activity of T5 in response to motion stimuli. These findings demonstrate that direction selectivity emerges from the enhancement of responses to motion in the preferred direction, as well as the suppression of responses to motion in the null direction. Thus, remarkably, T5 incorporates the essential algorithmic strategies used by the Hassenstein–Reichardt correlator and the Barlow–Levick detector. Our model for T5 also provides an algorithmic explanation for the selectivity of T5 for moving dark edges: our model captures all two- and three-point spacetime correlations relevant to motion in this stimulus class. More broadly, our findings reveal the contribution of input pathway visual processing, specifically center-surround, temporally biphasic receptive fields, to the generation of direction selectivity in T5. As the spatiotemporal receptive field of T5 in Drosophila is common to the simple cell in vertebrate visual cortex, our stimulus-response model of T5 will inform efforts in an experimentally tractable context to identify more detailed, mechanistic models of a prevalent computation. SIGNIFICANCE STATEMENT Feature selective neurons respond preferentially to astonishingly specific stimuli, providing the neurobiological basis for perception. Direction selectivity serves as a paradigmatic model of feature selectivity that has been examined in many species. While insect elementary motion detectors have served as premiere experimental models of direction selectivity for 60 years, the central question of their underlying algorithm remains unanswered. Using in vivo two-photon imaging of intracellular calcium signals, we measure the receptive fields of the first direction-selective cells in the Drosophila visual system, and define the algorithm used to compute the direction of motion. Computational modeling of these receptive fields predicts responses to motion and reveals how this circuit efficiently captures many useful correlations intrinsic to moving dark edges.
---
paper_title: Seeing Things in Motion: Models, Circuits, and Mechanisms

paper_content:

Motion vision provides essential cues for navigation and course control as well as for mate, prey, or predator detection. Consequently, neurons responding to visual motion in a direction-selective way are found in almost all species that see. However, directional information is not explicitly encoded at the level of a single photoreceptor. Rather, it has to be computed from the spatio-temporal excitation level of at least two photoreceptors. How this computation is done and how this computation is implemented in terms of neural circuitry and membrane biophysics have remained the focus of intense research over many decades. Here, we review recent progress made in this area with an emphasis on insects and the vertebrate retina.
---
paper_title: Common circuit design in fly and mammalian motion vision

paper_content:

Motion-sensitive neurons have long been studied in both the mammalian retina and the insect optic lobe, yet striking similarities have become obvious only recently. Detailed studies at the circuit level revealed that, in both systems, (i) motion information is extracted from primary visual information in parallel ON and OFF pathways; (ii) in each pathway, the process of elementary motion detection involves the correlation of signals with different temporal dynamics; and (iii) primary motion information from both pathways converges at the next synapse, resulting in four groups of ON-OFF neurons, selective for the four cardinal directions. Given that the last common ancestor of insects and mammals lived about 550 million years ago, this general strategy seems to be a robust solution for how to compute the direction of visual motion with neural hardware.
---
paper_title: Two identified looming detectors in the locust: ubiquitous lateral connections among their inputs contribute to selective responses to looming objects

paper_content:

Two identified looming detectors in the locust: ubiquitous lateral connections among their inputs contribute to selective responses to looming objects
---
paper_title: Vision for Mobile Robot Navigation: A Survey

paper_content:

Surveys the developments of the last 20 years in the area of vision for mobile robot navigation. Two major components of the paper deal with indoor navigation and outdoor navigation. For each component, we have further subdivided our treatment of the subject on the basis of structured and unstructured environments. For indoor robots in structured environments, we have dealt separately with the cases of geometrical and topological models of space. For unstructured environments, we have discussed the cases of navigation using optical flows, using methods from the appearance-based paradigm, and by recognition of specific objects in the environment.
---
paper_title: What does robotics offer animal behaviour?

paper_content:

There is a growing body of robot-based research that makes a serious claim to be a new methodology for biology. Robots can be used as models of specific animal systems to test hypotheses regarding the control of behaviour. At levels from learning algorithms to specific dendritic circuits, implementing a proposed controller in a robotic device tests it against real environments in a way that is difficult to simulate. This often provides insight into the true nature of the problem. It also enforces complete specifications and combines bodies of data. Current work can sometimes be criticized for drawing unjustified conclusions given the limited evaluation and inevitable inaccuracies of robot models. Nevertheless, this approach has led to novel hypotheses for animal behaviour and seems likely to provide fruitful results in the future. Copyright 2000 The Association for the Study of Animal Behaviour.
---
paper_title: Visuomotor control in flies and behavior-based agents

paper_content:

The development of artificial agents or robots using inspiration from living organisms is a promising approach to the study of biological systems, complementing traditional approaches in biology and the life sciences. The simulation of simple life forms and the implementation of behavioral models in robots are especially useful ways of testing models of biological information processing.
---
paper_title: Small Brains, Smart Machines: From Fly Vision to Robot Vision and Back Again

paper_content:

Neurobiological and neuroethological findings on insects can be used to design and construct small robots controlling their navigation on the basis of bio-inspired visual strategies and circuits. Animals' visual guidance is partly mediated by motion-sensitive neurons, which are responsible for gauging the optic flow. Although neurons of this kind were discovered in vertebrates' and invertebrates' visual systems more than 50 years ago, the principles and neural mechanisms involved have not yet been completely elucidated. Here, first, I propose to outline some of the findings we made during the last few decades by performing electrophysiological recordings on identified neurons in the housefly's eye while applying optical stimulation to identified photoreceptors. Whereas these findings shed light on the inner processing structure of an elementary motion detector (EMD), recent studies in which the latest genetic and neuroanatomical methods were applied to the fruitfly's visual system have identified some of the neurons in the visual chain which are possibly involved in the neural circuitry underlying a given EMD. Then, I will describe some of the proof-of-concept robots that we have developed on the basis of our biological findings. The 100-g robot OCTAVE, for example, is able to avoid the ground, react to wind, and land autonomously on a flat terrain without ever having to measure any state variables such as distances or speeds. The 100-g robots OSCAR 1 and OSCAR 2 inspired by the microscanner we discovered in the housefly's eye are able to stabilize their body using mainly visual means and track a moving edge with hyperacuity. These robots react to the optic flow, which is sensed by miniature optic flow sensors inspired by the housefly's EMDs. Constructing a “biorobot” gives us a unique opportunity of checking the soundness and robustness of a principle that is initially thought to be understood by bringing it face to face with the real physical world. Bio-inspired robotics not only help neurobiologists and neuroethologists to identify and investigate worthwhile problems in animals' sensory-motor systems, but they also provide engineers with ideas for developing novel devices and machines with promising future applications, in the field of smart autonomous vehicles and microvehicles, for example.
---
paper_title: Visual control of navigation in insects and its relevance for robotics

paper_content:

Flying insects display remarkable agility, despite their diminutive eyes and brains. This review describes our growing understanding of how these creatures use visual information to stabilize flight, avoid collisions with objects, regulate flight speed, detect and intercept other flying insects such as mates or prey, navigate to a distant food source, and orchestrate flawless landings. It also outlines the ways in which these insights are now being used to develop novel, biologically inspired strategies for the guidance of autonomous, airborne vehicles.
---
paper_title: Robot navigation inspired by principles of insect vision

paper_content:

Abstract Recent studies of insect visual behaviour and navigation reveal a number of elegant strategies that can be profitably applied to the design of autonomous robots. The peering behaviour of grasshoppers, for example, has inspired the design of new rangefinding systems. The centring response of bees flying through a tunnel has led to simple methods for navigating through corridors. Experimental investigation of the bee's “odometer” has led to the implementation of schemes for visually driven odometry. These and other visually mediated insect behaviours are described along with a number of applications to robot navigation.
---
paper_title: Honeybees as a Model for the Study of Visually Guided Flight, Navigation, and Biologically Inspired Robotics

paper_content:

Research over the past century has revealed the impressive capacities of the honeybee, Apis mellifera, in relation to visual perception, flight guidance, navigation, and learning and memory. These observations, coupled with the relative ease with which these creatures can be trained, and the relative simplicity of their nervous systems, have made honeybees an attractive model in which to pursue general principles of sensorimotor function in a variety of contexts, many of which pertain not just to honeybees, but several other animal species, including humans. This review begins by describing the principles of visual guidance that underlie perception of the world in three dimensions, obstacle avoidance, control of flight speed, and orchestrating smooth landings. We then consider how navigation over long distances is accomplished, with particular reference to how bees use information from the celestial compass to determine their flight bearing, and information from the movement of the environment in their eyes to gauge how far they have flown. Finally, we illustrate how some of the principles gleaned from these studies are now being used to design novel, biologically inspired algorithms for the guidance of unmanned aerial vehicles.
---
paper_title: New VLSI smart sensor for collision avoidance inspired by insect vision

paper_content:

An analog VLSI implementation of a smart microsensor that mimics the early visual processing stage in insects is described with an emphasis on the overall concept and the front- end detection. The system employs the `smart sensor' paradigm in that the detectors and processing circuitry are integrated on the one chip. The integrated circuit is composed of sixty channels of photodetectors and parallel processing elements. The photodetection circuitry includes p-well junction diodes on a 2 micrometers CMOS process and a logarithmic compression to increase the dynamic range of the system. The future possibility of gallium arsenide implementation is discussed. The processing elements behind each photodetector contain a low frequency differentiator where subthreshold design methods have been used. The completed IC is ideal for motion detection, particularly collision avoidance tasks, as it essentially detects distance, speed & bearing of an object. The Horridge Template Model for insect vision has been directly mapped into VLSI and therefore the IC truly exploits the beauty of nature in that the insect eye is so compact with parallel processing, enabling compact motion detection without the computational overhead of intensive imaging, full image extraction and interpretation. This world-first has exciting applications in the areas of automobile anti- collision, IVHS, autonomous robot guidance, aids for the blind, continuous process monitoring/web inspection and automated welding, for example.© (1995) COPYRIGHT SPIE--The International Society for Optical Engineering. Downloading of the abstract is permitted for personal use only.
---
paper_title: Optic flow-based collision-free strategies: From insects to robots

paper_content:

Flying insects are able to fly smartly in an unpredictable environment. It has been found that flying insects have smart neurons inside their tiny brains that are sensitive to visual motion also called optic flow. Consequently, flying insects rely mainly on visual motion during their flight maneuvers such as: takeoff or landing, terrain following, tunnel crossing, lateral and frontal obstacle avoidance, and adjusting flight speed in a cluttered environment. Optic flow can be defined as the vector field of the apparent motion of objects, surfaces, and edges in a visual scene generated by the relative motion between an observer (an eye or a camera) and the scene. Translational optic flow is particularly interesting for short-range navigation because it depends on the ratio between (i) the relative linear speed of the visual scene with respect to the observer and (ii) the distance of the observer from obstacles in the surrounding environment without any direct measurement of either speed or distance. In flying insects, roll stabilization reflex and yaw saccades attenuate any rotation at the eye level in roll and yaw respectively (i.e. to cancel any rotational optic flow) in order to ensure pure translational optic flow between two successive saccades. Our survey focuses on feedback-loops which use the translational optic flow that insects employ for collision-free navigation. Optic flow is likely, over the next decade to be one of the most important visual cues that can explain flying insects' behaviors for short-range navigation maneuvers in complex tunnels. Conversely, the biorobotic approach can therefore help to develop innovative flight control systems for flying robots with the aim of mimicking flying insects' abilities and better understanding their flight.
---
paper_title: From Wheels to Wings with Evolutionary Spiking Circuits

paper_content:

We give an overview of the EPFL indoor flying project, whose goal is to evolve neural controllers for autonomous, adaptive, indoor micro-flyers. Indoor flight is still a challenge because it requires miniaturization, energy efficiency, and control of nonlinear flight dynamics. This ongoing project consists of developing a flying, vision-based micro-robot, a bio-inspired controller composed of adaptive spiking neurons directly mapped into digital microcontrollers, and a method to evolve such a neural controller without human intervention. This article describes the motivation and methodology used to reach our goal as well as the results of a number of preliminary experiments on vision-based wheeled and flying robots.
---
paper_title: Fly visual course control: behaviour, algorithms and circuits

paper_content:

Understanding how the brain controls behaviour is undisputedly one of the grand goals of neuroscience research, and the pursuit of this goal has a long tradition in insect neuroscience. However, appropriate techniques were lacking for a long time. Recent advances in genetic and recording techniques now allow the participation of identified neurons in the execution of specific behaviours to be interrogated. By focusing on fly visual course control, I highlight what has been learned about the neuronal circuit modules that control visual guidance in Drosophila melanogaster through the use of these techniques.
---
paper_title: Small Brains, Smart Machines: From Fly Vision to Robot Vision and Back Again

paper_content:

Neurobiological and neuroethological findings on insects can be used to design and construct small robots controlling their navigation on the basis of bio-inspired visual strategies and circuits. Animals' visual guidance is partly mediated by motion-sensitive neurons, which are responsible for gauging the optic flow. Although neurons of this kind were discovered in vertebrates' and invertebrates' visual systems more than 50 years ago, the principles and neural mechanisms involved have not yet been completely elucidated. Here, first, I propose to outline some of the findings we made during the last few decades by performing electrophysiological recordings on identified neurons in the housefly's eye while applying optical stimulation to identified photoreceptors. Whereas these findings shed light on the inner processing structure of an elementary motion detector (EMD), recent studies in which the latest genetic and neuroanatomical methods were applied to the fruitfly's visual system have identified some of the neurons in the visual chain which are possibly involved in the neural circuitry underlying a given EMD. Then, I will describe some of the proof-of-concept robots that we have developed on the basis of our biological findings. The 100-g robot OCTAVE, for example, is able to avoid the ground, react to wind, and land autonomously on a flat terrain without ever having to measure any state variables such as distances or speeds. The 100-g robots OSCAR 1 and OSCAR 2 inspired by the microscanner we discovered in the housefly's eye are able to stabilize their body using mainly visual means and track a moving edge with hyperacuity. These robots react to the optic flow, which is sensed by miniature optic flow sensors inspired by the housefly's EMDs. Constructing a “biorobot” gives us a unique opportunity of checking the soundness and robustness of a principle that is initially thought to be understood by bringing it face to face with the real physical world. Bio-inspired robotics not only help neurobiologists and neuroethologists to identify and investigate worthwhile problems in animals' sensory-motor systems, but they also provide engineers with ideas for developing novel devices and machines with promising future applications, in the field of smart autonomous vehicles and microvehicles, for example.
---
paper_title: Optic flow-based collision-free strategies: From insects to robots

paper_content:

Flying insects are able to fly smartly in an unpredictable environment. It has been found that flying insects have smart neurons inside their tiny brains that are sensitive to visual motion also called optic flow. Consequently, flying insects rely mainly on visual motion during their flight maneuvers such as: takeoff or landing, terrain following, tunnel crossing, lateral and frontal obstacle avoidance, and adjusting flight speed in a cluttered environment. Optic flow can be defined as the vector field of the apparent motion of objects, surfaces, and edges in a visual scene generated by the relative motion between an observer (an eye or a camera) and the scene. Translational optic flow is particularly interesting for short-range navigation because it depends on the ratio between (i) the relative linear speed of the visual scene with respect to the observer and (ii) the distance of the observer from obstacles in the surrounding environment without any direct measurement of either speed or distance. In flying insects, roll stabilization reflex and yaw saccades attenuate any rotation at the eye level in roll and yaw respectively (i.e. to cancel any rotational optic flow) in order to ensure pure translational optic flow between two successive saccades. Our survey focuses on feedback-loops which use the translational optic flow that insects employ for collision-free navigation. Optic flow is likely, over the next decade to be one of the most important visual cues that can explain flying insects' behaviors for short-range navigation maneuvers in complex tunnels. Conversely, the biorobotic approach can therefore help to develop innovative flight control systems for flying robots with the aim of mimicking flying insects' abilities and better understanding their flight.
---
paper_title: Non-Linear Neuronal Responses as an Emergent Property of Afferent Networks: A Case Study of the Locust Lobula Giant Movement Detector

paper_content:

In principle it appears advantageous for single neurons to perform non-linear operations. Indeed it has been reported that some neurons show signatures of such operations in their electrophysiological response. A particular case in point is the Lobula Giant Movement Detector (LGMD) neuron of the locust, which is reported to locally perform a functional multiplication. Given the wide ramifications of this suggestion with respect to our understanding of neuronal computations, it is essential that this interpretation of the LGMD as a local multiplication unit is thoroughly tested. Here we evaluate an alternative model that tests the hypothesis that the non-linear responses of the LGMD neuron emerge from the interactions of many neurons in the opto-motor processing structure of the locust. We show, by exposing our model to standard LGMD stimulation protocols, that the properties of the LGMD that were seen as a hallmark of local non-linear operations can be explained as emerging from the dynamics of the pre-synaptic network. Moreover, we demonstrate that these properties strongly depend on the details of the synaptic projections from the medulla to the LGMD. From these observations we deduce a number of testable predictions. To assess the real-time properties of our model we applied it to a high-speed robot. These robot results show that our model of the locust opto-motor system is able to reliably stabilize the movement trajectory of the robot and can robustly support collision avoidance. In addition, these behavioural experiments suggest that the emergent non-linear responses of the LGMD neuron enhance the system's collision detection acuity. We show how all reported properties of this neuron are consistently reproduced by this alternative model, and how they emerge from the overall opto-motor processing structure of the locust. Hence, our results propose an alternative view on neuronal computation that emphasizes the network properties as opposed to the local transformations that can be performed by single neurons.
---
paper_title: Near range path navigation using LGMD visual neural networks

paper_content:

In this paper, we proposed a method for near range path navigation for a mobile robot by using a pair of biologically inspired visual neural network - lobula giant movement detector (LGMD). In the proposed binocular style visual system, each LGMD processes images covering a part of the wide field of view and extracts relevant visual cues as its output. The outputs from the two LGMDs are compared and translated into executable motor commands to control the wheels of the robot in real time. Stronger signal from the LGMD in one side pushes the robot away from this side step by step; therefore, the robot can navigate in a visual environment naturally with the proposed vision system. Our experiments showed that this bio-inspired system worked well in different scenarios.
---
paper_title: Bio-inspired collision detector with enhanced selectivity for ground robotic vision system

paper_content:

There are many ways of building collision-detecting systems. In this paper, we propose a novel collision selective visual neural network inspired by LGMD2 neurons in the juvenile locusts. Such collision-sensitive neuron matures early in the ﬁrst-aged or even hatching locusts, and is only selective to detect looming dark objects against bright background in depth, represents swooping predators, a situation which is similar to ground robots or vehicles. However, little has been done on modeling LGMD2, let alone its potential applications in robotics and other vision-based areas. Compared to other collision detectors, our major contributions are ﬁrst, enhancing the collision selectivity in a bio-inspired way, via constructing a computing efﬁcient visual sensor, and realizing the revealed speciﬁc characteristic sofLGMD2. Second, we applied the neural network to help rearrange path navigation of an autonomous ground miniature robot in an arena. We also examined its neural properties through systematic experiments challenged against image streams from a visual sensor of the micro-robot.
---
paper_title: Loom-Sensitive Neurons Link Computation to Action in the Drosophila Visual System

paper_content:

Summary Background Many animals extract specific cues from rich visual scenes to guide appropriate behaviors. Such cues include visual motion signals produced both by self-movement and by moving objects in the environment. The complexity of these signals requires neural circuits to link particular patterns of motion to specific behavioral responses. Results Through electrophysiological recordings, we characterize genetically identified neurons in the optic lobe of Drosophila that are specifically tuned to detect motion signals produced by looming objects on a collision course with the fly. Using a genetic manipulation to specifically silence these neurons, we demonstrate that signals from these cells are important for flies to efficiently initiate the loom escape response. Moreover, through targeted expression of channelrhodopsin in these cells, in flies that are blind, we reveal that optogenetic stimulation of these neurons is typically sufficient to elicit escape, even in the absence of any visual stimulus. Conclusions In this compact nervous system, a small group of neurons that extract a specific visual cue from local motion inputs serve to trigger the ethologically appropriate behavioral response.
---
paper_title: How accurate need sensory coding be for behaviour? Experiments using a mobile robot

paper_content:

Abstract This paper argues that for those neuronal systems which control behaviour, reliable responses are more appropriate than precise responses. We illustrate this argument using a mobile robot controlled by the responses of a neuronal model of the locust LGMD system, a visual system which responds to looming objects. Our experiments show that although the responses of the model LGMD vary widely as the robot approaches obstacles, they still trigger avoidance responses.
---
paper_title: Predator versus Prey: Locust Looming-Detector Neuron and Behavioural Responses to Stimuli Representing Attacking Bird Predators

paper_content:

Many arthropods possess escape-triggering neural mechanisms that help them evade predators. These mechanisms are important neuroethological models, but they are rarely investigated using predator-like stimuli because there is often insufficient information on real predator attacks. Locusts possess uniquely identifiable visual neurons (the descending contralateral movement detectors, DCMDs) that are well-studied looming motion detectors. The DCMDs trigger ‘glides’ in flying locusts, which are hypothesised to be appropriate last-ditch responses to the looms of avian predators. To date it has not been possible to study glides in response to stimuli simulating bird attacks because such attacks have not been characterised. We analyse video of wild black kites attacking flying locusts, and estimate kite attack speeds of 10.8±1.4 m/s. We estimate that the loom of a kite’s thorax towards a locust at these speeds should be characterised by a relatively low ratio of half size to speed (l/|v|) in the range 4–17 ms. Peak DCMD spike rate and gliding response occurrence are known to increase as l/|v| decreases for simple looming shapes. Using simulated looming discs, we investigate these trends and show that both DCMD and behavioural responses are strong to stimuli with kite-like l/|v| ratios. Adding wings to looming discs to produce a more realistic stimulus shape did not disrupt the overall relationships of DCMD and gliding occurrence to stimulus l/|v|. However, adding wings to looming discs did slightly reduce high frequency DCMD spike rates in the final stages of object approach, and slightly delay glide initiation. Looming discs with or without wings triggered glides closer to the time of collision as l/|v| declined, and relatively infrequently before collision at very low l/|v|. However, the performance of this system is in line with expectations for a last-ditch escape response.
---
paper_title: Computation of object approach by a system of visual motion-sensitive neurons in the crab Neohelice

paper_content:

Similar to most visual animals, crabs perform proper avoidance responses to objects directly approaching them. The monostratified lobula giant neurons of type 1 (MLG1) of crabs constitute an ensemble of 14–16 bilateral pairs of motion-detecting neurons projecting from the lobula (third optic neuropile) to the midbrain, with receptive fields that are distributed over the extensive visual field of the animal's eye. Considering the crab Neohelice (previously Chasmagnathus) granulata, here we describe the response of these neurons to looming stimuli that simulate objects approaching the animal on a collision course. We found that the peak firing time of MLG1 acts as an angular threshold detector signaling, with a delay of δ = 35 ms, the time at which an object reaches a fixed angular threshold of 49°. Using in vivo intracellular recordings, we detected the existence of excitatory and inhibitory synaptic currents that shape the neural response. Other functional features identified in the MLG1 neurons were phas...
---
paper_title: An obstacle avoidance method for two wheeled mobile robot

paper_content:

In this paper, we proposed an obstacle avoidance method of mobile robot by using lobula giant movement detector (LGMD) method. The LGMD is an identified neuron in the locust brain that responds most strongly to the image of an approaching object such as a predator. As an assistance to the avoidance method, we add a distance measurement algorithm to LGMD method. The measurement algorithm is based on information from camera to detect obstacles and calculate the route for two wheeled mobile robot. Simulation result confirmed the effectiveness of the proposed method.
---
paper_title: The anatomy and output connection of a locust visual interneurone; the lobular giant movement detector (LGMD) neurone

paper_content:

1. ::: ::: The anatomy of a giant movement detector neurone in the locust lobula (the LGMD) is described on the basis of both intracellular injection of cobalt (Fig. 2) and the reconstruction of osmium-ethyl gallate and silver impregnated serial sections (Fig. 3). ::: ::: ::: ::: ::: 2. ::: ::: It is shown that the LGMD has an anatomically complex junction with a previously described interneurone, the Descending Contralateral Movement Detector (Fig. 4), and that spikes in the LGMD precede 1∶1 with fixed latency spikes in the DCMD (Figs. 1,5). ::: ::: ::: ::: ::: 3. ::: ::: Three separate dendritic subfields are seen in the lobula complex (Figs. 2, 3); these are tentatively ascribed to the three different classes of input to the cell. ::: ::: ::: ::: ::: 4. ::: ::: A large part of the LGMD's terminal arborisation appears to serve only a single functional junction, that with the DCMD (Fig. 4).
---
paper_title: Seeing what is coming: building collision-sensitive neurones

paper_content:

Abstract The image of a rapidly approaching object has to elicit a quick response. An animal needs to know that the object is approaching on a collision course and how imminent a collision is. The relevant information can be computed from the way that the image of the object grows on the retina of one eye. Firm data about the types of neurones that react to such looming stimuli and trigger avoidance reactions come from recent studies on the pigeon and the locust. The neurones responsible are tightly tuned to detect objects that are approaching on a direct collision course. In the pigeon these neurones signal the time remaining before collision whereas in the locust they have a crucial role in the simple strategy this animal uses to detect an object approaching on a collision course.
---
paper_title: Two identified looming detectors in the locust: ubiquitous lateral connections among their inputs contribute to selective responses to looming objects

paper_content:

Two identified looming detectors in the locust: ubiquitous lateral connections among their inputs contribute to selective responses to looming objects
---
paper_title: Immunocytochemical evidence that collision sensing neurons in the locust visual system contain acetylcholine

paper_content:

The lobula giant movement detector (LGMD1 and -2) neurons in the locust visual system are parts of motion-sensitive pathways that detect objects approaching on a collision course. The dendritic processes of the LGMD1 and -2 in the lobula are localised to discrete regions, allowing the dendrites of each neuron to be distinguished uniquely. As was described previously for the LGMD1, the afferent processes onto the LGMD2 synapse directly with each other, and these synapses are immediately adjacent to their outputs onto the LGMD2. Here we present immunocytochemical evidence, using antibodies against choline-protein conjugates and a polyclonal antiserum against choline acetyltransferase (ChAT; Chemicon Ab 143), that the LGMD1 and -2 and the retinotopic units presynaptic to them contain acetylcholine (ACh). It is proposed that these retinotopic units excite the LGMD1 or -2 but inhibit each other. It is well established that ACh has both excitatory and inhibitory effects and may provide the substrate for a critical race in the LGMD1 or -2, between excitation caused by edges moving out over successive photoreceptors, and inhibition spreading laterally resulting in the selective response to objects approaching on a collision course. In the optic lobe, ACh was also found to be localised in discrete layers of the medulla and in the outer chiasm between the lamina and medulla. In the brain, the antennal lobes contained neurons that reacted positively for ACh. Silver- or haematoxylin and eosin-stained sections through the optic lobe confirmed the identities of the positively immunostained neurons.
---
paper_title: Synaptic Connections of First-Stage Visual Neurons in the Locust Schistocerca gregaria Extend Evolution of Tetrad Synapses Back 200 Million Years

paper_content:

The small size of some insects, and the crystalline regularity of their eyes, have made them ideal for large-scale reconstructions of visual circuits. In phylogenetically recent muscomorph flies, like Drosophila, precisely coordinated output to different motion-processing pathways is delivered by photoreceptors (R cells), targeting four different postsynaptic cells at each synapse (tetrad). Tetrads were linked to the evolution of aerial agility. To reconstruct circuits for vision in the larger brain of a locust, a phylogenetically old, flying insect, we adapted serial block-face scanning electron microscopy (SBEM). Locust lamina monopolar cells, L1 and L2, were the main targets of the R cell pathway, L1 and L2 each fed a different circuit, only L1 providing feedback onto R cells. Unexpectedly, 40% of all locust R cell synapses onto both L1 and L2 were tetrads, revealing the emergence of tetrads in an arthropod group present 200 million years before muscomorph flies appeared, coinciding with the early evolution of flight. J. Comp. Neurol. 523:298–312, 2015. © 2014 Wiley Periodicals, Inc.
---
paper_title: Spike frequency adaptation mediates looming stimulus selectivity in a collision-detecting neuron

paper_content:

Studying the mechanisms by which spike frequency adaptation shapes visual stimulus selectivity in the lobula giant movement detector interneuron of the locust visual system, the authors find that spike frequency adaptation selectively decreases this neuron's responses to nonpreferred stimuli.
---
paper_title: Activity of descending contralateral movement detector neurons and collision avoidance behaviour in response to head-on visual stimuli in locusts

paper_content:

Abstract. We recorded the activity of the right and left descending contralateral movement detectors responding to 10-cm (small) or 20-cm (large) computer-generated spheres approaching along different trajectories in the locust's frontal field of view. In separate experiments we examined the steering responses of tethered flying locusts to identical stimuli. The descending contralateral movement detectors were more sensitive to variations in target trajectory in the horizontal plane than in the vertical plane. Descending contralateral movement detector activity was related to target trajectory and to target size and was most sensitive to small objects converging on a direct collision course from above and to one side. Small objects failed to induce collision avoidance manoeuvres whereas large objects produced reliable collision avoidance responses. Large targets approaching along a converging trajectory produced steering responses that were either away from or toward the side of approach of the object, whereas targets approaching along trajectories that were offset from the locust's mid-longitudinal body axis primarily evoked responses away from the target. We detected no differences in the discharge properties of the descending contralateral movement detector pair that could account for the different collision avoidance behaviours evoked by varying the target size and trajectories. We suggest that descending contralateral movement detector properties are better suited to predator evasion than collision avoidance.
---
paper_title: Responses to object approach by a wide field visual neurone, the LGMD2 of the locust: Characterization and image cues

paper_content:

The LGMD2 belongs to a group of giant movement-detecting neurones which have fan-shaped arbors in the lobula of the locust optic lobe and respond to movements of objects. One of these neurones, the LGMD1, has been shown to respond directionally to movements of objects in depth, generating vigorous, maintained spike discharges during object approach. Here we compare the responses of the LGMD2 neurone with those of the LGMD1 to simulated movements of objects in depth and examine different image cues which could allow the LGMD2 to distinguish approaching from receding objects. In the absence of stimulation, the LGMD2 has a resting discharge of 10–40 spikes s−1 compared with <1 spike s−1 for the LGMD1. The most powerful excitatory stimulus for the LGMD2 is a dark object approaching the eye. Responses to approaching objects are suppressed by wide field movements of the background. Unlike the LGMD1, the LGMD2 is not excited by the approach of light objects; it specifically responds to movement of edges in the light to dark direction. Both neurones rely on the same monocular image cues to distinguish approaching from receding objects: an increase in the velocity with which edges of images travel over the eye; and an increase in the extent of edges in the image during approach.
---
paper_title: Looming detection by identified visual interneurons during larval development of the locust Locusta migratoria

paper_content:

Insect larvae clearly react to visual stimuli, but the ability of any visual neuron in a newly hatched insect to respond selectively to particular stimuli has not been directly tested. We characterised a pair of neurons in locust larvae that have been extensively studied in adults, where they are known to respond selectively to objects approaching on a collision course: the lobula giant motion detector (LGMD) and its postsynaptic partner, the descending contralateral motion detector (DCMD). Our physiological recordings of DCMD axon spikes reveal that at the time of hatching, the neurons already respond selectively to objects approaching the locust and they discriminate between stimulus approach speeds with differences in spike frequency. For a particular approaching stimulus, both the number and peak frequency of spikes increase with instar. In contrast, the number of spikes in responses to receding stimuli decreases with instar, so performance in discriminating approaching from receding stimuli improves as the locust goes through successive moults. In all instars, visual movement over one part of the visual field suppresses a response to movement over another part. Electron microscopy demonstrates that the anatomical substrate for the selective response to approaching stimuli is present in all larval instars: small neuronal processes carrying information from the eye make synapses both onto LGMD dendrites and with each other, providing pathways for lateral inhibition that shape selectivity for approaching objects.
---
paper_title: INTRACELLULAR CHARACTERIZATION OF NEURONS IN THE LOCUST BRAIN SIGNALING IMPENDING COLLISION

paper_content:

1. In response to a rapidly approaching object, intracellular recordings show that excitation in the locust lobula giant movement detecting (LGMD) neuron builds up exponentially, particularly during the final stages of object approach. After the cessation of object motion, inhibitory potentials in the LGMD then help to terminate this excitation. Excitation in the LGMD follows object recession with a short, constant latency but is cut back rapidly by hyperpolarizing potentials. The timing of these hyperpolarizing potentials in the LGMD is variable, and their latency following object recession is shortest with the highest velocities of motion simulated. The hyperpolarizing potentials last from 50-300 ms and are often followed by re-excitation. The observed hyperpolarizations of the LGMD can occur without any preceding excitation and are accompanied by a measurable conductance increase. The hyperpolarizations are likely to be inhibitory postsynaptic potentials (PSPs). The behavior of the intracellularly recorded inhibitory PSPs (IPSPs) closely parallels that of the feed forward inhibitory loop in the neural network described by Rind and Bramwell. 2. The preference of the LGMD for approaching versus receding objects remains over a wide range of starting and finishing distances. The response to object approach, measured both as membrane potential and spike rate, remains single peaked with starting distances of between 200 and 2,100 mm, and approach speeds of 0.5-2 m/s. These results confirm the behavior predicted by the neural network described by Rind and Bramwell but contradicts the findings of Rind and Simmons, forcing a re-evaluation of the suitability of some of the mechanical visual stimuli used in that study. 3. For depolarization of the LGMD neuron to be maintained or increased throughout the motion of image edges, the edges must move with increasing velocity over the eye. Membrane potential declines before the end of edge motion with constant velocities of edge motion. 4. A second identified neuron, the LGMD2 also is shown to respond directionally to approaching objects. In both the LGMD and LGMD2 neurons, postsynaptic inhibition shapes the directional response to object motion.
---
paper_title: Orthopteran DCMD neuron: a reevaluation of responses to moving objects. I. Selective responses to approaching objects

paper_content:

1. The "descending contralateral movement detector" (DCMD) neuron in the locust has been challenged with a variety of moving stimuli, including scenes from a film (Star Wars), moving disks, and images generated by computer. The neuron responds well to any rapid movement. For a dark object moving along a straight path at a uniform velocity, the DCMD gives the strongest response when the object travels directly toward the eye, and the weakest when the object travels away from the eye. Instead of expressing selectivity for movements of small rather than large objects, the DCMD responds preferentially to approaching objects. 2. The neuron shows a clear selectivity for approach over recession for a variety of sizes and velocities of movement both of real objects and in simulated movements. When a disk that subtends > or = 5 degrees at the eye approaches the eye, there are two peaks in spike rate: one immediately after the start of movement; and a second that builds up during the approach. When a disk recedes f...
---
paper_title: Neural network based on the input organization of an identified neuron signaling impending collision

paper_content:

1. We describe a four-layered neural network (Fig. 1), based on the input organization of a collision signaling neuron in the visual system of the locust, the lobula giant movement detector (LGMD)....
---
paper_title: Background visual motion affects responses of an insect motion‐sensitive neuron to objects deviating from a collision course

paper_content:

Stimulus complexity affects the response of looming sensitive neurons in a variety of animal taxa. The Lobula Giant Movement Detector/Descending Contralateral Movement Detector (LGMD/DCMD) pathway is well‐characterized in the locust visual system. It responds to simple objects approaching on a direct collision course (i.e., looming) as well as complex motion defined by changes in stimulus velocity, trajectory, and transitions, all of which are affected by the presence or absence of background visual motion. In this study, we focused on DCMD responses to objects transitioning away from a collision course, which emulates a successful locust avoidance behavior. We presented each of 20 locusts with a sequence of complex three‐dimensional visual stimuli in simple, scattered, and progressive flow field backgrounds while simultaneously recording DCMD activity extracellularly. DCMD responses to looming stimuli were generally characteristic irrespective of stimulus background. However, changing background complexity affected, peak firing rates, peak time, and caused changes in peak rise and fall phases. The DCMD response to complex object motion also varied with the azimuthal approach angle and the dynamics of object edge expansion. These data fit with an existing correlational model that relates expansion properties to firing rate modulation during trajectory changes.
---
paper_title: The locust DCMD, a movement-detecting neurone tightly tuned to collision trajectories

paper_content:

A Silicon Graphics computer was used to challenge the locust descending contralateral movement detector (DCMD) neurone with images of approaching objects. The DCMD gave its strongest response, measured as either total spike number or spike frequency, to objects approaching on a direct collision course. Deviation in either a horizontal or vertical direction from a direct collision course resulted in a reduced response. The decline in the DCMD response with increasing deviation from a collision course was used as a measure of the tightness of DCMD tuning for collision trajectories. Tuning was defined as the half-width of the response when it had fallen to half its maximum level. The response tuning, measured as averaged mean spike number versus deviation away from a collision course, had a half-width at half-maximum response of 2.4 ds3.0 d for a deviation in the horizontal direction and 3.0 d for a deviation in the vertical direction. Mean peak spike frequency showed an even sharper tuning, with a half-width at half-maximum response of 1.8 d for deviations away from a collision course in the horizontal plane.
---
paper_title: Multiplication and stimulus invariance in a looming-sensitive neuron

paper_content:

Multiplicative operations and invariance of neuronal responses are thought to play important roles in the processing of neural information in many sensory systems. Yet the biophysical mechanisms that underlie both multiplication and invariance of neuronal responses in vivo, either at the single cell or at the network level, remain to a large extent unknown. Recent work on an identified neuron in the locust visual system (the LGMD neuron) that responds well to objects looming on a collision course towards the animal suggests that this cell represents a good model to investigate the biophysical basis of multiplication and invariance at the single neuron level. Experimental and theoretical results are consistent with multiplication being implemented by subtraction of two logarithmic terms followed by exponentiation via active membrane conductances, according to a x 1/b = exp(log(a) - log(b)). Invariance appears to be in part due to non-linear integration of synaptic inputs within the dendritic tree of this neuron.
---
paper_title: The anatomy and output connection of a locust visual interneurone; the lobular giant movement detector (LGMD) neurone

paper_content:

1. ::: ::: The anatomy of a giant movement detector neurone in the locust lobula (the LGMD) is described on the basis of both intracellular injection of cobalt (Fig. 2) and the reconstruction of osmium-ethyl gallate and silver impregnated serial sections (Fig. 3). ::: ::: ::: ::: ::: 2. ::: ::: It is shown that the LGMD has an anatomically complex junction with a previously described interneurone, the Descending Contralateral Movement Detector (Fig. 4), and that spikes in the LGMD precede 1∶1 with fixed latency spikes in the DCMD (Figs. 1,5). ::: ::: ::: ::: ::: 3. ::: ::: Three separate dendritic subfields are seen in the lobula complex (Figs. 2, 3); these are tentatively ascribed to the three different classes of input to the cell. ::: ::: ::: ::: ::: 4. ::: ::: A large part of the LGMD's terminal arborisation appears to serve only a single functional junction, that with the DCMD (Fig. 4).
---
paper_title: Seeing what is coming: building collision-sensitive neurones

paper_content:

Abstract The image of a rapidly approaching object has to elicit a quick response. An animal needs to know that the object is approaching on a collision course and how imminent a collision is. The relevant information can be computed from the way that the image of the object grows on the retina of one eye. Firm data about the types of neurones that react to such looming stimuli and trigger avoidance reactions come from recent studies on the pigeon and the locust. The neurones responsible are tightly tuned to detect objects that are approaching on a direct collision course. In the pigeon these neurones signal the time remaining before collision whereas in the locust they have a crucial role in the simple strategy this animal uses to detect an object approaching on a collision course.
---
paper_title: Neuronal correlates of the visually elicited escape response of the crab Chasmagnathus upon seasonal variations, stimuli changes and perceptual alterations

paper_content:

When confronted with predators, animals are forced to take crucial decisions such as the timing and manner of escape. In the case of the crab Chasmagnathus, cumulative evidence suggests that the escape response to a visual danger stimulus (VDS) can be accounted for by the response of a group of lobula giant (LG) neurons. To further investigate this hypothesis, we examined the relationship between behavioral and neuronal activities within a variety of experimental conditions that affected the level of escape. The intensity of the escape response to VDS was influenced by seasonal variations, changes in stimulus features, and whether the crab perceived stimuli monocularly or binocularly. These experimental conditions consistently affected the response of LG neurons in a way that closely matched the effects observed at the behavioral level. In other words, the intensity of the stimulus-elicited spike activity of LG neurons faithfully reflected the intensity of the escape response. These results support the idea that the LG neurons from the lobula of crabs are deeply involved in the decision for escaping from VDS.
---
paper_title: Two identified looming detectors in the locust: ubiquitous lateral connections among their inputs contribute to selective responses to looming objects

paper_content:

Two identified looming detectors in the locust: ubiquitous lateral connections among their inputs contribute to selective responses to looming objects
---
paper_title: Non-Linear Neuronal Responses as an Emergent Property of Afferent Networks: A Case Study of the Locust Lobula Giant Movement Detector

paper_content:

In principle it appears advantageous for single neurons to perform non-linear operations. Indeed it has been reported that some neurons show signatures of such operations in their electrophysiological response. A particular case in point is the Lobula Giant Movement Detector (LGMD) neuron of the locust, which is reported to locally perform a functional multiplication. Given the wide ramifications of this suggestion with respect to our understanding of neuronal computations, it is essential that this interpretation of the LGMD as a local multiplication unit is thoroughly tested. Here we evaluate an alternative model that tests the hypothesis that the non-linear responses of the LGMD neuron emerge from the interactions of many neurons in the opto-motor processing structure of the locust. We show, by exposing our model to standard LGMD stimulation protocols, that the properties of the LGMD that were seen as a hallmark of local non-linear operations can be explained as emerging from the dynamics of the pre-synaptic network. Moreover, we demonstrate that these properties strongly depend on the details of the synaptic projections from the medulla to the LGMD. From these observations we deduce a number of testable predictions. To assess the real-time properties of our model we applied it to a high-speed robot. These robot results show that our model of the locust opto-motor system is able to reliably stabilize the movement trajectory of the robot and can robustly support collision avoidance. In addition, these behavioural experiments suggest that the emergent non-linear responses of the LGMD neuron enhance the system's collision detection acuity. We show how all reported properties of this neuron are consistently reproduced by this alternative model, and how they emerge from the overall opto-motor processing structure of the locust. Hence, our results propose an alternative view on neuronal computation that emphasizes the network properties as opposed to the local transformations that can be performed by single neurons.
---
paper_title: Modelling LGMD2 visual neuron system

paper_content:

Two Lobula Giant Movement Detectors (LGMDs) have been identified in the lobula region of the locust visual system: LGMD1 and LGMD2. LGMD1 had been successfully used in robot navigation to avoid impending collision. LGMD2 also responds to looming stimuli in depth, and shares most the same properties with LGMD1; however, LGMD2 has its specific collision selective responds when dealing with different visual stimulus. Therefore, in this paper, we propose a novel way to model LGMD2, in order to emulate its predicted bio-functions, moreover, to solve some defects of previous LGMD1 computational models. The mechanism of ON and OFF cells, as well as bio-inspired nonlinear functions, are introduced in our model, to achieve LGMD2's collision selectivity. Our model has been tested by a miniature mobile robot in real time. The results suggested this model has an ideal performance in both software and hardware for collision recognition.
---
paper_title: Synaptic Connections of First-Stage Visual Neurons in the Locust Schistocerca gregaria Extend Evolution of Tetrad Synapses Back 200 Million Years

paper_content:

The small size of some insects, and the crystalline regularity of their eyes, have made them ideal for large-scale reconstructions of visual circuits. In phylogenetically recent muscomorph flies, like Drosophila, precisely coordinated output to different motion-processing pathways is delivered by photoreceptors (R cells), targeting four different postsynaptic cells at each synapse (tetrad). Tetrads were linked to the evolution of aerial agility. To reconstruct circuits for vision in the larger brain of a locust, a phylogenetically old, flying insect, we adapted serial block-face scanning electron microscopy (SBEM). Locust lamina monopolar cells, L1 and L2, were the main targets of the R cell pathway, L1 and L2 each fed a different circuit, only L1 providing feedback onto R cells. Unexpectedly, 40% of all locust R cell synapses onto both L1 and L2 were tetrads, revealing the emergence of tetrads in an arthropod group present 200 million years before muscomorph flies appeared, coinciding with the early evolution of flight. J. Comp. Neurol. 523:298–312, 2015. © 2014 Wiley Periodicals, Inc.
---
paper_title: A Collision Detection System for a Mobile Robot Inspired by the Locust Visual System

paper_content:

The lobula giant movement detector (LGMD) is an identified neuron in the locust brain that responds most strongly to the image of an approaching object such as a predator. A computational neural network model based on the structure of the LGMD and its afferent inputs is also able to detect approaching objects. In order for the LGMD network to be used as a robust collision detector for robotic applications, we proposed a new mechanism to enhance the feature of colliding objects before the excitations are gathered by LGMD cell. The new model favours grouped excitation but tends to ignore isolated excitation with selective passing coefficients. Experiments with a Khepera robot showed the proposed collision detector worked in real time in an arena surrounded with blocks.
---
paper_title: Reactive direction control for a mobile robot: a locust-like control of escape direction emerges when a bilateral pair of model locust visual neurons are integrated

paper_content:

Locusts possess a bilateral pair of uniquely identifiable visual neurons that respond vigorously to the image of an approaching object. These neurons are called the lobula giant movement detectors (LGMDs). The locust LGMDs have been extensively studied and this has lead to the development of an LGMD model for use as an artificial collision detector in robotic applications. To date, robots have been equipped with only a single, central artificial LGMD sensor, and this triggers a non-directional stop or rotation when a potentially colliding object is detected. Clearly, for a robot to behave autonomously, it must react differently to stimuli approaching from different directions. In this study, we implement a bilateral pair of LGMD models in Khepera robots equipped with normal and panoramic cameras. We integrate the responses of these LGMD models using methodologies inspired by research on escape direction control in cockroaches. Using `randomised winner-take-all' or `steering wheel' algorithms for LGMD model integration, the Khepera robots could escape an approaching threat in real time and with a similar distribution of escape directions as real locusts. We also found that by optimising these algorithms, we could use them to integrate the left and right DCMD responses of real jumping locusts offline and reproduce the actual escape directions that the locusts took in a particular trial. Our results significantly advance the development of an artificial collision detection and evasion system based on the locust LGMD by allowing it reactive control over robot behaviour. The success of this approach may also indicate some important areas to be pursued in future biological research.
---
paper_title: Modeling disinhibition within a layered structure of the LGMD neuron

paper_content:

Due to their relatively simple nervous system, insects are an excellent way through which we can investigate how visual information is acquired and processed in order to trigger specific behaviours, as flight stabilization, flying speed adaptation, collision avoidance responses, among others. From the behaviours previously mentioned, we are particularly interested in visually evoked collision avoidance responses. These behaviors are, by necessity, fast and robust, making them excellent systems to study the neural basis of behavior. On the other hand, artificial collision avoidance is a complex task, in which the algorithms used need to be fast to process the captured data and then perform real time decisions. Consequently, neurorobotic models may provide a foundation for the development of more effective and autonomous devices. In this paper, we will focus our attention in the Lobula Giant Movement Detector (LGMD), which is a visual neuron, located in the third layer of the locust optic lobe, that responds selectively to approaching objects, being responsible for trigger collision avoidance maneuvers in locusts. This selectivity of the LGMD neuron to approaching objects seems to result from the dynamics of the network pre-synaptic to this neuron. Tipically, this modelation is done by a conventional Difference of Gaussians (DoG) filter. In this paper, we propose the integration of a different model, an Inversed Difference of Gaussians (IDoG) filter, which preserves the different level of brightness in the captured image, enhancing the contrast at the edges. This change is expected to increase the performance of the LGMD model. Finally, a comparative analysis of both modelations, as well as its effect in the final response of the LGMD neuron, will be performed.
---
paper_title: Time-dependent activation of feedforward inhibition in a looming-sensitive neuron

paper_content:

The lobula giant movement detector (LGMD) is an identified neuron in the locust visual system that responds preferentially to objects approaching on a collision course with the animal. For such looming stimuli, the LGMD firing rate gradually increases, peaks, and decays toward the end of approach. The LGMD receives both excitatory and feed-forward inhibitory inputs on distinct branches of its dendritic tree, but little is known about the contribution of feed-forward inhibition to its response properties. We used picrotoxin, a chloride channel blocker, to selectively block feed-forward inhibition to the LGMD. We then computed differences in firing rate and membrane potential between control and picrotoxin conditions to study the activation of feed-forward inhibition. For looming stimuli, a significant activation of inhibition was observed early, as objects exceeded on average ∼23° in angular extent at the retina. Inhibition then increased in parallel with excitation over the remainder of approach trials. Experiments in which the final angular size of the approaching objects was systematically varied revealed that the relative activation of excitation and inhibition remains well balanced over most of the course of looming trials. Feed-forward inhibition actively contributed to the termination of the response to approaching objects and was particularly effective for large or slowly moving objects. Suddenly appearing and receding objects activated excitation and feed-forward inhibition nearly simultaneously, in contrast to looming stimuli. Under these conditions, the activation of excitation and feed-forward inhibition was weaker than for approaching objects, suggesting that both are preferentially tuned to approaching objects. These results support a phenomenological model of multiplication within the LGMD and provide new constraints for biophysical models of its responses to looming and receding stimuli.
---
paper_title: Performance of an insect-inspired target tracker in natural conditions

paper_content:

Robust and efficient target-tracking algorithms embedded on moving platforms, are a requirement for many computer vision and robotic applications. However, deployment of a real-time system is challenging, even with the computational power of modern hardware. As inspiration, we look to biological lightweight solutions-lightweight and low-powered flying insects. For example, dragonflies pursue prey and mates within cluttered, natural environments, deftly selecting their target amidst swarms. In our laboratory, we study the physiology and morphology of dragonfly 'small target motion detector' neurons likely to underlie this pursuit behaviour. Here we describe our insect-inspired tracking model derived from these data and compare its efficacy and efficiency with state-of-the-art engineering models. For model inputs, we use both publicly available video sequences, as well as our own task-specific dataset (small targets embedded within natural scenes). In the context of the tracking problem, we describe differences in object statistics within the video sequences. For the general dataset, our model often locks on to small components of larger objects, tracking these moving features. When input imagery includes small moving targets, for which our highly nonlinear filtering is matched, the robustness outperforms state-of-the-art trackers. In all scenarios, our insect-inspired tracker runs at least twice the speed of the comparison algorithms. (Less)
---
paper_title: Processing of figure and background motion in the visual system of the fly

paper_content:

The visual system of the fly is able to extract different types of global retinal motion patterns as may be induced on the eyes during different flight maneuvers and to use this information to control visual orientation. The mechanisms underlying these tasks were analyzed by a combination of quantitative behavioral experiments on tethered flying flies (Musca domestica) and model simulations using different conditions of oscillatory large-field motion and relative motion of different segments of the stimulus pattern. Only torque responses about the vertical axis of the animal were determined. The stimulus patterns consisted of random dot textures ("Julesz patterns") which could be moved either horizontally or vertically. Horizontal rotatory large-field motion leads to compensatory optomotor turning responses, which under natural conditions would tend to stabilize the retinal image. The response amplitude depends on the oscillation frequency: It is much larger at low oscillation frequencies than at high ones. When an object and its background move relative to each other, the object may, in principle, be discriminated and then induce turning responses of the fly towards the object. However, whether the object is distinguished by the fly depends not only on the phase relationship between object and background motion but also on the oscillation frequency. At all phase relations tested, the object is detected only at high oscillation frequencies. For the patterns used here, the turning responses are only affected by motion along the horizontal axis of the eye. No influences caused by vertical motion could be detected. The experimental data can be explained best by assuming two parallel control systems with different temporal and spatial integration properties: TheLF-system which is most sensitive to coherent rotatory large-field motion and mediates compensatory optomotor responses mainly at low oscillation frequencies. In contrast, theSF-system is tuned to small-field and relative motion and thus specialized to discriminate a moving object from its background; it mediates turning responses towards objects mainly at high oscillation frequencies. The principal organization of the neural networks underlying these control systems could be derived from the characteristic features of the responses to the different stimulus conditions. The input to the model circuits responsible for the characteristic sensitivity of the SF-system to small-field and relative motion is provided by retinotopic arrays of local movement detectors. The movement detectors are integrated by a large-field element, the output cell of the network. The synapses between the detectors and the output cells have nonlinear transmission characteristics. Another type of large-field elements ("pool cells") which respond to motion in front of both eyes and have characteristic direction selectivities are assumed to interact with the local movement detector channels by inhibitory synapses of the shunting type, before the movement detectors are integrated by the output cells. The properties of the LF-system can be accounted for by similar model circuits which, however, differ with respect to the transmission characteristic of the synapses between the movement detectors and the output cell; moreover, their pool cells are only monocular. This type of network, however, is not necessary to account for the functional properties of the LF-system. Instead, intrinsic properties of single neurons may be sufficient. Computer simulations of the postulated mechanisms of the SF-and LF-system reveal that these can account for the specific features of the behavioral responses under quite different conditions of coherent large-field motion and relative motion of different pattern segments.
---
paper_title: Insect Detection of Small Targets Moving in Visual Clutter

paper_content:

Detection of targets that move within visual clutter is a common task for animals searching for prey or conspecifics, a task made even more difficult when a moving pursuer needs to analyze targets against the motion of background texture (clutter). Despite the limited optical acuity of the compound eye of insects, this challenging task seems to have been solved by their tiny visual system. Here we describe neurons found in the male hoverfly, Eristalis tenax, that respond selectively to small moving targets. Although many of these target neurons are inhibited by the motion of a background pattern, others respond to target motion within the receptive field under a surprisingly large range of background motion stimuli. Some neurons respond whether or not there is a speed differential between target and background. Analysis of responses to very small targets (smaller than the size of the visual field of single photoreceptors) or those targets with reduced contrast shows that these neurons have extraordinarily high contrast sensitivity. Our data suggest that rejection of background motion may result from extreme selectivity for small targets contrasting against local patches of the background, combined with this high sensitivity, such that background patterns rarely contain features that satisfactorily drive the neuron.
---
paper_title: A Model for the Detection of Moving Targets in Visual Clutter Inspired by Insect Physiology

paper_content:

We present a computational model for target discrimination based on intracellular recordings from neurons in the fly visual system. Determining how insects detect and track small moving features, often against cluttered moving backgrounds, is an intriguing challenge, both from a physiological and a computational perspective. Previous research has characterized higher-order neurons within the fly brain, known as 'small target motion detectors' (STMD), that respond robustly to moving features, even when the velocity of the target is matched to the background (i.e. with no relative motion cues). We recorded from intermediate-order neurons in the fly visual system that are well suited as a component along the target detection pathway. This full-wave rectifying, transient cell (RTC) reveals independent adaptation to luminance changes of opposite signs (suggesting separate ON and OFF channels) and fast adaptive temporal mechanisms, similar to other cell types previously described. From this physiological data we have created a numerical model for target discrimination. This model includes nonlinear filtering based on the fly optics, the photoreceptors, the 1(st) order interneurons (Large Monopolar Cells), and the newly derived parameters for the RTC. We show that our RTC-based target detection model is well matched to properties described for the STMDs, such as contrast sensitivity, height tuning and velocity tuning. The model output shows that the spatiotemporal profile of small targets is sufficiently rare within natural scene imagery to allow our highly nonlinear 'matched filter' to successfully detect most targets from the background. Importantly, this model can explain this type of feature discrimination without the need for relative motion cues.
---
paper_title: LGMD-based bio-inspired algorithm for detecting risk of collision of a road vehicle

paper_content:

LGMD (Lobula Giant Movement Detector) is part of a visual system of a locust, used to detect and evade approaching predators. Similar algorithm can be used in man-made systems like autonomous robots or safety systems in vehicles for detecting objects approaching on collision course. In this article usage of LGMD-based algorithms in road vehicles is investigated and a new solution is proposed. Video stream recorded from a moving vehicle inherently contains a lot of movement of the background due to vibrations and due to turning of the vehicle. This cause high output of LGMD and could trigger false alarms. The new approach, proposed in this article, enhances LGMD information and adds estimated expansion of objects in X and Y direction. Combination of all three sources of information gives a very good estimate of risk of collision. The new algorithm was developed using a reference set of test videos, where only one parameter was changed. Algorithm was tested with videos recorded from a moving vehicle in normal traffic and on the test ground, including real collisions in carton target at different speeds. Time between triggering of the alert and actual collision was measured.
---
paper_title: A modified neural network model for Lobula Giant Movement Detector with additional depth movement feature

paper_content:

The Lobula Giant Movement Detector (LGMD) is a wide-field visual neuron that is located in the Lobula layer of the Locust nervous system. The LGMD increases its firing rate in response to both the velocity of the approaching object and its proximity. It has been found that it can respond to looming stimuli very quickly and can trigger avoidance reactions whenever a rapidly approaching object is detected. It has been successfully applied in visual collision avoidance systems for vehicles and robots. This paper proposes a modified LGMD model that provides additional movement depth direction information. The proposed model retains the simplicity of the previous neural network model, adding only a few new cells. It has been tested on both simulated and recorded video data sets. The experimental results shows that the modified model can very efficiently provide stable information on the depth direction of movement.
---
paper_title: Neural circuit tuning fly visual interneurons to motion of small objects. I. Dissection of the circuit by pharmacological and photoinactivation techniques

paper_content:

1. Visual interneurons tuned to the motion of small objects are found in many animal species and are assumed to be the neuronal basis of figure-ground discrimination by relative motion. A well-examined example is the FD1-cell in the third visual neuropil of blowflies. This cell type responds best to motion of small objects. Motion of extended patterns elicits only small responses. As a neuronal mechanism that leads to such a response characteristic, it was proposed that the FD1-cell is inhibited by the two presumably GABAergic and, thus, inhibitory CH-cells, the VCH- and the DCH-cell. The CH-cells respond best to exactly that type of motion by which the activity of the FD1-cell is reduced. The hypothesis that the CH-cells inhibit the FD1-cell and, thus, mediate its selectivity to small moving objects was tested by ablating the CH-cells either pharmacologically or by photoinactivation. ::: 2. After application of the gamma-aminobutyric acid (GABA) antagonist picrotoxinin, the FD1-cell responds more strongly to large-field than to small-field motion, i.e., it has lost its small-field selectivity. This suggests that the tuning of the FD1-cell to small moving objects relies on a GABAergic mechanism and, thus, most likely on the CH-cells. ::: 3. The role of each CH-cell for small-field tuning was determined by inactivating them individually. They were injected with a fluorescent dye and then ablated by laser illumination. Only photoinactivation of the VCH-cell liminated the specific selectivity of the FD1-cell for small-field motion. Ablation of the DCH-cell did not significantly change the response characteristic of the FD1-cell. This reveals the important role of the VCH-cells in mediating the characteristic sensitivity of the FD1-cell to motion of small objects. ::: 4. The FD1-cell is most sensitive to motion of small objects in the ventral part of the ipsilateral visual field, whereas motion in the dorsal part influences the cell only weakly. This specific feature fits well to the sensitivity of the VCH-cell to ipsilateral motion that is most pronounced in the ventral part of the visual field. The spatial sensitivity distribution of the FD1-cell matches also the characteristics of figure-ground discrimination and fixation behavior.
---
paper_title: Near range path navigation using LGMD visual neural networks

paper_content:

In this paper, we proposed a method for near range path navigation for a mobile robot by using a pair of biologically inspired visual neural network - lobula giant movement detector (LGMD). In the proposed binocular style visual system, each LGMD processes images covering a part of the wide field of view and extracts relevant visual cues as its output. The outputs from the two LGMDs are compared and translated into executable motor commands to control the wheels of the robot in real time. Stronger signal from the LGMD in one side pushes the robot away from this side step by step; therefore, the robot can navigate in a visual environment naturally with the proposed vision system. Our experiments showed that this bio-inspired system worked well in different scenarios.
---
paper_title: The centrifugal horizontal cells in the lobula plate of the blowfly, Phaenicia sericata

paper_content:

Intracellular recordings combined with iontophoretic injection of Procion Yellow M4RAN were used to study the anatomy and physiology of the centrifugal horizontal cells (CH-cells) in the lobula plate of the blowfly, Phaenicia sericata. ::: ::: Anatomy: The CH-cells comprise a set of two homolateral, giant visual interneurones (DCH, VCH) at the rostral surface of each lobula plate. Their extensive arborizations in the lobula plate possess bulbous swellings (boutons terminaux). The arborization of one cell (DCH) covers the dorsal, and the arborization of the other cell (VCH) the ventral half of the lobula plate. Their axons run jointly with those of the horizontal cells through the chiasma internum and the optic peduncle. Their protocerebral arborization possesses spines; they form a dense network together with the axonal arborization of the horizontal cells, a second type of giant homolateral cell most sensitive to horizontal motion. The protocerebral arborization of the CH-cells gives rise to a cell body fibre which traverses the protocerebrum dorsally to the oesophageal canal. The cell body lies on the contralateral side laterally and slightly dorsally to the oesophageal canal in the frontal cell body layer. ::: ::: Physiology: The CH-cells respond with graded potentials to rotatory movements of their surround. Cells in the right lobula plate respond with excitation (excitatory postsynaptic potentials, membrane depolarization) to clockwise motion (contralateral regressive, ipsilateral progressive), and with inhibition (inhibitory postsynaptic potentials, membrane hyperpolarization) to counterclockwise motion in either or both receptive fields; CH-cells respond to motion presented to the ipsilateral and/or contralateral eye. Cells of the left lobula plate respond correspondingly to the reverse directions of motion. Vertical pattern motion and stationary patterns are ineffective. ::: ::: The heterolateral H1-neurone elicits excitatory postsynaptic potentials in the DCH-cell; these postsynaptic potentials are tightly correlated 1:1 to the preceding H1-action potential. The delay between the peak of the action potential and the beginning of the DCH-postsynaptic potential is 1.15 msec, agreeing very well with the value reported previously for the blowfly, Calliphora (Hausen, 1976a). The synaptic input and output connections of the CH-cells are discussed.
---
paper_title: A bio-inspired visual collision detection mechanism for cars: Combining insect inspired neurons to create a robust system

paper_content:

The lobula giant movement detector (LGMD) of locusts is a visual interneuron that responds with an increasing spike frequency to an object approaching on a direct collision course. Recent studies involving the use of LGMD models to detect car collisions showed that it could detect collisions, but the neuron produced collision alerts to non-colliding, translating, stimuli in many cases. This study presents a modified model to address these problems. It shows how the neurons pre-synaptic to the LGMD show a remarkable ability to filter images, and only colliding and translating stimuli produce excitation in the neuron. It then integrates the LGMD network with models based on the elementary movement detector (EMD) neurons from the fly visual system, which are used to analyse directional excitation patterns in the biologically filtered images. Combining the information from the LGMD neuron and four directionally sensitive neurons produces a robust collision detection system for a wide range of automotive test situations.
---
paper_title: Properties of neuronal facilitation that improve target tracking in natural pursuit simulations

paper_content:

Although flying insects have limited visual acuity (approx. 1°) and relatively small brains, many species pursue tiny targets against cluttered backgrounds with high success. Our previous computational model, inspired by electrophysiological recordings from insect 'small target motion detector' (STMD) neurons, did not account for several key properties described from the biological system. These include the recent observations of response 'facilitation' (a slow build-up of response to targets that move on long, continuous trajectories) and 'selective attention', a competitive mechanism that selects one target from alternatives. Here, we present an elaborated STMD-inspired model, implemented in a closed loop target-tracking system that uses an active saccadic gaze fixation strategy inspired by insect pursuit. We test this system against heavily cluttered natural scenes. Inclusion of facilitation not only substantially improves success for even short-duration pursuits, but it also enhances the ability to 'attend' to one target in the presence of distracters. Our model predicts optimal facilitation parameters that are static in space and dynamic in time, changing with respect to the amount of background clutter and the intended purpose of the pursuit. Our results provide insights into insect neurophysiology and show the potential of this algorithm for implementation in artificial visual systems and robotic applications.
---
paper_title: Responses to object approach by a wide field visual neurone, the LGMD2 of the locust: Characterization and image cues

paper_content:

The LGMD2 belongs to a group of giant movement-detecting neurones which have fan-shaped arbors in the lobula of the locust optic lobe and respond to movements of objects. One of these neurones, the LGMD1, has been shown to respond directionally to movements of objects in depth, generating vigorous, maintained spike discharges during object approach. Here we compare the responses of the LGMD2 neurone with those of the LGMD1 to simulated movements of objects in depth and examine different image cues which could allow the LGMD2 to distinguish approaching from receding objects. In the absence of stimulation, the LGMD2 has a resting discharge of 10–40 spikes s−1 compared with <1 spike s−1 for the LGMD1. The most powerful excitatory stimulus for the LGMD2 is a dark object approaching the eye. Responses to approaching objects are suppressed by wide field movements of the background. Unlike the LGMD1, the LGMD2 is not excited by the approach of light objects; it specifically responds to movement of edges in the light to dark direction. Both neurones rely on the same monocular image cues to distinguish approaching from receding objects: an increase in the velocity with which edges of images travel over the eye; and an increase in the extent of edges in the image during approach.
---
paper_title: Collision selective LGMDs neuron models research benefits from a vision-based autonomous micro robot

paper_content:

The developments of robotics inform research across a broad range of disciplines. In this paper, we will study and compare two collision selective neuron models via a vision-based autonomous micro robot. In the locusts' visual brain, two Lobula Giant Movement Detectors (LGMDs), i.e. LGMD1 and LGMD2, have been identified as looming sensitive neurons responding to rapidly expanding objects, yet with different collision selectivity. Both neurons have been modeled and successfully applied in robotic vision system for perceiving potential collisions in an efficient and reliable manner. In this research, we conduct binocular neuronal models, for the first time combining the functionalities of LGMD1 and LGMD2 neurons, in the visual modality of a ground mobile robot. The results of systematic on-line experiments demonstrated three contributions of this research: (1) The arena tests involving multiple robots verified the effectiveness and robustness of a reactive motion control strategy via integrating a bilateral pair of LGMD1 and LGMD2 models for collision detection in dynamic scenarios. (2) We pinpointed the different collision selectivity between LGMD1 and LGMD2 neuron models, which fulfill corresponding biological research. (3) The utilized micro robot may also benefit researches on other embedded vision systems as well as swarm robotics.
---
paper_title: Computation of Object Approach by a Wide-Field, Motion-Sensitive Neuron

paper_content:

The lobula giant motion detector (LGMD) in the locust visual system is a wide-field, motion-sensitive neuron that responds vigorously to objects approaching the animal on a collision course. We investigated the computation performed by LGMD when it responds to approaching objects by recording the activity of its postsynaptic target, the descending contralateral motion detector (DCMD). In each animal, peak DCMD activity occurred a fixed delay δ (15 ≤ δ ≤ 35 msec) after the approaching object had reached a specific angular threshold θthres on the retina (15° ≤ θthres ≤ 40°). θthres was independent of the size or velocity of the approaching object. This angular threshold computation was quite accurate: the error of LGMD and DCMD in estimating θthres(3.1–11.9°) corresponds to the angular separation between two and six ommatidia at each edge of the expanding object on the locust retina. It was also resistant to large amplitude changes in background luminosity, contrast, and body temperature. Using several experimentally derived assumptions, the firing rate of LGMD and DCMD could be shown to depend on the product ψ(t − δ) · e^(−αθ(t−δ0)), where θ(t) is the angular size subtended by the object during approach, ψ(t) is the angular edge velocity of the object and the constant, and α is related to the angular threshold size [α = 1/tan(θthres^(/2))]. Because LGMD appears to receive distinct input projections, respectively motion- and size-sensitive, this result suggests that a multiplication operation is implemented by LGMD. Thus, LGMD might be an ideal model to investigate the biophysical implementation of a multiplication operation by single neurons.
---
paper_title: Bio-inspired collision detector with enhanced selectivity for ground robotic vision system

paper_content:

There are many ways of building collision-detecting systems. In this paper, we propose a novel collision selective visual neural network inspired by LGMD2 neurons in the juvenile locusts. Such collision-sensitive neuron matures early in the ﬁrst-aged or even hatching locusts, and is only selective to detect looming dark objects against bright background in depth, represents swooping predators, a situation which is similar to ground robots or vehicles. However, little has been done on modeling LGMD2, let alone its potential applications in robotics and other vision-based areas. Compared to other collision detectors, our major contributions are ﬁrst, enhancing the collision selectivity in a bio-inspired way, via constructing a computing efﬁcient visual sensor, and realizing the revealed speciﬁc characteristic sofLGMD2. Second, we applied the neural network to help rearrange path navigation of an autonomous ground miniature robot in an arena. We also examined its neural properties through systematic experiments challenged against image streams from a visual sensor of the micro-robot.
---
paper_title: A modified model for the Lobula Giant Movement Detector and its FPGA implementation

paper_content:

Bio-inspired vision sensors are particularly appropriate candidates for navigation of vehicles or mobile robots due to their computational simplicity, allowing compact hardware implementations with low power dissipation. The Lobula Giant Movement Detector (LGMD) is a wide-field visual neuron located in the Lobula layer of the Locust nervous system. The LGMD increases its firing rate in response to both the velocity of an approaching object and the proximity of this object. It has been found that it can respond to looming stimuli very quickly and trigger avoidance reactions. It has been successfully applied in visual collision avoidance systems for vehicles and robots. This paper introduces a modified neural model for LGMD that provides additional depth direction information for the movement. The proposed model retains the simplicity of the previous model by adding only a few new cells. It has been simplified and implemented on a Field Programmable Gate Array (FPGA), taking advantage of the inherent parallelism exhibited by the LGMD, and tested on real-time video streams. Experimental results demonstrate the effectiveness as a fast motion detector.
---
paper_title: Multiplicative computation in a visual neuron sensitive to looming

paper_content:

Multiplicative operations are important in sensory processing, but their biophysical implementation remains largely unknown. We investigated an identified neuron (the lobula giant movement detector, LGMD, of locusts) whose output firing rate in response to looming visual stimuli has been described by two models, one of which involves a multiplication. In this model, the LGMD multiplies postsynaptically two inputs (one excitatory, one inhibitory) that converge onto its dendritic tree; in the other model, inhibition is presynaptic to the LGMD. By using selective activation and inactivation of pre- and postsynaptic inhibition, we show that postsynaptic inhibition has a predominant role, suggesting that multiplication is implemented within the neuron itself. Our pharmacological experiments and measurements of firing rate versus membrane potential also reveal that sodium channels act both to advance the response of the LGMD in time and to map membrane potential to firing rate in a nearly exponential manner. These results are consistent with an implementation of multiplication based on dendritic subtraction of two converging inputs encoded logarithmically, followed by exponentiation through active membrane conductances.
---
paper_title: Neural circuit tuning fly visual neurons to motion of small objects. II. Input organization of inhibitory circuit elements revealed by electrophysiological and optical recording techniques

paper_content:

1. The FD1-cell in the visual system of the fly is an identified visual interneuron that is specifically tuned to motion of small objects. In the companion paper it was shown that this response pro...
---
paper_title: Collision detection in complex dynamic scenes using an LGMD-based visual neural network with feature enhancement

paper_content:

The lobula giant movement detector (LGMD) is an identified neuron in the locust brain that responds most strongly to the images of an approaching object such as a predator. Its computational model can cope with unpredictable environments without using specific object recognition algorithms. In this paper, an LGMD-based neural network is proposed with a new feature enhancement mechanism to enhance the expanded edges of colliding objects via grouped excitation for collision detection with complex backgrounds. The isolated excitation caused by background detail will be filtered out by the new mechanism. Offline tests demonstrated the advantages of the presented LGMD-based neural network in complex backgrounds. Real time robotics experiments using the LGMD-based neural network as the only sensory system showed that the system worked reliably in a wide range of conditions; in particular, the robot was able to navigate in arenas with structured surrounds and complex backgrounds
---
paper_title: Invariance of Angular Threshold Computation in a Wide-Field Looming-Sensitive Neuron

paper_content:

The lobula giant motion detector (LGMD) is a wide-field bilateral visual interneuron in North American locusts that acts as an angular threshold detector during the approach of a solid square along a trajectory perpendicular to the long axis of the animal (Gabbiani et al., 1999a). We investigated the dependence of this angular threshold computation on several stimulus parameters that alter the spatial and temporal activation patterns of inputs onto the dendritic tree of the LGMD, across three locust species. The same angular threshold computation was implemented by LGMD in all three species. The angular threshold computation was invariant to changes in target shape (from solid squares to solid discs) and to changes in target texture (checkerboard and concentric patterns). Finally, the angular threshold computation did not depend on object approach angle, over at least 135 degrees in the horizontal plane. A two-dimensional model of the responses of the LGMD based on linear summation of motion-related excitatory and size-dependent inhibitory inputs successfully reproduced the experimental results for squares and discs approaching perpendicular to the long axis of the animal. Linear summation, however, was unable to account for invariance to object texture or approach angle. These results indicate that LGMD is a reliable neuron with which to study the biophysical mechanisms underlying the generation of complex but invariant visual responses by dendritic integration. They also suggest that invariance arises in part from non-linear integration of excitatory inputs within the dendritic tree of the LGMD.
---
paper_title: A Bio-inspired Collision Detector for Small Quadcopter

paper_content:

The sense and avoid capability enables insects to fly versatilely and robustly in dynamic and complex environment. Their biological principles are so practical and efficient that inspired we human imitating them in our flying machines. In this paper, we studied a novel bio-inspired collision detector and its application on a quadcopter. The detector is inspired from Lobula giant movement detector (LGMD) neurons in the locusts, and modeled into an STM32F407 Microcontroller Unit (MCU). Compared to other collision detecting methods applied on quadcopters, we focused on enhancing the collision accuracy in a bio-inspired way that can considerably increase the computing efficiency during an obstacle detecting task even in complex and dynamic environment. We designed the quadcopter's responding operation to imminent collisions and tested this bio-inspired system in an indoor arena. The observed results from the experiments demonstrated that the LGMD collision detector is feasible to work as a vision module for the quadcopter's collision avoidance task.
---
paper_title: How accurate need sensory coding be for behaviour? Experiments using a mobile robot

paper_content:

Abstract This paper argues that for those neuronal systems which control behaviour, reliable responses are more appropriate than precise responses. We illustrate this argument using a mobile robot controlled by the responses of a neuronal model of the locust LGMD system, a visual system which responds to looming objects. Our experiments show that although the responses of the model LGMD vary widely as the robot approaches obstacles, they still trigger avoidance responses.
---
paper_title: Local and large-range inhibition in feature detection.

paper_content:

Lateral inhibition is perhaps the most ubiquitous of neuronal mechanisms, having been demonstrated in early stages of processing in many different sensory pathways of both mammals and invertebrates. Recent work challenges the long-standing view that assumes that similar mechanisms operate to tune neuronal responses to higher order properties. Scant evidence for lateral inhibition exists beyond the level of the most peripheral stages of visual processing, leading to suggestions that many features of the tuning of higher order visual neurons can be accounted for by the receptive field and other intrinsic coding properties of visual neurons. Using insect target neurons as a model, we present unequivocal evidence that feature tuning is shaped not by intrinsic properties but by potent spatial lateral inhibition operating well beyond the first stages of visual processing. In addition, we present evidence for a second form of higher-order spatial inhibition--a long-range interocular transfer of information that we argue serves a role in establishing interocular rivalry and thus potentially a neural substrate for directing attention to single targets in the presence of distracters. In so doing, we demonstrate not just one, but two levels of spatial inhibition acting beyond the level of peripheral processing.
---
paper_title: Collision avoidance using a model of the locust LGMD neuron

paper_content:

Abstract The lobula giant movement detector (LGMD) system in the locust responds selectively to objects approaching the animal on a collision course. In earlier work we have presented a neural network model based on the LGMD system which shared this preference for approaching objects. We have extended this model in order to evaluate its responses in a real-world environment using a miniature mobile robot. This extended model shows reliable obstacle detection over an eight-fold range of speeds, and raises interesting questions about basic properties of the biological system.
---
paper_title: A collision avoidance model based on the Lobula giant movement detector (LGMD) neuron of the locust

paper_content:

In insects, we can find very complex and compact neural structures that are task specific. These neural structures allow them to perform complex tasks such as visual navigation, including obstacle avoidance, landing, self-stabilization, etc. Obstacle avoidance is fundamental for successful navigation, and it can be combined with more systems to make up more complex behaviors. In this paper, we present a model for collision avoidance based on the Lobula giant movement detector (LGMD) cell of the locust. This is a wide-field visual neuron that responds to looming stimuli and that can trigger avoidance reactions whenever a rapidly approaching object is detected. Here, we present result based on both an offline study of the model and its application to a flying robot.
---
paper_title: On the neuronal basis of figure-ground discrimination by relative motion in the visual system of the fly. III: Possible input circuitries and behavioural significance of the FD-cells

paper_content:

It has been concluded in the preceding papers (Egelhaaf, 1985a, b) that two functional classes of output elements of the visual ganglia might be involved in figure-ground discrimination by relative motion in the fly: The Horizontal Cells which respond best to the motion of large textured patterns and the FD-cells which are most sensitive to small moving objects. In this paper it is studied by computer simulations (1) in what way the input circuitry of the FD-cells might be organized and (2) the role the FD-cells play in figure-ground discrimination. The characteristic functional properties of the FD-cells can be explained by various alternative model networks. In all models the main input to the FD-cells is formed by two retinotopic arrays of small-field elementary movement detectors, responding to either front-to-back or back-to-front motion. According to their preferred direction of motion the FD-cells are excited by one of these movement detector classes and inhibited by the other. The synaptic transmission between the movement detectors and the FD-cells is assumed to be non-linear. It is a common property of all these model circuits that the inhibition of the FD-cells induced by large-field motion is mediated by pool cells which cover altogether the entire horizontal extent of the visual field of both eyes. These pool cells affect the response of the FD-cells either by pre- or postsynaptic shunting inhibition. Depending on the FD-cell under consideration, the pool cells are directionally selective for motion or sensitive to motion in either horizontal direction. The role the FD-cells and the Horizontal Cells are likely to play in figure-ground discrimination can be demonstrated by computer simulations of a composite neuronal model consisting of the model circuits for these cell types. According to their divergent spatial integration properties they perform different tasks in figure-ground discrimination: Whereas the Horizontal Cells mainly mediate information on wide-field motion, the FD-cells are selectively tuned to efficient detection of relatively small targets. Both cell classes together appear to be sufficient to account for figure-ground discrimination as it has been shown by analysis at the behavioural level.
---
paper_title: Neural network based on the input organization of an identified neuron signaling impending collision

paper_content:

1. We describe a four-layered neural network (Fig. 1), based on the input organization of a collision signaling neuron in the visual system of the locust, the lobula giant movement detector (LGMD)....
---
paper_title: An obstacle avoidance method for two wheeled mobile robot

paper_content:

In this paper, we proposed an obstacle avoidance method of mobile robot by using lobula giant movement detector (LGMD) method. The LGMD is an identified neuron in the locust brain that responds most strongly to the image of an approaching object such as a predator. As an assistance to the avoidance method, we add a distance measurement algorithm to LGMD method. The measurement algorithm is based on information from camera to detect obstacles and calculate the route for two wheeled mobile robot. Simulation result confirmed the effectiveness of the proposed method.
---
paper_title: Dynamic Range Enhance of Visual Sensor Circuits and Application for Multi-object Motion Detection

paper_content:

This paper presents a dynamic range enhanced visual sensor circuit and its application for multi-object detection based on LGMD (Lobula Giant Movement Detection) method. The proposed visual sensor incorporates a DR (Dynamic Range) extension circuit with the general pipeline AD Converter. Therefore little changes are needed to achieve the DR extension. Together with the modified LGMD model, our system would effectively enhance the reliability of motion detection compared with traditional motion detecting system.
---
paper_title: Localized direction selective responses in the dendrites of visual interneurons of the fly

paper_content:

BackgroundThe various tasks of visual systems, including course control, collision avoidance and the detection of small objects, require at the neuronal level the dendritic integration and subsequent processing of many spatially distributed visual motion inputs. While much is known about the pooled output in these systems, as in the medial superior temporal cortex of monkeys or in the lobula plate of the insect visual system, the motion tuning of the elements that provide the input has yet received little attention. In order to visualize the motion tuning of these inputs we examined the dendritic activation patterns of neurons that are selective for the characteristic patterns of wide-field motion, the lobula-plate tangential cells (LPTCs) of the blowfly. These neurons are known to sample direction-selective motion information from large parts of the visual field and combine these signals into axonal and dendro-dendritic outputs.ResultsFluorescence imaging of intracellular calcium concentration allowed us to take a direct look at the local dendritic activity and the resulting local preferred directions in LPTC dendrites during activation by wide-field motion in different directions. These 'calcium response fields' resembled a retinotopic dendritic map of local preferred directions in the receptive field, the layout of which is a distinguishing feature of different LPTCs.ConclusionsOur study reveals how neurons acquire selectivity for distinct visual motion patterns by dendritic integration of the local inputs with different preferred directions. With their spatial layout of directional responses, the dendrites of the LPTCs we investigated thus served as matched filters for wide-field motion patterns.
---
paper_title: Obstacle avoidance with LGMD neuron: Towards a neuromorphic UAV implementation

paper_content:

We present a neuromorphic adaptation of a spiking neural network model of the locust Lobula Giant Movement Detector (LGMD), which detects objects increasing in size in the field of vision (looming) and can be used to facilitate obstacle avoidance in robotic applications. Our model is constrained by the parameters of a mixed signal analog-digital neuromorphic device, developed by our group, and is driven by the output of a neuromorphic vision sensor. We demonstrate the performance of the model and how it may be used for obstacle avoidance on an unmanned areal vehicle (UAV).
---
paper_title: Shaping the collision selectivity in a looming sensitive neuron model with parallel ON and OFF pathways and spike frequency adaptation

paper_content:

Abstract Shaping the collision selectivity in vision-based artificial collision-detecting systems is still an open challenge. This paper presents a novel neuron model of a locust looming detector, i.e. the lobula giant movement detector (LGMD1), in order to provide effective solutions to enhance the collision selectivity of looming objects over other visual challenges. We propose an approach to model the biologically plausible mechanisms of ON and OFF pathways and a biophysical mechanism of spike frequency adaptation (SFA) in the proposed LGMD1 visual neural network. The ON and OFF pathways can separate both dark and light looming features for parallel spatiotemporal computations. This works effectively on perceiving a potential collision from dark or light objects that approach; such a bio-plausible structure can also separate LGMD1’s collision selectivity to its neighbouring looming detector — the LGMD2. The SFA mechanism can enhance the LGMD1’s collision selectivity to approaching objects rather than receding and translating stimuli, which is a significant improvement compared with similar LGMD1 neuron models. The proposed framework has been tested using off-line tests of synthetic and real-world stimuli, as well as on-line bio-robotic tests. The enhanced collision selectivity of the proposed model has been validated in systematic experiments. The computational simplicity and robustness of this work have also been verified by the bio-robotic tests, which demonstrates potential in building neuromorphic sensors for collision detection in both a fast and reliable manner.
---
paper_title: Dendro-Dendritic Interactions between Motion-Sensitive Large-Field Neurons in the Fly

paper_content:

For visual course control, flies rely on a set of motion-sensitive neurons called lobula plate tangential cells (LPTCs). Among these cells, the so-called CH (centrifugal horizontal) cells shape by their inhibitory action the receptive field properties of other LPTCs called FD (figure detection) cells specialized for figure-ground discrimination based on relative motion. Studying the ipsilateral input circuitry of CH cells by means of dual-electrode and combined electrical-optical recordings, we find that CH cells receive graded input from HS (large-field horizontal system) cells via dendro-dendritic electrical synapses. This particular wiring scheme leads to a spatial blur of the motion image on the CH cell dendrite, and, after inhibiting FD cells, to an enhancement of motion contrast. This could be crucial for enabling FD cells to discriminate object from self motion.
---
paper_title: Neural based obstacle avoidance with CPG controlled hexapod walking robot

paper_content:

In this work, we are proposing a collision avoidance system for a hexapod crawling robot based on the detection of intercepting objects using the Lobula giant movement detector (LGMD) connected directly to the locomotion control unit based on the Central pattern generator (CPG). We have designed and experimentally verified the proposed approach that maps the output of the LGMD directly on the locomotion control parameters of the CPG. The results of the experimental verification of the system with real mobile hexapod crawling robot support the feasibility of the proposed approach in collision avoidance scenarios.
---
paper_title: Multiplication and stimulus invariance in a looming-sensitive neuron

paper_content:

Multiplicative operations and invariance of neuronal responses are thought to play important roles in the processing of neural information in many sensory systems. Yet the biophysical mechanisms that underlie both multiplication and invariance of neuronal responses in vivo, either at the single cell or at the network level, remain to a large extent unknown. Recent work on an identified neuron in the locust visual system (the LGMD neuron) that responds well to objects looming on a collision course towards the animal suggests that this cell represents a good model to investigate the biophysical basis of multiplication and invariance at the single neuron level. Experimental and theoretical results are consistent with multiplication being implemented by subtraction of two logarithmic terms followed by exponentiation via active membrane conductances, according to a x 1/b = exp(log(a) - log(b)). Invariance appears to be in part due to non-linear integration of synaptic inputs within the dendritic tree of this neuron.
---
paper_title: Emergence of Multiplication in a Biophysical Model of a Wide-Field Visual Neuron for Computing Object Approaches: Dynamics, Peaks, & Fits

paper_content:

Many species show avoidance reactions in response to looming object approaches. In locusts, the corresponding escape behavior correlates with the activity of the lobula giant movement detector (LGMD) neuron. During an object approach, its firing rate was reported to gradually increase until a peak is reached, and then it declines quickly. The η-function predicts that the LGMD activity is a product between an exponential function of angular size exp(–Θ) and angular velocity Θ, and that peak activity is reached before time-to-contact (ttc). The η-function has become the prevailing LGMD model because it reproduces many experimental observations, and even experimental evidence for the multiplicative operation was reported. Several inconsistencies remain unresolved, though. Here we address these issues with a new model (ψ-model), which explicitly connects Θ and Θ to biophysical quantities. The ψ-model avoids biophysical problems associated with implementing exp(·), implements the multiplicative operation of η via divisive inhibition, and explains why activity peaks could occur after ttc. It consistently predicts response features of the LGMD, and provides excellent fits to published experimental data, with goodness of fit measures comparable to corresponding fits with the η-function.
---
paper_title: Simplified bionic solutions: a simple bio-inspired vehicle collision detection system

paper_content:

Modern cars are equipped with both active and passive sensor systems that can detect potential collisions. In contrast, locusts avoid collisions solely by responding to certain visual cues that are associated with object looming. In neurophysiological experiments, I investigated the possibility that the 'collision-detector neurons' of locusts respond to impending collisions in films recorded with dashboard cameras of fast driving cars. In a complementary modelling approach, I developed a simple algorithm to reproduce the neuronal response that was recorded during object approach. Instead of applying elaborate algorithms that factored in object recognition and optic flow discrimination, I tested the hypothesis that motion detection restricted to a 'danger zone', in which frontal collisions on the motorways are most likely, is sufficient to estimate the risk of a collision. Furthermore, I investigated whether local motion vectors, obtained from the differential excitation of simulated direction-selective networks, could be used to predict evasive steering maneuvers and prevent undesired responses to motion artifacts. The results of the study demonstrate that the risk of impending collisions in real traffic scenes is mirrored in the excitation of the collision-detecting neuron (DCMD) of locusts. The modelling approach was able to reproduce this neuronal response even when the vehicle was driving at high speeds and image resolution was low (about 200 × 100 pixels). Furthermore, evasive maneuvers that involved changing the steering direction and steering force could be planned by comparing the differences in the overall excitation levels of the simulated right and left direction-selective networks. Additionally, it was possible to suppress undesired responses of the algorithm to translatory movements, camera shake and ground shadows by evaluating local motion vectors. These estimated collision risk values and evasive steering vectors could be used as input for a driving assistant, converting the first into braking force and the latter into steering responses to avoid collisions. Since many processing steps were computed on the level of pixels and involved elements of direction-selective networks, this algorithm can be implemented in hardware so that parallel computations enhance the processing speed significantly.
---
paper_title: A neural model of the locust visual system for detection of object approaches with real-world scenes

paper_content:

In the central nervous systems of animals like pigeons and locusts, neurons were identified which signal objects approaching the animal on a direct collision course. Unraveling the neural circuitry for collision avoidance, and identifying the underlying computational principles, is promising for building vision-based neuromorphic architectures, which in the near future could find applications in cars or planes. At the present there is no published model available for robust detection of approaching objects under real-world conditions. Here we present a computational architecture for signalling impending collisions, based on known anatomical data of the locust \emph{lobula giant movement detector} (LGMD) neuron. Our model shows robust performance even in adverse situations, such as with approaching low-contrast objects, or with highly textured and moving backgrounds. We furthermore discuss which components need to be added to our model to convert it into a full-fledged real-world-environment collision detector. KEYWORDS: Locust, LGMD, collision detection, lateral inhibition, diffusion, ON-OFF-pathways, neuronal dynamics, computer vision, image processing
---
paper_title: An autonomous robot inspired by insect neurophysiology pursues moving features in natural environments

paper_content:

Objective. Many computer vision and robotic applications require the implementation of robust and efficient target-tracking algorithms on a moving platform. However, deployment of a real-time system is challenging, even with the computational power of modern hardware. Lightweight and low-powered flying insects, such as dragonflies, track prey or conspecifics within cluttered natural environments, illustrating an efficient biological solution to the target-tracking problem. Approach. We used our recent recordings from 'small target motion detector' neurons in the dragonfly brain to inspire the development of a closed-loop target detection and tracking algorithm. This model exploits facilitation, a slow build-up of response to targets which move along long, continuous trajectories, as seen in our electrophysiological data. To test performance in real-world conditions, we implemented this model on a robotic platform that uses active pursuit strategies based on insect behaviour. Main results. Our robot performs robustly in closed-loop pursuit of targets, despite a range of challenging conditions used in our experiments; low contrast targets, heavily cluttered environments and the presence of distracters. We show that the facilitation stage boosts responses to targets moving along continuous trajectories, improving contrast sensitivity and detection of small moving targets against textured backgrounds. Moreover, the temporal properties of facilitation play a useful role in handling vibration of the robotic platform. We also show that the adoption of feed-forward models which predict the sensory consequences of self-movement can significantly improve target detection during saccadic movements. Significance. Our results provide insight into the neuronal mechanisms that underlie biological target detection and selection (from a moving platform), as well as highlight the effectiveness of our bio-inspired algorithm in an artificial visual system.
---
paper_title: Computational model of the LGMD neuron for automatic collision detection

paper_content:

In many animal species it is essential to recognize approach predators from complex, dynamic visual scenes and timely initiate escape behavior. Such sophisticated behaviours are often achieved with low neuronal complexity, such as in locusts, suggesting that emulating these biological models in artificial systems would enable the generation of similar complex behaviours with low computational overhead. On the other hand, artificial collision detection is a complex task that requires both real time data acquisition and important features extraction from a captured image. In order to accomplish this task, the algorithms used need to be fast to process the captured data and then perform real time decisions. Taking into account the previous considerations, neurorobotic models may provide a foundation for the development of more effective and autonomous devices/robots, based on an improved understanding of the biological basis of adaptive behavior. In this paper, we make a comparative analysis between the new computational model of a locust looming-detecting pathway and the model previously proposed by us. The obtained results proved the improvement provided by the pixel remapping in the model performance.
---
paper_title: Biologically Inspired Feature Detection Using Cascaded Correlations of off and on Channels

paper_content:

Abstract Flying insects are valuable animal models for elucidating computational processes underlying visual motion detection. For example, optical flow analysis by wide-field motion processing neurons in the insect visual system has been investigated from both behavioral and physiological perspectives [1]. This has resulted in useful computational models with diverse applications [2,3]. In addition, some insects must also extract the movement of their prey or conspecifics from their environment. Such insects have the ability to detect and interact with small moving targets, even amidst a swarm of others [4,5]. We use electrophysiological techniques to record from small target motion detector (STMD) neurons in the insect brain that are likely to subserve these behaviors. Inspired by such recordings, we previously proposed an ‘elementary’ small target motion detector (ESTMD) model that accounts for the spatial and temporal tuning of such neurons and even their ability to discriminate targets against cluttered surrounds [6-8]. However, other properties such as direction selectivity [9] and response facilitation for objects moving on extended trajectories [10] are not accounted for by this model. We therefore propose here two model variants that cascade an ESTMD model with a traditional motion detection model algorithm, the Hassenstein Reichardt ‘elementary motion detector’ (EMD) [11]. We show that these elaborations maintain the principal attributes of ESTMDs (i.e. spatiotemporal tuning and background clutter rejection) while also capturing the direction selectivity observed in some STMD neurons. By encapsulating the properties of biological STMD neurons we aim to develop computational models that can simulate the remarkable capabilities of insects in target discrimination and pursuit for applications in robotics and artificial vision systems.
---
paper_title: Neural specializations for small target detection in insects

paper_content:

Despite being equipped with low-resolution eyes and tiny brains, many insects show exquisite abilities to detect and pursue targets even in highly textured surrounds. Target tracking behavior is subserved by neurons that are sharply tuned to the motion of small high-contrast targets. These neurons respond robustly to target motion, even against self-generated optic flow. A recent model, supported by neurophysiology, generates target selectivity by being sharply tuned to the unique spatiotemporal profile associated with target motion. Target neurons are likely connected in a complex network where some provide more direct output to behavior, whereas others serve an inter-regulatory role. These interactions may regulate attention and aid in the robust detection of targets in clutter observed in behavior.
---
paper_title: Object detection in the fly during simulated translatory flight

paper_content:

Abstract Translatory movement of an animal in its environment induces optic flow that contains information about the three-dimensional layout of the surroundings: as a rule, images of objects that are closer to the animal move faster across the retina than those of more distant objects. Such relative motion cues are used by flies to detect objects in front of a structured background. We confronted flying flies, tethered to a torque meter, with front-to-back motion of patterns displayed on two CRT screens, thereby simulating translatory motion of the background as experienced by an animal during straight flight. The torque meter measured the instantaneous turning responses of the fly around its vertical body axis. During short time intervals, object motion was superimposed on background pattern motion. The average turning response towards such an object depends on both object and background velocity in a characteristic way: (1) in order to elicit significant responses object motion has to be faster than background motion; (2) background motion within a certain range of velocities improves object detection. These properties can be interpreted as adaptations to situations as they occur in natural free flight. We confirmed that the measured responses were mediated mainly by a control system specialized for the detection of objects rather than by the compensatory optomotor system responsible for course stabilization.
---
paper_title: Neuronal encoding of object and distance information: a model simulation study on naturalistic optic flow processing

paper_content:

We developed a model of the input circuitry of the FD1 cell, an identified motion-sensitive interneuron in the blowfly’s visual system. The model circuit successfully reproduces the FD1 cell’s most conspicuous property: Its larger responses to objects than to spatially extended patterns. The model circuit also mimics the time-dependent responses of FD1 to dynamically complex naturalistic stimuli, shaped by the blowfly’s saccadic flight and gaze strategy: The FD1 responses are enhanced when, as a consequence of self-motion, a nearby object crosses the receptive field during intersaccadic intervals. Moreover, the model predicts that these object-induced responses are superimposed by pronounced pattern-dependent fluctuations during movements on virtual test flights in a three-dimensional environment with systematic modifications of the environmental patterns. Hence, the FD1 cell is predicted to detect not unambiguously objects defined by the spatial layout of the environment, but to be also sensitive to objects distinguished by textural features. These ambiguous detection abilities suggest an encoding of information about objects - irrespective of the features by which the objects are defined - by a population of cells, with the FD1 cell presumably playing a prominent role in such an ensemble.
---
paper_title: Development of a bio-inspired vision system for mobile micro-robots

paper_content:

In this paper, we present a new bio-inspired vision system for mobile micro-robots. The processing method takes inspiration from vision of locusts in detecting the fast approaching objects. Research suggested that locusts use wide- field visual neuron called the lobula giant movement detector to respond to imminent collisions. We employed the locusts' vision mechanism to motion control of a mobile robot. The selected image processing method is implemented on a developed extension module using a low-cost and fast ARM processor. The vision module is placed on top of a micro-robot to control its trajectory and to avoid obstacles. The observed results from several performed experiments demonstrated that the developed extension module and the inspired vision system are feasible to employ as a vision module for obstacle avoidance and motion control.
---
paper_title: Distributed Dendritic Processing Facilitates Object Detection: A Computational Analysis on the Visual System of the Fly

paper_content:

BACKGROUND ::: Detecting objects is an important task when moving through a natural environment. Flies, for example, may land on salient objects or may avoid collisions with them. The neuronal ensemble of Figure Detection cells (FD-cells) in the visual system of the fly is likely to be involved in controlling these behaviours, as these cells are more sensitive to objects than to extended background structures. Until now the computations in the presynaptic neuronal network of FD-cells and, in particular, the functional significance of the experimentally established distributed dendritic processing of excitatory and inhibitory inputs is not understood. ::: ::: ::: METHODOLOGY/PRINCIPAL FINDINGS ::: We use model simulations to analyse the neuronal computations responsible for the preference of FD-cells for small objects. We employed a new modelling approach which allowed us to account for the spatial spread of electrical signals in the dendrites while avoiding detailed compartmental modelling. The models are based on available physiological and anatomical data. Three models were tested each implementing an inhibitory neural circuit, but differing by the spatial arrangement of the inhibitory interaction. Parameter optimisation with an evolutionary algorithm revealed that only distributed dendritic processing satisfies the constraints arising from electrophysiological experiments. In contrast to a direct dendro-dendritic inhibition of the FD-cell (Direct Distributed Inhibition model), an inhibition of its presynaptic retinotopic elements (Indirect Distributed Inhibition model) requires smaller changes in input resistance in the inhibited neurons during visual stimulation. ::: ::: ::: CONCLUSIONS/SIGNIFICANCE ::: Distributed dendritic inhibition of retinotopic elements as implemented in our Indirect Distributed Inhibition model is the most plausible wiring scheme for the neuronal circuit of FD-cells. This microcircuit is computationally similar to lateral inhibition between the retinotopic elements. Hence, distributed inhibition might be an alternative explanation of perceptual phenomena currently explained by lateral inhibition networks.
---
paper_title: Detection of object motion by a fly neuron during simulated flight

paper_content:

Object detection on the basis of relative motion was investigated in the fly at the neuronal level. A representative of the figure detection cells (FD-cells), the FD1b-cell, was characterized with respect to its responses to optic flow which simulated the presence of an object during translatory flight. The figure detection cells reside in the fly's third visual neuropil and are believed to play a central role in mediating object-directed turning behaviour. The dynamical response properties as well as the mean response amplitudes of the FD1b-cell depend on the temporal frequency of object motion and on the presence or absence of background motion. The responses of the FD1b-cell to object motion during simulated translatory flight were compared to behavioural responses of the fly as obtained with identical stimuli in a previous study. The behavioural responses could only partly be explained on the basis of the FD1b-cell's responses. Further processing between the third visual neuropil and the final motor output has to be assumed which involves (1) facilitation of the object-induced responses during translatory background motion at moderate temporal frequencies, and (2) inhibition of the object-induced turning responses during translatory background motion at high temporal frequencies.
---
paper_title: A bio-inspired visual collision detection mechanism for cars: Optimisation of a model of a locust neuron to a novel environment

paper_content:

The lobula giant movement detector (LGMD) neuron of locusts has been shown to preferentially respond to objects approaching the eye of a locust on a direct collision course. Computer simulations of the neuron have been developed and have demonstrated the ability of mobile robots, interfaced with a simulated LGMD model, to avoid collisions. In this study, a model of the LGMD neuron is presented and the functional parameters of the model identified. Models with different parameters were presented with a range of automotive video sequences, including collisions with cars. The parameters were optimised to respond correctly to the video sequences using a range of genetic algorithms (GAs). The model evolved most rapidly using GAs with high clone rates into a form suitable for detecting collisions with cars and not producing false collision alerts to most non-collision scenes.
---
paper_title: Spatial response properties of contralateral inhibited lobula plate tangential cells in the fly visual system.

paper_content:

This study describes the spatial response properties of a particular group of motion-sensitive and directionally selective neurons located in the lobula plate of the fly visual system. Their preferred motion direction is front-to-back (depolarization), and their null direction is back-to-front (hyperpolarization). They receive inhibitory input from the contralateral eye during pattern motion from back to front (regressive). In this study, we call these neurons regressive contralateral inhibited lobula plate tangential cells (rCI-LPTCs). Three physiologic groups of rCI-neurons (rCI-I, rCI-IIa, and rCI-IIb) can be distinguished on the basis of their ipsilateral pattern size dependence and their inhibitory contralateral input. rCI-I neurons depolarize during the motion of small ipsilateral patterns from front to back, but they become hyperpolarized by large ipsilateral patterns moving in the same direction. rCI-IIa and rCI-IIb neurons receive bidirectional inhibitory input from the contralateral eye. rCI-IIa neurons respond best to small ipsilateral pattern sizes, but unlike rCI-I neurons, their net response to large patterns is positive. rCI-IIb neurons respond best to large ipsilateral patterns. The anatomical and physiologic variability of the rCI-neurons suggests that more than three types of rCI-neurons exist. The suggested physiologic groups might be preliminary. We recorded one neuron that could mediate the bidirectional inhibitory input that rCI-IIa and rCI-IIb neurons receive from the contralateral eye. In the case of the rCI-IIa neurons at least one further contralateral inhibitory element has to be assumed. The tuning of rCI-IIa neurons to small ipsilateral pattern sizes is likely to be based on an on-center/off-surround organization of their synaptic input. J. Comp. Neurol. 406:51–71, 1999. © 1999 Wiley-Liss, Inc.
---
paper_title: Bio-Inspired Embedded Vision System for Autonomous Micro-Robots: The LGMD Case

paper_content:

In this paper, we present a new bio-inspired vision system embedded for micro-robots. The vision system takes inspiration from locusts in detecting fast approaching objects. Neurophysiological research suggested that locusts use a wide-field visual neuron called lobula giant movement detector (LGMD) to respond to imminent collisions. In this paper, we present the implementation of the selected neuron model by a low-cost ARM processor as part of a composite vision module. As the first embedded LGMD vision module fits to a micro-robot, the developed system performs all image acquisition and processing independently. The vision module is placed on top of a micro-robot to initiate obstacle avoidance behavior autonomously. Both simulation and real-world experiments were carried out to test the reliability and robustness of the vision system. The results of the experiments with different scenarios demonstrated the potential of the bio-inspired vision system as a low-cost embedded module for autonomous robots.
---
paper_title: Motion detectors in the locust visual system: From biology to robot sensors.

paper_content:

Motion detectors in the locust optic lobe and brain fall into two categories: neurones that respond selectively to approaching vs. receding objects and neurones that respond selectively to a particular pattern of image motion over a substantial part of the eye, generated by the locust's own movements through its environment. Neurones from the two categories can be differentiated on the basis of their response to motion at a constant velocity at a fixed distance from the locust: neurones of the first category respond equally well to motion in any direction whereas neurones in the second category respond selectively to one preferred direction of motion. Several of the motion detectors of the first category, responding to approaching objects, share the same input organisation, suggesting that it is important in generating a tuning for approaching objects. Anatomical, physiological, and modelling studies have revealed how the selectivity of the response is generated. The selectivity arises as a result of a critical race between excitation, generated when image edges move out over the eye and delayed inhibition, generated by the same edge movements. For excitation to build up, the velocity and extent of edge motion over the eye must increase rapidly. The ultrastructure of the afferent inputs onto the dendrites of collision sensitive neurones reveals a possible substrate for the interaction between excitation and inhibition. This interpretation is supported by both physiological and immunocytochemical evidence. The input organisation of these neurones has been incorporated into the control structure of a small mobile robot, which successfully avoids collisions with looming objects. The ecological role of motion detectors of the second category that respond to image motion over a substantial part of the visual field, is discussed as is the input organisation that generates this selective response. The broad tuning of these neurones, particularly at low velocities (<0.02 degree/s), suggests they may have a role in navigation during migratory flights at altitude. By contrast, their optimum tuning to high-image velocities suggests these motion detectors are adapted for use in a fast flying insect, which does not spend significant time hovering.
---
paper_title: Binocular Integration of Visual Information: A Model Study on Naturalistic Optic Flow Processing

paper_content:

The computation of visual information from both visual hemispheres is often of functional relevance when solving orientation and navigation tasks. The vCH-cell is a motion-sensitive wide-field neuron in the visual system of the blowfly Calliphora, a model system in the field of optic flow processing. The vCH-cell receives input from various other identified wide-field cells, the receptive fields of which are located in both the ipsilateral and the contralateral visual field. The relevance of this connectivity to the processing of naturalistic image sequences, with their peculiar dynamical characteristics, is still unresolved. To disentangle the contributions of the different input components to the cell’s overall response, we used electrophysiologically determined responses of the vCH-cell and its various input elements to tune a model of the vCH-circuit. Their impact on the vCH-cell response could be distinguished by stimulating not only extended parts of the visual field of the fly, but also selected regions in the ipsi- and contralateral visual field with behaviorally generated optic flow. We show that a computational model of the vCH-circuit is able to account for the neuronal activities of the counterparts in the blowfly’s visual system. Furthermore, we offer an insight into the dendritic integration of binocular visual input.
---
paper_title: Non-Linear Neuronal Responses as an Emergent Property of Afferent Networks: A Case Study of the Locust Lobula Giant Movement Detector

paper_content:

In principle it appears advantageous for single neurons to perform non-linear operations. Indeed it has been reported that some neurons show signatures of such operations in their electrophysiological response. A particular case in point is the Lobula Giant Movement Detector (LGMD) neuron of the locust, which is reported to locally perform a functional multiplication. Given the wide ramifications of this suggestion with respect to our understanding of neuronal computations, it is essential that this interpretation of the LGMD as a local multiplication unit is thoroughly tested. Here we evaluate an alternative model that tests the hypothesis that the non-linear responses of the LGMD neuron emerge from the interactions of many neurons in the opto-motor processing structure of the locust. We show, by exposing our model to standard LGMD stimulation protocols, that the properties of the LGMD that were seen as a hallmark of local non-linear operations can be explained as emerging from the dynamics of the pre-synaptic network. Moreover, we demonstrate that these properties strongly depend on the details of the synaptic projections from the medulla to the LGMD. From these observations we deduce a number of testable predictions. To assess the real-time properties of our model we applied it to a high-speed robot. These robot results show that our model of the locust opto-motor system is able to reliably stabilize the movement trajectory of the robot and can robustly support collision avoidance. In addition, these behavioural experiments suggest that the emergent non-linear responses of the LGMD neuron enhance the system's collision detection acuity. We show how all reported properties of this neuron are consistently reproduced by this alternative model, and how they emerge from the overall opto-motor processing structure of the locust. Hence, our results propose an alternative view on neuronal computation that emphasizes the network properties as opposed to the local transformations that can be performed by single neurons.
---
paper_title: Immunocytochemical evidence that collision sensing neurons in the locust visual system contain acetylcholine

paper_content:

The lobula giant movement detector (LGMD1 and -2) neurons in the locust visual system are parts of motion-sensitive pathways that detect objects approaching on a collision course. The dendritic processes of the LGMD1 and -2 in the lobula are localised to discrete regions, allowing the dendrites of each neuron to be distinguished uniquely. As was described previously for the LGMD1, the afferent processes onto the LGMD2 synapse directly with each other, and these synapses are immediately adjacent to their outputs onto the LGMD2. Here we present immunocytochemical evidence, using antibodies against choline-protein conjugates and a polyclonal antiserum against choline acetyltransferase (ChAT; Chemicon Ab 143), that the LGMD1 and -2 and the retinotopic units presynaptic to them contain acetylcholine (ACh). It is proposed that these retinotopic units excite the LGMD1 or -2 but inhibit each other. It is well established that ACh has both excitatory and inhibitory effects and may provide the substrate for a critical race in the LGMD1 or -2, between excitation caused by edges moving out over successive photoreceptors, and inhibition spreading laterally resulting in the selective response to objects approaching on a collision course. In the optic lobe, ACh was also found to be localised in discrete layers of the medulla and in the outer chiasm between the lamina and medulla. In the brain, the antennal lobes contained neurons that reacted positively for ACh. Silver- or haematoxylin and eosin-stained sections through the optic lobe confirmed the identities of the positively immunostained neurons.
---
paper_title: IDENTIFICATION OF DIRECTIONALLY SELECTIVE MOTION-DETECTING NEURONES IN THE LOCUST LOBULA AND THEIR SYNAPTIC CONNECTIONS WITH AN IDENTIFIED DESCENDING NEURONE

paper_content:

The anatomy and physiology of two directionally selective motion-detecting neurones in the locust are described. Both neurones had dendrites in the lobula, and projected to the ipsilateral protocerebrum. Their cell bodies were located on the posterio-dorsal junction of the optic lobe with the protocerebrum. The neurones were sensitive to horizontal motion of a visual stimulus. One neurone, LDSMD(F), had a preferred direction forwards over the ipsilateral eye, and a null direction backwards. The other neurone, LDSMD(B), had a preferred direction backwards over the ipsilateral eye ::: ::: 1. 1. Motion in the preferred direction caused EPSPs and spikes in the LDSMD neurones. Motion in the null direction resulted in IPSPs ::: ::: 2. 2. Both excitatory and inhibitory inputs were derived from the ipsilateral eye ::: ::: 3. 3. The DSMD neurones responded to velocities of movement up to and beyond 270°s−1 ::: ::: 4. 4. The response of both LDSMD neurones showed no evidence of adaptation during maintained apparent or real movement ::: ::: 5. 5. There was a delay of 60–80 ms between a single step of apparent movement, either the preferred or the null direction, and the start of the response ::: ::: 6. 6. There was a monosynaptic, excitatory connection between the LDSMD(B) neurone and the protocerebral, descending DSMD neurone (PDDSMD) identified in the preceding paper (Rind, 1990). At resting membrane potential, a single presynaptic spike did not give rise to a spike in the postsynaptic neurone
---
paper_title: Responses to object approach by a wide field visual neurone, the LGMD2 of the locust: Characterization and image cues

paper_content:

The LGMD2 belongs to a group of giant movement-detecting neurones which have fan-shaped arbors in the lobula of the locust optic lobe and respond to movements of objects. One of these neurones, the LGMD1, has been shown to respond directionally to movements of objects in depth, generating vigorous, maintained spike discharges during object approach. Here we compare the responses of the LGMD2 neurone with those of the LGMD1 to simulated movements of objects in depth and examine different image cues which could allow the LGMD2 to distinguish approaching from receding objects. In the absence of stimulation, the LGMD2 has a resting discharge of 10–40 spikes s−1 compared with <1 spike s−1 for the LGMD1. The most powerful excitatory stimulus for the LGMD2 is a dark object approaching the eye. Responses to approaching objects are suppressed by wide field movements of the background. Unlike the LGMD1, the LGMD2 is not excited by the approach of light objects; it specifically responds to movement of edges in the light to dark direction. Both neurones rely on the same monocular image cues to distinguish approaching from receding objects: an increase in the velocity with which edges of images travel over the eye; and an increase in the extent of edges in the image during approach.
---
paper_title: Bio-inspired collision detector with enhanced selectivity for ground robotic vision system

paper_content:

There are many ways of building collision-detecting systems. In this paper, we propose a novel collision selective visual neural network inspired by LGMD2 neurons in the juvenile locusts. Such collision-sensitive neuron matures early in the ﬁrst-aged or even hatching locusts, and is only selective to detect looming dark objects against bright background in depth, represents swooping predators, a situation which is similar to ground robots or vehicles. However, little has been done on modeling LGMD2, let alone its potential applications in robotics and other vision-based areas. Compared to other collision detectors, our major contributions are ﬁrst, enhancing the collision selectivity in a bio-inspired way, via constructing a computing efﬁcient visual sensor, and realizing the revealed speciﬁc characteristic sofLGMD2. Second, we applied the neural network to help rearrange path navigation of an autonomous ground miniature robot in an arena. We also examined its neural properties through systematic experiments challenged against image streams from a visual sensor of the micro-robot.
---
paper_title: INTRACELLULAR CHARACTERIZATION OF NEURONS IN THE LOCUST BRAIN SIGNALING IMPENDING COLLISION

paper_content:

1. In response to a rapidly approaching object, intracellular recordings show that excitation in the locust lobula giant movement detecting (LGMD) neuron builds up exponentially, particularly during the final stages of object approach. After the cessation of object motion, inhibitory potentials in the LGMD then help to terminate this excitation. Excitation in the LGMD follows object recession with a short, constant latency but is cut back rapidly by hyperpolarizing potentials. The timing of these hyperpolarizing potentials in the LGMD is variable, and their latency following object recession is shortest with the highest velocities of motion simulated. The hyperpolarizing potentials last from 50-300 ms and are often followed by re-excitation. The observed hyperpolarizations of the LGMD can occur without any preceding excitation and are accompanied by a measurable conductance increase. The hyperpolarizations are likely to be inhibitory postsynaptic potentials (PSPs). The behavior of the intracellularly recorded inhibitory PSPs (IPSPs) closely parallels that of the feed forward inhibitory loop in the neural network described by Rind and Bramwell. 2. The preference of the LGMD for approaching versus receding objects remains over a wide range of starting and finishing distances. The response to object approach, measured both as membrane potential and spike rate, remains single peaked with starting distances of between 200 and 2,100 mm, and approach speeds of 0.5-2 m/s. These results confirm the behavior predicted by the neural network described by Rind and Bramwell but contradicts the findings of Rind and Simmons, forcing a re-evaluation of the suitability of some of the mechanical visual stimuli used in that study. 3. For depolarization of the LGMD neuron to be maintained or increased throughout the motion of image edges, the edges must move with increasing velocity over the eye. Membrane potential declines before the end of edge motion with constant velocities of edge motion. 4. A second identified neuron, the LGMD2 also is shown to respond directionally to approaching objects. In both the LGMD and LGMD2 neurons, postsynaptic inhibition shapes the directional response to object motion.
---
paper_title: Shaping the collision selectivity in a looming sensitive neuron model with parallel ON and OFF pathways and spike frequency adaptation

paper_content:

Abstract Shaping the collision selectivity in vision-based artificial collision-detecting systems is still an open challenge. This paper presents a novel neuron model of a locust looming detector, i.e. the lobula giant movement detector (LGMD1), in order to provide effective solutions to enhance the collision selectivity of looming objects over other visual challenges. We propose an approach to model the biologically plausible mechanisms of ON and OFF pathways and a biophysical mechanism of spike frequency adaptation (SFA) in the proposed LGMD1 visual neural network. The ON and OFF pathways can separate both dark and light looming features for parallel spatiotemporal computations. This works effectively on perceiving a potential collision from dark or light objects that approach; such a bio-plausible structure can also separate LGMD1’s collision selectivity to its neighbouring looming detector — the LGMD2. The SFA mechanism can enhance the LGMD1’s collision selectivity to approaching objects rather than receding and translating stimuli, which is a significant improvement compared with similar LGMD1 neuron models. The proposed framework has been tested using off-line tests of synthetic and real-world stimuli, as well as on-line bio-robotic tests. The enhanced collision selectivity of the proposed model has been validated in systematic experiments. The computational simplicity and robustness of this work have also been verified by the bio-robotic tests, which demonstrates potential in building neuromorphic sensors for collision detection in both a fast and reliable manner.
---
paper_title: The locust DCMD, a movement-detecting neurone tightly tuned to collision trajectories

paper_content:

A Silicon Graphics computer was used to challenge the locust descending contralateral movement detector (DCMD) neurone with images of approaching objects. The DCMD gave its strongest response, measured as either total spike number or spike frequency, to objects approaching on a direct collision course. Deviation in either a horizontal or vertical direction from a direct collision course resulted in a reduced response. The decline in the DCMD response with increasing deviation from a collision course was used as a measure of the tightness of DCMD tuning for collision trajectories. Tuning was defined as the half-width of the response when it had fallen to half its maximum level. The response tuning, measured as averaged mean spike number versus deviation away from a collision course, had a half-width at half-maximum response of 2.4 ds3.0 d for a deviation in the horizontal direction and 3.0 d for a deviation in the vertical direction. Mean peak spike frequency showed an even sharper tuning, with a half-width at half-maximum response of 1.8 d for deviations away from a collision course in the horizontal plane.
---
paper_title: A look into the cockpit of the developing locust: Looming detectors and predator avoidance

paper_content:

For many animals, the visual detection of looming stimuli is crucial at any stage of their lives. For example, human babies of only 6 days old display evasive responses to looming stimuli (Bower et al. [1971]: Percept Psychophys 9: 193-196). This means the neuronal pathways involved in looming detection should mature early in life. Locusts have been used extensively to examine the neural circuits and mechanisms involved in sensing looming stimuli and triggering visually evoked evasive actions, making them ideal subjects in which to investigate the development of looming sensitivity. Two lobula giant movement detectors (LGMD) neurons have been identified in the lobula region of the locust visual system: the LGMD1 neuron responds selectively to looming stimuli and provides information that contributes to evasive responses such as jumping and emergency glides. The LGMD2 responds to looming stimuli and shares many response properties with the LGMD1. Both neurons have only been described in the adult. In this study, we describe a practical method combining classical staining techniques and 3D neuronal reconstructions that can be used, even in small insects, to reveal detailed anatomy of individual neurons. We have used it to analyze the anatomy of the fan-shaped dendritic tree of the LGMD1 and the LGMD2 neurons in all stages of the post-embryonic development of Locusta migratoria. We also analyze changes seen during the ontogeny of escape behaviors triggered by looming stimuli, specially the hiding response.
---
paper_title: Two identified looming detectors in the locust: ubiquitous lateral connections among their inputs contribute to selective responses to looming objects

paper_content:

Two identified looming detectors in the locust: ubiquitous lateral connections among their inputs contribute to selective responses to looming objects
---
paper_title: Non-Linear Neuronal Responses as an Emergent Property of Afferent Networks: A Case Study of the Locust Lobula Giant Movement Detector

paper_content:

In principle it appears advantageous for single neurons to perform non-linear operations. Indeed it has been reported that some neurons show signatures of such operations in their electrophysiological response. A particular case in point is the Lobula Giant Movement Detector (LGMD) neuron of the locust, which is reported to locally perform a functional multiplication. Given the wide ramifications of this suggestion with respect to our understanding of neuronal computations, it is essential that this interpretation of the LGMD as a local multiplication unit is thoroughly tested. Here we evaluate an alternative model that tests the hypothesis that the non-linear responses of the LGMD neuron emerge from the interactions of many neurons in the opto-motor processing structure of the locust. We show, by exposing our model to standard LGMD stimulation protocols, that the properties of the LGMD that were seen as a hallmark of local non-linear operations can be explained as emerging from the dynamics of the pre-synaptic network. Moreover, we demonstrate that these properties strongly depend on the details of the synaptic projections from the medulla to the LGMD. From these observations we deduce a number of testable predictions. To assess the real-time properties of our model we applied it to a high-speed robot. These robot results show that our model of the locust opto-motor system is able to reliably stabilize the movement trajectory of the robot and can robustly support collision avoidance. In addition, these behavioural experiments suggest that the emergent non-linear responses of the LGMD neuron enhance the system's collision detection acuity. We show how all reported properties of this neuron are consistently reproduced by this alternative model, and how they emerge from the overall opto-motor processing structure of the locust. Hence, our results propose an alternative view on neuronal computation that emphasizes the network properties as opposed to the local transformations that can be performed by single neurons.
---
paper_title: Modelling LGMD2 visual neuron system

paper_content:

Two Lobula Giant Movement Detectors (LGMDs) have been identified in the lobula region of the locust visual system: LGMD1 and LGMD2. LGMD1 had been successfully used in robot navigation to avoid impending collision. LGMD2 also responds to looming stimuli in depth, and shares most the same properties with LGMD1; however, LGMD2 has its specific collision selective responds when dealing with different visual stimulus. Therefore, in this paper, we propose a novel way to model LGMD2, in order to emulate its predicted bio-functions, moreover, to solve some defects of previous LGMD1 computational models. The mechanism of ON and OFF cells, as well as bio-inspired nonlinear functions, are introduced in our model, to achieve LGMD2's collision selectivity. Our model has been tested by a miniature mobile robot in real time. The results suggested this model has an ideal performance in both software and hardware for collision recognition.
---
paper_title: Spike-Frequency Adaptation and Intrinsic Properties of an Identified, Looming-Sensitive Neuron

paper_content:

We investigated in vivo the characteristics of spike-frequency adaptation and the intrinsic membrane properties of an identified, looming-sensitive interneuron of the locust optic lobe, the lobula giant movement detector (LGMD). The LGMD had an input resistance of 4–5 MΩ, a membrane time constant of about 8 ms, and exhibited inward rectification and rebound spiking after hyperpolarizing current pulses. Responses to depolarizing current pulses revealed the neuron's intrinsic bursting properties and pronounced spike-frequency adaptation. The characteristics of adaptation, including its time course, the attenuation of the firing rate, the mutual dependency of these two variables, and their dependency on injected current, closely followed the predictions of a model first proposed to describe the adaptation of cat visual cortex pyramidal neurons in vivo. Our results thus validate the model in an entirely different context and suggest that it might be applicable to a wide variety of neurons across species. Spik...
---
paper_title: Spike frequency adaptation mediates looming stimulus selectivity in a collision-detecting neuron

paper_content:

Studying the mechanisms by which spike frequency adaptation shapes visual stimulus selectivity in the lobula giant movement detector interneuron of the locust visual system, the authors find that spike frequency adaptation selectively decreases this neuron's responses to nonpreferred stimuli.
---
paper_title: A Collision Detection System for a Mobile Robot Inspired by the Locust Visual System

paper_content:

The lobula giant movement detector (LGMD) is an identified neuron in the locust brain that responds most strongly to the image of an approaching object such as a predator. A computational neural network model based on the structure of the LGMD and its afferent inputs is also able to detect approaching objects. In order for the LGMD network to be used as a robust collision detector for robotic applications, we proposed a new mechanism to enhance the feature of colliding objects before the excitations are gathered by LGMD cell. The new model favours grouped excitation but tends to ignore isolated excitation with selective passing coefficients. Experiments with a Khepera robot showed the proposed collision detector worked in real time in an arena surrounded with blocks.
---
paper_title: Role of spike-frequency adaptation in shaping neuronal response to dynamic stimuli

paper_content:

Spike-frequency adaptation is the reduction of a neuron’s firing rate to a stimulus of constant intensity. In the locust, the Lobula Giant Movement Detector (LGMD) is a visual interneuron that exhibits rapid adaptation to both current injection and visual stimuli. Here, a reduced compartmental model of the LGMD is employed to explore adaptation’s role in selectivity for stimuli whose intensity changes with time. We show that supralinearly increasing current injection stimuli are best at driving a high spike count in the response, while linearly increasing current injection stimuli (i.e., ramps) are best at attaining large firing rate changes in an adapting neuron. This result is extended with in vivo experiments showing that the LGMD’s response to translating stimuli having a supralinear velocity profile is larger than the response to constant or linearly increasing velocity translation. Furthermore, we show that the LGMD’s preference for approaching versus receding stimuli can partly be accounted for by adaptation. Finally, we show that the LGMD’s adaptation mechanism appears well tuned to minimize sensitivity for the level of basal input.
---
paper_title: Reactive direction control for a mobile robot: a locust-like control of escape direction emerges when a bilateral pair of model locust visual neurons are integrated

paper_content:

Locusts possess a bilateral pair of uniquely identifiable visual neurons that respond vigorously to the image of an approaching object. These neurons are called the lobula giant movement detectors (LGMDs). The locust LGMDs have been extensively studied and this has lead to the development of an LGMD model for use as an artificial collision detector in robotic applications. To date, robots have been equipped with only a single, central artificial LGMD sensor, and this triggers a non-directional stop or rotation when a potentially colliding object is detected. Clearly, for a robot to behave autonomously, it must react differently to stimuli approaching from different directions. In this study, we implement a bilateral pair of LGMD models in Khepera robots equipped with normal and panoramic cameras. We integrate the responses of these LGMD models using methodologies inspired by research on escape direction control in cockroaches. Using `randomised winner-take-all' or `steering wheel' algorithms for LGMD model integration, the Khepera robots could escape an approaching threat in real time and with a similar distribution of escape directions as real locusts. We also found that by optimising these algorithms, we could use them to integrate the left and right DCMD responses of real jumping locusts offline and reproduce the actual escape directions that the locusts took in a particular trial. Our results significantly advance the development of an artificial collision detection and evasion system based on the locust LGMD by allowing it reactive control over robot behaviour. The success of this approach may also indicate some important areas to be pursued in future biological research.
---
paper_title: LGMD-based bio-inspired algorithm for detecting risk of collision of a road vehicle

paper_content:

LGMD (Lobula Giant Movement Detector) is part of a visual system of a locust, used to detect and evade approaching predators. Similar algorithm can be used in man-made systems like autonomous robots or safety systems in vehicles for detecting objects approaching on collision course. In this article usage of LGMD-based algorithms in road vehicles is investigated and a new solution is proposed. Video stream recorded from a moving vehicle inherently contains a lot of movement of the background due to vibrations and due to turning of the vehicle. This cause high output of LGMD and could trigger false alarms. The new approach, proposed in this article, enhances LGMD information and adds estimated expansion of objects in X and Y direction. Combination of all three sources of information gives a very good estimate of risk of collision. The new algorithm was developed using a reference set of test videos, where only one parameter was changed. Algorithm was tested with videos recorded from a moving vehicle in normal traffic and on the test ground, including real collisions in carton target at different speeds. Time between triggering of the alert and actual collision was measured.
---
paper_title: A modified neural network model for Lobula Giant Movement Detector with additional depth movement feature

paper_content:

The Lobula Giant Movement Detector (LGMD) is a wide-field visual neuron that is located in the Lobula layer of the Locust nervous system. The LGMD increases its firing rate in response to both the velocity of the approaching object and its proximity. It has been found that it can respond to looming stimuli very quickly and can trigger avoidance reactions whenever a rapidly approaching object is detected. It has been successfully applied in visual collision avoidance systems for vehicles and robots. This paper proposes a modified LGMD model that provides additional movement depth direction information. The proposed model retains the simplicity of the previous neural network model, adding only a few new cells. It has been tested on both simulated and recorded video data sets. The experimental results shows that the modified model can very efficiently provide stable information on the depth direction of movement.
---
paper_title: Near range path navigation using LGMD visual neural networks

paper_content:

In this paper, we proposed a method for near range path navigation for a mobile robot by using a pair of biologically inspired visual neural network - lobula giant movement detector (LGMD). In the proposed binocular style visual system, each LGMD processes images covering a part of the wide field of view and extracts relevant visual cues as its output. The outputs from the two LGMDs are compared and translated into executable motor commands to control the wheels of the robot in real time. Stronger signal from the LGMD in one side pushes the robot away from this side step by step; therefore, the robot can navigate in a visual environment naturally with the proposed vision system. Our experiments showed that this bio-inspired system worked well in different scenarios.
---
paper_title: A bio-inspired visual collision detection mechanism for cars: Combining insect inspired neurons to create a robust system

paper_content:

The lobula giant movement detector (LGMD) of locusts is a visual interneuron that responds with an increasing spike frequency to an object approaching on a direct collision course. Recent studies involving the use of LGMD models to detect car collisions showed that it could detect collisions, but the neuron produced collision alerts to non-colliding, translating, stimuli in many cases. This study presents a modified model to address these problems. It shows how the neurons pre-synaptic to the LGMD show a remarkable ability to filter images, and only colliding and translating stimuli produce excitation in the neuron. It then integrates the LGMD network with models based on the elementary movement detector (EMD) neurons from the fly visual system, which are used to analyse directional excitation patterns in the biologically filtered images. Combining the information from the LGMD neuron and four directionally sensitive neurons produces a robust collision detection system for a wide range of automotive test situations.
---
paper_title: Collision selective LGMDs neuron models research benefits from a vision-based autonomous micro robot

paper_content:

The developments of robotics inform research across a broad range of disciplines. In this paper, we will study and compare two collision selective neuron models via a vision-based autonomous micro robot. In the locusts' visual brain, two Lobula Giant Movement Detectors (LGMDs), i.e. LGMD1 and LGMD2, have been identified as looming sensitive neurons responding to rapidly expanding objects, yet with different collision selectivity. Both neurons have been modeled and successfully applied in robotic vision system for perceiving potential collisions in an efficient and reliable manner. In this research, we conduct binocular neuronal models, for the first time combining the functionalities of LGMD1 and LGMD2 neurons, in the visual modality of a ground mobile robot. The results of systematic on-line experiments demonstrated three contributions of this research: (1) The arena tests involving multiple robots verified the effectiveness and robustness of a reactive motion control strategy via integrating a bilateral pair of LGMD1 and LGMD2 models for collision detection in dynamic scenarios. (2) We pinpointed the different collision selectivity between LGMD1 and LGMD2 neuron models, which fulfill corresponding biological research. (3) The utilized micro robot may also benefit researches on other embedded vision systems as well as swarm robotics.
---
paper_title: Bio-inspired collision detector with enhanced selectivity for ground robotic vision system

paper_content:

There are many ways of building collision-detecting systems. In this paper, we propose a novel collision selective visual neural network inspired by LGMD2 neurons in the juvenile locusts. Such collision-sensitive neuron matures early in the ﬁrst-aged or even hatching locusts, and is only selective to detect looming dark objects against bright background in depth, represents swooping predators, a situation which is similar to ground robots or vehicles. However, little has been done on modeling LGMD2, let alone its potential applications in robotics and other vision-based areas. Compared to other collision detectors, our major contributions are ﬁrst, enhancing the collision selectivity in a bio-inspired way, via constructing a computing efﬁcient visual sensor, and realizing the revealed speciﬁc characteristic sofLGMD2. Second, we applied the neural network to help rearrange path navigation of an autonomous ground miniature robot in an arena. We also examined its neural properties through systematic experiments challenged against image streams from a visual sensor of the micro-robot.
---
paper_title: A modified model for the Lobula Giant Movement Detector and its FPGA implementation

paper_content:

Bio-inspired vision sensors are particularly appropriate candidates for navigation of vehicles or mobile robots due to their computational simplicity, allowing compact hardware implementations with low power dissipation. The Lobula Giant Movement Detector (LGMD) is a wide-field visual neuron located in the Lobula layer of the Locust nervous system. The LGMD increases its firing rate in response to both the velocity of an approaching object and the proximity of this object. It has been found that it can respond to looming stimuli very quickly and trigger avoidance reactions. It has been successfully applied in visual collision avoidance systems for vehicles and robots. This paper introduces a modified neural model for LGMD that provides additional depth direction information for the movement. The proposed model retains the simplicity of the previous model by adding only a few new cells. It has been simplified and implemented on a Field Programmable Gate Array (FPGA), taking advantage of the inherent parallelism exhibited by the LGMD, and tested on real-time video streams. Experimental results demonstrate the effectiveness as a fast motion detector.
---
paper_title: Collision detection in complex dynamic scenes using an LGMD-based visual neural network with feature enhancement

paper_content:

The lobula giant movement detector (LGMD) is an identified neuron in the locust brain that responds most strongly to the images of an approaching object such as a predator. Its computational model can cope with unpredictable environments without using specific object recognition algorithms. In this paper, an LGMD-based neural network is proposed with a new feature enhancement mechanism to enhance the expanded edges of colliding objects via grouped excitation for collision detection with complex backgrounds. The isolated excitation caused by background detail will be filtered out by the new mechanism. Offline tests demonstrated the advantages of the presented LGMD-based neural network in complex backgrounds. Real time robotics experiments using the LGMD-based neural network as the only sensory system showed that the system worked reliably in a wide range of conditions; in particular, the robot was able to navigate in arenas with structured surrounds and complex backgrounds
---
paper_title: Neural network based on the input organization of an identified neuron signaling impending collision

paper_content:

1. We describe a four-layered neural network (Fig. 1), based on the input organization of a collision signaling neuron in the visual system of the locust, the lobula giant movement detector (LGMD)....
---
paper_title: Visual motion pattern extraction and fusion for collision detection in complex dynamic scenes

paper_content:

Detecting colliding objects in complex dynamic scenes is a difficult task for conventional computer vision techniques. However, visual processing mechanisms in animals such as insects may provide very simple and effective solutions for detecting colliding objects in complex dynamic scenes. In this paper, we propose a robust collision detecting system, which consists of a lobula giant movement detector (LGMD) based neural network and a translating sensitive neural network (TSNN), to recognise objects on a direct collision course in complex dynamic scenes. The LGMD based neural network is specialized for recognizing looming objects that are on a direct collision course. The TSNN, which fuses the extracted visual motion cues from several whole field direction selective neural networks, is only sensitive to translating movements in the dynamic scenes. The looming cue and translating cue revealed by the two specialized visual motion detectors are fused in the present system via a decision making mechanism. In the system, the LGMD plays a key role in detecting imminent collision; the decision from TSNN becomes useful only when a collision alarm has been issued by the LGMD network. Using driving scenarios as an example, we showed that the bio-inspired system can reliably detect imminent colliding objects in complex driving scenes.
---
paper_title: Shaping the collision selectivity in a looming sensitive neuron model with parallel ON and OFF pathways and spike frequency adaptation

paper_content:

Abstract Shaping the collision selectivity in vision-based artificial collision-detecting systems is still an open challenge. This paper presents a novel neuron model of a locust looming detector, i.e. the lobula giant movement detector (LGMD1), in order to provide effective solutions to enhance the collision selectivity of looming objects over other visual challenges. We propose an approach to model the biologically plausible mechanisms of ON and OFF pathways and a biophysical mechanism of spike frequency adaptation (SFA) in the proposed LGMD1 visual neural network. The ON and OFF pathways can separate both dark and light looming features for parallel spatiotemporal computations. This works effectively on perceiving a potential collision from dark or light objects that approach; such a bio-plausible structure can also separate LGMD1’s collision selectivity to its neighbouring looming detector — the LGMD2. The SFA mechanism can enhance the LGMD1’s collision selectivity to approaching objects rather than receding and translating stimuli, which is a significant improvement compared with similar LGMD1 neuron models. The proposed framework has been tested using off-line tests of synthetic and real-world stimuli, as well as on-line bio-robotic tests. The enhanced collision selectivity of the proposed model has been validated in systematic experiments. The computational simplicity and robustness of this work have also been verified by the bio-robotic tests, which demonstrates potential in building neuromorphic sensors for collision detection in both a fast and reliable manner.
---
paper_title: Multiplication and stimulus invariance in a looming-sensitive neuron

paper_content:

Multiplicative operations and invariance of neuronal responses are thought to play important roles in the processing of neural information in many sensory systems. Yet the biophysical mechanisms that underlie both multiplication and invariance of neuronal responses in vivo, either at the single cell or at the network level, remain to a large extent unknown. Recent work on an identified neuron in the locust visual system (the LGMD neuron) that responds well to objects looming on a collision course towards the animal suggests that this cell represents a good model to investigate the biophysical basis of multiplication and invariance at the single neuron level. Experimental and theoretical results are consistent with multiplication being implemented by subtraction of two logarithmic terms followed by exponentiation via active membrane conductances, according to a x 1/b = exp(log(a) - log(b)). Invariance appears to be in part due to non-linear integration of synaptic inputs within the dendritic tree of this neuron.
---
paper_title: LGMD and DSNs neural networks integration for collision predication

paper_content:

An ability to predict collisions is essential for current vehicles and autonomous robots. In this paper, an integrated collision predication system is proposed based on neural subsystems inspired from Lobula giant movement detector (LGMD) and directional selective neurons (DSNs) which focus on different part of the visual field separately. The two type of neurons found in the visual pathways of insects respond most strongly to moving objects with preferred motion patterns, i.e., the LGMD prefers looming stimuli and DSNs prefer specific lateral movements. We fuse the extracted information by each type of neurons to make final decision. By dividing the whole field of view into four regions for each subsystem to process, the proposed approaches can detect hazardous situations that had been difficult for single subsystem only. Our experiments show that the integrated system works in most of the hazardous scenarios.
---
paper_title: A bio-inspired visual collision detection mechanism for cars: Optimisation of a model of a locust neuron to a novel environment

paper_content:

The lobula giant movement detector (LGMD) neuron of locusts has been shown to preferentially respond to objects approaching the eye of a locust on a direct collision course. Computer simulations of the neuron have been developed and have demonstrated the ability of mobile robots, interfaced with a simulated LGMD model, to avoid collisions. In this study, a model of the LGMD neuron is presented and the functional parameters of the model identified. Models with different parameters were presented with a range of automotive video sequences, including collisions with cars. The parameters were optimised to respond correctly to the video sequences using a range of genetic algorithms (GAs). The model evolved most rapidly using GAs with high clone rates into a form suitable for detecting collisions with cars and not producing false collision alerts to most non-collision scenes.
---
paper_title: Bio-Inspired Embedded Vision System for Autonomous Micro-Robots: The LGMD Case

paper_content:

In this paper, we present a new bio-inspired vision system embedded for micro-robots. The vision system takes inspiration from locusts in detecting fast approaching objects. Neurophysiological research suggested that locusts use a wide-field visual neuron called lobula giant movement detector (LGMD) to respond to imminent collisions. In this paper, we present the implementation of the selected neuron model by a low-cost ARM processor as part of a composite vision module. As the first embedded LGMD vision module fits to a micro-robot, the developed system performs all image acquisition and processing independently. The vision module is placed on top of a micro-robot to initiate obstacle avoidance behavior autonomously. Both simulation and real-world experiments were carried out to test the reliability and robustness of the vision system. The results of the experiments with different scenarios demonstrated the potential of the bio-inspired vision system as a low-cost embedded module for autonomous robots.
---
paper_title: Characterization of lobula giant neurons responsive to visual stimuli that elicit escape behaviors in the crab Chasmagnathus

paper_content:

In the grapsid crab Chasmagnathus , a visual danger stimulus elicits a strong escape response that diminishes rapidly on stimulus repetition. This behavioral modification can persist for several days as a result of the formation of an associative memory. We have previously shown that a generic group of large motion-sensitive neurons from the lobula of the crab respond to visual stimuli and accurately reflect the escape performance. Additional evidence indicates that these neurons play a key role in visual memory and in the decision to initiate an escape. Although early studies recognized that the group of lobula giant (LG) neurons consisted of different classes of motion-sensitive cells, a distinction between these classes has been lacking. Here, we recorded in vivo the responses of individual LG neurons to a wide range of visual stimuli presented in different segments of the animal's visual field. Physiological characterizations were followed by intracellular dye injections, which permitted comparison of the functional and morphological features of each cell. All LG neurons consisted of large tangential arborizations in the lobula with axons projecting toward the midbrain. Functionally, these cells proved to be more sensitive to single objects than to flow field motion. Despite these commonalities, clear differences in morphology and physiology allowed us to identify four distinct classes of LG neurons. These results will permit analysis of the role of each neuronal type for visually guided behaviors and will allow us to address specific questions on the neuronal plasticity of LGs that underlie the well-recognized memory model of the crab.
---
paper_title: Speed tuning in elementary motion detectors of the correlation type

paper_content:

A prominent model of visual motion detection is the so-called correlation or Reichardt detector. Whereas this model can account for many properties of motion vision, from humans to insects (review, Borst and Egelhaaf 1989), it has been commonly assumed that this scheme of motion detection is not well suited to the measurement of image velocity. This is because the commonly used version of the model, which incorporates two unidirectional motion detectors with opposite preferred directions, produces a response which varies not only with the velocity of the image, but also with its spatial structure and contrast. On the other hand, information on image velocity can be crucial in various contexts, and a number of recent behavioural experiments suggest that insects do extract velocity for navigational purposes (review, Srinivasan et al. 1996). Here we show that other versions of the correlation model, which consists of a single unidirectional motion detector or incorporates two oppositely directed detectors with unequal sensitivities, produce responses which vary with image speed and display tuning curves that are substantially independent of the spatial structure of the image. This surprising feature suggests simple strategies of reducing ambiguities in the estimation of speed by using components of neural hardware that are already known to exist in the visual system.
---
paper_title: Loom-Sensitive Neurons Link Computation to Action in the Drosophila Visual System

paper_content:

Summary Background Many animals extract specific cues from rich visual scenes to guide appropriate behaviors. Such cues include visual motion signals produced both by self-movement and by moving objects in the environment. The complexity of these signals requires neural circuits to link particular patterns of motion to specific behavioral responses. Results Through electrophysiological recordings, we characterize genetically identified neurons in the optic lobe of Drosophila that are specifically tuned to detect motion signals produced by looming objects on a collision course with the fly. Using a genetic manipulation to specifically silence these neurons, we demonstrate that signals from these cells are important for flies to efficiently initiate the loom escape response. Moreover, through targeted expression of channelrhodopsin in these cells, in flies that are blind, we reveal that optogenetic stimulation of these neurons is typically sufficient to elicit escape, even in the absence of any visual stimulus. Conclusions In this compact nervous system, a small group of neurons that extract a specific visual cue from local motion inputs serve to trigger the ethologically appropriate behavioral response.
---
paper_title: Computation of object approach by a system of visual motion-sensitive neurons in the crab Neohelice

paper_content:

Similar to most visual animals, crabs perform proper avoidance responses to objects directly approaching them. The monostratified lobula giant neurons of type 1 (MLG1) of crabs constitute an ensemble of 14–16 bilateral pairs of motion-detecting neurons projecting from the lobula (third optic neuropile) to the midbrain, with receptive fields that are distributed over the extensive visual field of the animal's eye. Considering the crab Neohelice (previously Chasmagnathus) granulata, here we describe the response of these neurons to looming stimuli that simulate objects approaching the animal on a collision course. We found that the peak firing time of MLG1 acts as an angular threshold detector signaling, with a delay of δ = 35 ms, the time at which an object reaches a fixed angular threshold of 49°. Using in vivo intracellular recordings, we detected the existence of excitatory and inhibitory synaptic currents that shape the neural response. Other functional features identified in the MLG1 neurons were phas...
---
paper_title: Escape Behavior: Linking Neural Computation to Action

paper_content:

A new study uses a combination of physiological and optogenetic techniques to identify visual neurons in fruit flies that detect approaching objects, and whose activation is integral in escaping an oncoming threat.
---
paper_title: Collision-avoidance and landing responses are mediated by separate pathways in the fruit fly, Drosophila melanogaster.

paper_content:

Flies rely heavily on visual feedback for several aspects of flight control. As a fly approaches an object, the image projected across its retina expands, providing the fly with visual feedback that can be used either to trigger a collision-avoidance maneuver or a landing response. To determine how a fly makes the decision to land on or avoid a looming object, we measured the behaviors generated in response to an expanding image during tethered flight in a visual closed-loop flight arena. During these experiments, each fly varied its wing-stroke kinematics to actively control the azimuth position of a 15°×15° square within its visual field. Periodically, the square symmetrically expanded in both the horizontal and vertical directions. We measured changes in the fly's wing-stroke amplitude and frequency in response to the expanding square while optically tracking the position of its legs to monitor stereotyped landing responses. Although this stimulus could elicit both the landing responses and collision-avoidance reactions, separate pathways appear to mediate the two behaviors. For example, if the square is in the lateral portion of the fly's field of view at the onset of expansion, the fly increases stroke amplitude in one wing while decreasing amplitude in the other, indicative of a collision-avoidance maneuver. In contrast, frontal expansion elicits an increase in wing-beat frequency and leg extension, indicative of a landing response. To further characterize the sensitivity of these responses to expansion rate, we tested a range of expansion velocities from 100 to 10000° s^(-1). Differences in the latency of both the collision-avoidance reactions and the landing responses with expansion rate supported the hypothesis that the two behaviors are mediated by separate pathways. To examine the effects of visual feedback on the magnitude and time course of the two behaviors, we presented the stimulus under open-loop conditions, such that the fly's response did not alter the position of the expanding square. From our results we suggest a model that takes into account the spatial sensitivities and temporal latencies of the collision-avoidance and landing responses, and is sufficient to schematically represent how the fly uses integration of motion information in deciding whether to turn or land when confronted with an expanding object.
---
paper_title: Organization of columnar inputs in the third optic ganglion of a highly visual crab

paper_content:

Motion information provides essential cues for a wide variety of animal behaviors such as mate, prey, or predator detection. In decapod crustaceans and pterygote insects, visual codification of object motion is associated with visual processing in the third optic neuropile, the lobula. In this neuropile, tangential neurons collect motion information from small field columnar neurons and relay it to the midbrain where behavioral responses would be finally shaped. In highly ordered structures, detailed knowledge of the neuroanatomy can give insight into their function. In spite of the relevance of the lobula in processing motion information, studies on the neuroarchitecture of this neuropile are scant. Here, by applying dextran-conjugated dyes in the second optic neuropile (the medulla) of the crab Neohelice, we mass stained the columnar neurons that convey visual information into the lobula. We found that the arborizations of these afferent columnar neurons lie at four main lobula depths. A detailed examination of serial optical sections of the lobula revealed that these input strata are composed of different number of substrata and that the strata are thicker in the centre of the neuropile. Finally, by staining the different lobula layers composed of tangential processes we combined the present characterization of lobula input strata with the previous characterization of the neuroarchitecture of the crab's lobula based on reduced-silver preparations. We found that the third lobula input stratum overlaps with the dendrites of lobula giant tangential neurons. This suggests that columnar neurons projecting from the medulla can directly provide visual input to the crab's lobula giant neurons.
---
paper_title: Flies Evade Looming Targets by Executing Rapid Visually Directed Banked Turns

paper_content:

Avoiding predators is an essential behavior in which animals must quickly transform sensory cues into evasive actions. Sensory reflexes are particularly fast in flying insects such as flies, but the means by which they evade aerial predators is not known. Using high-speed videography and automated tracking of flies in combination with aerodynamic measurements on flapping robots, we show that flying flies react to looming stimuli with directed banked turns. The maneuver consists of a rapid body rotation followed immediately by an active counter-rotation and is enacted by remarkably subtle changes in wing motion. These evasive maneuvers of flies are substantially faster than steering maneuvers measured previously and indicate the existence of sensory-motor circuitry that can reorient the fly’s flight path within a few wingbeats.
---
paper_title: Escape behavior and neuronal responses to looming stimuli in the crab Chasmagnathus granulatus (Decapoda: Grapsidae)

paper_content:

Behavioral responses to looming stimuli have been studied in many vertebrate and invertebrate species, but neurons sensitive to looming have been investigated in very few animals. In this paper we introduce a new experimental model using the crab Chasmagnathus granulatus, which allows investigation of the processes of looming detection and escape decision at both the behavioral and neuronal levels. By analyzing the escape response of the crab in a walking simulator device we show that: (i) a robust and reliable escape response can be elicited by computer-generated looming stimuli in all tested animals; (ii) parameters such as distance, speed, timing and directionality of the escape run, are easy to record and quantify precisely in the walking device; (iii) although the magnitude of escape varies between animals and stimulus presentations, the timing of the response is remarkably consistent and does not habituate at 3 min stimulus intervals. We then study the response of neurons from the brain of the crab by means of intracellular recordings in the intact animal and show that: (iv) two subclasses of previously identified movement detector neurons from the lobula (third optic neuropil) exhibit robust and reliable responses to the same looming stimuli that trigger the behavioral response; (v) the neurons respond to the object approach by increasing their rate of firing in a way that closely matches the dynamics of the image expansion. Finally, we compare the neuronal with the behavioral response showing that: (vi) differences in the neuronal responses to looming, receding or laterally moving stimuli closely reflect the behavioral differences to such stimuli; (vii) during looming, the crab starts to run soon after the looming-sensitive neurons begin to increase their firing rate. The increase in the running speed during stimulus approach faithfully follows the increment in the firing rate, until the moment of maximum stimulus expansion. Thereafter, the neurons abruptly stop firing and the animal immediately decelerates its run. The results are discussed in connection with studies of responses to looming stimuli in the locust.
---
paper_title: Elementary motion detectors

paper_content:

A quick guide to the elementary motion detector— a model of how a simple neural circuit can detect visual motion, developed from work on insect vision but which seems also to be relevant to vertebrate visual systems.
---
paper_title: Neuronal correlates of the visually elicited escape response of the crab Chasmagnathus upon seasonal variations, stimuli changes and perceptual alterations

paper_content:

When confronted with predators, animals are forced to take crucial decisions such as the timing and manner of escape. In the case of the crab Chasmagnathus, cumulative evidence suggests that the escape response to a visual danger stimulus (VDS) can be accounted for by the response of a group of lobula giant (LG) neurons. To further investigate this hypothesis, we examined the relationship between behavioral and neuronal activities within a variety of experimental conditions that affected the level of escape. The intensity of the escape response to VDS was influenced by seasonal variations, changes in stimulus features, and whether the crab perceived stimuli monocularly or binocularly. These experimental conditions consistently affected the response of LG neurons in a way that closely matched the effects observed at the behavioral level. In other words, the intensity of the stimulus-elicited spike activity of LG neurons faithfully reflected the intensity of the escape response. These results support the idea that the LG neurons from the lobula of crabs are deeply involved in the decision for escaping from VDS.
---
paper_title: IDENTIFICATION OF DIRECTIONALLY SELECTIVE MOTION-DETECTING NEURONES IN THE LOCUST LOBULA AND THEIR SYNAPTIC CONNECTIONS WITH AN IDENTIFIED DESCENDING NEURONE

paper_content:

The anatomy and physiology of two directionally selective motion-detecting neurones in the locust are described. Both neurones had dendrites in the lobula, and projected to the ipsilateral protocerebrum. Their cell bodies were located on the posterio-dorsal junction of the optic lobe with the protocerebrum. The neurones were sensitive to horizontal motion of a visual stimulus. One neurone, LDSMD(F), had a preferred direction forwards over the ipsilateral eye, and a null direction backwards. The other neurone, LDSMD(B), had a preferred direction backwards over the ipsilateral eye ::: ::: 1. 1. Motion in the preferred direction caused EPSPs and spikes in the LDSMD neurones. Motion in the null direction resulted in IPSPs ::: ::: 2. 2. Both excitatory and inhibitory inputs were derived from the ipsilateral eye ::: ::: 3. 3. The DSMD neurones responded to velocities of movement up to and beyond 270°s−1 ::: ::: 4. 4. The response of both LDSMD neurones showed no evidence of adaptation during maintained apparent or real movement ::: ::: 5. 5. There was a delay of 60–80 ms between a single step of apparent movement, either the preferred or the null direction, and the start of the response ::: ::: 6. 6. There was a monosynaptic, excitatory connection between the LDSMD(B) neurone and the protocerebral, descending DSMD neurone (PDDSMD) identified in the preceding paper (Rind, 1990). At resting membrane potential, a single presynaptic spike did not give rise to a spike in the postsynaptic neurone
---
paper_title: Neural networks in the cockpit of the fly

paper_content:

Flies have been buzzing around on earth for over 300 million years. During this time they have radiated into more than 125,000 different species (Yeates and Wiegmann 1999), so that, by now, roughly every tenth described species is a fly. They thus represent one of the most successful animal groups on our planet. This evolutionary success might, at least in part, be a result of their acrobatic maneuverability, which enables them, for example, to chase mates at turning velocities of more than 3000° s–1 with delay times of less than 30 ms (Land and Collett 1974; Wagner 1986). It is this fantastic behavior, which has initiated much research during the last decades, both on its sensory control and the biophysical and aerodynamic principles of the flight output (Dickinson et al. 1999, 2000). Here, we review the current state of knowledge about the neural processing of visual motion, which represents one sensory component intimately involved in flight control. Other reviews on this topic have been published with a similar (Hausen 1981, 1984; Hausen and Egelhaaf 1989; Borst 1996) or different emphasis (Frye and Dickinson 2001; Borst and Dickinson 2002). Because of space limitations, we do not review the extensive work that has been done on fly motion-sensitive neurons to advance our understanding of neural coding (Bialek et al. 1991; Rieke et al. 1997; de Ruyter et al. 1997, 2000; Haag and Borst 1997, 1998; Borst and Haag 2001). Unless stated otherwise, all data presented in the following were obtained on the blowfly Calliphora vicina which we will often casually refer to as 'the fly'.
---
paper_title: Directionally Selective Motion Detection by Insect Neurons

paper_content:

Animals have several good reasons for detecting motion with their eyes. First, the motion of other animals — potential preys, mates, intruders or predators — provides essential information on which to base vital moves such as escape or chase. Secondly, information about self-motion is crucial, especially in the context of navigation, course stabilization, obstacle avoidance, and collision-free goal reaching. In fact, the wealth of information provided by passive, non-contact self-motion evaluation in visual systems has been likened to a kind of “visual kinaesthesis” (Gibson 1958). Even the 3D structure of the environment can be picked up by a moving observer (revs. Collett and Harkness 1982; Buchner 1984; Nakayama 1985; Hildreth and Koch 1987). Von Helmholtz (1867) was the first to clearly state the importance of this “motion parallax” in locomotion, and Exner (1891) proposed that arthropods make use of motion parallax as well as stereopsis to estimate distances (see also Horridge 1986).
---
paper_title: Common circuit design in fly and mammalian motion vision

paper_content:

Motion-sensitive neurons have long been studied in both the mammalian retina and the insect optic lobe, yet striking similarities have become obvious only recently. Detailed studies at the circuit level revealed that, in both systems, (i) motion information is extracted from primary visual information in parallel ON and OFF pathways; (ii) in each pathway, the process of elementary motion detection involves the correlation of signals with different temporal dynamics; and (iii) primary motion information from both pathways converges at the next synapse, resulting in four groups of ON-OFF neurons, selective for the four cardinal directions. Given that the last common ancestor of insects and mammals lived about 550 million years ago, this general strategy seems to be a robust solution for how to compute the direction of visual motion with neural hardware.
---
paper_title: IDENTIFICATION OF DIRECTIONALLY SELECTIVE MOTION-DETECTING NEURONES IN THE LOCUST LOBULA AND THEIR SYNAPTIC CONNECTIONS WITH AN IDENTIFIED DESCENDING NEURONE

paper_content:

The anatomy and physiology of two directionally selective motion-detecting neurones in the locust are described. Both neurones had dendrites in the lobula, and projected to the ipsilateral protocerebrum. Their cell bodies were located on the posterio-dorsal junction of the optic lobe with the protocerebrum. The neurones were sensitive to horizontal motion of a visual stimulus. One neurone, LDSMD(F), had a preferred direction forwards over the ipsilateral eye, and a null direction backwards. The other neurone, LDSMD(B), had a preferred direction backwards over the ipsilateral eye ::: ::: 1. 1. Motion in the preferred direction caused EPSPs and spikes in the LDSMD neurones. Motion in the null direction resulted in IPSPs ::: ::: 2. 2. Both excitatory and inhibitory inputs were derived from the ipsilateral eye ::: ::: 3. 3. The DSMD neurones responded to velocities of movement up to and beyond 270°s−1 ::: ::: 4. 4. The response of both LDSMD neurones showed no evidence of adaptation during maintained apparent or real movement ::: ::: 5. 5. There was a delay of 60–80 ms between a single step of apparent movement, either the preferred or the null direction, and the start of the response ::: ::: 6. 6. There was a monosynaptic, excitatory connection between the LDSMD(B) neurone and the protocerebral, descending DSMD neurone (PDDSMD) identified in the preceding paper (Rind, 1990). At resting membrane potential, a single presynaptic spike did not give rise to a spike in the postsynaptic neurone
---
paper_title: Postsynaptic organisations of directional selective visual neural networks for collision detection

paper_content:

In this paper, we studied the postsynaptic organisations of directional selective visual neurons for collision detection. Directional selective neurons can extract different directional visual motion cues fast and reliably by allowing inhibition spreads to further layers in specific directions with one or several time steps delay. Whether these directional selective neurons can be easily organised for other specific visual tasks is not known. Taking collision detection as the primary visual task, we investigated the postsynaptic organisations of these directional selective neurons through evolutionary processes. The evolved postsynaptic organisations demonstrated robust properties in detecting imminent collisions in complex visual environments with many of which achieved 94% success rate after evolution suggesting active roles in collision detection directional selective neurons and its postsynaptic organisations can play.
---
paper_title: A Synthetic Vision System Using Directionally Selective Motion Detectors to Recognize Collision

paper_content:

Reliably recognizing objects approaching on a collision course is extremely important. A synthetic vision system is proposed to tackle the problem of collision recognition in dynamic environments. The system combines the outputs of four whole-field motion-detecting neurons, each receiving inputs from a network of neurons employing asymmetric lateral inhibition to suppress their responses to one direction of motion. An evolutionary algorithm is then used to adjust the weights between the four motion-detecting neurons to tune the system to detect collisions in two test environments. To do this, a population of agents, each representing a proposed synthetic visual system, either were shown images generated by a mobile Khepera robot navigating in a simplified laboratory environment or were shown images videoed outdoors from a moving vehicle. The agents had to cope with the local environment correctly in order to survive. After 400 generations, the best agent recognized imminent collisions reliably in the familiar environment where it had evolved. However, when the environment was swapped, only the agent evolved to cope in the robotic environment still signaled collision reliably. This study suggests that whole-field direction-selective neurons, with selectivity based on asymmetric lateral inhibition, can be organized into a synthetic vision system, which can then be adapted to play an important role in collision detection in complex dynamic scenes.
---
paper_title: Redundant Neural Vision Systems—Competing for Collision Recognition Roles

paper_content:

Ability to detect collisions is vital for future robots that interact with humans in complex visual environments. Lobula giant movement detectors (LGMD) and directional selective neurons (DSNs) are two types of identified neurons found in the visual pathways of insects such as locusts. Recent modeling studies showed that the LGMD or grouped DSNs could each be tuned for collision recognition. In both biological and artificial vision systems, however, which one should play the collision recognition role and the way the two types of specialized visual neurons could be functioning together are not clear. In this modeling study, we compared the competence of the LGMD and the DSNs, and also investigate the cooperation of the two neural vision systems for collision recognition via artificial evolution. We implemented three types of collision recognition neural subsystems - the LGMD, the DSNs and a hybrid system which combines the LGMD and the DSNs subsystems together, in each individual agent. A switch gene determines which of the three redundant neural subsystems plays the collision recognition role. We found that, in both robotics and driving environments, the LGMD was able to build up its ability for collision recognition quickly and robustly therefore reducing the chance of other types of neural networks to play the same role. The results suggest that the LGMD neural network could be the ideal model to be realized in hardware for collision recognition.
---
paper_title: Sampling of the Visual Environment by the Compound Eye of the Fly: Fundamentals and Applications

paper_content:

This paper reviews some experiments which have been done to elucidate how a fly looks at its surroundings. Stimulation for such a study arose out of the conviction that the neural processing of information in the compound eye might better be unravelled with a precise stimulation of single receptor cells and that such a fine stimulation requires a better understanding of the optics.
---
paper_title: Complementary mechanisms create direction selectivity in the fly

paper_content:

The brain extracts information from signals delivered from the eyes and other sensory organs in order to direct behavior. Understanding how the interactions and wiring of a multitude of individual nerve cells process and transmit this critical information to the brain is a fundamental goal in the field of neuroscience. One question many neuroscientists have tried to understand is how nerve cells in an animal’s brain detect direction when an animal sees movement of some kind – so-called motion vision. The raw signal from the light receptors in the eye does not discriminate whether the light moves in one direction or the other. So, the nerve cells in the brain must somehow compute the direction of movement based on the information relayed by the eye. For more than half a century, major debates have revolved around two rival models that could explain how motion vision works. Both models could in principle lead to neurons that prefer images moving in one direction over images moving in the opposite direction – so-called direction selectivity. In both models, the information about the changing light levels hitting two light-sensitive cells at two points on the eye are compared across time. In one model, signals from images moving in a cell’s preferred direction become amplified. In the other model, signals moving in the unfavored direction become canceled out. However, neither model perfectly explains motion vision. Now, Haag, Arenz et al. show that both models are partially correct and that the two mechanisms work together to detect motion across the field of vision more accurately. In the experiments, both models were tested in tiny fruit flies by measuring the activity of the first nerve cells that respond to the direction of visual motion. While each mechanism alone only produces a fairly weak and error-prone signal of direction, together the two mechanisms produce a stronger and more precise directional signal. Further research is now needed to determine which individual neurons amplify or cancel the signals to achieve such a high degree of direction selectivity.
---
paper_title: Brain Connectivity: Revealing the Fly Visual Motion Circuit

paper_content:

Summary A new semi-automated method for high-throughput identification of visual neurons and their synaptic partners has been combined with optical recording of activity and behavioral analysis to give the first complete description of an elementary circuit for detecting visual motion.
---
paper_title: Object-Detecting Neurons in Drosophila

paper_content:

Summary Many animals rely on vision to detect objects such as conspecifics, predators, and prey. Hypercomplex cells found in feline cortex and small target motion detectors found in dragonfly and hoverfly optic lobes demonstrate robust tuning for small objects, with weak or no response to larger objects or movement of the visual panorama [1–3]. However, the relationship among anatomical, molecular, and functional properties of object detection circuitry is not understood. Here we characterize a specialized object detector in Drosophila , the lobula columnar neuron LC11 [4]. By imaging calcium dynamics with two-photon excitation microscopy, we show that LC11 responds to the omni-directional movement of a small object darker than the background, with little or no responses to static flicker, vertically elongated bars, or panoramic gratings. LC11 dendrites innervate multiple layers of the lobula, and each dendrite spans enough columns to sample 75° of visual space, yet the area that evokes calcium responses is only 20° wide and shows robust responses to a 2.2° object spanning less than half of one facet of the compound eye. The dendrites of neighboring LC11s encode object motion retinotopically, but the axon terminals fuse into a glomerular structure in the central brain where retinotopy is lost. Blocking inhibitory ionic currents abolishes small object sensitivity and facilitates responses to elongated bars and gratings. Our results reveal high-acuity object motion detection in the Drosophila optic lobe.
---
paper_title: A visual motion detection circuit suggested by Drosophila connectomics

paper_content:

Animal behaviour arises from computations in neuronal circuits, but our understanding of these computations has been frustrated by the lack of detailed synaptic connection maps, or connectomes. For example, despite intensive investigations over half a century, the neuronal implementation of local motion detection in the insect visual system remains elusive. Here we develop a semi-automated pipeline using electron microscopy to reconstruct a connectome, containing 379 neurons and 8,637 chemical synaptic contacts, within the Drosophila optic medulla. By matching reconstructed neurons to examples from light microscopy, we assigned neurons to cell types and assembled a connectome of the repeating module of the medulla. Within this module, we identified cell types constituting a motion detection circuit, and showed that the connections onto individual motion-sensitive neurons in this circuit were consistent with their direction selectivity. Our results identify cellular targets for future functional investigations, and demonstrate that connectomes can provide key insights into neuronal computations.
---
paper_title: Visual Motion: Cellular Implementation of a Hybrid Motion Detector

paper_content:

Visual motion detection in insects is mediated by three-input detectors that compare inputs of different spatiotemporal properties. A new modeling study shows that only a small subset of possible arrangements of the input elements provides high direction-selectivity.
---
paper_title: Neural networks in the cockpit of the fly

paper_content:

Flies have been buzzing around on earth for over 300 million years. During this time they have radiated into more than 125,000 different species (Yeates and Wiegmann 1999), so that, by now, roughly every tenth described species is a fly. They thus represent one of the most successful animal groups on our planet. This evolutionary success might, at least in part, be a result of their acrobatic maneuverability, which enables them, for example, to chase mates at turning velocities of more than 3000° s–1 with delay times of less than 30 ms (Land and Collett 1974; Wagner 1986). It is this fantastic behavior, which has initiated much research during the last decades, both on its sensory control and the biophysical and aerodynamic principles of the flight output (Dickinson et al. 1999, 2000). Here, we review the current state of knowledge about the neural processing of visual motion, which represents one sensory component intimately involved in flight control. Other reviews on this topic have been published with a similar (Hausen 1981, 1984; Hausen and Egelhaaf 1989; Borst 1996) or different emphasis (Frye and Dickinson 2001; Borst and Dickinson 2002). Because of space limitations, we do not review the extensive work that has been done on fly motion-sensitive neurons to advance our understanding of neural coding (Bialek et al. 1991; Rieke et al. 1997; de Ruyter et al. 1997, 2000; Haag and Borst 1997, 1998; Borst and Haag 2001). Unless stated otherwise, all data presented in the following were obtained on the blowfly Calliphora vicina which we will often casually refer to as 'the fly'.
---
paper_title: A look into the cockpit of the fly: visual orientation, algorithms, and identified neurons

paper_content:

The top-down approach to understanding brain function seeks to account for the behavior of an animal in terms of biophysical properties of nerve cells and synaptic interactions via a series of progressively reductive levels of explanation. Using the fly as a model system, this approach was pioneered by Werner Reichardt and his colleagues in the late 1950s. Quantitative input-output analyses led them to formal algorithms that related the input of the fly's eye to the orientation behavior of the animal. But it has been possible only recently to track down the implementation of part of these algorithms to the computations performed by individual neurons and small neuronal ensembles. Thus, the visually guided flight maneuvers of the fly have turned out to be one of the few cases in which it has been feasible to reach an understanding of the mechanisms underlying a complex behavioral performance at successively reductive levels of analysis. These recent findings illuminate some of the fundamental questions that are being debated in computational neuroscience (Marr and Poggio, 1977; Sejnowski et al., 1988; Churchland and Sejnowski, 1992): (1) Are some brain functions emergent properties present only at the systems level? (2) Does an understanding of brain function at the systems level help in understanding function at the cellular and subcellular level? (3) Can different levels of organization be understood independently of each other? In this review we concentrate on two basic computational tasks that have to be solved by the fly, as well as by many other moving animals: (1) stabilization of an intended course against disturbances and (2) intended deviations from a straight course in order to orient toward salient objecs. Performing these tasks depends on the extraction of motion information from the changing distribution of light intensity received by the eyes.
---
paper_title: Directionally Selective Motion Detection by Insect Neurons

paper_content:

Animals have several good reasons for detecting motion with their eyes. First, the motion of other animals — potential preys, mates, intruders or predators — provides essential information on which to base vital moves such as escape or chase. Secondly, information about self-motion is crucial, especially in the context of navigation, course stabilization, obstacle avoidance, and collision-free goal reaching. In fact, the wealth of information provided by passive, non-contact self-motion evaluation in visual systems has been likened to a kind of “visual kinaesthesis” (Gibson 1958). Even the 3D structure of the environment can be picked up by a moving observer (revs. Collett and Harkness 1982; Buchner 1984; Nakayama 1985; Hildreth and Koch 1987). Von Helmholtz (1867) was the first to clearly state the importance of this “motion parallax” in locomotion, and Exner (1891) proposed that arthropods make use of motion parallax as well as stereopsis to estimate distances (see also Horridge 1986).
---
paper_title: Feedback Network Controls Photoreceptor Output at the Layer of First Visual Synapses in Drosophila

paper_content:

At the layer of first visual synapses, information from photoreceptors is processed and transmitted towards the brain. In fly compound eye, output from photoreceptors (R1-R6) that share the same visual field is pooled and transmitted via histaminergic synapses to two classes of interneuron, large monopolar cells (LMCs) and amacrine cells (ACs). The interneurons also feed back to photoreceptor terminals via numerous ligand-gated synapses, yet the significance of these connections has remained a mystery. We investigated the role of feedback synapses by comparing intracellular responses of photoreceptors and LMCs in wild-type Drosophila and in synaptic mutants, to light and current pulses and to naturalistic light stimuli. The recordings were further subjected to rigorous statistical and information-theoretical analysis. We show that the feedback synapses form a negative feedback loop that controls the speed and amplitude of photoreceptor responses and hence the quality of the transmitted signals. These results highlight the benefits of feedback synapses for neural information processing, and suggest that similar coding strategies could be used in other nervous systems.
---
paper_title: The Temporal Tuning of the Drosophila Motion Detectors Is Determined by the Dynamics of Their Input Elements

paper_content:

Summary Detecting the direction of motion contained in the visual scene is crucial for many behaviors. However, because single photoreceptors only signal local luminance changes, motion detection requires a comparison of signals from neighboring photoreceptors across time in downstream neuronal circuits. For signals to coincide on readout neurons that thus become motion and direction selective, different input lines need to be delayed with respect to each other. Classical models of motion detection rely on non-linear interactions between two inputs after different temporal filtering. However, recent studies have suggested the requirement for at least three, not only two, input signals. Here, we comprehensively characterize the spatiotemporal response properties of all columnar input elements to the elementary motion detectors in the fruit fly, T4 and T5 cells, via two-photon calcium imaging. Between these input neurons, we find large differences in temporal dynamics. Based on this, computer simulations show that only a small subset of possible arrangements of these input elements maps onto a recently proposed algorithmic three-input model in a way that generates a highly direction-selective motion detector, suggesting plausible network architectures. Moreover, modulating the motion detection system by octopamine-receptor activation, we find the temporal tuning of T4 and T5 cells to be shifted toward higher frequencies, and this shift can be fully explained by the concomitant speeding of the input elements.
---
paper_title: The Emergence of Directional Selectivity in the Visual Motion Pathway of Drosophila

paper_content:

Summary The perception of visual motion is critical for animal navigation, and flies are a prominent model system for exploring this neural computation. In Drosophila , the T4 cells of the medulla are directionally selective and necessary for ON motion behavioral responses. To examine the emergence of directional selectivity, we developed genetic driver lines for the neuron types with the most synapses onto T4 cells. Using calcium imaging, we found that these neuron types are not directionally selective and that selectivity arises in the T4 dendrites. By silencing each input neuron type, we identified which neurons are necessary for T4 directional selectivity and ON motion behavioral responses. We then determined the sign of the connections between these neurons and T4 cells using neuronal photoactivation. Our results indicate a computational architecture for motion detection that is a hybrid of classic theoretical models.
---
paper_title: Principles of visual motion detection

paper_content:

Motion information is required for the solution of many complex tasks of the visual system such as depth perception by motion parallax and figure/ground discrimination by relative motion. However, motion information is not explicitly encoded at the level of the retinal input. Instead, it has to be computed from the time-dependent brightness patterns of the retinal image as sensed by the two-dimensional array of photoreceptors. Different models have been proposed which describe the neural computations underlying motion detection in various ways. To what extent do biological motion detectors approximate any of these models? As will be argued here, there is increasing evidence from the different disciplines studying biological motion vision, that, throughout the animal kingdom ranging from invertebrates to vertebrates including man, the mechanisms underlying motion detection can be attributed to only a few, essentially equivalent computational principles. Motion detection may, therefore, be one of the first examples in computational neurosciences where common principles can be found not only at the cellular level (e.g. dendritic integration, spike propagation, synaptic transmission) but also at the level of computations performed by small neural networks.
---
paper_title: Impact and sources of neuronal variability in the fly’s motion vision pathway

paper_content:

Abstract Nervous systems encode information about dynamically changing sensory input by changes in neuronal activity. Neuronal activity changes, however, also arise from noise sources within and outside the nervous system or from changes of the animal’s behavioral state. The resulting variability of neuronal responses in representing sensory stimuli limits the reliability with which animals can respond to stimuli and may thus even affect the chances for survival in certain situations. Relevant sources of noise arising at different stages along the motion vision pathway have been investigated from the sensory input to the initiation of behavioral reactions. Here, we concentrate on the reliability of processing visual motion information in flies. Flies rely on visual motion information to guide their locomotion. They are among the best established model systems for the processing of visual motion information allowing us to bridge the gap between behavioral performance and underlying neuronal computations. It has been possible to directly assess the consequences of noise at major stages of the fly’s visual motion processing system on the reliability of neuronal signals. Responses of motion sensitive neurons and their variability have been related to optomotor movements as indicators for the overall performance of visual motion computation. We address whether and how noise already inherent in the stimulus, e.g. photon noise for the visual system, influences later processing stages and to what extent variability at the output level of the sensory system limits behavioral performance. Recent advances in circuit analysis and the progress in monitoring neuronal activity in behaving animals should now be applied to understand how the animal meets the requirements of fast and reliable manoeuvres in naturalistic situations.
---
paper_title: Redundant Neural Vision Systems—Competing for Collision Recognition Roles

paper_content:

Ability to detect collisions is vital for future robots that interact with humans in complex visual environments. Lobula giant movement detectors (LGMD) and directional selective neurons (DSNs) are two types of identified neurons found in the visual pathways of insects such as locusts. Recent modeling studies showed that the LGMD or grouped DSNs could each be tuned for collision recognition. In both biological and artificial vision systems, however, which one should play the collision recognition role and the way the two types of specialized visual neurons could be functioning together are not clear. In this modeling study, we compared the competence of the LGMD and the DSNs, and also investigate the cooperation of the two neural vision systems for collision recognition via artificial evolution. We implemented three types of collision recognition neural subsystems - the LGMD, the DSNs and a hybrid system which combines the LGMD and the DSNs subsystems together, in each individual agent. A switch gene determines which of the three redundant neural subsystems plays the collision recognition role. We found that, in both robotics and driving environments, the LGMD was able to build up its ability for collision recognition quickly and robustly therefore reducing the chance of other types of neural networks to play the same role. The results suggest that the LGMD neural network could be the ideal model to be realized in hardware for collision recognition.
---
paper_title: Direction Selectivity in Drosophila Emerges from Preferred-Direction Enhancement and Null-Direction Suppression

paper_content:

Across animal phyla, motion vision relies on neurons that respond preferentially to stimuli moving in one, preferred direction over the opposite, null direction. In the elementary motion detector of Drosophila , direction selectivity emerges in two neuron types, T4 and T5, but the computational algorithm underlying this selectivity remains unknown. We find that the receptive fields of both T4 and T5 exhibit spatiotemporally offset light-preferring and dark-preferring subfields, each obliquely oriented in spacetime. In a linear-nonlinear modeling framework, the spatiotemporal organization of the T5 receptive field predicts the activity of T5 in response to motion stimuli. These findings demonstrate that direction selectivity emerges from the enhancement of responses to motion in the preferred direction, as well as the suppression of responses to motion in the null direction. Thus, remarkably, T5 incorporates the essential algorithmic strategies used by the Hassenstein–Reichardt correlator and the Barlow–Levick detector. Our model for T5 also provides an algorithmic explanation for the selectivity of T5 for moving dark edges: our model captures all two- and three-point spacetime correlations relevant to motion in this stimulus class. More broadly, our findings reveal the contribution of input pathway visual processing, specifically center-surround, temporally biphasic receptive fields, to the generation of direction selectivity in T5. As the spatiotemporal receptive field of T5 in Drosophila is common to the simple cell in vertebrate visual cortex, our stimulus-response model of T5 will inform efforts in an experimentally tractable context to identify more detailed, mechanistic models of a prevalent computation. SIGNIFICANCE STATEMENT Feature selective neurons respond preferentially to astonishingly specific stimuli, providing the neurobiological basis for perception. Direction selectivity serves as a paradigmatic model of feature selectivity that has been examined in many species. While insect elementary motion detectors have served as premiere experimental models of direction selectivity for 60 years, the central question of their underlying algorithm remains unanswered. Using in vivo two-photon imaging of intracellular calcium signals, we measure the receptive fields of the first direction-selective cells in the Drosophila visual system, and define the algorithm used to compute the direction of motion. Computational modeling of these receptive fields predicts responses to motion and reveals how this circuit efficiently captures many useful correlations intrinsic to moving dark edges.
---
paper_title: Chromatic Organization and Sexual Dimorphism of the Fly Retinal Mosaic

paper_content:

Whether in vertebrates or in invertebrates, mapping the spectral organization of a retinal mosaic with single cell resolution often remains an insuperable task, for which microspectrophotometry and intracellular recordings appear as cumbersome tools. The need for methods capable of revealing at a glimpse the mosaic pattern of the individual spectral types across a large receptor array has led to many ingenious techniques, some of which are listed in Table 1.
---
paper_title: Complementary mechanisms create direction selectivity in the fly

paper_content:

The brain extracts information from signals delivered from the eyes and other sensory organs in order to direct behavior. Understanding how the interactions and wiring of a multitude of individual nerve cells process and transmit this critical information to the brain is a fundamental goal in the field of neuroscience. One question many neuroscientists have tried to understand is how nerve cells in an animal’s brain detect direction when an animal sees movement of some kind – so-called motion vision. The raw signal from the light receptors in the eye does not discriminate whether the light moves in one direction or the other. So, the nerve cells in the brain must somehow compute the direction of movement based on the information relayed by the eye. For more than half a century, major debates have revolved around two rival models that could explain how motion vision works. Both models could in principle lead to neurons that prefer images moving in one direction over images moving in the opposite direction – so-called direction selectivity. In both models, the information about the changing light levels hitting two light-sensitive cells at two points on the eye are compared across time. In one model, signals from images moving in a cell’s preferred direction become amplified. In the other model, signals moving in the unfavored direction become canceled out. However, neither model perfectly explains motion vision. Now, Haag, Arenz et al. show that both models are partially correct and that the two mechanisms work together to detect motion across the field of vision more accurately. In the experiments, both models were tested in tiny fruit flies by measuring the activity of the first nerve cells that respond to the direction of visual motion. While each mechanism alone only produces a fairly weak and error-prone signal of direction, together the two mechanisms produce a stronger and more precise directional signal. Further research is now needed to determine which individual neurons amplify or cancel the signals to achieve such a high degree of direction selectivity.
---
paper_title: Evaluation of optical motion information by movement detectors

paper_content:

The paper is dealing in its first part with a system-theoretical approach for the decomposition of multi-input systems into the sum of simpler systems. By this approach the algorithm for the computations underlying the extraction of motion information from the optical environment by biological movement detectors is analysed. In the second part it concentrates on a specific model for motion computation known to be realized by the visual system of insects and of man. These motion detectors provide the visual system with information on both, velocity and structural properties of a moving pattern. The last part of the paper deals with the functional properties of two-dimensional arrays of movement detectors. They are analyzed and their relations to meaningful physiological responses are discussed.
---
paper_title: Adaptation and the temporal delay filter of fly motion detectors

paper_content:

Recent accounts attribute motion adaptation to a shortening of the delay filter in elementary motion detectors (EMDs). Using computer modelling and recordings from HS neurons in the drone-fly Eristalis tenax, we present evidence that challenges this theory. (i) Previous evidence for a change in the delay filter comes from ‘image step’ (or ‘velocity impulse’) experiments. We note a large discrepancy between the temporal frequency tuning predicted from these experiments and the observed tuning of motion sensitive cells. (ii) The results of image step experiments are highly sensitive to the experimental method used. (iii) An apparent motion stimulus reveals a much shorter EMD delay than suggested by previous ‘image step’ experiments. This short delay agrees with the observed temporal frequency sensitivity of the unadapted cell. (iv) A key prediction of a shortening delay filter is that the temporal frequency optimum of the cell should show a large shift to higher temporal frequencies after motion adaptation. We show little change in the temporal or spatial frequency (and hence velocity) optima following adaptation.
---
paper_title: The Temporal Tuning of the Drosophila Motion Detectors Is Determined by the Dynamics of Their Input Elements

paper_content:

Summary Detecting the direction of motion contained in the visual scene is crucial for many behaviors. However, because single photoreceptors only signal local luminance changes, motion detection requires a comparison of signals from neighboring photoreceptors across time in downstream neuronal circuits. For signals to coincide on readout neurons that thus become motion and direction selective, different input lines need to be delayed with respect to each other. Classical models of motion detection rely on non-linear interactions between two inputs after different temporal filtering. However, recent studies have suggested the requirement for at least three, not only two, input signals. Here, we comprehensively characterize the spatiotemporal response properties of all columnar input elements to the elementary motion detectors in the fruit fly, T4 and T5 cells, via two-photon calcium imaging. Between these input neurons, we find large differences in temporal dynamics. Based on this, computer simulations show that only a small subset of possible arrangements of these input elements maps onto a recently proposed algorithmic three-input model in a way that generates a highly direction-selective motion detector, suggesting plausible network architectures. Moreover, modulating the motion detection system by octopamine-receptor activation, we find the temporal tuning of T4 and T5 cells to be shifted toward higher frequencies, and this shift can be fully explained by the concomitant speeding of the input elements.
---
paper_title: Principles of visual motion detection

paper_content:

Motion information is required for the solution of many complex tasks of the visual system such as depth perception by motion parallax and figure/ground discrimination by relative motion. However, motion information is not explicitly encoded at the level of the retinal input. Instead, it has to be computed from the time-dependent brightness patterns of the retinal image as sensed by the two-dimensional array of photoreceptors. Different models have been proposed which describe the neural computations underlying motion detection in various ways. To what extent do biological motion detectors approximate any of these models? As will be argued here, there is increasing evidence from the different disciplines studying biological motion vision, that, throughout the animal kingdom ranging from invertebrates to vertebrates including man, the mechanisms underlying motion detection can be attributed to only a few, essentially equivalent computational principles. Motion detection may, therefore, be one of the first examples in computational neurosciences where common principles can be found not only at the cellular level (e.g. dendritic integration, spike propagation, synaptic transmission) but also at the level of computations performed by small neural networks.
---
paper_title: Elementary motion detectors

paper_content:

A quick guide to the elementary motion detector— a model of how a simple neural circuit can detect visual motion, developed from work on insect vision but which seems also to be relevant to vertebrate visual systems.
---
paper_title: Time-of-Travel Methods for Measuring Optical Flow on Board a Micro Flying Robot

paper_content:

For use in autonomous micro air vehicles, visual sensors must not only be small, lightweight and insensitive to light variations; on-board autopilots also require fast and accurate optical flow measurements over a wide range of speeds. Using an auto-adaptive bio-inspired Michaelis–Menten Auto-adaptive Pixel (M 2 APix) analog silicon retina, in this article, we present comparative tests of two optical flow calculation algorithms operating under lighting conditions from 6 × 10 − 7 to 1 . 6 × 10 − 2 W·cm − 2 (i.e., from 0.2 to 12,000 lux for human vision). Contrast “time of travel” between two adjacent light-sensitive pixels was determined by thresholding and by cross-correlating the two pixels’ signals, with measurement frequency up to 5 kHz for the 10 local motion sensors of the M 2 APix sensor. While both algorithms adequately measured optical flow between 25 ∘ /s and 1000 ∘ /s, thresholding gave rise to a lower precision, especially due to a larger number of outliers at higher speeds. Compared to thresholding, cross-correlation also allowed for a higher rate of optical flow output (99 Hz and 1195 Hz, respectively) but required substantially more computational resources.
---
paper_title: Minimalistic optic flow sensors applied to indoor and outdoor visual guidance and odometry on a car-like robot

paper_content:

Here we present a novel bio-inspired optic flow (OF) sensor and its application to visual guidance and odometry on a low-cost car-like robot called BioCarBot. The minimalistic OF sensor was robust to high-dynamic-range lighting conditions and to various visual patterns encountered thanks to its M2APIX auto-adaptive pixels and the new cross-correlation OF algorithm implemented. The low-cost car-like robot estimated its velocity and steering angle, and therefore its position and orientation, via an extended Kalman filter (EKF) using only two downward-facing OF sensors and the Ackerman steering model. Indoor and outdoor experiments were carried out in which the robot was driven in the closed-loop mode based on the velocity and steering angle estimates. The experimental results obtained show that our novel OF sensor can deliver high-frequency measurements ([Formula: see text]) in a wide OF range (1.5-[Formula: see text]) and in a 7-decade high-dynamic light level range. The OF resolution was constant and could be adjusted as required (up to [Formula: see text]), and the OF precision obtained was relatively high (standard deviation of [Formula: see text] with an average OF of [Formula: see text], under the most demanding lighting conditions). An EKF-based algorithm gave the robot's position and orientation with a relatively high accuracy (maximum errors outdoors at a very low light level: [Formula: see text] and [Formula: see text] over about [Formula: see text] and [Formula: see text]) despite the low-resolution control systems of the steering servo and the DC motor, as well as a simplified model identification and calibration. Finally, the minimalistic OF-based odometry results were compared to those obtained using measurements based on an inertial measurement unit (IMU) and a motor's speed sensor.
---
paper_title: A bee in the corridor: centering and wall-following

paper_content:

In an attempt to better understand the mechanism underlying lateral collision avoidance in flying insects, we trained honeybees (Apis mellifera) to fly through a large (95-cm wide) flight tunnel. We found that, depending on the entrance and feeder positions, honeybees would either center along the corridor midline or fly along one wall. Bees kept following one wall even when a major (150-cm long) part of the opposite wall was removed. These findings cannot be accounted for by the “optic flow balance” hypothesis that has been put forward to explain the typical bees’ “centering response” observed in narrower corridors. Both centering and wall-following behaviors are well accounted for, however, by a control scheme called the lateral optic flow regulator, i.e., a feedback system that strives to maintain the unilateral optic flow constant. The power of this control scheme is that it would allow the bee to guide itself visually in a corridor without having to measure its speed or distance from the walls.
---
paper_title: Visual Guidance Of A Mobile Robot Equipped With A Network Of Self-Motion Sensors

paper_content:

This paper reports on the principles of an on-board electro-optical system for the guidance of an autonomous mobile robot. Some of the signal processing adopted here was directly inspired by natural visual systems, in particular by the compound eye of the fly. The visual system has compound optics with a panoramic field but relatively low spatial resolution. It makes use of elementary motion detectors (E.M.D's) to estimate the distance to objects from the optic flow. Each E.M.D. constitutes one mesh of an analog network. It measures the relative angular velocity of any contrast point that passes across its receptive field as a result of the robot's own motion and evaluates the radial distance to this contrast point from the motion parallax. For this purpose, the mobile makes translation steps at constant speed during each visual acquisition. An obstacle avoidance algorithm is implemented on a parallel, analog network. This network integrates the numerous data provided by the E.M.D's and controls the drive motor and steering motor of the robot platform in real time. Other navigation modules may be added without altering the basic hardware architecture of the system. For example, a target detector has been associated with the system. No stringent hypothesis needs to be made as to the shape of objects in the environment. Both the visual processing principles and the obstacle avoidance strategy are described.
---
paper_title: Speed tuning in elementary motion detectors of the correlation type

paper_content:

A prominent model of visual motion detection is the so-called correlation or Reichardt detector. Whereas this model can account for many properties of motion vision, from humans to insects (review, Borst and Egelhaaf 1989), it has been commonly assumed that this scheme of motion detection is not well suited to the measurement of image velocity. This is because the commonly used version of the model, which incorporates two unidirectional motion detectors with opposite preferred directions, produces a response which varies not only with the velocity of the image, but also with its spatial structure and contrast. On the other hand, information on image velocity can be crucial in various contexts, and a number of recent behavioural experiments suggest that insects do extract velocity for navigational purposes (review, Srinivasan et al. 1996). Here we show that other versions of the correlation model, which consists of a single unidirectional motion detector or incorporates two oppositely directed detectors with unequal sensitivities, produce responses which vary with image speed and display tuning curves that are substantially independent of the spatial structure of the image. This surprising feature suggests simple strategies of reducing ambiguities in the estimation of speed by using components of neural hardware that are already known to exist in the visual system.
---
paper_title: Miniature curved artificial compound eyes.

paper_content:

In most animal species, vision is mediated by compound eyes, which offer lower resolution than vertebrate single-lens eyes, but significantly larger fields of view with negligible distortion and spherical aberration, as well as high temporal resolution in a tiny package. Compound eyes are ideally suited for fast panoramic motion perception. Engineering a miniature artificial compound eye is challenging because it requires accurate alignment of photoreceptive and optical components on a curved surface. Here, we describe a unique design method for biomimetic compound eyes featuring a panoramic, undistorted field of view in a very thin package. The design consists of three planar layers of separately produced arrays, namely, a microlens array, a neuromorphic photodetector array, and a flexible printed circuit board that are stacked, cut, and curved to produce a mechanically flexible imager. Following this method, we have prototyped and characterized an artificial compound eye bearing a hemispherical field of view with embedded and programmable low-power signal processing, high temporal resolution, and local adaptation to illumination. The prototyped artificial compound eye possesses several characteristics similar to the eye of the fruit fly Drosophila and other arthropod species. This design method opens up additional vistas for a broad range of applications in which wide field motion detection is at a premium, such as collision-free navigation of terrestrial and aerospace vehicles, and for the experimental testing of insect vision theories.
---
paper_title: Toward an insect-inspired event-based autopilot combining both visual and control events

paper_content:

This paper presents the autopilot and the behavior of a “simulated bee” traveling along two different tunnels using both visual and control events. The computational gain of an event-based PID controller compared to its time-based version is usually discussed because the event detector is computationally expensive. By combining visual and control events, the newly suggested event-based autopilot requires very low computational resources. In particular, the event detector which computes the control error and tests its magnitude is activated only when a new contrast is detected by the optic motion detectors that assesses the optic flow, i.e. only when the magnitude of the optic flow error could have changed. This new event-based control strategy used faithfully the visual information already available in the optic flow sensor to reduce even further the computational cost. The “simulated bee” was equipped with: (i) a minimalistic compound eye comprising 10 or 8 local motion sensors (depending of the tunnel configuration) measuring the optic flow magnitude, (ii) two optic flow regulators updating the control signals whenever specific optic flow criteria changed and (iii) three event-based controllers taking into account both error signals and visual events, each one in charge of its own translational dynamics. The “simulated bee” managed to travel safely along the tunnels without requiring any speed or distance measurements, using very low computational resources, by (i) concomitantly adjusting the side thrust, vertical lift and forward thrust only when both a visual contrast and a change of optic flow control error were detected, and (ii) avoiding collisions with the surface of the tunnels and decreasing or increasing its speed, depending on the clutter rate perceived by motion sensors.
---
paper_title: Fly visual system inspired artificial neural network for collision detection

paper_content:

This work investigates one bio-inspired collision detection system based on fly visual neural structures, in which collision alarm is triggered if an approaching object in a direct collision course appears in the field of view of a camera or a robot, together with the relevant time region of collision. One such artificial system consists of one artificial fly visual neural network model and one collision detection mechanism. The former one is a computational model to capture membrane potentials produced by neurons. The latter one takes the outputs of the former one as its inputs, and executes three detection schemes: (i) identifying when a spike takes place through the membrane potentials and one threshold scheme; (ii) deciding the motion direction of a moving object by the Reichardt detector model; and (iii) sending collision alarms and collision regions. Experimentally, relying upon a series of video image sequences with different scenes, numerical results illustrated that the artificial system with some striking characteristics is a potentially alternative tool for collision detection.
---
paper_title: Navigation in an autonomous flying robot by using a biologically inspired visual odometer

paper_content:

While mobile robots and walking insects can use proprioceptive information (specialized receptors in the insects' leg, or wheel encoders in robots) to estimate distance traveled, flying agents have to rely mainly on visual cues. Experiments with bees provide evidence that flying insects might be using optical flow induced by egomotion to estimate distance traveled. Recently some details of this odometer have been unraveled. In this study, we propose a biologically inspired model of the bee's visual odometer based on Elementary Motion Detectors (EMDs), and present results from goal-directed navigation experiments with an autonomous flying robot platform that we developed specifically for this purpose. The robot is equipped with a panoramic vision system, which is used to provide input to the EMDs of the left and right visual fields. The outputs of the EMDs are in later stage spatially integrated by wide field motion detectors, and their accumulated response is directly used for the odometer. In a set of initial experiments, the robot moves through a corridor on a fixed route, and the outputs of EMDs, the odometer, are recorded. The results show that the proposed model can be used to provide an estimate of the distance traveled, but the performance depends on the route the robot follows, something which is biologically plausible since natural insects tend to adopt a fixed route during foraging. Given these results, we assumed that the optomotor response plays an important role in the context of goal-directed navigation, and we conducted experiments with an autonomous freely flying robot. The experiments demonstrate that this computationally cheap mechanism can be successfully employed in natural indoor environments.© (2000) COPYRIGHT SPIE--The International Society for Optical Engineering. Downloading of the abstract is permitted for personal use only.
---
paper_title: Insect Inspired Autopilots

paper_content:

This paper deals with the control problems involved in insects' and robots' visually guided piloting. Explicit control schemes are presented which may explain how insects navigate by relying on optic flow cues, without requiring any distance or speed measurements. The concept of the optic flow regulator, a feedback control system based on OF sensors, is presented. We tested our control schemes in simulation, and implemented them on-board two types of miniature aerial robots, a helicopter and a hovercraft. Their electronic OF sensors were inspired by the results of our microelectrode studies on motion sensitive neurons in the housefly's compound eye. The control schemes described do without any conventional avionic sensors like rangefinders or speedometers, and therefore show great potential for safe autonomous control of aerial, underwater and space vehicles in unchartered environments.
---
paper_title: Obstacle avoidance and speed control in a mobile vehicle equipped with a compound eye

paper_content:

Shows that the use of onboard visual motion sensors has advantages in visually guided navigation. An agent equipped with a given array of motion detectors detects obstacles all the more distant when it is running faster. Its "radius of vision" becomes simply proportional to speed, with the interesting consequence that the agent can easily adapt its speed to the environmental context. The simulation study presented shows that the newly designed visuomotor system allows the robot to reach a target while avoiding obstacles in cluttered environments, adapt its own speed to the situation, follow walls, retreat from dead-ends and bypass impenetrable clusters of obstacles. These performances result all from one and the same visuomotor algorithm and do not require any "arbitration" between different "behaviors" as is the case with the subsumption architecture. Nor do they require any kind of high level reasoning, world model and long term memory. The described system could be part of a hierarchical system and carry out the basic and reputedly difficult task of fast piloting in uncertain environments, for example, between intermediary waypoints that would be specified by a high level mission planner.
---
paper_title: Optic Flow Regulation in Unsteady Environments: A Tethered MAV Achieves Terrain Following and Targeted Landing Over a Moving Platform

paper_content:

The present study deals with the risky and daunting tasks of flying and landing in non-stationary environments. Using a two Degree-Of-Freedom (DOF) tethered micro-air vehicle (MAV), we show the benefits of an autopilot dealing with a variable - the optic flow - which depends directly on two relative variables, the groundspeed and the groundheight. The micro-helicopter was shown to follow the ups and downs of a rotating platform that was also oscillated vertically. At no time did the MAV know in terms of ground height whether it was approaching the moving ground or whether the ground itself was rising to it dangerously. Nor did it know whether its current groundspeed was caused only by its forward thrust or whether it was partly due to the ground moving backwards or forwards. Furthermore, the MAV was shown to land safely on a platform set into motion along two directions, vertical and horizontal. This paper extends to non-stationary environments a former approach that introduced the principle of “optic flow regulation” for altitude control. Whereas in the former approach no requirement was set on the robot’s landing target, the target’s elevation angle was used here in a second feedback loop that gradually altered the robot’s pitch and therefore its airspeed, leading to smooth landing in the vicinity of the target. Whether dealing with terrain following or landing, the MAV followed followed appropriately the unpredictable changes in the environment although it had no explicit knowledge of groundheight and groundspeed. The MAV did not make use of any rangefinders or velocimeters and was simply equipped with a 2-gram vision-based autopilot.
---
paper_title: Motion Detection Circuits for a Time-To-Travel Algorithm

paper_content:

The paper describes a new motion detection circuit that extracts motion information based on a time-to-travel algorithm. The front-end photoreceptor adapts over 7 decades of background intensity and motion information can be extracted down to a contrast value of 2.5%. Results from the circuits which were fabricated in a 2-metal 2-poly 1.5mum CMOS process, show that the motion information can be extracted over 2 decades of speed.
---
paper_title: A Bio-Inspired Flying Robot Sheds Light on Insect Piloting Abilities

paper_content:

When insects are flying forward, the image of the ground sweeps backward across their ventral viewfield and forms an "optic flow," which depends on both the groundspeed and the groundheight. To explain how these animals manage to avoid the ground by using this visual motion cue, we suggest that insect navigation hinges on a visual-feedback loop we have called the optic-flow regulator, which controls the vertical lift. To test this idea, we built a micro-helicopter equipped with an optic-flow regulator and a bio-inspired optic-flow sensor. This fly-by-sight micro-robot can perform exacting tasks such as take-off, level flight, and landing. Our control scheme accounts for many hitherto unexplained findings published during the last 70 years on insects' visually guided performances; for example, it accounts for the fact that honeybees descend in a headwind, land with a constant slope, and drown when travelling over mirror-smooth water. Our control scheme explains how insects manage to fly safely without any of the instruments used onboard aircraft to measure the groundheight, groundspeed, and descent speed. An optic-flow regulator is quite simple in terms of its neural implementation and just as appropriate for insects as it would be for aircraft.
---
paper_title: Obstacle avoidance in a terrestrial mobile robot provided with a scanning retina

paper_content:

This study describes the role of a novel visual sensor, called the "scanning local motion detector" (SLMD) for the visual navigation of a terrestrial mobile agent. Following a brief theoretical description of the principle of retinal scanning coupled with notion parallax, we show that a single retina of 24 pixels - when oriented in the mobile agent's forward line of move - allows the latter to avoid the visually detected obstacles in a reflex manner by correcting its trajectory in real time. The results of the preliminary computer simulations yield credit to a zig-zag algorithm which allows the agent to avoid the oncoming obstacles while immediately "keeping an eye" on its lateral regions. In parallel with the simulation, we designed and built a hardware prototype of a scanning retina with coarse optical resolution (average angular sampling of 3/spl deg/) mounted on top of a circular mobile platform. The preliminary results of the robot's successful performances confirm the validity of the zig-zag algorithm.
---
paper_title: Nondirectional motion may underlie insect behavioral dependence on image speed

paper_content:

Behavioral experiments suggest that insects make use of the apparent image speed on their compound eyes to navigate through obstacles, control flight speed, land smoothly, and measure the distance they have flown. However, the vast majority of electrophysiological recordings from motion-sensitive insect neurons show responses which are tuned in spatial and temporal frequency and are thus unable to unambiguously represent image speed. We suggest that this contradiction may be resolved at an early stage of visual motion processing using nondirectional motion sensors that respond proportionally to image speed until their peak response. We describe and characterize a computational model of these sensors and propose a model by which a spatial collation of such sensors could be used to generate speed-dependent behavior.
---
paper_title: A fully-autonomous hovercraft inspired by bees: Wall following and speed control in straight and tapered corridors

paper_content:

The small autonomous vehicles of the future will have to navigate close to obstacles in highly unpredictable environments. Risky tasks of this kind may require novel sensors and control methods that differ from conventional approaches. Recent ethological findings have shown that complex navigation tasks such as obstacle avoidance and speed control are performed by flying insects on the basis of optic flow (OF) cues, although insects' compound eyes have a very poor spatial resolution. The present paper deals with the implementation of an optic flow-based autopilot on a fully autonomous hovercraft. Tests were performed on this small (878-gram) innovative robotic platform in straight and tapered corridors lined with natural panoramas. A bilateral OF regulator controls the robot's forward speed (up to 0.8m/s), while a unilateral OF regulator controls the robot's clearance from the two walls. A micro-gyrometer and a tiny magnetic compass ensure that the hovercraft travels forward in the corridor without yawing. The lateral OFs are measured by two minimalist eyes mounted sideways opposite to each other. For the first time, the hovercraft was found to be capable of adjusting both its forward speed and its clearance from the walls, in both straight and tapered corridors, without requiring any distance or speed measurements, that is, without any need for on-board rangefinders or tachometers.
---
paper_title: A robotic aircraft that follows terrain using a neuromorphic eye

paper_content:

Future Unmanned Air Vehicles (UAV) and Micro Air Vehicles (MAV) will fly in urban areas and very close to obstacles. We have built a miniature (35 cm, 0.840 kg) electrically-powered aircraft which uses a motion-sensing visual system to follow terrain and avoid obstacles. Signals from the 20-photoreceptor onboard eye are processed by 19 custom Elementary Motion Detection (EMD) circuits which are derived from those of the fly. Visual, inertial, and rotor RPM signals from the aircraft are acquired by a flight computer which runs the real-time Linux operating system. Vision-guided trajectories and landings were simulated and automatic terrain-following flights at 2 m/s were demonstrated with the aircraft tethered to a whirling-arm. This UAV project is at the intersection of neurobiology, robotics, and aerospace. It provides technologies for MAV operations.
---
paper_title: Contrast saturation in a neuronally-based model of elementary motion detection

paper_content:

The Hassenstein-Reichardt (HR) correlation model is commonly used to model elementary motion detection in the fly. Recently, a neuronally-based computational model was proposed which, unlike the HR model, is based on identified neurons. The response of both models increases as the square of contrast, although the response of insect neurons saturates at high contrasts. We introduce a saturating nonlinearity into the neuronally-based model in order to produce contrast saturation and discuss the neuronal implications of these elements. Furthermore, we show that features of the contrast sensitivity of movement-detecting neurons are predicted by the modified model.
---
paper_title: Small Brains, Smart Machines: From Fly Vision to Robot Vision and Back Again

paper_content:

Neurobiological and neuroethological findings on insects can be used to design and construct small robots controlling their navigation on the basis of bio-inspired visual strategies and circuits. Animals' visual guidance is partly mediated by motion-sensitive neurons, which are responsible for gauging the optic flow. Although neurons of this kind were discovered in vertebrates' and invertebrates' visual systems more than 50 years ago, the principles and neural mechanisms involved have not yet been completely elucidated. Here, first, I propose to outline some of the findings we made during the last few decades by performing electrophysiological recordings on identified neurons in the housefly's eye while applying optical stimulation to identified photoreceptors. Whereas these findings shed light on the inner processing structure of an elementary motion detector (EMD), recent studies in which the latest genetic and neuroanatomical methods were applied to the fruitfly's visual system have identified some of the neurons in the visual chain which are possibly involved in the neural circuitry underlying a given EMD. Then, I will describe some of the proof-of-concept robots that we have developed on the basis of our biological findings. The 100-g robot OCTAVE, for example, is able to avoid the ground, react to wind, and land autonomously on a flat terrain without ever having to measure any state variables such as distances or speeds. The 100-g robots OSCAR 1 and OSCAR 2 inspired by the microscanner we discovered in the housefly's eye are able to stabilize their body using mainly visual means and track a moving edge with hyperacuity. These robots react to the optic flow, which is sensed by miniature optic flow sensors inspired by the housefly's EMDs. Constructing a “biorobot” gives us a unique opportunity of checking the soundness and robustness of a principle that is initially thought to be understood by bringing it face to face with the real physical world. Bio-inspired robotics not only help neurobiologists and neuroethologists to identify and investigate worthwhile problems in animals' sensory-motor systems, but they also provide engineers with ideas for developing novel devices and machines with promising future applications, in the field of smart autonomous vehicles and microvehicles, for example.
---
paper_title: Bio-inspired optical flow circuits for the visual guidance of micro air vehicles

paper_content:

In the framework of our research on biologically inspired microrobotics, we have developed a visually based autopilot for micro air vehicles (MAV), which we have called OCTAVE (optical altitude control system for autonomous vehicles). Here, we show the feasibility of a joint altitude and speed control system based on a low complexity optronic velocity sensor that estimates the optic flow in the downward direction. This velocity sensor draws on electrophysiological findings of on the fly elementary motion detectors (EMDs) obtained at our laboratory. We built an elementary, 100-gram tethered helicopter system that carries out terrain following above a randomly textured ground. The overall processing system is light enough to be mounted on-board MAVs with an avionic payload of only some grams.
---
paper_title: Movement-induced motion signal distributions in outdoor scenes

paper_content:

The movement of an observer generates a characteristic field of velocity vectors on the retina (Gibson 1950). Because such optic flow-fields are useful for navigation, many theoretical, psychophysical and physiological studies have addressed the question how egomotion parameters such as direction of heading can be estimated from optic flow. Little is known, however, about the structure of optic flow under natural conditions. To address this issue, we recorded sequences of panoramic images along accurately defined paths in a variety of outdoor locations and used these sequences as input to a two-dimensional array of correlation-based motion detectors (2DMD). We find that (a) motion signal distributions are sparse and noisy with respect to local motion directions; (b) motion signal distributions contain patches (motion streaks) which are systematically oriented along the principal flow-field directions; (c) motion signal distributions show a distinct, dorso-ventral topography, reflecting the distance anisotropy of terrestrial environments; (d) the spatiotemporal tuning of the local motion detector we used has little influence on the structure of motion signal distributions, at least for the range of conditions we tested; and (e) environmental motion is locally noisy throughout the visual field, with little spatial or temporal correlation; it can therefore be removed by temporal averaging and is largely over-ridden by image motion caused by observer movement. Our results suggest that spatial or temporal integration is important to retrieve reliable information on the local direction and size of motion vectors, because the structure of optic flow is clearly detectable in the temporal average of motion signal distributions. Egomotion parameters can be reliably retrieved from such averaged distributions under a range of environmental conditions. These observations raise a number of questions about the role of specific environmental and computational constraints in the processing of natural optic flow.
---
paper_title: Fast global motion estimation algorithm based on elementary motion detectors

paper_content:

This paper presents a fast global motion estimation algorithm based on so called elementary motion detectors or EMD. EMD, modeling insect visual signal processing systems, have low computational complexity aspects and thus can be key components to realize such a fast global motion estimation algorithm. The developed algorithm is evaluated by being applied to various types of image sequences and is found to provide accurate estimation results.
---
paper_title: Optic-Flow-Based Collision Avoidance

paper_content:

Flying in and around caves, tunnels, and buildings demands more than one sensing modality. This article presented an optic-flow- based approach inspired by flying insects for avoiding lateral collisions. However, there were a few real-world scenarios in which optic flow sensing failed. This occurred when obstacles on approach were directly in front of the aircraft. Here, a simple sonar or infrared sensor can be used to trigger a quick transition into the hovering mode to avoid the otherwise fatal collision. Toward this end, we have demonstrated a fixed-wing prototype capable of manually transitioning from conventional cruise flight into the hovering mode. The prototype was then equipped with an IMU and a flight control system to automate the hovering process. The next step in this research is to automate the transition from cruise to hover flight.
---
paper_title: Pulse-Based Analog VLSI Velocity Sensors

paper_content:

We present two algorithms for estimating the velocity of a visual stimulus and their implementations with analog circuits using CMOS VLSI technology. Both are instances of so-called token methods, where velocity is computed by identifying particular features in the image at different locations; in our algorithms, these features are abrupt temporal changes in image irradiance. Our circuits integrate photoreceptors and associated electronics for computing motion onto a single chip and unambiguously extract bidirectional velocity for stimuli of high and intermediate contrasts over considerable irradiance and velocity ranges. At low contrasts, the output signal for a given velocity tends to decrease gracefully with contrast, while direction-selectivity is maintained. The individual motion-sensing cells are compact and highly suitable for use in dense 1-D or 2-D imaging arrays.
---
paper_title: Responses of blowfly motion-sensitive neurons to reconstructed optic flow along outdoor flight paths

paper_content:

The retinal image flow a blowfly experiences in its daily life on the wing is determined by both the structure of the environment and the animal’s own movements. To understand the design of visual processing mechanisms, there is thus a need to analyse the performance of neurons under natural operating conditions. To this end, we recorded flight paths of flies outdoors and reconstructed what they had seen, by moving a panoramic camera along exactly the same paths. The reconstructed image sequences were later replayed on a fast, panoramic flight simulator to identified, motion sensitive neurons of the so-called horizontal system (HS) in the lobula plate of the blowfly, which are assumed to extract self-motion parameters from optic flow. We show that under real life conditions HS-cells not only encode information about self-rotation, but are also sensitive to translational optic flow and, thus, indirectly signal information about the depth structure of the environment. These properties do not require an elaboration of the known model of these neurons, because the natural optic flow sequences generate—at least qualitatively—the same depth-related response properties when used as input to a computational HS-cell model and to real neurons.
---
paper_title: A universal strategy for visually guided landing

paper_content:

Landing is a challenging aspect of flight because, to land safely, speed must be decreased to a value close to zero at touchdown. The mechanisms by which animals achieve this remain unclear. When landing on horizontal surfaces, honey bees control their speed by holding constant the rate of front-to-back image motion (optic flow) generated by the surface as they reduce altitude. As inclination increases, however, this simple pattern of optic flow becomes increasingly complex. How do honey bees control speed when landing on surfaces that have different orientations? To answer this, we analyze the trajectories of honey bees landing on a vertical surface that produces various patterns of motion. We find that landing honey bees control their speed by holding the rate of expansion of the image constant. We then test and confirm this hypothesis rigorously by analyzing landings when the apparent rate of expansion generated by the surface is manipulated artificially. This strategy ensures that speed is reduced, gradually and automatically, as the surface is approached. We then develop a mathematical model of this strategy and show that it can effectively be used to guide smooth landings on surfaces of any orientation, including horizontal surfaces. This biological strategy for guiding landings does not require knowledge about either the distance to the surface or the speed at which it is approached. The simplicity and generality of this landing strategy suggests that it is likely to be exploited by other flying animals and makes it ideal for implementation in the guidance systems of flying robots.
---
paper_title: A neural network for pursuit tracking inspired by the fly visual system

paper_content:

Abstract This paper presents an artificial neural network that detects and tracks an object moving within its field of view. This novel network is inspired by processing functions observed in the fly visual system. The network detects changes in input light intensities, determines motion on both the local and the wide-field levels, and outputs displacement information necessary to control pursuit tracking. Software simulations demonstrate the current prototype successfully follows a moving target within specified radiance and motion constraints. The paper reviews these limiting constraints and suggests future network augmentations to remove them. Despite its current limitations, the existing prototype serves as a solid foundation for a future network that promises to provide machines with the improved abilities to do high-speed pursuit tracking, interception, and collision avoidance.
---
paper_title: Object tracking in motion-blind flies

paper_content:

In response to the movement of its visual world, Drosophila is capable of optomotor response in head and body turning, as well as a visual fixation response. This study shows that blocking the visual pathway activity responsible for optokinetic response in flies does not affect the visual fixation response, suggesting two distinct pathways for processing each set of information. By doing so, the authors also devised a neural and behavioral hierarchy in fly visual system where fixation behavior and the neurons mediating fixation response are upstream of optokinetic response as performed by lobula plate neurons.
---
paper_title: Low-speed optic-flow sensor onboard an unmanned helicopter flying outside over fields

paper_content:

The 6-pixel low-speed Visual Motion Sensor (VMS) inspired by insects' visual systems presented here performs local 1-D angular speed measurements ranging from 1.5°/s to 25°/s and weighs only 2.8 g. The entire optic flow processing system, including the spatial and temporal filtering stages, has been updated with respect to the original design. This new lightweight sensor was tested under free-flying outdoor conditions over various fields onboard a 80 kg unmanned helicopter called ReSSAC. The visual disturbances encountered included helicopter vibrations, uncontrolled illuminance, trees, roads, and houses. The optic flow measurements obtained were finely analyzed online and also offline, using the sensors of various kinds mounted onboard ReSSAC. The results show that the optic flow measured despite the complex disturbances encountered closely matched the approximate ground-truth optic flow.
---
paper_title: Biologically inspired visual odometer for navigation of a flying robot

paper_content:

Experimental research in biology has uncovered a number of different ways in which flying insects use cues derived from optical flow for navigational purposes, such as safe landing, obstacle avoidance and dead reckoning. In this study, we use a synthetic methodology to gain additional insights into the navigation behavior of bees. Specifically, we focus on the mechanisms of course stabilization behavior and visually mediated odometer by using a biological model of motion detector for the purpose of long-range goal-directed navigation in 3D environment. The performance tests of the proposed navigation method are conducted by using a blimp-type flying robot platform in uncontrolled indoor environments. The result shows that the proposed mechanism can be used for goal-directed navigation. Further analysis is also conducted in order to enhance the navigation performance of autonomous aerial vehicles. © 2003 Elsevier B.V. All rights reserved.
---
paper_title: Estimation of self-motion by optic flow processing in single visual interneurons

paper_content:

HUMANS, animals and some mobile robots use visual motion cues for object detection and navigation in structured surroundings1–4. Motion is commonly sensed by large arrays of small field movement detectors, each preferring motion in a particular direction5,6. Self-motion generates distinct 'optic flow fields' in the eyes that depend on the type and direction of the momentary locomotion (rotation, translation) 7. To investigate how the optic flow is processed at the neuronal level, we recorded intracellularly from identified interneurons in the third visual neuropile of the blowfly8. The distribution of local motion tuning over their huge receptive fields was mapped in detail. The global structure of the resulting 'motion response fields' is remarkably similar to optic flow fields. Thus, the organization of the receptive fields of the so-called VS neurons9,10 strongly suggests that each of these neurons specifically extracts the rotatory component of the optic flow around a particular horizontal axis. Other neurons are probably adapted to extract translatory flow components. This study shows how complex visual discrimination can be achieved by task-oriented preprocessing in single neurons.
---
paper_title: Adaptation and the temporal delay filter of fly motion detectors

paper_content:

Recent accounts attribute motion adaptation to a shortening of the delay filter in elementary motion detectors (EMDs). Using computer modelling and recordings from HS neurons in the drone-fly Eristalis tenax, we present evidence that challenges this theory. (i) Previous evidence for a change in the delay filter comes from ‘image step’ (or ‘velocity impulse’) experiments. We note a large discrepancy between the temporal frequency tuning predicted from these experiments and the observed tuning of motion sensitive cells. (ii) The results of image step experiments are highly sensitive to the experimental method used. (iii) An apparent motion stimulus reveals a much shorter EMD delay than suggested by previous ‘image step’ experiments. This short delay agrees with the observed temporal frequency sensitivity of the unadapted cell. (iv) A key prediction of a shortening delay filter is that the temporal frequency optimum of the cell should show a large shift to higher temporal frequencies after motion adaptation. We show little change in the temporal or spatial frequency (and hence velocity) optima following adaptation.
---
paper_title: Flying over uneven moving terrain based on optic-flow cues without any need for reference frames or accelerometers.

paper_content:

Two bio-inspired guidance principles involving no reference frame are presented here and were implemented in a rotorcraft, which was equipped with panoramic optic flow (OF) sensors but (as in flying insects) no accelerometer. To test these two guidance principles, we built a tethered tandem rotorcraft called BeeRotor (80 grams), which was tested flying along a high-roofed tunnel. The aerial robot adjusts its pitch and hence its speed, hugs the ground and lands safely without any need for an inertial reference frame. The rotorcraft's altitude and forward speed are adjusted via two OF regulators piloting the lift and the pitch angle on the basis of the common-mode and differential rotor speeds, respectively. The robot equipped with two wide-field OF sensors was tested in order to assess the performances of the following two systems of guidance involving no inertial reference frame: (i) a system with a fixed eye orientation based on the curved artificial compound eye (CurvACE) sensor, and (ii) an active system of reorientation based on a quasi-panoramic eye which constantly realigns its gaze, keeping it parallel to the nearest surface followed. Safe automatic terrain following and landing were obtained with CurvACE under dim light to daylight conditions and the active eye-reorientation system over rugged, changing terrain, without any need for an inertial reference frame.
---
paper_title: A Model for Detection of Angular Velocity of Image Motion Based on the Temporal Tuning of the Drosophila

paper_content:

We propose a new bio-plausible model based on the visual systems of Drosophila for estimating angular velocity of image motion in insects’ eyes. The model implements both preferred direction motion enhancement and non-preferred direction motion suppression which is discovered in Drosophila’s visual neural circuits recently to give a stronger directional selectivity. In addition, the angular velocity detecting model (AVDM) produces a response largely independent of the spatial frequency in grating experiments which enables insects to estimate the flight speed in cluttered environments. This also coincides with the behaviour experiments of honeybee flying through tunnels with stripes of different spatial frequencies.
---
paper_title: Outdoor performance of a motion-sensitive neuron in the blowfly

paper_content:

We studied an identified motion-sensitive neuron of the blowfly under outdoor conditions. The neuron was stimulated by oscillating the fly in a rural environment. We analysed whether the motion-induced neuronal activity is affected by brightness changes ranging between bright sunlight and dusk. In addition, the relationship between spike rate and ambient temperature was determined. The main results are: (1) The mean spike rate elicited by visual motion is largely independent of brightness changes over several orders of magnitude as they occur as a consequence of positional changes of the sun. Even during dusk the neuron responds strongly and directionally selective to motion. (2) The neuronal spike rate is not significantly affected by short-term brightness changes caused by clouds temporarily occluding the sun. (3) In contrast, the neuronal activity is much affected by changes in ambient temperature.
---
paper_title: Optic flow-based collision-free strategies: From insects to robots

paper_content:

Flying insects are able to fly smartly in an unpredictable environment. It has been found that flying insects have smart neurons inside their tiny brains that are sensitive to visual motion also called optic flow. Consequently, flying insects rely mainly on visual motion during their flight maneuvers such as: takeoff or landing, terrain following, tunnel crossing, lateral and frontal obstacle avoidance, and adjusting flight speed in a cluttered environment. Optic flow can be defined as the vector field of the apparent motion of objects, surfaces, and edges in a visual scene generated by the relative motion between an observer (an eye or a camera) and the scene. Translational optic flow is particularly interesting for short-range navigation because it depends on the ratio between (i) the relative linear speed of the visual scene with respect to the observer and (ii) the distance of the observer from obstacles in the surrounding environment without any direct measurement of either speed or distance. In flying insects, roll stabilization reflex and yaw saccades attenuate any rotation at the eye level in roll and yaw respectively (i.e. to cancel any rotational optic flow) in order to ensure pure translational optic flow between two successive saccades. Our survey focuses on feedback-loops which use the translational optic flow that insects employ for collision-free navigation. Optic flow is likely, over the next decade to be one of the most important visual cues that can explain flying insects' behaviors for short-range navigation maneuvers in complex tunnels. Conversely, the biorobotic approach can therefore help to develop innovative flight control systems for flying robots with the aim of mimicking flying insects' abilities and better understanding their flight.
---
paper_title: Collision-avoidance and landing responses are mediated by separate pathways in the fruit fly, Drosophila melanogaster.

paper_content:

Flies rely heavily on visual feedback for several aspects of flight control. As a fly approaches an object, the image projected across its retina expands, providing the fly with visual feedback that can be used either to trigger a collision-avoidance maneuver or a landing response. To determine how a fly makes the decision to land on or avoid a looming object, we measured the behaviors generated in response to an expanding image during tethered flight in a visual closed-loop flight arena. During these experiments, each fly varied its wing-stroke kinematics to actively control the azimuth position of a 15°×15° square within its visual field. Periodically, the square symmetrically expanded in both the horizontal and vertical directions. We measured changes in the fly's wing-stroke amplitude and frequency in response to the expanding square while optically tracking the position of its legs to monitor stereotyped landing responses. Although this stimulus could elicit both the landing responses and collision-avoidance reactions, separate pathways appear to mediate the two behaviors. For example, if the square is in the lateral portion of the fly's field of view at the onset of expansion, the fly increases stroke amplitude in one wing while decreasing amplitude in the other, indicative of a collision-avoidance maneuver. In contrast, frontal expansion elicits an increase in wing-beat frequency and leg extension, indicative of a landing response. To further characterize the sensitivity of these responses to expansion rate, we tested a range of expansion velocities from 100 to 10000° s^(-1). Differences in the latency of both the collision-avoidance reactions and the landing responses with expansion rate supported the hypothesis that the two behaviors are mediated by separate pathways. To examine the effects of visual feedback on the magnitude and time course of the two behaviors, we presented the stimulus under open-loop conditions, such that the fly's response did not alter the position of the expanding square. From our results we suggest a model that takes into account the spatial sensitivities and temporal latencies of the collision-avoidance and landing responses, and is sufficient to schematically represent how the fly uses integration of motion information in deciding whether to turn or land when confronted with an expanding object.
---
paper_title: Optic flow regulation: the key to aircraft automatic guidance

paper_content:

Abstract We have developed a visually based autopilot which is able to make an air vehicle automatically take off, cruise and land, while reacting appropriately to wind disturbances (head wind and tail wind). This autopilot consists of a visual control system that adjusts the thrust so as to keep the downward optic flow (OF) at a constant value. This autopilot is therefore based on an optic flow regulation loop. It makes use of a sensor, which is known as an elementary motion detector (EMD). The functional structure of this EMD was inspired by that of the housefly, which was previously investigated at our Laboratory by performing electrophysiological recordings while applying optical microstimuli to single photoreceptor cells of the insect's compound eye. We built a proof-of-concept, tethered rotorcraft that circles indoors over an environment composed of contrasting features randomly arranged on the floor. The autopilot, which we have called OCTAVE (Optic flow based Control sysTem for Aerial VEhicles), enables this miniature (100 g) rotorcraft to carry out complex tasks such as ground avoidance and terrain following, to control risky maneuvers such as automatic take off and automatic landing, and to respond appropriately to wind disturbances. A single visuomotor control loop suffices to perform all these reputedly demanding tasks. As the electronic processing system required is extremely light-weight (only a few grams), it can be mounted on-board micro-air vehicles (MAVs) as well as larger unmanned air vehicles (UAVs) or even submarines and autonomous underwater vehicles (AUVs). But the OCTAVE autopilot could also provide guidance and/or warning signals to prevent the pilots of manned aircraft from colliding with shallow terrain, for example.
---
paper_title: A Model of Temporal Adaptation in Fly Motion Vision

paper_content:

A computational model is proposed to account for the adaptive properties of the fly motion system. The response properties of motion-sensitive neurons in the fly are modelled using an underdamped adaptive scheme to adjust the time constants of delay filters in an array of Reichardt detectors. It is shown that the increase in both temporal resolution and sensitivity to velocity change observed following adaptation to constant motion can be understood as a consequence of local adaptation of the filter time constants on the basis of the outputs of elementary motion detectors.
---
paper_title: Minimum viewing angle for visually guided ground speed control in bumblebees.

paper_content:

SUMMARY To control flight, flying insects extract information from the pattern of visual motion generated during flight, known as optic flow. To regulate their ground speed, insects such as honeybees and Drosophila hold the rate of optic flow in the axial direction (front-to-back) constant. A consequence of this strategy is that its performance varies with the minimum viewing angle (the deviation from the frontal direction of the longitudinal axis of the insect) at which changes in axial optic flow are detected. The greater this angle, the later changes in the rate of optic flow, caused by changes in the density of the environment, will be detected. The aim of the present study is to examine the mechanisms of ground speed control in bumblebees and to identify the extent of the visual range over which optic flow for ground speed control is measured. Bumblebees were trained to fly through an experimental tunnel consisting of parallel vertical walls. Flights were recorded when (1) the distance between the tunnel walls was either 15 or 30 cm, (2) the visual texture on the tunnel walls provided either strong or weak optic flow cues and (3) the distance between the walls changed abruptly halfway along the tunnel9s length. The results reveal that bumblebees regulate ground speed using optic flow cues and that changes in the rate of optic flow are detected at a minimum viewing angle of 23–30 deg., with a visual field that extends to approximately 155 deg. By measuring optic flow over a visual field that has a low minimum viewing angle, bumblebees are able to detect and respond to changes in the proximity of the environment well before they are encountered.
---
paper_title: Man-made velocity estimators based on insect vision

paper_content:

The study of insect vision is of significant interest to engineers for inspiring the design of future motion-sensitive smart sensor devices, for collision avoidance applications. Although insects are relatively simple organisms compared to vertebrates, they are blessed with a very efficient visual system, which enables them to navigate with great ease and accuracy. Biologically inspired motion detection models are bound to replace the conventional machine vision technology because of their simplicity and significant advantages in a number of applications. The dominant model for insect motion detection, first proposed by Hassentein and Reichardt in 1956, has gained widespread acceptance in the invertebrate vision community. The template model is another known model proposed later by Horridge in 1990, which permits simple tracking techniques and lends itself easily to both hardware and software. In this paper, we compare these two different motion detecting strategies. It was found from the data obtained from the intracellular recordings of the steady-state responses of wide-field neurons in the hoverfly Volucella, that the shape of the curves obtained agree with the theoretical predictions made by Dror. In order to compare this with the template model, we carried out an experiment to obtain the velocity response curves of the template model to the same image statistics. The results lead us to believe that the fly motion detector emulates a modified Reichardt correlator. In the second part of the paper, modifications are made to the Reichardt detector that improve its performance in velocity detection by reducing its dependance on contrast and image structure. Our recent neurobiological experiments suggest that adaptive mechanisms decrease the EMD (elementary motion detector) dependence on pattern contrast and improve reliability. So appropriate modelling of an adaptive feedback mechanism is carried out to normalize contrast of input signals in order to improve the reliability and robustness of velocity estimation.
---
paper_title: Encoding of Naturalistic Optic Flow by a Population of Blowfly Motion-Sensitive Neurons

paper_content:

In sensory systems information is encoded by the activity of populations of neurons. To analyze the coding properties of neuronal populations sensory stimuli have usually been used that were much simpler than those encountered in real life. It has been possible only recently to stimulate visual interneurons of the blowfly with naturalistic visual stimuli reconstructed from eye movements measured during free flight. Therefore we now investigate with naturalistic optic flow the coding properties of a small neuronal population of identified visual interneurons in the blowfly, the so-called VS and HS neurons. These neurons are motion sensitive and directionally selective and are assumed to extract information about the animal's self-motion from optic flow. We could show that neuronal responses of VS and HS neurons are mainly shaped by the characteristic dynamical properties of the fly's saccadic flight and gaze strategy. Individual neurons encode information about both the rotational and the translational components of the animal's self-motion. Thus the information carried by individual neurons is ambiguous. The ambiguities can be reduced by considering neuronal population activity. The joint responses of different subpopulations of VS and HS neurons can provide unambiguous information about the three rotational and the three translational components of the animal's self-motion and also, indirectly, about the three-dimensional layout of the environment.
---
paper_title: A neural model of how the brain computes heading from optic flow in realistic scenes

paper_content:

Abstract Visually-based navigation is a key competence during spatial cognition. Animals avoid obstacles and approach goals in novel cluttered environments using optic flow to compute heading with respect to the environment. Most navigation models try either explain data, or to demonstrate navigational competence in real-world environments without regard to behavioral and neural substrates. The current article develops a model that does both. The ViSTARS neural model describes interactions among neurons in the primate magnocellular pathway, including V1, MT + , and MST d . Model outputs are quantitatively similar to human heading data in response to complex natural scenes. The model estimates heading to within 1.5° in random dot or photo-realistically rendered scenes, and within 3° in video streams from driving in real-world environments. Simulated rotations of less than 1°/s do not affect heading estimates, but faster simulated rotation rates do, as in humans. The model is part of a larger navigational system that identifies and tracks objects while navigating in cluttered environments.
---
paper_title: Extraction of optical velocity by use of multi-input Reichardt detectors

paper_content:

We study the possibility of a metrical readout of velocity from an ensemble of Reichardt correlators. We show that with a suitable choice of spatial and temporal prefiltering of the correlator input it is possible to devise reliable (nonaliasing) velocity-tuned Reichardt detectors. However, because of the well-known covariance of spatial and velocity tuning of velocity detectors in biological motion vision, an ensemble consisting of these detectors has problems in extracting a pattern-invariant velocity. We find that pattern invariance of the motion estimate can be closely approximated with Reichardt correlators that sample the luminance pattern at more than the minimum number of two locations.
---
paper_title: Honeybees change their height to restore their optic flow

paper_content:

To further elucidate the mechanisms underlying insects' height and speed control, we trained outdoor honeybees to fly along a high-roofed tunnel, part of which was equipped with a moving floor. Honeybees followed the stationary part of the floor at a given height. On encountering the moving part of the floor, which moved in the same direction as their flight, honeybees descended and flew at a lower height, thus gradually restoring their ventral optic flow (OF) to a similar value to that they had percieved when flying over the stationary part of the floor. This was therefore achieved not by increasing their airspeed, but by lowering their height of flight. These results can be accounted for by a control system called an optic flow regulator, as proposed in previous studies. This visuo-motor control scheme explains how honeybees can navigate safely along tunnels on the sole basis of OF measurements, without any need to measure either their speed or the clearance from the surrounding walls.
---
paper_title: A biomimetic vision-based hovercraft accounts for bees’ complex behaviour in various corridors

paper_content:

Here we present the first systematic comparison between the visual guidance behaviour of a biomimetic robot and those of honeybees flying in similar environments. We built a miniature hovercraft which can travel safely along corridors with various configurations. For the first time, we implemented on a real physical robot the ?lateral optic flow regulation autopilot?, which we previously studied computer simulations. This autopilot inspired by the results of experiments on various species of hymenoptera consists of two intertwined feedback loops, the speed and lateral control loops, each of which has its own optic flow (OF) set-point. A heading-lock system makes the robot move straight ahead as fast as 69?cm s?1 with a clearance from one wall as small as 31?cm, giving an unusually high translational OF value (125? s?1). Our biomimetic robot was found to navigate safely along straight, tapered and bent corridors, and to react appropriately to perturbations such as the lack of texture on one wall, the presence of a tapering or non-stationary section of the corridor and even a sloping terrain equivalent to a wind disturbance. The front end of the visual system consists of only two local motion sensors (LMS), one on each side. This minimalistic visual system measuring the lateral OF suffices to control both the robot's forward speed and its clearance from the walls without ever measuring any speeds or distances. We added two additional LMSs oriented at +/?45? to improve the robot's performances in stiffly tapered corridors. The simple control system accounts for worker bees? ability to navigate safely in six challenging environments: straight corridors, single walls, tapered corridors, straight corridors with part of one wall moving or missing, as well as in the presence of wind.
---
paper_title: Time-of-Travel Methods for Measuring Optical Flow on Board a Micro Flying Robot

paper_content:

For use in autonomous micro air vehicles, visual sensors must not only be small, lightweight and insensitive to light variations; on-board autopilots also require fast and accurate optical flow measurements over a wide range of speeds. Using an auto-adaptive bio-inspired Michaelis–Menten Auto-adaptive Pixel (M 2 APix) analog silicon retina, in this article, we present comparative tests of two optical flow calculation algorithms operating under lighting conditions from 6 × 10 − 7 to 1 . 6 × 10 − 2 W·cm − 2 (i.e., from 0.2 to 12,000 lux for human vision). Contrast “time of travel” between two adjacent light-sensitive pixels was determined by thresholding and by cross-correlating the two pixels’ signals, with measurement frequency up to 5 kHz for the 10 local motion sensors of the M 2 APix sensor. While both algorithms adequately measured optical flow between 25 ∘ /s and 1000 ∘ /s, thresholding gave rise to a lower precision, especially due to a larger number of outliers at higher speeds. Compared to thresholding, cross-correlation also allowed for a higher rate of optical flow output (99 Hz and 1195 Hz, respectively) but required substantially more computational resources.
---
paper_title: Direct Observation of ON and OFF Pathways in the Drosophila Visual System

paper_content:

Summary Visual motion perception is critical to many animal behaviors, and flies have emerged as a powerful model system for exploring this fundamental neural computation. Although numerous studies have suggested that fly motion vision is governed by a simple neural circuit [1–3], the implementation of this circuit has remained mysterious for decades. Connectomics and neurogenetics have produced a surge in recent progress, and several studies have shown selectivity for light increments (ON) or decrements (OFF) in key elements associated with this circuit [4–7]. However, related studies have reached disparate conclusions about where this selectivity emerges and whether it plays a major role in motion vision [8–13]. To address these questions, we examined activity in the neuropil thought to be responsible for visual motion detection, the medulla, of Drosophila melanogaster in response to a range of visual stimuli using two-photon calcium imaging. We confirmed that the input neurons of the medulla, the LMCs, are not responsible for light-on and light-off selectivity. We then examined the pan-neural response of medulla neurons and found prominent selectivity for light-on and light-off in layers of the medulla associated with two anatomically derived pathways (L1/L2 associated) [14, 15]. We next examined the activity of prominent interneurons within each pathway (Mi1 and Tm1) and found that these neurons have corresponding selectivity for light-on or light-off. These results provide direct evidence that motion is computed in parallel light-on and light-off pathways, demonstrate that this selectivity emerges in neurons immediately downstream of the LMCs, and specify where crucial elements of motion computation occur.
---
paper_title: A Genetic Push to Understand Motion Detection

paper_content:

Two articles in this issue of Neuron (Eichner et al. and Clark et al.) attack the problem of explaining how neuronal hardware in Drosophila implements the Reichardt motion detector, one of the most famous computational models in neuroscience, which has proven intractable up to now.
---
paper_title: Complementary mechanisms create direction selectivity in the fly

paper_content:

The brain extracts information from signals delivered from the eyes and other sensory organs in order to direct behavior. Understanding how the interactions and wiring of a multitude of individual nerve cells process and transmit this critical information to the brain is a fundamental goal in the field of neuroscience. One question many neuroscientists have tried to understand is how nerve cells in an animal’s brain detect direction when an animal sees movement of some kind – so-called motion vision. The raw signal from the light receptors in the eye does not discriminate whether the light moves in one direction or the other. So, the nerve cells in the brain must somehow compute the direction of movement based on the information relayed by the eye. For more than half a century, major debates have revolved around two rival models that could explain how motion vision works. Both models could in principle lead to neurons that prefer images moving in one direction over images moving in the opposite direction – so-called direction selectivity. In both models, the information about the changing light levels hitting two light-sensitive cells at two points on the eye are compared across time. In one model, signals from images moving in a cell’s preferred direction become amplified. In the other model, signals moving in the unfavored direction become canceled out. However, neither model perfectly explains motion vision. Now, Haag, Arenz et al. show that both models are partially correct and that the two mechanisms work together to detect motion across the field of vision more accurately. In the experiments, both models were tested in tiny fruit flies by measuring the activity of the first nerve cells that respond to the direction of visual motion. While each mechanism alone only produces a fairly weak and error-prone signal of direction, together the two mechanisms produce a stronger and more precise directional signal. Further research is now needed to determine which individual neurons amplify or cancel the signals to achieve such a high degree of direction selectivity.
---
paper_title: Miniature curved artificial compound eyes.

paper_content:

In most animal species, vision is mediated by compound eyes, which offer lower resolution than vertebrate single-lens eyes, but significantly larger fields of view with negligible distortion and spherical aberration, as well as high temporal resolution in a tiny package. Compound eyes are ideally suited for fast panoramic motion perception. Engineering a miniature artificial compound eye is challenging because it requires accurate alignment of photoreceptive and optical components on a curved surface. Here, we describe a unique design method for biomimetic compound eyes featuring a panoramic, undistorted field of view in a very thin package. The design consists of three planar layers of separately produced arrays, namely, a microlens array, a neuromorphic photodetector array, and a flexible printed circuit board that are stacked, cut, and curved to produce a mechanically flexible imager. Following this method, we have prototyped and characterized an artificial compound eye bearing a hemispherical field of view with embedded and programmable low-power signal processing, high temporal resolution, and local adaptation to illumination. The prototyped artificial compound eye possesses several characteristics similar to the eye of the fruit fly Drosophila and other arthropod species. This design method opens up additional vistas for a broad range of applications in which wide field motion detection is at a premium, such as collision-free navigation of terrestrial and aerospace vehicles, and for the experimental testing of insect vision theories.
---
paper_title: Fly visual course control: behaviour, algorithms and circuits

paper_content:

Understanding how the brain controls behaviour is undisputedly one of the grand goals of neuroscience research, and the pursuit of this goal has a long tradition in insect neuroscience. However, appropriate techniques were lacking for a long time. Recent advances in genetic and recording techniques now allow the participation of identified neurons in the execution of specific behaviours to be interrogated. By focusing on fly visual course control, I highlight what has been learned about the neuronal circuit modules that control visual guidance in Drosophila melanogaster through the use of these techniques.
---
paper_title: Brain Connectivity: Revealing the Fly Visual Motion Circuit

paper_content:

Summary A new semi-automated method for high-throughput identification of visual neurons and their synaptic partners has been combined with optical recording of activity and behavioral analysis to give the first complete description of an elementary circuit for detecting visual motion.
---
paper_title: Mimicking fly motion tracking and fixation behaviors with a hybrid visual neural network

paper_content:

How do animals like insects perceive meaningful visual motion cues involving directional and locational information of moving objects in visual clutter accurately and efficiently? In this paper, with respect to latest biological research progress made in underlying motion detection circuitry in the fly's preliminary visual system, we conduct a novel hybrid visual neural network, combining the functionality of two bio-plausible, namely the motion and the position pathways, for mimicking motion tracking and fixation behaviors. This modeling study extends a former direction selective neurons model to the higher level of behavior. The motivated algorithms can be used to guide a system that extracts location information of moving objects in a scene regardless of background clutter, using entirely low-level visual processing. We tested it against translational movements in synthetic and real-world scenes. The results demonstrated the following contributions: (1) The proposed computational structure fulfills the characteristics of a putative signal tuning map of the fly's physiology. (2) It also satisfies a biological implication that visual fixation behaviors could be simply tuned via the position pathway; nevertheless, the motion-detecting pathway improves the tracking precision. (3) Contrary to segmentation and registration based computer vision techniques, its computational simplicity benefits the building of neuromorphic visual sensor for robots.
---
paper_title: A common evolutionary origin for the ON- and OFF-edge motion detection pathways of the Drosophila visual system

paper_content:

Synaptic circuits for identified behaviors in the Drosophila brain have typically been considered from either a developmental or functional perspective without reference to how the circuits might have been inherited from ancestral forms. For example, two candidate pathways for ON- and OFF-edge motion detection in the visual system act via circuits that use respectively either T4 or T5, two cell types of the fourth neuropil, or lobula plate, that exhibit narrow-field direction-selective responses and provide input to wide-field tangential neurons. T4 or T5 both have four subtypes that terminate one each in the four strata of the lobula plate. Representatives are reported in a wide range of Diptera, and both cell types exhibit various similarities in: 1) the morphology of their dendritic arbors; 2) their four morphological and functional subtypes; 3) their cholinergic profile in Drosophila; 4) their input from the pathways of L3 cells in the first neuropil, or lamina, and by one of a pair of lamina cells, L1 (to the T4 pathway) and L2 (to the T5 pathway); and 5) their innervation by a single, wide-field contralateral tangential neuron from the central brain. Progenitors of both also express the gene atonal early in their proliferation from the inner anlage of the developing optic lobe, being alone among many other cell type progeny to do so. Yet T4 receives input in the second neuropil, or medulla, and T5 in the third neuropil or lobula. Here we suggest that these two cell types were originally one, that their ancestral cell population duplicated and split to innervate separate medulla and lobula neuropils, and that a fiber crossing – the internal chiasma – arose between the two neuropils. The split most plausibly occurred, we suggest, with the formation of the lobula as a new neuropil that formed when it separated from its ancestral neuropil to leave the medulla, suggesting additionally that medulla input neurons to T4 and T5 may also have had a common origin.
---
paper_title: Small Brains, Smart Machines: From Fly Vision to Robot Vision and Back Again

paper_content:

Neurobiological and neuroethological findings on insects can be used to design and construct small robots controlling their navigation on the basis of bio-inspired visual strategies and circuits. Animals' visual guidance is partly mediated by motion-sensitive neurons, which are responsible for gauging the optic flow. Although neurons of this kind were discovered in vertebrates' and invertebrates' visual systems more than 50 years ago, the principles and neural mechanisms involved have not yet been completely elucidated. Here, first, I propose to outline some of the findings we made during the last few decades by performing electrophysiological recordings on identified neurons in the housefly's eye while applying optical stimulation to identified photoreceptors. Whereas these findings shed light on the inner processing structure of an elementary motion detector (EMD), recent studies in which the latest genetic and neuroanatomical methods were applied to the fruitfly's visual system have identified some of the neurons in the visual chain which are possibly involved in the neural circuitry underlying a given EMD. Then, I will describe some of the proof-of-concept robots that we have developed on the basis of our biological findings. The 100-g robot OCTAVE, for example, is able to avoid the ground, react to wind, and land autonomously on a flat terrain without ever having to measure any state variables such as distances or speeds. The 100-g robots OSCAR 1 and OSCAR 2 inspired by the microscanner we discovered in the housefly's eye are able to stabilize their body using mainly visual means and track a moving edge with hyperacuity. These robots react to the optic flow, which is sensed by miniature optic flow sensors inspired by the housefly's EMDs. Constructing a “biorobot” gives us a unique opportunity of checking the soundness and robustness of a principle that is initially thought to be understood by bringing it face to face with the real physical world. Bio-inspired robotics not only help neurobiologists and neuroethologists to identify and investigate worthwhile problems in animals' sensory-motor systems, but they also provide engineers with ideas for developing novel devices and machines with promising future applications, in the field of smart autonomous vehicles and microvehicles, for example.
---
paper_title: Candidate Neural Substrates for Off-Edge Motion Detection in Drosophila

paper_content:

BACKGROUND ::: In the fly's visual motion pathways, two cell types-T4 and T5-are the first known relay neurons to signal small-field direction-selective motion responses [1]. These cells then feed into large tangential cells that signal wide-field motion. Recent studies have identified two types of columnar neurons in the second neuropil, or medulla, that relay input to T4 from L1, the ON-channel neuron in the first neuropil, or lamina, thus providing a candidate substrate for the elementary motion detector (EMD) [2]. Interneurons relaying the OFF channel from L1's partner, L2, to T5 are so far not known, however. ::: ::: ::: RESULTS ::: Here we report that multiple types of transmedulla (Tm) neurons provide unexpectedly complex inputs to T5 at their terminals in the third neuropil, or lobula. From the L2 pathway, single-column input comes from Tm1 and Tm2 and multiple-column input from Tm4 cells. Additional input to T5 comes from Tm9, the medulla target of a third lamina interneuron, L3, providing a candidate substrate for L3's combinatorial action with L2 [3]. Most numerous, Tm2 and Tm9's input synapses are spatially segregated on T5's dendritic arbor, providing candidate anatomical substrates for the two arms of a T5 EMD circuit; Tm1 and Tm2 provide a second. Transcript profiling indicates that T5 expresses both nicotinic and muscarinic cholinoceptors, qualifying T5 to receive cholinergic inputs from Tm9 and Tm2, which both express choline acetyltransferase (ChAT). ::: ::: ::: CONCLUSIONS ::: We hypothesize that T5 computes small-field motion signals by integrating multiple cholinergic Tm inputs using nicotinic and muscarinic cholinoceptors.
---
paper_title: Columnar cells necessary for motion responses of wide-field visual interneurons in Drosophila

paper_content:

Wide-field motion-sensitive neurons in the lobula plate (lobula plate tangential cells, LPTCs) of the fly have been studied for decades. However, it has never been conclusively shown which cells constitute their major presynaptic elements. LPTCs are supposed to be rendered directionally selective by integrating excitatory as well as inhibitory input from many local motion detectors. Based on their stratification in the different layers of the lobula plate, the columnar cells T4 and T5 are likely candidates to provide some of this input. To study their role in motion detection, we performed whole-cell recordings from LPTCs in Drosophila with T4 and T5 cells blocked using two different genetically encoded tools. In these flies, motion responses were abolished, while flicker responses largely remained. We thus demonstrate that T4 and T5 cells indeed represent those columnar cells that provide directionally selective motion information to LPTCs. Contrary to previous assumptions, flicker responses seem to be largely mediated by a third, independent pathway. This work thus represents a further step towards elucidating the complete motion detection circuitry of the fly.
---
paper_title: A visual motion detection circuit suggested by Drosophila connectomics

paper_content:

Animal behaviour arises from computations in neuronal circuits, but our understanding of these computations has been frustrated by the lack of detailed synaptic connection maps, or connectomes. For example, despite intensive investigations over half a century, the neuronal implementation of local motion detection in the insect visual system remains elusive. Here we develop a semi-automated pipeline using electron microscopy to reconstruct a connectome, containing 379 neurons and 8,637 chemical synaptic contacts, within the Drosophila optic medulla. By matching reconstructed neurons to examples from light microscopy, we assigned neurons to cell types and assembled a connectome of the repeating module of the medulla. Within this module, we identified cell types constituting a motion detection circuit, and showed that the connections onto individual motion-sensitive neurons in this circuit were consistent with their direction selectivity. Our results identify cellular targets for future functional investigations, and demonstrate that connectomes can provide key insights into neuronal computations.
---
paper_title: Optic-Flow-Based Collision Avoidance

paper_content:

Flying in and around caves, tunnels, and buildings demands more than one sensing modality. This article presented an optic-flow- based approach inspired by flying insects for avoiding lateral collisions. However, there were a few real-world scenarios in which optic flow sensing failed. This occurred when obstacles on approach were directly in front of the aircraft. Here, a simple sonar or infrared sensor can be used to trigger a quick transition into the hovering mode to avoid the otherwise fatal collision. Toward this end, we have demonstrated a fixed-wing prototype capable of manually transitioning from conventional cruise flight into the hovering mode. The prototype was then equipped with an IMU and a flight control system to automate the hovering process. The next step in this research is to automate the transition from cruise to hover flight.
---
paper_title: Visual Motion: Cellular Implementation of a Hybrid Motion Detector

paper_content:

Visual motion detection in insects is mediated by three-input detectors that compare inputs of different spatiotemporal properties. A new modeling study shows that only a small subset of possible arrangements of the input elements provides high direction-selectivity.
---
paper_title: Object tracking in motion-blind flies

paper_content:

In response to the movement of its visual world, Drosophila is capable of optomotor response in head and body turning, as well as a visual fixation response. This study shows that blocking the visual pathway activity responsible for optokinetic response in flies does not affect the visual fixation response, suggesting two distinct pathways for processing each set of information. By doing so, the authors also devised a neural and behavioral hierarchy in fly visual system where fixation behavior and the neurons mediating fixation response are upstream of optokinetic response as performed by lobula plate neurons.
---
paper_title: Directionally Selective Motion Detection by Insect Neurons

paper_content:

Animals have several good reasons for detecting motion with their eyes. First, the motion of other animals — potential preys, mates, intruders or predators — provides essential information on which to base vital moves such as escape or chase. Secondly, information about self-motion is crucial, especially in the context of navigation, course stabilization, obstacle avoidance, and collision-free goal reaching. In fact, the wealth of information provided by passive, non-contact self-motion evaluation in visual systems has been likened to a kind of “visual kinaesthesis” (Gibson 1958). Even the 3D structure of the environment can be picked up by a moving observer (revs. Collett and Harkness 1982; Buchner 1984; Nakayama 1985; Hildreth and Koch 1987). Von Helmholtz (1867) was the first to clearly state the importance of this “motion parallax” in locomotion, and Exner (1891) proposed that arthropods make use of motion parallax as well as stereopsis to estimate distances (see also Horridge 1986).
---
paper_title: A Class of Visual Neurons with Wide-Field Properties Is Required for Local Motion Detection

paper_content:

Summary Visual motion cues are used by many animals to guide navigation across a wide range of environments. Long-standing theoretical models have made predictions about the computations that compare light signals across space and time to detect motion. Using connectomic and physiological approaches, candidate circuits that can implement various algorithmic steps have been proposed in the Drosophila visual system. These pathways connect photoreceptors, via interneurons in the lamina and the medulla, to direction-selective cells in the lobula and lobula plate. However, the functional architecture of these circuits remains incompletely understood. Here, we use a forward genetic approach to identify the medulla neuron Tm9 as critical for motion-evoked behavioral responses. Using in vivo calcium imaging combined with genetic silencing, we place Tm9 within motion-detecting circuitry. Tm9 receives functional inputs from the lamina neurons L3 and, unexpectedly, L1 and passes information onto the direction-selective T5 neuron. Whereas the morphology of Tm9 suggested that this cell would inform circuits about local points in space, we found that the Tm9 spatial receptive field is large. Thus, this circuit informs elementary motion detectors about a wide region of the visual scene. In addition, Tm9 exhibits sustained responses that provide a tonic signal about incoming light patterns. Silencing Tm9 dramatically reduces the response amplitude of T5 neurons under a broad range of different motion conditions. Thus, our data demonstrate that sustained and wide-field signals are essential for elementary motion processing.
---
paper_title: The Temporal Tuning of the Drosophila Motion Detectors Is Determined by the Dynamics of Their Input Elements

paper_content:

Summary Detecting the direction of motion contained in the visual scene is crucial for many behaviors. However, because single photoreceptors only signal local luminance changes, motion detection requires a comparison of signals from neighboring photoreceptors across time in downstream neuronal circuits. For signals to coincide on readout neurons that thus become motion and direction selective, different input lines need to be delayed with respect to each other. Classical models of motion detection rely on non-linear interactions between two inputs after different temporal filtering. However, recent studies have suggested the requirement for at least three, not only two, input signals. Here, we comprehensively characterize the spatiotemporal response properties of all columnar input elements to the elementary motion detectors in the fruit fly, T4 and T5 cells, via two-photon calcium imaging. Between these input neurons, we find large differences in temporal dynamics. Based on this, computer simulations show that only a small subset of possible arrangements of these input elements maps onto a recently proposed algorithmic three-input model in a way that generates a highly direction-selective motion detector, suggesting plausible network architectures. Moreover, modulating the motion detection system by octopamine-receptor activation, we find the temporal tuning of T4 and T5 cells to be shifted toward higher frequencies, and this shift can be fully explained by the concomitant speeding of the input elements.
---
paper_title: Optic flow-based collision-free strategies: From insects to robots

paper_content:

Flying insects are able to fly smartly in an unpredictable environment. It has been found that flying insects have smart neurons inside their tiny brains that are sensitive to visual motion also called optic flow. Consequently, flying insects rely mainly on visual motion during their flight maneuvers such as: takeoff or landing, terrain following, tunnel crossing, lateral and frontal obstacle avoidance, and adjusting flight speed in a cluttered environment. Optic flow can be defined as the vector field of the apparent motion of objects, surfaces, and edges in a visual scene generated by the relative motion between an observer (an eye or a camera) and the scene. Translational optic flow is particularly interesting for short-range navigation because it depends on the ratio between (i) the relative linear speed of the visual scene with respect to the observer and (ii) the distance of the observer from obstacles in the surrounding environment without any direct measurement of either speed or distance. In flying insects, roll stabilization reflex and yaw saccades attenuate any rotation at the eye level in roll and yaw respectively (i.e. to cancel any rotational optic flow) in order to ensure pure translational optic flow between two successive saccades. Our survey focuses on feedback-loops which use the translational optic flow that insects employ for collision-free navigation. Optic flow is likely, over the next decade to be one of the most important visual cues that can explain flying insects' behaviors for short-range navigation maneuvers in complex tunnels. Conversely, the biorobotic approach can therefore help to develop innovative flight control systems for flying robots with the aim of mimicking flying insects' abilities and better understanding their flight.
---
paper_title: Processing properties of ON and OFF pathways for Drosophila motion detection

paper_content:

Four medulla neurons implement two critical processing steps to incoming signals in Drosophila motion detection.
---
paper_title: The Emergence of Directional Selectivity in the Visual Motion Pathway of Drosophila

paper_content:

Summary The perception of visual motion is critical for animal navigation, and flies are a prominent model system for exploring this neural computation. In Drosophila , the T4 cells of the medulla are directionally selective and necessary for ON motion behavioral responses. To examine the emergence of directional selectivity, we developed genetic driver lines for the neuron types with the most synapses onto T4 cells. Using calcium imaging, we found that these neuron types are not directionally selective and that selectivity arises in the T4 dendrites. By silencing each input neuron type, we identified which neurons are necessary for T4 directional selectivity and ON motion behavioral responses. We then determined the sign of the connections between these neurons and T4 cells using neuronal photoactivation. Our results indicate a computational architecture for motion detection that is a hybrid of classic theoretical models.
---
paper_title: Dissection of the Peripheral Motion Channel in the Visual System of Drosophila melanogaster

paper_content:

In the eye, visual information is segregated into modalities such as color and motion, these being transferred to the central brain through separate channels. Here, we genetically dissect the achromatic motion channel in the fly Drosophila melanogaster at the level of the first relay station in the brain, the lamina, where it is split into four parallel pathways (L1-L3, amc/T1). The functional relevance of this divergence is little understood. We now show that the two most prominent pathways, L1 and L2, together are necessary and largely sufficient for motion-dependent behavior. At high pattern contrast, the two pathways are redundant. At intermediate contrast, they mediate motion stimuli of opposite polarity, L2 front-to-back, L1 back-to-front motion. At low contrast, L1 and L2 depend upon each other for motion processing. Of the two minor pathways, amc/T1 specifically enhances the L1 pathway at intermediate contrast. L3 appears not to contribute to motion but to orientation behavior.
---
paper_title: ON and OFF pathways in Drosophila motion vision

paper_content:

Ramon y Cajal, the founding father of neuroscience, observed similarities between the vertebrate retina and the insect eye, but that was based purely on anatomy. Using state-of-the-art genetics and electrophysiology in the fruitfly, these authors distinguish motion-sensitive neurons responding to abrupt increases in light from those specific to light decrements, thus bringing the similarity with vertebrate circuitry to the functional level.
---
paper_title: Figure Tracking by Flies Is Supported by Parallel Visual Streams

paper_content:

Summary Visual figures may be distinguished based on elementary motion or higher-order non-Fourier features, and flies track both [1]. The canonical elementary motion detector, a compact computation for Fourier motion direction and amplitude, can also encode higher-order signals provided elaborate preprocessing [2–4]. However, the way in which a fly tracks a moving figure containing both elementary and higher-order signals has not been investigated. Using a novel white noise approach, we demonstrate that (1) the composite response to an object containing both elementary motion (EM) and uncorrelated higher-order figure motion (FM) reflects the linear superposition of each component; (2) the EM-driven component is velocity-dependent, whereas the FM component is driven by retinal position; (3) retinotopic variation in EM and FM responses are different from one another; (4) the FM subsystem superimposes saccadic turns upon smooth pursuit; and (5) the two systems in combination are necessary and sufficient to predict the full range of figure tracking behaviors, including those that generate no EM cues at all [1]. This analysis requires an extension of the model that fly motion vision is based on simple elementary motion detectors [5] and provides a novel method to characterize the subsystems responsible for the pursuit of visual figures.
---
paper_title: The First Steps in Drosophila Motion Detection

paper_content:

The visual system, with its ability to perceive motion, is crucial for most animals to walk or fly steadily. Theoretical models of motion detection exist, but the underlying cellular mechanisms are still poorly understood. In this issue of Neuron, Rister and colleagues dissect the function of neuronal subtypes in the optic lobe of Drosophila to reveal their role in motion detection.
---
paper_title: Direction Selectivity in Drosophila Emerges from Preferred-Direction Enhancement and Null-Direction Suppression

paper_content:

Across animal phyla, motion vision relies on neurons that respond preferentially to stimuli moving in one, preferred direction over the opposite, null direction. In the elementary motion detector of Drosophila , direction selectivity emerges in two neuron types, T4 and T5, but the computational algorithm underlying this selectivity remains unknown. We find that the receptive fields of both T4 and T5 exhibit spatiotemporally offset light-preferring and dark-preferring subfields, each obliquely oriented in spacetime. In a linear-nonlinear modeling framework, the spatiotemporal organization of the T5 receptive field predicts the activity of T5 in response to motion stimuli. These findings demonstrate that direction selectivity emerges from the enhancement of responses to motion in the preferred direction, as well as the suppression of responses to motion in the null direction. Thus, remarkably, T5 incorporates the essential algorithmic strategies used by the Hassenstein–Reichardt correlator and the Barlow–Levick detector. Our model for T5 also provides an algorithmic explanation for the selectivity of T5 for moving dark edges: our model captures all two- and three-point spacetime correlations relevant to motion in this stimulus class. More broadly, our findings reveal the contribution of input pathway visual processing, specifically center-surround, temporally biphasic receptive fields, to the generation of direction selectivity in T5. As the spatiotemporal receptive field of T5 in Drosophila is common to the simple cell in vertebrate visual cortex, our stimulus-response model of T5 will inform efforts in an experimentally tractable context to identify more detailed, mechanistic models of a prevalent computation. SIGNIFICANCE STATEMENT Feature selective neurons respond preferentially to astonishingly specific stimuli, providing the neurobiological basis for perception. Direction selectivity serves as a paradigmatic model of feature selectivity that has been examined in many species. While insect elementary motion detectors have served as premiere experimental models of direction selectivity for 60 years, the central question of their underlying algorithm remains unanswered. Using in vivo two-photon imaging of intracellular calcium signals, we measure the receptive fields of the first direction-selective cells in the Drosophila visual system, and define the algorithm used to compute the direction of motion. Computational modeling of these receptive fields predicts responses to motion and reveals how this circuit efficiently captures many useful correlations intrinsic to moving dark edges.
---
paper_title: Seeing Things in Motion: Models, Circuits, and Mechanisms

paper_content:

Motion vision provides essential cues for navigation and course control as well as for mate, prey, or predator detection. Consequently, neurons responding to visual motion in a direction-selective way are found in almost all species that see. However, directional information is not explicitly encoded at the level of a single photoreceptor. Rather, it has to be computed from the spatio-temporal excitation level of at least two photoreceptors. How this computation is done and how this computation is implemented in terms of neural circuitry and membrane biophysics have remained the focus of intense research over many decades. Here, we review recent progress made in this area with an emphasis on insects and the vertebrate retina.
---
paper_title: A directional tuning map of Drosophila elementary motion detectors

paper_content:

This study uses calcium imaging to show that T4 and T5 neurons are divided in specific subpopulations responding to motion in four cardinal directions, and are specific to ON versus OFF edges, respectively; when either T4 or T5 neurons were genetically blocked, tethered flies walking on air-suspended beads failed to respond to the corresponding visual stimuli.
---
paper_title: Common circuit design in fly and mammalian motion vision

paper_content:

Motion-sensitive neurons have long been studied in both the mammalian retina and the insect optic lobe, yet striking similarities have become obvious only recently. Detailed studies at the circuit level revealed that, in both systems, (i) motion information is extracted from primary visual information in parallel ON and OFF pathways; (ii) in each pathway, the process of elementary motion detection involves the correlation of signals with different temporal dynamics; and (iii) primary motion information from both pathways converges at the next synapse, resulting in four groups of ON-OFF neurons, selective for the four cardinal directions. Given that the last common ancestor of insects and mammals lived about 550 million years ago, this general strategy seems to be a robust solution for how to compute the direction of visual motion with neural hardware.
---
paper_title: A biomimetic vision-based hovercraft accounts for bees’ complex behaviour in various corridors

paper_content:

Here we present the first systematic comparison between the visual guidance behaviour of a biomimetic robot and those of honeybees flying in similar environments. We built a miniature hovercraft which can travel safely along corridors with various configurations. For the first time, we implemented on a real physical robot the ?lateral optic flow regulation autopilot?, which we previously studied computer simulations. This autopilot inspired by the results of experiments on various species of hymenoptera consists of two intertwined feedback loops, the speed and lateral control loops, each of which has its own optic flow (OF) set-point. A heading-lock system makes the robot move straight ahead as fast as 69?cm s?1 with a clearance from one wall as small as 31?cm, giving an unusually high translational OF value (125? s?1). Our biomimetic robot was found to navigate safely along straight, tapered and bent corridors, and to react appropriately to perturbations such as the lack of texture on one wall, the presence of a tapering or non-stationary section of the corridor and even a sloping terrain equivalent to a wind disturbance. The front end of the visual system consists of only two local motion sensors (LMS), one on each side. This minimalistic visual system measuring the lateral OF suffices to control both the robot's forward speed and its clearance from the walls without ever measuring any speeds or distances. We added two additional LMSs oriented at +/?45? to improve the robot's performances in stiffly tapered corridors. The simple control system accounts for worker bees? ability to navigate safely in six challenging environments: straight corridors, single walls, tapered corridors, straight corridors with part of one wall moving or missing, as well as in the presence of wind.
---
paper_title: Internal Structure of the Fly Elementary Motion Detector

paper_content:

Recent experiments have shown that motion detection in Drosophila starts with splitting the visual input into two parallel channels encoding brightness increments (ON) or decrements (OFF). This suggests the existence of either two (ON-ON, OFF-OFF) or four (for all pairwise interactions) separate motion detectors. To decide between these possibilities, we stimulated flies using sequences of ON and OFF brightness pulses while recording from motion-sensitive tangential cells. We found direction-selective responses to sequences of same sign (ON-ON, OFF-OFF), but not of opposite sign (ON-OFF, OFF-ON), refuting the existence of four separate detectors. Based on further measurements, we propose a model that reproduces a variety of additional experimental data sets, including ones that were previously interpreted as support for four separate detectors. Our experiments and the derived model mark an important step in guiding further dissection of the fly motion detection circuit.
---
paper_title: Functional Specialization of Parallel Motion Detection Circuits in the Fly

paper_content:

In the fly Drosophila melanogaster , photoreceptor input to motion vision is split into two parallel pathways as represented by first-order interneurons L1 and L2 ([Rister et al., 2007][1]; [Joesch et al., 2010][2]). However, how these pathways are functionally specialized remains controversial. One study ([Eichner et al., 2011][3]) proposed that the L1-pathway evaluates only sequences of brightness increments (ON-ON), while the L2-pathway processes exclusively brightness decrements (OFF-OFF). Another study ([Clark et al., 2011][4]) proposed that each of the two pathways evaluates both ON-ON and OFF-OFF sequences. To decide between these alternatives, we recorded from motion-sensitive neurons in flies in which the output from either L1 or L2 was genetically blocked. We found that blocking L1 abolishes ON-ON responses but leaves OFF-OFF responses intact. The opposite was true, when the output from L2 was blocked. We conclude that the L1 and L2 pathways are functionally specialized to detect ON-ON and OFF-OFF sequences, respectively. ::: ::: [1]: #ref-17 ::: [2]: #ref-11 ::: [3]: #ref-6 ::: [4]: #ref-4
---
paper_title: Defining the Computational Structure of the Motion Detector in Drosophila

paper_content:

Summary Many animals rely on visual motion detection for survival. Motion information is extracted from spatiotemporal intensity patterns on the retina, a paradigmatic neural computation. A phenomenological model, the Hassenstein-Reichardt correlator (HRC), relates visual inputs to neural activity and behavioral responses to motion, but the circuits that implement this computation remain unknown. By using cell-type specific genetic silencing, minimal motion stimuli, and in vivo calcium imaging, we examine two critical HRC inputs. These two pathways respond preferentially to light and dark moving edges. We demonstrate that these pathways perform overlapping but complementary subsets of the computations underlying the HRC. A numerical model implementing differential weighting of these operations displays the observed edge preferences. Intriguingly, these pathways are distinguished by their sensitivities to a stimulus correlation that corresponds to an illusory percept, "reverse phi," that affects many species. Thus, this computational architecture may be widely used to achieve edge selectivity in motion detection.
---
paper_title: Performance of a Visual Fixation Model in an Autonomous Micro Robot Inspired by Drosophila Physiology

paper_content:

In nature, lightweight and low-powered insects are ideal model systems to study motion perception strategies. Understanding the underlying characteristics and functionality of insects' visual systems is not only attractive to neural system modellers but also critical in providing effective solutions to future robotics. This paper presents a novel modelling of dynamic vision system inspired by Drosophila physiology for mimicking fast motion tracking and a closed-loop behavioural response to fixation. The proposed model was realised on the embedded system in an autonomous micro robot which has limited computational resources. A monocular camera was applied as the only motion sensing modality. Systematic experiments including open-loop and closed-loop bio-robotic tests validated the proposed visual fixation model: the robot showed motion tracking and fixation behaviours similarly to insects; the image processing frequency can maintain 25 ~ 45Hz. Arena tests also demonstrated a successful following behaviour aroused by fixation in navigation.
---
paper_title: Mimicking fly motion tracking and fixation behaviors with a hybrid visual neural network

paper_content:

How do animals like insects perceive meaningful visual motion cues involving directional and locational information of moving objects in visual clutter accurately and efficiently? In this paper, with respect to latest biological research progress made in underlying motion detection circuitry in the fly's preliminary visual system, we conduct a novel hybrid visual neural network, combining the functionality of two bio-plausible, namely the motion and the position pathways, for mimicking motion tracking and fixation behaviors. This modeling study extends a former direction selective neurons model to the higher level of behavior. The motivated algorithms can be used to guide a system that extracts location information of moving objects in a scene regardless of background clutter, using entirely low-level visual processing. We tested it against translational movements in synthetic and real-world scenes. The results demonstrated the following contributions: (1) The proposed computational structure fulfills the characteristics of a putative signal tuning map of the fly's physiology. (2) It also satisfies a biological implication that visual fixation behaviors could be simply tuned via the position pathway; nevertheless, the motion-detecting pathway improves the tracking precision. (3) Contrary to segmentation and registration based computer vision techniques, its computational simplicity benefits the building of neuromorphic visual sensor for robots.
---
paper_title: An improved LPTC neural model for background motion direction estimation

paper_content:

A class of specialized neurons, called lobula plate tangential cells (LPTCs) has been shown to respond strongly to wide-field motion. The classic model, elementary motion detector (EMD) and its improved model, two-quadrant detector (TQD) have been proposed to simulate LPTCs. Although EMD and TQD can percept background motion, their outputs are so cluttered that it is difficult to discriminate actual motion direction of the background. In this paper, we propose a max operation mechanism to model a newly-found transmedullary neuron Tm9 whose physiological properties do not map onto EMD and TQD. This proposed max operation mechanism is able to improve the detection performance of TQD in cluttered background by filtering out irrelevant motion signals. We will demonstrate the functionality of this proposed mechanism in wide-field motion perception.
---
paper_title: Modeling direction selective visual neural network with ON and OFF pathways for extracting motion cues from cluttered background

paper_content:

The nature endows animals robust vision systems for extracting and recognizing different motion cues, detecting predators, chasing preys/mates in dynamic and cluttered environments. Direction selective neurons (DSNs), with preference to certain orientation visual stimulus, have been found in both vertebrates and invertebrates for decades. In this paper, with respect to recent biological research progress in motion-detecting circuitry, we propose a novel way to model DSNs for recognizing movements on four cardinal directions. It is based on an architecture of ON and OFF visual pathways underlies a theory of splitting motion signals into parallel channels, encoding brightness increments and decrements separately. To enhance the edge selectivity and speed response to moving objects, we put forth a bio-plausible spatial-temporal network structure with multiple connections of same polarity ON/OFF cells. Each pair-wised combination is filtered with dynamic delay depending on sampling distance. The proposed vision system was challenged against image streams from both synthetic and cluttered real physical scenarios. The results demonstrated three major contributions: first, the neural network fulfilled the characteristics of a postulated physiological map of conveying visual information through different neuropile layers; second, the DSNs model can extract useful directional motion cues from cluttered background robustly and timely, which hits at potential of quick implementation in vision-based micro mobile robots; moreover, it also represents better speed response compared to a state-of-the-art elementary motion detector.
---
paper_title: Retinotopic Organization of Small-Field-Target-Detecting Neurons in the Insect Visual System

paper_content:

BACKGROUND ::: Despite having tiny brains and relatively low-resolution compound eyes, many fly species frequently engage in precisely controlled aerobatic pursuits of conspecifics. Recent investigations into high-order processing in the fly visual system have revealed a class of neurons, coined small-target-motion detectors (STMDs), capable of responding robustly to target motion against the motion of background clutter. Despite limited spatial acuity in the insect eye, these neurons display exquisite sensitivity to small targets. ::: ::: ::: RESULTS ::: We recorded intracellularly from morphologically identified columnar neurons in the lobula complex of the hoverfly Eristalis tenax. We show that these columnar neurons with exquisitely small receptive fields, like their large-field counterparts recently described from both male and female flies, have an extreme selectivity for the motion of small targets. In doing so, we provide the first physiological characterization of small-field neurons in female flies. These retinotopically organized columnar neurons include both direction-selective and nondirection-selective classes covering a large area of visual space. ::: ::: ::: CONCLUSIONS ::: The retinotopic arrangement of lobula columnar neurons sensitive to the motion of small targets makes a strong case for these neurons as important precursors in the local processing of target motion. Furthermore, the continued response of STMDs with such small receptive fields to the motion of small targets in the presence of moving background clutter places further constraints on the potential mechanisms underlying their small-target tuning.
---
paper_title: Neural mechanisms underlying target detection in a dragonfly centrifugal neuron

paper_content:

Visual identification of targets is an important task for many animals searching for prey or conspecifics. Dragonflies utilize specialized optics in the dorsal acute zone, accompanied by higher-order visual neurons in the lobula complex, and descending neural pathways tuned to the motion of small targets. While recent studies describe the physiology of insect small target motion detector (STMD) neurons, little is known about the mechanisms that underlie their exquisite sensitivity to target motion. Lobula plate tangential cells (LPTCs), a group of neurons in dipteran flies selective for wide-field motion, have been shown to take input from local motion detectors consistent with the classic correlation model developed by Hassenstein and Reichardt in the 1950s. We have tested the hypothesis that similar mechanisms underlie the response of dragonfly STMDs. We show that an anatomically characterized centrifugal STMD neuron (CSTMD1) gives responses that depend strongly on target contrast, a clear prediction of the correlation model. Target stimuli are more complex in spatiotemporal terms than the sinusoidal grating patterns used to study LPTCs, so we used a correlation-based computer model to predict response tuning to velocity and width of moving targets. We show that increasing target width in the direction of travel causes a shift in response tuning to higher velocities, consistent with our model. Finally, we show how the morphology of CSTMD1 allows for impressive spatial interactions when more than one target is present in the visual field.
---
paper_title: Insect Detection of Small Targets Moving in Visual Clutter

paper_content:

Detection of targets that move within visual clutter is a common task for animals searching for prey or conspecifics, a task made even more difficult when a moving pursuer needs to analyze targets against the motion of background texture (clutter). Despite the limited optical acuity of the compound eye of insects, this challenging task seems to have been solved by their tiny visual system. Here we describe neurons found in the male hoverfly, Eristalis tenax, that respond selectively to small moving targets. Although many of these target neurons are inhibited by the motion of a background pattern, others respond to target motion within the receptive field under a surprisingly large range of background motion stimuli. Some neurons respond whether or not there is a speed differential between target and background. Analysis of responses to very small targets (smaller than the size of the visual field of single photoreceptors) or those targets with reduced contrast shows that these neurons have extraordinarily high contrast sensitivity. Our data suggest that rejection of background motion may result from extreme selectivity for small targets contrasting against local patches of the background, combined with this high sensitivity, such that background patterns rarely contain features that satisfactorily drive the neuron.
---
paper_title: Object- and self-movement detectors in the ventral nerve cord of the dragonfly

paper_content:

1. ::: ::: Descending, movement-sensitive visual interneurons in the ventral nerve cord of the dragonfly,Anax junius, fall into two categories, based upon their responses to a variety of stimulus patterns. One group (object-movement detectors) is sensitive only to movement of small patterns; the other (self-movement detectors) responds maximally to movement of very large patterns or to rotation of the animal in the lighted laboratory. ::: ::: ::: ::: ::: 2. ::: ::: Object-movement-detector responses to repeated identical stimuli habituate very rapidly. The habituation is region specific; pattern movement elsewhere in the receptive field elicits a renewed response (Fig. 3). The habituation is very long lasting and is not subject to dishabituation by mechanical or visual stimulation. Self-movement detectors, in contrast, show little or no habituation (Fig. 3). ::: ::: ::: ::: ::: 3. ::: ::: Increasing the extent of the stimulus pattern in the direction of motion decreases responses of object-movement detectors slightly and greatly increases self-movement-detector responses (Fig. 4). ::: ::: ::: ::: ::: 4. ::: ::: Increasing the length of the advancing edges perpendicular to the line of motion dramatically reduces object-movement-detector responses (Fig. 5). Such increases enhance self-movement-detector responses only slightly, unless they result in the pattern occupying especially sensitive regions of the receptive field (Fig. 7). ::: ::: ::: ::: ::: 5. ::: ::: Over a wide velocity range, self-movement-detector responses are not dependent on pattern wavelength (Fig. 8). ::: ::: ::: ::: ::: 6. ::: ::: These results indicate that the parameter upon which the object/world discrimination is based is different for the two groups of interneurons. The critical parameter for the self-movement detectors is the extent of the pattern in the direction of motion, whereas for the object-movement detectors, the critical parameter is the extent of the pattern perpendicular to the direction of motion.
---
paper_title: Local and large-range inhibition in feature detection.

paper_content:

Lateral inhibition is perhaps the most ubiquitous of neuronal mechanisms, having been demonstrated in early stages of processing in many different sensory pathways of both mammals and invertebrates. Recent work challenges the long-standing view that assumes that similar mechanisms operate to tune neuronal responses to higher order properties. Scant evidence for lateral inhibition exists beyond the level of the most peripheral stages of visual processing, leading to suggestions that many features of the tuning of higher order visual neurons can be accounted for by the receptive field and other intrinsic coding properties of visual neurons. Using insect target neurons as a model, we present unequivocal evidence that feature tuning is shaped not by intrinsic properties but by potent spatial lateral inhibition operating well beyond the first stages of visual processing. In addition, we present evidence for a second form of higher-order spatial inhibition--a long-range interocular transfer of information that we argue serves a role in establishing interocular rivalry and thus potentially a neural substrate for directing attention to single targets in the presence of distracters. In so doing, we demonstrate not just one, but two levels of spatial inhibition acting beyond the level of peripheral processing.
---
paper_title: Visual control of flight behaviour in the hoverflySyritta pipiens L.

paper_content:

1. ::: ::: The visually guided flight behaviour of groups of male and femaleSyritta pipiens was filmed at 50 f.p.s. and analysed frame by frame. Sometimes the flies cruise around ignoring each other. At other times males but not females track other flies closely, during which the body axis points accurately towards the leading fly. ::: ::: ::: ::: ::: 2. ::: ::: The eyes of males but not females have a forward directed region of enlarged facets where the resolution is 2 to 3 times greater than elsewhere. The inter-ommatidial angle in this “fovea” is 0.6°. ::: ::: ::: ::: ::: 3. ::: ::: Targets outside the fovea are fixated by accurately directed, intermittent, open-loop body saccades. Fixation of moving targets within the fovea is maintained by “continuous” tracking in which the angular position of the target on the retina (Θe) is continuously translated into the angular velocity of the tracking fly (\(\dot \Phi _p \)) with a latency of roughly 20 ms (\(\dot \Phi _p = k \Theta _e \), wherek≏30 s−1). ::: ::: ::: ::: ::: 4. ::: ::: The tracking fly maintains a roughly constant distance (in the range 5–15 cm) from the target. If the distance between the two flies is more than some set value the fly moves forwards, if it is less the fly moves backwards. The forward or backward velocity (\(\dot F_p \)) increases with the difference (D-D0) between the actual and desired distance (\(\dot F_p = k^\prime (D - D_0 )\)), wherek′=10 to 20 s−1). It is argued that the fly computes distance by measuring the vertical substense of the target image on the retina. ::: ::: ::: ::: ::: 5. ::: ::: Angular tracking is sometimes, at the tracking fly's choice,supplemented by changes in sideways velocity. The fly predicts a suitable sideways velocity probably on the basis of a running averageΘ e , but not its instantaneous value. Alternatively, when the target is almost stationary, angular tracking may bereplaced by sideways tracking. In this case the sideways velocity (\(\dot S\)) is related toΘ e about 30 ms earlier (\(\dot S_p = k\prime \prime \Theta _e \), wherek″=2.5 cm · s−1 · deg−1), and the angular tracking system is inoperative. ::: ::: ::: ::: ::: 6. ::: ::: When the leading fly settles the tracking fly often moves rapidly sideways in an arc centred on the leading fly. During thesevoluntary sideways movements the male continues to point his head at the target. He does this not by correctingΘ e , which is usually zero, but by predicting the angular velocity needed to maintain fixation. This prediction requires knowledge of both the distance between the flies and the tracking fly's sideways velocity. It is shown that the fly tends to over-estimate distance by about 20%. ::: ::: ::: ::: ::: 7. ::: ::: When two males meet head on during tracking the pursuit may be cut short as a result of vigorous sideways oscillations of both flies. These side-to-side movements are synchronised so that the males move in opposite directions, and the oscillations usually grow in size until the males separate. The angular tracking system is active during “wobbling” and it is shown that to synchronise the two flies the sideways tracking system must also be operative. The combined action of both systems in the two flies leads to instability and so provides a simple way of automatically separating two males. ::: ::: ::: ::: ::: 8. ::: ::: Tracking is probably sexual in function and often culminates in a rapid dart towards the leading fly, after the latter has settled. During these “rapes” the male accelerates continuously at about 500 cm · s−2, turning just before it lands so that it is in the copulatory position. The male rapes flies of either sex indicating that successful copulation involves more trial and error than recognition. ::: ::: ::: ::: ::: 9. ::: ::: During cruising flight the angular velocity of the fly is zero except for brief saccadic turns. There is often a sideways component to flight which means that the body axis is not necessarily in the direction of flight. Changes in flight direction are made either by means of saccades or by adjusting the ratio of sideways to forward velocity (\(\dot S/\dot F\)). Changes in body axis are frequently made without any change in the direction of flight. On these occasions, when the fly makes an angular saccade, it simultaneously adjusts\(\dot S/\dot F\) by an appropriate amount. ::: ::: ::: ::: ::: 10. ::: ::: Flies change course when they approach flowers using the same variety of mechanisms: a series of saccades, adjustments to\(\dot S/\dot F\), or by a mixture of the two. ::: ::: ::: ::: ::: 11. ::: ::: The optomotor response, which tends to prevent rotation except during saccades, is active both during cruising and tracking flight.
---
paper_title: Identified target-selective visual interneurons descending from the dragonfly brain

paper_content:

1. ::: ::: Eight large interneurons descending in the dragonfly (Aeshna umbrosa, Anax junius) ventral nerve cord from the brain to the thoracic ganglia were identified anatomically with intracellular dye injection (Fig. 3). All eight were strictly visual and responded only to movements of small patterns, such as black squares, ‘targets’, moving on a white background. ::: ::: ::: ::: ::: 2. ::: ::: The target interneurons all projected from the protocerebrum at least as far as the metathoracic ganglion. Within the protocerebrum they arborized in the posterodorsal neuropil region, near the base of the circumesophageal connectives (Fig. 3). ::: ::: ::: ::: ::: 3. ::: ::: The receptive fields of six of the cells were large, including most of the forward hemisphere of vision. For five of these, spiking responses were often restricted to a much smaller region within the receptive field, with stimulation of other areas yielding only subthreshold responses (Figs. 4 and 5, Table 1). ::: ::: ::: ::: ::: 4. ::: ::: The pattern of selectivity for target size varied, with some neurons responding only to small targets, some showing consistent responses over a wide range of target sizes, and one preferring larger targets (Fig. 6, Table 1). ::: ::: ::: ::: ::: 5. ::: ::: Five of the interneurons were directionally selective. Movement in the antipreferred direction elicited hyperpolarizing responses in two of them. Movements of large patterns, such as a checkerboard pattern covering the forward hemisphere, elicited opposite directional responses, i.e., hyperpolarizations in the preferred target direction and subthreshold depolarizations in the antipreferred direction (Fig. 7). A large pattern moving in any direction inhibited the response to target movement (Fig. 8). ::: ::: ::: ::: ::: 6. ::: ::: These neurons mediate, in part, the visual control of flight orientation. I propose that they convey turning signals to the wing motor in response to objects moving relative to the animal.
---
paper_title: Neural specializations for small target detection in insects

paper_content:

Despite being equipped with low-resolution eyes and tiny brains, many insects show exquisite abilities to detect and pursue targets even in highly textured surrounds. Target tracking behavior is subserved by neurons that are sharply tuned to the motion of small high-contrast targets. These neurons respond robustly to target motion, even against self-generated optic flow. A recent model, supported by neurophysiology, generates target selectivity by being sharply tuned to the unique spatiotemporal profile associated with target motion. Target neurons are likely connected in a complex network where some provide more direct output to behavior, whereas others serve an inter-regulatory role. These interactions may regulate attention and aid in the robust detection of targets in clutter observed in behavior.
---
paper_title: Small object detection neurons in female hoverflies

paper_content:

While predators such as dragonflies are dependent on visual detection of moving prey, social interactions make conspecific detection equally important for many non-predatory insects. Specialized 'acute zones' associated with target detection have evolved in several insect groups and are a prominent male-specific feature in many dipteran flies. The physiology of target selective neurons associated with these specialized eye regions has previously been described only from male flies. We show here that female hoverflies (Eristalis tenax) have several classes of neurons within the third optic ganglion (lobula) capable of detecting moving objects smaller than 1 degrees . These neurons have frontal receptive fields covering a large part of the ipsilateral world and are tuned to a broad range of target speeds and sizes. This could make them suitable for detecting targets under a range of natural conditions such as required during predator avoidance or conspecific interactions.
---
paper_title: Modelling LGMD2 visual neuron system

paper_content:

Two Lobula Giant Movement Detectors (LGMDs) have been identified in the lobula region of the locust visual system: LGMD1 and LGMD2. LGMD1 had been successfully used in robot navigation to avoid impending collision. LGMD2 also responds to looming stimuli in depth, and shares most the same properties with LGMD1; however, LGMD2 has its specific collision selective responds when dealing with different visual stimulus. Therefore, in this paper, we propose a novel way to model LGMD2, in order to emulate its predicted bio-functions, moreover, to solve some defects of previous LGMD1 computational models. The mechanism of ON and OFF cells, as well as bio-inspired nonlinear functions, are introduced in our model, to achieve LGMD2's collision selectivity. Our model has been tested by a miniature mobile robot in real time. The results suggested this model has an ideal performance in both software and hardware for collision recognition.
---
paper_title: Collision selective LGMDs neuron models research benefits from a vision-based autonomous micro robot

paper_content:

The developments of robotics inform research across a broad range of disciplines. In this paper, we will study and compare two collision selective neuron models via a vision-based autonomous micro robot. In the locusts' visual brain, two Lobula Giant Movement Detectors (LGMDs), i.e. LGMD1 and LGMD2, have been identified as looming sensitive neurons responding to rapidly expanding objects, yet with different collision selectivity. Both neurons have been modeled and successfully applied in robotic vision system for perceiving potential collisions in an efficient and reliable manner. In this research, we conduct binocular neuronal models, for the first time combining the functionalities of LGMD1 and LGMD2 neurons, in the visual modality of a ground mobile robot. The results of systematic on-line experiments demonstrated three contributions of this research: (1) The arena tests involving multiple robots verified the effectiveness and robustness of a reactive motion control strategy via integrating a bilateral pair of LGMD1 and LGMD2 models for collision detection in dynamic scenarios. (2) We pinpointed the different collision selectivity between LGMD1 and LGMD2 neuron models, which fulfill corresponding biological research. (3) The utilized micro robot may also benefit researches on other embedded vision systems as well as swarm robotics.
---
paper_title: Bio-inspired collision detector with enhanced selectivity for ground robotic vision system

paper_content:

There are many ways of building collision-detecting systems. In this paper, we propose a novel collision selective visual neural network inspired by LGMD2 neurons in the juvenile locusts. Such collision-sensitive neuron matures early in the ﬁrst-aged or even hatching locusts, and is only selective to detect looming dark objects against bright background in depth, represents swooping predators, a situation which is similar to ground robots or vehicles. However, little has been done on modeling LGMD2, let alone its potential applications in robotics and other vision-based areas. Compared to other collision detectors, our major contributions are ﬁrst, enhancing the collision selectivity in a bio-inspired way, via constructing a computing efﬁcient visual sensor, and realizing the revealed speciﬁc characteristic sofLGMD2. Second, we applied the neural network to help rearrange path navigation of an autonomous ground miniature robot in an arena. We also examined its neural properties through systematic experiments challenged against image streams from a visual sensor of the micro-robot.
---
paper_title: Review of state-of-the-art artificial compound eye imaging systems.

paper_content:

The natural compound eye has received much attention in recent years due to its remarkable properties, such as its large field of view (FOV), compact structure, and high sensitivity to moving objects. Many studies have been devoted to mimicking the imaging system of the natural compound eye. The paper gives a review of state-of-the-art artificial compound eye imaging systems. Firstly, we introduce the imaging principle of three types of natural compound eye. Then, we divide current artificial compound eye imaging systems into four categories according to the difference of structural composition. Readers can easily grasp methods to build an artificial compound eye imaging system from the perspective of structural composition. Moreover, we compare the imaging performance of state-of-the-art artificial compound eye imaging systems, which provides a reference for readers to design system parameters of an artificial compound eye imaging system. Next, we present the applications of the artificial compound eye imaging system including imaging with a large FOV, imaging with high resolution, object distance detection, medical imaging, egomotion estimation, and navigation. Finally, an outlook of the artificial compound eye imaging system is highlighted.
---
paper_title: Computation of object approach by a system of visual motion-sensitive neurons in the crab Neohelice

paper_content:

Similar to most visual animals, crabs perform proper avoidance responses to objects directly approaching them. The monostratified lobula giant neurons of type 1 (MLG1) of crabs constitute an ensemble of 14–16 bilateral pairs of motion-detecting neurons projecting from the lobula (third optic neuropile) to the midbrain, with receptive fields that are distributed over the extensive visual field of the animal's eye. Considering the crab Neohelice (previously Chasmagnathus) granulata, here we describe the response of these neurons to looming stimuli that simulate objects approaching the animal on a collision course. We found that the peak firing time of MLG1 acts as an angular threshold detector signaling, with a delay of δ = 35 ms, the time at which an object reaches a fixed angular threshold of 49°. Using in vivo intracellular recordings, we detected the existence of excitatory and inhibitory synaptic currents that shape the neural response. Other functional features identified in the MLG1 neurons were phas...
---
paper_title: Shaping the collision selectivity in a looming sensitive neuron model with parallel ON and OFF pathways and spike frequency adaptation

paper_content:

Abstract Shaping the collision selectivity in vision-based artificial collision-detecting systems is still an open challenge. This paper presents a novel neuron model of a locust looming detector, i.e. the lobula giant movement detector (LGMD1), in order to provide effective solutions to enhance the collision selectivity of looming objects over other visual challenges. We propose an approach to model the biologically plausible mechanisms of ON and OFF pathways and a biophysical mechanism of spike frequency adaptation (SFA) in the proposed LGMD1 visual neural network. The ON and OFF pathways can separate both dark and light looming features for parallel spatiotemporal computations. This works effectively on perceiving a potential collision from dark or light objects that approach; such a bio-plausible structure can also separate LGMD1’s collision selectivity to its neighbouring looming detector — the LGMD2. The SFA mechanism can enhance the LGMD1’s collision selectivity to approaching objects rather than receding and translating stimuli, which is a significant improvement compared with similar LGMD1 neuron models. The proposed framework has been tested using off-line tests of synthetic and real-world stimuli, as well as on-line bio-robotic tests. The enhanced collision selectivity of the proposed model has been validated in systematic experiments. The computational simplicity and robustness of this work have also been verified by the bio-robotic tests, which demonstrates potential in building neuromorphic sensors for collision detection in both a fast and reliable manner.
---
paper_title: Two identified looming detectors in the locust: ubiquitous lateral connections among their inputs contribute to selective responses to looming objects

paper_content:

Two identified looming detectors in the locust: ubiquitous lateral connections among their inputs contribute to selective responses to looming objects
---
paper_title: A Genetic Push to Understand Motion Detection

paper_content:

Two articles in this issue of Neuron (Eichner et al. and Clark et al.) attack the problem of explaining how neuronal hardware in Drosophila implements the Reichardt motion detector, one of the most famous computational models in neuroscience, which has proven intractable up to now.
---
paper_title: Internal Structure of the Fly Elementary Motion Detector

paper_content:

Recent experiments have shown that motion detection in Drosophila starts with splitting the visual input into two parallel channels encoding brightness increments (ON) or decrements (OFF). This suggests the existence of either two (ON-ON, OFF-OFF) or four (for all pairwise interactions) separate motion detectors. To decide between these possibilities, we stimulated flies using sequences of ON and OFF brightness pulses while recording from motion-sensitive tangential cells. We found direction-selective responses to sequences of same sign (ON-ON, OFF-OFF), but not of opposite sign (ON-OFF, OFF-ON), refuting the existence of four separate detectors. Based on further measurements, we propose a model that reproduces a variety of additional experimental data sets, including ones that were previously interpreted as support for four separate detectors. Our experiments and the derived model mark an important step in guiding further dissection of the fly motion detection circuit.
---
paper_title: Bio-inspired small target motion detector with a new lateral inhibition mechanism

paper_content:

In nature, it is an important task for animals to detect small targets which move within cluttered background. In recent years, biologists have found that a class of neurons in the lobula complex, called STMDs (small target motion detectors) which have extreme selectivity for small targets moving within visual clutter. At the same time, some researchers assert that lateral inhibition plays an important role in discriminating the motion of the target from the motion of the background, even account for many features of the tuning of higher order visual neurons. Inspired by the finding that complete lateral inhibition can only be seen when the motion of the central region is identical to the motion of the peripheral region, we propose a new lateral inhibition mechanism combined with motion velocity and direction to improve the performance of ESTMD model (elementary small target motion detector). In this paper, we will elaborate on the biological plausibility and functionality of this new lateral inhibition mechanism in small target motion detection.
---
paper_title: Postsynaptic organisations of directional selective visual neural networks for collision detection

paper_content:

In this paper, we studied the postsynaptic organisations of directional selective visual neurons for collision detection. Directional selective neurons can extract different directional visual motion cues fast and reliably by allowing inhibition spreads to further layers in specific directions with one or several time steps delay. Whether these directional selective neurons can be easily organised for other specific visual tasks is not known. Taking collision detection as the primary visual task, we investigated the postsynaptic organisations of these directional selective neurons through evolutionary processes. The evolved postsynaptic organisations demonstrated robust properties in detecting imminent collisions in complex visual environments with many of which achieved 94% success rate after evolution suggesting active roles in collision detection directional selective neurons and its postsynaptic organisations can play.
---
paper_title: A Model for the Detection of Moving Targets in Visual Clutter Inspired by Insect Physiology

paper_content:

We present a computational model for target discrimination based on intracellular recordings from neurons in the fly visual system. Determining how insects detect and track small moving features, often against cluttered moving backgrounds, is an intriguing challenge, both from a physiological and a computational perspective. Previous research has characterized higher-order neurons within the fly brain, known as 'small target motion detectors' (STMD), that respond robustly to moving features, even when the velocity of the target is matched to the background (i.e. with no relative motion cues). We recorded from intermediate-order neurons in the fly visual system that are well suited as a component along the target detection pathway. This full-wave rectifying, transient cell (RTC) reveals independent adaptation to luminance changes of opposite signs (suggesting separate ON and OFF channels) and fast adaptive temporal mechanisms, similar to other cell types previously described. From this physiological data we have created a numerical model for target discrimination. This model includes nonlinear filtering based on the fly optics, the photoreceptors, the 1(st) order interneurons (Large Monopolar Cells), and the newly derived parameters for the RTC. We show that our RTC-based target detection model is well matched to properties described for the STMDs, such as contrast sensitivity, height tuning and velocity tuning. The model output shows that the spatiotemporal profile of small targets is sufficiently rare within natural scene imagery to allow our highly nonlinear 'matched filter' to successfully detect most targets from the background. Importantly, this model can explain this type of feature discrimination without the need for relative motion cues.
---
paper_title: Functional Specialization of Parallel Motion Detection Circuits in the Fly

paper_content:

In the fly Drosophila melanogaster , photoreceptor input to motion vision is split into two parallel pathways as represented by first-order interneurons L1 and L2 ([Rister et al., 2007][1]; [Joesch et al., 2010][2]). However, how these pathways are functionally specialized remains controversial. One study ([Eichner et al., 2011][3]) proposed that the L1-pathway evaluates only sequences of brightness increments (ON-ON), while the L2-pathway processes exclusively brightness decrements (OFF-OFF). Another study ([Clark et al., 2011][4]) proposed that each of the two pathways evaluates both ON-ON and OFF-OFF sequences. To decide between these alternatives, we recorded from motion-sensitive neurons in flies in which the output from either L1 or L2 was genetically blocked. We found that blocking L1 abolishes ON-ON responses but leaves OFF-OFF responses intact. The opposite was true, when the output from L2 was blocked. We conclude that the L1 and L2 pathways are functionally specialized to detect ON-ON and OFF-OFF sequences, respectively. ::: ::: [1]: #ref-17 ::: [2]: #ref-11 ::: [3]: #ref-6 ::: [4]: #ref-4
---
paper_title: Navigation in an autonomous flying robot by using a biologically inspired visual odometer

paper_content:

While mobile robots and walking insects can use proprioceptive information (specialized receptors in the insects' leg, or wheel encoders in robots) to estimate distance traveled, flying agents have to rely mainly on visual cues. Experiments with bees provide evidence that flying insects might be using optical flow induced by egomotion to estimate distance traveled. Recently some details of this odometer have been unraveled. In this study, we propose a biologically inspired model of the bee's visual odometer based on Elementary Motion Detectors (EMDs), and present results from goal-directed navigation experiments with an autonomous flying robot platform that we developed specifically for this purpose. The robot is equipped with a panoramic vision system, which is used to provide input to the EMDs of the left and right visual fields. The outputs of the EMDs are in later stage spatially integrated by wide field motion detectors, and their accumulated response is directly used for the odometer. In a set of initial experiments, the robot moves through a corridor on a fixed route, and the outputs of EMDs, the odometer, are recorded. The results show that the proposed model can be used to provide an estimate of the distance traveled, but the performance depends on the route the robot follows, something which is biologically plausible since natural insects tend to adopt a fixed route during foraging. Given these results, we assumed that the optomotor response plays an important role in the context of goal-directed navigation, and we conducted experiments with an autonomous freely flying robot. The experiments demonstrate that this computationally cheap mechanism can be successfully employed in natural indoor environments.© (2000) COPYRIGHT SPIE--The International Society for Optical Engineering. Downloading of the abstract is permitted for personal use only.
---
paper_title: Collision selective LGMDs neuron models research benefits from a vision-based autonomous micro robot

paper_content:

The developments of robotics inform research across a broad range of disciplines. In this paper, we will study and compare two collision selective neuron models via a vision-based autonomous micro robot. In the locusts' visual brain, two Lobula Giant Movement Detectors (LGMDs), i.e. LGMD1 and LGMD2, have been identified as looming sensitive neurons responding to rapidly expanding objects, yet with different collision selectivity. Both neurons have been modeled and successfully applied in robotic vision system for perceiving potential collisions in an efficient and reliable manner. In this research, we conduct binocular neuronal models, for the first time combining the functionalities of LGMD1 and LGMD2 neurons, in the visual modality of a ground mobile robot. The results of systematic on-line experiments demonstrated three contributions of this research: (1) The arena tests involving multiple robots verified the effectiveness and robustness of a reactive motion control strategy via integrating a bilateral pair of LGMD1 and LGMD2 models for collision detection in dynamic scenarios. (2) We pinpointed the different collision selectivity between LGMD1 and LGMD2 neuron models, which fulfill corresponding biological research. (3) The utilized micro robot may also benefit researches on other embedded vision systems as well as swarm robotics.
---
paper_title: Bio-inspired collision detector with enhanced selectivity for ground robotic vision system

paper_content:

There are many ways of building collision-detecting systems. In this paper, we propose a novel collision selective visual neural network inspired by LGMD2 neurons in the juvenile locusts. Such collision-sensitive neuron matures early in the ﬁrst-aged or even hatching locusts, and is only selective to detect looming dark objects against bright background in depth, represents swooping predators, a situation which is similar to ground robots or vehicles. However, little has been done on modeling LGMD2, let alone its potential applications in robotics and other vision-based areas. Compared to other collision detectors, our major contributions are ﬁrst, enhancing the collision selectivity in a bio-inspired way, via constructing a computing efﬁcient visual sensor, and realizing the revealed speciﬁc characteristic sofLGMD2. Second, we applied the neural network to help rearrange path navigation of an autonomous ground miniature robot in an arena. We also examined its neural properties through systematic experiments challenged against image streams from a visual sensor of the micro-robot.
---
paper_title: Mimicking fly motion tracking and fixation behaviors with a hybrid visual neural network

paper_content:

How do animals like insects perceive meaningful visual motion cues involving directional and locational information of moving objects in visual clutter accurately and efficiently? In this paper, with respect to latest biological research progress made in underlying motion detection circuitry in the fly's preliminary visual system, we conduct a novel hybrid visual neural network, combining the functionality of two bio-plausible, namely the motion and the position pathways, for mimicking motion tracking and fixation behaviors. This modeling study extends a former direction selective neurons model to the higher level of behavior. The motivated algorithms can be used to guide a system that extracts location information of moving objects in a scene regardless of background clutter, using entirely low-level visual processing. We tested it against translational movements in synthetic and real-world scenes. The results demonstrated the following contributions: (1) The proposed computational structure fulfills the characteristics of a putative signal tuning map of the fly's physiology. (2) It also satisfies a biological implication that visual fixation behaviors could be simply tuned via the position pathway; nevertheless, the motion-detecting pathway improves the tracking precision. (3) Contrary to segmentation and registration based computer vision techniques, its computational simplicity benefits the building of neuromorphic visual sensor for robots.
---
paper_title: Collision detection in complex dynamic scenes using an LGMD-based visual neural network with feature enhancement

paper_content:

The lobula giant movement detector (LGMD) is an identified neuron in the locust brain that responds most strongly to the images of an approaching object such as a predator. Its computational model can cope with unpredictable environments without using specific object recognition algorithms. In this paper, an LGMD-based neural network is proposed with a new feature enhancement mechanism to enhance the expanded edges of colliding objects via grouped excitation for collision detection with complex backgrounds. The isolated excitation caused by background detail will be filtered out by the new mechanism. Offline tests demonstrated the advantages of the presented LGMD-based neural network in complex backgrounds. Real time robotics experiments using the LGMD-based neural network as the only sensory system showed that the system worked reliably in a wide range of conditions; in particular, the robot was able to navigate in arenas with structured surrounds and complex backgrounds
---
paper_title: A Synthetic Vision System Using Directionally Selective Motion Detectors to Recognize Collision

paper_content:

Reliably recognizing objects approaching on a collision course is extremely important. A synthetic vision system is proposed to tackle the problem of collision recognition in dynamic environments. The system combines the outputs of four whole-field motion-detecting neurons, each receiving inputs from a network of neurons employing asymmetric lateral inhibition to suppress their responses to one direction of motion. An evolutionary algorithm is then used to adjust the weights between the four motion-detecting neurons to tune the system to detect collisions in two test environments. To do this, a population of agents, each representing a proposed synthetic visual system, either were shown images generated by a mobile Khepera robot navigating in a simplified laboratory environment or were shown images videoed outdoors from a moving vehicle. The agents had to cope with the local environment correctly in order to survive. After 400 generations, the best agent recognized imminent collisions reliably in the familiar environment where it had evolved. However, when the environment was swapped, only the agent evolved to cope in the robotic environment still signaled collision reliably. This study suggests that whole-field direction-selective neurons, with selectivity based on asymmetric lateral inhibition, can be organized into a synthetic vision system, which can then be adapted to play an important role in collision detection in complex dynamic scenes.
---
paper_title: Small Brains, Smart Machines: From Fly Vision to Robot Vision and Back Again

paper_content:

Neurobiological and neuroethological findings on insects can be used to design and construct small robots controlling their navigation on the basis of bio-inspired visual strategies and circuits. Animals' visual guidance is partly mediated by motion-sensitive neurons, which are responsible for gauging the optic flow. Although neurons of this kind were discovered in vertebrates' and invertebrates' visual systems more than 50 years ago, the principles and neural mechanisms involved have not yet been completely elucidated. Here, first, I propose to outline some of the findings we made during the last few decades by performing electrophysiological recordings on identified neurons in the housefly's eye while applying optical stimulation to identified photoreceptors. Whereas these findings shed light on the inner processing structure of an elementary motion detector (EMD), recent studies in which the latest genetic and neuroanatomical methods were applied to the fruitfly's visual system have identified some of the neurons in the visual chain which are possibly involved in the neural circuitry underlying a given EMD. Then, I will describe some of the proof-of-concept robots that we have developed on the basis of our biological findings. The 100-g robot OCTAVE, for example, is able to avoid the ground, react to wind, and land autonomously on a flat terrain without ever having to measure any state variables such as distances or speeds. The 100-g robots OSCAR 1 and OSCAR 2 inspired by the microscanner we discovered in the housefly's eye are able to stabilize their body using mainly visual means and track a moving edge with hyperacuity. These robots react to the optic flow, which is sensed by miniature optic flow sensors inspired by the housefly's EMDs. Constructing a “biorobot” gives us a unique opportunity of checking the soundness and robustness of a principle that is initially thought to be understood by bringing it face to face with the real physical world. Bio-inspired robotics not only help neurobiologists and neuroethologists to identify and investigate worthwhile problems in animals' sensory-motor systems, but they also provide engineers with ideas for developing novel devices and machines with promising future applications, in the field of smart autonomous vehicles and microvehicles, for example.
---
paper_title: Movement-induced motion signal distributions in outdoor scenes

paper_content:

The movement of an observer generates a characteristic field of velocity vectors on the retina (Gibson 1950). Because such optic flow-fields are useful for navigation, many theoretical, psychophysical and physiological studies have addressed the question how egomotion parameters such as direction of heading can be estimated from optic flow. Little is known, however, about the structure of optic flow under natural conditions. To address this issue, we recorded sequences of panoramic images along accurately defined paths in a variety of outdoor locations and used these sequences as input to a two-dimensional array of correlation-based motion detectors (2DMD). We find that (a) motion signal distributions are sparse and noisy with respect to local motion directions; (b) motion signal distributions contain patches (motion streaks) which are systematically oriented along the principal flow-field directions; (c) motion signal distributions show a distinct, dorso-ventral topography, reflecting the distance anisotropy of terrestrial environments; (d) the spatiotemporal tuning of the local motion detector we used has little influence on the structure of motion signal distributions, at least for the range of conditions we tested; and (e) environmental motion is locally noisy throughout the visual field, with little spatial or temporal correlation; it can therefore be removed by temporal averaging and is largely over-ridden by image motion caused by observer movement. Our results suggest that spatial or temporal integration is important to retrieve reliable information on the local direction and size of motion vectors, because the structure of optic flow is clearly detectable in the temporal average of motion signal distributions. Egomotion parameters can be reliably retrieved from such averaged distributions under a range of environmental conditions. These observations raise a number of questions about the role of specific environmental and computational constraints in the processing of natural optic flow.
---
paper_title: Neural network based on the input organization of an identified neuron signaling impending collision

paper_content:

1. We describe a four-layered neural network (Fig. 1), based on the input organization of a collision signaling neuron in the visual system of the locust, the lobula giant movement detector (LGMD)....
---
paper_title: Shaping the collision selectivity in a looming sensitive neuron model with parallel ON and OFF pathways and spike frequency adaptation

paper_content:

Abstract Shaping the collision selectivity in vision-based artificial collision-detecting systems is still an open challenge. This paper presents a novel neuron model of a locust looming detector, i.e. the lobula giant movement detector (LGMD1), in order to provide effective solutions to enhance the collision selectivity of looming objects over other visual challenges. We propose an approach to model the biologically plausible mechanisms of ON and OFF pathways and a biophysical mechanism of spike frequency adaptation (SFA) in the proposed LGMD1 visual neural network. The ON and OFF pathways can separate both dark and light looming features for parallel spatiotemporal computations. This works effectively on perceiving a potential collision from dark or light objects that approach; such a bio-plausible structure can also separate LGMD1’s collision selectivity to its neighbouring looming detector — the LGMD2. The SFA mechanism can enhance the LGMD1’s collision selectivity to approaching objects rather than receding and translating stimuli, which is a significant improvement compared with similar LGMD1 neuron models. The proposed framework has been tested using off-line tests of synthetic and real-world stimuli, as well as on-line bio-robotic tests. The enhanced collision selectivity of the proposed model has been validated in systematic experiments. The computational simplicity and robustness of this work have also been verified by the bio-robotic tests, which demonstrates potential in building neuromorphic sensors for collision detection in both a fast and reliable manner.
---
paper_title: An improved LPTC neural model for background motion direction estimation

paper_content:

A class of specialized neurons, called lobula plate tangential cells (LPTCs) has been shown to respond strongly to wide-field motion. The classic model, elementary motion detector (EMD) and its improved model, two-quadrant detector (TQD) have been proposed to simulate LPTCs. Although EMD and TQD can percept background motion, their outputs are so cluttered that it is difficult to discriminate actual motion direction of the background. In this paper, we propose a max operation mechanism to model a newly-found transmedullary neuron Tm9 whose physiological properties do not map onto EMD and TQD. This proposed max operation mechanism is able to improve the detection performance of TQD in cluttered background by filtering out irrelevant motion signals. We will demonstrate the functionality of this proposed mechanism in wide-field motion perception.
---
paper_title: Correlation between OFF and ON Channels Underlies Dark Target Selectivity in an Insect Visual System

paper_content:

In both vertebrates and invertebrates, evidence supports separation of luminance increments and decrements (ON and OFF channels) in early stages of visual processing (Hartline, 1938; Joesch et al., 2010); however, less is known about how these parallel pathways are recombined to encode form and motion. In Drosophila, genetic knockdown of inputs to putative ON and OFF pathways and direct recording from downstream neurons in the wide-field motion pathway reveal that local elementary motion detectors exist in pairs that separately correlate contrast polarity channels, ON with ON and OFF with OFF (Joesch et al., 2013). However, behavioral responses to reverse-phi motion of discrete features reveal additional correlations of the opposite signs (Clark et al., 2011). We here present intracellular recordings from feature detecting neurons in the dragonfly that provide direct physiological evidence for the correlation of OFF and ON pathways. These neurons show clear polarity selectivity for feature contrast, responding strongly to targets that are darker than the background and only weakly to dark contrasting edges. These dark target responses are much stronger than the linear combination of responses to ON and OFF edges. We compare these data with output from elementary motion detector-based models (Eichner et al., 2011; Clark et al., 2011), with and without stages of strong center-surround antagonism. Our data support an alternative elementary small target motion detector model, which derives dark target selectivity from the correlation of a delayed OFF with an un-delayed ON signal at each individual visual processing unit (Wiederman et al., 2008, 2009).
---
paper_title: A Model for Detection of Angular Velocity of Image Motion Based on the Temporal Tuning of the Drosophila

paper_content:

We propose a new bio-plausible model based on the visual systems of Drosophila for estimating angular velocity of image motion in insects’ eyes. The model implements both preferred direction motion enhancement and non-preferred direction motion suppression which is discovered in Drosophila’s visual neural circuits recently to give a stronger directional selectivity. In addition, the angular velocity detecting model (AVDM) produces a response largely independent of the spatial frequency in grating experiments which enables insects to estimate the flight speed in cluttered environments. This also coincides with the behaviour experiments of honeybee flying through tunnels with stripes of different spatial frequencies.
---
paper_title: Optic flow-based collision-free strategies: From insects to robots

paper_content:

Flying insects are able to fly smartly in an unpredictable environment. It has been found that flying insects have smart neurons inside their tiny brains that are sensitive to visual motion also called optic flow. Consequently, flying insects rely mainly on visual motion during their flight maneuvers such as: takeoff or landing, terrain following, tunnel crossing, lateral and frontal obstacle avoidance, and adjusting flight speed in a cluttered environment. Optic flow can be defined as the vector field of the apparent motion of objects, surfaces, and edges in a visual scene generated by the relative motion between an observer (an eye or a camera) and the scene. Translational optic flow is particularly interesting for short-range navigation because it depends on the ratio between (i) the relative linear speed of the visual scene with respect to the observer and (ii) the distance of the observer from obstacles in the surrounding environment without any direct measurement of either speed or distance. In flying insects, roll stabilization reflex and yaw saccades attenuate any rotation at the eye level in roll and yaw respectively (i.e. to cancel any rotational optic flow) in order to ensure pure translational optic flow between two successive saccades. Our survey focuses on feedback-loops which use the translational optic flow that insects employ for collision-free navigation. Optic flow is likely, over the next decade to be one of the most important visual cues that can explain flying insects' behaviors for short-range navigation maneuvers in complex tunnels. Conversely, the biorobotic approach can therefore help to develop innovative flight control systems for flying robots with the aim of mimicking flying insects' abilities and better understanding their flight.
---
paper_title: Optic flow regulation: the key to aircraft automatic guidance

paper_content:

Abstract We have developed a visually based autopilot which is able to make an air vehicle automatically take off, cruise and land, while reacting appropriately to wind disturbances (head wind and tail wind). This autopilot consists of a visual control system that adjusts the thrust so as to keep the downward optic flow (OF) at a constant value. This autopilot is therefore based on an optic flow regulation loop. It makes use of a sensor, which is known as an elementary motion detector (EMD). The functional structure of this EMD was inspired by that of the housefly, which was previously investigated at our Laboratory by performing electrophysiological recordings while applying optical microstimuli to single photoreceptor cells of the insect's compound eye. We built a proof-of-concept, tethered rotorcraft that circles indoors over an environment composed of contrasting features randomly arranged on the floor. The autopilot, which we have called OCTAVE (Optic flow based Control sysTem for Aerial VEhicles), enables this miniature (100 g) rotorcraft to carry out complex tasks such as ground avoidance and terrain following, to control risky maneuvers such as automatic take off and automatic landing, and to respond appropriately to wind disturbances. A single visuomotor control loop suffices to perform all these reputedly demanding tasks. As the electronic processing system required is extremely light-weight (only a few grams), it can be mounted on-board micro-air vehicles (MAVs) as well as larger unmanned air vehicles (UAVs) or even submarines and autonomous underwater vehicles (AUVs). But the OCTAVE autopilot could also provide guidance and/or warning signals to prevent the pilots of manned aircraft from colliding with shallow terrain, for example.
---
paper_title: Principles of visual motion detection

paper_content:

Motion information is required for the solution of many complex tasks of the visual system such as depth perception by motion parallax and figure/ground discrimination by relative motion. However, motion information is not explicitly encoded at the level of the retinal input. Instead, it has to be computed from the time-dependent brightness patterns of the retinal image as sensed by the two-dimensional array of photoreceptors. Different models have been proposed which describe the neural computations underlying motion detection in various ways. To what extent do biological motion detectors approximate any of these models? As will be argued here, there is increasing evidence from the different disciplines studying biological motion vision, that, throughout the animal kingdom ranging from invertebrates to vertebrates including man, the mechanisms underlying motion detection can be attributed to only a few, essentially equivalent computational principles. Motion detection may, therefore, be one of the first examples in computational neurosciences where common principles can be found not only at the cellular level (e.g. dendritic integration, spike propagation, synaptic transmission) but also at the level of computations performed by small neural networks.
---
paper_title: Defining the Computational Structure of the Motion Detector in Drosophila

paper_content:

Summary Many animals rely on visual motion detection for survival. Motion information is extracted from spatiotemporal intensity patterns on the retina, a paradigmatic neural computation. A phenomenological model, the Hassenstein-Reichardt correlator (HRC), relates visual inputs to neural activity and behavioral responses to motion, but the circuits that implement this computation remain unknown. By using cell-type specific genetic silencing, minimal motion stimuli, and in vivo calcium imaging, we examine two critical HRC inputs. These two pathways respond preferentially to light and dark moving edges. We demonstrate that these pathways perform overlapping but complementary subsets of the computations underlying the HRC. A numerical model implementing differential weighting of these operations displays the observed edge preferences. Intriguingly, these pathways are distinguished by their sensitivities to a stimulus correlation that corresponds to an illusory percept, "reverse phi," that affects many species. Thus, this computational architecture may be widely used to achieve edge selectivity in motion detection.
---
paper_title: A bio-inspired visual collision detection mechanism for cars: Optimisation of a model of a locust neuron to a novel environment

paper_content:

The lobula giant movement detector (LGMD) neuron of locusts has been shown to preferentially respond to objects approaching the eye of a locust on a direct collision course. Computer simulations of the neuron have been developed and have demonstrated the ability of mobile robots, interfaced with a simulated LGMD model, to avoid collisions. In this study, a model of the LGMD neuron is presented and the functional parameters of the model identified. Models with different parameters were presented with a range of automotive video sequences, including collisions with cars. The parameters were optimised to respond correctly to the video sequences using a range of genetic algorithms (GAs). The model evolved most rapidly using GAs with high clone rates into a form suitable for detecting collisions with cars and not producing false collision alerts to most non-collision scenes.
---
paper_title: A Feedback Neural Network for Small Target Motion Detection in Cluttered Backgrounds

paper_content:

Small target motion detection is critical for insects to search for and track mates or prey which always appear as small dim speckles in the visual field. A class of specific neurons, called small target motion detectors (STMDs), has been characterized by exquisite sensitivity for small target motion. Understanding and analyzing visual pathway of STMD neurons are beneficial to design artificial visual systems for small target motion detection. Feedback loops have been widely identified in visual neural circuits and play an important role in target detection. However, if there exists a feedback loop in the STMD visual pathway or if a feedback loop could significantly improve the detection performance of STMD neurons, is unclear. In this paper, we propose a feedback neural network for small target motion detection against naturally cluttered backgrounds. In order to form a feedback loop, model output is temporally delayed and relayed to previous neural layer as feedback signal. Extensive experiments showed that the significant improvement of the proposed feedback neural network over the existing STMD-based models for small target motion detection.
---
paper_title: Redundant Neural Vision Systems—Competing for Collision Recognition Roles

paper_content:

Ability to detect collisions is vital for future robots that interact with humans in complex visual environments. Lobula giant movement detectors (LGMD) and directional selective neurons (DSNs) are two types of identified neurons found in the visual pathways of insects such as locusts. Recent modeling studies showed that the LGMD or grouped DSNs could each be tuned for collision recognition. In both biological and artificial vision systems, however, which one should play the collision recognition role and the way the two types of specialized visual neurons could be functioning together are not clear. In this modeling study, we compared the competence of the LGMD and the DSNs, and also investigate the cooperation of the two neural vision systems for collision recognition via artificial evolution. We implemented three types of collision recognition neural subsystems - the LGMD, the DSNs and a hybrid system which combines the LGMD and the DSNs subsystems together, in each individual agent. A switch gene determines which of the three redundant neural subsystems plays the collision recognition role. We found that, in both robotics and driving environments, the LGMD was able to build up its ability for collision recognition quickly and robustly therefore reducing the chance of other types of neural networks to play the same role. The results suggest that the LGMD neural network could be the ideal model to be realized in hardware for collision recognition.
---
paper_title: Two identified looming detectors in the locust: ubiquitous lateral connections among their inputs contribute to selective responses to looming objects

paper_content:

Two identified looming detectors in the locust: ubiquitous lateral connections among their inputs contribute to selective responses to looming objects
---
paper_title: Modeling direction selective visual neural network with ON and OFF pathways for extracting motion cues from cluttered background

paper_content:

The nature endows animals robust vision systems for extracting and recognizing different motion cues, detecting predators, chasing preys/mates in dynamic and cluttered environments. Direction selective neurons (DSNs), with preference to certain orientation visual stimulus, have been found in both vertebrates and invertebrates for decades. In this paper, with respect to recent biological research progress in motion-detecting circuitry, we propose a novel way to model DSNs for recognizing movements on four cardinal directions. It is based on an architecture of ON and OFF visual pathways underlies a theory of splitting motion signals into parallel channels, encoding brightness increments and decrements separately. To enhance the edge selectivity and speed response to moving objects, we put forth a bio-plausible spatial-temporal network structure with multiple connections of same polarity ON/OFF cells. Each pair-wised combination is filtered with dynamic delay depending on sampling distance. The proposed vision system was challenged against image streams from both synthetic and cluttered real physical scenarios. The results demonstrated three major contributions: first, the neural network fulfilled the characteristics of a postulated physiological map of conveying visual information through different neuropile layers; second, the DSNs model can extract useful directional motion cues from cluttered background robustly and timely, which hits at potential of quick implementation in vision-based micro mobile robots; moreover, it also represents better speed response compared to a state-of-the-art elementary motion detector.
---
paper_title: Bio-plausible visual neural network for spatio-temporally spiral motion perception

paper_content:

Abstract Neurophysiological studies validate that the primate cerebral cortex includes spiral neurons whose visual properties respond preferentially to spiral motion patterns. However, the biological mechanism of which a vision system perceives spiral motion is unclear, while few computational models are reported to discuss the problem of spiral motion perception. In order to fill this gap, this work develops a spiral motion perception neural network in terms of the recent achievements in neurophysiology and simulates the visual response characteristics of spiral neurons. One such network, inspired by two stages of biological visual information processing includes two subnetworks- presynaptic and postsynaptic neural networks. The former comprises multiple lateral inhibition neural sub-networks for the capture of visual motion information, whereas the latter extracts different rotational and radial motion cues and synthesizes them to detect the process of the spiral motion of an object. Experimentally, the proposed neural network is sufficiently examined by different types of spiral motion patterns in non-interference or interference environments. Numerically comparative experiments show that it can effectively detect spiral motion patterns of the object and also does not respond to any non-spiral motion, which is consistent with the metaphor of spiral neurons’ performance perception.
---
paper_title: Performance of a Visual Fixation Model in an Autonomous Micro Robot Inspired by Drosophila Physiology

paper_content:

In nature, lightweight and low-powered insects are ideal model systems to study motion perception strategies. Understanding the underlying characteristics and functionality of insects' visual systems is not only attractive to neural system modellers but also critical in providing effective solutions to future robotics. This paper presents a novel modelling of dynamic vision system inspired by Drosophila physiology for mimicking fast motion tracking and a closed-loop behavioural response to fixation. The proposed model was realised on the embedded system in an autonomous micro robot which has limited computational resources. A monocular camera was applied as the only motion sensing modality. Systematic experiments including open-loop and closed-loop bio-robotic tests validated the proposed visual fixation model: the robot showed motion tracking and fixation behaviours similarly to insects; the image processing frequency can maintain 25 ~ 45Hz. Arena tests also demonstrated a successful following behaviour aroused by fixation in navigation.
---
paper_title: Postsynaptic organisations of directional selective visual neural networks for collision detection

paper_content:

In this paper, we studied the postsynaptic organisations of directional selective visual neurons for collision detection. Directional selective neurons can extract different directional visual motion cues fast and reliably by allowing inhibition spreads to further layers in specific directions with one or several time steps delay. Whether these directional selective neurons can be easily organised for other specific visual tasks is not known. Taking collision detection as the primary visual task, we investigated the postsynaptic organisations of these directional selective neurons through evolutionary processes. The evolved postsynaptic organisations demonstrated robust properties in detecting imminent collisions in complex visual environments with many of which achieved 94% success rate after evolution suggesting active roles in collision detection directional selective neurons and its postsynaptic organisations can play.
---
paper_title: Collision selective LGMDs neuron models research benefits from a vision-based autonomous micro robot

paper_content:

The developments of robotics inform research across a broad range of disciplines. In this paper, we will study and compare two collision selective neuron models via a vision-based autonomous micro robot. In the locusts' visual brain, two Lobula Giant Movement Detectors (LGMDs), i.e. LGMD1 and LGMD2, have been identified as looming sensitive neurons responding to rapidly expanding objects, yet with different collision selectivity. Both neurons have been modeled and successfully applied in robotic vision system for perceiving potential collisions in an efficient and reliable manner. In this research, we conduct binocular neuronal models, for the first time combining the functionalities of LGMD1 and LGMD2 neurons, in the visual modality of a ground mobile robot. The results of systematic on-line experiments demonstrated three contributions of this research: (1) The arena tests involving multiple robots verified the effectiveness and robustness of a reactive motion control strategy via integrating a bilateral pair of LGMD1 and LGMD2 models for collision detection in dynamic scenarios. (2) We pinpointed the different collision selectivity between LGMD1 and LGMD2 neuron models, which fulfill corresponding biological research. (3) The utilized micro robot may also benefit researches on other embedded vision systems as well as swarm robotics.
---
paper_title: Mimicking fly motion tracking and fixation behaviors with a hybrid visual neural network

paper_content:

How do animals like insects perceive meaningful visual motion cues involving directional and locational information of moving objects in visual clutter accurately and efficiently? In this paper, with respect to latest biological research progress made in underlying motion detection circuitry in the fly's preliminary visual system, we conduct a novel hybrid visual neural network, combining the functionality of two bio-plausible, namely the motion and the position pathways, for mimicking motion tracking and fixation behaviors. This modeling study extends a former direction selective neurons model to the higher level of behavior. The motivated algorithms can be used to guide a system that extracts location information of moving objects in a scene regardless of background clutter, using entirely low-level visual processing. We tested it against translational movements in synthetic and real-world scenes. The results demonstrated the following contributions: (1) The proposed computational structure fulfills the characteristics of a putative signal tuning map of the fly's physiology. (2) It also satisfies a biological implication that visual fixation behaviors could be simply tuned via the position pathway; nevertheless, the motion-detecting pathway improves the tracking precision. (3) Contrary to segmentation and registration based computer vision techniques, its computational simplicity benefits the building of neuromorphic visual sensor for robots.
---
paper_title: A Synthetic Vision System Using Directionally Selective Motion Detectors to Recognize Collision

paper_content:

Reliably recognizing objects approaching on a collision course is extremely important. A synthetic vision system is proposed to tackle the problem of collision recognition in dynamic environments. The system combines the outputs of four whole-field motion-detecting neurons, each receiving inputs from a network of neurons employing asymmetric lateral inhibition to suppress their responses to one direction of motion. An evolutionary algorithm is then used to adjust the weights between the four motion-detecting neurons to tune the system to detect collisions in two test environments. To do this, a population of agents, each representing a proposed synthetic visual system, either were shown images generated by a mobile Khepera robot navigating in a simplified laboratory environment or were shown images videoed outdoors from a moving vehicle. The agents had to cope with the local environment correctly in order to survive. After 400 generations, the best agent recognized imminent collisions reliably in the familiar environment where it had evolved. However, when the environment was swapped, only the agent evolved to cope in the robotic environment still signaled collision reliably. This study suggests that whole-field direction-selective neurons, with selectivity based on asymmetric lateral inhibition, can be organized into a synthetic vision system, which can then be adapted to play an important role in collision detection in complex dynamic scenes.
---
paper_title: A Rotational Motion Perception Neural Network Based on Asymmetric Spatiotemporal Visual Information Processing

paper_content:

All complex motion patterns can be decomposed into several elements, including translation, expansion/contraction, and rotational motion. In biological vision systems, scientists have found that specific types of visual neurons have specific preferences to each of the three motion elements. There are computational models on translation and expansion/contraction perceptions; however, little has been done in the past to create computational models for rotational motion perception. To fill this gap, we proposed a neural network that utilizes a specific spatiotemporal arrangement of asymmetric lateral inhibited direction selective neural networks (DSNNs) for rotational motion perception. The proposed neural network consists of two parts—presynaptic and postsynaptic parts. In the presynaptic part, there are a number of lateral inhibited DSNNs to extract directional visual cues. In the postsynaptic part, similar to the arrangement of the directional columns in the cerebral cortex, these direction selective neurons are arranged in a cyclic order to perceive rotational motion cues. In the postsynaptic network, the delayed excitation from each direction selective neuron is multiplied by the gathered excitation from this neuron and its unilateral counterparts depending on which rotation, clockwise (cw) or counter-cw (ccw), to perceive. Systematic experiments under various conditions and settings have been carried out and validated the robustness and reliability of the proposed neural network in detecting cw or ccw rotational motion. This research is a critical step further toward dynamic visual information processing.
---
paper_title: Visual motion pattern extraction and fusion for collision detection in complex dynamic scenes

paper_content:

Detecting colliding objects in complex dynamic scenes is a difficult task for conventional computer vision techniques. However, visual processing mechanisms in animals such as insects may provide very simple and effective solutions for detecting colliding objects in complex dynamic scenes. In this paper, we propose a robust collision detecting system, which consists of a lobula giant movement detector (LGMD) based neural network and a translating sensitive neural network (TSNN), to recognise objects on a direct collision course in complex dynamic scenes. The LGMD based neural network is specialized for recognizing looming objects that are on a direct collision course. The TSNN, which fuses the extracted visual motion cues from several whole field direction selective neural networks, is only sensitive to translating movements in the dynamic scenes. The looming cue and translating cue revealed by the two specialized visual motion detectors are fused in the present system via a decision making mechanism. In the system, the LGMD plays a key role in detecting imminent collision; the decision from TSNN becomes useful only when a collision alarm has been issued by the LGMD network. Using driving scenarios as an example, we showed that the bio-inspired system can reliably detect imminent colliding objects in complex driving scenes.
---
paper_title: LGMD and DSNs neural networks integration for collision predication

paper_content:

An ability to predict collisions is essential for current vehicles and autonomous robots. In this paper, an integrated collision predication system is proposed based on neural subsystems inspired from Lobula giant movement detector (LGMD) and directional selective neurons (DSNs) which focus on different part of the visual field separately. The two type of neurons found in the visual pathways of insects respond most strongly to moving objects with preferred motion patterns, i.e., the LGMD prefers looming stimuli and DSNs prefer specific lateral movements. We fuse the extracted information by each type of neurons to make final decision. By dividing the whole field of view into four regions for each subsystem to process, the proposed approaches can detect hazardous situations that had been difficult for single subsystem only. Our experiments show that the integrated system works in most of the hazardous scenarios.
---
paper_title: Redundant Neural Vision Systems—Competing for Collision Recognition Roles

paper_content:

Ability to detect collisions is vital for future robots that interact with humans in complex visual environments. Lobula giant movement detectors (LGMD) and directional selective neurons (DSNs) are two types of identified neurons found in the visual pathways of insects such as locusts. Recent modeling studies showed that the LGMD or grouped DSNs could each be tuned for collision recognition. In both biological and artificial vision systems, however, which one should play the collision recognition role and the way the two types of specialized visual neurons could be functioning together are not clear. In this modeling study, we compared the competence of the LGMD and the DSNs, and also investigate the cooperation of the two neural vision systems for collision recognition via artificial evolution. We implemented three types of collision recognition neural subsystems - the LGMD, the DSNs and a hybrid system which combines the LGMD and the DSNs subsystems together, in each individual agent. A switch gene determines which of the three redundant neural subsystems plays the collision recognition role. We found that, in both robotics and driving environments, the LGMD was able to build up its ability for collision recognition quickly and robustly therefore reducing the chance of other types of neural networks to play the same role. The results suggest that the LGMD neural network could be the ideal model to be realized in hardware for collision recognition.
---
paper_title: Time-of-Travel Methods for Measuring Optical Flow on Board a Micro Flying Robot

paper_content:

For use in autonomous micro air vehicles, visual sensors must not only be small, lightweight and insensitive to light variations; on-board autopilots also require fast and accurate optical flow measurements over a wide range of speeds. Using an auto-adaptive bio-inspired Michaelis–Menten Auto-adaptive Pixel (M 2 APix) analog silicon retina, in this article, we present comparative tests of two optical flow calculation algorithms operating under lighting conditions from 6 × 10 − 7 to 1 . 6 × 10 − 2 W·cm − 2 (i.e., from 0.2 to 12,000 lux for human vision). Contrast “time of travel” between two adjacent light-sensitive pixels was determined by thresholding and by cross-correlating the two pixels’ signals, with measurement frequency up to 5 kHz for the 10 local motion sensors of the M 2 APix sensor. While both algorithms adequately measured optical flow between 25 ∘ /s and 1000 ∘ /s, thresholding gave rise to a lower precision, especially due to a larger number of outliers at higher speeds. Compared to thresholding, cross-correlation also allowed for a higher rate of optical flow output (99 Hz and 1195 Hz, respectively) but required substantially more computational resources.
---
paper_title: Minimalistic optic flow sensors applied to indoor and outdoor visual guidance and odometry on a car-like robot

paper_content:

Here we present a novel bio-inspired optic flow (OF) sensor and its application to visual guidance and odometry on a low-cost car-like robot called BioCarBot. The minimalistic OF sensor was robust to high-dynamic-range lighting conditions and to various visual patterns encountered thanks to its M2APIX auto-adaptive pixels and the new cross-correlation OF algorithm implemented. The low-cost car-like robot estimated its velocity and steering angle, and therefore its position and orientation, via an extended Kalman filter (EKF) using only two downward-facing OF sensors and the Ackerman steering model. Indoor and outdoor experiments were carried out in which the robot was driven in the closed-loop mode based on the velocity and steering angle estimates. The experimental results obtained show that our novel OF sensor can deliver high-frequency measurements ([Formula: see text]) in a wide OF range (1.5-[Formula: see text]) and in a 7-decade high-dynamic light level range. The OF resolution was constant and could be adjusted as required (up to [Formula: see text]), and the OF precision obtained was relatively high (standard deviation of [Formula: see text] with an average OF of [Formula: see text], under the most demanding lighting conditions). An EKF-based algorithm gave the robot's position and orientation with a relatively high accuracy (maximum errors outdoors at a very low light level: [Formula: see text] and [Formula: see text] over about [Formula: see text] and [Formula: see text]) despite the low-resolution control systems of the steering servo and the DC motor, as well as a simplified model identification and calibration. Finally, the minimalistic OF-based odometry results were compared to those obtained using measurements based on an inertial measurement unit (IMU) and a motor's speed sensor.
---
paper_title: A QVGA 143 dB Dynamic Range Frame-Free PWM Image Sensor With Lossless Pixel-Level Video Compression and Time-Domain CDS

paper_content:

The biomimetic CMOS dynamic vision and image sensor described in this paper is based on a QVGA (304×240) array of fully autonomous pixels containing event-based change detection and pulse-width-modulation (PWM) imaging circuitry. Exposure measurements are initiated and carried out locally by the individual pixel that has detected a change of brightness in its field-of-view. Pixels do not rely on external timing signals and independently and asynchronously request access to an (asynchronous arbitrated) output channel when they have new grayscale values to communicate. Pixels that are not stimulated visually do not produce output. The visual information acquired from the scene, temporal contrast and grayscale data, are communicated in the form of asynchronous address-events (AER), with the grayscale values being encoded in inter-event intervals. The pixel-autonomous and massively parallel operation ideally results in lossless video compression through complete temporal redundancy suppression at the pixel level. Compression factors depend on scene activity and peak at ~1000 for static scenes. Due to the time-based encoding of the illumination information, very high dynamic range - intra-scene DR of 143 dB static and 125 dB at 30 fps equivalent temporal resolution - is achieved. A novel time-domain correlated double sampling (TCDS) method yields array FPN of 56 dB (9.3 bit) for >10 Lx illuminance.
---
paper_title: Biologically Inspired CMOS Image Sensor for Fast Motion and Polarization Detection

paper_content:

A complementary-metal-oxide semiconductor (CMOS) image sensor replicating the perception of vision in insects is presented for machine vision applications. The sensor is equipped with in-pixel analog and digital memories that allow in-pixel binarization in real time. The binary output of the pixel tries to replicate the flickering effect of an insect's eye to detect the smallest possible motion based on the change in state of each pixel. The pixel level optical flow generation reduces the need for digital hardware and simplifies the process of motion detection. A built-in counter counts the changes in states for each row to estimate the direction of the motion. The designed image sensor can also sense polarization information in real time using a metallic wire grid micropolarizer. An extinction ratio of 7.7 is achieved. The 1-D binary optical flow is shown to vary with the polarization angle of the incoming light ray. The image sensor consists of an array of 128 × 128 pixels, occupies an area of 5 × 4 mm2 and it is designed and fabricated in a 180-nm CMOS process.
---
paper_title: The VODKA Sensor: A Bio-Inspired Hyperacute Optical Position Sensing Device

paper_content:

We have designed and built a simple optical sensor called Vibrating Optical Device for the Kontrol of Autonomous robots (VODKA), that was inspired by the “tremor” eye movements observed in many vertebrate and invertebrate animals. In the initial version presented here, the sensor relies on the repetitive micro-translation of a pair of photoreceptors set behind a small lens, and on the processing designed to locate a target from the two photoreceptor signals. The VODKA sensor, in which retinal micro-scanning movements are performed via a small piezo-bender actuator driven at a frequency of 40 Hz, was found to be able to locate a contrasting edge with an outstandingly high resolution 900-fold greater than its static resolution (which is constrained by the interreceptor angle), regardless of the scanning law imposed on the retina. Hyperacuity is thus obtained at a very low cost, thus opening new vistas for the accurate visuo-motor control of robotic platforms. As an example, the sensor was mounted onto a miniature aerial robot that became able to track a moving target accurately by exploiting the robot's uncontrolled random vibrations as the source of its ocular microscanning movement. The simplicity, small size, low mass and low power consumption of this optical sensor make it highly suitable for many applications in the fields of metrology, astronomy, robotics, automotive, and aerospace engineering. The basic operating principle may also shed new light on the whys and wherefores of the tremor eye movements occurring in both animals and humans.
---
paper_title: Miniature curved artificial compound eyes.

paper_content:

In most animal species, vision is mediated by compound eyes, which offer lower resolution than vertebrate single-lens eyes, but significantly larger fields of view with negligible distortion and spherical aberration, as well as high temporal resolution in a tiny package. Compound eyes are ideally suited for fast panoramic motion perception. Engineering a miniature artificial compound eye is challenging because it requires accurate alignment of photoreceptive and optical components on a curved surface. Here, we describe a unique design method for biomimetic compound eyes featuring a panoramic, undistorted field of view in a very thin package. The design consists of three planar layers of separately produced arrays, namely, a microlens array, a neuromorphic photodetector array, and a flexible printed circuit board that are stacked, cut, and curved to produce a mechanically flexible imager. Following this method, we have prototyped and characterized an artificial compound eye bearing a hemispherical field of view with embedded and programmable low-power signal processing, high temporal resolution, and local adaptation to illumination. The prototyped artificial compound eye possesses several characteristics similar to the eye of the fruit fly Drosophila and other arthropod species. This design method opens up additional vistas for a broad range of applications in which wide field motion detection is at a premium, such as collision-free navigation of terrestrial and aerospace vehicles, and for the experimental testing of insect vision theories.
---
paper_title: Bio-inspired optic flow sensors based on FPGA: Application to Micro-Air-Vehicles

paper_content:

Tomorrow's Micro-Air-Vehicles (MAVs) could be used as scouts in many civil and military missions without any risk to human life. MAVs have to be equipped with sensors of several kinds for stabilization and guidance purposes. Many recent findings have shown, for example, that complex tasks such as 3-D navigation can be performed by insects using optic flow (OF) sensors although insects' eyes have a rather poor spatial resolution. At our Laboratory, we have been performing electrophysiological, micro-optical, neuroanatomical and behavioral studies for several decades on the housefly's visual system, with a view to understanding the neural principles underlying OF detection and establishing how OF sensors might contribute to performing basic navigational tasks. Based on these studies, we developed a functional model for an Elementary Motion Detector (EMD), which we first transcribed into electronic terms in 1986 and subsequently used onboard several terrestrial and aerial robots. Here we present a Field Programmable Gate Array (FPGA) implementation of an EMD array, which was designed for estimating the OF in various parts of the visual field of a MAV. FPGA technology is particularly suitable for applications of this kind, where a single Integrated Circuit (IC) can receive inputs from several photoreceptors of similar (or different) shapes and sizes located in various parts of the visual field. In addition, the remarkable characteristics of present-day FPGA applications (their high clock frequency, large number of system gates, embedded RAM blocks and Intellectual Property (IP) functions, small size, light weight, low cost, etc.) make for the flexible design of a multi-EMD visual system and its installation onboard MAVs with extremely low permissible avionic payloads.
---
paper_title: Energy-Efficient Design and Control of a Vibro-Driven Robot

paper_content:

Vibro-driven robotic (VDR) systems use stick-slip motions for locomotion. Due to the underactuated nature of the system, efficient design and control are still open problems. We present a new energy preserving design based on a spring-augmented pendulum. We indirectly control the friction-induced stick-slip motions by exploiting the passive dynamics in order to achieve an improvement in overall travelling distance and energy efficiency. Both collocated and non-collocated constraint conditions are elaborately analysed and considered to obtain a desired trajectory generation profile. For tracking control, we develop a partial feedback controller for the driving pendulum which counteracts the dynamic contributions from the platform. Comparative simulation studies show the effectiveness and intriguing performance of the proposed approach, while its feasibility is experimentally verified through a physical robot. Our robot is to the best of our knowledge the first nonlinear-motion prototype in literature towards the VDR systems.
---
paper_title: Motion Detection Circuits for a Time-To-Travel Algorithm

paper_content:

The paper describes a new motion detection circuit that extracts motion information based on a time-to-travel algorithm. The front-end photoreceptor adapts over 7 decades of background intensity and motion information can be extracted down to a contrast value of 2.5%. Results from the circuits which were fabricated in a 2-metal 2-poly 1.5mum CMOS process, show that the motion information can be extracted over 2 decades of speed.
---
paper_title: Bioinspired event-driven collision avoidance algorithm based on optic flow

paper_content:

Any mobile agent, whether biological or robotic, needs to avoid collisions with obstacles. Insects, such as bees and flies, use optic flow to estimate the relative nearness to obstacles. Optic flow induced by ego-motion is composed of a translational and a rotational component. The segregation of both components is computationally and thus energetically expensive. Flies and bees actively separate the rotational and translational optic flow components via behaviour, i.e. by employing a saccadic strategy of flight and gaze control. Although robotic systems are able to mimic this gaze-strategy, the calculation of optic-flow fields from standard camera images remains time and energy consuming. To overcome this problem, we use a dynamic vision sensor (DVS), which provides event-based information about changes in contrast over time at each pixel location. To extract optic flow from this information, a plane-fitting algorithm estimating the relative velocity in a small spatio-temporal cuboid is used. The depth-structure is derived from the translational optic flow by using local properties of the retina. A collision avoidance direction is then computed from the event-based depth-structure of the environment. The system has successfully been tested on a robotic platform in open loop.
---
paper_title: A modified model for the Lobula Giant Movement Detector and its FPGA implementation

paper_content:

Bio-inspired vision sensors are particularly appropriate candidates for navigation of vehicles or mobile robots due to their computational simplicity, allowing compact hardware implementations with low power dissipation. The Lobula Giant Movement Detector (LGMD) is a wide-field visual neuron located in the Lobula layer of the Locust nervous system. The LGMD increases its firing rate in response to both the velocity of an approaching object and the proximity of this object. It has been found that it can respond to looming stimuli very quickly and trigger avoidance reactions. It has been successfully applied in visual collision avoidance systems for vehicles and robots. This paper introduces a modified neural model for LGMD that provides additional depth direction information for the movement. The proposed model retains the simplicity of the previous model by adding only a few new cells. It has been simplified and implemented on a Field Programmable Gate Array (FPGA), taking advantage of the inherent parallelism exhibited by the LGMD, and tested on real-time video streams. Experimental results demonstrate the effectiveness as a fast motion detector.
---
paper_title: Bio-inspired optical flow circuits for the visual guidance of micro air vehicles

paper_content:

In the framework of our research on biologically inspired microrobotics, we have developed a visually based autopilot for micro air vehicles (MAV), which we have called OCTAVE (optical altitude control system for autonomous vehicles). Here, we show the feasibility of a joint altitude and speed control system based on a low complexity optronic velocity sensor that estimates the optic flow in the downward direction. This velocity sensor draws on electrophysiological findings of on the fly elementary motion detectors (EMDs) obtained at our laboratory. We built an elementary, 100-gram tethered helicopter system that carries out terrain following above a randomly textured ground. The overall processing system is light enough to be mounted on-board MAVs with an avionic payload of only some grams.
---
paper_title: Visually guided micro-aerial vehicle: automatic take off, terrain following, landing and wind reaction

paper_content:

We have developed a visually based autopilot which is able to make a micro air vehicle (MAV) automatically take off, cruise and land, while reacting adequately to wind disturbances. We built a proof-of-concept, tethered rotorcraft that can travel indoors over an environment composed of contrasting features randomly arranged on the floor. Here we show the feasibility of a visuomotor control loop that acts upon the thrust so as to maintain the optic flow (OF) estimated in the downward direction to a reference value. The sensor involved in this OF regulator is an elementary motion detector (EMD). The functional structure of the EMD was inspired by that of the housefly, which was previously investigated at our laboratory by performing electrophysiological recordings while applying optical microstimuli to single photoreceptor cells of the compound eye. The vision based autopilot, which we have called OCTAVE (optic flow control system for aerospace vehicles) solves complex problems such as terrain following, controls risky maneuvers such as take off and landing and responds appropriately to wind disturbances. All these reputedly demanding tasks are performed with one and the same visuomotor control loop. The non-emissive sensor and simple processing system are particularly suitable for use with MAV, since the tolerated avionic payload of these micro-aircraft is only a few grams. OCTAVE autopilot could also contribute to relieve a remote operator from the lowly and difficult task of continuously piloting and guiding an UAV. It could also provide guiding assistance to pilots of manned aircraft.
---
paper_title: Pulse-Based Analog VLSI Velocity Sensors

paper_content:

We present two algorithms for estimating the velocity of a visual stimulus and their implementations with analog circuits using CMOS VLSI technology. Both are instances of so-called token methods, where velocity is computed by identifying particular features in the image at different locations; in our algorithms, these features are abrupt temporal changes in image irradiance. Our circuits integrate photoreceptors and associated electronics for computing motion onto a single chip and unambiguously extract bidirectional velocity for stimuli of high and intermediate contrasts over considerable irradiance and velocity ranges. At low contrasts, the output signal for a given velocity tends to decrease gracefully with contrast, while direction-selectivity is maintained. The individual motion-sensing cells are compact and highly suitable for use in dense 1-D or 2-D imaging arrays.
---
paper_title: Characteristics of Three Miniature Bio-inspired Optic Flow Sensors in Natural Environments

paper_content:

Considerable attention has been paid during the last decade to vision-based navigation systems based on optic flow (OF) cues. OF-based systems have been implemented on an increasingly large number of sighted autonomous robotic platforms. Nowadays, the OF is measured using conventional cameras, custom-made sensors and even optical mouse chips. However, very few studies have dealt so far with the reliability of these OF sensors in terms of their precision, range and sensitivity to illuminance variations. Three miniature custom-made OF sensors developed at our laboratory, which were composed of photosensors connected to an OF processing unit were tested and compared in this study, focusing on their responses and characteristics in real indoor and outdoor environments in a large range of illuminance. It was concluded that by combining a custom-made aVLSI retina equipped with Adaptive Pixels for Insect-based Sensor (APIS) with a bio-inspired visual processing system, it is possible to obtain highly effective miniature sensors for measuring the OF under real environmental conditions.
---
paper_title: Obstacle avoidance with LGMD neuron: Towards a neuromorphic UAV implementation

paper_content:

We present a neuromorphic adaptation of a spiking neural network model of the locust Lobula Giant Movement Detector (LGMD), which detects objects increasing in size in the field of vision (looming) and can be used to facilitate obstacle avoidance in robotic applications. Our model is constrained by the parameters of a mixed signal analog-digital neuromorphic device, developed by our group, and is driven by the output of a neuromorphic vision sensor. We demonstrate the performance of the model and how it may be used for obstacle avoidance on an unmanned areal vehicle (UAV).
---
paper_title: Low-speed optic-flow sensor onboard an unmanned helicopter flying outside over fields

paper_content:

The 6-pixel low-speed Visual Motion Sensor (VMS) inspired by insects' visual systems presented here performs local 1-D angular speed measurements ranging from 1.5°/s to 25°/s and weighs only 2.8 g. The entire optic flow processing system, including the spatial and temporal filtering stages, has been updated with respect to the original design. This new lightweight sensor was tested under free-flying outdoor conditions over various fields onboard a 80 kg unmanned helicopter called ReSSAC. The visual disturbances encountered included helicopter vibrations, uncontrolled illuminance, trees, roads, and houses. The optic flow measurements obtained were finely analyzed online and also offline, using the sensors of various kinds mounted onboard ReSSAC. The results show that the optic flow measured despite the complex disturbances encountered closely matched the approximate ground-truth optic flow.
---
paper_title: New VLSI smart sensor for collision avoidance inspired by insect vision

paper_content:

An analog VLSI implementation of a smart microsensor that mimics the early visual processing stage in insects is described with an emphasis on the overall concept and the front- end detection. The system employs the `smart sensor' paradigm in that the detectors and processing circuitry are integrated on the one chip. The integrated circuit is composed of sixty channels of photodetectors and parallel processing elements. The photodetection circuitry includes p-well junction diodes on a 2 micrometers CMOS process and a logarithmic compression to increase the dynamic range of the system. The future possibility of gallium arsenide implementation is discussed. The processing elements behind each photodetector contain a low frequency differentiator where subthreshold design methods have been used. The completed IC is ideal for motion detection, particularly collision avoidance tasks, as it essentially detects distance, speed & bearing of an object. The Horridge Template Model for insect vision has been directly mapped into VLSI and therefore the IC truly exploits the beauty of nature in that the insect eye is so compact with parallel processing, enabling compact motion detection without the computational overhead of intensive imaging, full image extraction and interpretation. This world-first has exciting applications in the areas of automobile anti- collision, IVHS, autonomous robot guidance, aids for the blind, continuous process monitoring/web inspection and automated welding, for example.© (1995) COPYRIGHT SPIE--The International Society for Optical Engineering. Downloading of the abstract is permitted for personal use only.
---
paper_title: Flying over uneven moving terrain based on optic-flow cues without any need for reference frames or accelerometers.

paper_content:

Two bio-inspired guidance principles involving no reference frame are presented here and were implemented in a rotorcraft, which was equipped with panoramic optic flow (OF) sensors but (as in flying insects) no accelerometer. To test these two guidance principles, we built a tethered tandem rotorcraft called BeeRotor (80 grams), which was tested flying along a high-roofed tunnel. The aerial robot adjusts its pitch and hence its speed, hugs the ground and lands safely without any need for an inertial reference frame. The rotorcraft's altitude and forward speed are adjusted via two OF regulators piloting the lift and the pitch angle on the basis of the common-mode and differential rotor speeds, respectively. The robot equipped with two wide-field OF sensors was tested in order to assess the performances of the following two systems of guidance involving no inertial reference frame: (i) a system with a fixed eye orientation based on the curved artificial compound eye (CurvACE) sensor, and (ii) an active system of reorientation based on a quasi-panoramic eye which constantly realigns its gaze, keeping it parallel to the nearest surface followed. Safe automatic terrain following and landing were obtained with CurvACE under dim light to daylight conditions and the active eye-reorientation system over rugged, changing terrain, without any need for an inertial reference frame.
---
paper_title: A biologically inspired analog IC for visual collision detection

paper_content:

We have designed and tested a single-chip analog VLSI sensor that detects imminent collisions by measuring radially expanding optic flow. The design of the chip is based on a model proposed to explain leg-extension behavior in flies during landing approaches. We evaluated a detailed version of this model in simulation using a library of 50 test movies taken through a fisheye lens. The algorithm was evaluated on its ability to distinguish movies ending in collisions from movies in which no collision occurred. This biologically inspired algorithm is capable of 94% correct performance in this task using an ultra-low-resolution (132-pixel) image as input. A new elementary motion detector (EMD) circuit was developed to measure optic flow on a CMOS focal-plane sensor. This EMD circuit models the bandpass nature of large monopolar cells (LMCs) immediately postsynaptic to photoreceptors in the fly visual system as well as a saturating multiplication operation proposed for Reichart-type motion detectors. A 16/spl times/16 array of two-dimensional motion detectors was fabricated in a standard 0.5-/spl mu/m CMOS process. The chip consumes 140 /spl mu/W of power from a 5 V supply. With the addition of wide-angle optics, the sensor is able to detect collisions 100-400 ms before impact in complex, real-world scenes.
---
paper_title: An FPGA implementation of insect-inspired motion detector for high-speed vision systems

paper_content:

In this paper, an array of biologically inspired elementary motion detectors (EMDs) is implemented on an FPGA (field programmable gate array) platform. The well-known Reichardt-type EMD, modeling the insect's visual signal processing system, is very sensitive to motion direction and has low computational cost. A modified structure of EMD is used to detect local optical flow. Six templates of receptive fields, according to the fly's vision system, are designed for simple ego-motion estimation. The results of several typical experiments demonstrate local detection of optical flow and simple motion estimation under specific backgrounds. The performance of the real-time implementation is sufficient to deal with a video frame rate of 350 fps at 256 times 256 pixels resolution. The execution of the motion detection algorithm and the resulting time delay is only 0.25 mus. This hardware is suited for obstacle detection, motion estimation and UAV/MAV attitude control.
---
paper_title: Optimized adaptive tracking control for an underactuated vibro-driven capsule system

paper_content:

This paper studies the issue of adaptive trajectory tracking control for an underactuated vibro-driven capsule system and presents a novel motion generation framework. In this framework, feasible motion trajectory is derived through investigating dynamic constraints and kernel control indexes that underlie the underactuated dynamics. Due to the underactuated nature of the capsule system, the global motion dynamics cannot be directly controlled. The main objective of optimization is to indirectly control the friction-induced stick–slip motions to reshape the passive dynamics and, by doing so, to obtain optimal system performance in terms of average speed and energy efficacy. Two tracking control schemes are designed using a closed-loop feedback linearization approach and an adaptive variable structure control method with an auxiliary control variable, respectively. The reference model is accurately matched in a finite-time horizon. The key point is to define an exogenous state variable whose dynamics is employed as a control input. The tracking performance and system stability are investigated through rigorous theoretic analysis. Extensive simulation studies are conducted to demonstrate the effectiveness and feasibility of the developed trajectory model and optimized adaptive control system.
---

Title: Towards Computational Models and Applications of Insect Visual Systems for Motion Perception: A Review
Section 1: Introduction
Description 1: Introduce the significance of motion perception in insects and its relevance to computational modeling and applications in intelligent machines.

Section 2: Related Survey of Research on Biological Visual Systems
Description 2: Review past research on the cellular and subcellular mechanisms of insect visual systems, highlighting key discoveries and studies.

Section 3: Related Survey of Bio-inspired Models and Applications
Description 3: Provide an overview of bio-inspired computational models derived from insect visual systems and their applications in robotics, especially focusing on vision-based navigation and control.

Section 4: Taxonomy of This Review
Description 4: Outline the review's structure, categorizing computational models by their direction and size selectivity, and summarizing the significance of motion patterns in biological research and artificial systems.

Section 5: Neuron Models of Looming Perception
Description 5: Discuss computational models and applications of looming-sensitive neurons inspired by locust visual systems, detailing methods to shape looming selectivity and applications in robotics.

Section 6: Biological Research Background
Description 6: Provide a detailed background of the biological research on looming-sensitive neurons, focusing on LGMD1 and LGMD2 in locusts.

Section 7: Computational Models and Applications
Description 7: Review various computational models of looming-sensitive neurons and their applications in mobile robots, UAVs, and ground vehicles, highlighting key methodologies and implementations.

Section 8: Neural Systems for Translation Perception
Description 8: Review computational models and applications of translation-sensitive neural networks inspired by insect visual neurons and pathways, focusing on direction-selective neurons (DSN) and fly elementary motion detectors (EMDs).

Section 9: Computational Models of Locust DSNs
Description 9: Discuss the modeling of directionally selective neurons in locusts, detailing the computational structures and their applications in real-world environments.

Section 10: Fly Motion Detectors
Description 10: Review classic and modern theories on fly motion detectors, including Reichardt detectors and EMDs, and their applications to optic flow-based robotic navigation.

Section 11: EMD Models and OF-Based Applications to Robotics
Description 11: Discuss specific implementations of EMD models and their use in optic flow-based control strategies for various robotic applications.

Section 12: Modeling of Fly ON and OFF Pathways and LPTCs
Description 12: Present cutting-edge biological findings on fly visual systems, focusing on ON and OFF pathways and Lobula Plate Tangential Cells (LPTCs) and their role in motion detection.

Section 13: Small-Target Motion Perception Models
Description 13: Review computational models of small-target motion-sensitive neurons, focusing on the small-target motion detector (STMD) and figure detection neuron (FDN) in insect visual systems.

Section 14: Discussion
Description 14: Summarize similarities in computational modeling of insect motion detectors, discuss the generation of direction and size selectivity in models, and point out existing and potential hardware implementations.

Section 15: Integration of Multiple Neural Systems
Description 15: Discuss the benefits of integrating multiple neural systems to handle complex visual tasks in dynamic environments and enhance motion perception in artificial systems.

Section 16: Hardware Realization of Insect Motion Perception Models
Description 16: Explore the hardware implementation of computational models, focusing on single-chip and high-performance solutions, and discuss future trends and applications.

Section 17: Conclusion
Description 17: Summarize the review, emphasizing the potential of insect-inspired motion perception models for future neuromorphic sensors and intelligent machines.