Retinal prostheses for treating incurable blindness are designed to electrically stimulate surviving retinal neurons,  causing them to send artificial visual signals to the brain. However, electrical stimulation generally cannot precisely reproduce  normal patterns of neural activity in the retina. Therefore, an electrical stimulus must be selected that produces a neural response as close as possible to the desired response. This requires a technique for computing a distance between the desired response and the achievable response that is meaningful in terms of the visual signal being conveyed. Here we propose a method to learn such a metric on neural responses, directly from recorded light responses of a population of retinal ganglion cells (RGCs) in the primate retina. The learned metric produces a measure of similarity of RGC population responses that accurately reflects the similarity of the visual input. Using data from electrical stimulation experiments, we demonstrate that this metric may improve the performance of a prosthesis. An important application of neuroscience research is the development of electronic devices to replace the function of diseased or damaged neural circuits BID57 BID39 . Artificial vision has been a particularly challenging modality due to the richness of visual information, its diverse uses in perception and behavior, and the complexity of fabricating a device that can interface effectively with neural circuitry BID47 BID56 BID20 .The most advanced example is a retinal prosthesis: a device that replaces the function of neural circuitry in the retina lost to degenerative disease. Most of the computational work related to this application has focused on building encoding models that use the visual image to accurately predict the spiking activity of populations of retinal ganglion cells (RGCs), the output neurons of the retina that convey visual information to the brain. Leading models include linear models BID7 , probabilistic point-process models BID33 and recently proposed models employing rich nonlinearities BID30 BID2 BID41 .However, an accurate encoding model, although valuable, is insufficient. Any retinal prosthesiswhether based on electrical stimulation BID40 or optical stimulation BID6 BID3 -is limited in its ability to create arbitrary desired patterns of neural activity, due to inefficiencies or lack of specificity in the stimulation modality BID1 BID20 . Thus, a given stimulation system can only achieve a limited vocabulary of elicited spike patterns. Although a powerful and accurate encoding model might indicate that a particular spike pattern would be the natural biological response to the incident visual stimulus, the desired spike pattern might not reside within the feasible set of the stimulation device FIG0 ).Previous studies BID21 have addressed this problem by selecting the electrical stimulation which minimizes the number of unmatched spikes across cells -equivalent to the Hamming distance between two binary vectors. Even though a Hamming distance is easy to compute, this solution is not necessarily optimal. The goal of a prosthetics device should be to instead select an Real recorded spiking activity in a two populations of primate retinal ganglion cells (RGCs) demonstrating the lack of specificity from electrical stimulation. The electrodes are in blue and the stimulated electrode is shaded green. C. The target firing pattern r often lies outside the set of firing patterns achievable with the prosthesis. The goal of the learned metric is to define a distance measure to identify the nearest feasible electrical stimulationr. R denotes the set of all neural population responses.electrical stimulation pattern that produces a response as close as possible to the desired pattern of activity in terms of the elicited visual sensation ( FIG0 ). In lieu of measuring the visual sensation produced by a prosthetic, we instead posit that one may infer a distance metric based on the signal and noise properties of individual and populations of neurons BID43 BID33 BID11 . In contrast, previous approaches to spike metrics have focused on user-specified, parameteric functions BID52 BID51 or unsupervised techniques to cluster nearby spike patterns BID50 BID9 BID14 .In this work, we propose a neural response metric learned directly from the statistics and structure of firing patterns in neural populations, with the aim of using it to select optimal electrical stimulation patterns in a prosthesis device. In particular, we learn a neural response metric by applying ideas from metric learning to recordings of RGC populations in non-human primate retina. We demonstrate that the learned metric provides an intuitive, meaningful representation of the similarity between spike patterns in the RGC population, capturing the statistics of neural responses as well as similarity between visual images. Finally, we use this metric to select the optimal electrical stimulation pattern within the constraints of the electrical interface to a population of RGCs. The learned metric approach has two major potential implications for visual neuroscience. First, it provides a novel method to find "symbols" in the neural code of the retina that are similar in the sense that they indicate the presence of similar stimuli BID14 . Second, it has an application to retinal prosthesis technology, in which hardware constraints demand that the set of neural responses that can be generated with a device be used to effectively transmit useful visual information. For this application, a metric on responses that reflects visual stimulus similarity could be extremely useful.The present approach differs from previously proposed spike train metrics (reviewed in BID51 ). Previous approaches have employed unsupervised techniques to cluster nearby spike patterns BID14 BID34 BID15 or employed user-specified, paramteric approaches BID53 BID0 . In the case of single snapshots in time used here, the latter approach (Victor-Purpura metric) has only one degree of freedom which is a user-specified cost associated with moving spikes from one cell to another. In our proposed method, the relative importance of cell identity is learned directly from the statistics of population firing patterns.The present work is a stepping stone towards building an encoding algorithm for retinal prostheses. In this paper, we learn the metric using light evoked responses. However, we need to estimate this metric in a blind retina, which has no light evoked responses. The convolutional metric is adaptable to any RGC population by merely noting cell types and center locations. Thus a convolutional metric could be trained on multiple healthy retinas and applied to a blind retina. Preliminary results in this direction indicate that a convolutional metric trained on half of the cells in a retinal recording (training data) generalizes to the other half (validation data), yielding performance higher than a quadratic metric (and comparable to a convolutional metric) trained directly on the validation data.Additional techniques may also be helpful in extending our method to data involving many cells, temporal responses, and additional response structure. For example, using recurrent neural networks BID26 to embed responses may help compute distances between spiking patterns consisting of multiple time bins, perhaps of unequal length. Boosting BID13 may help combine multiple efficiently learned metrics for a smaller, spatially localized groups of cells. Other metrics may be developed to capture invariances learned by commonly used encoding models BID7 BID33 . Also, triplet mining techniques (i.e., choosing hard negatives), a commonly used trick in computer vision, may improve efficiency BID38 BID32 . Novel metrics could also be learned with additional structure in population responses, such as the highly structured correlated activity in RGCs Mastronarde (1983); BID17 . This noise correlation structure may be learnable using negative examples that destroy the noise correlations in data while preserving light response properties, by taking responses of different cells from different repeats of the stimulus.Note that the convolutional metric outperforms the quadratic metric at both global (ROC curves) and local (precision recall curves) scales. However, using current retinal prosthesis technologies, we might be able to resolve information only up to a particular scale. For current retinal prostheses, capturing global structure may be of greatest importance, because state-of-the-art technology has a relatively coarse vocabulary for stimulating RGCs (Humayun et al., 2012; BID58 ) (see also FIG0 ). Specifically, the "nearest" elicited firing pattern is "far" in terms of the corresponding visual stimulus ( FIG5 ) . In terms of the proposed learned metric, the nearest feasible firing pattern achievable by electrical stimulation in our experiments is at the 10th percentile of all possible firing patterns. In this context, the average closest stimulation pattern, expressed as a percentile of the learned metric distances, provides a valuable benchmark to measure the performance of a prosthesis and how that performance is affected by advances in the underlying hardware and software. In particular, the network employs knowledge of the receptive field locations and firing rates of individual cells but the network is independent of the number of cells in the retina. The latter point is achieved by embedding the responses of neurons into pathways grouped by cell type. In our experiments, we focus on 2 cell types (ON and OFF parasols), thus we employ a 2 channel pathway BID22 .The network receives as input the spiking activity of ON and OFF parasols and embeds these spike patterns as one-hot vectors placed at the spatial locations of each cell's receptive field. The resulting pattern of activations is summed across all cells in the ON and OFF populations, respectively, and passed through several convolutional layers of a network. Successive layers shrink the spatial activation size of the representation, while increasing the number of filter channels BID24 BID44 . The final embedding response vector has 1/16th number of pixels in the stimulus and represents the flattened representation of the last layer of the network.Let c denote the number of different cells. The RGC population response is a vector r ∈ {0, 1} c .• Represent responses as vectors over {+1, −1} withr = 2(r − 0.5).• Compute the scale for each cell as a function of the mean firing rate: DISPLAYFORM0 • Map each cell to its center location on a grid with spatial dimensions same as those of visual stimulus. Let M i be grid embedding on cell i. So, M i has zero for all positions except center of cell.• Perform a separable 5 × 5 convolution of stride 1 on each M i to get RF estimate of cell,M i .• Add the activation of cells of the same type to get the total activation for a given cell type.Hence, activation map for each cell type A i = ir i s iMi . Subsequent layers receive input as a two layered activation map corresponding to ON and OFF parasol cells.• The convolutional layers further combine information accross multiple cells, of different types. The details of different layers are shown in FIG6 and Normalization Batch normalization after every convolution Optimizer Adam BID23 ) (α = 0.01, β 1 = 0.9, β 2 = 0.999) Parameter updates 20,000Batch size 100 Weight initialization Xavier initialization BID16
Using triplets to learn a metric for comparing neural responses and improve the performance of a prosthesis. Authors develop new spike train distance metrics, including neural networks and quadratic metrics. These metrics are shown to outperform the naive Hamming distance metric, and implicitly captures some structure in neural code. With the application of improving neural prosthesis in mind, the authors propose to learn a metric between neural responses by either optimizing a quadratic form or a deep neural network .