Matrix coefficients of intertwining operators and the Bruhat order. Let ( pi mathbf z ,V mathbf z ) be an unramified principal series representation of a reductive group over a nonarchimedean local field, parametrized by an element mathbf z of the maximal torus in the Langlands dual group. If v is an element of the Weyl group W , then the standard intertwining integral mathcal A v maps V mathbf z to V v mathbf z . Letting psi mathbf z w with w in W be a suitable basis of the Iwahori fixed vectors in V mathbf z , and widehat psi mathbf z w a basis of the contragredient representation, we define sigma(u,v,w) (for u,v,w in W ) to be langle mathcal A v psi u mathbf z , widehat psi v mathbf z w rangle . This is an interesting function and we initiate its study. We show that given u and w , there is a minimal v such that sigma(u,v,w) neq 0 . Denoting this v as v hbox min v hbox min (u,w) , we will prove that sigma(u,v hbox min ,w) is a polynomial of the cardinality q of the residue field. Indeed if v v hbox min , then sigma(u,v,w) is a rational function of mathbf z and q , whose denominator we describe. But if v v hbox min , the dependence on mathbf z disappears. We will express sigma(u,v hbox min ,w) as the Poincar 'e polynomial of a Bruhat interval. The proof leads to fairly intricate considerations of the Bruhat order. Thus our results require us to prove some facts that may be of independent interest, relating the Bruhat order leqslant and the weak Bruhat order leqslant R . For example we will prove (for finite Coxeter groups) the following "mixed meet" property. If u, w are elements of W , then there exists a unique element m in W that is maximal with respect to the condition that m leqslant R u and m leqslant w . Thus if z leqslant R u and z leqslant w , then x leqslant m . The value v hbox min is m 1 u .


A Two stage Framework and Reinforcement Learning based Optimization Algorithms for Complex Scheduling Problems. There hardly exists a general solver that is efficient for scheduling problems due to their diversity and complexity. In this study, we develop a two stage framework, in which reinforcement learning (RL) and traditional operations research (OR) algorithms are combined together to efficiently deal with complex scheduling problems. The scheduling problem is solved in two stages, including a finite Markov decision process (MDP) and a mixed integer programming process, respectively. This offers a novel and general paradigm that combines RL with OR approaches to solving scheduling problems, which leverages the respective strengths of RL and OR The MDP narrows down the search space of the original problem through an RL method, while the mixed integer programming process is settled by an OR algorithm. These two stages are performed iteratively and interactively until the termination criterion has been met. Under this idea, two implementation versions of the combination methods of RL and OR are put forward. The agile Earth observation satellite scheduling problem is selected as an example to demonstrate the effectiveness of the proposed scheduling framework and methods. The convergence and generalization capability of the methods are verified by the performance of training scenarios, while the efficiency and accuracy are tested in 50 untrained scenarios. The results show that the proposed algorithms could stably and efficiently obtain satisfactory scheduling schemes for agile Earth observation satellite scheduling problems. In addition, it can be found that RL based optimization algorithms have stronger scalability than non learning algorithms. This work reveals the advantage of combining reinforcement learning methods with heuristic methods or mathematical programming methods for solving complex combinatorial optimization problems.


Sense representations for Portuguese experiments with sense embeddings and deep neural language models. Sense representations have gone beyond word representations like Word2Vec, GloVe and FastText and achieved innovative performance on a wide range of natural language processing tasks. Although very useful in many applications, the traditional approaches for generating word embeddings have a strict drawback they produce a single vector representation for a given word ignoring the fact that ambiguous words can assume different meanings. In this paper, we explore unsupervised sense representations which, different from traditional word embeddings, are able to induce different senses of a word by analyzing its contextual semantics in a text. The unsupervised sense representations investigated in this paper are sense embeddings and deep neural language models. We present the first experiments carried out for generating sense embeddings for Portuguese. Our experiments show that the sense embedding model (Sense2vec) outperformed traditional word embeddings in syntactic and semantic analogies task, proving that the language resource generated here can improve the performance of NLP tasks in Portuguese. We also evaluated the performance of pre trained deep neural language models (ELMo and BERT) in two transfer learning approaches feature based and fine tuning, in the semantic textual similarity task. Our experiments indicate that the fine tuned Multilingual and Portuguese BERT language models were able to achieve better accuracy than the ELMo model and baselines.


Space time multilevel quadrature methods and their application for cardiac electrophysiology. We present a novel approach which aims at high performance uncertainty quantification for cardiac electrophysiology simulations. Employing the monodomain equation to model the transmembrane potential inside the cardiac cells, we evaluate the effect of spatially correlated perturbations of the heart fibers on the statistics of the resulting quantities of interest. Our methodology relies on a close integration of multilevel quadrature methods, parallel iterative solvers and space time finite element discretizations, allowing for a fully parallelized framework in space, time and stochastics. Extensive numerical studies are presented to evaluate convergence rates and to compare the performance of classical Monte Carlo methods such as standard Monte Carlo (MC) and quasi Monte Carlo (QMC), as well as multilevel strategies, i.e. multilevel Monte Carlo (MLMC) and multilevel quasi Monte Carlo (MLQMC) on hierarchies of nested meshes. Finally, we employ a recently suggested variant of the multilevel approach for non nested meshes to deal with a realistic heart geometry.


Generalizing Neural Networks by Reflecting Deviating Data in Production. Trained with a sufficiently large training and testing dataset, Deep Neural Networks (DNNs) are expected to generalize. However, inputs may deviate from the training dataset distribution in real deployments. This is a fundamental issue with using a finite dataset. Even worse, real inputs may change over time from the expected distribution. Taken together, these issues may lead deployed DNNs to mis predict in production. In this work, we present a runtime approach that mitigates DNN mis predictions caused by the unexpected runtime inputs to the DNN. In contrast to previous work that considers the structure and parameters of the DNN itself, our approach treats the DNN as a blackbox and focuses on the inputs to the DNN. Our approach has two steps. First, it recognizes and distinguishes "unseen" semantically preserving inputs. For this we use a distribution analyzer based on the distance metric learned by a Siamese network. Second, our approach transforms those unexpected inputs into inputs from the training set that are identified as having similar semantics. We call this process input reflection and formulate it as a search problem over the embedding space on the training set. This embedding space is learned by a Quadruplet network as an auxiliary model for the subject model to improve the generalization. We implemented a tool called InputReflector based on the above two step approach and evaluated it with experiments on three DNN models trained on CIFAR 10, MNIST, and FMINST image datasets. The results show that InputReflector can effectively distinguish inputs that retain semantics of the distribution (e.g., blurred, brightened, contrasted, and zoomed images) and out of distribution inputs from normal inputs.


Client Selection in Federated Learning based on Gradients Importance. Federated learning (FL) enables multiple devices to collaboratively learn a global model without sharing their personal data. In real world applications, the different parties are likely to have heterogeneous data distribution and limited communication bandwidth. In this paper, we are interested in improving the communication efficiency of FL systems. We investigate and design a device selection strategy based on the importance of the gradient norms. In particular, our approach consists of selecting devices with the highest norms of gradient values at each communication round. We study the convergence and the performance of such a selection technique and compare it to existing ones. We perform several experiments with non iid set up. The results show the convergence of our method with a considerable increase of test accuracy comparing to the random selection.


Graph Attention Recurrent Neural Networks for Correlated Time Series Forecasting Full version. We consider a setting where multiple entities inter act with each other over time and the time varying statuses of the entities are represented as multiple correlated time series. For example, speed sensors are deployed in different locations in a road network, where the speed of a specific location across time is captured by the corresponding sensor as a time series, resulting in multiple speed time series from different locations, which are often correlated. To enable accurate forecasting on correlated time series, we proposes graph attention recurrent neural networks.First, we build a graph among different entities by taking into account spatial proximity and employ a multi head attention mechanism to derive adaptive weight matrices for the graph to capture the correlations among vertices (e.g., speeds at different locations) at different timestamps. Second, we employ recurrent neural networks to take into account temporal dependency while taking into account the adaptive weight matrices learned from the first step to consider the correlations among time series.Experiments on a large real world speed time series data set suggest that the proposed method is effective and outperforms the state of the art in most settings. This manuscript provides a full version of a workshop paper 1 .


Saturating PI control of stable nonlinear systems using singular perturbations. This paper presents an anti windup PI controller, using a saturating integrator, for a single input single output stable nonlinear plant, whose steady state input output map is increasing. We prove that, under reasonable assumptions, there exists an upper bound on the controller gain such that for any constant reference input, the corresponding equilibrium point of the closed loop system is exponentially stable, with a "large" region of attraction. When the state of the closed loop system converges to this equilibrium point, then the tracking error tends to zero. The closed loop stability analysis employs Lyapunov methods in the framework of singular perturbations theory. Finally, we show that if the plant satisfies the asymptotic gain property, then the closed loop system is globally asymptotically stable for any sufficiently small controller gain. The effectiveness of the proposed PI controller is proved by showing how it performs as part of the control algorithm of a synchronverter (a special type of DC to AC power converter).


Quantization of restricted Lagrangian subvarieties in positive characteristic. Bezrukavnikov and Kaledin introduced quantizations of symplectic varieties X in positive characteristic which endow the Poisson bracket on X with the structure of a restricted Lie algebra. We consider deformation quantization of line bundles on Lagrangian subvarieties Y of X to modules over such quantizations. If the ideal sheaf of Y is a restricted Lie subalgebra of the structure sheaf of X, we show that there is a certain cohomology class which vanishes if and only if a line bundle on Y admits a quantization.


Spherical Matrix Factorization. Matrix Factorization plays an important role in machine learning such as Non negative Matrix Factorization, Principal Component Analysis, Dictionary Learning, etc. However, most of the studies aim to minimize the loss by measuring the Euclidean distance, though in some fields, angle distance is known to be more important and critical for analysis. In this paper, we propose a method by adding constraints on factors to unify the Euclidean and angle distance. However, due to non convexity of the objective and constraints, the optimized solution is not easy to obtain. In this paper we propose a general framework to systematically solve it with provable convergence guarantee with various constraints.


Charon Load Aware Load Balancing in P4. Load Balancers play an important role in data centers as they distribute network flows across application servers and guarantee per connection consistency. It is hard however to make fair load balancing decisions so that all resources are efficiently occupied yet not overloaded. Tracking connection states allows to infer server load states and make informed decisions, but at the cost of additional memory space consumption. This makes it hard to implement on programmable hardware, which has constrained memory but offers line rate performance. This paper presents Charon, a stateless load aware load balancer that has line rate performance implemented in P4 NetFPGA. Charon passively collects load states from application servers and employs the power of 2 choices scheme to make data driven load balancing decisions and improve resource utilization. Perconnection consistency is preserved statelessly by encoding server ID in a covert channel. The prototype design and implementation details are described in this paper. Simulation results show performance gains in terms of load distribution fairness, quality of service, throughput and processing latency.


Analysis of COVID 19 infection waves in Japan by Avrami equation. Last time, an attempt was made to analyze the first wave of COVID 19 in Japan from February to May 2020 by the Avrami equation. This time, all infection waves that occurred repeatedly from February 2020 to September 2021 were simulated by developing the last work. The entire waveform was basically simulated by the superposition of five major waves, and the least squares method was applied to each wave to determine the parameters of the equation. Despite the simulation using the single equation assuming that the parameters were constants, the accuracy of the simulated cumulative infection D was as good as 95 CI D 2.5 after the second wave when D exceeded 10,000. Since the simulation was highly accurate, it was effective in predicting the number of infections in the near future and in considering the cause of changes in the number of infections observed as the detailed structure of the waveform.


Self Supervised Multi Object Tracking with Cross Input Consistency. In this paper, we propose a self supervised learning procedure for training a robust multi object tracking (MOT) model given only unlabeled video. While several self supervisory learning signals have been proposed in prior work on single object tracking, such as color propagation and cycle consistency, these signals cannot be directly applied for training RNN models, which are needed to achieve accurate MOT they yield degenerate models that, for instance, always match new detections to tracks with the closest initial detections. We propose a novel self supervisory signal that we call cross input consistency we construct two distinct inputs for the same sequence of video, by hiding different information about the sequence in each input. We then compute tracks in that sequence by applying an RNN model independently on each input, and train the model to produce consistent tracks across the two inputs. We evaluate our unsupervised method on MOT17 and KITTI remarkably, we find that, despite training only on unlabeled video, our unsupervised approach outperforms four supervised methods published in the last 1 2 years, including Tracktor , FAMNet, GSM, and mmMOT.


Deep Hough Voting for Robust Global Registration. Point cloud registration is the task of estimating the rigid transformation that aligns a pair of point cloud fragments. We present an efficient and robust framework for pairwise registration of real world 3D scans, leveraging Hough voting in the 6D transformation parameter space. First, deep geometric features are extracted from a point cloud pair to compute putative correspondences. We then construct a set of triplets of correspondences to cast votes on the 6D Hough space, representing the transformation parameters in sparse tensors. Next, a fully convolutional refinement module is applied to refine the noisy votes. Finally, we identify the consensus among the correspondences from the Hough space, which we use to predict our final transformation parameters. Our method outperforms state of the art methods on 3DMatch and 3DLoMatch benchmarks while achieving comparable performance on KITTI odometry dataset. We further demonstrate the generalizability of our approach by setting a new state of the art on ICL NUIM dataset, where we integrate our module into a multi way registration pipeline.


Towards optimized actions in critical situations of soccer games with deep reinforcement learning. Soccer is a sparse rewarding game any smart or careless action in critical situations can change the result of the match. Therefore players, coaches, and scouts are all curious about the best action to be performed in critical situations, such as the times with a high probability of losing ball possession or scoring a goal. This work proposes a new state representation for the soccer game and a batch reinforcement learning to train a smart policy network. This network gets the contextual information of the situation and proposes the optimal action to maximize the expected goal for the team. We performed extensive numerical experiments on the soccer logs made by InStat for 104 European soccer matches. The results show that in all 104 games, the optimized policy obtains higher rewards than its counterpart in the behavior policy. Besides, our framework learns policies that are close to the expected behavior in the real world. For instance, in the optimized policy, we observe that some actions such as foul, or ball out can be sometimes more rewarding than a shot in specific situations.


Exploiting Spatial Dimensions of Latent in GAN for Real time Image Editing. Generative adversarial networks (GANs) synthesize realistic images from random latent vectors. Although manipulating the latent vectors controls the synthesized outputs, editing real images with GANs suffers from i) time consuming optimization for projecting real images to the latent vectors, ii) or inaccurate embedding through an encoder. We propose StyleMapGAN the intermediate latent space has spatial dimensions, and a spatially variant modulation replaces AdaIN. It makes the embedding through an encoder more accurate than existing optimization based methods while maintaining the properties of GANs. Experimental results demonstrate that our method significantly outperforms state of the art models in various image manipulation tasks such as local editing and image interpolation. Last but not least, conventional editing methods on GANs are still valid on our StyleMapGAN. Source code is available at https github.com naver ai StyleMapGAN.


Twin Subsequence Search in Time Series. We address the problem of subsequence search in time series using Chebyshev distance, to which we refer as twin subsequence search. We first show how existing time series indices can be extended to perform twin subsequence search. Then, we introduce TS Index, a novel index tailored to this problem. Our experimental evaluation compares these approaches against real time series datasets, and demonstrates that TS Index can retrieve twin subsequences much faster under various query conditions. This paper has been published in the 24th International Conference on Extending Database Technology (EDBT 2021).


MigCast in Monte Carlo The Impact of Data Model Evolution in NoSQL Databases. During the development of NoSQL backed software, the data model evolves naturally alongside the application code. Especially in agile development, new application releases are deployed frequently causing schema changes. Eventually, decisions have to be made regarding the migration of versioned legacy data which is persisted in the cloud hosted production database. We solve this schema evolution problem and present the results of near exhaustive calculations by means of which software project stakeholders can manage the operative costs for data model evolution and adapt their software release strategy accordingly in order to comply with service level agreements regarding the competing metrics of migration costs and latency. We clarify conclusively how data model evolution in NoSQL databases impacts the metrics while taking all relevant characteristics of migration scenarios into account. As calculating all possible combinatorics in the search space of migration scenarios would by far exceed computational means, we used a probabilistic Monte Carlo method of repeated sampling, serving as a well established means to bring the complexity of data model evolution under control. Our experiments show the qualitative and quantitative impact on the performance of migration strategies with respect to intensity and distribution of data entity accesses, the kinds of schema changes, and the characteristics of the underlying data model.


Statistical Inference with M Estimators on Adaptively Collected Data. Bandit algorithms are increasingly used in real world sequential decision making problems. Associated with this is an increased desire to be able to use the resulting datasets to answer scientific questions like Did one type of ad lead to more purchases? In which contexts is a mobile health intervention effective? However, classical statistical approaches fail to provide valid confidence intervals when used with data collected with bandit algorithms. Alternative methods have recently been developed for simple models (e.g., comparison of means). Yet there is a lack of general methods for conducting statistical inference using more complex models on data collected with (contextual) bandit algorithms for example, current methods cannot be used for valid inference on parameters in a logistic regression model for a binary reward. In this work, we develop theory justifying the use of M estimators which includes estimators based on empirical risk minimization as well as maximum likelihood on data collected with adaptive algorithms, including (contextual) bandit algorithms. Specifically, we show that M estimators, modified with particular adaptive weights, can be used to construct asymptotically valid confidence regions for a variety of inferential targets.


Geometric criteria for mathbb A 1 connectedness and applications to norm varieties. We show that mathbb A 1 connectedness of a large class of varieties over a field k can be characterized as the condition that their generic point can be connected to a k rational point using (not necessarily naive) mathbb A 1 homotopies. We also show that symmetric powers of mathbb A 1 connected varieties (over an arbitrary field), as well as smooth proper models of them (over an algebraically closed field of characteristic 0 ), are mathbb A 1 connected. As an application of these results, we show that the standard norm varieties over a field k of characteristic 0 become mathbb A 1 connected (and consequently, universally R trivial) after base change to an algebraic closure of k .


Scalable Rule Based Representation Learning for Interpretable Classification. Rule based models, e.g., decision trees, are widely used in scenarios demanding high model interpretability for their transparent inner structures and good model expressivity. However, rule based models are hard to optimize, especially on large data sets, due to their discrete parameters and structures. Ensemble methods and fuzzy soft rules are commonly used to improve performance, but they sacrifice the model interpretability. To obtain both good scalability and interpretability, we propose a new classifier, named Rule based Representation Learner (RRL), that automatically learns interpretable non fuzzy rules for data representation and classification. To train the non differentiable RRL effectively, we project it to a continuous space and propose a novel training method, called Gradient Grafting, that can directly optimize the discrete model using gradient descent. An improved design of logical activation functions is also devised to increase the scalability of RRL and enable it to discretize the continuous features end to end. Exhaustive experiments on nine small and four large data sets show that RRL outperforms the competitive interpretable approaches and can be easily adjusted to obtain a trade off between classification accuracy and model complexity for different scenarios. Our code is available at https github.com 12wang3 rrl.


Automatic Symmetry Discovery with Lie Algebra Convolutional Network. Existing equivariant neural networks require prior knowledge of the symmetry group and discretization for continuous groups. We propose to work with Lie algebras (infinitesimal generators) instead of Lie groups. Our model, the Lie algebra convolutional network (L conv) can automatically discover symmetries and does not require discretization of the group. We show that L conv can serve as a building block to construct any group equivariant feedforward architecture. Both CNNs and Graph Convolutional Networks can be expressed as L conv with appropriate groups. We discover direct connections between L conv and physics (1) group invariant loss generalizes field theory (2) Euler Lagrange equation measures the robustness, and (3) equivariance leads to conservation laws and Noether current.These connections open up new avenues for designing more general equivariant networks and applying them to important problems in physical sciences


B 3 block representations of dimension 6 and braid reversions. We construct a family of six dimensional block representations of the braid group B 3 on three strings. We show that some of these representations can be used to detect braid vertibility of some known knots and others of 9 and 10 crossings.


A fractional glance to the theory of edge dislocations. We revisit some recents results inspired by the Peierls Nabarro model on edge dislocations for crystals which rely on the fractional Laplace representation of the corresponding equation. In particular, we discuss results related to heteroclinic, homoclinic and multibump patterns for the atom dislocation function, the large space and time scale of the solutions of the parabolic problem, the dynamics of the dislocation points and the large time asymptotics after possible dislocation collisions.


Context Free TextSpotter for Real Time and Mobile End to End Text Detection and Recognition. In the deployment of scene text spotting systems on mobile platforms, lightweight models with low computation are preferable. In concept, end to end (E2E) text spotting is suitable for such purposes because it performs text detection and recognition in a single model. However, current state of the art E2E methods rely on heavy feature extractors, recurrent sequence modellings, and complex shape aligners to pursue accuracy, which means their computations are still heavy. We explore the opposite direction How far can we go without bells and whistles in E2E text spotting? To this end, we propose a text spotting method that consists of simple convolutions and a few post processes, named Context Free TextSpotter. Experiments using standard benchmarks show that Context Free TextSpotter achieves real time text spotting on a GPU with only three million parameters, which is the smallest and fastest among existing deep text spotters, with an acceptable transcription quality degradation compared to heavier ones. Further, we demonstrate that our text spotter can run on a smartphone with affordable latency, which is valuable for building stand alone OCR applications.


Generalization in Mean Field Games by Learning Master Policies. Mean Field Games (MFGs) can potentially scale multi agent systems to extremely large populations of agents. Yet, most of the literature assumes a single initial distribution for the agents, which limits the practical applications of MFGs. Machine Learning has the potential to solve a wider diversity of MFG problems thanks to generalizations capacities. We study how to leverage these generalization properties to learn policies enabling a typical agent to behave optimally against any population distribution. In reference to the Master equation in MFGs, we coin the term Master policies'' to describe them and we prove that a single Master policy provides a Nash equilibrium, whatever the initial distribution. We propose a method to learn such Master policies. Our approach relies on three ingredients adding the current population distribution as part of the observation, approximating Master policies with neural networks, and training via Reinforcement Learning and Fictitious Play. We illustrate on numerical examples not only the efficiency of the learned Master policy but also its generalization capabilities beyond the distributions used for training.


Homological properties of pinched Veronese rings. Pinched Veronese rings are formed by removing an algebra generator from a Veronese subring of a polynomial ring. We study the homological properties of such rings, including the Cohen Macaulay, Gorenstein, and complete intersection properties. Greco and Martino classified Cohen Macaulayness of pinched Veronese rings by the maximum entry of the exponent vector of the pinched monomial we re prove their results with semigroup methods and correct an omission of a small class of examples of Cohen Macaulay pinched Veronese rings. When the underlying field is of prime characteristic, we show that pinched Veronese rings exhibit a variety of F singularities, including F regular, F injective, and F nilpotent. We also compute upper bounds on the Frobenius test exponents of pinched Veronese rings, a computational invariant which controls the Frobenius closure of all parameter ideals simultaneously.


Developing parsimonious ensembles using predictor diversity within a reinforcement learning framework. Heterogeneous ensembles that can aggregate an unrestricted number and variety of base predictors can effectively address challenging prediction problems. In particular, accurate ensembles that are also parsimonious, i.e., consist of as few base predictors as possible, can help reveal potentially useful knowledge about the target problem domain. Although ensemble selection offers a potential approach to achieving these goals, the currently available algorithms are limited in their abilities. In this paper, we present several algorithms that incorporate ensemble diversity into a reinforcement learning (RL) based ensemble selection framework to build accurate and parsimonious ensembles. These algorithms, as well as several baselines, are rigorously evaluated on datasets from diverse domains in terms of the predictive performance and parsimony of their ensembles. This evaluation demonstrates that our diversity incorporated RL based algorithms perform better than the others for constructing simultaneously accurate and parsimonious ensembles. These algorithms can eventually aid the interpretation or reverse engineering of predictive models assimilated into effective ensembles. To enable such a translation, an implementation of these algorithms, as well the experimental setup they are evaluated in, has been made available at https github.com GauravPandeyLab lens learning ensembles using reinforcement learning.


Fourier coefficients of Siegel cusp forms of degree 2 in the Atkin Lehner type newspace. We prove that a non zero F in the Atkin Lehner type Siegel newspace of degree 2 and an odd level N is determined by fundamental Fourier coefficients up to a divisor of N . In particular when the level is odd and square free we show that F is determined by its fundamental Fourier coefficients.


On the energy stability of Strang splitting for Cahn Hilliard. We consider a Strang type second order operator splitting discretization for the Cahn Hilliard equation. We introduce a new theoretical framework and prove uniform energy stability of the numerical solution and persistence of all higher Sobolev norms. This is the first strong stability result for second order operator splitting methods for the Cahn Hilliard equation. In particular we settle several long standing open issues in the work of Cheng, Kurganov, Qu and Tang cite Tang15 .


POWTEX visits POWGEN. The high intensity time of flight neutron diffractometer POWTEX for powder and texture analysis is currently being built to be operated in the eastern guide hall of the research reactor FRM II near Munich. Having heavily faced the world wide 3He crisis in 2009, we promptly initiated the development of 3He free detector alternatives that are tailor made for the requirements of large area diffractometers. Herein, we report about the 2017 enterprise to operate one building unit of the final POWTEX detector at the neutron powder diffractometer POWGEN at the Spallation Neutron Source of the Oak Ridge National Lab, USA. As a result, we present the first angular and wavelength dependent data of the POWTEX detector, current data reduction steps as implemented in Mantid and also multi dimensional refinement results of two samples (diamond and BaZn(NCN)2) by use of the GSAS II software suite.


The S global dimensions of commutative rings. Let R be a commutative ring with identity and S a multiplicative subset of R . First, we introduce and study the S projective dimensions and S injective dimensions of R modules, and then explore the S global dimension S gl.dim (R) of a commutative ring R which is defined to be the supremum of S projective dimensions of all R modules. Finally, we investigated the S global dimension of factor rings and polynomial rings.


New families of quantum stabilizer codes from Hermitian self orthogonal algebraic geometry codes. There has been a lot of effort to construct good quantum codes from the classical error correcting codes. Constructing new quantum codes, using Hermitian self orthogonal codes, seems to be a difficult problem in general. In this paper, Hermitian self orthogonal codes are studied from algebraic function fields. Sufficient conditions for the Hermitian self orthogonality of an algebraic geometry code are presented. New Hermitian self orthogonal codes are constructed from projective lines, elliptic curves, hyper elliptic curves, Hermitian curves, and Artin Schreier curves. In addition, over the projective lines, we construct new families of MDS quantum codes with parameters N,N 2K,K 1 q under the following conditions i) N t(q 1) 1 or t(q 1) 2 with t (q 1) and K lfloor frac t(q 1) 1 2t rfloor 1 ii) (n 1) (q 2 1) , N n or N n 1 , K 0 lfloor frac n q 1 q 1 rfloor , and K ge K 0 1 iii) N tq 1 , forall 1 le t le q and K lfloor frac tq q 1 q 1 rfloor 1 iv) n (q 2 1) , n 2 frac n gcd (n,q 1) , forall 1 le t le frac q 1 n 2 1 , N (t 1)n 2 and K lfloor frac (t 1)n 1 q 1 q 1 rfloor 1 .


Rockfall susceptibility and network ranked susceptibility along the Italian railway. Rockfalls pose a substantial threat to ground transportation, due to their rapidity, destructive potential and high probability of occurrence on steep topographies, found along roads and railways. Approaches for assessment of rockfall susceptibility range from purely phenomenological methods and statistical methods, suitable for modeling large areas, to purely deterministic ones, usually easier to use in local analyses. A common requirement is the need to locate potential detachment points, often found uphill on cliffs, and the subsequent assessment of the runout areas of rockfalls stemming from such points. Here, we apply a physically based model to calculate rockfall trajectories along the whole Italian railway network, within a corridor of total length of about 17,000 km and varying width. We propose a data driven method for the location of rockfall source points based on expert mapping of potential source areas on sample representative locations. Using empirical distributions of gridded slope values in source areas mapped by experts, we derived probabilistic maps of rockfall sources in the proximity of the railway network, regardless of a particular trigger. Source areas act as starting points of simulated trajectories in the three dimensional model STONE. The program provides a pixel by pixel trajectory count, covering 24,500 km2, the largest homogeneous application of the model to date. We classified the map into a vector susceptibility map of the segments of the railway, for which we provide segment wise rockfall susceptibility. Eventually, we considered a graph representation of the network to classify the segments both on the basis of rockfall susceptibility and the role of each segment in the network, resulting in a network ranked susceptibility. Both maps are useful for subsequent hazard assessment, and to prioritize safety improvements along the railway, at national scale.


Successive Cancellation Decoding of Reed Muller Codes with Fast Hadamard Transform. A novel permuted fast successive cancellation list decoding algorithm with fast Hadamard transform (FHT FSCL) is presented. The proposed decoder initializes L (L ge1) active decoding paths with L random codeword permutations sampled from the full symmetry group of the codes. The path extension in the permutation domain is carried out until the first constituent RM code of order 1 is visited. Conventional path extension of the successive cancellation list decoder is then utilized in the information bit domain. The simulation results show that for a RM code of length 512 with 46 information bits, by running 20 parallel permuted FHT FSCL decoders with L 4 , we reduce 72 of the computational complexity, 22 of the decoding latency, and 84 of the memory consumption of the state of the art simplified successive cancellation decoder that uses 512 permutations sampled from the full symmetry group of the code, with similar error correction performance at the target frame error rate of 10 4 .


Optimization models for fair horizontal collaboration in demand responsive transportation. The advances in information and communication technology are changing theway people move. Companies that offer demand responsive transportation serviceshave the opportunity to reduce their costs and increase their revenues throughcollaboration, while at the same time reducing the environmental impact of theiroperations. We consider the case of companies, offering a shared taxi service, thatare involved in horizontal collaboration and present mixed integer programmingmodels for the optimization of their routes that embed constraints aimed at bal ancing the workload exchange. These constraints bound the imbalance in terms oftraveled time and or served customers to be less than thresholds agreed in advanceby the companies. We also present a heuristic algorithm and show the benefits ofthe collaboration.


Primitive normalisers in quasipolynomial time. The normaliser problem has as input two subgroups H and K of the symmetric group S n , and asks for a generating set for N K(H) it is not known to have a subexponential time solution. It is proved in Roney Dougal Siccha, 2020 that if H is primitive then the normaliser problem can be solved in quasipolynomial time. We show that for all subgroups H and K of S n , in quasipolynomial time we can decide whether N S n (H) is primitive, and if so compute N K(H) . Hence we reduce the question of whether one can solve the normaliser problem in quasipolynomial time to the case where the normaliser is known not to be primitive.


Asymptotic Behavior of the Steady Prandtl Equation. We study the asymptotic behavior of the Oleinik's solution to the steady Prandtl equation when the outer flow U(x) 1 . Serrin proved that the Oleinik's solution converges to the famous Blasius solution bar u in L infty y sense as x rightarrow infty . The explicit decay estimates of u bar u and its derivatives were proved by Iyer ARMA 237(2020) when the initial data is a small localized perturbation of the Blasius profile. In this paper, we prove the explicit decay estimate of u(x,y) bar u (x,y) L infty y for general initial data with exponential decay. We also prove the decay estimates of its derivatives when the data has an additional concave assumption. Our proof is based on the maximum principle technique. The key ingredient is to find a series of barrier functions.


I Know What You Imported Last Summer A study of security threats in thePython ecosystem. The popularity of Python has risen rapidly over the past 15 years. It is a major language in some of the most exciting technologies today. This popularity has led to a large ecosystem of third party packages available via the pip package registry which hosts more than 200,000 packages. These third party packages can be reused by simply importing the package after installing using package managers like pip. The ease of reuse of third party software comes with security risks putting millions of users in danger. In this project, we study the ecosystem to analyze this threat. The mature ecosystem of Python has multiple weak spots that we highlight in our project. First, we demonstrate how trivial it is to exploit the Python ecosystem. Then, we systematically analyze dependencies amongst packages, maintainers, and publicly reported security issues. Most attacks are possible only if users install malicious packages. We thus try to analyze and evaluate different methods used by attackers to force incorrect downloads. We quantify your ideas by estimating the potential threat that can be caused by exploiting a popular Python package. We also discuss methods used in the industry to defend against such attacks


Connectivity of spaces of directed paths in geometric models for concurrent computation. Higher Dimensional Automata (HDA) are higher dimensional relatives to transition systems in concurrency theory taking into account to which degree various actions commute. Mathematically, they take the form of labelled cubical complexes. It is important to know, and challenging from a geometric topological perspective, whether the space of directed paths (executions in the model) between two vertices (states) is connected more generally, to estimate higher connectedness of these path spaces. This paper presents an approach for such an estimation for particularly simple HDA modelling the access of a number of processors to a number of resources with given limited capacity each. It defines a spare capacity for a concurrent program with prescribed periods of access of the processors to the resources. It shows that the connectedness of spaces of directed paths can be estimated (from above) by spare capacities. Moreover, spare capacities can also be used to detect deadlocks and critical states in such a HDA. The key theoretical ingredient is a transition from the calculation of local connectedness bounds (of the upper links of vertices of an HDA) to global ones by applying a version of the nerve lemma due to Anders Bj "orner.


ProSTformer Pre trained Progressive Space Time Self attention Model for Traffic Flow Forecasting. Traffic flow forecasting is essential and challenging to intelligent city management and public safety. Recent studies have shown the potential of convolution free Transformer approach to extract the dynamic dependencies among complex influencing factors. However, two issues prevent the approach from being effectively applied in traffic flow forecasting. First, it ignores the spatiotemporal structure of the traffic flow videos. Second, for a long sequence, it is hard to focus on crucial attention due to the quadratic times dot product computation. To address the two issues, we first factorize the dependencies and then design a progressive space time self attention mechanism named ProSTformer. It has two distinctive characteristics (1) corresponding to the factorization, the self attention mechanism progressively focuses on spatial dependence from local to global regions, on temporal dependence from inside to outside fragment (i.e., closeness, period, and trend), and finally on external dependence such as weather, temperature, and day of week (2) by incorporating the spatiotemporal structure into the self attention mechanism, each block in ProSTformer highlights the unique dependence by aggregating the regions with spatiotemporal positions to significantly decrease the computation. We evaluate ProSTformer on two traffic datasets, and each dataset includes three separate datasets with big, medium, and small scales. Despite the radically different design compared to the convolutional architectures for traffic flow forecasting, ProSTformer performs better or the same on the big scale datasets than six state of the art baseline methods by RMSE. When pre trained on the big scale datasets and transferred to the medium and small scale datasets, ProSTformer achieves a significant enhancement and behaves best.


Linear stability analysis of the Couette flow for the two dimensional non isentropic compressible Euler equations. This note is devoted to the linear stability of the Couette flow for the non isentropic compressible Euler equations in a domain mathbb T times mathbb R . Exploiting the several conservation laws originated from the special structure of the linear system, we obtain a Lyapunov type instability for the density, the temperature, the compressible part of the velocity field, and also obtain an inviscid damping for the incompressible part of the velocity field.


Attention, please! A survey of Neural Attention Models in Deep Learning. In humans, Attention is a core property of all perceptual and cognitive operations. Given our limited ability to process competing sources, attention mechanisms select, modulate, and focus on the information most relevant to behavior. For decades, concepts and functions of attention have been studied in philosophy, psychology, neuroscience, and computing. For the last six years, this property has been widely explored in deep neural networks. Currently, the state of the art in Deep Learning is represented by neural attention models in several application domains. This survey provides a comprehensive overview and analysis of developments in neural attention models. We systematically reviewed hundreds of architectures in the area, identifying and discussing those in which attention has shown a significant impact. We also developed and made public an automated methodology to facilitate the development of reviews in the area. By critically analyzing 650 works, we describe the primary uses of attention in convolutional, recurrent networks and generative models, identifying common subgroups of uses and applications. Furthermore, we describe the impact of attention in different application domains and their impact on neural networks' interpretability. Finally, we list possible trends and opportunities for further research, hoping that this review will provide a succinct overview of the main attentional models in the area and guide researchers in developing future approaches that will drive further improvements.


Fair and Adventurous Enumeration of Quantifier Instantiations. SMT solvers generally tackle quantifiers by instantiating their variables with tuples of terms from the ground part of the formula. Recent enumerative approaches for quantifier instantiation consider tuples of terms in some heuristic order. This paper studies different strategies to order such tuples and their impact on performance. We decouple the ordering problem into two parts. First is the order of the sequence of terms to consider for each quantified variable, and second is the order of the instantiation tuples themselves. While the most and least preferred tuples, i.e. those with all variables assigned to the most or least preferred terms, are clear, the combinations in between allow flexibility in an implementation. We look at principled strategies of complete enumeration, where some strategies are more fair, meaning they treat all the variables the same but some strategies may be more adventurous, meaning that they may venture further down the preference list. We further describe new techniques for discarding irrelevant instantiations which are crucial for the performance of these strategies in practice. These strategies are implemented in the SMT solver cvc5, where they contribute to the diversification of the solver's configuration space, as shown by our experimental results.


L p solutions of backward stochastic differential equations with time delayed generators. This article is devoted to study the class of backward stochastic differential equation with delayed generator. We suppose the terminal value and the generator to be L p integrable with p 1 . We derive a new type of estimation related to this BSDE. Next, we establish the existence and uniqueness result in two ways. First, an approximation technics used by Briand et al. (Stochastic Process. Appl. 108 (2003) 109 129) and hence the well know Picard iterative procedure. Using Picard iterative procedure, we revisit the result of Dos Reis et al. (Stochastic Process. Appl. 121 (9) (2011) 2114 2150), simplifying the proof and give an explicit existence and uniqueness condition related to the Lipschitz constant K and the terminal time T .


A Cooperative Optimal Mining Model for Bitcoin. We analyze Bitcoin mining from the perspective of a game and propose an optimal mining model that maximizes profits of pools and miners. The model is a two stage Stackelberg game in which each stage forms a sub game. In stage I, pools are the leaders who assign a computing power to be consumed by miners. In stage II, miners decide of their power consumption and distribution. They find themselves in a social dilemma in which they must choose between mining in solo, therefore prioritizing their individual preferences, and participating in a pool for the collective interest. The model relies on a pool protocol based on a simulated game in which the miners compete for the reward won by the pool. The solutions for the stage I sub game and the simulated protocol game are unique and stable Nash equilibriums while the stage II sub game leads to a stable cooperative equilibrium only when miners choose their strategies according to certain criteria. We conclude that the cooperative optimal mining model has the potential to favor Bitcoin decentralization and stability. Mainly, the social dilemma faced by miners together with the balance of incentives ensure a certain distribution of the network computing power between pools and solo miners, while equilibriums in the game solutions provide stability to the system.


Scalable Anytime Algorithms for Learning Fragments of Linear Temporal Logic. Linear temporal logic (LTL) is a specification language for finite sequences (called traces) widely used in program verification, motion planning in robotics, process mining, and many other areas. We consider the problem of learning LTL formulas for classifying traces despite a growing interest of the research community, existing solutions suffer from two limitations they do not scale beyond small formulas, and they may exhaust computational resources without returning any result. We introduce a new algorithm addressing both issues our algorithm is able to construct formulas an order of magnitude larger than previous methods, and it is anytime, meaning that it in most cases successfully outputs a formula, albeit possibly not of minimal size. We evaluate the performances of our algorithm using an open source implementation against publicly available benchmarks.


Few Shot Video Object Detection. We introduce Few Shot Video Object Detection (FSVOD) with three contributions to visual learning in our highly diverse and dynamic world 1) a large scale video dataset FSVOD 500 comprising of 500 classes with class balanced videos in each category for few shot learning 2) a novel Tube Proposal Network (TPN) to generate high quality video tube proposals for aggregating feature representation for the target video object which can be highly dynamic 3) a strategically improved Temporal Matching Network (TMN ) for matching representative query tube features with better discriminative ability thus achieving higher diversity. Our TPN and TMN are jointly and end to end trained. Extensive experiments demonstrate that our method produces significantly better detection results on two few shot video object detection datasets compared to image based methods and other naive video based extensions. Codes and datasets will be released at https github.com fanq15 FewX.


Provably Efficient Black Box Action Poisoning Attacks Against Reinforcement Learning. Due to the broad range of applications of reinforcement learning (RL), understanding the effects of adversarial attacks against RL model is essential for the safe applications of this model. Prior theoretical works on adversarial attacks against RL mainly focus on either observation poisoning attacks or environment poisoning attacks. In this paper, we introduce a new class of attacks named action poisoning attacks, where an adversary can change the action signal selected by the agent. Compared with existing attack models, the attacker's ability in the proposed action poisoning attack model is more restricted, which brings some design challenges. We study the action poisoning attack in both white box and black box settings. We introduce an adaptive attack scheme called LCB H, which works for most RL agents in the black box setting. We prove that the LCB H attack can force any efficient RL agent, whose dynamic regret scales sublinearly with the total number of steps taken, to choose actions according to a policy selected by the attacker very frequently, with only sublinear cost. In addition, we apply LCB H attack against a popular model free RL algorithm UCB H. We show that, even in the black box setting, by spending only logarithm cost, the proposed LCB H attack scheme can force the UCB H agent to choose actions according to the policy selected by the attacker very frequently.


A Survey on Deep Learning Approaches for Vehicle Trajectory Prediction in Autonomous Driving. With the rapid development of machine learning, autonomous driving has become a hot issue, making urgent demands for more intelligent perception and planning systems. Self driving cars can avoid traffic crashes with precisely predicted future trajectories of surrounding vehicles. In this work, we review and categorize existing learning based trajectory forecasting methods from perspectives of representation, modeling, and learning. Moreover, we make our implementation of Target driveN Trajectory Prediction publicly available at https github.com Henry1iu TNT Trajectory Predition, demonstrating its outstanding performance whereas its original codes are withheld. Enlightenment is expected for researchers seeking to improve trajectory prediction performance based on the achievement we have made.


TENT Efficient Quantization of Neural Networks on the tiny Edge with Tapered FixEd PoiNT. In this research, we propose a new low precision framework, TENT, to leverage the benefits of a tapered fixed point numerical format in TinyML models. We introduce a tapered fixed point quantization algorithm that matches the numerical format's dynamic range and distribution to that of the deep neural network model's parameter distribution at each layer. An accelerator architecture for the tapered fixed point with TENT framework is proposed. Results show that the accuracy on classification tasks improves up to 31 with an energy overhead of 17 30 as compared to fixed point, for ConvNet and ResNet 18 models.


Temperature sensitivity of pest reproductive numbers in age structured PDE models, with a focus on the invasive spotted lanternfly. Invasive pest establishment is a pervasive threat to global ecosystems, agriculture, and public health. The recent establishment of the invasive spotted lanternfly in the northeastern United States has proven devastating to farms and vineyards, necessitating urgent development of population dynamical models and effective control practices. In this paper, we propose a stage and age structured system of PDEs to model insect pest populations, in which underlying dynamics are dictated by ambient temperature through rates of development, fecundity, and mortality. The model incorporates diapause and non diapause pathways, and is calibrated to experimental and field data on the spotted lanternfly. We develop a novel moving mesh method for capturing age advection accurately, even for coarse discretization parameters. We define a one year reproductive number ( R 0 ) from the spectrum of a one year solution operator, and study its sensitivity to variations in the mean and amplitude of the annual temperature profile. We quantify assumptions sufficient to give rise to the low rank structure of the solution operator characteristic of part of the parameter domain. We discuss establishment potential as it results from the pairing of a favorable R 0 value and transient population survival, and address implications for pest control strategies.


Generative Adversarial Networks A Survey Towards Private and Secure Applications. Generative Adversarial Networks (GAN) have promoted a variety of applications in computer vision, natural language processing, etc. due to its generative model's compelling ability to generate realistic examples plausibly drawn from an existing distribution of samples. GAN not only provides impressive performance on data generation based tasks but also stimulates fertilization for privacy and security oriented research because of its game theoretic optimization strategy. Unfortunately, there are no comprehensive surveys on GAN in privacy and security, which motivates this survey paper to summarize those state of the art works systematically. The existing works are classified into proper categories based on privacy and security functions, and this survey paper conducts a comprehensive analysis of their advantages and drawbacks. Considering that GAN in privacy and security is still at a very initial stage and has imposed unique challenges that are yet to be well addressed, this paper also sheds light on some potential privacy and security applications with GAN and elaborates on some future research directions.


Improved convergence rates and trajectory convergence for primal dual dynamical systems with vanishing damping. In this work, we approach the minimization of a continuously differentiable convex function under linear equality constraints by a second order dynamical system with asymptotically vanishing damping term. The system is formulated in terms of the augmented Lagrangian associated to the minimization problem. We show fast convergence of the primal dual gap, the feasibility measure, and the objective function value along the generated trajectories. In case the objective function has Lipschitz continuous gradient, we show that the primal dual trajectory asymptotically weakly converges to a primal dual optimal solution of the underlying minimization problem. To the best of our knowledge, this is the first result which guarantees the convergence of the trajectory generated by a primal dual dynamical system with asymptotic vanishing damping. Moreover, we will rediscover in case of the unconstrained minimization of a convex differentiable function with Lipschitz continuous gradient all convergence statements obtained in the literature for Nesterov's accelerated gradient method.


DeepAnalyze Learning to Localize Crashes at Scale. Crash localization, an important step in debugging crashes, is challenging when dealing with an extremely large number of diverse applications and platforms and underlying root causes. Large scale error reporting systems, e.g., Windows Error Reporting (WER), commonly rely on manually developed rules and heuristics to localize blamed frames causing the crashes. As new applications and features are routinely introduced and existing applications are run under new environments, developing new rules and maintaining existing ones become extremely challenging. We propose a data driven solution to address the problem. We start with the first large scale empirical study of 362K crashes and their blamed methods reported to WER by tens of thousands of applications running in the field. The analysis provides valuable insights on where and how the crashes happen and what methods to blame for the crashes. These insights enable us to develop DeepAnalyze, a novel multi task sequence labeling approach for identifying blamed frames in stack traces. We evaluate our model with over a million real world crashes from four popular Microsoft applications and show that DeepAnalyze, trained with crashes from one set of applications, not only accurately localizes crashes of the same applications, but also bootstraps crash localization for other applications with zero to very little additional training data.


Point line based RGB D SLAM and Bundle Adjustment Uncertainty Analysis. Most of the state of the art indirect visual SLAM methods are based on the sparse point features. However, it is hard to find enough reliable point features for state estimation in the case of low textured scenes. Line features are abundant in urban and indoor scenes. Recent studies have shown that the combination of point and line features can provide better accuracy despite the decrease in computational efficiency. In this paper, measurements of point and line features are extracted from RGB D data to create map features, and points on a line are treated as keypoints. We propose an extended approach to make more use of line observation information. And we prove that, in the local bundle adjustment, the estimation uncertainty of keyframe poses can be reduced when considering more landmarks with independent measurements in the optimization process. Experimental results on two public RGB D datasets demonstrate that the proposed method has better robustness and accuracy in challenging environments.


Border SegGCN Improving Semantic Segmentation by Refining the Border Outline using Graph Convolutional Network. We present Border SegGCN, a novel architecture to improve semantic segmentation by refining the border outline using graph convolutional networks (GCN). The semantic segmentation network such as Unet or DeepLabV3 is used as a base network to have pre segmented output. This output is converted into a graphical structure and fed into the GCN to improve the border pixel prediction of the pre segmented output. We explored and studied the factors such as border thickness, number of edges for a node, and the number of features to be fed into the GCN by performing experiments. We demonstrate the effectiveness of the Border SegGCN on the CamVid and Carla dataset, achieving a test set performance of 81.96 without any post processing on CamVid dataset. It is higher than the reported state of the art mIoU achieved on CamVid dataset by 0.404


Learning with Instance Bundles for Reading Comprehension. When training most modern reading comprehension models, all the questions associated with a context are treated as being independent from each other. However, closely related questions and their corresponding answers are not independent, and leveraging these relationships could provide a strong supervision signal to a model. Drawing on ideas from contrastive estimation, we introduce several new supervision techniques that compare question answer scores across multiple related instances. Specifically, we normalize these scores across various neighborhoods of closely contrasting questions and or answers, adding another cross entropy loss term that is used in addition to traditional maximum likelihood estimation. Our techniques require bundles of related question answer pairs, which we can either mine from within existing data or create using various automated heuristics. We empirically demonstrate the effectiveness of training with instance bundles on two datasets HotpotQA and ROPES showing up to 11 absolute gains in accuracy.


CLIP4Caption Multi CLIP for Video Caption. This report describes our solution to the VALUE Challenge 2021 in the captioning task. Our solution, named CLIP4Caption , is built on X Linear X Transformer, which is an advanced model with encoder decoder architecture. We make the following improvements on the proposed CLIP4Caption We employ an advanced encoder decoder model architecture X Transformer as our main framework and make the following improvements 1) we utilize three strong pre trained CLIP models to extract the text related appearance visual features. 2) we adopt the TSN sampling strategy for data enhancement. 3) we involve the video subtitle information to provide richer semantic information. 3) we introduce the subtitle information, which fuses with the visual features as guidance. 4) we design word level and sentence level ensemble strategies. Our proposed method achieves 86.5, 148.4, 64.5 CIDEr scores on VATEX, YC2C, and TVC datasets, respectively, which shows the superior performance of our proposed CLIP4Caption on all three datasets.


On the Role of System Software in Energy Management of Neuromorphic Computing. Neuromorphic computing systems such as DYNAPs and Loihi have recently been introduced to the computing community to improve performance and energy efficiency of machine learning programs, especially those that are implemented using Spiking Neural Network (SNN). The role of a system software for neuromorphic systems is to cluster a large machine learning model (e.g., with many neurons and synapses) and map these clusters to the computing resources of the hardware. In this work, we formulate the energy consumption of a neuromorphic hardware, considering the power consumed by neurons and synapses, and the energy consumed in communicating spikes on the interconnect. Based on such formulation, we first evaluate the role of a system software in managing the energy consumption of neuromorphic systems. Next, we formulate a simple heuristic based mapping approach to place the neurons and synapses onto the computing resources to reduce energy consumption. We evaluate our approach with 10 machine learning applications and demonstrate that the proposed mapping approach leads to a significant reduction of energy consumption of neuromorphic computing systems.


SomnNET An SpO2 Based Deep Learning Network for Sleep Apnea Detection in Smartwatches. The abnormal pause or rate reduction in breathing is known as the sleep apnea hypopnea syndrome and affects the quality of sleep of an individual. A novel method for the detection of sleep apnea events (pause in breathing) from peripheral oxygen saturation (SpO2) signals obtained from wearable devices is discussed in this paper. The paper details an apnea detection algorithm of a very high resolution on a per second basis for which a 1 dimensional convolutional neural network which we termed SomnNET is developed. This network exhibits an accuracy of 97.08 and outperforms several lower resolution state of the art apnea detection methods. The feasibility of model pruning and binarization to reduce the computational complexity is explored. The pruned network with 80 sparsity exhibited an accuracy of 89.75 , and the binarized network exhibited an accuracy of 68.22 . The performance of the proposed networks is compared against several state of the art algorithms.


Majorization Minimization Methods for Distributed Pose Graph Optimization. We consider the problem of distributed pose graph optimization (PGO) that has important applications in multi robot simultaneous localization and mapping (SLAM). We propose the majorization minimization (MM) method for distributed PGO ( mathsf MM ! ! ! !PGO ) that applies to a broad class of robust loss kernels. The mathsf MM ! ! ! !PGO method is guaranteed to converge to first order critical points under mild conditions. Furthermore, noting that the mathsf MM ! ! ! !PGO method is reminiscent of proximal methods, we leverage Nesterov's method and adopt adaptive restarts to accelerate convergence. The resulting accelerated MM methods for distributed PGO both with a master node in the network ( mathsf AMM ! ! ! !PGO ) and without ( mathsf AMM ! ! ! !PGO ) have faster convergence in contrast to the mathsf MM ! ! ! !PGO method without sacrificing theoretical guarantees. In particular, the mathsf AMM ! ! ! !PGO method, which needs no master node and is fully decentralized, features a novel adaptive restart scheme and has a rate of convergence comparable to that of the mathsf AMM ! ! ! !PGO method using a master node to aggregate information from all the other nodes. The efficacy of this work is validated through extensive applications to 2D and 3D SLAM benchmark datasets and comprehensive comparisons against existing state of the art methods, indicating that our MM methods converge faster and result in better solutions to distributed PGO.


Rational functions as new variables. In multicentric calculus one takes a polynomial p with distinct roots as a new variable and represents complex valued functions by mathbb C d valued functions, where d is the degree of p . An application is e.g. the possibility to represent a piecewise constant holomorphic function as a convergent power series, simultaneously in all components of p(z) le rho . In this paper we study the necessary modifications needed, if we take a rational function r p q as the new variable instead. This allows to consider functions defined in neighborhoods of any compact set as opposed to the polynomial case where the domains p(z) le rho are always polynomially convex. Two applications are formulated. One giving a convergent power series expression for Sylvester equations AX XB C in the general case of A,B being bounded operators in Banach spaces with distinct spectra. The other application formulates a K spectral result for bounded operators in Hilbert spaces.


Thermal Signals from Collective Electronic Excitations in Inhomogeneous Warm Dense Matter. We predict the emergence of novel collective electronic excitations in warm dense matter with an inhomogeneous electronic structure based on first principles calculations. The emerging modes are controlled by the imposed perturbation amplitude. They include satellite signals around the standard plasmon feature, transformation of plasmons to optical modes, and double plasmon modes. Most importantly, these modes exhibit a pronounced dependence on the temperature. This makes them potentially invaluable for the diagnostics of plasma parameters in the warm dense matter regime. We demonstrate that these modes can be probed with present experimental techniques.


Using Shape to Categorize Low Shot Learning with an Explicit Shape Bias. It is widely accepted that reasoning about object shape is important for object recognition. However, the most powerful object recognition methods today do not explicitly make use of object shape during learning. In this work, motivated by recent developments in low shot learning, findings in developmental psychology, and the increased use of synthetic data in computer vision research, we investigate how reasoning about 3D shape can be used to improve low shot learning methods' generalization performance. We propose a new way to improve existing low shot learning approaches by learning a discriminative embedding space using 3D object shape, and using this embedding by learning how to map images into it. Our new approach improves the performance of image only low shot learning approaches on multiple datasets. We also introduce Toys4K, a 3D object dataset with the largest number of object categories currently available, which supports low shot learning.


math.NA cs.NA physics.comp-ph physics.flu-dyn

Geometric Quasilinearization Framework for Analysis and Design of Bound Preserving Schemes. Solutions to many partial differential equations satisfy certain bounds or constraints. For example, the density and pressure are positive for equations of fluid dynamics, and in the relativistic case the fluid velocity is upper bounded by the speed of light, etc. As widely realized, it is crucial to develop bound preserving numerical methods that preserve such intrinsic constraints. Exploring provably bound preserving schemes has attracted much attention and is actively studied in recent years. This is however still a challenging task for many systems especially those involving nonlinear constraints. Based on some key insights from geometry, we systematically propose an innovative and general framework, referred to as geometric quasilinearization (GQL), which paves a new effective way for studying bound preserving problems with nonlinear constraints. The essential idea of GQL is to equivalently transfer all nonlinear constraints into linear ones, through properly introducing some free auxiliary variables. We establish the fundamental principle and general theory of GQL via the geometric properties of convex regions, and propose three simple effective methods for constructing GQL. We apply the GQL approach to a variety of partial differential equations, and demonstrate its effectiveness and remarkable advantages for studying bound preserving schemes, by diverse challenging examples and applications which cannot be easily handled by direct or traditional approaches.


Low Precision Hardware Architectures Meet Recommendation Model Inference at Scale. Tremendous success of machine learning (ML) and the unabated growth in ML model complexity motivated many ML specific designs in both CPU and accelerator architectures to speed up the model inference. While these architectures are diverse, highly optimized low precision arithmetic is a component shared by most. Impressive compute throughputs are indeed often exhibited by these architectures on benchmark ML models. Nevertheless, production models such as recommendation systems important to Facebook's personalization services are demanding and complex These systems must serve billions of users per month responsively with low latency while maintaining high prediction accuracy, notwithstanding computations with many tens of billions parameters per inference. Do these low precision architectures work well with our production recommendation systems? They do. But not without significant effort. We share in this paper our search strategies to adapt reference recommendation models to low precision hardware, our optimization of low precision compute kernels, and the design and development of tool chain so as to maintain our models' accuracy throughout their lifespan during which topic trends and users' interests inevitably evolve. Practicing these low precision technologies helped us save datacenter capacities while deploying models with up to 5X complexity that would otherwise not be deployed on traditional general purpose CPUs. We believe these lessons from the trenches promote better co design between hardware architecture and software engineering and advance the state of the art of ML in industry.


Joint Text and Label Generation for Spoken Language Understanding. Generalization is a central problem in machine learning, especially when data is limited. Using prior information to enforce constraints is the principled way of encouraging generalization. In this work, we propose to leverage the prior information embedded in pretrained language models (LM) to improve generalization for intent classification and slot labeling tasks with limited training data. Specifically, we extract prior knowledge from pretrained LM in the form of synthetic data, which encode the prior implicitly. We fine tune the LM to generate an augmented language, which contains not only text but also encodes both intent labels and slot labels. The generated synthetic data can be used to train a classifier later. Since the generated data may contain noise, we rephrase the learning from generated data as learning with noisy labels. We then utilize the mixout regularization for the classifier and prove its effectiveness to resist label noise in generated data. Empirically, our method demonstrates superior performance and outperforms the baseline by a large margin.


On combinatorial properties and the zero distribution of certain Sheffer sequences. We present combinatorial and analytical results concerning a Sheffer sequence with a generating function of the form G(x,z) Q(z) x Q( z) 1 x , where Q is a quadratic polynomial with real zeros. By using the properties of Riordan matrices we address combinatorial properties and interpretations of our Sheffer sequence of polynomials and their coefficients. We also show that apart from two exceptional zeros, the zeros of polynomials with large enough degree in such a Sheffer sequence lie on the line x 1 2 it .


Automatic Large Scale Detection of Red Palm Weevil Infestation using Aerial and Street View Images. The spread of the Red Palm Weevil has dramatically affected date growers, homeowners and governments, forcing them to deal with a constant threat to their palm trees. Early detection of palm tree infestation has been proven to be critical in order to allow treatment that may save trees from irreversible damage, and is most commonly performed by local physical access for individual tree monitoring. Here, we present a novel method for surveillance of Red Palm Weevil infested palm trees utilizing state of the art deep learning algorithms, with aerial and street level imagery data. To detect infested palm trees we analyzed over 100,000 aerial and street images, mapping the location of palm trees in urban areas. Using this procedure, we discovered and verified infested palm trees at various locations.


Does enhanced shape bias improve neural network robustness to common corruptions?. Convolutional neural networks (CNNs) learn to extract representations of complex features, such as object shapes and textures to solve image recognition tasks. Recent work indicates that CNNs trained on ImageNet are biased towards features that encode textures and that these alone are sufficient to generalize to unseen test data from the same distribution as the training data but often fail to generalize to out of distribution data. It has been shown that augmenting the training data with different image styles decreases this texture bias in favor of increased shape bias while at the same time improving robustness to common corruptions, such as noise and blur. Commonly, this is interpreted as shape bias increasing corruption robustness. However, this relationship is only hypothesized. We perform a systematic study of different ways of composing inputs based on natural images, explicit edge information, and stylization. While stylization is essential for achieving high corruption robustness, we do not find a clear correlation between shape bias and robustness. We conclude that the data augmentation caused by style variation accounts for the improved corruption robustness and increased shape bias is only a byproduct.


WikiGraphs A Wikipedia Text Knowledge Graph Paired Dataset. We present a new dataset of Wikipedia articles each paired with a knowledge graph, to facilitate the research in conditional text generation, graph generation and graph representation learning. Existing graph text paired datasets typically contain small graphs and short text (1 or few sentences), thus limiting the capabilities of the models that can be learned on the data. Our new dataset WikiGraphs is collected by pairing each Wikipedia article from the established WikiText 103 benchmark (Merity et al., 2016) with a subgraph from the Freebase knowledge graph (Bollacker et al., 2008). This makes it easy to benchmark against other state of the art text generative models that are capable of generating long paragraphs of coherent text. Both the graphs and the text data are of significantly larger scale compared to prior graph text paired datasets. We present baseline graph neural network and transformer model results on our dataset for 3 tasks graph text generation, graph text retrieval and text graph retrieval. We show that better conditioning on the graph provides gains in generation and retrieval quality but there is still large room for improvement.


Ammonia induced Calcium Phosphate Nanostructure A Potential Assay for Studying Osteoporosis and Bone Metastasis. Osteoclastic resorption of bone plays a central role in both osteoporosis and bone metastasis. A reliable in vitro assay that simulates osteoclastic resorption in vivo would significantly speed up the process of devel oping effective therapeutic solutions for those diseases. Here we reported the development of a novel and robust nano structured calcium phosphate coating with unique functions on the track etched porous mem brane by using an ammonia induced mineralization (AiM) technique. The calcium phosphate coating uni formly covers one side of the PET membrane enabling testing for osteoclastic resorption. The track etched pores in the PET membrane allow calcium phosphate mineral pins to grow inside, which, on one hand, enhances coating integration with membrane substrate, and on the other hand provides diffusion channels for delivering drugs from the lower chamber of a double chamber cell culture system. The applications of the processed calcium phosphate coating was first demonstrated as a drug screening device by using alen dronate, a widely used drug for osteoporosis. It was confirmed that the delivery of alendronate significant ly decreased both the number of monocyte differentiated osteoclasts and coating resorption. To demon strate the application in studying bone metastasis, we delivered PC3 prostate cancer conditioned medium and confirmed that both the differentiation of monocytes into osteoclasts and the osteoclastic resorption of the calcium phosphate coating were significantly enhanced. This novel assay thus provides a new platform for studying osteoclastic activities and assessing drug efficacy in vitro.


Shape or Texture Understanding Discriminative Features in CNNs. Contrasting the previous evidence that neurons in the later layers of a Convolutional Neural Network (CNN) respond to complex object shapes, recent studies have shown that CNNs actually exhibit a texture bias' given an image with both texture and shape cues (e.g., a stylized image), a CNN is biased towards predicting the category corresponding to the texture. However, these previous studies conduct experiments on the final classification output of the network, and fail to robustly evaluate the bias contained (i) in the latent representations, and (ii) on a per pixel level. In this paper, we design a series of experiments that overcome these issues. We do this with the goal of better understanding what type of shape information contained in the network is discriminative, where shape information is encoded, as well as when the network learns about object shape during training. We show that a network learns the majority of overall shape information at the first few epochs of training and that this information is largely encoded in the last few layers of a CNN. Finally, we show that the encoding of shape does not imply the encoding of localized per pixel semantic information. The experimental results and findings provide a more accurate understanding of the behaviour of current CNNs, thus helping to inform future design choices.


Fundamental component enhancement via adaptive nonlinear activation functions. In many real world oscillatory signals, the fundamental component of a signal f(t) might be weak or does not exist. This makes it difficult to estimate the instantaneous frequency of the signal. Traditionally, researchers apply the rectification trick, working with f(t) or mbox ReLu (f(t)) instead, to enhance the fundamental component. This raises an interesting question what type of nonlinear function g mathbb R rightarrow mathbb R has the property that g(f(t)) has a more pronounced fundamental frequency? g(t) t and g(t) mbox ReLu (t) seem to work well in practice we propose a variant of g(t) 1 (1 t ) and provide a theoretical guarantee. Several simulated signals and real signals are analyzed to demonstrate the performance of the proposed solution.


The Reproducibility of Programming Related Issues in Stack Overflow Questions. Software developers often look for solutions to their code level problems using the Stack Overflow Q A website. To receive help, developers frequently submit questions containing sample code segments and the description of the programming issue. Unfortunately, it is not always possible to reproduce the issues from the code segments that may impede questions from receiving prompt and appropriate solutions. We conducted an exploratory study on the reproducibility of issues discussed in 400 Java and 400 Python questions. We parsed, compiled, executed, and carefully examined the code segments from these questions to reproduce the reported programming issues. The outcomes of our study are three fold. First, we found that we can reproduce approximately 68 of Java and 71 of Python issues, whereas we were unable to reproduce approximately 22 of Java and 19 of Python issues using the code segments. Of the issues that were reproducible, approximately 67 of the Java code segments and 20 of the Python code segments required minor or major modifications to reproduce the issues. Second, we carefully investigated why programming issues could not be reproduced and provided evidence based guidelines for writing effective code examples for Stack Overflow questions. Third, we investigated the correlation between the issue reproducibility status of questions and the corresponding answer meta data, such as the presence of an accepted answer. According to our analysis, a reproducible question has at least two times higher chance of receiving an accepted answer than an irreproducible question. Besides, the median time delay in receiving accepted answers is double if the issues reported in questions could not be reproduced. We also investigate the confounding factors (e.g., reputation) and find that confounding factors do not hurt the correlation between reproducibility status and answer meta data.


Denoised Labels for Financial Time Series Data via Self Supervised Learning. The introduction of electronic trading platforms effectively changed the organisation of traditional systemic trading from quote driven markets into order driven markets. Its convenience led to an exponentially increasing amount of financial data, which is however hard to use for the prediction of future prices, due to the low signal to noise ratio and the non stationarity of financial time series. Simpler classification tasks where the goal is to predict the directions of future price movement via supervised learning algorithms, need sufficiently reliable labels to generalise well. Labelling financial data is however less well defined than other domains did the price go up because of noise or because of signal? The existing labelling methods have limited countermeasures against noise and limited effects in improving learning algorithms. This work takes inspiration from image classification in trading and success in self supervised learning. We investigate the idea of applying computer vision techniques to financial time series to reduce the noise exposure and hence generate correct labels. We look at the label generation as the pretext task of a self supervised learning approach and compare the naive (and noisy) labels, commonly used in the literature, with the labels generated by a denoising autoencoder for the same downstream classification task. Our results show that our denoised labels improve the performances of the downstream learning algorithm, for both small and large datasets. We further show that the signals we obtain can be used to effectively trade with binary strategies. We suggest that with proposed techniques, self supervised learning constitutes a powerful framework for generating "better" financial labels that are useful for studying the underlying patterns of the market.


Changing the Narrative Perspective From Deictic to Anaphoric Point of View. We introduce the task of changing the narrative point of view, where characters are assigned a narrative perspective that is different from the one originally used by the writer. The resulting shift in the narrative point of view alters the reading experience and can be used as a tool in fiction writing or to generate types of text ranging from educational to self help and self diagnosis. We introduce a benchmark dataset containing a wide range of types of narratives annotated with changes in point of view from deictic (first or second person) to anaphoric (third person) and describe a pipeline for processing raw text that relies on a neural architecture for mention selection. Evaluations on the new benchmark dataset show that the proposed architecture substantially outperforms the baselines by generating mentions that are less ambiguous and more natural.


MM Pyramid Multimodal Pyramid Attentional Network for Audio Visual Event Localization and Video Parsing. Recognizing and localizing events in videos is a fundamental task for video understanding. Since events may occur in auditory and visual modalities, multimodal detailed perception is essential for complete scene comprehension. Most previous works attempted to analyze videos from a holistic perspective. However, they do not consider semantic information at multiple scales, which makes the model difficult to localize events in various lengths. In this paper, we present a Multimodal Pyramid Attentional Network (MM Pyramid) that captures and integrates multi level temporal features for audio visual event localization and audio visual video parsing. Specifically, we first propose the attentive feature pyramid module. This module captures temporal pyramid features via several stacking pyramid units, each of them is composed of a fixed size attention block and dilated convolution block. We also design an adaptive semantic fusion module, which leverages a unit level attention block and a selective fusion block to integrate pyramid features interactively. Extensive experiments on audio visual event localization and weakly supervised audio visual video parsing tasks verify the effectiveness of our approach.


An Empirical Analysis of VM Startup Times in Public IaaS Clouds An Extended Report. VM startup time is an essential factor in designing elastic cloud applications. For example, a cloud application with autoscaling can reduce under and over provisioning of VM instances with a precise estimation of VM startup time, and in turn, it is likely to guarantee the application's performance and improve the cost efficiency. However, VM startup time has been little studied, and available measurement results performed previously did not consider various configurations of VMs for modern cloud applications. In this work, we perform comprehensive measurements and analysis of VM startup time from two major cloud providers, namely Amazon Web Services (AWS) and Google Cloud Platform (GCP). With three months of measurements, we collected more than 300,000 data points from each provider by applying a set of configurations, including 11 VM types, four different data center locations, four VM image sizes, two OS types, and two purchase models (e.g., spot preemptible VMs vs. on demand VMs). With extensive analysis, we found that VM startup time can vary significantly because of several important factors, such as VM image sizes, data center locations, VM types, and OS types. Moreover, by comparing with previous measurement results, we confirm that cloud providers (specifically AWS) made significant improvements for the VM startup times and currently have much quicker VM startup times than in the past.


A stochastic model for the hydrodynamic force in Euler Lagrange simulations of particle laden flows. Standard Eulerian Lagrangian (EL) methods generally employ drag force models that only represent the mean hydrodynamic force acting upon a particle laden suspension. Consequently, higher order drag force statistics, arising from neighbor induced flow perturbations, are not accounted for with implications on predictions for particle velocity variance and dispersion. We develop a force Langevin (FL) model that treats neighbor induced drag fluctuations as a stochastic force within an EL framework. The stochastic drag force follows an Ornstein Uhlenbeck process and requires closure of the integral time scale for the fluctuating hydrodynamic force and the standard deviation in drag. The former is closed using the mean free time between successive collisions, derived from the kinetic theory of non uniform gases. For the latter, particle resolved direct numerical simulation (PR DNS) of fixed particle assemblies is utilized to develop a correlation. The stochastic EL framework specifies unresolved drag force statistics, leading to the correct evolution and sustainment of particle velocity variance over a wide range of Reynolds numbers and solids volume fractions when compared to PR DNS of freely evolving homogeneous suspensions. By contrast, standard EL infers drag statistics from variations in the resolved flow and thus under predicts the growth and steady particle velocity variance in homogeneous suspensions. Velocity statistics from standard EL approaches are found to depend on the bandwidth of the projection function used for two way momentum coupling, while results obtained from the stochastic EL approach are insensitive to the projection bandwidth.


A decomposition theorem for 0 cycles and applications to class field theory. We describe the abelian 'e tale fundamental group with modulus in terms of 0 cycles on a class of smooth quasi projective varieties over finite fields which may not admit smooth compactifications. Using a limit process, this yields a cycle theoretic analogue of the K theoretic reciprocity theorem of Kato Saito for such varieties. The proof is based on a decomposition theorem for the cohomological Chow group of 0 cycles on the double of a quasi projective R 1 scheme over a field along a closed subscheme in terms of the Chow groups with and without modulus of the scheme. This also extends the decomposition theorem of Binda Krishna to smooth schemes over imperfect fields.


Solving incompressible Navier Stokes equations on irregular domains and quadtrees by monolithic approach. We present a second order monolithic method for solving incompressible Navier Stokes equations on irregular domains with quadtree grids. A semi collocated grid layout is adopted, where velocity variables are located at cell vertices, and pressure variables are located at cell centers. Compact finite difference methods with ghost values are used to discretize the advection and diffusion terms of the velocity. A pressure gradient and divergence operator on the quadtree that use compact stencils are developed. Furthermore, the proposed method is extended to cubical domains with octree grids. Numerical results demonstrate that the method is second order convergent in L infty norms and can handle irregular domains for various Reynolds numbers.


Slopes of modular forms and reducible Galois representations an oversight in the ghost conjecture. The ghost conjecture, formulated by this article's authors, predicts the list of p adic valuations of the non zero p th eigenvalues ("slopes") for overconvergent p adic modular eigenforms in terms of the Newton polygon of an easy to describe power series (the "ghost series"). The prediction is restricted to eigenforms whose Galois representation modulo p is reducible on a decomposition group at p. It has been discovered, however, that the conjecture is not formulated correctly. Here we explain the issue and propose a salvage.


Comments on "A Framework for Control System Design Subject to Average Data Rate Constraints". Theorem 4.1 in the 2011 paper "A Framework for Control System Design Subject to Average Data Rate Constraints" allows one to lower bound average operational data rates in feedback loops (including the situation in which encoder and decoder have side information). Unfortunately, its proof is invalid. In this note we first state the theorem and explain why its proof is flawed, and then provide a correct proof under weaker assumptions.


Data Generation for Learning to Grasp in a Bin picking Scenario. The rise of deep learning has greatly transformed the pipeline of robotic grasping from model based approach to data driven stream. Along this line, a large scale of grasping data either collected from simulation or from real world examples become extremely important. In this paper, we present our recent work on data generation in simulation for a bin picking scene. 77 objects from the YCB object data sets are used to generate the dataset with PyBullet, where different environment conditions are taken into account including lighting, camera pose, sensor noise and so on. In all, 100K data samples are collected in terms of ground truth segmentation, RGB, 6D pose and point cloud. All the data examples including the source code are made available online.


Global Convergence and Stability of Stochastic Gradient Descent. In machine learning, stochastic gradient descent (SGD) is widely deployed to train models using highly non convex objectives with equally complex noise models. Unfortunately, SGD theory often makes restrictive assumptions that fail to capture the non convexity of real problems, and almost entirely ignore the complex noise models that exist in practice. In this work, we make substantial progress on this shortcoming. First, we establish that SGD's iterates will either globally converge to a stationary point or diverge under nearly arbitrary nonconvexity and noise models. Under a slightly more restrictive assumption on the joint behavior of the non convexity and noise model that generalizes current assumptions in the literature, we show that the objective function cannot diverge, even if the iterates diverge. As a consequence of our results, SGD can be applied to a greater range of stochastic optimization problems with confidence about its global convergence behavior and stability.