Unified approach for solving exit problems for additive increase and multiplicative decrease processes. We analyse an additive increase and multiplicative decrease (aka growth collapse) process that grows linearly in time and that experiences downward jumps at Poisson epochs that are (deterministically) proportional to its present position. This process is used for example in modelling of Transmission Control Protocol (TCP) and can be viewed as a particular example of the so called shot noise model, a basic tool in modeling earthquakes, avalanches and neuron firings. For this process, and also for its reflected versions, we consider one and two sided exit problems that concern the identification of the laws of exit times from fixed intervals and half lines. All proofs are based on a unified first step analysis approach at the first jump epoch, which allows us to give explicit, yet involved, formulas for their Laplace transforms. All the eight Laplace transforms can be described in terms of two so called scale functions Z uparrow and L uparrow . Here Z uparrow is described in terms of multiple explicit sums, and L uparrow in terms of an explicit recursion formula. All other Laplace transforms can be obtained from Z uparrow and L uparrow by taking limits, derivatives, integrals and combinations of these.


Technical Report for A Joint User Scheduling and Trajectory Planning Data Collection Strategy for the UAV assisted WSN. Unmanned aerial vehicles (UAVs) are usually dispatched as mobile sinks to assist data collection in large scale wireless sensor networks (WSNs). However, when considering the limitations of UAV's mobility and communication capabilities in a large scale WSN, some sensor nodes may run out of storage space as they fail to offload their data to the UAV for an extended period of time. To minimize the data loss caused by the above issue, a joint user scheduling and trajectory planning data collection strategy is proposed in this letter, which is formulated as a non convex optimization problem. The problem is further divided into two sub problems and solved sequentially. Simulation results show that the proposed strategy is more effective in minimizing data loss rate than other strategies.


A Product Space Reformulation with Reduced Dimension for Splitting Algorithms. In this paper we propose a product space reformulation to transform monotone inclusions described by finitely many operators on a Hilbert space into equivalent two operator problems. Our approach relies on Pierra's classical reformulation with a different decomposition, which results on a reduction of the dimension of the outcoming product Hilbert space. We discuss the case of not necessarily convex feasibility and best approximation problems. By applying existing splitting methods to the proposed reformulation we obtain new parallel variants of them with a reduction in the number of variables. The convergence of the new algorithms is straightforwardly derived with no further assumptions. The computational advantage is illustrated through some numerical experiments.


A comparison of LSTM and GRU networks for learning symbolic sequences. We explore relations between the hyper parameters of a recurrent neural network (RNN) and the complexity of string sequences it is able to memorize. We compare long short term memory (LSTM) networks and gated recurrent units (GRUs). We find that an increase of RNN depth does not necessarily result in better memorization capability when the training time is constrained. Our results also indicate that the learning rate and the number of units per layer are among the most important hyper parameters to be tuned. Generally, GRUs outperform LSTM networks on low complexity sequences while on high complexity sequences LSTMs perform better.


CoreDiag Eliminating Redundancy in Constraint Sets. Constraint based environments such as configuration systems, recommender systems, and scheduling systems support users in different decision making scenarios. These environments exploit a knowledge base for determining solutions of interest for the user. The development and maintenance of such knowledge bases is an extremely time consuming and error prone task. Users often specify constraints which do not reflect the real world. For example, redundant constraints are specified which often increase both, the effort for calculating a solution and efforts related to knowledge base development and maintenance. In this paper we present a new algorithm (CoreDiag) which can be exploited for the determination of minimal cores (minimal non redundant constraint sets). The algorithm is especially useful for distributed knowledge engineering scenarios where the degree of redundancy can become high. In order to show the applicability of our approach, we present an empirical study conducted with commercial configuration knowledge bases.


Reduced etch lag and high aspect ratios by deep reactive ion etching (DRIE). Deep reactive ion etching (DRIE) with the Bosch process is one of the key procedures used to manufacture micron sized structures for MEMS and microfluidic applications in silicon and, hence, of increasing importance for miniaturization in biomedical research. While guaranteeing high aspect ratio structures and providing high design flexibility, the etching procedure suffers from reactive ion etching lag and often relies on complex oxide masks to enable deep etching. In this work, we introduce an optimized Bosch process that reduces the etch lag to below 1.5 . Furthermore, we improved a three step Bosch process, allowing the fabrication of structures with 6 micrometre thickness at depths up to 180 micrometre while maintaining their stability.


Local Patch AutoAugment with Multi Agent Collaboration. Data augmentation (DA) plays a critical role in improving the generalization of deep learning models. Recent works on automatically searching for DA policies from data have achieved great success. However, existing automated DA methods generally perform the search at the image level, which limits the exploration of diversity in local regions. In this paper, we propose a more fine grained automated DA approach, dubbed Patch AutoAugment, to divide an image into a grid of patches and search for the joint optimal augmentation policies for the patches. We formulate it as a multi agent reinforcement learning (MARL) problem, where each agent learns an augmentation policy for each patch based on its content together with the semantics of the whole image. The agents cooperate with each other to achieve the optimal augmentation effect of the entire image by sharing a team reward. We show the effectiveness of our method on multiple benchmark datasets of image classification and fine grained image recognition (e.g., CIFAR 10, CIFAR 100, ImageNet, CUB 200 2011, Stanford Cars and FGVC Aircraft). Extensive experiments demonstrate that our method outperforms the state of the art DA methods while requiring fewer computational resources.


The Power of D hops in Matching Power Law Graphs. This paper studies seeded graph matching for power law graphs. Assume that two edge correlated graphs are independently edge sampled from a common parent graph with a power law degree distribution. A set of correctly matched vertex pairs is chosen at random and revealed as initial seeds. Our goal is to use the seeds to recover the remaining latent vertex correspondence between the two graphs. Departing from the existing approaches that focus on the use of high degree seeds in 1 hop neighborhoods, we develop an efficient algorithm that exploits the low degree seeds in suitably defined D hop neighborhoods. Specifically, we first match a set of vertex pairs with appropriate degrees (which we refer to as the first slice) based on the number of low degree seeds in their D hop neighborhoods. This significantly reduces the number of initial seeds needed to trigger a cascading process to match the rest of the graphs. Under the Chung Lu random graph model with n vertices, max degree Theta( sqrt n ) , and the power law exponent 2 beta 3 , we show that as soon as D frac 4 beta 3 beta , by optimally choosing the first slice, with high probability our algorithm can correctly match a constant fraction of the true pairs without any error, provided with only Omega(( log n) 4 beta ) initial seeds. Our result achieves an exponential reduction in the seed size requirement, as the best previously known result requires n 1 2 epsilon seeds (for any small constant epsilon 0 ). Performance evaluation with synthetic and real data further corroborates the improved performance of our algorithm.


Unsupervised Data Selection for Data Centric Semi Supervised Learning. We study unsupervised data selection for semi supervised learning (SSL), where a large scale unlabeled dataset is available and a small subset of data is budgeted for label acquisition. Existing SSL methods focus on learning a model that effectively integrates information from given small labeled data and large unlabeled data, whereas we focus on selecting the right data to annotate for SSL without requiring any label or task information. Intuitively, instances to be labeled shall collectively have maximum diversity and coverage for downstream tasks, and individually have maximum information propagation utility for SSL. We formalize these concepts in a three step data centric SSL method that improves FixMatch in stability and accuracy by 8 on CIFAR 10 (0.08 labeled) and 14 on ImageNet 1K (0.2 labeled). It is also a universal framework that works with various SSL methods, delivering consistent performance gains. Our work demonstrates that small computation spent on carefully selecting data for annotation brings big annotation efficiency and model performance gain without changing the learning pipeline. Our completely unsupervised data selection can be easily extended to other weakly supervised learning settings.


Learning on a Budget via Teacher Imitation. Deep Reinforcement Learning (RL) techniques can benefit greatly from leveraging prior experience, which can be either self generated or acquired from other entities. Action advising is a framework that provides a flexible way to transfer such knowledge in the form of actions between teacher student peers. However, due to the realistic concerns, the number of these interactions is limited with a budget therefore, it is crucial to perform these in the most appropriate moments. There have been several promising studies recently that address this problem setting especially from the student's perspective. Despite their success, they have some shortcomings when it comes to the practical applicability and integrity as an overall solution to the learning from advice challenge. In this paper, we extend the idea of advice reusing via teacher imitation to construct a unified approach that addresses both advice collection and advice utilisation problems. We also propose a method to automatically tune the relevant hyperparameters of these components on the fly to make it able to adapt to any task with minimal human intervention. The experiments we performed in 5 different Atari games verify that our algorithm either surpasses or performs on par with its top competitors while being far simpler to be employed. Furthermore, its individual components are also found to be providing significant advantages alone.


Make Sure You're Unsure A Framework for Verifying Probabilistic Specifications. Most real world applications require dealing with stochasticity like sensor noise or predictive uncertainty, where formal specifications of desired behavior are inherently probabilistic. Despite the promise of formal verification in ensuring the reliability of neural networks, progress in the direction of probabilistic specifications has been limited. In this direction, we first introduce a general formulation of probabilistic specifications for neural networks, which captures both probabilistic networks (e.g., Bayesian neural networks, MC Dropout networks) and uncertain inputs (distributions over inputs arising from sensor noise or other perturbations). We then propose a general technique to verify such specifications by generalizing the notion of Lagrangian duality, replacing standard Lagrangian multipliers with "functional multipliers" that can be arbitrary functions of the activations at a given layer. We show that an optimal choice of functional multipliers leads to exact verification (i.e., sound and complete verification), and for specific forms of multipliers, we develop tractable practical verification algorithms. We empirically validate our algorithms by applying them to Bayesian Neural Networks (BNNs) and MC Dropout Networks, and certifying properties such as adversarial robustness and robust detection of out of distribution (OOD) data. On these tasks we are able to provide significantly stronger guarantees when compared to prior work for instance, for a VGG 64 MC Dropout CNN trained on CIFAR 10, we improve the certified AUC (a verified lower bound on the true AUC) for robust OOD detection (on CIFAR 100) from 0 rightarrow 29 . Similarly, for a BNN trained on MNIST, we improve on the robust accuracy from 60.2 rightarrow 74.6 . Further, on a novel specification distributionally robust OOD detection we improve the certified AUC from 5 rightarrow 23 .


Learning to Stylize Novel Views. We tackle a 3D scene stylization problem generating stylized images of a scene from arbitrary novel views given a set of images of the same scene and a reference image of the desired style as inputs. Direct solution of combining novel view synthesis and stylization approaches lead to results that are blurry or not consistent across different views. We propose a point cloud based method for consistent 3D scene stylization. First, we construct the point cloud by back projecting the image features to the 3D space. Second, we develop point cloud aggregation modules to gather the style information of the 3D scene, and then modulate the features in the point cloud with a linear transformation matrix. Finally, we project the transformed features to 2D space to obtain the novel views. Experimental results on two diverse datasets of real world scenes validate that our method generates consistent stylized novel view synthesis results against other alternative approaches.


Sections and projections of nested convex bodies. One of the most important problems in Geometric Tomography is to establish properties of a given convex body if we know some properties over its sections or its projections. There are many interesting and deep results that provide characterizations of the sphere and the ellipsoid in terms of the properties of its sections or projections. Another kind of characterizations of the ellipsoid is when we consider properties of the support cones. However, in almost all the known characterizations, we have only a convex body and the sections, projections, or support cones, are considered for this given body. In this article we proved some results that characterizes the Euclidean ball or the ellipsoid when the sections or projections are taken for a pair of nested convex bodies, i.e., two convex bodies K , L such that L subset text int , K. We impose some relations between the corresponding sections or projections and some apparently new characterizations of the ball or the ellipsoid appear. We also deal with properties of support cones or point source shadow boundaries when the apexes are taken in the boundary of K .


Spaces where all closed sets are alpha limit sets. Metrizable spaces are studied in which every closed set is an alpha limit set for some continuous map and some point. It is shown that this property is enjoyed by every space containing sufficiently many arcs (formalized in the notion of a space with enough arcs), though such a space need not be arcwise connected. Further it is shown that this property is not preserved by topological sums, products and continuous images and quotients. However, positive results do hold for metrizable spaces obtained by those constructions from spaces with enough arcs.


Firing statistics of a neuron with delayed feedback inhibition stimulated with a renewal process. In this paper, we study the impact of an inhibitory autapse on neuronal activity. In order to do this, we consider a class of spiking neuron models with delayed feedback inhibition stimulated with a series of excitatory impulses, representing a stochastic point renewal process. We calculate exactly the probability density function (PDF) p(t) for the distribution of output interspike intervals (ISIs). The calculation is based on the known PDF of ISIs p 0(t) for the same neuron without feedback and the PDF of ISIs for the input stream p in (t) . Obtained results are applied to the case of a neuron with threshold 2 when the time intervals between input impulses are distributed according to the Erlang 2 distribution. Further, for the binding neuron model with threshold 2 with delayed feedback inhibition stimulated with the Erlang 2 stream of excitatory impulses, the first two moments of the ISI PDF are computed. Our results indicate that depending on the time delay of the feedback inhibition, the spike regularity can lower or rise in comparison with the case of the neuron without delayed feedback inhibition.


Cascaded Head colliding Attention. Transformers have advanced the field of natural language processing (NLP) on a variety of important tasks. At the cornerstone of the Transformer architecture is the multi head attention (MHA) mechanism which models pairwise interactions between the elements of the sequence. Despite its massive success, the current framework ignores interactions among different heads, leading to the problem that many of the heads are redundant in practice, which greatly wastes the capacity of the model. To improve parameter efficiency, we re formulate the MHA as a latent variable model from a probabilistic perspective. We present cascaded head colliding attention (CODA) which explicitly models the interactions between attention heads through a hierarchical variational distribution. We conduct extensive experiments and demonstrate that CODA outperforms the transformer baseline, by 0.6 perplexity on texttt Wikitext 103 in language modeling, and by 0.6 BLEU on texttt WMT14 EN DE in machine translation, due to its improvements on the parameter efficiency. footnote Our implementation is publicly available at url https github.com LZhengisme CODA .


The probability of spanning a classical space by two non degenerate subspaces of complementary dimension. Let n,n' be positive integers and let V be an (n n') dimensional vector space over a finite field mathbb F equipped with a non degenerate alternating, hermitian or quadratic form. We estimate the proportion of pairs (U, U') , where U is a non degenerate n subspace and U' is a non degenerate n' subspace of V , such that U U' V (usually such spaces U and U' are not perpendicular). The proportion is shown to be at least 1 c mathbb F for some constant c leqslant 2 in the symplectic or unitary cases, and c 3 in the orthogonal case.


Partitioning the real line into Borel sets. For which infinite cardinals kappa is there a partition of the real line mathbb R into precisely kappa Borel sets? Hausdorff famously proved that there is a partition of mathbb R into aleph 1 Borel sets. But other than this, we show that the spectrum of possible sizes of partitions of mathbb R into Borel sets can be fairly arbitrary. For example, given any A subseteq omega with 0,1 in A , there is a forcing extension in which A n , text there is a partition of mathbb R text into aleph n text Borel sets . We also look at the corresponding question for partitions of mathbb R into closed sets. We show that, like with partitions into Borel sets, the set of all uncountable kappa such that there is a partition of mathbb R into precisely kappa closed sets can be fairly arbitrary.


Exploring Multi Dimensional Events Characterizing Tech Start Up Emergence in the Nigerian Entrepreneurial Ecosystem. Most countries across the globe identify technology based start ups as a driving force for job creation, economic growth and national development, and a critical tool for economic sustenance during pandemic crises like covid 19. However, its emergence are been argued to be problematic. Especially in a developing economy like Nigeria, where tech start up founders are faced with diverse form of constraints and environmental uncertainties. Extant literature indicated that studies are been conducted to explain tech start up emergence. However, such studies are fragmented with findings that are determinants to tech start up emergence, with several determinants studied in isolation, and the emergence as linear and unidimensional events. Consequently, neglecting multi dimensional perspective, which aggregate the dimensions of events characterizing tech start up emergence. Given the iterative, event based process, and interactive dependent nature of tech start up ventures to create activity based products services in an open, uncertain, nonlinear and dynamic environment, we argue that little are been known about tech start up emergence. Thus, by drawing from synthesize literature review, activity theory, and exploratory case study design we identify opportunity discovery and selection team formation and domain consensus bootstrapping minimum viable product development and market experimentation feedback as interdependent multi dimensional events constituting tech start up emergence in Nigerian tech start up ecosystem.


Minimax Adaptive Estimation for Finite Sets of Linear Systems. For linear time invariant systems with uncertain parameters belonging to a finite set, we present a purely deterministic approach to multiple model estimation and propose an algorithm based on the minimax criterion using constrained quadratic programming. The estimator tends to learn the dynamics of the system, and once the uncertain parameters have been sufficiently estimated, the estimator behaves like a standard Kalman filter.


Score sequences of bitournaments. The score of a vertex x in an oriented graph is defined to be its outdegree, emph i.e. , the number of arcs with initial vertex x . The score sequence of an oriented graph is the sequence of all scores arranged in nondecreasing order. An oriented complete bipartite graph is called a bitournament. The score sequence of a bitournament consists of two nondecreasing sequences of nonnegative integers, one for each of the two partite sets. Moon has characterized the score sequences of bitournaments. This paper introduces the concept of trimming a sequence and gives a characterization of score sequences of bitournaments utilizing this concept.


Using Non Verbal Expressions as a Tool in Naming Research. Variable and function names are extremely important for program comprehension. It is therefore also important to study how developers select names. But controlled experiments on naming are hindered by the need to describe to experimental subjects what it is they need to name. Words appearing in these descriptions may then find their way into the names, leading to a bias in the results. We suggest that this problem can be alleviated by using emojis or other small graphics in lieu of key words in the descriptions. A replication of previous work on naming, this time including such emojis and graphics, indeed led to a more diverse and less biased choice of words in the names than when using English descriptions.


Characterizations of recurrence and Poisson stability of flows on surfaces. Long time behaviors of orbits are one of the most fundamental properties in dynamical systems. Poincar 'e studied the Poisson stability, to capture a property whether points return arbitrarily near the initial positions after a sufficiently long time. Birkhoff introduced and studied the concept of recurrent points. We show that the recurrence and Poisson stability of flows on surfaces are topological properties of the orbit spaces. In fact, a flow on a compact connected surface is Poisson stable (resp. recurrent) if and only if the Kolmogorov quotient of the orbit space satisfies T 1 (resp. T 1 2 ) separation axiom. Using such characterizations, we characterize Hausdorff separation axiom for orbit spaces and their Kolmogorov quotients of flows on compact connected surfaces.


Combinatorics of NC Probability Spaces with Independent Constants. Unlike classical and free independence, the boolean and monotone notions of independence lack of the property of independent constants. In the scalar case, this leads to restrictions for the central limit theorems, as observed by F. Oravecz. We characterize the property of independent constants from a combinatorial point of view, based on cumulants and set partitions. This characterization also holds for the operator valued extension. Our considerations lead rather directly to very mild variations of boolean and monotone cumulants, where constants are now independent. These alternative probability theories are closely related to the usual notions. Hence, an important part of the boolean monotone probability theories can be imported directly. We describe some standard combinatorial aspects of these variations (and their cyclic versions), such as their M obius functions, which feature well known combinatorial integer sequences. The new notions with independent constants seem also more strongly related to the operator valued extension of c free probability.


Numerical simulation of baffled circulating fluidized bed with Geldart B particles. As most classical fluidization technologies, Circulating Fluidized Beds (CFB) are widely employed in chemical and physical operations. However, CFB Reactors still have some disadvantages such as the non uniform solid particles distribution and the backflow of the solid particles near the wall, which deteriorate the system mixing, and lead to low system conversion heat and mass transfer rate. To improve the CFB performance, we propose in the current work to study the effect of the ring baffle configuration on the fluidization system hydrodynamics using a multifluid Eulerian CFD model, incorporating the Kinetic Theory of Granular Flow. In this approach, two different types of ring baffles, circular and trapezoidal, were considered and the corresponding results were compared. The results revealed that the incorporation of ring baffles improved the system mixing and reduced the backflow near the wall. Nevertheless, the shape of baffles had a limited impact on the system hydrodynamics.


Topological semantics for paraconsistent and paracomplete logics in Isabelle HOL. We encode a topological semantics for paraconsistent and paracomplete logics enriched with recovery operators, by drawing upon early works on topological Boolean algebras (by Kuratowski, Zarycki, McKinsey Tarski, etc.). This work exemplarily illustrates the shallow semantical embedding approach using Isabelle HOL and shows how we can effectively harness theorem provers, model finders and "hammers" for reasoning with quantified non classical logics.


Property Inference Attacks on Convolutional Neural Networks Influence and Implications of Target Model's Complexity. Machine learning models' goal is to make correct predictions for specific tasks by learning important properties and patterns from data. By doing so, there is a chance that the model learns properties that are unrelated to its primary task. Property Inference Attacks exploit this and aim to infer from a given model ( ie the target model) properties about the training dataset seemingly unrelated to the model's primary goal. If the training data is sensitive, such an attack could lead to privacy leakage. This paper investigates the influence of the target model's complexity on the accuracy of this type of attack, focusing on convolutional neural network classifiers. We perform attacks on models that are trained on facial images to predict whether someone's mouth is open. Our attacks' goal is to infer whether the training dataset is balanced gender wise. Our findings reveal that the risk of a privacy breach is present independently of the target model's complexity for all studied architectures, the attack's accuracy is clearly over the baseline. We discuss the implication of the property inference on personal data in the light of Data Protection Regulations and Guidelines.


Evaluating Federated Learning for Intrusion Detection in Internet of Things Review and Challenges. The application of Machine Learning (ML) techniques to the well known intrusion detection systems (IDS) is key to cope with increasingly sophisticated cybersecurity attacks through an effective and efficient detection process. In the context of the Internet of Things (IoT), most ML enabled IDS approaches use centralized approaches where IoT devices share their data with data centers for further analysis. To mitigate privacy concerns associated with centralized approaches, in recent years the use of Federated Learning (FL) has attracted a significant interest in different sectors, including healthcare and transport systems. However, the development of FL enabled IDS for IoT is in its infancy, and still requires research efforts from various areas, in order to identify the main challenges for the deployment in real world scenarios. In this direction, our work evaluates a FL enabled IDS approach based on a multiclass classifier considering different data distributions for the detection of different attacks in an IoT scenario. In particular, we use three different settings that are obtained by partitioning the recent ToN IoT dataset according to IoT devices' IP address and types of attack. Furthermore, we evaluate the impact of different aggregation functions according to such setting by using the recent IBMFL framework as FL implementation. Additionally, we identify a set of challenges and future directions based on the existing literature and the analysis of our evaluation results.


Homogenization of Boundary Value Problems in Perforated Lipschitz Domains. This paper is concerned with boundary regularity estimates in the homogenization of elliptic equations with rapidly oscillating and high contrast coefficients. We establish uniform nontangential maximal function estimates for the Dirichlet, regularity, and Neumann problems with L 2 boundary data in a periodically perforated Lipschitz domain.


Frontrunner Jones and the Raiders of the Dark Forest An Empirical Study of Frontrunning on the Ethereum Blockchain. Ethereum prospered the inception of a plethora of smart contract applications, ranging from gambling games to decentralized finance. However, Ethereum is also considered a highly adversarial environment, where vulnerable smart contracts will eventually be exploited. Recently, Ethereum's pool of pending transaction has become a far more aggressive environment. In the hope of making some profit, attackers continuously monitor the transaction pool and try to frontrun their victims' transactions by either displacing or suppressing them, or strategically inserting their transactions. This paper aims to shed some light into what is known as a dark forest and uncover these predators' actions. We present a methodology to efficiently measure the three types of frontrunning displacement, insertion, and suppression. We perform a large scale analysis on more than 11M blocks and identify almost 200K attacks with an accumulated profit of 18.41M USD for the attackers, providing evidence that frontrunning is both, lucrative and a prevalent issue.


Uncertainty Estimation in SARS CoV 2 B cell Epitope Prediction for Vaccine Development. B cell epitopes play a key role in stimulating B cells, triggering the primary immune response which results in antibody production as well as the establishment of long term immunity in the form of memory cells. Consequently, being able to accurately predict appropriate linear B cell epitope regions would pave the way for the development of new protein based vaccines. Knowing how much confidence there is in a prediction is also essential for gaining clinicians' trust in the technology. In this article, we propose a calibrated uncertainty estimation in deep learning to approximate variational Bayesian inference using MC DropWeights to predict epitope regions using the data from the immune epitope database. Having applied this onto SARS CoV 2, it can more reliably predict B cell epitopes than standard methods. This will be able to identify safe and effective vaccine candidates against Covid 19.


Practical Resources for Enhancing the Reproducibility of Mechanistic Modeling in Systems Biology. Although reproducibility is a core tenet of the scientific method, it remains challenging to reproduce many results. Surprisingly, this also holds true for computational results in domains such as systems biology where there have been extensive standardization efforts. For example, Tiwari et al. recently found that they could only repeat 50 of published simulation results in systems biology. Toward improving the reproducibility of computational systems research, we identified several resources that investigators can leverage to make their research more accessible, executable, and comprehensible by others. In particular, we identified several domain standards and curation services, as well as powerful approaches pioneered by the software engineering industry that we believe many investigators could adopt. Together, we believe these approaches could substantially enhance the reproducibility of systems biology research. In turn, we believe enhanced reproducibility would accelerate the development of more sophisticated models that could inform precision medicine and synthetic biology.


Tracing Text Provenance via Context Aware Lexical Substitution. Text content created by humans or language models is often stolen or misused by adversaries. Tracing text provenance can help claim the ownership of text content or identify the malicious users who distribute misleading content like machine generated fake news. There have been some attempts to achieve this, mainly based on watermarking techniques. Specifically, traditional text watermarking methods embed watermarks by slightly altering text format like line spacing and font, which, however, are fragile to cross media transmissions like OCR. Considering this, natural language watermarking methods represent watermarks by replacing words in original sentences with synonyms from handcrafted lexical resources (e.g., WordNet), but they do not consider the substitution's impact on the overall sentence's meaning. Recently, a transformer based network was proposed to embed watermarks by modifying the unobtrusive words (e.g., function words), which also impair the sentence's logical and semantic coherence. Besides, one well trained network fails on other different types of text content. To address the limitations mentioned above, we propose a natural language watermarking scheme based on context aware lexical substitution (LS). Specifically, we employ BERT to suggest LS candidates by inferring the semantic relatedness between the candidates and the original sentence. Based on this, a selection strategy in terms of synchronicity and substitutability is further designed to test whether a word is exactly suitable for carrying the watermark signal. Extensive experiments demonstrate that, under both objective and subjective metrics, our watermarking scheme can well preserve the semantic integrity of original sentences and has a better transferability than existing methods. Besides, the proposed LS approach outperforms the state of the art approach on the Stanford Word Substitution Benchmark.


Invertible and isometric weighted composition operators. We consider abstract Banach spaces of analytic functions on general bounded domains that satisfy only a minimum number of axioms. We describe all invertible (equivalently, surjective) weighted composition operators acting on such spaces. For the spaces of analytic functions in the disk whose norm is given in terms of a natural seminorm (such as the typical spaces given in terms of derivatives), we describe all surjective weighted composition operators that are isometric. This generalizes a number of known results.


Balanced Weighing Matrices. A unified approach to the construction of weighing matrices and certain symmetric designs is presented. Assuming the weight p in a weighing matrix W(n,p) is a prime power, it is shown that there is a W left( frac p m 1 1 p 1 (n 1) 1,p m 1 right) for each positive integer m . The case of n p 1 reduces to the balanced weighing matrices with classical parameters W left( frac p m 2 1 p 1 ,p m 1 right). The equivalence with certain classes of association schemes is discussed in details.


Method of Alternating Projection for the Absolute Value Equation. A novel approach for solving the general absolute value equation Ax B x c where A,B in mathbb R m times n and c in mathbb R m is presented. We reformulate the equation as a feasibility problem which we solve via the method of alternating projections (MAP). The fixed points set of the alternating projections map is characterized under nondegeneracy conditions on A and B . Furthermore, we prove linear convergence of the algorithm. Unlike most of the existing approaches in the literature, the algorithm presented here is capable of handling problems with m neq n , both theoretically and numerically.


Very High Dynamic Range, 10,000 frames second Pixel Array Detector for Electron Microscopy. Precision and accuracy of quantitative scanning transmission electron microscopy (STEM) methods such as ptychography, and the mapping of electric, magnetic and strain fields depend on the dose. Reasonable acquisition time requires high beam current and the ability to quantitatively detect both large and minute changes in signal. A new hybrid pixel array detector (PAD), the second generation Electron Microscope Pixel Array Detector (EMPAD G2), addresses this challenge by advancing the technology of a previous generation PAD, the EMPAD. The EMPAD G2 images continuously at a frame rates up to 10 kHz with a dynamic range that spans from low noise detection of single electrons to electron beam currents exceeding 180 pA per pixel, even at electron energies of 300 keV. The EMPAD G2 enables rapid collection of high quality STEM data that simultaneously contain full diffraction information from unsaturated bright field disks to usable Kikuchi bands and higher order Laue zones. Test results from 80 to 300 keV are presented, as are first experimental results demonstrating ptychographic reconstructions, strain and polarization maps. We introduce a new information metric, the Maximum Usable Imaging Speed (MUIS), to identify when a detector becomes electron starved, saturated or its pixel count is mismatched with the beam current.


Socially Aware Multi Agent Following with 2D Laser Scans via Deep Reinforcement Learning and Potential Field. Target following in dynamic pedestrian environments is an important task for mobile robots. However, it is challenging to keep tracking the target while avoiding collisions in crowded environments, especially with only one robot. In this paper, we propose a multi agent method for an arbitrary number of robots to follow the target in a socially aware manner using only 2D laser scans. The multi agent following problem is tackled by utilizing the complementary strengths of both reinforcement learning and potential field, in which the reinforcement learning part handles local interactions while navigating to the goals assigned by the potential field. Specifically, with the help of laser scans in obstacle map representation, the learning based policy can help the robots avoid collisions with both static obstacles and dynamic obstacles like pedestrians in advance, namely socially aware. While the formation control and goal assignment for each robot is obtained from a target centered potential field constructed using aggregated state information from all the following robots. Experiments are conducted in multiple settings, including random obstacle distributions and different numbers of robots. Results show that our method works successfully in unseen dynamic environments. The robots can follow the target in a socially compliant manner with only 2D laser scans.


additive bases of abelian groups of rank 2. Let G be a finite abelian group and p be the smallest prime dividing G . Let S be a sequence over G . We say that S is regular if for every proper subgroup H subsetneq G , S contains at most H 1 terms from H . Let mathsf c 0(G) be the smallest integer t such that every regular sequence S over G of length S geq t forms an additive basis of G , i.e., sum(S) G . The invariant mathsf c 0(G) was first studied by Olson and Peng in 1980's, and since then it has been determined for all finite abelian groups except for the groups with rank 2 and a few groups of rank 3 or 4 with order less than 10 8 . In this paper, we focus on the remaining case concerning groups of rank 2. It was conjectured by Gao et al. (Acta Arith. 168 (2015) 247 267) that mathsf c 0(G) m(G) . We confirm the conjecture for the case when G C n 1 oplus C n 2 with n 1 n 2 , n 1 geq 2p , p geq 3 and n 1n 2 geq 72p 6 .


TDIOT Target driven Inference for Deep Video Object Tracking. Recent tracking by detection approaches use deep object detectors as target detection baseline, because of their high performance on still images. For effective video object tracking, object detection is integrated with a data association step performed by either a custom design inference architecture or an end to end joint training for tracking purpose. In this work, we adopt the former approach and use the pre trained Mask R CNN deep object detector as the baseline. We introduce a novel inference architecture placed on top of FPN ResNet101 backbone of Mask R CNN to jointly perform detection and tracking, without requiring additional training for tracking purpose. The proposed single object tracker, TDIOT, applies an appearance similarity based temporal matching for data association. In order to tackle tracking discontinuities, we incorporate a local search and matching module into the inference head layer that exploits SiamFC for short term tracking. Moreover, in order to improve robustness to scale changes, we introduce a scale adaptive region proposal network that enables to search the target at an adaptively enlarged spatial neighborhood specified by the trace of the target. In order to meet long term tracking requirements, a low cost verification layer is incorporated into the inference architecture to monitor presence of the target based on its LBP histogram model. Performance evaluation on videos from VOT2016, VOT2018 and VOT LT2018 datasets demonstrate that TDIOT achieves higher accuracy compared to the state of the art short term trackers while it provides comparable performance in long term tracking.


A Fast Randomized Algorithm for Massive Text Normalization. Many popular machine learning techniques in natural language processing and data mining rely heavily on high quality text sources. However real world text datasets contain a significant amount of spelling errors and improperly punctuated variants where the performance of these models would quickly deteriorate. Moreover, real world, web scale datasets contain hundreds of millions or even billions of lines of text, where the existing text cleaning tools are prohibitively expensive to execute over and may require an overhead to learn the corrections. In this paper, we present FLAN, a scalable randomized algorithm to clean and canonicalize massive text data. Our algorithm relies on the Jaccard similarity between words to suggest correction results. We efficiently handle the pairwise word to word comparisons via Locality Sensitive Hashing (LSH). We also propose a novel stabilization process to address the issue of hash collisions between dissimilar words, which is a consequence of the randomized nature of LSH and is exacerbated by the massive scale of real world datasets. Compared with existing approaches, our method is more efficient, both asymptotically and in empirical evaluations, and does not rely on additional features, such as lexical phonetic similarity or word embedding features. In addition, FLAN does not require any annotated data or supervised learning. We further theoretically show the robustness of our algorithm with upper bounds on the false positive and false negative rates of corrections. Our experimental results on real world datasets demonstrate the efficiency and efficacy of FLAN.


Nonlinear effects of stabilization in ship models with non smooth nonlinearities using P control. Nonlinear terms in standard models of ship maneuvering entail continuous non smooth terms so that bifurcations of straight motion are not amenable to standard center manifold reduction. In this paper, we determine the criticality of Hopf bifurcations that arise in stabilizing the straight motion based on a recently developed analytical approach. For such a 3 DOF model of ship motion with yaw damping and yaw restoring control, we present a detailed analysis of the possibilities to stabilize the straight motion, and the resulting nonlinear effects. To facilitate the analysis, we consider a combination of rudder and propeller forces into an effective thruster force. We identify existence, location and geometry of the stability boundary in terms of the controls, including the dependence on the propeller diameter and the thruster position. We find that safe' supercritical Andronov Hopf bifurcations are predominant and by means of numerical continuation we provide a global bifurcation analysis, which identifies the arrangement and relative location of stable and unstable equilibria and periodic orbits. We illustrate the resulting stable quasi periodic ship motions in Earth fixed coordinates and present some direct numerical simulations.


Disentangled Self Attentive Neural Networks for Click Through Rate Prediction. Click Through Rate (CTR) prediction, whose aim is to predict the probability of whether a user will click on an item, is an essential task for many online applications. Due to the nature of data sparsity and high dimensionality of CTR prediction, a key to making effective prediction is to model high order feature interaction. An efficient way to do this is to perform inner product of feature embeddings with self attentive neural networks. To better model complex feature interaction, in this paper we propose a novel DisentanglEd Self atTentIve NEtwork (DESTINE) framework for CTR prediction that explicitly decouples the computation of unary feature importance from pairwise interaction. Specifically, the unary term models the general importance of one feature on all other features, whereas the pairwise interaction term contributes to learning the pure impact for each feature pair. We conduct extensive experiments using two real world benchmark datasets. The results show that DESTINE not only maintains computational efficiency but achieves consistent improvements over state of the art baselines.


A Combined Machine Learning Optimization Based Approach for Inverse Design of Nonuniform Bianisotropic Metasurfaces. Electromagnetic metasurface design based on far field constraints without the complete knowledge of the fields on both sides of the metasurface is typically a time consuming and iterative process, which relies heavily on heuristics and ad hoc methods. This paper proposes an end to end systematic and efficient approach where the designer inputs high level far field constraints such as nulls, sidelobe levels, and main beam level(s) and a 3 layer non uniform passive, lossless, omega type bianisotropic electromagnetic metasurface design to satisfy them is returned. The surface parameters to realize the far field criteria are found using the alternating direction method of multipliers on a homogenized model derived from the method of moments. This model incorporates edge effects of the finite surface and mutual coupling in the inhomogeneous impedance sheet. Optimization through the physical unit cell space integrated with machine learning based surrogate models is used to realize the desired surface parameters from physical meta atom (or unit cell) designs. Two passive lossless examples with different feeding systems and far field constraints are shown to demonstrate the effectiveness of this method.


Staying in Shape Learning Invariant Shape Representations using Contrastive Learning. Creating representations of shapes that are invari ant to isometric or almost isometric transforma tions has long been an area of interest in shape anal ysis, since enforcing invariance allows the learningof more effective and robust shape representations.Most existing invariant shape representations arehandcrafted, and previous work on learning shaperepresentations do not focus on producing invariantrepresentations. To solve the problem of learningunsupervised invariant shape representations, weuse contrastive learning, which produces discrimi native representations through learning invarianceto user specified data augmentations. To producerepresentations that are specifically isometry andalmost isometry invariant, we propose new dataaugmentations that randomly sample these transfor mations. We show experimentally that our methodoutperforms previous unsupervised learning ap proaches in both effectiveness and robustness.


A Framework for Rationale Extraction for Deep QA models. As neural network based QA models become deeper and more complex, there is a demand for robust frameworks which can access a model's rationale for its prediction. Current techniques that provide insights on a model's working are either dependent on adversarial datasets or are proposing models with explicit explanation generation components. These techniques are time consuming and challenging to extend to existing models and new datasets. In this work, we use Integrated Gradients' to extract rationale for existing state of the art models in the task of Reading Comprehension based Question Answering (RCQA). On detailed analysis and comparison with collected human rationales, we find that though 40 80 words of extracted rationale coincide with the human rationale (precision), only 6 19 of human rationale is present in the extracted rationale (recall).


Orthogonal decomposition of composition operators on the H 2 space of Dirichlet series. Let mathscr H 2 denote the Hilbert space of Dirichlet series with square summable coefficients. We study composition operators mathscr C varphi on mathscr H 2 which are generated by symbols of the form varphi(s) c 0s sum n geq1 c n n s , in the case that c 0 geq 1 . If only a subset mathbb P of prime numbers features in the Dirichlet series of varphi , then the operator mathscr C varphi admits an associated orthogonal decomposition. Under sparseness assumptions on mathbb P we use this to asymptotically estimate the approximation numbers of mathscr C varphi . Furthermore, in the case that varphi is supported on a single prime number, we affirmatively settle the problem of describing the compactness of mathscr C varphi in terms of the ordinary Nevanlinna counting function. We give detailed applications of our results to affine symbols and to angle maps.


Comprehensive Integration of API Usage Patterns. Nowadays, developers often reuse existing APIs to implement their programming tasks. A lot of API usage patterns are mined to help developers learn API usage rules. However, there are still many missing variables to be synthesized when developers integrate the patterns into their programming context. To deal with this issue, we propose a comprehensive approach to integrate API usage patterns in this paper. We first perform an empirical study by analyzing how API usage patterns are integrated in real world projects. We find the expressions for variable synthesis is often non trivial and can be divided into 5 syntax types. Based on the observation, we promote an approach to help developers interactively complete API usage patterns. Compared to the existing code completion techniques, our approach can recommend infrequent expressions accompanied with their real world usage examples according to the user intent. The evaluation shows that our approach could assist users to integrate APIs more efficiently and complete the programming tasks faster than existing works.


Towards a new cohomology theory for strict Lie 2 groups. In this article, we introduce the first degrees of a cochain complex associated to a strict Lie 2 group whose cohomology is shown to extend the classical cohomology theory of Lie groups. In particular, we show that the second cohomology group classifies an appropriate type of extension. We conclude by putting forward evidence that this complex can be extended to arbitrary degrees.


Unitarily invariant Norms on Operators. Let f be a symmetric norm on mathbb R n and let mathcal B ( mathcal H ) be the set of all bounded linear operators on a Hilbert space mathcal H of dimension at least n . Define a norm on mathcal B ( mathcal H ) by A f f(s 1(A), dots, s n(A)) , where s k(A) inf A X X in mathcal B ( mathcal H ) hbox has rank less than k is the k th singular value of A . Basic properties of the norm cdot f are obtained including some norm inequalities and characterization of the equality case. Geometric properties of the unit ball of the norm are obtained the results are used to determine the structure of maps L satisfying L(A) L(B) f A B f for any A, B in mathcal B ( mathcal H ) .


Preparing for Performance Analysis at Exascale. Performance tools for emerging heterogeneous exascale platforms must address two principal challenges when analyzing execution measurements. First, measurement of large scale executions may record mountains of performance data. Second, performance measurements for parallel programs are sparse in two ways the set of metrics present for any context and the set of contexts present in different threads. For GPU accelerated applications, an important source of sparsity is that none of the myriad of GPU metrics apply to any of the many CPU contexts. To address these challenges, we developed a novel streaming aggregation approach to postmortem analysis that employs both shared and distributed memory parallelism to aggregate sparse performance measurements from every rank, thread, and GPU stream of an application, and attributes heterogeneous call path profiles and traces to source code. Using the same amount of resources, our approach analyzes large scale performance measurements of GPU accelerated applications over an order of magnitude faster than HPCToolkit and its sparse analysis results are as much as three orders of magnitude smaller than HPCToolkit's dense representation of metrics.


An abstract critical point theorem with applications to elliptic problems with combined nonlinearities. We prove an abstract critical point theorem based on a cohomological index theory that produces pairs of nontrivial critical points with nontrivial higher critical groups. This theorem yields pairs of nontrivial solutions that are neither local minimizers nor of mountain pass type for problems with combined nonlinearities. Applications are given to subcritical and critical p Laplacian problems, Kirchhoff type nonlocal problems, and critical fractional p Laplacian problems.


Continuity of the gradient of the fractional maximal operator on W 1,1 ( mathbb R d) . We establish that the map f mapsto nabla mathcal M alpha f is continuous from W 1,1 ( mathbb R d) to L q ( mathbb R d) , where alpha in (0,d) , q frac d d alpha and mathcal M alpha denotes either the centered or non centered fractional Hardy Littlewood maximal operator. In particular, we cover the cases d 1 and alpha in (0,1) in full generality, for which results were only known for radial functions.


BPHZ Renormalization in Gaussian Rough Paths. We construct a procedure for Bogoliubov Parasiuk Hepp Zimmermann (BPHZ) renormalization of a rough path in view of the relation between rough path theory and regularity structure. We also provide a plain expression of the BPHZ renormalized model in a rough path. BPHZ renormalization plays a central role in the theory of singular stochastic partial differential equations and assures the convergence of the model in the regularity structure. Here we demonstrate that the renormalization is also effective in a rough path setting by considering its application to rough path theory and mathematical finance.


An Uncertainty Informed Framework for Trustworthy Fault Diagnosis in Safety Critical Applications. There has been a growing interest in deep learning based prognostic and health management (PHM) for building end to end maintenance decision support systems, especially due to the rapid development of autonomous systems. However, the low trustworthiness of PHM hinders its applications in safety critical assets when handling data from an unknown distribution that differs from the training dataset, referred to as the out of distribution (OOD) dataset. To bridge this gap, we propose an uncertainty informed framework to diagnose faults and meanwhile detect the OOD dataset, enabling the capability of learning unknowns and achieving trustworthy fault diagnosis. Particularly, we develop a probabilistic Bayesian convolutional neural network (CNN) to quantify both epistemic and aleatory uncertainties in fault diagnosis. The fault diagnosis model flags the OOD dataset with large predictive uncertainty for expert intervention and is confident in providing predictions for the data within tolerable uncertainty. This results in trustworthy fault diagnosis and reduces the risk of erroneous decision making, thus potentially avoiding undesirable consequences. The proposed framework is demonstrated by the fault diagnosis of bearings with three OOD datasets attributed to random number generation, an unknown fault mode, and four common sensor faults, respectively. The results show that the proposed framework is of particular advantage in tackling unknowns and enhancing the trustworthiness of fault diagnosis in safety critical applications.


ISETAuto Detecting vehicles with depth and radiance information. Autonomous driving applications use two types of sensor systems to identify vehicles depth sensing LiDAR and radiance sensing cameras. We compare the performance (average precision) of a ResNet for vehicle detection in complex, daytime, driving scenes when the input is a depth map (D d(x,y)), a radiance image (L r(x,y)), or both D,L . (1) When the spatial sampling resolution of the depth map and radiance image are equal to typical camera resolutions, a ResNet detects vehicles at higher average precision from depth than radiance. (2) As the spatial sampling of the depth map declines to the range of current LiDAR devices, the ResNet average precision is higher for radiance than depth. (3) For a hybrid system that combines a depth map and radiance image, the average precision is higher than using depth or radiance alone. We established these observations in simulation and then confirmed them using realworld data. The advantage of combining depth and radiance can be explained by noting that the two type of information have complementary weaknesses. The radiance data are limited by dynamic range and motion blur. The LiDAR data have relatively low spatial resolution. The ResNet combines the two data sources effectively to improve overall vehicle detection.


Hyperbolic Brunnian links. We present practical methods to show a Brunnian link is hyperbolic. The methods, belonging to geometric topology, are based on observation, worked by hand and generally applicable. Typical examples illustrate their efficiency. We thus determine hyperbolicity for all of the known Brunnian links in literatures except two series. Especially, we discover 6 infinite series of hyperbolic Brunnian links.


Non intrusive Surrogate Modeling for Parametrized Time dependent PDEs using Convolutional Autoencoders. This work presents a non intrusive surrogate modeling scheme based on machine learning technology for predictive modeling of complex systems, described by parametrized time dependent PDEs. For these problems, typical finite element approaches involve the spatiotemporal discretization of the PDE and the solution of the corresponding linear system of equations at each time step. Instead, the proposed method utilizes a convolutional autoencoder in conjunction with a feed forward neural network to establish a low cost and accurate mapping from the problem's parametric space to its solution space. For this purpose, time history response data are collected by solving the high fidelity model via FEM for a reduced set of parameter values. Then, by applying the convolutional autoencoder to this data set, a low dimensional representation of the high dimensional solution matrices is provided by the encoder, while the reconstruction map is obtained by the decoder. Using the latent representation given by the encoder, a feed forward neural network is efficiently trained to map points from the problem's parametric space to the compressed version of the respective solution matrices. This way, the encoded response of the system at new parameter values is given by the neural network, while the entire response is delivered by the decoder. This approach effectively bypasses the need to serially formulate and solve the system's governing equations at each time increment, thus resulting in a significant cost reduction and rendering the method ideal for problems requiring repeated model evaluations or 'real time' computations. The elaborated methodology is demonstrated on the stochastic analysis of time dependent PDEs solved with the Monte Carlo method, however, it can be straightforwardly applied to other similar type problems, such as sensitivity analysis, design optimization, etc.


Diffraction at the Open End of Dielectric Lined Circular Waveguide. A rigorous approach for solving canonical circular open ended dielectric lined waveguide diffraction problems is presented. This is continuation of our recent paper 1 where a simpler case of uniform dielectric filling has been considered. Here we deal with the case of an open ended circular waveguide with layered dielectric filling which is closer to potential applications. The presented method uses the solution of corresponding Wiener Hopf Fock equation and leads to an infinite linear system for reflection coefficients (S parameters) of the waveguide, the latter can be efficiently solved numerically using the reducing technique. As a specific example directly applicable to beam driven radiation sources based on dielectric lined capillaries, diffraction of a slow TM symmetrical mode at the open end of the described waveguide is considered. A series of such modes forms the wakefield (Cherenkov radiation field) generated by a charged particle bunch during its passage along the vacuum channel axis. Calculated S parameters were compared with those obtained from COMSOL simulation and an excellent agreement was shown. This method is expected to be very convenient for analytical investigation of various electromagnetic interactions of Terahertz (THz) waves (both free and guided) and charged particle bunches with slow wave structures prospective in context of modern beam driven THz emitters, THz accererators and THz based bunch manipulation and bunch diagnostic systems.


Measuring the photoelectron emission delay in the molecular frame. If matter absorbs a photon of sufficient energy it emits an electron. The question of the duration of the emission process has intrigued scientists for decades. With the advent of attosecond metrology, experiments addressing such ultrashort intervals became possible. While these types of studies require attosecond experimental precision, we present here a novel measurement approach that avoids those experimental difficulties. We instead extract the emission delay from the interference pattern generated as the emitted photoelectron is diffracted by the parent ion's potential. Targeting core electrons in CO, we measured a 2d map of photoelectron emission delays in the molecular frame over a wide range of electron energies. The measured emission times depend drastically on the emission direction and exhibit characteristic changes along the shape resonance of the molecule. Our approach can be routinely extended to other electron orbitals and more complex molecules.


A 2 FPN Attention Aggregation based Feature Pyramid Network for Instance Segmentation. Learning pyramidal feature representations is crucial for recognizing object instances at different scales. Feature Pyramid Network (FPN) is the classic architecture to build a feature pyramid with high level semantics throughout. However, intrinsic defects in feature extraction and fusion inhibit FPN from further aggregating more discriminative features. In this work, we propose Attention Aggregation based Feature Pyramid Network (A 2 FPN), to improve multi scale feature learning through attention guided feature aggregation. In feature extraction, it extracts discriminative features by collecting distributing multi level global context features, and mitigates the semantic information loss due to drastically reduced channels. In feature fusion, it aggregates complementary information from adjacent features to generate location wise reassembly kernels for content aware sampling, and employs channel wise reweighting to enhance the semantic consistency before element wise addition. A 2 FPN shows consistent gains on different instance segmentation frameworks. By replacing FPN with A 2 FPN in Mask R CNN, our model boosts the performance by 2.1 and 1.6 mask AP when using ResNet 50 and ResNet 101 as backbone, respectively. Moreover, A 2 FPN achieves an improvement of 2.0 and 1.4 mask AP when integrated into the strong baselines such as Cascade Mask R CNN and Hybrid Task Cascade.


Semi Supervised Domain Adaptation via Selective Pseudo Labeling and Progressive Self Training. Domain adaptation (DA) is a representation learning methodology that transfers knowledge from a label sufficient source domain to a label scarce target domain. While most of early methods are focused on unsupervised DA (UDA), several studies on semi supervised DA (SSDA) are recently suggested. In SSDA, a small number of labeled target images are given for training, and the effectiveness of those data is demonstrated by the previous studies. However, the previous SSDA approaches solely adopt those data for embedding ordinary supervised losses, overlooking the potential usefulness of the few yet informative clues. Based on this observation, in this paper, we propose a novel method that further exploits the labeled target images for SSDA. Specifically, we utilize labeled target images to selectively generate pseudo labels for unlabeled target images. In addition, based on the observation that pseudo labels are inevitably noisy, we apply a label noise robust learning scheme, which progressively updates the network and the set of pseudo labels by turns. Extensive experimental results show that our proposed method outperforms other previous state of the art SSDA methods.


Lower and upper order of harmonic mappings. In this paper, we define both the upper and lower order of a sense preserving harmonic mapping in mathbb D . We generalize to the harmonic case some known results about holomorphic functions with positive lower order and we show some consequences of a function having finite upper order. In addition, we improve a related result in the case when there is equality in a known distortion theorem for harmonic mappings with finite upper order. Some examples are provided to illustrate the developed theory.


Spatio temporal Weather Forecasting and Attention Mechanism on Convolutional LSTMs. Numerical weather forecasting on high resolution physical models consume hours of computations on supercomputers. Application of deep learning and machine learning methods in forecasting revealed new solutions in this area. In this paper, we forecast high resolution numeric weather data using both input weather data and observations by providing a novel deep learning architecture. We formulate the problem as spatio temporal prediction. Our model is composed of Convolutional Long short Term Memory, and Convolutional Neural Network units with encoder decoder structure. We enhance the short long term performance and interpretability with an attention and a context matcher mechanism. We perform experiments on high scale, real life, benchmark numerical weather dataset, ERA5 hourly data on pressure levels, and forecast the temperature. The results show significant improvements in capturing both spatial and temporal correlations with attention matrices focusing on different parts of the input series. Our model obtains the best validation and the best test score among the baseline models, including ConvLSTM forecasting network and U Net. We provide qualitative and quantitative results and show that our model forecasts 10 time steps with 3 hour frequency with an average of 2 degrees error. Our code and the data are publicly available.


Calculus rules of the generalized concave Kurdyka L ojasiewicz property. In this paper, we propose several calculus rules for the generalized concave Kurdyka L ojasiewicz (KL) property, which generalize Li and Pong's results for KL exponents. The optimal concave desingularizing function has various forms and may be nondifferentiable. Our calculus rules do not assume desingularizing functions to have any specific form nor differentiable, while the known results do. Several examples are also given to show that our calculus rules are applicable to a broader class of functions than the known ones.


FFR FD Effective and Fast Detection of DeepFakes Based on Feature Point Defects. The internet is filled with fake face images and videos synthesized by deep generative models. These realistic DeepFakes pose a challenge to determine the authenticity of multimedia content. As countermeasures, artifact based detection methods suffer from insufficiently fine grained features that lead to limited detection performance. DNN based detection methods are not efficient enough, given that a DeepFake can be created easily by mobile apps and DNN based models require high computational resources. For the first time, we show that DeepFake faces have fewer feature points than real ones, especially in certain facial regions. Inspired by feature point detector descriptors to extract discriminative features at the pixel level, we propose the Fused Facial Region Feature Descriptor (FFR FD) for effective and fast DeepFake detection. FFR FD is only a vector extracted from the face, and it can be constructed from any feature point detector descriptors. We train a random forest classifier with FFR FD and conduct extensive experiments on six large scale DeepFake datasets, whose results demonstrate that our method is superior to most state of the art DNN based models.


Evaluating Document Representations for Content based Legal Literature Recommendations. Recommender systems assist legal professionals in finding relevant literature for supporting their case. Despite its importance for the profession, legal applications do not reflect the latest advances in recommender systems and representation learning research. Simultaneously, legal recommender systems are typically evaluated in small scale user study without any public available benchmark datasets. Thus, these studies have limited reproducibility. To address the gap between research and practice, we explore a set of state of the art document representation methods for the task of retrieving semantically related US case law. We evaluate text based (e.g., fastText, Transformers), citation based (e.g., DeepWalk, Poincar 'e), and hybrid methods. We compare in total 27 methods using two silver standards with annotations for 2,964 documents. The silver standards are newly created from Open Case Book and Wikisource and can be reused under an open license facilitating reproducibility. Our experiments show that document representations from averaged fastText word vectors (trained on legal corpora) yield the best results, closely followed by Poincar 'e citation embeddings. Combining fastText and Poincar 'e in a hybrid manner further improves the overall result. Besides the overall performance, we analyze the methods depending on document length, citation count, and the coverage of their recommendations. We make our source code, models, and datasets publicly available at https github.com malteos legal document similarity .


Entailment as Few Shot Learner. Large pre trained language models (LMs) have demonstrated remarkable ability as few shot learners. However, their success hinges largely on scaling model parameters to a degree that makes it challenging to train and serve. In this paper, we propose a new approach, named as EFL, that can turn small LMs into better few shot learners. The key idea of this approach is to reformulate potential NLP task into an entailment one, and then fine tune the model with as little as 8 examples. We further demonstrate our proposed method can be (i) naturally combined with an unsupervised contrastive learning based data augmentation method (ii) easily extended to multilingual few shot learning. A systematic evaluation on 18 standard NLP tasks demonstrates that this approach improves the various existing SOTA few shot learning methods by 12 , and yields competitive few shot performance with 500 times larger models, such as GPT 3.


On some zeta integrals for unramified representations of GSp(4). This article is a companion to several works of the author and others on the arithmetic of automorphic forms for GSp(4), and their associated L functions and Galois representations. These works require, at various points, an input from smooth representation theory over p adic local fields the computation of values of the unique trilinear form on an unramified representation of GSp(4) x GL(2) x GL(2) invariant under the diagonal GL(2) x GL(2), for various different input data. In this note, we carry out these local computations, using a general formula for Shintani functions due to Gejima.


An initial review of hypersonic vehicle accidents. Hypersonic flight, generally defined as the region in which the speed of a vehicle exceeds Mach 5 and in which thermal loads become dominant, has seen more attention over the past several decades due to the potential military applications of such vehicles. Vehicles capable of flight at these speeds, whilst seemingly a novel prospect, have been in development for more than 70 years. The nature of flight in this environment as well as the new challenges it introduces have led to relatively high failure rates. This paper presents a review of hypersonic vehicle flights that have resulted in failure from conception to modern day with the purpose of identifying trends to aid and guide the development of future vehicles. The collected data is used to formulate a failure taxonomy to accurately identify and classify past hypersonic vehicle failures as well as potential future ones. The trends and features of the categorical data collected are explored using an ex post facto study.


MT6 Multilingual Pretrained Text to Text Transformer with Translation Pairs. Multilingual T5 (mT5) pretrains a sequence to sequence model on massive monolingual texts, which has shown promising results on many cross lingual tasks. In this paper, we improve multilingual text to text transfer Transformer with translation pairs (mT6). Specifically, we explore three cross lingual text to text pre training tasks, namely, machine translation, translation pair span corruption, and translation span corruption. In addition, we propose a partially non autoregressive objective for text to text pre training. We evaluate the methods on eight multilingual benchmark datasets, including sentence classification, named entity recognition, question answering, and abstractive summarization. Experimental results show that the proposed mT6 improves cross lingual transferability over mT5.


Generate, Prune, Select A Pipeline for Counterspeech Generation against Online Hate Speech. Countermeasures to effectively fight the ever increasing hate speech online without blocking freedom of speech is of great social interest. Natural Language Generation (NLG), is uniquely capable of developing scalable solutions. However, off the shelf NLG methods are primarily sequence to sequence neural models and they are limited in that they generate commonplace, repetitive and safe responses regardless of the hate speech (e.g., "Please refrain from using such language.") or irrelevant responses, making them ineffective for de escalating hateful conversations. In this paper, we design a three module pipeline approach to effectively improve the diversity and relevance. Our proposed pipeline first generates various counterspeech candidates by a generative model to promote diversity, then filters the ungrammatical ones using a BERT model, and finally selects the most relevant counterspeech response using a novel retrieval based method. Extensive Experiments on three representative datasets demonstrate the efficacy of our approach in generating diverse and relevant counterspeech.


Multi Class Unsourced Random Access via Coded Demixing. Unsourced random access (URA) is a recently proposed communication paradigm attuned to machine driven data transfers. In the original URA formulation, all the active devices share the same number of bits per packet. The scenario where several classes of devices transmit concurrently has so far received little attention. An initial solution to this problem takes the form of group successive interference cancellation, where codewords from a class of devices with more resources are recovered first, followed by the decoding of the remaining messages. This article introduces a joint iterative decoding approach rooted in approximate message passing. This framework has a concatenated coding structure borrowed from the single class coded compressed sensing and admits a solution that offers performance improvement at little added computational complexity. Our findings point to new connections between multi class URA and compressive demixing. The performance of the envisioned algorithm is validated through numerical simulations.


Visualizing Event Sequence Data for User Behavior Evaluation of In Vehicle Information Systems. With modern IVIS becoming more capable and complex than ever, their evaluation becomes increasingly difficult. The analysis of large amounts of user behavior data can help to cope with this complexity and can support UX experts in designing IVIS that serve customer needs and are safe to operate while driving. We, therefore, propose a Multi level User Behavior Visualization Framework providing effective visualizations of user behavior data that is collected via telematics from production vehicles. Our approach visualizes user behavior data on three different levels (1) The Task Level View aggregates event sequence data generated through touchscreen interactions to visualize user flows. (2) The Flow Level View allows comparing the individual flows based on a chosen metric. (3) The Sequence Level View provides detailed insights into touch interactions, glance, and driving behavior. Our case study proves that UX experts consider our approach a useful addition to their design process.


Flows, growth rates, and the veering polynomial. For certain pseudo Anosov flows phi on closed 3 manifolds, unpublished work of Agol Gu 'eritaud produces a veering triangulation tau on the manifold M obtained by deleting phi 's singular orbits. We show that tau can be realized in M so that its 2 skeleton is positively transverse to phi , and that the combinatorially defined flow graph Phi embedded in M uniformly codes phi 's orbits in a precise sense. Together with these facts we use a modified version of the veering polynomial, previously introduced by the authors, to compute the growth rates of phi 's closed orbits after cutting M along certain transverse surfaces, thereby generalizing work of McMullen in the fibered setting. These results are new even in the case where the transverse surface represents a class in the boundary of a fibered cone of M . Our work can be used to study the flow phi on the original closed manifold. Applications include counting growth rates of closed orbits after cutting along closed transverse surfaces, defining a continuous, convex entropy function on the positive' cone in H 1 of the cut open manifold, and answering a question of Leininger about the closure of the set of all stretch factors arising as monodromies within a single fibered cone of a 3 manifold. This last application connects to the study of endperiodic automorphisms of infinite type surfaces and the growth rates of their periodic points.


Detecting Logical Relation In Contract Clauses. Contracts underlie most modern commercial transactions defining define the duties and obligations of the related parties in an agreement. Ensuring such agreements are error free is crucial for modern society and their analysis of a contract requires understanding the logical relations between clauses and identifying potential contradictions. This analysis depends on error prone human effort to understand each contract clause. In this work, we develop an approach to automate the extraction of logical relations between clauses in a contract. We address this problem as a Natural Language Inference task to detect the entailment type between two clauses in a contract. The resulting approach should help contract authors detecting potential logical conflicts between clauses.


Pyramid Medical Transformer for Medical Image Segmentation. Deep neural networks have been a prevailing technique in the field of medical image processing. However, the most popular convolutional neural networks (CNNs) based methods for medical image segmentation are imperfect because they model long range dependencies by stacking layers or enlarging filters. Transformers and the self attention mechanism are recently proposed to effectively learn long range dependencies by modeling all pairs of word to word attention regardless of their positions. The idea has also been extended to the computer vision field by creating and treating image patches as embeddings. Considering the computation complexity for whole image self attention, current transformer based models settle for a rigid partitioning scheme that potentially loses informative relations. Besides, current medical transformers model global context on full resolution images, leading to unnecessary computation costs. To address these issues, we developed a novel method to integrate multi scale attention and CNN feature extraction using a pyramidal network architecture, namely Pyramid Medical Transformer (PMTrans). The PMTrans captured multi range relations by working on multi resolution images. An adaptive partitioning scheme was implemented to retain informative relations and to access different receptive fields efficiently. Experimental results on three medical image datasets (gland segmentation, MoNuSeg, and HECKTOR datasets) showed that PMTrans outperformed the latest CNN based and transformer based models for medical image segmentation.


Slope Gap Distribution of Saddle Connections on the 2n gon. We explicitly compute the limiting slope gap distribution for saddle connections on any 2n gon. Our calculations show that the slope gap distribution for a translation surface is not always unimodal, answering a question of Athreya. We also give linear lower and upper bounds for number of non differentiability points as n grows. The latter result exhibits the first example of a non trivial bound on an infinite family of translation surfaces and answers a question by Kumanduri Wang.


A Generalized Cross Ratio. In one complex variable, the cross ratio is a well known quantity associated with four given points in the complex plane that remains invariant under linear fractional maps. In particular, if one knows where three points in the complex plane are mapped under a linear fractional map, one can use this invariance to explicitly determine the map and to show that linear fractional maps are 3 transitive. In this paper, we define a generalized cross ratio and determine some of its basic properties. In particular, we determine which hypotheses must be made to guarantee that our generalized cross ratio is well defined. We thus obtain a class of maps that obey similar transitivity properties as in one complex dimension, under some more restrictive conditions.


Cloud Enabled High Altitude Platform Systems Challenges and Opportunities. Augmenting ground level communications with flying networks, such as the high altitude platform system (HAPS), is among the major innovative initiatives of the next generation of wireless systems (6G). Given HAPS quasi static positioning at the stratosphere, HAPS to ground and HAPS to air connectivity frameworks are expected to be prolific in terms of data acquisition and computing, especially given the mild weather and quasi constant wind speed characteristics of the stratospheric layer. This paper explores the opportunities stemming from the realization of cloud enabled HAPS in the context of telecommunications applications and services. The paper first advocates for the potential physical advantages of deploying HAPS as flying data centers, also known as super macro base stations. The paper then describes various cloud services that can be offered from the HAPS and the merits that can be achieved by this integration, such as enhancing the quality, speed, and range of the offered services. The proposed services span a wide range of fields, including satellites, Internet of Things (IoT), ad hoc networks (such as sensor vehicular and aerial networks), gaming, and social networks. For each service, the paper illustrates the methods that would be used by cloud providers to offload the service data to the HAPS and enable the cloud customers to consume the service. The paper further sheds light on the challenges that need to be addressed for realizing practical cloud enabled HAPS, mainly, those related to high energy, processing power, quality of service (QoS), and security considerations. Finally, the paper discusses some open issues on the topic, namely, HAPS mobility and message routing, HAPS security via blockchain and machine learning, artificial intelligence based resource allocation in cloud enabled HAPS, and integration with vertical heterogeneous networks.


On line feasible wrench polytope evaluation based on human musculoskeletal models an iterative convex hull method. Many recent human robot collaboration strategies, such as Assist As Needed (AAN), are promoting humancentered robot control, where the robot continuously adapts its assistance level based on the real time need of its human counterpart. One of the fundamental assumptions of these approaches is the ability to measure or estimate the physical capacity of humans in real time. In this work, we propose an algorithm for the feasibility set analysis of a generic class of linear algebra problems. This novel iterative convex hull method is applied to the determination of the feasible Cartesian wrench polytope associated to a musculoskeletal model of the human upper limb. The method is capable of running in real time and allows the user to define the desired estimation accuracy. The algorithm performance analysis shows that the execution time has near linear relationship to the considered number of muscles, as opposed to the exponential relationship of the conventional methods. Finally, real time robot control application of the algorithm is demonstrated in a Collaborative carrying experiment, where a human operator and a Franka Emika Panda robot jointly carry a 7kg object. The robot is controlled in accordance to the AAN paradigm maintaining the load carried by the human operator at 30 of its carrying capacity.


Fully Self Supervised Class Awareness in Dense Object Descriptors. We address the problem of inferring self supervised dense semantic correspondences between objects in multi object scenes. The method introduces learning of class aware dense object descriptors by providing either unsupervised discrete labels or confidence in object similarities. We quantitatively and qualitatively show that the introduced method outperforms previous techniques with more robust pixel to pixel matches. An example robotic application is also shown grasping of objects in clutter based on corresponding points.


A Euclidean Distance Matrix Model for Convex Clustering. Clustering has been one of the most basic and essential problems in unsupervised learning due to various applications in many critical fields. The recently proposed sum of nums (SON) model by Pelckmans et al. (2005), Lindsten et al. (2011) and Hocking et al. (2011) has received a lot of attention. The advantage of the SON model is the theoretical guarantee in terms of perfect recovery, established by Sun et al. (2018). It also provides great opportunities for designing efficient algorithms for solving the SON model. The semismooth Newton based augmented Lagrangian method by Sun et al. (2018) has demonstrated its superior performance over the alternating direction method of multipliers (ADMM) and the alternating minimization algorithm (AMA). In this paper, we propose a Euclidean distance matrix model based on the SON model. An efficient majorization penalty algorithm is proposed to solve the resulting model. Extensive numerical experiments are conducted to demonstrate the efficiency of the proposed model and the majorization penalty algorithm.


Gathering of seven autonomous mobile robots on triangular grids. In this paper, we consider the gathering problem of seven autonomous mobile robots on triangular grids. The gathering problem requires that, starting from any connected initial configuration where a subgraph induced by all robot nodes (nodes where a robot exists) constitutes one connected graph, robots reach a configuration such that the maximum distance between two robots is minimized. For the case of seven robots, gathering is achieved when one robot has six adjacent robot nodes (they form a shape like a hexagon). In this paper, we aim to clarify the relationship between the capability of robots and the solvability of gathering on a triangular grid. In particular, we focus on visibility range of robots. To discuss the solvability of the problem in terms of the visibility range, we consider strong assumptions except for visibility range. Concretely, we assume that robots are fully synchronous and they agree on the direction and orientation of the x axis, and chirality in the triangular grid. In this setting, we first consider the weakest assumption about visibility range, i.e., robots with visibility range 1. In this case, we show that there exists no collision free algorithm to solve the gathering problem. Next, we extend the visibility range to 2. In this case, we show that our algorithm can solve the problem from any connected initial configuration. Thus, the proposed algorithm is optimal in terms of visibility range.


Decomposing Complex Questions Makes Multi Hop QA Easier and More Interpretable. Multi hop QA requires the machine to answer complex questions through finding multiple clues and reasoning, and provide explanatory evidence to demonstrate the machine reasoning process. We propose Relation Extractor Reader and Comparator (RERC), a three stage framework based on complex question decomposition, which is the first work that the RERC model has been proposed and applied in solving the multi hop QA challenges. The Relation Extractor decomposes the complex question, and then the Reader answers the sub questions in turn, and finally the Comparator performs numerical comparison and summarizes all to get the final answer, where the entire process itself constitutes a complete reasoning evidence path. In the 2WikiMultiHopQA dataset, our RERC model has achieved the most advanced performance, with a winning joint F1 score of 53.58 on the leaderboard. All indicators of our RERC are close to human performance, with only 1.95 behind the human level in F1 score of support fact. At the same time, the evidence path provided by our RERC framework has excellent readability and faithfulness.


Determinantal modules over preprojective algebras and representations of Dynkin quivers. In this paper, we study extension groups of modules over a preprojective algebra via the Auslander Reiten translation of the quiver associated with it. More precisely, based on the recent work given by Aizenbud and Lapid, we give a description of extension groups of a preprojective algebra in terms of the AR translation and nilpotent matrices. As a result, we calculate the extension group of a sort of so called determinantal modules, which is an analog of quantum minors in quantum coordinate rings. In particular, we give an equivalent combinatorial condition when the product of two quantum minors (up to q power rescaling) belongs to the dual canonical basis of quantum coordinate rings in the Dynkin case.


The MSR Video to Text Dataset with Clean Annotations. Video captioning automatically generates short descriptions of the video content, usually in form of a single sentence. Many methods have been proposed for solving this task. A large dataset called MSR Video to Text (MSR VTT) is often used as the benchmark dataset for testing the performance of the methods. However, we found that the human annotations, i.e., the descriptions of video contents in the dataset are quite noisy, e.g., there are many duplicate captions and many captions contain grammatical problems. These problems may pose difficulties to video captioning models for learning underlying patterns. We cleaned the MSR VTT annotations by removing these problems, then tested several typical video captioning models on the cleaned dataset. Experimental results showed that data cleaning boosted the performances of the models measured by popular quantitative metrics. We recruited subjects to evaluate the results of a model trained on the original and cleaned datasets. The human behavior experiment demonstrated that trained on the cleaned dataset, the model generated captions that were more coherent and more relevant to the contents of the video clips.


Homogeneously polyanalytic kernels on the unit ball and the Siegel domain. We prove that the homogeneously polyanalytic functions of total order m , defined by the system of equations overline D (k 1, ldots,k n) f 0 with k 1 cdots k n m , can be written as polynomials of total degree m in variables overline z 1 , ldots, overline z n , with some analytic coefficients. We establish a weighted mean value property for such functions, using a reproducing property of Jacobi polynomials. After that, we give a general recipe to transform a reproducing kernel by a weighted change of variables. Applying these tools, we compute the reproducing kernel of the Bergman space of homogeneously polyanalytic functions on the unit ball in mathbb C n and on the Siegel domain. For the one dimensional case, analogous results were obtained by Koshelev (1977), Pessoa (2014), Hachadi and Youssfi (2019).


FrugalMCT Efficient Online ML API Selection for Multi Label Classification Tasks. Multi label classification tasks such as OCR and multi object recognition are a major focus of the growing machine learning as a service industry. While many multi label prediction APIs are available, it is challenging for users to decide which API to use for their own data and budget, due to the heterogeneity in those APIs' price and performance. Recent work shows how to select from single label prediction APIs. However the computation complexity of the previous approach is exponential in the number of labels and hence is not suitable for settings like OCR. In this work, we propose FrugalMCT, a principled framework that adaptively selects the APIs to use for different data in an online fashion while respecting user's budget. The API selection problem is cast as an integer linear program, which we show has a special structure that we leverage to develop an efficient online API selector with strong performance guarantees. We conduct systematic experiments using ML APIs from Google, Microsoft, Amazon, IBM, Tencent and other providers for tasks including multi label image classification, scene text recognition and named entity recognition. Across diverse tasks, FrugalMCT can achieve over 90 cost reduction while matching the accuracy of the best single API, or up to 8 better accuracy while matching the best API's cost.


Using Eye tracking Data to Predict Situation Awareness in Real Time during Takeover Transitions in Conditionally Automated Driving. Situation awareness (SA) is critical to improving takeover performance during the transition period from automated driving to manual driving. Although many studies measured SA during or after the driving task, few studies have attempted to predict SA in real time in automated driving. In this work, we propose to predict SA during the takeover transition period in conditionally automated driving using eye tracking and self reported data. First, a tree ensemble machine learning model, named LightGBM (Light Gradient Boosting Machine), was used to predict SA. Second, in order to understand what factors influenced SA and how, SHAP (SHapley Additive exPlanations) values of individual predictor variables in the LightGBM model were calculated. These SHAP values explained the prediction model by identifying the most important factors and their effects on SA, which further improved the model performance of LightGBM through feature selection. We standardized SA between 0 and 1 by aggregating three performance measures (i.e., placement, distance, and speed estimation of vehicles with regard to the ego vehicle) of SA in recreating simulated driving scenarios, after 33 participants viewed 32 videos with six lengths between 1 and 20 s. Using only eye tracking data, our proposed model outperformed other selected machine learning models, having a root mean squared error (RMSE) of 0.121, a mean absolute error (MAE) of 0.096, and a 0.719 correlation coefficient between the predicted SA and the ground truth. The code is available at https github.com refengchou Situation awareness prediction. Our proposed model provided important implications on how to monitor and predict SA in real time in automated driving using eye tracking data.