Optimal networks for dynamical spreading. The inverse problem of finding the optimal network structure for a specific type of dynamical process stands out as one of the most challenging problems in network science. Focusing on the susceptible infected susceptible type of dynamics on annealed networks whose structures are fully characterized by the degree distribution, we develop an analytic framework to solve the inverse problem. We find that, for relatively low or high infection rates, the optimal degree distribution is unique, which consists of no more than two distinct nodal degrees. For intermediate infection rates, the optimal degree distribution is multitudinous and can have a broader support. We also find that, in general, the heterogeneity of the optimal networks decreases with the infection rate. A surprising phenomenon is the existence of a specific value of the infection rate for which any degree distribution would be optimal in generating maximum spreading prevalence. The analytic framework and the findings provide insights into the interplay between network structure and dynamical processes with practical implications.


Independence equivalence classes of cycles. The independence equivalence class of a graph G is the set of graphs that have the same independence polynomial as G . Beaton, Brown and Cameron (arXiv 1810.05317) found the independence equivalence classes of even cycles, and raised the problem of finding the independence equivalence class of odd cycles. The problem is completely solved in this paper.


The Cyclic Graph of a 2 Frobenius Group. The cyclic graph of a group G is the graph whose vertices are the nonidentity elements of G and whose edges connect distinct elements x and y if and only if the subgroup langle x,y rangle is cyclic. We obtain information about the cyclic graph of 2 Frobenius groups. The cyclic graph of a 2 Frobenius group is disconnected. In this paper, we determine the number of connected components of the cyclic graph of any 2 Frobenius group.


On the Global Convergence of Momentum based Policy Gradient. Policy gradient (PG) methods are popular and efficient for large scale reinforcement learning due to their relative stability and incremental nature. In recent years, the empirical success of PG methods has led to the development of a theoretical foundation for these methods. In this work, we generalize this line of research by studying the global convergence of stochastic PG methods with momentum terms, which have been demonstrated to be efficient recipes for improving PG methods. We study both the soft max and the Fisher non degenerate policy parametrizations, and show that adding a momentum improves the global optimality sample complexity of vanilla PG methods by tilde mathcal O ( epsilon 1.5 ) and tilde mathcal O ( epsilon 1 ) , respectively, where epsilon 0 is the target tolerance. Our work is the first one that obtains global convergence results for the momentum based PG methods. For the generic Fisher non degenerate policy parametrizations, our result is the first single loop and finite batch PG algorithm achieving tilde O ( epsilon 3 ) global optimality sample complexity. Finally, as a by product, our methods also provide general framework for analyzing the global convergence rates of stochastic PG methods, which can be easily applied and extended to different PG estimators.


Emulating Aerosol Microphysics with Machine Learning. Aerosol particles play an important role in the climate system by absorbing and scattering radiation and influencing cloud properties. They are also one of the biggest sources of uncertainty for climate modeling. Many climate models do not include aerosols in sufficient detail. In order to achieve higher accuracy, aerosol microphysical properties and processes have to be accounted for. This is done in the ECHAM HAM global climate aerosol model using the M7 microphysics model, but increased computational costs make it very expensive to run at higher resolutions or for a longer time. We aim to use machine learning to approximate the microphysics model at sufficient accuracy and reduce the computational cost by being fast at inference time. The original M7 model is used to generate data of input output pairs to train a neural network on it. By using a special logarithmic transform we are able to learn the variables tendencies achieving an average R 2 score of 89 . On a GPU we achieve a speed up of 120 compared to the original model.


Explicit Moduli of Superelliptic Curves with Level Structure. In this article we give an explicit construction of the moduli space of trigonal superelliptic curves with level 3 structure. The construction is given in terms of point sets on the projective line and leads to a closed formula for the number of connected (and irreducible) components of the moduli space. The results of the article generalise the description of the moduli space of hyperelliptic curves with level 2 structure, due to Dolgachev and Ortland, Runge and Tsuyumine.


Initial nonrepetitive complexity of regular episturmian words and their Diophantine exponents. Regular episturmian words are episturmian words whose directive words have a regular and restricted form making them behave more like Sturmian words than general episturmian words. We present a method to evaluate the initial nonrepetitive complexity of regular episturmian words extending the work of Wojcik on Sturmian words. For this, we develop a theory of generalized Ostrowski numeration systems and show how to associate with each episturmian word a unique sequence of numbers written in this numeration system. The description of the initial nonrepetitive complexity allows us to obtain novel results on the Diophantine exponents of regular episturmian words. We prove that the Diophantine exponent of a regular episturmian word is finite if and only if its directive word has bounded partial quotients. Moreover, we prove that the Diophantine exponent of a regular episturmian word is strictly greater than 2 if the sequence of partial quotients is eventually at least 3 . Given an infinite word x over an integer alphabet, we may consider a real number xi x having x as a fractional part. The Diophantine exponent of x is a lower bound for the irrationality exponent of xi x . Our results thus yield nontrivial lower bounds for the irrationality exponents of real numbers whose fractional parts are regular episturmian words. As a consequence, we identify a new uncountable class of transcendental numbers whose irrationality exponents are strictly greater than 2 . This class contains an uncountable subclass of Liouville numbers.


COMPAS Representation Learning with Compositional Part Sharing for Few Shot Classification. Few shot image classification consists of two consecutive learning processes 1) In the meta learning stage, the model acquires a knowledge base from a set of training classes. 2) During meta testing, the acquired knowledge is used to recognize unseen classes from very few examples. Inspired by the compositional representation of objects in humans, we train a neural network architecture that explicitly represents objects as a set of parts and their spatial composition. In particular, during meta learning, we train a knowledge base that consists of a dictionary of part representations and a dictionary of part activation maps that encode common spatial activation patterns of parts. The elements of both dictionaries are shared among the training classes. During meta testing, the representation of unseen classes is learned using the part representations and the part activation maps from the knowledge base. Finally, an attention mechanism is used to strengthen those parts that are most important for each category. We demonstrate the value of our compositional learning framework for a few shot classification using miniImageNet, tieredImageNet, CIFAR FS, and FC100, where we achieve state of the art performance.


A data driven convergence criterion for iterative unfolding of smeared spectra. A data driven convergence criterion for the D'Agostini (Richardson Lucy) iterative unfolding is presented. It relies on the unregularized spectrum (infinite number of iterations), and allows a safe estimation of the bias and undercoverage induced by truncating the algorithm. In addition, situations where the response matrix is not perfectly known are also discussed, and show that in most cases the unregularized spectrum is not an unbiased estimator of the true distribution. Whenever a bias is introduced, either by truncation of by poor knowledge of the response, a way to retrieve appropriate coverage properties is proposed.


Finite difference method for inhomogeneous fractional Dirichlet problem. We make the split of the integral fractional Laplacian as ( Delta) s u ( Delta)( Delta) s 1 u , where s in(0, frac 1 2 ) cup( frac 1 2 ,1) . Based on this splitting, we respectively discretize the one and two dimensional integral fractional Laplacian with the inhomogeneous Dirichlet boundary condition and give the corresponding truncation errors with the help of the interpolation estimate. Moreover, the suitable corrections are proposed to guarantee the convergence in solving the inhomogeneous fractional Dirichlet problem and an mathcal O (h 1 alpha 2s ) convergence rate is obtained when the solution u in C 1, alpha ( bar Omega delta n ) , where n is the dimension of the space, alpha in( max(0,2s 1),1 , delta is a fixed positive constant, and h denotes mesh size. Finally, the performed numerical experiments confirm the theoretical results.


Homotopy Based Reinforcement Learning with Maximum Entropy for Autonomous Air Combat. The Intelligent decision of the unmanned combat aerial vehicle (UCAV) has long been a challenging problem. The conventional search method can hardly satisfy the real time demand during high dynamics air combat scenarios. The reinforcement learning (RL) method can significantly shorten the decision time via using neural networks. However, the sparse reward problem limits its convergence speed and the artificial prior experience reward can easily deviate its optimal convergent direction of the original task, which raises great difficulties for the RL air combat application. In this paper, we propose a homotopy based soft actor critic method (HSAC) which focuses on addressing these problems via following the homotopy path between the original task with sparse reward and the auxiliary task with artificial prior experience reward. The convergence and the feasibility of this method are also proved in this paper. To confirm our method feasibly, we construct a detailed 3D air combat simulation environment for the RL based methods training firstly, and we implement our method in both the attack horizontal flight UCAV task and the self play confrontation task. Experimental results show that our method performs better than the methods only utilizing the sparse reward or the artificial prior experience reward. The agent trained by our method can reach more than 98.3 win rate in the attack horizontal flight UCAV task and average 67.4 win rate when confronted with the agents trained by the other two methods.


G VOM A GPU Accelerated Voxel Off Road Mapping System. We present a local 3D voxel mapping framework for off road path planning and navigation. Our method provides both hard and soft positive obstacle detection, negative obstacle detection, slope estimation, and roughness estimation. By using a 3D array lookup table data structure and by leveraging the GPU it can provide online performance. We then demonstrate the system working on three vehicles, a Clearpath Robotics Warthog, Moose, and a Polaris Ranger, and compare against a set of pre recorded waypoints. This was done at 4.5 m s in autonomous operation and 12 m s in manual operation with a map update rate of 10 Hz. Finally, an open source ROS implementation is provided. https github.com unmannedlab G VOM


Julia for Biologists. Increasing emphasis on data and quantitative methods in the biomedical sciences is making biological research more computational. Collecting, curating, processing, and analysing large genomic and imaging data sets poses major computational challenges, as does simulating larger and more realistic models in systems biology. Here we discuss how a relative newcomer among computer programming languages Julia is poised to meet the current and emerging demands in the computational biosciences, and beyond. Speed, flexibility, a thriving package ecosystem, and readability are major factors that make high performance computing and data analysis available to an unprecedented degree to "gifted amateurs". We highlight how Julia's design is already enabling new ways of analysing biological data and systems, and we provide a, necessarily incomplete, list of resources that can facilitate the transition into the Julian way of computing.


Activation function design for deep networks linearity and effective initialisation. The activation function deployed in a deep neural network has great influence on the performance of the network at initialisation, which in turn has implications for training. In this paper we study how to avoid two problems at initialisation identified in prior works rapid convergence of pairwise input correlations, and vanishing and exploding gradients. We prove that both these problems can be avoided by choosing an activation function possessing a sufficiently large linear region around the origin, relative to the bias variance sigma b 2 of the network's random initialisation. We demonstrate empirically that using such activation functions leads to tangible benefits in practice, both in terms test and training accuracy as well as training time. Furthermore, we observe that the shape of the nonlinear activation outside the linear region appears to have a relatively limited impact on training. Finally, our results also allow us to train networks in a new hyperparameter regime, with a much larger bias variance than has previously been possible.


A compendium of covariances and correlation coefficients of coalescent tree properties. Gene genealogies are frequently studied by measuring properties such as their height ( H ), length ( L ), sum of external branches ( E ), sum of internal branches ( I ), and mean of their two basal branches ( B ), and the coalescence times that contribute to the other genealogical features ( T ). These tree properties and their relationships can provide insight into the effects of population genetic processes on genealogies and genetic sequences. Here, under the coalescent model, we study the 15 correlations among pairs of features of genealogical trees H n , L n , E n , I n , B n , and T k for a sample of size n , with 2 leq k leq n . We report high correlations among H n , L n , I n, and B n , with all pairwise correlations of these quantities having values greater than or equal to sqrt 6 6 zeta(3) 6 pi 2 ( pi sqrt 18 9 pi 2 pi 4 ) approx 0.84930 in the limit as n rightarrow infty . Although E n has an expectation of 2 for all n and H n has expectation 2 in the limit as n rightarrow infty , their limiting correlation is 0. The results contribute toward understanding features of the shapes of coalescent trees.


LatentCLR A Contrastive Learning Approach for Unsupervised Discovery of Interpretable Directions. Recent research has shown that it is possible to find interpretable directions in the latent spaces of pre trained Generative Adversarial Networks (GANs). These directions enable controllable image generation and support a wide range of semantic editing operations, such as zoom or rotation. The discovery of such directions is often done in a supervised or semi supervised manner and requires manual annotations which limits their use in practice. In comparison, unsupervised discovery allows finding subtle directions that are difficult to detect a priori. In this work, we propose a contrastive learning based approach to discover semantic directions in the latent space of pre trained GANs in a self supervised manner. Our approach finds semantically meaningful dimensions comparable with state of the art methods.


Security For System On Chip (SoC) Using Neural Networks. With the growth of embedded systems, VLSI design phases complexity and cost factors across the globe and has become outsourced. Modern computing ICs are now using system on chip for better on chip processing and communication. In the era of Internet of Things (IoT), security has become one of the most crucial parts of a System on Chip (SoC). Malicious activities generate abnormal traffic patterns which affect the operation of the system and its performance which cannot be afforded in a computation hungry world. SoCs have a chance of functionality failure, leakage of information, even a denial of services (DoS), Hardware Trojan Horses and many more factors which are categorized as security threats. In this paper, we aim to compare and describe different types of malicious security threats and how neural networks can be used to prevent those attacks. Spiking Neural Networks (SNN), Runtime Neural Architecture (RTNA) are some of the neural networks which prevent SoCs from attacks. Finally, the development trends in SoC security are also highlighted.


Alignment of electron optical beam shaping elements using a convolutional neural network. A convolutional neural network is used to align an orbital angular momentum sorter in a transmission electron microscope. The method is demonstrated using simulations and experimentally. As a result of its accuracy and speed, it offers the possibility of real time tuning of other electron optical devices and electron beam shaping configurations.


3D High Fidelity Mask Face Presentation Attack Detection Challenge. The threat of 3D masks to face recognition systems is increasingly serious and has been widely concerned by researchers. To facilitate the study of the algorithms, a large scale High Fidelity Mask dataset, namely CASIA SURF HiFiMask (briefly HiFiMask) has been collected. Specifically, it consists of a total amount of 54, 600 videos which are recorded from 75 subjects with 225 realistic masks under 7 new kinds of sensors. Based on this dataset and Protocol 3 which evaluates both the discrimination and generalization ability of the algorithm under the open set scenarios, we organized a 3D High Fidelity Mask Face Presentation Attack Detection Challenge to boost the research of 3D mask based attack detection. It attracted 195 teams for the development phase with a total of 18 teams qualifying for the final round. All the results were verified and re run by the organizing team, and the results were used for the final ranking. This paper presents an overview of the challenge, including the introduction of the dataset used, the definition of the protocol, the calculation of the evaluation criteria, and the summary and publication of the competition results. Finally, we focus on introducing and analyzing the top ranking algorithms, the conclusion summary, and the research ideas for mask attack detection provided by this competition.


Fluctuation results for size of the vacant set for random walks on discrete torus. We consider one or more independent random walks on the d ge 3 dimensional discrete torus. The walks start from vertices chosen independently and uniformly at random. We analyze the fluctuation behavior of the size of some random sets arising from the trajectories of the random walks at a time proportional to the size of the torus. Examples include vacant sets and the intersection of ranges. The proof relies on a refined analysis of tail estimates for hitting time and can be applied for other vertex transitive graphs.


Module classes induced by complexes and lambda pure injective modules. We prove that, if mathcal GP is the class of all Gorenstein projective modules over a ring R , then mathfrak GP ( mathcal GP , mathcal GP perp) is a cotorsion pair. Moreover, mathfrak GP is complete when all projective modules are lambda pure injective for some infinite regular cardinal lambda (in particular, if R is right Sigma pure injective). We obtain these results, on the one hand, studying the class of totally acyclic complexes over R . We prove that, when R is Sigma pure injective, this class is deconstructible and forms a coreflective subcategory of the homotopy category of the projective modules. On the other hand, we use results about lambda pure injective modules for infinite regular cardinals lambda . Finally, under different set theoretical hypotheses, we show that for an arbitrary ring R , the following hold (1) There exists an infinite regular cardinal number lambda such that every projective module is lambda pure injective (and mathfrak GP is complete). (2) R is right pure semisimple if and only if there exists a regular uncountable lambda such that mathrm Mod R has enough lambda pure injective objects.


Existence, stability and long time behaviour of weak solutions of the three dimensional compressible Navier Stokes equations with potential force. We address the global in time existence, stability and long time behaviour of weak solutions of the three dimensional compressible Navier Stokes equations with potential force. We show the details of the alpha dependence of different smoothing rates for weak solutions near t 0 under the assumption on the initial velocity u 0 that u 0 in H alpha for alpha in( frac 1 2 ,1 and obtain long time convergence of weak solutions in various norms. We then make use of the Lagrangean framework in comparing the instantaneous states of corresponding fluid particles in two different solutions. The present work provides qualitative results on the long time behaviour of weak solutions and how the weak solutions depend continuously on initial data and steady states.


Strong conciseness of coprime and anti coprime commutators. A coprime commutator in a profinite group G is an element of the form x,y , where x and y have coprime order and an anti coprime commutator is a commutator x,y such that the orders of x and y are divisible by the same primes. In the present paper we establish that a profinite group G is finite by pronilpotent if the cardinality of the set of coprime commutators in G is less than 2 aleph 0 . Moreover, a profinite group G has finite commutator subgroup G' if the cardinality of the set of anti coprime commutators in G is less than 2 aleph 0 .


TSSuBERT Tweet Stream Summarization Using BERT. The development of deep neural networks and the emergence of pre trained language models such as BERT allow to increase performance on many NLP tasks. However, these models do not meet the same popularity for tweet summarization, which can probably be explained by the lack of existing collections for training and evaluation. Our contribution in this paper is twofold (1) we introduce a large dataset for Twitter event summarization, and (2) we propose a neural model to automatically summarize huge tweet streams. This extractive model combines in an original way pre trained language models and vocabulary frequency based representations to predict tweet salience. An additional advantage of the model is that it automatically adapts the size of the output summary according to the input tweet stream. We conducted experiments using two different Twitter collections, and promising results are observed in comparison with state of the art baselines.


Anti Backdoor Learning Training Clean Models on Poisoned Data. Backdoor attack has emerged as a major security threat to deep neural networks (DNNs). While existing defense methods have demonstrated promising results on detecting or erasing backdoors, it is still not clear whether robust training methods can be devised to prevent the backdoor triggers being injected into the trained model in the first place. In this paper, we introduce the concept of emph anti backdoor learning , aiming to train emph clean models given backdoor poisoned data. We frame the overall learning process as a dual task of learning the emph clean and the emph backdoor portions of data. From this view, we identify two inherent characteristics of backdoor attacks as their weaknesses 1) the models learn backdoored data much faster than learning with clean data, and the stronger the attack the faster the model converges on backdoored data 2) the backdoor task is tied to a specific class (the backdoor target class). Based on these two weaknesses, we propose a general learning scheme, Anti Backdoor Learning (ABL), to automatically prevent backdoor attacks during training. ABL introduces a two stage emph gradient ascent mechanism for standard training to 1) help isolate backdoor examples at an early training stage, and 2) break the correlation between backdoor examples and the target class at a later training stage. Through extensive experiments on multiple benchmark datasets against 10 state of the art attacks, we empirically show that ABL trained models on backdoor poisoned data achieve the same performance as they were trained on purely clean data. Code is available at url https github.com bboylyg ABL .


Does referent predictability affect the choice of referential form? A computational approach using masked coreference resolution. It is often posited that more predictable parts of a speaker's meaning tend to be made less explicit, for instance using shorter, less informative words. Studying these dynamics in the domain of referring expressions has proven difficult, with existing studies, both psycholinguistic and corpus based, providing contradictory results. We test the hypothesis that speakers produce less informative referring expressions (e.g., pronouns vs. full noun phrases) when the context is more informative about the referent, using novel computational estimates of referent predictability. We obtain these estimates training an existing coreference resolution system for English on a new task, masked coreference resolution, giving us a probability distribution over referents that is conditioned on the context but not the referring expression. The resulting system retains standard coreference resolution performance while yielding a better estimate of human derived referent predictability than previous attempts. A statistical analysis of the relationship between model output and mention form supports the hypothesis that predictability affects the form of a mention, both its morphosyntactic type and its length.


Han's conjecture for bounded extensions. Let B subset A be a left or right bounded extension of finite dimensional algebras. We use the Jacobi Zariski long nearly exact sequence to show that B satisfies Han's conjecture if and only if A does, regardless if the extension splits or not. We provide conditions ensuring that an extension by arrows and relations is left or right bounded. Finally we give a structure result for extensions of an algebra given by a quiver and admissible relations, and examples of non split left or right bounded extensions.


Directional bounded complexity, mean equicontinuity and discrete spectrum for mathbb Z q actions. Given q in mathbb N , let (X,T) be a mathbb Z q system, vec v in mathbb R q setminus vec 0 be a direction vector and textbf b in mathbb R q 1 . We study (X,T) that has bounded complexity with respect to three kinds of metrics defined along direction vec v the directional Bowen metric d k vec v , textbf b , the directional max mean metric hat d k vec v , textbf b and the directional mean metric bar d k vec v , textbf b . It is shown that (X,T) has bounded topological complexity with respect to d k vec v , textbf b k 1 infty (resp. hat d k vec v , textbf b k 1 infty ) if and only if T is ( vec v , textbf b ) equicontinuous (resp. ( vec v , textbf b ) equicontinuous in the mean). Meanwhile, it turns out that an invariant Borel probability measure mu on X has bounded complexity with respect to d k vec v , textbf b k 1 infty if and only if T is ( mu, vec v , textbf b ) equicontinuous. Moreover, it is shown that mu has bounded complexity with respect to bar d k vec v , textbf b k 1 infty if and only if mu has bounded complexity with respect to hat d k vec v , textbf b k 1 infty if and only if T is ( mu, vec v , textbf b ) mean equicontinuous if and only if T is ( mu, vec v , textbf b ) equicontinuous in the mean if and only if mu has vec v discrete spectrum.


HyObscure Hybrid Obscuring for Privacy Preserving Data Publishing. Minimizing privacy leakage while ensuring data utility is a critical problem to data holders in a privacy preserving data publishing task. Most prior research concerns only with one type of data and resorts to a single obscuring method, eg, obfuscation or generalization, to achieve a privacy utility tradeoff, which is inadequate for protecting real life heterogeneous data and is hard to defend ever growing machine learning based inference attacks. This work takes a pilot study on privacy preserving data publishing when both generalization and obfuscation operations are employed for heterogeneous data protection. To this end, we first propose novel measures for privacy and utility quantification and formulate the hybrid privacy preserving data obscuring problem to account for the joint effect of generalization and obfuscation. We then design a novel hybrid protection mechanism called HyObscure, to cross iteratively optimize the generalization and obfuscation operations for maximum privacy protection under a certain utility guarantee. The convergence of the iterative process and the privacy leakage bound of HyObscure are also provided in theory. Extensive experiments demonstrate that HyObscure significantly outperforms a variety of state of the art baseline methods when facing various inference attacks under different scenarios. HyObscure also scales linearly to the data size and behaves robustly with varying key parameters.


Where to cut to delay a pandemic with minimum disruption? Mathematical analysis based on the SIS model. We consider the problem of modifying a network topology in such a way as to delay the propagation of a disease with minimal disruption of the network capacity to reroute goods items passengers. We find an approximate solution to the Susceptible Infected Susceptible (SIS) model, which constitutes a tight upper bound to its exact solution. This upper bound allows direct structure epidemic dynamic relations via the total communicability function. Using this approach we propose a strategy to remove edges in a network that significantly delays the propagation of a disease across the network with minimal disruption of its capacity to deliver goods items passengers. We apply this strategy to the analysis of the U.K. airport transportation network weighted by the number of passengers transported in the year 2003. We find that the removal of all flights connecting four origin destination pairs in the U.K. delays the propagation of a disease by more than 300 , with a minimal deterioration of the transportation capacity of this network. These time delays in the propagation of a disease represent an important non pharmaceutical intervention to confront an epidemics, allowing for better preparations of the health systems, while keeping the economy moving with minimal disruptions.


Skeleton Based Mutually Assisted Interacted Object Localization and Human Action Recognition. Skeleton data carries valuable motion information and is widely explored in human action recognition. However, not only the motion information but also the interaction with the environment provides discriminative cues to recognize the action of persons. In this paper, we propose a joint learning framework for mutually assisted "interacted object localization" and "human action recognition" based on skeleton data. The two tasks are serialized together and collaborate to promote each other, where preliminary action type derived from skeleton alone helps improve interacted object localization, which in turn provides valuable cues for the final human action recognition. Besides, we explore the temporal consistency of interacted object as constraint to better localize the interacted object with the absence of ground truth labels. Extensive experiments on the datasets of SYSU 3D, NTU60 RGB D and Northwestern UCLA show that our method achieves the best or competitive performance with the state of the art methods for human action recognition. Visualization results show that our method can also provide reasonable interacted object localization results.


DSRN an Efficient Deep Network for Image Relighting. Custom and natural lighting conditions can be emulated in images of the scene during post editing. Extraordinary capabilities of the deep learning framework can be utilized for such purpose. Deep image relighting allows automatic photo enhancement by illumination specific retouching. Most of the state of the art methods for relighting are run time intensive and memory inefficient. In this paper, we propose an efficient, real time framework Deep Stacked Relighting Network (DSRN) for image relighting by utilizing the aggregated features from input image at different scales. Our model is very lightweight with total size of about 42 MB and has an average inference time of about 0.0116s for image of resolution 1024 times 1024 which is faster as compared to other multi scale models. Our solution is quite robust for translating image color temperature from input image to target image and also performs moderately for light gradient generation with respect to the target image. Additionally, we show that if images illuminated from opposite directions are used as input, the qualitative results improve over using a single input image.


Binary classification of proteins by a Machine Learning approach. In this work we present a system based on a Deep Learning approach, by using a Convolutional Neural Network, capable of classifying protein chains of amino acids based on the protein description contained in the Protein Data Bank. Each protein is fully described in its chemical physical geometric properties in a file in XML format. The aim of the work is to design a prototypical Deep Learning machinery for the collection and management of a huge amount of data and to validate it through its application to the classification of a sequences of amino acids. We envisage applying the described approach to more general classification problems in biomolecules, related to structural properties and similarities.


Spark Optimization of Linear Codes for Reliable Data Delivery by Relay Drones. Data gathering operations in remote locations often rely on relay drones, which collect, store and deliver transmitted information to a ground control station. The probability of the ground control station successfully reconstructing the gathered data can be increased if random linear coding (RLC) is used, especially when feedback channels between the drones and the transmitter are not available. RLC decoding can be complemented by partial packet recovery (PPR), which utilizes sparse recovery principles to repair erroneously received data before RLC decoding takes place. We explain that the spark of the transpose of the parity check matrix of the linear code, that is, the smallest number of linearly dependent columns of the matrix, influences the effectiveness of PPR. We formulate a spark optimization problem and obtain code designs that achieve a gain over PPR assisted RLC, in terms of the probability that the ground control station will decode the delivered data.


Complexity of Paired Domination in AT free and Planar Graphs. For a graph G (V,E) , a subset D of vertex set V , is a dominating set of G if every vertex not in D is adjacent to atleast one vertex of D . A dominating set D of a graph G with no isolated vertices is called a paired dominating set (PD set), if G D , the subgraph induced by D in G has a perfect matching. The Min PD problem requires to compute a PD set of minimum cardinality. The decision version of the Min PD problem remains NP complete even when G belongs to restricted graph classes such as bipartite graphs, chordal graphs etc. On the positive side, the problem is efficiently solvable for many graph classes including intervals graphs, strongly chordal graphs, permutation graphs etc. In this paper, we study the complexity of the problem in AT free graphs and planar graph. The class of AT free graphs contains cocomparability graphs, permutation graphs, trapezoid graphs, and interval graphs as subclasses. We propose a polynomial time algorithm to compute a minimum PD set in AT free graphs. In addition, we also present a linear time 2 approximation algorithm for the problem in AT free graphs. Further, we prove that the decision version of the problem is NP complete for planar graphs, which answers an open question asked by Lin et al. (in Theor. Comput. Sci., 591 (2015) 99 105 and Algorithmica, 82 (2020) 2809 2840 ).


TUDublin team at Constraint AAAI2021 COVID19 Fake News Detection. The paper is devoted to the participation of the TUDublin team in Constraint AAAI2021 COVID19 Fake News Detection Challenge. Today, the problem of fake news detection is more acute than ever in connection with the pandemic. The number of fake news is increasing rapidly and it is necessary to create AI tools that allow us to identify and prevent the spread of false information about COVID 19 urgently. The main goal of the work was to create a model that would carry out a binary classification of messages from social media as real or fake news in the context of COVID 19. Our team constructed the ensemble consisting of Bidirectional Long Short Term Memory, Support Vector Machine, Logistic Regression, Naive Bayes and a combination of Logistic Regression and Naive Bayes. The model allowed us to achieve 0.94 F1 score, which is within 5 of the best result.


Subspace Regularizers for Few Shot Class Incremental Learning. Few shot class incremental learning the problem of updating a trained classifier to discriminate among an expanded set of classes with limited labeled data is a key challenge for machine learning systems deployed in non stationary environments. Existing approaches to the problem rely on complex model architectures and training procedures that are difficult to tune and re use. In this paper, we present an extremely simple approach that enables the use of ordinary logistic regression classifiers for few shot incremental learning. The key to this approach is a new family of subspace regularization schemes that encourage weight vectors for new classes to lie close to the subspace spanned by the weights of existing classes. When combined with pretrained convolutional feature extractors, logistic regression models trained with subspace regularization outperform specialized, state of the art approaches to few shot incremental image classification by up to 22 on the miniImageNet dataset. Because of its simplicity, subspace regularization can be straightforwardly extended to incorporate additional background information about the new classes (including class names and descriptions specified in natural language) these further improve accuracy by up to 2 . Our results show that simple geometric regularization of class representations offers an effective tool for continual learning.


Improved Lang Weil bounds for a geometrically irreducible hypersurface over a finite field. We sharpen to nearly optimal the known asymptotic and explicit bounds for the number of mathbb F q rational points on a geometrically irreducible hypersurface over a (large) finite field. The proof involves a Bertini type probabilistic combinatorial technique. Namely, we study the number of mathbb F q points on the intersection of the given hypersurface with a random plane.


FIBER Fill in the Blanks as a Challenging Video Understanding Evaluation Framework. We propose fill in the blanks as a video understanding evaluation framework and introduce FIBER a novel dataset consisting of 28,000 videos and descriptions in support of this evaluation framework. The fill in the blanks setting tests a model's understanding of a video by requiring it to predict a masked noun phrase in the caption of the video, given the video and the surrounding text. The FIBER benchmark does not share the weaknesses of the current state of the art language informed video understanding tasks, namely (1) video question answering using multiple choice questions, where models perform relatively well because they exploit linguistic biases in the task formulation, thus making our framework challenging for the current state of the art systems to solve and (2) video captioning, which relies on an open ended evaluation framework that is often inaccurate because system answers may be perceived as incorrect if they differ in form from the ground truth. The FIBER dataset and our code are available at https lit.eecs.umich.edu fiber .


The Scales of Human Mobility. There is a contradiction at the heart of our current understanding of individual and collective mobility patterns. On one hand, a highly influential stream of literature on human mobility driven by analyses of massive empirical datasets finds that human movements show no evidence of characteristic spatial scales. There, human mobility is described as scale free. On the other hand, in geography, the concept of scale, referring to meaningful levels of description from individual buildings through neighborhoods, cities, regions, and countries, is central for the description of various aspects of human behavior such as socio economic interactions, or political and cultural dynamics. Here, we resolve this apparent paradox by showing that day to day human mobility does indeed contain meaningful scales, corresponding to spatial containers restricting mobility behavior. The scale free results arise from aggregating displacements across containers. We present a simple model, which given a person's trajectory, infers their neighborhoods, cities, and so on, as well as the sizes of these geographical containers. We find that the containers characterizing the trajectories of more than 700,000 individuals do indeed have typical sizes. We show that our model generates highly realistic trajectories without overfitting and provides a new lens through which to understand the differences in mobility behaviour across countries, gender groups, and urban rural areas.


High Fidelity and Arbitrary Face Editing. Cycle consistency is widely used for face editing. However, we observe that the generator tends to find a tricky way to hide information from the original image to satisfy the constraint of cycle consistency, making it impossible to maintain the rich details (e.g., wrinkles and moles) of non editing areas. In this work, we propose a simple yet effective method named HifaFace to address the above mentioned problem from two perspectives. First, we relieve the pressure of the generator to synthesize rich details by directly feeding the high frequency information of the input image into the end of the generator. Second, we adopt an additional discriminator to encourage the generator to synthesize rich details. Specifically, we apply wavelet transformation to transform the image into multi frequency domains, among which the high frequency parts can be used to recover the rich details. We also notice that a fine grained and wider range control for the attribute is of great importance for face editing. To achieve this goal, we propose a novel attribute regression loss. Powered by the proposed framework, we achieve high fidelity and arbitrary face editing, outperforming other state of the art approaches.


Proceedings AI ML for Cybersecurity Challenges, Solutions, and Novel Ideas at SIAM Data Mining 2021. Malicious cyber activity is ubiquitous and its harmful effects have dramatic and often irreversible impacts on society. Given the shortage of cybersecurity professionals, the ever evolving adversary, the massive amounts of data which could contain evidence of an attack, and the speed at which defensive actions must be taken, innovations which enable autonomy in cybersecurity must continue to expand, in order to move away from a reactive defense posture and towards a more proactive one. The challenges in this space are quite different from those associated with applying AI in other domains such as computer vision. The environment suffers from an incredibly high degree of uncertainty, stemming from the intractability of ingesting all the available data, as well as the possibility that malicious actors are manipulating the data. Another unique challenge in this space is the dynamism of the adversary causes the indicators of compromise to change frequently and without warning. In spite of these challenges, machine learning has been applied to this domain and has achieved some success in the realm of detection. While this aspect of the problem is far from solved, a growing part of the commercial sector is providing ML enhanced capabilities as a service. Many of these entities also provide platforms which facilitate the deployment of these automated solutions. Academic research in this space is growing and continues to influence current solutions, as well as strengthen foundational knowledge which will make autonomous agents in this space a possibility.


Dynamic pricing and assortment under a contextual MNL demand. We consider dynamic multi product pricing and assortment problems under an unknown demand over T periods, where in each period, the seller decides on the price for each product or the assortment of products to offer to a customer who chooses according to an unknown Multinomial Logit Model (MNL). Such problems arise in many applications, including online retail and advertising. We propose a randomized dynamic pricing policy based on a variant of the Online Newton Step algorithm (ONS) that achieves a O(d sqrt T log(T)) regret guarantee under an adversarial arrival model. We also present a new optimistic algorithm for the adversarial MNL contextual bandits problem, which achieves a better dependency than the state of the art algorithms in a problem dependent constant kappa (potentially exponentially small). Our regret upper bounds scale as tilde O (d sqrt kappa T log(T) kappa) , which gives a significantly stronger bound than the existing tilde O (d sqrt T kappa) guarantees.


Over the Air Federated Multi Task Learning. In this letter, we introduce over the air computation into the communication design of federated multi task learning (FMTL), and propose an over the air federated multi task learning (OA FMTL) framework, where multiple learning tasks deployed on edge devices share a non orthogonal fading channel under the coordination of an edge server (ES). Specifically, the model updates for all the tasks are transmitted and superimposed concurrently over a non orthogonal uplink fading channel, and the model aggregations of all the tasks are reconstructed at the ES through a modified version of the turbo compressed sensing algorithm (Turbo CS) that overcomes inter task interference. Both convergence analysis and numerical results show that the OA FMTL framework can significantly improve the system efficiency in terms of reducing the number of channel uses without causing substantial learning performance degradation.


Appearance of the Kashiwara Saito singularity in the representation theory of p adic mathop GL 16 . In 1993 David Vogan proposed a basis for the vector space of stable distributions on p adic groups using the microlocal geometry of moduli spaces of Langlands parameters. In the case of general linear groups, distribution characters of irreducible admissible representations, taken up to equivalence, form a basis for the vector space of stable distributions. In this paper we show that these two bases, one putative, cannot be equal. Specifically, we use the Kashiwara Saito singularity to find a non Arthur type irreducible admissible representation of p adic mathop GL 16 whose ABV packet, as defined in earlier work, contains exactly one other representation remarkably, this other admissible representation is of Arthur type. In the course of this study we strengthen the main result concerning the Kashiwara Saito singularity. The irreducible admissible representations in this paper illustrate a fact we found surprising for general linear groups, while all A packets are singletons, some ABV packets are not.


Visual Search at Alibaba. This paper introduces the large scale visual search algorithm and system infrastructure at Alibaba. The following challenges are discussed under the E commercial circumstance at Alibaba (a) how to handle heterogeneous image data and bridge the gap between real shot images from user query and the online images. (b) how to deal with large scale indexing for massive updating data. (c) how to train deep models for effective feature representation without huge human annotations. (d) how to improve the user engagement by considering the quality of the content. We take advantage of large image collection of Alibaba and state of the art deep learning techniques to perform visual search at scale. We present solutions and implementation details to overcome those problems and also share our learnings from building such a large scale commercial visual search engine. Specifically, model and search based fusion approach is introduced to effectively predict categories. Also, we propose a deep CNN model for joint detection and feature learning by mining user click behavior. The binary index engine is designed to scale up indexing without compromising recall and precision. Finally, we apply all the stages into an end to end system architecture, which can simultaneously achieve highly efficient and scalable performance adapting to real shot images. Extensive experiments demonstrate the advancement of each module in our system. We hope visual search at Alibaba becomes more widely incorporated into today's commercial applications.


On Positive Braids, Monodromy Groups and Framings. We associate to every positive braid a braid monodromy group, generalizing the geometric monodromy group of an isolated plane curve singularity. If the closure of the braid is a knot, we identify the corresponding group with a framed mapping class group. In particular, this gives a well defined knot invariant. As an application, we obtain that the geometric monodromy group of an irreducible singularity is determined by the genus and the Arf invariant of the associated knot.


Domain Generalization for Document Authentication against Practical Recapturing Attacks. Recapturing attack can be employed as a simple but effective anti forensic tool for digital document images. Inspired by the document inspection process that compares a questioned document against a reference sample, we proposed a document recapture detection scheme by employing Siamese network to compare and extract distinct features in a recapture document image. The proposed algorithm takes advantages of both metric learning and image forensic techniques. Instead of adopting Euclidean distance based loss function, we integrate the forensic similarity function with a triplet loss and a normalized softmax loss. After training with the proposed triplet selection strategy, the resulting feature embedding clusters the genuine samples near the reference while pushes the recaptured samples apart. In the experiment, we consider practical domain generalization problems, such as the variations in printing imaging devices, substrates, recapturing channels, and document types. To evaluate the robustness of different approaches, we benchmark some popular off the shelf machine learning based approaches, a state of the art document image detection scheme, and the proposed schemes with different network backbones under various experimental protocols. Experimental results show that the proposed schemes with different network backbones have consistently outperformed the state of the art approaches under different experimental settings. Specifically, under the most challenging scenario in our experiment, i.e., evaluation across different types of documents that produced by different devices, we have achieved less than 5.00 APCER (Attack Presentation Classification Error Rate) and 5.56 BPCER (Bona Fide Presentation Classification Error Rate) by the proposed network with ResNeXt101 backbone at 5 BPCER decision threshold.


Vision Based Mobile Robotics Obstacle Avoidance With Deep Reinforcement Learning. Obstacle avoidance is a fundamental and challenging problem for autonomous navigation of mobile robots. In this paper, we consider the problem of obstacle avoidance in simple 3D environments where the robot has to solely rely on a single monocular camera. In particular, we are interested in solving this problem without relying on localization, mapping, or planning techniques. Most of the existing work consider obstacle avoidance as two separate problems, namely obstacle detection, and control. Inspired by the recent advantages of deep reinforcement learning in Atari games and understanding highly complex situations in Go, we tackle the obstacle avoidance problem as a data driven end to end deep learning approach. Our approach takes raw images as input and generates control commands as output. We show that discrete action spaces are outperforming continuous control commands in terms of expected average reward in maze like environments. Furthermore, we show how to accelerate the learning and increase the robustness of the policy by incorporating predicted depth maps by a generative adversarial network.


Depth as Attention for Face Representation Learning. Face representation learning solutions have recently achieved great success for various applications such as verification and identification. However, face recognition approaches that are based purely on RGB images rely solely on intensity information, and therefore are more sensitive to facial variations, notably pose, occlusions, and environmental changes such as illumination and background. A novel depth guided attention mechanism is proposed for deep multi modal face recognition using low cost RGB D sensors. Our novel attention mechanism directs the deep network "where to look" for visual features in the RGB image by focusing the attention of the network using depth features extracted by a Convolution Neural Network (CNN). The depth features help the network focus on regions of the face in the RGB image that contains more prominent person specific information. Our attention mechanism then uses this correlation to generate an attention map for RGB images from the depth features extracted by CNN. We test our network on four public datasets, showing that the features obtained by our proposed solution yield better results on the Lock3DFace, CurtinFaces, IIIT D RGB D, and KaspAROV datasets which include challenging variations in pose, occlusion, illumination, expression, and time lapse. Our solution achieves average (increased) accuracies of 87.3 ( 5.0 ), 99.1 ( 0.9 ), 99.7 ( 0.6 ) and 95.3 ( 0.5 ) for the four datasets respectively, thereby improving the state of the art. We also perform additional experiments with thermal images, instead of depth images, showing the high generalization ability of our solution when adopting other modalities for guiding the attention mechanism instead of depth information


An Interpretive Constrained Linear Model for ResNet and MgNet. We propose a constrained linear data feature mapping model as an interpretable mathematical model for image classification using a convolutional neural network (CNN). From this viewpoint, we establish detailed connections between the traditional iterative schemes for linear systems and the architectures of the basic blocks of ResNet and MgNet type models. Using these connections, we present some modified ResNet models that compared with the original models have fewer parameters and yet can produce more accurate results, thereby demonstrating the validity of this constrained learning data feature mapping assumption. Based on this assumption, we further propose a general data feature iterative scheme to show the rationality of MgNet. We also provide a systematic numerical study on MgNet to show its success and advantages in image classification problems and demonstrate its advantages in comparison with established networks.


Pulse interactions in weakly nonlinear coherent optical communication links. The intrachannel interaction of pulses in weakly nonlinear coherent optical fiber lines is theoretically investigated. It is shown that the main contribution to the perturbation of the optical field comes from resonant interactions of ordered triplets of pulses. The structure of triplets is determined. The weight contributions of such interactions are calculated. A classification of interactions using Loeschian numbers is proposed. Using computer simulation, the dependence of the average energy of the optical field perturbations on the distance is shown. Based on the performed analysis, an effective algorithm is proposed for assessing the perturbations resulting from intrachannel interaction.


To Charge or To Sell? EV Pack Useful Life Estimation via LSTMs and Autoencoders. Electric Vehicles (EVs) are spreading fast as they promise to provide better performances and comfort, but above all, to help facing climate change. Despite their success, their cost is still a challenge. One of the most expensive components of EVs is lithium ion batteries, which became the standard for energy storage in a wide range of applications. Precisely estimating the Remaining Useful Life (RUL) of battery packs can open to their reuse and thus help to reduce the cost of EVs and improve sustainability. A correct RUL estimation can be used to quantify the residual market value of the battery pack. The customer can then decide to sell the battery when it still has a value, i.e., before it exceeds its end of life of the target application and can still be reused in a second domain without compromising safety and reliability. In this paper, we propose to use a Deep Learning approach based on LSTMs and Autoencoders to estimate the RUL of li ion batteries. Compared to what has been proposed so far in the literature, we employ measures to ensure the applicability of the method also in the real deployed application. Such measures include (1) avoid using non measurable variables as input, (2) employ appropriate datasets with wide variability and different conditions, (3) do not use cycles to define the RUL.


Insertion in constructed normal numbers. Defined by Borel, a real number is normal to an integer base b , greater than or equal to 2 , if in its base b expansion every block of digits occurs with the same limiting frequency as every other block of the same length. We consider the problem of insertion in constructed base b normal expansions to obtain normality to base (b 1) .


Degree distributions in AB random geometric graphs. In this paper, we provide degree distributions for AB random geometric graphs, in which points of type A connect to the closest k points of type B . The motivating example to derive such degree distributions is in 5G wireless networks with multi connectivity, where users connect to their closest k base stations. It is important to know how many users a particular base station serves, which gives the degree of that base station. To obtain these degree distributions, we investigate the distribution of area sizes of the k th order Voronoi cells of B points. Assuming that the A points are Poisson distributed, we investigate the amount of users connected to a certain B point, which is equal to the degree of this point. In the simple case where the B points are placed in an hexagonal grid, we show that all k th order Voronoi areas are equal and thus all degrees follow a Poisson distribution. However, this observation does not hold for Poisson distributed B points, for which we show that the degree distribution follows a compound Poisson Erlang distribution in the 1 dimensional case. We then approximate the degree distribution in the 2 dimensional case with a compound Poisson Gamma degree distribution and show that this one parameter fit performs well for different values of k . Moreover, we show that for increasing k , these degree distributions become more concentrated around the mean. This means that k connected AB random graphs balance the loads of B type nodes more evenly as k increases. Finally, we provide a case study on real data of base stations. We show that with little shadowing in the distances between users and base stations, the Poisson distribution does not capture the degree distribution of these data, especially for k 1 . However, under strong shadowing, our degree approximations perform quite good even for these non Poissonian location data.


Experimental Demonstration of a New Oscillator Concept Based on Degenerate Band Edge in Microstrip Circuit. The first experimental demonstration of an oscillator based on a periodic, resonant microstrip circuit with a degenerate band edge (DBE) is presented. The DBE is a fourth order exceptional degeneracy of the eigenmodes in a lossless periodic waveguide that is visible in the wavenumber frequency dispersion diagram, and the periodic microstrip behaves as a frequency selective medium. The presence of the DBE condition and the associated DBE resonance allow for a stable, single mode oscillation as well as stability with varying the load resistance.


Average size of Selmer group in large q limit. In this paper, we prove a function field analogue of Poonen Rains heuristics on the average size of p Selmer group. Let E be an elliptic curve defined over mathbb Z t . Then E is also defined over mathbb F q for any q of prime power. We show that for large enough q , the average size of the p Selmer groups over the family of quadratic twists of E over mathbb F q t is equal to p 1 for all but finitely many primes p . Namely, if we twist the curve in mathbb F q t by polynomials of fixed degree n and let both n and q approach to infinity, then the average rank of p Selmer group converges to p 1 .


On Best approximations to compact operators. We study best approximations to compact operators between Banach spaces and Hilbert spaces, from the point of view of Birkhoff James orthogonality and semi inner products. As an application of the present study, some distance formulae are presented in the space of compact operators. The special case of bounded linear functionals as compact operators is treated separately and some applications to best approximations in reflexive, strictly convex and smooth Banach spaces are discussed. An explicit example is presented in ell p n spaces, where 1 p infty, to illustrate the applicability of the methods developed in this article. A comparative analysis of the results presented in this article with the well known classical duality principle in approximation theory is conducted to demonstrate the advantage in the former case, from a computational point of view.


On sets of linear forms of maximal complexity. We present a uniform description of sets of m linear forms in n variables over the field of rational numbers whose computation requires m(n 1) additions.


Matrix valued positive definite kernels related to the generalized Aitken's integral for Gaussians. We introduce a method to construct general multivariate positive definite kernels on a nonempty set X that employs a prescribed bounded completely monotone function and special multivariate functions on X . The method is consistent with a generalized version of Aitken's integral formula for Gaussians. In the case where X is a cartesian product, the method produces nonseparable positive definite kernels that may be useful in multivariate interpolation. In addition, it can be interpreted as an abstract multivariate generalization of the well established Gneiting's model for constructing space time covariances commonly cited in the literature. Many parametric models discussed in statistics can be interpreted as particular cases of the method.


A Framework for Automatic Monitoring of Norms that regulate Time Constrained Actions. This paper addresses the problem of proposing a model of norms and a framework for automatically computing their violation or fulfilment. The proposed T NORM model can be used to express abstract norms able to regulate classes of actions that should or should not be performed in a temporal interval. We show how the model can be used to formalize obligations and prohibitions and for inhibiting them by introducing permissions and exemptions. The basic building blocks for norm specification consists of rules with suitably nested components. The activation condition, the regulated actions, and the temporal constrains of norms are specified using the W3C Web Ontology Language (OWL 2). Thanks to this choice, it is possible to use OWL reasoning for computing the effects that the logical implication between actions has on norms fulfilment or violation. The operational semantics of the T NORM model is specified by providing an unambiguous procedure for translating every norm and every exception into production rules.


Motivic cohomology of fat points in Milnor range via formal and rigid geometries. We present a formal scheme based cycle model for the motivic cohomology of the fat points defined by the truncated polynomial rings k t (t m) with m geq 2 , in one variable over a field k. We compute their Milnor range cycle class groups when the field has sufficiently many elements. With some aids from rigid analytic geometry, we prove that the resulting cycle class groups are isomorphic to the Milnor K groups of the truncated polynomial rings, generalizing a theorem of Nesterenko Suslin and Totaro. We deduce that their relative parts are isomorphic to the big de Rham Witt forms of k. In particular, this provides another cycle theoretic description of them after a result of K. R " u lling (J. Algebraic Geom., 2007).


Global smooth solutions to the 3D non resistive MHD equations with low regularity axisymmetric data. The purpose of this paper is to study the incompressible non resistive MHD equations in mathbb R 3 . We establish the global well posedness of classical solutions if the initial data is axially symmetric and the swirl components of the velocity and magnetic vorticity vanish. In particular, the special axially symmetric initial data can be arbitrarily large and satisfy low regularity assumptions.


New Binary Addition Tree Algorithm for the All Multiterminal Binary State Network Reliability Problem. Various real life applications, for example, Internet of Things, wireless sensor networks, smart grids, transportation networks, communication networks, social networks, and computer grid systems, are always modeled as network structures. The binary state network composed of binary state (e.g., functioning or failed) components (arcs and or nodes) is one of the most popular network structures. The two terminal network reliability is a success probability that the network is still functioning and can be calculated by verifying the connectivity between two specific nodes, and is an effective and popular technique for evaluating the performance of all types of networks. To obtain complete information for a making better decisions, a multi terminal network reliability extends the two specific nodes to a specific node subset in which all nodes are connected. In this study, a new algorithm called the all multiterminal BAT is proposed by revising the binary addition tree algorithm (BAT) and the layered search algorithm (LSA) to calculate all multi terminal reliabilities. The efficiency and effectiveness of the proposed all multiterminal BAT are analyzed from the perspective of time complexity and explained via numerical experiments to solve the all multiterminal network reliability problems.


ManiSkill Generalizable Manipulation Skill Benchmark with Large Scale Demonstrations. Object manipulation from 3D visual inputs poses many challenges on building generalizable perception and policy models. However, 3D assets in existing benchmarks mostly lack the diversity of 3D shapes that align with real world intra class complexity in topology and geometry. Here we propose SAPIEN Manipulation Skill Benchmark (ManiSkill) to benchmark manipulation skills over diverse objects in a full physics simulator. 3D assets in ManiSkill include large intra class topological and geometric variations. Tasks are carefully chosen to cover distinct types of manipulation challenges. Latest progress in 3D vision also makes us believe that we should customize the benchmark so that the challenge is inviting to researchers working on 3D deep learning. To this end, we simulate a moving panoramic camera that returns ego centric point clouds or RGB D images. In addition, we would like ManiSkill to serve a broad set of researchers interested in manipulation research. Besides supporting the learning of policies from interactions, we also support learning from demonstrations (LfD) methods, by providing a large number of high quality demonstrations ( 36,000 successful trajectories, 1.5M point cloud RGB D frames in total). We provide baselines using 3D deep learning and LfD algorithms. All code of our benchmark (simulator, environment, SDK, and baselines) is open sourced, and a challenge facing interdisciplinary researchers will be held based on the benchmark.


Beyond Self Supervision A Simple Yet Effective Network Distillation Alternative to Improve Backbones. Recently, research efforts have been concentrated on revealing how pre trained model makes a difference in neural network performance. Self supervision and semi supervised learning technologies have been extensively explored by the community and are proven to be of great potential in obtaining a powerful pre trained model. However, these models require huge training costs (i.e., hundreds of millions of images or training iterations). In this paper, we propose to improve existing baseline networks via knowledge distillation from off the shelf pre trained big powerful models. Different from existing knowledge distillation frameworks which require student model to be consistent with both soft label generated by teacher model and hard label annotated by humans, our solution performs distillation by only driving prediction of the student model consistent with that of the teacher model. Therefore, our distillation setting can get rid of manually labeled data and can be trained with extra unlabeled data to fully exploit capability of teacher model for better learning. We empirically find that such simple distillation settings perform extremely effective, for example, the top 1 accuracy on ImageNet 1k validation set of MobileNetV3 large and ResNet50 D can be significantly improved from 75.2 to 79 and 79.1 to 83 , respectively. We have also thoroughly analyzed what are dominant factors that affect the distillation performance and how they make a difference. Extensive downstream computer vision tasks, including transfer learning, object detection and semantic segmentation, can significantly benefit from the distilled pretrained models. All our experiments are implemented based on PaddlePaddle, codes and a series of improved pretrained models with ssld suffix are available in PaddleClas.


A decreasing scaling transition scheme from Adam to SGD. Adaptive gradient algorithm (AdaGrad) and its variants, such as RMSProp, Adam, AMSGrad, etc, have been widely used in deep learning. Although these algorithms are faster in the early phase of training, their generalization performance is often not as good as stochastic gradient descent (SGD). Hence, a trade off method of transforming Adam to SGD after a certain iteration to gain the merits of both algorithms is theoretically and practically significant. To that end, we propose a decreasing scaling transition scheme to achieve a smooth and stable transition from Adam to SGD, which is called DSTAdam. The convergence of the proposed DSTAdam is also proved in an online convex setting. Finally, the effectiveness of the DSTAdam is verified on the CIFAR 10 100 datasets. Our implementation is available at https github.com kunzeng DSTAdam.


Never Go Full Batch (in Stochastic Convex Optimization). We study the generalization performance of text full batch optimization algorithms for stochastic convex optimization these are first order methods that only access the exact gradient of the empirical risk (rather than gradients with respect to individual data points), that include a wide range of algorithms such as gradient descent, mirror descent, and their regularized and or accelerated variants. We provide a new separation result showing that, while algorithms such as stochastic gradient descent can generalize and optimize the population risk to within epsilon after O(1 epsilon 2) iterations, full batch methods either need at least Omega(1 epsilon 4) iterations or exhibit a dimension dependent sample complexity.


Signed (0,2) graphs with few eigenvalues and a symmetric spectrum. We investigate properties of signed graphs that have few distinct eigenvalues together with a symmetric spectrum. Our main contribution is to determine all signed (0,2) graphs with vertex degree at most 6 that have precisely two distinct eigenvalues pm lambda . Next, we consider to what extent induced subgraphs of signed graph with two distinct eigenvalues pm lambda are determined by their spectra. Lastly, we classify signed (0,2) graphs that have a symmetric spectrum with three distinct eigenvalues and give a partial classification for those with four distinct eigenvalues.


Mixing second and third order nonlinear interactions in nanophotonic lithium niobate waveguides. In this paper, we have investigated the interplay between the second and third order nonlinearities in lithium niobate waveguides with strong waveguide dispersion using uniform and linearly chirped poling patterns at input powers in the pico joule range. We have implemented the accurate unidirectional pulse propagation model to take into account all the possible nonlinear interactions inside these structures. In particular, the poling period has been designed to quasi phase match single and multiple sum and difference frequency generation processes. We have shown how the poling period can be used as an additional degree of freedom to tailor the output spectra of chip based nonlinear waveguides in an unprecedented way.


Real Variable Theory of Local Variable Hardy Spaces. In this paper, we give a complete real variable theory of local variable Hardy spaces. First, we present various real variable characterizations in terms of several local maximal functions. Next, the new atomic and the finite atomic decomposition for the local variable Hardy spaces are established. As an application, we also introduce the local variable Campanato space which is showed to be the dual space of the local variable Hardy spaces. Analogous to the homogeneous case, some equivalent definitions of the dual of local variable Hardy spaces are also considered. Finally, we show that the boundedness of inhomogeneous Calder 'on Zygmund singular integrals and local fractional integrals on local variable Hardy spaces and their duals.


Residual Tensor Train a Flexible and Efficient Approach for Learning Multiple Multilinear Correlations. Tensor Train (TT) approach has been successfully applied in the modelling of the multilinear interaction of features. Nevertheless, the existing models lack flexibility and generalizability, as they only model a single type of high order correlation. In practice, multiple multilinear correlations may exist within the features. In this paper, we present a novel Residual Tensor Train (ResTT) which integrates the merits of TT and residual structure to capture the multilinear feature correlations, from low to higher orders, within the same model. In particular, we prove that the fully connected layer in neural networks and the Volterra series can be taken as special cases of ResTT. Furthermore, we derive the rule for weight initialization that stabilizes the training of ResTT based on a mean field analysis. We prove that such a rule is much more relaxed than that of TT, which means ResTT can easily address the vanishing and exploding gradient problem that exists in the current TT models. Numerical experiments demonstrate that ResTT outperforms the state of the art tensor network approaches, and is competitive with the benchmark deep learning models on MNIST and Fashion MNIST datasets.


Learning Rays via Deep Neural Network in a Ray based IPDG Method for High Frequency Helmholtz Equations in Inhomogeneous Media. We develop a deep learning approach to extract ray directions at discrete locations by analyzing highly oscillatory wave fields. A deep neural network is trained on a set of local plane wave fields to predict ray directions at discrete locations. The resulting deep neural network is then applied to a reduced frequency Helmholtz solution to extract the directions, which are further incorporated into a ray based interior penalty discontinuous Galerkin (IPDG) method to solve the Helmholtz equations at higher frequencies. In this way, we observe no apparent pollution effects in the resulting Helmholtz solutions in inhomogeneous media. Our 2D and 3D numerical results show that the proposed scheme is very efficient and yields highly accurate solutions.


Training dataset and dictionary sizes matter in BERT models the case of Baltic languages. Large pretrained masked language models have become state of the art solutions for many NLP problems. While studies have shown that monolingual models produce better results than multilingual models, the training datasets must be sufficiently large. We trained a trilingual LitLat BERT like model for Lithuanian, Latvian, and English, and a monolingual Est RoBERTa model for Estonian. We evaluate their performance on four downstream tasks named entity recognition, dependency parsing, part of speech tagging, and word analogy. To analyze the importance of focusing on a single language and the importance of a large training set, we compare created models with existing monolingual and multilingual BERT models for Estonian, Latvian, and Lithuanian. The results show that the newly created LitLat BERT and Est RoBERTa models improve the results of existing models on all tested tasks in most situations.


Machine Learning in LiDAR 3D point clouds. LiDAR point clouds contain measurements of complicated natural scenes and can be used to update digital elevation models, glacial monitoring, detecting faults and measuring uplift detecting, forest inventory, detect shoreline and beach volume changes, landslide risk analysis, habitat mapping, and urban development, among others. A very important application is the classification of the 3D cloud into elementary classes. For example, it can be used to differentiate between vegetation, man made structures, and water. Our goal is to present a preliminary comparison study for the classification of 3D point cloud LiDAR data that includes several types of feature engineering. In particular, we demonstrate that providing context by augmenting each point in the LiDAR point cloud with information about its neighboring points can improve the performance of downstream learning algorithms. We also experiment with several dimension reduction strategies, ranging from Principal Component Analysis (PCA) to neural network based auto encoders, and demonstrate how they affect classification performance in LiDAR point clouds. For instance, we observe that combining feature engineering with a dimension reduction a method such as PCA, there is an improvement in the accuracy of the classification with respect to doing a straightforward classification with the raw data.


Unsteady pulsating flowfield over spiked axisymmetric forebody at hypersonic flows. The paper gives experimental observations on the hypersonic flow past an axisymmetric flat face cylinder with a protruding sharp tip spike at a freestream Mach number of M infty 8.16 at two different freestream Reynolds numbers based on the base body diameter ( Re D 0.76 times 10 6 , and 3.05 times 10 6 ). Furthermore, modal analysis is done on schlieren images to understand the flow dynamics parallel with the unsteady pressure measurements. The protruding spike of length to base body diameter ratio of l D 1 creates a familiar unsteady flowfield called 'pulsation.' Pressure loading and fluctuation intensity at two different Re D cases are calculated. A maximum drop of 98.24 is observed in both parameters between the high and low ReD cases. Based on the analysis, a difference in the pulsation characteristics are noticed, which arise from two vortical zones, each from a system of two lambda ' shocks formed during the collapse' phase ahead of the base body. The interaction of shedding vortices from the lambda shocks' triple points, along with the rotating stationary waves, contributes to the asymmetric high pressure loading and the observation of shock pulsation on the flat face cylinder. The vortical interactions form the second dominant spatial mode with a temporal mode carrying a dimensionless frequency ( f 2D u infty approx 0.34 ) almost twice that of the fundamental frequency ( f 1D u infty approx 0.17 ). The observed frequencies are invariant irrespective of the ReD cases. However, for the high frequency range, the spectral pressure decay is observed to follow an inverse and 7 3 law for the low and high Re D cases, respectively.


Facilitating Knowledge Sharing from Domain Experts to Data Scientists for Building NLP Models. Data scientists face a steep learning curve in understanding a new domain for which they want to build machine learning (ML) models. While input from domain experts could offer valuable help, such input is often limited, expensive, and generally not in a form readily consumable by a model development pipeline. In this paper, we propose Ziva, a framework to guide domain experts in sharing essential domain knowledge to data scientists for building NLP models. With Ziva, experts are able to distill and share their domain knowledge using domain concept extractors and five types of label justification over a representative data sample. The design of Ziva is informed by preliminary interviews with data scientists, in order to understand current practices of domain knowledge acquisition process for ML development projects. To assess our design, we run a mix method case study to evaluate how Ziva can facilitate interaction of domain experts and data scientists. Our results highlight that (1) domain experts are able to use Ziva to provide rich domain knowledge, while maintaining low mental load and stress levels and (2) data scientists find Ziva's output helpful for learning essential information about the domain, offering scalability of information, and lowering the burden on domain experts to share knowledge. We conclude this work by experimenting with building NLP models using the Ziva output by our case study.


Banach spaces containing c 0 and elements in the fourth dual. A recent result of T. Abrahamsen, P. H 'ajek and S. Troyanski states that a separable Banach space is almost square if and only if there exists h in S X such that x h max x ,1 for all x in X . The proof passes through a sequential version of being almost square which we call being textit sequentially almost square . In this article we study these conditions in the nonseparable setting. On one hand, we show that a Banach space X contains a copy of c 0 if and only if there exists an equivalent renorming cdot on X for which there exists h in S X such that x h max x ,1 for every x in X . On the other hand, although it is unclear whether the aforementioned result of T. Abrahamsen et al. holds in the nonseparable setting, we show that, under the existence of selective ultrafilters, if X is a sequentially almost square Banach space then there exists h in S X such that x h max x ,1 for all x in X .


Well tuned Simple Nets Excel on Tabular Datasets. Tabular datasets are the last "unconquered castle" for deep learning, with traditional ML methods like Gradient Boosted Decision Trees still performing strongly even against recent specialized neural architectures. In this paper, we hypothesize that the key to boosting the performance of neural networks lies in rethinking the joint and simultaneous application of a large set of modern regularization techniques. As a result, we propose regularizing plain Multilayer Perceptron (MLP) networks by searching for the optimal combination cocktail of 13 regularization techniques for each dataset using a joint optimization over the decision on which regularizers to apply and their subsidiary hyperparameters. We empirically assess the impact of these regularization cocktails for MLPs in a large scale empirical study comprising 40 tabular datasets and demonstrate that (i) well regularized plain MLPs significantly outperform recent state of the art specialized neural network architectures, and (ii) they even outperform strong traditional ML methods, such as XGBoost.


Schema Aware Deep Graph Convolutional Networks for Heterogeneous Graphs. Graph convolutional network (GCN) based approaches have achieved significant progress for solving complex, graph structured problems. GCNs incorporate the graph structure information and the node (or edge) features through message passing and computes 'deep' node representations. Despite significant progress in the field, designing GCN architectures for heterogeneous graphs still remains an open challenge. Due to the schema of a heterogeneous graph, useful information may reside multiple hops away. A key question is how to perform message passing to incorporate information of neighbors multiple hops away while avoiding the well known over smoothing problem in GCNs. To address this question, we propose our GCN framework 'Deep Heterogeneous Graph Convolutional Network (DHGCN)', which takes advantage of the schema of a heterogeneous graph and uses a hierarchical approach to effectively utilize information many hops away. It first computes representations of the target nodes based on their 'schema derived ego network' (SEN). It then links the nodes of the same type with various pre defined metapaths and performs message passing along these links to compute final node representations. Our design choices naturally capture the way a heterogeneous graph is generated from the schema. The experimental results on real and synthetic datasets corroborate the design choice and illustrate the performance gains relative to competing alternatives.


Guerrilla Science creating scientific instruments in high tropicallity' conditions. The 1980s was a flourishing time for Cuban physics, with various achievements ranging from the design of several experiments to be performed by a Cuban cosmonaut in 1980, to the synthesis of the first superconductor with critical temperature above 77 K shortly after being originally reported by US scientists. By the early 1990s, there was a profound economic crisis in the country. The situation strongly affected the availability of scientific instruments to Cuban physicists a process that continues today. Doing science in such High Tropicallity' scenario has been challenging. Many Cuban researchers have been forced to create competitive instruments with minimal resources. Here we put examples of some of these experimental setups, created by members of the Group of Complex Systems and Statistical Physics, University of Habana. Most of them involve the recycling of old laboratory instruments, and even the use of daily life devices.


Machine Learning and Ensemble Approach Onto Predicting Heart Disease. The four essential chambers of one's heart that lie in the thoracic cavity are crucial for one's survival, yet ironically prove to be the most vulnerable. Cardiovascular disease (CVD) also commonly referred to as heart disease has steadily grown to the leading cause of death amongst humans over the past few decades. Taking this concerning statistic into consideration, it is evident that patients suffering from CVDs need a quick and correct diagnosis in order to facilitate early treatment to lessen the chances of fatality. This paper attempts to utilize the data provided to train classification models such as Logistic Regression, K Nearest Neighbors, Support Vector Machine, Decision Tree, Gaussian Naive Bayes, Random Forest, and Multi Layer Perceptron (Artificial Neural Network) and eventually using a soft voting ensemble technique in order to attain as many correct diagnoses as possible.


Augmenting Decompiler Output with Learned Variable Names and Types. A common tool used by security professionals for reverse engineering binaries found in the wild is the decompiler. A decompiler attempts to reverse compilation, transforming a binary to a higher level language such as C. High level languages ease reasoning about programs by providing useful abstractions such as loops, typed variables, and comments, but these abstractions are lost during compilation. Decompilers are able to deterministically reconstruct structural properties of code, but comments, variable names, and custom variable types are technically impossible to recover. In this paper we present DIRTY (DecompIled variable ReTYper), a novel technique for improving the quality of decompiler output that automatically generates meaningful variable names and types. Empirical evaluation on a novel dataset of C code mined from GitHub shows that DIRTY outperforms prior work approaches by a sizable margin, recovering the original names written by developers 66.4 of the time and the original types 75.8 of the time.


Degenerating products of flag varieties and applications to the Breuil Mezard conjecture. We consider closed subschemes in the affine grassmannian obtained by degenerating e fold products of flag varieties, embedded via a tuple of dominant cocharacters. For G operatorname GL 2 , and cocharacters small relative to the characteristic, we relate the cycles of these degenerations to the representation theory of G . We then show that these degenerations smoothly model the geometry of (the special fibre of) low weight crystalline subspaces inside the Emerton Gee stack classifying p adic representations of the Galois group of a finite extension of mathbb Q p . As an application we prove new cases of the Breuil M 'ezard conjecture in dimension two.


A General Purpose Transpiler for Fully Homomorphic Encryption. Fully homomorphic encryption (FHE) is an encryption scheme which enables computation on encrypted data without revealing the underlying data. While there have been many advances in the field of FHE, developing programs using FHE still requires expertise in cryptography. In this white paper, we present a fully homomorphic encryption transpiler that allows developers to convert high level code (e.g., C ) that works on unencrypted data into high level code that operates on encrypted data. Thus, our transpiler makes transformations possible on encrypted data. Our transpiler builds on Google's open source XLS SDK (https github.com google xls) and uses an off the shelf FHE library, TFHE (https tfhe.github.io tfhe ), to perform low level FHE operations. The transpiler design is modular, which means the underlying FHE library as well as the high level input and output languages can vary. This modularity will help accelerate FHE research by providing an easy way to compare arbitrary programs in different FHE schemes side by side. We hope this lays the groundwork for eventual easy adoption of FHE by software developers. As a proof of concept, we are releasing an experimental transpiler (https github.com google fully homomorphic encryption tree main transpiler) as open source software.


Uniqueness of convex ancient solutions to hypersurface flows. We show that every convex ancient solution of mean curvature flow with Type I curvature growth is either spherical, cylindrical, or planar. We then prove the corresponding statement for flows by a natural class of curvature functions which are convex or concave in the second fundamental form. Neither of these results assumes interior noncollapsing.


On spherical 4 distance 7 designs. We investigate spherical 4 distance 7 designs by studying their distance distributions. We compute these distance distributions and use their product (an integer) to derive certain divisibility conditions relating the dimension n and the cardinality M of our designs. It follows that n divides 12M and n 1 divides 4M 2 . This result provides a good base for computer experiments to support the folklore conjecture that the only spherical 4 distance 7 designs are the tight spherical 7 designs. We then proceed with a computer assisted proof of this conjecture in all dimensions n leq 1000 .


Consensus Formation in First Order Graphon Models with Time Varying Topologies. In this article, we investigate the asymptotic formation of consensus for several classes of time dependent cooperative graphon dynamics. After motivating the use of this type of macroscopic model to describe multi agent systems, we adapt the classical notion of scrambling coefficient to this setting, and leverage it to establish sufficient conditions ensuring the exponential convergence to consensus with respect to the L infty norm topology. We then shift our attention to consensus formation expressed in terms of the L 2 norm, and prove three different consensus result for symmetric, balanced and strongly connected topologies, which involve a suitable generalisation of the notion of algebraic connectivity to this infinite dimensional framework. We then show that, just as in the finite dimensional setting, the notion of algebraic connectivity that we propose encodes information about the connectivity properties of the underlying interaction topology. We finally use the corresponding results to shed some light on the relation between L 2 and L infty consensus formation, and illustrate our contributions by a series of numerical simulations.


On the RND under Heston's stochastic volatility model. We consider Heston's (1993) stochastic volatility model for valuation of European options to which (semi) closed form solutions are available and are given in terms of characteristic functions. We prove that the class of scale parameter distributions with mean being the forward spot price satisfies Heston's solution. Thus, we show that any member of this class could be used for the direct risk neutral valuation of the option price under Heston's SV model. In fact, we also show that any RND with mean being the forward spot price that satisfies Hestons' option valuation solution, must be a member of a scale family of distributions in that mean. As particular examples, we show that one parameter versions of the it Log Normal, Inverse Gaussian, Gamma, Weibull and the it Inverse Weibull distributions are all members of this class and thus provide explicit risk neutral densities (RND) for Heston's pricing model. We demonstrate, via exact calculations and Monte Carlo simulations, the applicability and suitability of these explicit RNDs using already published Index data with a calibrated Heston model (S P500, Bakshi, Cao and Chen (1997), and ODAX, Mr 'azek and Posp 'i v s il (2017)), as well as current option market data (AMD).


C 1, alpha rectifiability in low codimension in Heisenberg groups. A natural notion of higher order rectifiability is introduced for subsets of Heisenberg groups mathbb H n in terms of covering a set almost everywhere by a countable union of ( mathbf C H 1, alpha , mathbb H ) regular surfaces, for some 0 alpha leq 1 . We prove that a sufficient condition for C 1, alpha rectifiability of low codimensional subsets in Heisenberg groups is the almost everywhere existence of suitable approximate tangent paraboloids.


Demonstrating Cloth Folding to Robots Design and Evaluation of a 2D and a 3D User Interface. An appropriate user interface to collect human demonstration data for deformable object manipulation has been mostly overlooked in the literature. We present an interaction design for demonstrating cloth folding to robots. Users choose pick and place points on the cloth and can preview a visualization of a simulated cloth before real robot execution. Two interfaces are proposed A 2D display and mouse interface where points are placed by clicking on an image of the cloth, and a 3D Augmented Reality interface where the chosen points are placed by hand gestures. We conduct a user study with 18 participants, in which each user completed two sequential folds to achieve a cloth goal shape. Results show that while both interfaces were acceptable, the 3D interface was found to be more suitable for understanding the task, and the 2D interface suitable for repetition. Results also found that fold previews improve three key metrics task efficiency, the ability to predict the final shape of the cloth and overall user satisfaction.


Conversational Entity Linking Problem Definition and Datasets. Machine understanding of user utterances in conversational systems is of utmost importance for enabling engaging and meaningful conversations with users. Entity Linking (EL) is one of the means of text understanding, with proven efficacy for various downstream tasks in information retrieval. In this paper, we study entity linking for conversational systems. To develop a better understanding of what EL in a conversational setting entails, we analyze a large number of dialogues from existing conversational datasets and annotate references to concepts, named entities, and personal entities using crowdsourcing. Based on the annotated dialogues, we identify the main characteristics of conversational entity linking. Further, we report on the performance of traditional EL systems on our Conversational Entity Linking dataset, ConEL, and present an extension to these methods to better fit the conversational setting. The resources released with this paper include annotated datasets, detailed descriptions of crowdsourcing setups, as well as the annotations produced by various EL systems. These new resources allow for an investigation of how the role of entities in conversations is different from that in documents or isolated short text utterances like queries and tweets, and complement existing conversational datasets.