Causal Modeling With Infinitely Many Variables. Structural equations models (SEMs) are perhaps the most commonly used framework for modeling causality. However, as we show, naively extending this framework to infinitely many variables, which is necessary, for example, to model dynamical systems, runs into several problems. We introduce GSEMs (generalized SEMs), a flexible generalization of SEMs that directly specify the results of interventions, in which (1) systems of differential equations can be represented in a natural and intuitive manner, (2) certain natural situations, which cannot be represented by SEMs at all, can be represented easily, (3) the definition of actual causality in SEMs carries over essentially without change.


A Time Resolved Imaging System for the Diagnosis of X ray Self Emission in High Energy Density Physics Experiments. A diagnostic capable of recording spatially and temporally resolved X ray self emission data was developed to characterise experiments on the MAGPIE pulsed power generator. The diagnostic used two separate imaging systems A pinhole imaging system with two dimensional spatial resolution and a slit imaging system with one dimensional spatial resolution. The two dimensional imaging system imaged light onto image plate. The one dimensional imaging system imaged light onto the same piece of image plate and a linear array of silicon photodiodes. This design allowed the cross comparison of different images, allowing a picture of the spatial and temporal distribution of X ray self emission to be established. The design was tested in a series of pulsed power driven magnetic reconnection experiments.


PipAttack Poisoning Federated Recommender Systems forManipulating Item Promotion. Due to the growing privacy concerns, decentralization emerges rapidly in personalized services, especially recommendation. Also, recent studies have shown that centralized models are vulnerable to poisoning attacks, compromising their integrity. In the context of recommender systems, a typical goal of such poisoning attacks is to promote the adversary's target items by interfering with the training dataset and or process. Hence, a common practice is to subsume recommender systems under the decentralized federated learning paradigm, which enables all user devices to collaboratively learn a global recommender while retaining all the sensitive data locally. Without exposing the full knowledge of the recommender and entire dataset to end users, such federated recommendation is widely regarded safe' towards poisoning attacks. In this paper, we present a systematic approach to backdooring federated recommender systems for targeted item promotion. The core tactic is to take advantage of the inherent popularity bias that commonly exists in data driven recommenders. As popular items are more likely to appear in the recommendation list, our innovatively designed attack model enables the target item to have the characteristics of popular items in the embedding space. Then, by uploading carefully crafted gradients via a small number of malicious users during the model update, we can effectively increase the exposure rate of a target (unpopular) item in the resulted federated recommender. Evaluations on two real world datasets show that 1) our attack model significantly boosts the exposure rate of the target item in a stealthy way, without harming the accuracy of the poisoned recommender and 2) existing defenses are not effective enough, highlighting the need for new defenses against our local model poisoning attacks to federated recommender systems.


On the Feasibility of Predicting Questions being Forgotten in Stack Overflow. For their attractiveness, comprehensiveness and dynamic coverage of relevant topics, community based question answering sites such as Stack Overflow heavily rely on the engagement of their communities Questions on new technologies, technology features as well as technology versions come up and have to be answered as technology evolves (and as community members gather experience with it). At the same time, other questions cease in importance over time, finally becoming irrelevant to users. Beyond filtering low quality questions, "forgetting" questions, which have become redundant, is an important step for keeping the Stack Overflow content concise and useful. In this work, we study this managed forgetting task for Stack Overflow. Our work is based on data from more than a decade (2008 2019) covering 18.1M questions, that are made publicly available by the site itself. For establishing a deeper understanding, we first analyze and characterize the set of questions about to be forgotten, i.e., questions that get a considerable number of views in the current period but become unattractive in the near future. Subsequently, we examine the capability of a wide range of features in predicting such forgotten questions in different categories. We find some categories in which those questions are more predictable. We also discover that the text based features are surprisingly not helpful in this prediction task, while the meta information is much more predictive.


Network wide link travel time and station waiting time estimation using automatic fare collection data A computational graph approach. Urban rail transit (URT) system plays a dominating role in many megacities like Beijing and Hong Kong. Due to its important role and complex nature, it is always in great need for public agencies to better understand the performance of the URT system. This paper focuses on an essential and hard problem to estimate the network wide link travel time and station waiting time using the automatic fare collection (AFC) data in the URT system, which is beneficial to better understand the system wide real time operation state. The emerging data driven techniques, such as computational graph (CG) models in the machine learning field, provide a new solution for solving this problem. In this study, we first formulate a data driven estimation optimization framework to estimate the link travel time and station waiting time. Then, we cast the estimation optimization model into a CG framework to solve the optimization problem and obtain the estimation results. The methodology is verified on a synthetic URT network and applied to a real world URT network using the synthetic and real world AFC data, respectively. Results show the robustness and effectiveness of the CG based framework. To the best of our knowledge, this is the first time that the CG is applied to the URT. This study can provide critical insights to better understand the operational state in URT.


Shuffle algebras for quivers as quantum groups. We define a quantum loop group mathbf U Q associated to an arbitrary quiver Q (I,E) and maximal set of deformation parameters, with generators indexed by I times mathbb Z and some explicit quadratic and cubic relations. We prove that mathbf U Q is isomorphic to the (generic, small) shuffle algebra associated to the quiver Q and hence, by Neg21a , to the localized K theoretic Hall algebra of Q . For the quiver with one vertex and g loops, this yields a presentation of the spherical Hall algebra of a (generic) smooth projective curve of genus g (invoking the results of SV12 ). We extend the above results to the case of non generic parameters satisfying a certain natural metric condition. As an application, we obtain a description by generators and relations of the subalgebra generated by absolutely cuspidal eigenforms of the Hall algebra of an arbitrary smooth projective curve (invoking the results of KSV17 ).


Improving Adversarial Transferability with Gradient Refining. Deep neural networks are vulnerable to adversarial examples, which are crafted by adding human imperceptible perturbations to original images. Most existing adversarial attack methods achieve nearly 100 attack success rates under the white box setting, but only achieve relatively low attack success rates under the black box setting. To improve the transferability of adversarial examples for the black box setting, several methods have been proposed, e.g., input diversity, translation invariant attack, and momentum based attack. In this paper, we propose a method named Gradient Refining, which can further improve the adversarial transferability by correcting useless gradients introduced by input diversity through multiple transformations. Our method is generally applicable to many gradient based attack methods combined with input diversity. Extensive experiments are conducted on the ImageNet dataset and our method can achieve an average transfer success rate of 82.07 for three different models under single model setting, which outperforms the other state of the art methods by a large margin of 6.0 averagely. And we have applied the proposed method to the competition CVPR 2021 Unrestricted Adversarial Attacks on ImageNet organized by Alibaba and won the second place in attack success rates among 1558 teams.


Stateless actor critic for instance segmentation with high level priors. Instance segmentation is an important computer vision problem which remains challenging despite impressive recent advances due to deep learning based methods. Given sufficient training data, fully supervised methods can yield excellent performance, but annotation of ground truth data remains a major bottleneck, especially for biomedical applications where it has to be performed by domain experts. The amount of labels required can be drastically reduced by using rules derived from prior knowledge to guide the segmentation. However, these rules are in general not differentiable and thus cannot be used with existing methods. Here, we relax this requirement by using stateless actor critic reinforcement learning, which enables non differentiable rewards. We formulate the instance segmentation problem as graph partitioning and the actor critic predicts the edge weights driven by the rewards, which are based on the conformity of segmented instances to high level priors on object shape, position or size. The experiments on toy and real datasets demonstrate that we can achieve excellent performance without any direct supervision based only on a rich set of priors.


Thermal corrections for positronium. Thermal corrections, including relativistic effects, for the positronium atom are discussed. The theoretical description of thermal corrections is carried out within the framework of relativistic quantum electrodynamics. As a result, thermal corrections to atomic energy levels with a fine and hyperfine structure and to the probabilities of annihilation of a positronium atom placed in a thermal environment (blackbody radiation) are taken into account. Numerical results are discussed throughout the paper in view of modern experiments and theoretical searches for verification of fundamental interactions.


Building Beyond HLS Graph Analysis and Others. High Level Synthesis has introduced reconfigurable logic to a new world that of software development. The newest wave of HLS tools has been successful, and the future looks bright. But is HLS the end all be all to FPGA acceleration? Is it enough to allow non experts to program FPGAs successfully, even when dealing with troublesome data structures and complex control flows such as those often encountered in graph algorithms? We take a look at the panorama of adoption of HLS by the software community, focusing on graph analysis in particular in order to generalise to textit FPGA unfriendly problems. We argue for the existence of shortcomings in current HLS development flows which hinder adoption, and present our perspective on the path forward, including how these issues may be remedied via higher level tooling.


New examples of Weierstrass semigroups associated with a double covering of a curve on a Hirzebruch surface of degree one. Let varphi Sigma 1 longrightarrow mathbb P 2 be a blow up at a point on mathbb P 2 . Let C be the proper transform of a smooth plane curve of degree d geq 4 by varphi , and let P be a point on C . Let pi tilde C longrightarrow C be a double covering branched along the reduced divisor on C obtained as the intersection of C and a reduced divisor in 2K Sigma 1 containing P . In this paper, we investigate the Weierstrass semigroup H( tilde P ) at the ramification point tilde P of pi over P , in the case where the intersection multiplicity at varphi(P) of varphi(C) and the tangent line at varphi(P) of varphi(C) is d 1 .


Personalized Rehabilitation Robotics based on Online Learning Control. The use of rehabilitation robotics in clinical applications gains increasing importance, due to therapeutic benefits and the ability to alleviate labor intensive works. However, their practical utility is dependent on the deployment of appropriate control algorithms, which adapt the level of task assistance according to each individual patient's need. Generally, the required personalization is achieved through manual tuning by clinicians, which is cumbersome and error prone. In this work we propose a novel online learning control architecture, which is able to personalize the control force at run time to each individual user. To this end, we deploy Gaussian process based online learning with previously unseen prediction and update rates. Finally, we evaluate our method in an experimental user study, where the learning controller is shown to provide personalized control, while also obtaining safe interaction forces.


Probabilistic Analysis of Operating Modes in Cache Enabled Full Duplex D2D Networks. With the extensive acquisition of various mobile applications, cellular networks are facing challenges due to exponentially growing demand for high data rate, which causes a great burden on mobile core networks and backhaul links. Cache enabled Device to Device (D2D) communication, which is recognized as one of the key enablers of the fifth generation (5G) cellular network, is a promising solution to alleviate this problem. However, conventional half duplex (HD) communication may not be sufficient to provide fast enough content delivery over D2D links in order to meet strict latency targets of emerging D2D applications. In band full duplex (FD), with its capability of allowing simultaneous transmission and reception, can provide more content delivery opportunities, thus resulting improved spectral efficiency and latency reduction. However, given the random nature of the cached contents in user devices and users' random requests, it is unlikely to consider all involving nodes in content exchange collaborations as a pure HD or FD network. In this paper, we aim to analyze the caching perspective of a finite network of D2D nodes in which each node is endowed with FD capability and utilize a more realistic caching policy. We model and analyze all possible operating modes for an arbitrary device, which we compute the probability of occurrence of each mode along with the Probability Mass Functions (PMFs) of nodes that are operating in all possible modes. Our analysis concretely quantize all possible outcomes that strongly depend on the random nature of the caching parameters, yielding to have an accurate insight on the caching performance and all possible outcomes of the cache enabled FD D2D network.


Weierstrass Representations of Lorentzian Minimal Surfaces in mathbb R 4 2 . The minimal Lorentzian surfaces in mathbb R 4 2 whose first normal space is two dimensional and whose Gauss curvature K and normal curvature varkappa satisfy K 2 varkappa 2 0 are called minimal Lorentzian surfaces of general type. These surfaces admit canonical parameters and with respect to such parameters are determined uniquely up to a motion in mathbb R 4 2 by the curvatures K and varkappa satisfying a system of two natural PDEs. In the present paper we study minimal Lorentzian surfaces in mathbb R 4 2 and find a Weierstrass representation with respect to isothermal parameters of any minimal surface with two dimensional first normal space. We also obtain a Weierstrass representation with respect to canonical parameters of any minimal Lorentzian surface of general type and solve explicitly the system of natural PDEs expressing any solution to this system by means of four real functions of one variable.


The b weight distribution for MDS codes. For a positive integer b ge2 , the b symbol code is a new coding framework proposed to combat b errors in b symbol read channels. Especially, a 2 symbol code is called a symbol pair code. Remarkably, a classical maximum distance separable (MDS) code is also an MDS b symbol code. Recently, for any MDS code mathcal C , Ma and Luo determined the symbol pair weight distribution for mathcal C . In this paper, by calculating the number of codewords in mathcal C with special shape, we obtain the b weight distribution for mathcal C , and then generalize Theorem 1 in cite ML .


Managing Bias in Human Annotated Data Moving Beyond Bias Removal. Due to the widespread use of data powered systems in our everyday lives, the notions of bias and fairness gained significant attention among researchers and practitioners, in both industry and academia. Such issues typically emerge from the data, which comes with varying levels of quality, used to train systems. With the commercialization and employment of such systems that are sometimes delegated to make life changing decisions, a significant effort is being made towards the identification and removal of possible sources of bias that may surface to the final end user. In this position paper, we instead argue that bias is not something that should necessarily be removed in all cases, and the attention and effort should shift from bias removal to the identification, measurement, indexing, surfacing, and adjustment of bias, which we name bias management. We argue that if correctly managed, bias can be a resource that can be made transparent to the the users and empower them to make informed choices about their experience with the system.


DRO Deep Recurrent Optimizer for Structure from Motion. There are increasing interests of studying the structure from motion (SfM) problem with machine learning techniques. While earlier methods directly learn a mapping from images to depth maps and camera poses, more recent works enforce multi view geometry constraints through optimization embedded in the learning framework. This paper presents a novel optimization method based on recurrent neural networks to further exploit the potential of neural networks in SfM. Specifically, our neural optimizer alternately updates the depth and camera poses through iterations to minimize a feature metric cost, and two gated recurrent units iteratively improve the results by tracing historical information. In this way, our network is a gradient free zeroth order optimizer designed for SfM and can be applied to both supervised and self supervised SfM. Extensive experimental results demonstrate that our method outperforms previous methods and is more efficient in computation and memory consumption than cost volume based methods. In particular, our self supervised method outperforms previous supervised methods on the KITTI and ScanNet datasets. Our source code is available at https github.com aliyun dro sfm.


Deep Evolutionary Learning for Molecular Design. In this paper, we propose a deep evolutionary learning (DEL) process that integrates fragment based deep generative model and multi objective evolutionary computation for molecular design. Our approach enables (1) evolutionary operations in the latent space of the generative model, rather than the structural space, to generate novel promising molecular structures for the next evolutionary generation, and (2) generative model fine tuning using newly generated high quality samples. Thus, DEL implements a data model co evolution concept which improves both sample population and generative model learning. Experiments on two public datasets indicate that sample population obtained by DEL exhibits improved property distributions, and dominates samples generated by multi objective Bayesian optimization algorithms.


Machine learning accelerated particle in cell plasma simulations. Particle In Cell (PIC) methods are frequently used for kinetic, high fidelity simulations of plasmas. Implicit formulations of PIC algorithms feature strong conservation properties, up to numerical round off errors, and are not subject to time step limitations which make them an attractive candidate to use in simulations fusion plasmas. Currently they remain prohibitively expensive for high fidelity simulation of macroscopic plasmas. We investigate how amortized solvers can be incorporated with PIC methods for simulations of plasmas. Incorporated into the amortized solver, a neural network predicts a vector space that entails an approximate solution of the PIC system. The network uses only fluid moments and the electric field as input and its output is used to augment the vector space of an iterative linear solver. We find that this approach reduces the average number of required solver iterations by about 25 when simulating electron plasma oscillations. This novel approach may allow to accelerate implicit PIC simulations while retaining all conservation laws and may also be appropriate for multi scale systems.


Open domain Topic Identification of Out of domain Utterances using Wikipedia. Users of spoken dialogue systems (SDS) expect high quality interactions across a wide range of diverse topics. However, the implementation of SDS capable of responding to every conceivable user utterance in an informative way is a challenging problem. Multi domain SDS must necessarily identify and deal with out of domain (OOD) utterances to generate appropriate responses as users do not always know in advance what domains the SDS can handle. To address this problem, we extend the current state of the art in multi domain SDS by estimating the topic of OOD utterances using external knowledge representation from Wikipedia. Experimental results on real human to human dialogues showed that our approach does not degrade domain prediction performance when compared to the base model. But more significantly, our joint training achieves more accurate predictions of the nearest Wikipedia article by up to about 30 when compared to the benchmarks.


On the Investment Strategies in Occupational Pension Plans. Demographic changes increase the necessity to base the pension system more and more on the second and the third pillar, namely the occupational and private pension plans this paper deals with Target Date Funds (TDFs), which are a typical investment opportunity for occupational pension planners. TDFs are usually identified with a decreasing fraction of wealth invested in equity (a so called glide path) as retirement comes closer, i.e., wealth is invested more risky the younger the saver is. We investigate whether this is actually optimal in the presence of non tradable income risk in a stochastic volatility environment. The retirement planning procedure is formulated as a stochastic optimization problem. We find it is the (random) contributions that induce the optimal path exhibiting a glide path structure, both in the constant and stochastic volatility environment. Moreover, the initial wealth and the initial contribution made to a retirement account strongly influence the fractional amount of wealth to be invested in risky assets. The risk aversion of an individual mainly determines the steepness of the glide path.


An aperiodic tiling made of one tile, a triangle. How many different tiles are needed at the minimum to create aperiodicity? Several tilings made of two tiles were discovered, the first one being by Penrose in the seventies. Since then, scientists discovered other aperiodic tilings made of two tiles, including the square triangle one, a tiling that has been particularly useful for the study of dodecagonal quasicrystals and soft matters. An open problem still exists Can one tile be sufficient to create aperiodicity? This is known as the ein stein problem. We present in this paper an aperiodic tiling made of one single tile an isosceles right triangle. The tile itself is not aperiodic and therefore not a solution to the ein stein problem but we present a set of substitution rules on the same tile that forces the tiling to be aperiodic. This paper presents its construction rules that proves its aperiodicity. We also show that this tiling offers an underlying dodecagonal structure close to the one of square triangle tiling.


The Role of Functional Programming in Management and Orchestration of Virtualized Network Resources Part II. Network Evolution and Design Principles. This is part II of the follow up lecture notes of the lectures given by the authors at the Three CO" (Composability, Comprehensibility, Correctness) Winter School held in Kov s ice, Slovakia, in January 2018, and Summer School held in Budapest, Hungary, in June 2019. In this part we explain the recent network evolution and the concept of virtualization, focusing on the management and orchestration of virtualized network resources. Network Functions Virtualization (NFV) is a new paradigm for changing the way networks are built and operated. Decoupling software implementation from network resources through a virtualization layer introduces a need for developing sets of NFV management and orchestration (MANO) functions. We discuss how this new point of view is highly inspired by the functional programming concepts. We provide examples and exercises on Open Stack virtual technology, and also discuss the challenges and problems inspired by telecommunication industry. Focus is on Reliable operation of Management and Orchestration functions of Virtualized resources. These notes provide an introduction to the subject, with the goal of explaining the necessity for new knowledge and skills in area of network programming. We introduce students with main problems and the network design principles, methods and techniques used for their solution. The worked examples and exercises serve students as the teaching material, from which they can learn how to use functional programming to effectively and efficiently coordinate management and orchestration functions in distributed complex systems using NFV.


Stabilizing Deep Q Learning with ConvNets and Vision Transformers under Data Augmentation. While agents trained by Reinforcement Learning (RL) can solve increasingly challenging tasks directly from visual observations, generalizing learned skills to novel environments remains very challenging. Extensive use of data augmentation is a promising technique for improving generalization in RL, but it is often found to decrease sample efficiency and can even lead to divergence. In this paper, we investigate causes of instability when using data augmentation in common off policy RL algorithms. We identify two problems, both rooted in high variance Q targets. Based on our findings, we propose a simple yet effective technique for stabilizing this class of algorithms under augmentation. We perform extensive empirical evaluation of image based RL using both ConvNets and Vision Transformers (ViT) on a family of benchmarks based on DeepMind Control Suite, as well as in robotic manipulation tasks. Our method greatly improves stability and sample efficiency of ConvNets under augmentation, and achieves generalization results competitive with state of the art methods for image based RL in environments with unseen visuals. We further show that our method scales to RL with ViT based architectures, and that data augmentation may be especially important in this setting.


"Zero Shot" Point Cloud Upsampling. Recent supervised point cloud upsampling methods are restricted by the size of training data and are limited in terms of covering all object shapes. Besides the challenges faced due to data acquisition, the networks also struggle to generalize on unseen records. In this paper, we present an internal point cloud upsampling approach at a holistic level referred to as "Zero Shot" Point Cloud Upsampling (ZSPU). Our approach is data agnostic and relies solely on the internal information provided by a particular point cloud without patching in both self training and testing phases. This single stream design significantly reduces the training time by learning the relation between low resolution (LR) point clouds and their high (original) resolution (HR) counterparts. This association will then provide super resolution (SR) outputs when original point clouds are loaded as input. ZSPU achieves competitive superior quantitative and qualitative performances on benchmark datasets when compared with other upsampling methods.


One Chatbot Per Person Creating Personalized Chatbots based on Implicit User Profiles. Personalized chatbots focus on endowing chatbots with a consistent personality to behave like real users, give more informative responses, and further act as personal assistants. Existing personalized approaches tried to incorporate several text descriptions as explicit user profiles. However, the acquisition of such explicit profiles is expensive and time consuming, thus being impractical for large scale real world applications. Moreover, the restricted predefined profile neglects the language behavior of a real user and cannot be automatically updated together with the change of user interests. In this paper, we propose to learn implicit user profiles automatically from large scale user dialogue history for building personalized chatbots. Specifically, leveraging the benefits of Transformer on language understanding, we train a personalized language model to construct a general user profile from the user's historical responses. To highlight the relevant historical responses to the input post, we further establish a key value memory network of historical post response pairs, and build a dynamic post aware user profile. The dynamic profile mainly describes what and how the user has responded to similar posts in history. To explicitly utilize users' frequently used words, we design a personalized decoder to fuse two decoding strategies, including generating a word from the generic vocabulary and copying one word from the user's personalized vocabulary. Experiments on two real world datasets show the significant improvement of our model compared with existing methods. Our code is available at https github.com zhengyima DHAP


Bound states in the continuum and Fano resonances in subwavelength resonator arrays. When wave scattering systems are subject to certain symmetries, resonant states may decouple from the far field continuum they remain localized to the structure and cannot be excited by incident waves from the far field. In this work, we use layer potential techniques to prove the existence of such states, known as bound states in the continuum, in systems of subwavelength resonators. When the symmetry is slightly broken, this resonant state can be excited from the far field. Remarkably, this may create asymmetric (Fano type) scattering behaviour where the transmission is fundamentally different for frequencies on either side of the resonant frequency. Using asymptotic analysis, we compute the scattering matrix of the system explicitly, thereby characterizing this Fano type transmission anomaly.


Training Conversational Agents with Generative Conversational Networks. Rich, open domain textual data available on the web resulted in great advancements for language processing. However, while that data may be suitable for language processing tasks, they are mostly non conversational, lacking many phenomena that appear in human interactions and this is one of the reasons why we still have many unsolved challenges in conversational AI. In this work, we attempt to address this by using Generative Conversational Networks to automatically generate data and train social conversational agents. We evaluate our approach on TopicalChat with automatic metrics and human evaluators, showing that with 10 of seed data it performs close to the baseline that uses 100 of the data.


The distance k dimension of graphs. The metric dimension, dim(G) , of a graph G is a graph parameter motivated by robot navigation that has been studied extensively. Let G be a graph with vertex set V(G) , and let d(x,y) denote the length of a shortest x y path in G . For a positive integer k and for distinct x,y in V(G) , let d k(x,y) min d(x,y), k 1 and let R k x,y z in V(G) d k(x,z) neq d k(y,z) . A subset S subseteq V(G) is a distance k resolving set of G if S cap R k x,y ge 1 for any pair of distinct x,y in V(G) , and the distance k dimension, dim k(G) , of G is the minimum cardinality over all distance k resolving sets of G . In this paper, we study the distance k dimension of graphs. We obtain some general bounds for distance k dimension. For all k ge 1 , we characterize connected graphs G of order n with dim k(G) ge n 2 . We determine dim k(G) when G is a cycle or a path. We also examine the effect of vertex or edge deletion on the distance k dimension of graphs.


Boosting Method in Approximately Solving Linear Programming with Fast Online Algorithm. In this paper, we develop a new algorithm combining the idea of boosting'' with the first order algorithm to approximately solve a class of (Integer) Linear programs(LPs) arisen in general resource allocation problems. Not only can this algorithm solve LPs directly, but also can be applied to accelerate the Column Generation method. As a direct solver, our algorithm achieves a provable O( sqrt n K ) optimality gap, where n is the number of variables and K is the number of data duplication bearing the same intuition as the boosting algorithm. We use numerical experiments to demonstrate the effectiveness of our algorithm and several variants.


Network Architecture Search for Face Enhancement. Various factors such as ambient lighting conditions, noise, motion blur, etc. affect the quality of captured face images. Poor quality face images often reduce the performance of face analysis and recognition systems. Hence, it is important to enhance the quality of face images collected in such conditions. We present a multi task face restoration network, called Network Architecture Search for Face Enhancement (NASFE), which can enhance poor quality face images containing a single degradation (i.e. noise or blur) or multiple degradations (noise blur low light). During training, NASFE uses clean face images of a person present in the degraded image to extract the identity information in terms of features for restoring the image. Furthermore, the network is guided by an identity loss so that the identity in formation is maintained in the restored image. Additionally, we propose a network architecture search based fusion network in NASFE which fuses the task specific features that are extracted using the task specific encoders. We introduce FFT op and deveiling operators in the fusion network to efficiently fuse the task specific features. Comprehensive experiments on synthetic and real images demonstrate that the proposed method outperforms many recent state of the art face restoration and enhancement methods in terms of quantitative and visual performance.


Effective calculation of the causal capacitance matrix of a multiconductor transmission line in the range of parameters by the method of moments. The paper proposes a technique of non uniform segmentation of multiconductor transmission lines with edge coupling. This technique allows obtaining causal capacitance matrices with the lowest computational costs on multivariate analysis in the range of the lines parameters. It was found that time savings could achieve 49 .


Exploring Classic and Neural Lexical Translation Models for Information Retrieval Interpretability, Effectiveness, and Efficiency Benefits. We study the utility of the lexical translation model (IBM Model 1) for English text retrieval, in particular, its neural variants that are trained end to end. We use the neural Model1 as an aggregator layer applied to context free or contextualized query document embeddings. This new approach to design a neural ranking system has benefits for effectiveness, efficiency, and interpretability. Specifically, we show that adding an interpretable neural Model 1 layer on top of BERT based contextualized embeddings (1) does not decrease accuracy and or efficiency and (2) may overcome the limitation on the maximum sequence length of existing BERT models. The context free neural Model 1 is less effective than a BERT based ranking model, but it can run efficiently on a CPU (without expensive index time precomputation or query time operations on large tensors). Using Model 1 we produced best neural and non neural runs on the MS MARCO document ranking leaderboard in late 2020.


Application of Neural Network Machine Learning to Solution of Black Scholes Equation. This paper presents a novel way to predict options price for one day in advance, utilizing the method of Quasi Reversibility for solving the Black Scholes equation. The Black Scholes equation solved forwards in time with Tikhonov regularization as an ill posed problem allows for extrapolation of option prices. This provides high accuracy results, which can be further improved by applying Neural Network Machine Learning to the solution of the Black Scholes equation as well as initial and boundary conditions and implied volatility. Using historical option and stock price data the results obtained from the method of Quasi Reversibility and Machine Learning method are compared in terms of accuracy, precision and recall. It is shown that these methods can be applied to the real world trading within a variety of trading strategies.


Representing ELMo embeddings as two dimensional text online. We describe a new addition to the WebVectors toolkit which is used to serve word embedding models over the Web. The new ELMoViz module adds support for contextualized embedding architectures, in particular for ELMo models. The provided visualizations follow the metaphor of two dimensional text' by showing lexical substitutes words which are most semantically similar in context to the words of the input sentence. The system allows the user to change the ELMo layers from which token embeddings are inferred. It also conveys corpus information about the query words and their lexical substitutes (namely their frequency tiers and parts of speech). The module is well integrated into the rest of the WebVectors toolkit, providing lexical hyperlinks to word representations in static embedding models. Two web services have already implemented the new functionality with pre trained ELMo models for Russian, Norwegian and English.


Existence of minimisers of variational problems posed in spaces of mixed smoothness. The present work constitutes a first step towards establishing a systematic framework for treating variational problems that depend on a given input function through a mixture of its derivatives of different orders in different directions. For a fixed vector mathbf a (a 1, ldots, a N) in mathbb N N and u colon mathbb R N supset Omega to mathbb R n we denote by nabla mathbf a u ( partial alpha u) langle alpha, mathbf a 1 rangle 1 the matrix whose i th row is composed of derivatives partial alpha u i of the i th component of the map u , and where the multi indices alpha satisfy langle alpha, mathbf a 1 rangle sum j 1 N frac alpha j a j 1 . We study functionals of the form mathrm W mathbf a ,p ( Omega mathbb R n) ni u mapsto int Omega F( nabla mathbf a u(x)) , mathrm d x, where mathrm W mathbf a ,p ( Omega mathbb R n) is an appropriate Sobolev space of mixed smoothness and F is the integrand. We study existence of minimisers of such functionals under prescribed Dirichlet boundary conditions. We characterise coercivity, lower semicontiuity, and envelopes of relaxation of such functionals, in terms of an appropriate generalisation of Morrey's quasiconvexity.


Firefly Neural Architecture Descent a General Approach for Growing Neural Networks. We propose firefly neural architecture descent, a general framework for progressively and dynamically growing neural networks to jointly optimize the networks' parameters and architectures. Our method works in a steepest descent fashion, which iteratively finds the best network within a functional neighborhood of the original network that includes a diverse set of candidate network structures. By using Taylor approximation, the optimal network structure in the neighborhood can be found with a greedy selection procedure. We show that firefly descent can flexibly grow networks both wider and deeper, and can be applied to learn accurate but resource efficient neural architectures that avoid catastrophic forgetting in continual learning. Empirically, firefly descent achieves promising results on both neural architecture search and continual learning. In particular, on a challenging continual image classification task, it learns networks that are smaller in size but have higher average accuracy than those learned by the state of the art methods.


Not all noise is accounted equally How differentially private learning benefits from large sampling rates. Learning often involves sensitive data and as such, privacy preserving extensions to Stochastic Gradient Descent (SGD) and other machine learning algorithms have been developed using the definitions of Differential Privacy (DP). In differentially private SGD, the gradients computed at each training iteration are subject to two different types of noise. Firstly, inherent sampling noise arising from the use of minibatches. Secondly, additive Gaussian noise from the underlying mechanisms that introduce privacy. In this study, we show that these two types of noise are equivalent in their effect on the utility of private neural networks, however they are not accounted for equally in the privacy budget. Given this observation, we propose a training paradigm that shifts the proportions of noise towards less inherent and more additive noise, such that more of the overall noise can be accounted for in the privacy budget. With this paradigm, we are able to improve on the state of the art in the privacy utility tradeoff of private end to end CNNs.


Nonlocal pseudopotentials and time step errors in diffusion Monte Carlo. We present a version of the T moves approach for treating nonlocal pseudopotentials in diffusion Monte Carlo which has much smaller time step errors than the existing T moves approaches, while at the same time preserving desirable features such as the upper bound property for the energy. In addition, we modify the reweighting factor of the projector used in diffusion Monte Carlo to reduce the time step error. The latter is applicable not only to pseudopotential calculations but to all electron calculations as well.


Flow over an espresso cup Inferring 3D velocity and pressure fields from tomographic background oriented schlieren videos via physics informed neural networks. Tomographic background oriented schlieren (Tomo BOS) imaging measures density or temperature fields in 3D using multiple camera BOS projections, and is particularly useful for instantaneous flow visualizations of complex fluid dynamics problems. We propose a new method based on physics informed neural networks (PINNs) to infer the full continuous 3D velocity and pressure fields from snapshots of 3D temperature fields obtained by Tomo BOS imaging. PINNs seamlessly integrate the underlying physics of the observed fluid flow and the visualization data, hence enabling the inference of latent quantities using limited experimental data. In this hidden fluid mechanics paradigm, we train the neural network by minimizing a loss function composed of a data mismatch term and residual terms associated with the coupled Navier Stokes and heat transfer equations. We first quantify the accuracy of the proposed method based on a 2D synthetic data set for buoyancy driven flow, and subsequently apply it to the Tomo BOS data set, where we are able to infer the instantaneous velocity and pressure fields of the flow over an espresso cup based only on the temperature field provided by the Tomo BOS imaging. Moreover, we conduct an independent PIV experiment to validate the PINN inference for the unsteady velocity field at a center plane. To explain the observed flow physics, we also perform systematic PINN simulations at different Reynolds and Richardson numbers and quantify the variations in velocity and pressure fields. The results in this paper indicate that the proposed deep learning technique can become a promising direction in experimental fluid mechanics.


WebSRC A Dataset for Web Based Structural Reading Comprehension. Web search is an essential way for humans to obtain information, but it's still a great challenge for machines to understand the contents of web pages. In this paper, we introduce the task of structural reading comprehension (SRC) on web. Given a web page and a question about it, the task is to find the answer from the web page. This task requires a system not only to understand the semantics of texts but also the structure of the web page. Moreover, we proposed WebSRC, a novel Web based Structural Reading Comprehension dataset. WebSRC consists of 400K question answer pairs, which are collected from 6.4K web pages. Along with the QA pairs, corresponding HTML source code, screenshots, and metadata are also provided in our dataset. Each question in WebSRC requires a certain structural understanding of a web page to answer, and the answer is either a text span on the web page or yes no. We evaluate various baselines on our dataset to show the difficulty of our task. We also investigate the usefulness of structural information and visual features. Our dataset and baselines have been publicly available at https x lance.github.io WebSRC .


How to help university students to manage their interruptions and improve their attention and time management. Technology has changed both our way of life and the way in which we learn. Students now attend lectures with laptops and mobile phones, and this situation is accentuated in the case of students on Computer Science degrees, since they require their computers in order to participate in both theoretical and practical lessons. Problems, however, arise when the students' social networks are opened on their computers and they receive notifications that interrupt their work. We set up a workshop regarding time, thoughts and attention management with the objective of teaching our students techniques that would allow them to manage interruptions, concentrate better and definitively make better use of their time. Those who took part in the workshop were then evaluated to discover its effects. The results obtained are quite optimistic and are described in this paper with the objective of encouraging other universities to perform similar initiatives.


Asymptotics of the eigenvalues of seven diagonal Toeplitz matrices of a special form. We find uniform asymptotic formulas for all the eigenvalues of certain 7 diagonal symmetric Toeplitz matrices of large dimension. The entries of the matrices are real and we consider the case where the real valued generating function such that its first five derivatives at the one endpoint of interval are equal zero. This is not the simple loop case considered earlier. We obtain nonlinear equations for the eigenvalues. It should be noted that our equations have a more complicated structure than the equations for the simple loop case.


Improving Neural Cross Lingual Summarization via Employing Optimal Transport Distance for Knowledge Distillation. Current state of the art cross lingual summarization models employ multi task learning paradigm, which works on a shared vocabulary module and relies on the self attention mechanism to attend among tokens in two languages. However, correlation learned by self attention is often loose and implicit, inefficient in capturing crucial cross lingual representations between languages. The matter worsens when performing on languages with separate morphological or structural features, making the cross lingual alignment more challenging, resulting in the performance drop. To overcome this problem, we propose a novel Knowledge Distillation based framework for Cross Lingual Summarization, seeking to explicitly construct cross lingual correlation by distilling the knowledge of the monolingual summarization teacher into the cross lingual summarization student. Since the representations of the teacher and the student lie on two different vector spaces, we further propose a Knowledge Distillation loss using Sinkhorn Divergence, an Optimal Transport distance, to estimate the discrepancy between those teacher and student representations. Due to the intuitively geometric nature of Sinkhorn Divergence, the student model can productively learn to align its produced cross lingual hidden states with monolingual hidden states, hence leading to a strong correlation between distant languages. Experiments on cross lingual summarization datasets in pairs of distant languages demonstrate that our method outperforms state of the art models under both high and low resourced settings.


A simple evaluation of a theta value and the Kronecker limit formula. We evaluate the classic sum sum n in mathbb Z e pi n 2 . The novelty of our approach is that it does not require any prior knowledge about modular forms, elliptic functions or analytic continuations. Even the Gamma function, in terms of which the result is expressed, only appears as a complex function in the computation of a real integral by the residue theorem. Another contribution of this note is to provide a very simple proof of the Kronecker limit formula.


PARIS Personalized Activity Recommendation for Improving Sleep Quality. The quality of sleep has a deep impact on people's physical and mental health. People with insufficient sleep are more likely to report physical and mental distress, activity limitation, anxiety, and pain. Moreover, in the past few years, there has been an explosion of applications and devices for activity monitoring and health tracking. Signals collected from these wearable devices can be used to study and improve sleep quality. In this paper, we utilize the relationship between physical activity and sleep quality to find ways of assisting people improve their sleep using machine learning techniques. People usually have several behavior modes that their bio functions can be divided into. Performing time series clustering on activity data, we find cluster centers that would correlate to the most evident behavior modes for a specific subject. Activity recipes are then generated for good sleep quality for each behavior mode within each cluster. These activity recipes are supplied to an activity recommendation engine for suggesting a mix of relaxed to intense activities to subjects during their daily routines. The recommendations are further personalized based on the subjects' lifestyle constraints, i.e. their age, gender, body mass index (BMI), resting heart rate, etc, with the objective of the recommendation being the improvement of that night's quality of sleep. This would in turn serve a longer term health objective, like lowering heart rate, improving the overall quality of sleep, etc.


Computing Sensitivities in Reaction Networks using Finite Difference Methods. In this article, we investigate various numerical methods for computing scaled or logarithmic sensitivities of the form partial ln y partial ln x . The methods tested include One Point, Two Point, Five Point, and the Richardson Extrapolation. The different methods were applied to a variety of mathematical functions as well as a reaction network model. The algorithms were validated by comparing results with known analytical solutions for functions and using the Reder method for computing the sensitivities in reaction networks via the Tellurium package. For evaluation, two aspects were looked at, accuracy and time taken to compute the sensitivities. Of the four methods, Richardson's extrapolation was by far the most accurate but also the slowest in terms of performance. For fast, reasonably accurate estimates, we recommend the two point method. For most other cases where the derivatives are changing rapidly, the five point method is a good choice, although it is three times slower than the two point method. For ultimate accuracy which would apply particularly to very fast changing derivatives the Richardson method is without doubt the best, but it is seven times slower than the two point method. We do not recommend the one point method in any circumstance. The Python software that was used in the study with documentation is available at url https github.com evanyfyip SensitivityAnalysis .


Simulation of a high energy electron beam transmission through titanium and Kapton circledR thin films. The results of computer simulation of the high energy electrons passage through thin layers of titanium (Ti) and polyimide Kapton circledR in the energy range from 3 to 20 MeV are presented. Simulation is carried out using the Geant4 toolkit. The thickness of the Ti foil in the model experiment is 50 microns, the thickness of the Kapton circledR film is 110 microns. The purpose of the calculations is to reveal the possibility of using the Kapton circledR film in the output devices of linear electron accelerators. It was necessary to calculate the probable values of the energy absorbed in a Kapton circledR film and in a Ti foil for each value of primary electrons energy. Another important characteristic is the divergence radius of the electron beam at a predetermined distance from the film, or the electron scattering angle. As a result of calculations, the energy spectra of bremsstrahlung gamma quanta, formed during the passage of electrons through the materials of the films, are obtained. The most probable values of the energy absorbed in the Ti foil and in the Kapton circledR film are calculated. The scattering radii of an electron beam for the Kapton circledR film and also for the Ti foil at a distance of 20 cm are estimated. It is shown that the ratio of the total amount of bremsstrahlung gamma quanta in the case of use the Kapton circledR film is approximately 0.56 of the total amount of bremsstrahlung gamma quanta when using the Ti foil. The analysis of the calculated data showed that the use of Kapton circledR as a material for the manufacture of output devices for high energy electron beams is more preferable in comparison to Ti films. The use of Kapton circledR instead of Ti makes it possible to significantly reduce the background of the generated bremsstrahlung gamma quanta and reduce the scattering radius of the electron beam.


Harnack inequality for fractional Laplacian type operators on hyperbolic spaces. The Krylov Safonov theory for fully nonlinear nonlocal operators on hyperbolic spaces of dimension three is established. Since the operators on hyperbolic spaces exhibit qualitatively different behavior than those on manifolds with nonnegative curvature, new scale functions are introduced which take the effect of negative curvature into account. The regularity theory in this work provides unified regularity results for fractional order and second order operators in the sense that the regularity estimates stay uniform as the fractional order approaches 2. In the unified regularity theory, the asymptotic behavior of the normalizing constant for the fractional Laplacian plays a fundamental role. The dimension restriction has been imposed to compute the explicit value of this constant by using the Fourier analysis on hyperbolic spaces.


On Subvarieties of Degenerations of Separably Rationally Connected Varieties. The goal of this work is to consider some geometric properties of geometrically irreducible subschemes on degenerations of separably rationally connected varieties. It is known that these geometrically irreducible subschemes exist when the ground field has characteristic zero or contains an algebraically closed subfield. In this note, we show that the dimension of this geometrically irreducible subscheme has a lower bound by the Fano index of the generic fiber.


The role of grounded conducting pointy objects during Thunderstorm Ground Enhancements (TGEs). A mechanism for increased fluxes of high energy particles detected in Thunderstorm Ground Enhancements (TGEs) is suggested. When the electric field Eambient at the ground beneath thunderstorms reaches modest magnitudes (6 12 kV m), corona discharge occurs from the tips of some grounded conducting pointy objects (GCPOs). Past theory and modeling of prolate semi ellipsoidal GCPOs suggest the corona threshold value for free electron availability via seed electron detachment is E 67 V m Pa at the tip. The field enhancement, Etip Eambient, at the GCPO tip depends only on its length tip radius of curvature ratio. Thus, Etip at all GCPOs is substantially enhanced by Eambient, whether or not they are in corona. Cosmic ray electrons striking these pointy tips will gain MeV energies. These high energy electrons may be detected with instrumentation located nearby, or they may produce gamma rays via bremsstrahlung within the tips before being detected, thereby contributing to TGEs.


FairCanary Rapid Continuous Explainable Fairness. Machine Learning (ML) models are being used in all facets of today's society to make high stake decisions like bail granting or credit lending, with very minimal regulations. Such systems are extremely vulnerable to both propagating and amplifying social biases, and have therefore been subject to growing research interest. One of the main issues with conventional fairness metrics is their narrow definitions which hide the complete extent of the bias by focusing primarily on positive and or negative outcomes, whilst not paying attention to the overall distributional shape. Moreover, these metrics are often contradictory to each other, are severely restrained by the contextual and legal landscape of the problem, have technical constraints like poor support for continuous outputs, the requirement of class labels, and are not explainable. In this paper, we present Quantile Demographic Drift, which addresses the shortcomings mentioned above. This metric can also be used to measure intra group privilege. It is easily interpretable via existing attribution techniques, and also extends naturally to individual fairness via the principle of like for like comparison. We make this new fairness score the basis of a new system that is designed to detect bias in production ML models without the need for labels. We call the system FairCanary because of its capability to detect bias in a live deployed model and narrow down the alert to the responsible set of features, like the proverbial canary in a coal mine.


Improving the quality of generative models through Smirnov transformation. Solving the convergence issues of Generative Adversarial Networks (GANs) is one of the most outstanding problems in generative models. In this work, we propose a novel activation function to be used as output of the generator agent. This activation function is based on the Smirnov probabilistic transformation and it is specifically designed to improve the quality of the generated data. In sharp contrast with previous works, our activation function provides a more general approach that deals not only with the replication of categorical variables but with any type of data distribution (continuous or discrete). Moreover, our activation function is derivable and therefore, it can be seamlessly integrated in the backpropagation computations during the GAN training processes. To validate this approach, we evaluate our proposal against two different data sets a) an artificially rendered data set containing a mixture of discrete and continuous variables, and b) a real data set of flow based network traffic data containing both normal connections and cryptomining attacks. To evaluate the fidelity of the generated data, we analyze both their results in terms of quality measures of statistical nature and also regarding the use of these synthetic data to feed a nested machine learning based classifier. The experimental results evince a clear outperformance of the GAN network tuned with this new activation function with respect to both a na "ive mean based generator and a standard GAN. The quality of the data is so high that the generated data can fully substitute real data for training the nested classifier without a fall in the obtained accuracy. This result encourages the use of GANs to produce high quality synthetic data that are applicable in scenarios in which data privacy must be guaranteed.


A high flux, adjustable, compact cold atom source. Magneto optical traps (MOTs) are widely used for laser cooling of atoms. We have developed a high flux compact cold atom source based on a pyramid MOT with a unique adjustable aperture that is highly suitable for portable quantum technology devices, including space based experiments. The adjustability enabled an investigation into the previously unexplored impact of aperture size on the atomic flux, and optimisation of the aperture size allowed us to demonstrate a higher flux than any reported cold atom sources that use a pyramid, LVIS, 3D MOT or grating MOT. We achieved 2.0(1)x10 10 atoms s of 87 Rb with a mean velocity of 32(1)m s, FWHM of 27.6(9)m s and divergence of 58(3)mrad. Halving the total optical power to 195mW caused only a 26 reduction of the flux, and a 33 decrease in mean velocity. Methods to further decrease the velocity as required have been identified. The low power consumption and small size make this design suitable for a wide range of cold atom technologies.


Point set Distances for Learning Representations of 3D Point Clouds. Learning an effective representation of 3D point clouds requires a good metric to measure the discrepancy between two 3D point sets, which is non trivial due to their irregularity. Most of the previous works resort to using the Chamfer discrepancy or Earth Mover's distance, but those metrics are either ineffective in measuring the differences between point clouds or computationally expensive. In this paper, we conduct a systematic study with extensive experiments on distance metrics for 3D point clouds. From this study, we propose to use sliced Wasserstein distance and its variants for learning representations of 3D point clouds. In addition, we introduce a new algorithm to estimate sliced Wasserstein distance that guarantees that the estimated value is close enough to the true one. Experiments show that the sliced Wasserstein distance and its variants allow the neural network to learn a more efficient representation compared to the Chamfer discrepancy. We demonstrate the efficiency of the sliced Wasserstein metric and its variants on several tasks in 3D computer vision including training a point cloud autoencoder, generative modeling, transfer learning, and point cloud registration.


Bifurcation preserving discretisations of optimal control problems. The first order optimality conditions of optimal control problems (OCPs) can be regarded as boundary value problems for Hamiltonian systems. Variational or symplectic discretisation methods are classically known for their excellent long term behaviour. As boundary value problems are posed on intervals of fixed, moderate length, it is not immediately clear whether methods can profit from structure preservation in this context. When parameters are present, solutions can undergo bifurcations, for instance, two solutions can merge and annihilate one another as parameters are varied. We will show that generic bifurcations of an OCP are preserved under discretisation when the OCP is either directly discretised to a discrete OCP (direct method) or translated into a Hamiltonian boundary value problem using first order necessary conditions of optimality which is then solved using a symplectic integrator (indirect method). Moreover, certain bifurcations break when a non symplectic scheme is used. The general phenomenon is illustrated on the example of a cut locus of an ellipsoid.


The upper logarithmic density of monochromatic subset sums. We show that in any two coloring of the positive integers there is a color for which the set of positive integers that can be represented as a sum of distinct elements with this color has upper logarithmic density at least (2 sqrt 3 ) 4 and this is best possible. This answers a forty year old question of Erd H o s.


Remote Anomaly Detection in Industry 4.0 Using Resource Constrained Devices. A central use case for the Internet of Things (IoT) is the adoption of sensors to monitor physical processes, such as the environment and industrial manufacturing processes, where they provide data for predictive maintenance, anomaly detection, or similar. The sensor devices are typically resource constrained in terms of computation and power, and need to rely on cloud or edge computing for data processing. However, the capacity of the wireless link and their power constraints limit the amount of data that can be transmitted to the cloud. While this is not problematic for the monitoring of slowly varying processes such as temperature, it is more problematic for complex signals such as those captured by vibration and acoustic sensors. In this paper, we consider the specific problem of remote anomaly detection based on signals that fall into the latter category over wireless channels with resource constrained sensors. We study the impact of source coding on the detection accuracy with both an anomaly detector based on Principal Component Analysis (PCA) and one based on an autoencoder. We show that the coded transmission is beneficial when the signal to noise ratio (SNR) of the channel is low, while uncoded transmission performs best in the high SNR regime.


Majority dynamics on sparse random graphs. Majority dynamics on a graph G is a deterministic process such that every vertex updates its pm 1 assignment according to the majority assignment on its neighbor simultaneously at each step. Benjamini, Chan, O'Donnel, Tamuz and Tan conjectured that, in the Erd H o s R 'enyi random graph G(n,p) , the random initial pm 1 assignment converges to a 99 agreement with high probability whenever p omega(1 n) . This conjecture was first confirmed for p geq lambda n 1 2 for a large constant lambda by Fountoulakis, Kang and Makai. Although this result has been reproved recently by Tran and Vu and by Berkowitz and Devlin, it was unknown whether the conjecture holds for p lambda n 1 2 . We break this Omega(n 1 2 ) barrier by proving the conjecture for sparser random graphs G(n,p) , where lambda' n 3 5 log n leq p leq lambda n 1 2 with a large constant lambda' 0 .


On commutator relations in 2 spherical RGD systems. In this paper we investigate the commutator relations for prenilpotent roots which are nested. These commutator relations are trivial in a lot of cases.


A convex form that is not a sum of squares. Every convex homogeneous polynomial (or form) is nonnegative. Blekherman has shown that there exist convex forms that are not sums of squares via a nonconstructive argument. We provide an explicit example of a convex form of degree four in 272 variables that is not a sum of squares. The form is related to the Cauchy Schwarz inequality over the octonions. The proof uses symmetry reduction together with the fact (due to Blekherman) that forms of even degree, that are near constant on the unit sphere, are convex. Using this same connection, we obtain improved bounds on the approximation quality achieved by the basic sum of squares relaxation for optimizing quaternary quartic forms on the sphere.


Extra large type Artin groups are hierarchically hyperbolic. We show that Artin groups of extra large type, and more generally Artin groups of large and hyperbolic type, are hierarchically hyperbolic. This implies in particular that these groups have finite asymptotic dimension and uniform exponential growth. We prove these results by using a combinatorial approach to hierarchical hyperbolicity, via the action of these groups on a new complex that is quasi isometric both to the coned off Deligne complex introduced by Martin Przytycki and to a generalisation due to Morris Wright of the graph of irreducible parabolic subgroups of finite type introduced by Cumplido Gebhardt Gonz 'alez Meneses Wiest.


BDPGO Balanced Distributed Pose Graph Optimization Framework for Swarm Robotics. Distributed pose graph optimization (DPGO) is one of the fundamental techniques of swarm robotics. Currently, the sub problems of DPGO are built on the native poses. Our validation proves that this approach may introduce an imbalance in the sizes of the sub problems in real world scenarios, which affects the speed of DPGO optimization, and potentially increases communication requirements. In addition, the coherence of the estimated poses is not guaranteed when the robots in the swarm fail, or partial robots are disconnected. In this paper, we propose BDPGO, a balanced distributed pose graph optimization framework using the idea of decoupling the robot poses and DPGO. BDPGO re distributes the poses in the pose graph to the robot swarm in a balanced way by introducing a two stage graph partitioning method to build balanced subproblems. Our validation demonstrates that BDPGO significantly improves the optimization speed without changing the specific algorithm of DPGO in realistic datasets. What's more, we also validate that BDPGO is robust to robot failure, changes in the wireless network. BDPGO has capable of keeps the coherence of the estimated poses in these situations. The framework also has the potential to be applied to other collaborative simultaneous localization and mapping (CSLAM) problems involved in distributedly solving the factor graph.


Deep Reinforcement Learning for Equal Risk Pricing and Hedging under Dynamic Expectile Risk Measures. Recently equal risk pricing, a framework for fair derivative pricing, was extended to consider dynamic risk measures. However, all current implementations either employ a static risk measure that violates time consistency, or are based on traditional dynamic programming solution schemes that are impracticable in problems with a large number of underlying assets (due to the curse of dimensionality) or with incomplete asset dynamics information. In this paper, we extend for the first time a famous off policy deterministic actor critic deep reinforcement learning (ACRL) algorithm to the problem of solving a risk averse Markov decision process that models risk using a time consistent recursive expectile risk measure. This new ACRL algorithm allows us to identify high quality time consistent hedging policies (and equal risk prices) for options, such as basket options, that cannot be handled using traditional methods, or in context where only historical trajectories of the underlying assets are available. Our numerical experiments, which involve both a simple vanilla option and a more exotic basket option, confirm that the new ACRL algorithm can produce 1) in simple environments, nearly optimal hedging policies, and highly accurate prices, simultaneously for a range of maturities 2) in complex environments, good quality policies and prices using reasonable amount of computing resources and 3) overall, hedging strategies that actually outperform the strategies produced using static risk measures when the risk is evaluated at later points of time.


Incentivizing Mobile Edge Caching and Sharing An Evolutionary Game Approach. Mobile Edge Caching is a promising technique to enhance the content delivery quality and reduce the backhaul link congestion, by storing popular content at the network edge or mobile devices (e.g. base stations and smartphones) that are proximate to content requesters. In this work, we study a novel mobile edge caching framework, which enables mobile devices to cache and share popular contents with each other via device to device (D2D) links. We are interested in the following incentive problem of mobile device users whether and which users are willing to cache and share what contents, taking the user mobility and cost reward into consideration. The problem is challenging in a large scale network with a large number of users. We introduce the evolutionary game theory, an effective tool for analyzing large scale dynamic systems, to analyze the mobile users' content caching and sharing strategies. Specifically, we first derive the users' best caching and sharing strategies, and then analyze how these best strategies change dynamically over time, based on which we further characterize the system equilibrium systematically. Simulation results show that the proposed caching scheme outperforms the existing schemes in terms of the total transmission cost and the cellular load. In particular, in our simulation, the total transmission cost can be reduced by 42.5 55.2 and the cellular load can be reduced by 21.5 56.4 .


Dynamically Modelling Heterogeneous Higher Order Interactions for Malicious Behavior Detection in Event Logs. Anomaly detection in event logs is a promising approach for intrusion detection in enterprise networks. By building a statistical model of usual activity, it aims to detect multiple kinds of malicious behavior, including stealthy tactics, techniques and procedures (TTPs) designed to evade signature based detection systems. However, finding suitable anomaly detection methods for event logs remains an important challenge. This results from the very complex, multi faceted nature of the data event logs are not only combinatorial, but also temporal and heterogeneous data, thus they fit poorly in most theoretical frameworks for anomaly detection. Most previous research focuses on either one of these three aspects, building a simplified representation of the data that can be fed to standard anomaly detection algorithms. In contrast, we propose to simultaneously address all three of these characteristics through a specifically tailored statistical model. We introduce textsc Decades , a underline d ynamic, h underline e terogeneous and underline c ombinatorial model for underline a nomaly underline d etection in underline e vent underline s treams, and we demonstrate its effectiveness at detecting malicious behavior through experiments on a real dataset containing labelled red team activity. In particular, we empirically highlight the importance of handling the multiple characteristics of the data by comparing our model with state of the art baselines relying on various data representations.


Estimating the Total Volume of Queries to a Search Engine. We study the problem of estimating the total number of searches (volume) of queries in a specific domain, which were submitted to a search engine in a given time period. Our statistical model assumes that the distribution of searches follows a Zipf's law, and that the observed sample volumes are biased accordingly to three possible scenarios. These assumptions are consistent with empirical data, with keyword research practices, and with approximate algorithms used to take counts of query frequencies. A few estimators of the parameters of the distribution are devised and experimented, based on the nature of the empirical simulated data. For continuous data, we recommend using nonlinear least square regression (NLS) on the top volume queries, where the bound on the volume is obtained from the well known Clauset, Shalizi and Newman (CSN) estimation of power law parameters. For binned data, we propose using a Chi square minimization approach restricted to the top volume queries, where the bound is obtained by the binned version of the CSN method. Estimations are then derived for the total number of queries and for the total volume of the population, including statistical error bounds. We apply the methods on the domain of recipes and cooking queries searched in Italian in 2017. The observed volumes of sample queries are collected from Google Trends (continuous data) and SearchVolume (binned data). The estimated total number of queries and total volume are computed for the two cases, and the results are compared and discussed.


SOLD2 Self supervised Occlusion aware Line Description and Detection. Compared to feature point detection and description, detecting and matching line segments offer additional challenges. Yet, line features represent a promising complement to points for multi view tasks. Lines are indeed well defined by the image gradient, frequently appear even in poorly textured areas and offer robust structural cues. We thus hereby introduce the first joint detection and description of line segments in a single deep network. Thanks to a self supervised training, our method does not require any annotated line labels and can therefore generalize to any dataset. Our detector offers repeatable and accurate localization of line segments in images, departing from the wireframe parsing approach. Leveraging the recent progresses in descriptor learning, our proposed line descriptor is highly discriminative, while remaining robust to viewpoint changes and occlusions. We evaluate our approach against previous line detection and description methods on several multi view datasets created with homographic warps as well as real world viewpoint changes. Our full pipeline yields higher repeatability, localization accuracy and matching metrics, and thus represents a first step to bridge the gap with learned feature points methods. Code and trained weights are available at https github.com cvg SOLD2.


An empirical evaluation of the usefulness of Tree Kernels for Commit time Defect Detection in large software systems. Defect detection at commit check in time prevents the introduction of defects into software systems. Current defect detection approaches rely on metric based models which are not very accurate and whose results are not directly useful for developers. We propose a method to detect bug inducing commits by comparing the incoming changes with all past commits in the project, considering both those that introduced defects and those that did not. Our method considers individual changes in the commit separately, at the method level granularity. Doing so helps developers as they are informed of specific methods that need further attention instead of being told that the entire commit is problematic. Our approach represents source code as abstract syntax trees and uses tree kernels to estimate the similarity of the code with previous commits. We experiment with subtree kernels (STK), subset tree kernels (SSTK), or partial tree kernels (PTK). An incoming change is then classified using a K NN classifier on the past changes. We evaluate our approach on the BigCloneBench benchmark and on the Technical Debt dataset, using the NiCad clone detector as the baseline. Our experiments with the BigCloneBench benchmark show that the tree kernel approach can detect clones with a comparable MAP to that of NiCad. Also, on defect detection with the Technical Debt dataset, tree kernels are least as effective as NiCad with MRR, F score, and Accuracy of 0.87, 0.80, and 0.82 respectively.


MMF Multi Task Multi Structure Fusion for Hierarchical Image Classification. Hierarchical classification is significant for complex tasks by providing multi granular predictions and encouraging better mistakes. As the label structure decides its performance, many existing approaches attempt to construct an excellent label structure for promoting the classification results. In this paper, we consider that different label structures provide a variety of prior knowledge for category recognition, thus fusing them is helpful to achieve better hierarchical classification results. Furthermore, we propose a multi task multi structure fusion model to integrate different label structures. It contains two kinds of branches one is the traditional classification branch to classify the common subclasses, the other is responsible for identifying the heterogeneous superclasses defined by different label structures. Besides the effect of multiple label structures, we also explore the architecture of the deep model for better hierachical classification and adjust the hierarchical evaluation metrics for multiple label structures. Experimental results on CIFAR100 and Car196 show that our method obtains significantly better results than using a flat classifier or a hierarchical classifier with any single label structure.


Closed form solutions for the inverse kinematics of serial robots using conformal geometric algebra. This work addresses the inverse kinematics of serial robots using conformal geometric algebra. Classical approaches include either the use of homogeneous matrices, which entails high computational cost and execution time or the development of particular geometric strategies that cannot be generalized to arbitrary serial robots. In this work, we present a compact, elegant and intuitive formulation of robot kinematics based on conformal geometric algebra that provides a suitable framework for the closed form resolution of the inverse kinematic problem for manipulators with a spherical wrist. For serial robots of this kind, the inverse kinematics problem can be split in two subproblems the position and orientation problems. The latter is solved by appropriately splitting the rotor that defines the target orientation into three simpler rotors, while the former is solved by developing a geometric strategy for each combination of prismatic and revolute joints that forms the position part of the robot. Finally, the inverse kinematics of 7 DoF redundant manipulators with a spherical wrist is solved by extending the geometric solutions obtained in the non redundant case.


A Python Package to Detect Anti Vaccine Users on Twitter. Vaccine hesitancy has a long history but has been recently driven by the anti vaccine narratives shared online, which significantly degrades the efficacy of vaccination strategies, such as those for COVID 19. Despite broad agreement in the medical community about the safety and efficacy of available vaccines, a large number of social media users continue to be inundated with false information about vaccines and, partly because of this, became indecisive or unwilling to be vaccinated. The goal of this study is to better understand anti vaccine sentiment, and work to reduce its impact, by developing a system capable of automatically identifying the users responsible for spreading anti vaccine narratives. We introduce a publicly available Python package capable of analyzing Twitter profiles to assess how likely that profile is to spread anti vaccine sentiment in the future. The software package is built using text embedding methods, neural networks, and automated dataset generation. It is trained on over one hundred thousand accounts and several million tweets. This model will help researchers and policy makers understand anti vaccine discussion and misinformation strategies, which can further help tailor targeted campaigns seeking to inform and debunk the harmful anti vaccination myths currently being spread. Additionally, we leverage the data on such users to understand what are the moral and emotional characteristics of anti vaccine spreaders.


IIE NLP Eyas at SemEval 2021 Task 4 Enhancing PLM for ReCAM with Special Tokens, Re Ranking, Siamese Encoders and Back Translation. This paper introduces our systems for all three subtasks of SemEval 2021 Task 4 Reading Comprehension of Abstract Meaning. To help our model better represent and understand abstract concepts in natural language, we well design many simple and effective approaches adapted to the backbone model (RoBERTa). Specifically, we formalize the subtasks into the multiple choice question answering format and add special tokens to abstract concepts, then, the final prediction of question answering is considered as the result of subtasks. Additionally, we employ many finetuning tricks to improve the performance. Experimental results show that our approaches achieve significant performance compared with the baseline systems. Our approaches achieve eighth rank on subtask 1 and tenth rank on subtask 2.


Sub GMN The Subgraph Matching Network Model. As one of the most fundamental tasks in graph theory, subgraph matching is a crucial task in many fields, ranging from information retrieval, computer vision, biology, chemistry and natural language processing. Yet subgraph matching problem remains to be an NP complete problem. This study proposes an end to end learning based approximate method for subgraph matching task, called subgraph matching network (Sub GMN). The proposed Sub GMN firstly uses graph representation learning to map nodes to node level embedding. It then combines metric learning and attention mechanisms to model the relationship between matched nodes in the data graph and query graph. To test the performance of the proposed method, we applied our method on two databases. We used two existing methods, GNN and FGNN as baseline for comparison. Our experiment shows that, on dataset 1, on average the accuracy of Sub GMN are 12.21 and 3.2 higher than that of GNN and FGNN respectively. On average running time Sub GMN runs 20 40 times faster than FGNN. In addition, the average F1 score of Sub GMN on all experiments with dataset 2 reached 0.95, which demonstrates that Sub GMN outputs more correct node to node matches. Comparing with the previous GNNs based methods for subgraph matching task, our proposed Sub GMN allows varying query and data graphes in the test application stage, while most previous GNNs based methods can only find a matched subgraph in the data graph during the test application for the same query graph used in the training stage. Another advantage of our proposed Sub GMN is that it can output a list of node to node matches, while most existing end to end GNNs based methods cannot provide the matched node pairs.


Geometric control of hybrid systems. In this paper, we present a geometric approach for computing controlled invariant sets for hybrid control systems. While the problem is well studied in the ellipsoidal case, this family is quite conservative for constrained or switched linear systems. We reformulate the invariance of a set as an inequality for its support function that is valid for any convex set. This produces novel algebraic conditions for the invariance of sets with polynomial or piecewise quadratic support function.


Minimally Supervised Structure Rich Text Categorization via Learning on Text Rich Networks. Text categorization is an essential task in Web content analysis. Considering the ever evolving Web data and new emerging categories, instead of the laborious supervised setting, in this paper, we focus on the minimally supervised setting that aims to categorize documents effectively, with a couple of seed documents annotated per category. We recognize that texts collected from the Web are often structure rich, i.e., accompanied by various metadata. One can easily organize the corpus into a text rich network, joining raw text documents with document attributes, high quality phrases, label surface names as nodes, and their associations as edges. Such a network provides a holistic view of the corpus' heterogeneous data sources and enables a joint optimization for network based analysis and deep textual model training. We therefore propose a novel framework for minimally supervised categorization by learning from the text rich network. Specifically, we jointly train two modules with different inductive biases a text analysis module for text understanding and a network learning module for class discriminative, scalable network learning. Each module generates pseudo training labels from the unlabeled document set, and both modules mutually enhance each other by co training using pooled pseudo labels. We test our model on two real world datasets. On the challenging e commerce product categorization dataset with 683 categories, our experiments show that given only three seed documents per category, our framework can achieve an accuracy of about 92 , significantly outperforming all compared methods our accuracy is only less than 2 away from the supervised BERT model trained on about 50K labeled documents.


TwoPhaseFlow An OpenFOAM based framework for development of two phase flow solvers. One of the prevailing challenges in Computational Fluid Dynamics is accurate simulation of two phase flows involving heat and mass transfer across the fluid interface. This is currently an active field of research, which is to some extend impaired by a lack of a common programming framework for implementing and testing new models. Here we present a new OpenFOAM based open source framework allowing fast implementation and test of new phase change and surface tension force models. Capitalizing on the runtime selection mechanism in OpenFOAM, the new models can easily be selected and compared to analytical solutions and existing models. As a start, the framework includes the following curvature calculation methods for surface tension height function, parabolic fit, and reconstructed distance function method. As for phase change, interface heat resistance and direct heat flux models are available. These can be combined with three solvers covering the range from isothermal, incompressible flow to non isothermal, compressible flow with conjugated heat transfer. By design, addition of new models and solvers is straightforward and users are invited to contribute their specific models, solvers, and validation cases to the library.


Improving the Algorithm of Deep Learning with Differential Privacy. In this paper, an adjustment to the original differentially private stochastic gradient descent (DPSGD) algorithm for deep learning models is proposed. As a matter of motivation, to date, almost no state of the art machine learning algorithm hires the existing privacy protecting components due to otherwise serious compromise in their utility despite the vital necessity. The idea in this study is natural and interpretable, contributing to improve the utility with respect to the state of the art. Another property of the proposed technique is its simplicity which makes it again more natural and also more appropriate for real world and specially commercial applications. The intuition is to trim and balance out wild individual discrepancies for privacy reasons, and at the same time, to preserve relative individual differences for seeking performance. The idea proposed here can also be applied to the recurrent neural networks (RNN) to solve the gradient exploding problem. The algorithm is applied to benchmark datasets MNIST and CIFAR 10 for a classification task and the utility measure is calculated. The results outperformed the original work.


A Bayesian Approach for In Situ Stress Prediction and Uncertainty Quantification for Subsurface Engineering. Many subsurface engineering applications require accurate knowledge of the in situ state of stress for their safe design and operation. Existing methods to meet this need primarily include field measurements for estimating one or more of the principal stresses from a borehole, or optimization methods for constructing a 3D geomechanical model in terms of geophysical measurements. These methods, however, often contain considerable uncertainty in estimating the state of stress. In this paper, we build on a Bayesian approach to quantify uncertainty in stress estimations for subsurface engineering applications. This approach can provide an estimate of the 3D distribution of stress throughout the volume of interest and provide an estimate of the uncertainty arising from the stress measurement, the rheology parameters, and a paucity of measurements. The value of this approach is demonstrated using stress measurements from the In Salah carbon storage site, which was one of the first industrial carbon capture and storage projects in the world. This demonstration shows the application of this Bayesian approach for estimating the initial state of stress for In Salah and quantifying the uncertainty in the estimated stress. Also, an assessment of a maximum injection pressure to prevent geomechanical risks from CO2 injection pressures is provided in terms of the probability distribution of the minimum principal stress quantified by the approach. With the In Salah case study, this paper demonstrates that using the Bayesian approach can provide additional insights for site explorations and or project operations to make informed site decisions for subsurface engineering applications.


A parallel fast multipole method for a space time boundary element method for the heat equation. We present a novel approach to the parallelization of the parabolic fast multipole method for a space time boundary element method for the heat equation. We exploit the special temporal structure of the involved operators to provide an efficient distributed parallelization with respect to time and with a one directional communication pattern. On top, we apply a task based shared memory parallelization and SIMD vectorization. In the numerical tests we observe high efficiencies of our parallelization approach.


Ivy Templated Deep Learning for Inter Framework Portability. We introduce Ivy, a templated Deep Learning (DL) framework which abstracts existing DL frameworks. Ivy unifies the core functions of these frameworks to exhibit consistent call signatures, syntax and input output behaviour. New high level framework agnostic functions and classes, which are usable alongside framework specific code, can then be implemented as compositions of the unified low level Ivy functions. Ivy currently supports TensorFlow, PyTorch, MXNet, Jax and NumPy. We also release four pure Ivy libraries for mechanics, 3D vision, robotics, and differentiable environments. Through our evaluations, we show that Ivy can significantly reduce lines of code with a runtime overhead of less than 1 in most cases. We welcome developers to join the Ivy community by writing their own functions, layers and libraries in Ivy, maximizing their audience and helping to accelerate DL research through inter framework codebases. More information can be found at https ivy dl.org.


Dimensions of Kleinian orbital sets. Given a non empty bounded subset of hyperbolic space and a Kleinian group acting on that space, the orbital set is the orbit of the given set under the action of the group. We may view orbital sets as bounded (often fractal) subsets of Euclidean space. We prove that the upper box dimension of an orbital set is given by the maximum of three quantities the upper box dimension of the given set the Poincar 'e exponent of the Kleinian group and the upper box dimension of the limit set of the Kleinian group. Since we do not make any assumptions about the Kleinian group, none of the terms in the maximum can be removed in general. We show by constructing an explicit example that the (hyperbolic) boundedness assumption on C cannot be removed in general.


Sparse Lerner operators in infinite dimensions. We use the principle of almost orthogonality to give a new and simple proof that a sparse Lerner operator is bounded on a matrix or operator weighted space L W 2 ( mu) , where mu is a doubling measure on R d if and only if the weight W satisfies the Muckenhoupt A 2( mu) condition, restricted to the sparse collection in question. Our method extends to the infinite dimensional setting, thus allowing for applications to the multi parameter setting. For the class of Muckenhoupt A 2 weights, we obtain bounds in terms of mixed A 2 ( mu) A infty ( mu) conditions, which is independent of dimension and agrees with the best known bound in the finite dimensional vectorial setting. As an application, we prove a matrix weighted bound for the maximal Bergman projection, where we obtain a new sharper bound in terms of the B 'ekoll 'e Bonami characteristic. Furthermore, we consider commutators of sparse Lerner operators on operator valued weighted L 2 spaces and some applications to multi parameters


COMPARE A Taxonomy and Dataset of Comparison Discussions in Peer Reviews. Comparing research papers is a conventional method to demonstrate progress in experimental research. We present COMPARE, a taxonomy and a dataset of comparison discussions in peer reviews of research papers in the domain of experimental deep learning. From a thorough observation of a large set of review sentences, we build a taxonomy of categories in comparison discussions and present a detailed annotation scheme to analyze this. Overall, we annotate 117 reviews covering 1,800 sentences. We experiment with various methods to identify comparison sentences in peer reviews and report a maximum F1 Score of 0.49. We also pretrain two language models specifically on ML, NLP, and CV paper abstracts and reviews to learn informative representations of peer reviews. The annotated dataset and the pretrained models are available at https github.com shruti singh COMPARE .


Large N phenomena and quantization of the Loday Quillen Tsygan theorem. We offer a new approach to large N limits using the Batalin Vilkovisky formalism, both commutative and noncommutative, and we exhibit how the Loday Quillen Tsygan Theorem admits BV quantizations in that setting. Matrix integrals offer a key example we demonstrate how this formalism leads to a recurrence relation that in principle allows us to compute all multi point correlation functions. We also explain how the Harer Zagier relations may be expressed in terms of this noncommutative geometry derived from the BV formalism. As another application, we consider the problem of quantization in the large N limit and demonstrate how the Loday Quillen Tsygan Theorem leads us to a solution in terms of noncommutative geometry. These constructions are relevant to open topological field theories and string field theory, providing a mechanism that relates moduli of categories of branes to moduli of brane gauge theories.