A Unified Framework for Biphasic Facial Age Translation with Noisy Semantic Guided Generative Adversarial Networks. Biphasic facial age translation aims at predicting the appearance of the input face at any age. Facial age translation has received considerable research attention in the last decade due to its practical value in cross age face recognition and various entertainment applications. However, most existing methods model age changes between holistic images, regardless of the human face structure and the age changing patterns of individual facial components. Consequently, the lack of semantic supervision will cause infidelity of generated faces in detail. To this end, we propose a unified framework for biphasic facial age translation with noisy semantic guided generative adversarial networks. Structurally, we project the class aware noisy semantic layouts to soft latent maps for the following injection operation on the individual facial parts. In particular, we introduce two sub networks, ProjectionNet and ConstraintNet. ProjectionNet introduces the low level structural semantic information with noise map and produces soft latent maps. ConstraintNet disentangles the high level spatial features to constrain the soft latent maps, which endows more age related context into the soft latent maps. Specifically, attention mechanism is employed in ConstraintNet for feature disentanglement. Meanwhile, in order to mine the strongest mapping ability of the network, we embed two types of learning strategies in the training procedure, supervised self driven generation and unsupervised condition driven cycle consistent generation. As a result, extensive experiments conducted on MORPH and CACD datasets demonstrate the prominent ability of our proposed method which achieves state of the art performance.


Code generation for productive portable scalable finite element simulation in Firedrake. Creating scalable, high performance PDE based simulations requires a suitable combination of discretizations, differential operators, preconditioners and solvers. The required combination changes with the application and with the available hardware, yet software development time is a severely limited resource for most scientists and engineers. Here we demonstrate that generating simulation code from a high level Python interface provides an effective mechanism for creating high performance simulations from very few lines of user code. We demonstrate that moving from one supercomputer to another can require significant algorithmic changes to achieve scalable performance, but that the code generation approach enables these algorithmic changes to be achieved with minimal development effort.


COVID 19 Detecting Depression Signals during Stay At Home Period. The new coronavirus outbreak has been officially declared a global pandemic by the World Health Organization. To grapple with the rapid spread of this ongoing pandemic, most countries have banned indoor and outdoor gatherings and ordered their residents to stay home. Given the developing situation with coronavirus, mental health is an important challenge in our society today. In this paper, we discuss the investigation of social media postings to detect signals relevant to depression. To this end, we utilize topic modeling features and a collection of psycholinguistic and mental well being attributes to develop statistical models to characterize and facilitate representation of the more subtle aspects of depression. Furthermore, we predict whether signals relevant to depression are likely to grow significantly as time moves forward. Our best classifier yields F 1 scores as high as 0.8 and surpasses the utilized baseline by a considerable margin, 0.173. In closing, we propose several future research avenues.


Measuring Presence in Augmented Reality Environments Design and a First Test of a Questionnaire. Augmented Reality (AR) enriches a user's real environment by adding spatially aligned virtual objects (3D models, 2D textures, textual annotations, etc) by means of special display technologies. These are either worn on the body or placed in the working environment. From a technical point of view, AR faces three major challenges (1) to generate a high quality rendering, (2) to precisely register (in position and orientation) the virtual objects (VOs) with the real environment, and (3) to do so in interactive real time (Regenbrecht, Wagner, and Baratoff, 2002). The goal is to create the impression that the VOs are part of the real environment. Therefore, and similar to definitions of virtual reality (Steuer, 1992), it makes sense to define AR from a psychological point of view Augmented Reality conveys the impression that VOs are present in the real environment. In order to evaluate how well this goal is reached, a psychological measurement of this type of presence is necessary. In the following, we will describe technological features of AR systems that make a special questionnaire version necessary, describe our approach to the questionnaire development, and the data collection strategy. Finally we will present first results of the application of the questionnaire in a recent study with 385 participants.


BoningKnife Joint Entity Mention Detection and Typing for Nested NER via prior Boundary Knowledge. While named entity recognition (NER) is a key task in natural language processing, most approaches only target flat entities, ignoring nested structures which are common in many scenarios. Most existing nested NER methods traverse all sub sequences which is both expensive and inefficient, and also don't well consider boundary knowledge which is significant for nested entities. In this paper, we propose a joint entity mention detection and typing model via prior boundary knowledge (BoningKnife) to better handle nested NER extraction and recognition tasks. BoningKnife consists of two modules, MentionTagger and TypeClassifier. MentionTagger better leverages boundary knowledge beyond just entity start end to improve the handling of nesting levels and longer spans, while generating high quality mention candidates. TypeClassifier utilizes a two level attention mechanism to decouple different nested level representations and better distinguish entity types. We jointly train both modules sharing a common representation and a new dual info attention layer, which leads to improved representation focus on entity related information. Experiments over different datasets show that our approach outperforms previous state of the art methods and achieves 86.41, 85.46, and 94.2 F1 scores on ACE2004, ACE2005, and NNE, respectively.


Ranger21 a synergistic deep learning optimizer. As optimizers are critical to the performances of neural networks, every year a large number of papers innovating on the subject are published. However, while most of these publications provide incremental improvements to existing algorithms, they tend to be presented as new optimizers rather than composable algorithms. Thus, many worthwhile improvements are rarely seen out of their initial publication. Taking advantage of this untapped potential, we introduce Ranger21, a new optimizer which combines AdamW with eight components, carefully selected after reviewing and testing ideas from the literature. We found that the resulting optimizer provides significantly improved validation accuracy and training speed, smoother training curves, and is even able to train a ResNet50 on ImageNet2012 without Batch Normalization layers. A problem on which AdamW stays systematically stuck in a bad initial state.


Improving Dither Modulation based Robust Steganography by Overflow Suppression. Nowadays, people are sharing their pictures on online social networks (OSNs), so OSN is a good platform for Steganography. But OSNs usually perform JPEG compression on the uploaded image, which will invalidate most of the existing steganography algorithms. Recently, some works try to design robust steganography which can resist JPEG compression, such as Dither Modulation based robust Adaptive Steganography (DMAS) and Generalized dither Modulation based robust Adaptive Steganography (GMAS). They relieve the problem that the receivers cannot extract the message correctly when the quality factor of channel JPEG compression is larger than that of cover images. However, they only can realize limited resistance to detection and compression due to robust domain selection. To overcome this problem, we meticulously explore three lossy operations in the JPEG recompression and discover that the key problem is spatial overflow. Then two preprocessing methods Overall Scaling (OS) and Specific Truncation (ST) are presented to remove overflow before message embedding as well as generate a reference image. The reference image is employed as the guidance to build asymmetric distortion for removing overflow during embedding. Experimental results show that the proposed methods significantly surpass GMAS in terms of security and achieve comparable robustness.


Allee effect in models of interacting species. The search for more realistic models for interacting species has produced many adaptations of the original Lotka Volterra equations, such as the inclusion of the Allee effect and the different Holling's types of functional response. In the present work we show that a correct implementation of both ideas together requires a careful formulation. We focus our work in the fact that a density dependent carrying capacity, combined with the Allee effect, can lead to meaningless effects. We illustrate the difficulties in predator prey and two species competition models, together with our proposed solution of the careful inclusion of the corresponding cubic terms.


Bulwark Holistic and Verified Security Monitoring of Web Protocols. Modern web applications often rely on third party services to provide their functionality to users. The secure integration of these services is a non trivial task, as shown by the large number of attacks against Single Sign On and Cashier as a Service protocols. In this paper we present Bulwark, a new automatic tool which generates formally verified security monitors from applied pi calculus specifications of web protocols. The security monitors generated by Bulwark offer holistic protection, since they can be readily deployed both at the client side and at the server side, thus ensuring full visibility of the attack surface against web protocols. We evaluate the effectiveness of Bulwark by testing it against a pool of vulnerable web applications that use the OAuth 2.0 protocol or integrate the PayPal payment system.


Explainable Fuzzy Utility Mining on Sequences. Fuzzy systems have good modeling capabilities in several data science scenarios, and can provide human explainable intelligence models with explainability and interpretability. In contrast to transaction data, which have been extensively studied, sequence data are more common in real life applications. To obtain a human explainable data intelligence model for decision making, in this study, we investigate explainable fuzzy theoretic utility mining on multi sequences. Meanwhile, a more normative formulation of the problem of fuzzy utility mining on sequences is formulated. By exploring fuzzy set theory for utility mining, we propose a novel method termed pattern growth fuzzy utility mining (PGFUM) for mining fuzzy high utility sequences with linguistic meaning. In the case of sequence data, PGFUM reflects the fuzzy quantity and utility regions of sequences. To improve the efficiency and feasibility of PGFUM, we develop two compressed data structures with explainable fuzziness. Furthermore, one existing and two new upper bounds on the explainable fuzzy utility of candidates are adopted in three proposed pruning strategies to substantially reduce the search space and thus expedite the mining process. Finally, the proposed PGFUM algorithm is compared with PFUS, which is the only currently available method for the same task, through extensive experimental evaluation. It is demonstrated that PGFUM achieves not only human explainable mining results that contain the original nature of revealable intelligibility, but also high efficiency in terms of runtime and memory cost.


Palm theory for extremes of stationary regularly varying time series and random fields. The tail process bY (Y bi ) bi in Z d of a stationary regularly varying time series or random field bX (X bi ) bi in Z d represents the asymptotic local distribution of bX as seen from its typical exceedance over a threshold u as u to infty . Motivated by the standard Palm theory, we show that every tail process satisfies an invariance property called exceedance stationarity and that this property, together with the polar decomposition of the tail process, characterizes the class of all tail processes. We then restrict to the case when Y bi to 0 as bi to infty and establish a couple of Palm like dualities between the tail process and the so called anchored tail process which, under suitable conditions, represents the asymptotic distribution of a typical cluster of extremes of bX . The main message is that the distribution of the tail process is biased towards clusters with more exceedances. Finally, we use these results to determine the distribution of the typical cluster of extremes for moving average processes with random coefficients and heavy tailed innovations.


On 1 D PDE Based Cardiovascular Flow Bottleneck Modeling and Analysis A Vehicular Traffic Flow Inspired Approach. We illustrate the potential of PDE based traffic flow control in cardiovascular flow analysis, monitoring, and control, presenting a PDE based control oriented formulation, for 1 D blood flow dynamics in the presence of stenosis. This is achieved adopting an approach for modeling and analysis that relies on the potential correspondence of 1 D blood flow dynamics in the presence of stenosis, with 1 D traffic flow dynamics in the presence of bottleneck. We reveal such correspondence in relation to the respective (for the two flow types), speed dynamics and a (consistent with them) fundamental diagram based reduction bottleneck dynamic effects description and resulting boundary conditions and free flow congested regimes characterization.


Online Body Schema Adaptation through Cost Sensitive Active Learning. Humanoid robots have complex bodies and kinematic chains with several Degrees of Freedom (DoF) which are difficult to model. Learning the parameters of a kinematic model can be achieved by observing the position of the robot links during prospective motions and minimising the prediction errors. This work proposes a movement efficient approach for estimating online the body schema of a humanoid robot arm in the form of Denavit Hartenberg (DH) parameters. A cost sensitive active learning approach based on the A Optimality criterion is used to select optimal joint configurations. The chosen joint configurations simultaneously minimise the error in the estimation of the body schema and minimise the movement between samples. This reduces energy consumption, along with mechanical fatigue and wear, while not compromising the learning accuracy. The work was implemented in a simulation environment, using the 7DoF arm of the iCub robot simulator. The hand pose is measured with a single camera via markers placed in the palm and back of the robot's hand. A non parametric occlusion model is proposed to avoid choosing joint configurations where the markers are not visible, thus preventing worthless attempts. The results show cost sensitive active learning has similar accuracy to the standard active learning approach, while reducing in about half the executed movement.


A globally convergent numerical method for a 3D coefficient inverse problem for a wave like equation. A version of the convexification globally convergent numerical method is constructed for a coefficient inverse problem for a wave like partial differential equation. The presence of the Carleman Weight Function in the corresponding Tikhonov like cost functional ensures the global strict convexity of this functional. Numerical results are presented to illustrate the effectiveness and efficiency of the proposed method.


Chances for the honest in honest versus insider trading. We study a Black Scholes market with a finite time horizon and two investors an honest and an insider trader. We analyze it with anticipating stochastic calculus in two steps. First, we recover the classical result on portfolio optimization that shows that the expected logarithmic utility of the insider is strictly greater than that of the honest trader. Then, we prove that, whenever the market is viable, the honest trader can get a higher logarithmic utility, and therefore more wealth, than the insider with a strictly positive probability. Our proof relies on the analysis of a sort of forward integral variant of the Dol 'eans Dade exponential process. The main financial conclusion is that the logarithmic utility is perhaps too conservative for some insiders.


Developing a learning goal framework for computational thinking in computationally integrated physics classrooms. Computational thinking has been a recent focus of education research within the sciences. However, there is a dearth of scholarly literature on how best to teach and to assess this topic, especially in disciplinary science courses. Physics classes with computation integrated into the curriculum are a fitting setting for investigating computational thinking. In this paper, we lay the foundation for exploring computational thinking in introductory physics courses. First, we review relevant literature to synthesize a set of potential learning goals that students could engage in when working with computation. The computational thinking framework that we have developed features 14 practices contained within 6 different categories. We use in class video data as existence proofs of the computational thinking practices proposed in our framework. In doing this work, we hope to provide ways for teachers to assess their students' development of computational thinking, while also giving physics education researchers some guidance on how to study this topic in greater depth.


3D Tensor based Deep Learning Models for Predicting Option Price. Option pricing is a significant problem for option risk management and trading. In this article, we utilize a framework to present financial data from different sources. The data is processed and represented in a form of 2D tensors in three channels. Furthermore, we propose two deep learning models that can deal with 3D tensor data. Experiments performed on the Chinese market option dataset prove the practicability of the proposed strategies over commonly used ways, including B S model and vector based LSTM.


Open World Active Learning with Stacking Ensemble for Self Driving Cars. The environments, in which autonomous cars act, are high risky, dynamic, and full of uncertainty, demanding a continuous update of their sensory information and knowledge bases. The frequency of facing an unknown object is too high making hard the usage of Artificial Intelligence (AI) classical classification models that usually rely on the close world assumption. This problem of classifying objects in this domain is better faced with and open world AI approach. We propose an algorithm to identify not only all the known entities that may appear in front of the car, but also to detect and learn the classes of those unknown objects that may be rare to stand on an highway (e.g., a lost box from a truck). Our approach relies on the DOC algorithm from Lei Shu et. al. as well as on the Query by Committee algorithm.


Learning High Dimensional Distributions with Latent Neural Fokker Planck Kernels. Learning high dimensional distributions is an important yet challenging problem in machine learning with applications in various domains. In this paper, we introduce new techniques to formulate the problem as solving Fokker Planck equation in a lower dimensional latent space, aiming to mitigate challenges in high dimensional data space. Our proposed model consists of latent distribution morphing, a generator and a parameterized Fokker Planck kernel function. One fascinating property of our model is that it can be trained with arbitrary steps of latent distribution morphing or even without morphing, which makes it flexible and as efficient as Generative Adversarial Networks (GANs). Furthermore, this property also makes our latent distribution morphing an efficient plug and play scheme, thus can be used to improve arbitrary GANs, and more interestingly, can effectively correct failure cases of the GAN models. Extensive experiments illustrate the advantages of our proposed method over existing models.


Arithmetic Intensity Guided Fault Tolerance for Neural Network Inference on GPUs. Neural networks (NNs) are increasingly employed in safety critical domains and in environments prone to unreliability (e.g., soft errors), such as on spacecraft. Therefore, it is critical to impart fault tolerance to NN inference. Algorithm based fault tolerance (ABFT) is emerging as an efficient approach for fault tolerance in NNs. We propose an adaptive approach to ABFT for NN inference that exploits untapped opportunities in emerging deployment scenarios. GPUs have high compute to memory bandwidth ratios, while NN layers have a wide range of arithmetic intensities. This leaves some layers compute bound and others memory bandwidth bound, but current approaches to ABFT do not consider these differences. We first investigate ABFT schemes best suited for each of these scenarios. We then propose intensity guided ABFT, an adaptive, arithmetic intensity guided approach that selects the most efficient ABFT scheme for each NN layer. Intensity guided ABFT reduces execution time overhead by 1.09 5.3 times across many NNs compared to traditional approaches to ABFT.


Invariant Language Modeling. Modern pretrained language models are critical components of NLP pipelines. Yet, they suffer from spurious correlations, poor out of domain generalization, and biases. Inspired by recent progress in causal machine learning, in particular the invariant risk minimization (IRM) paradigm, we propose invariant language modeling, a framework for learning invariant representations that generalize better across multiple environments. In particular, we adapt a game theoretic implementation of IRM (IRM games) to language models, where the invariance emerges from a specific training schedule in which all the environments compete to optimize their own environment specific loss by updating subsets of the model in a round robin fashion. In a series of controlled experiments, we demonstrate the ability of our method to (i) remove structured noise, (ii) ignore specific spurious correlations without affecting global performance, and (iii) achieve better out of domain generalization. These benefits come with a negligible computational overhead compared to standard training, do not require changing the local loss, and can be applied to any language model architecture. We believe this framework is promising to help mitigate spurious correlations and biases in language models.


MOVO a dApp for DLT based Smart Mobility. Plenty of research on smart mobility is currently devoted to the inclusion of novel decentralized software architectures to these systems, due to the inherent advantages in terms of transparency, traceability, trustworthiness. MOVO is a decentralized application (dApp) for smart mobility. It includes (i) a module for collecting data from vehicles and smartphones sensors (ii) a component for interacting with Distributed Ledger Technologies (DLT) and Decentralized File Storages (DFS), for storing and validating sensor data (iii) a module for "offline" interaction between devices. The dApp consists of an Android application intended for use inside a vehicle, which helps the user driver collect contextually generated data (e.g. a driver's stress level, an electric vehicle's battery level), which can then be shared through the use of DLT (i.e., IOTA DLT and Ethereum smart contracts) and DFS (i.e., IPFS). The third module consists of an implementation of a communication channel that, via Wi Fi Direct, allows two devices to exchange data and payment information with respect to DLT (i.e. cryptocurrency and token) assets. In this paper, we describe the main software components and provide an experimental evaluation that confirms the viability of the MOVO dApp in real mobility scenarios.


Simple, Light, Yet Formally Verified, Global Common Subexpression Elimination and Loop Invariant Code Motion. We present an approach for implementing a formally certified loop invariant code motion optimization by composing an unrolling pass and a formally certified yet efficient global subexpression elimination.This approach is lightweight each pass comes with a simple and independent proof of correctness.Experiments show the approach significantly narrows the performance gap between the CompCert certified compiler and state of the art optimizing compilers.Our static analysis employs an efficient yet verified hashed set structure, resulting in fast compilation.


Estimation of Air Pollution with Remote Sensing Data Revealing Greenhouse Gas Emissions from Space. Air pollution is a major driver of climate change. Anthropogenic emissions from the burning of fossil fuels for transportation and power generation emit large amounts of problematic air pollutants, including Greenhouse Gases (GHGs). Despite the importance of limiting GHG emissions to mitigate climate change, detailed information about the spatial and temporal distribution of GHG and other air pollutants is difficult to obtain. Existing models for surface level air pollution rely on extensive land use datasets which are often locally restricted and temporally static. This work proposes a deep learning approach for the prediction of ambient air pollution that only relies on remote sensing data that is globally available and frequently updated. Combining optical satellite imagery with satellite based atmospheric column density air pollution measurements enables the scaling of air pollution estimates (in this case NO 2 ) to high spatial resolution (up to sim 10m) at arbitrary locations and adds a temporal component to these estimates. The proposed model performs with high accuracy when evaluated against air quality measurements from ground stations (mean absolute error 6 mu g m 3 ). Our results enable the identification and temporal monitoring of major sources of air pollution and GHGs.


Weighted quadrature for hierarchical B splines. We present weighted quadrature for hierarchical B splines to address the fast formation of system matrices arising from adaptive isogeometric Galerkin methods with suitably graded hierarchical meshes. By exploiting a local tensor product structure, we extend the construction of weighted rules from the tensor product to the hierarchical spline setting. The proposed algorithm has a computational cost proportional to the number of degrees of freedom and advantageous properties with increasing spline degree. To illustrate the performance of the method and confirm the theoretical estimates, a selection of 2D and 3D numerical tests is provided.


Topological Models of Abstract Commensurators. The full solenoid over a topological space X is the inverse limit of all finite covers. When X is a compact Hausdorff space admitting a locally path connected universal cover, we relate the pointed homotopy equivalences of the full solenoid to the abstract commensurator of the fundamental group pi 1(X) . The relationship is an isomorphism when X is an aspherical CW complex. If X is additionally a geodesic metric space and pi 1(X) is residually finite, we show that this topological model is compatible with the realization of the abstract commensurator as a subgroup of the quasi isometry group of pi 1(X) . This is a general topological analogue of work of Biswas, Nag, Odden, Sullivan, and others on the universal hyperbolic solenoid, the full solenoid over a closed surface of genus at least two.


Charged Fluids in Porous Media. The Nernst Planck Darcy system models ionic electrodiffusion in porous media. We consider the system for two ionic species with opposite valences. We prove that the initial value problem for the Nernst Planck Darcy system in periodic domains in two or three dimensions has global weak solutions in W 1, p ( p geq 2 ). We obtain furthermore global existence and uniqueness of smooth solutions for arbitrary large data.


Generalized Leibniz rules and Lipschitzian stability for expected integral mappings. This paper is devoted to the study of the expected integral multifunctions given in the form begin equation operatorname E Phi(x) int T Phi t(x)d mu, end equation where Phi colon T times mathbb R n rightrightarrows mathbb R m is a set valued mapping on a measure space (T, mathcal A , mu) . Such multifunctions appear in applications to stochastic programming, which require developing efficient calculus rules of generalized differentiation. Major calculus rules are developed in this paper for coderivatives of multifunctions operatorname E Phi and second order subdifferentials of the corresponding expected integral functionals with applications to constraint systems arising in stochastic programming. The paper is self contained with presenting in the preliminaries some needed results on sequential first order subdifferential calculus of expected integral functionals taken from the first paper of this series.


Local planar domination revisited. We show how to compute a 20 approximation of a minimum dominating set in a planar graph in a constant number of rounds in the LOCAL model of distributed computing. This improves on the previously best known approximation factor of 52, which was achieved by an elegant and simple algorithm of Lenzen et al. Our algorithm combines ideas from the algorithm of Lenzen et al. with recent work of Czygrinow et al. and Kublenz et al. to reduce to the case of bounded degree graphs, where we can simulate a distributed version of the classical greedy algorithm.


QK Iteration A Self Supervised Representation Learning Algorithm for Image Similarity. Self supervised representation learning is a fundamental problem in computer vision with many useful applications (e.g., image search, instance level recognition, copy detection). In this paper we present a new contrastive self supervised representation learning algorithm in the context of Copy Detection in the 2021 Image Similarity Challenge hosted by Facebook AI Research. Previous work in contrastive self supervised learning has identified the importance of being able to optimize representations while pushing'' against a large number of negative examples. Representative previous solutions either use large batches enabled by modern distributed training systems or maintain queues or memory banks holding recently evaluated representations while relaxing some consistency properties. We approach this problem from a new angle We directly learn a query model and a key model jointly and push representations against a very large number (e.g., 1 million) of negative representations in each SGD step. We achieve this by freezing the backbone on one side and by alternating between a Q optimization step and a K optimization step. During the competition timeframe, our algorithms achieved a micro AP score of 0.3401 on the Phase 1 leaderboard, significantly improving over the baseline mu AP of 0.1556. On the final Phase 2 leaderboard, our model scored 0.1919, while the baseline scored 0.0526. Continued training yielded further improvement. We conducted an empirical study to compare the proposed approach with a SimCLR style strategy where the negative examples are taken from the batch only. We found that our method ( mu AP of 0.3403) significantly outperforms this SimCLR style baseline ( mu AP of 0.2001).


Parameter Decoupling Strategy for Semi supervised 3D Left Atrium Segmentation. Consistency training has proven to be an advanced semi supervised framework and achieved promising results in medical image segmentation tasks through enforcing an invariance of the predictions over different views of the inputs. However, with the iterative updating of model parameters, the models would tend to reach a coupled state and eventually lose the ability to exploit unlabeled data. To address the issue, we present a novel semi supervised segmentation model based on parameter decoupling strategy to encourage consistent predictions from diverse views. Specifically, we first adopt a two branch network to simultaneously produce predictions for each image. During the training process, we decouple the two prediction branch parameters by quadratic cosine distance to construct different views in latent space. Based on this, the feature extractor is constrained to encourage the consistency of probability maps generated by classifiers under diversified features. In the overall training process, the parameters of feature extractor and classifiers are updated alternately by consistency regularization operation and decoupling operation to gradually improve the generalization performance of the model. Our method has achieved a competitive result over the state of the art semi supervised methods on the Atrial Segmentation Challenge dataset, demonstrating the effectiveness of our framework. Code is available at https github.com BX0903 PDC.


KAM renormalization and Herman rings for 2D maps. In this note, we extend the renormalization horseshoe we have recently constructed with N. Goncharuk for analytic diffeomorphisms of the circle to their small two dimensional perturbations. As one consequence, Herman rings with rotation numbers of bounded type survive on a codimension one set of parameters under small two dimensional perturbations.


Can Deep Neural Networks Predict Data Correlations from Column Names?. For humans, it is often possible to predict data correlations from column names. We conduct experiments to find out whether deep neural networks can learn to do the same. If so, e.g., it would open up the possibility of tuning tools that use NLP analysis on schema elements to prioritize their efforts for correlation detection. We analyze correlations for around 120,000 column pairs, taken from around 4,000 data sets. We try to predict correlations, based on column names alone. For predictions, we exploit pre trained language models, based on the recently proposed Transformer architecture. We consider different types of correlations, multiple prediction methods, and various prediction scenarios. We study the impact of factors such as column name length or the amount of training data on prediction accuracy. Altogether, we find that deep neural networks can predict correlations with a relatively high accuracy in many scenarios (e.g., with an accuracy of 95 for long column names).


The Effect of Model Size on Worst Group Generalization. Overparameterization is shown to result in poor test accuracy on rare subgroups under a variety of settings where subgroup information is known. To gain a more complete picture, we consider the case where subgroup information is unknown. We investigate the effect of model size on worst group generalization under empirical risk minimization (ERM) across a wide range of settings, varying 1) architectures (ResNet, VGG, or BERT), 2) domains (vision or natural language processing), 3) model size (width or depth), and 4) initialization (with pre trained or random weights). Our systematic evaluation reveals that increasing model size does not hurt, and may help, worst group test performance under ERM across all setups. In particular, increasing pre trained model size consistently improves performance on Waterbirds and MultiNLI. We advise practitioners to use larger pre trained models when subgroup labels are unknown.


Point Cloud Augmentation with Weighted Local Transformations. Despite the extensive usage of point clouds in 3D vision, relatively limited data are available for training deep neural networks. Although data augmentation is a standard approach to compensate for the scarcity of data, it has been less explored in the point cloud literature. In this paper, we propose a simple and effective augmentation method called PointWOLF for point cloud augmentation. The proposed method produces smoothly varying non rigid deformations by locally weighted transformations centered at multiple anchor points. The smooth deformations allow diverse and realistic augmentations. Furthermore, in order to minimize the manual efforts to search the optimal hyperparameters for augmentation, we present AugTune, which generates augmented samples of desired difficulties producing targeted confidence scores. Our experiments show our framework consistently improves the performance for both shape classification and part segmentation tasks. Particularly, with PointNet , PointWOLF achieves the state of the art 89.7 accuracy on shape classification with the real world ScanObjectNN dataset.


Laminar vortex dynamics around forward swept wings. Forward swept wings offer unique advantages in the aerodynamic performance of air vehicles. However, the low Reynolds number characteristics of such wings have not been explored in the past. In this work, we numerically study laminar separated flows over forward swept wings with semi aspect ratios sAR 0.5 to 2 at a chord based Reynolds number of 400. Forward swept wings generate wakes that are significantly different from those of backward swept wings. For low aspect ratio forward wings, the wakes remain steady due to the strong downwash effects induced by the tip vortices. For larger aspect ratio, the downwash effects weaken over the inboard regions of the wing, allowing unsteady vortex shedding to occur. Further larger aspect ratio allows for the formation of streamwise vortices for highly swept wings, stabilizing the flow. Forward swept wings can generate enhanced lift at high angles of attack than the unswept and backward swept wings, with the cost of high drag. We show through force element analysis that the increased lift of forward swept wings is attributed to the vortical structure that is maintained by the tip vortex induced downwash over the outboard region of wing span. The current findings offer a detailed understanding of the sweep effects on laminar separated flows over forward swept wings, and invite innovative designs of high lift devices.


Reinforcement Learning, Bit by Bit. Reinforcement learning agents have demonstrated remarkable achievements in simulated environments. Data efficiency poses an impediment to carrying this success over to real environments. The design of data efficient agents calls for a deeper understanding of information acquisition and representation. We discuss concepts and regret analysis that together offer principled guidance. This line of thinking sheds light on questions of what information to seek, how to seek that information, and what information to retain. To illustrate concepts, we design simple agents that build on them and present computational results that highlight data efficiency.


AdaptSum Towards Low Resource Domain Adaptation for Abstractive Summarization. State of the art abstractive summarization models generally rely on extensive labeled data, which lowers their generalization ability on domains where such data are not available. In this paper, we present a study of domain adaptation for the abstractive summarization task across six diverse target domains in a low resource setting. Specifically, we investigate the second phase of pre training on large scale generative models under three different settings 1) source domain pre training 2) domain adaptive pre training and 3) task adaptive pre training. Experiments show that the effectiveness of pre training is correlated with the similarity between the pre training data and the target domain task. Moreover, we find that continuing pre training could lead to the pre trained model's catastrophic forgetting, and a learning method with less forgetting can alleviate this issue. Furthermore, results illustrate that a huge gap still exists between the low resource and high resource settings, which highlights the need for more advanced domain adaptation methods for the abstractive summarization task.


Automatic Fairness Testing of Neural Classifiers through Adversarial Sampling. Although deep learning has demonstrated astonishing performance in many applications, there are still concerns about its dependability. One desirable property of deep learning applications with societal impact is fairness (i.e., non discrimination). Unfortunately, discrimination might be intrinsically embedded into the models due to the discrimination in the training data. As a countermeasure, fairness testing systemically identifies discriminatory samples, which can be used to retrain the model and improve the model's fairness. Existing fairness testing approaches however have two major limitations. Firstly, they only work well on traditional machine learning models and have poor performance (e.g., effectiveness and efficiency) on deep learning models. Secondly, they only work on simple structured (e.g., tabular) data and are not applicable for domains such as text. In this work, we bridge the gap by proposing a scalable and effective approach for systematically searching for discriminatory samples while extending existing fairness testing approaches to address a more challenging domain, i.e., text classification. Compared with state of the art methods, our approach only employs lightweight procedures like gradient computation and clustering, which is significantly more scalable and effective. Experimental results show that on average, our approach explores the search space much more effectively (9.62 and 2.38 times more than the state of the art methods respectively on tabular and text datasets) and generates much more discriminatory samples (24.95 and 2.68 times) within a same reasonable time. Moreover, the retrained models reduce discrimination by 57.2 and 60.2 respectively on average.


Thompson like groups, characters, and Reidemeister numbers. We investigate Reidemeister numbers of automorphisms of Thompson like groups. More precisely, we use the Sigma invariant of Bieri Neumann Strebel to prove that the Lodha Moore groups and the braided version of Thompson's group F have property R infty .


The singular set in the Stefan problem. In this paper we analyze the singular set in the Stefan problem and prove the following results The singular set has parabolic Hausdorff dimension at most n 1 . The solution admits a C infty expansion at all singular points, up to a set of parabolic Hausdorff dimension at most n 2 . In mathbb R 3 , the free boundary is smooth for almost every time t , and the set of singular times mathcal S subset mathbb R has Hausdorff dimension at most 1 2 . These results provide us with a refined understanding of the Stefan problem's singularities and answer some long standing open questions in the field.


Orbifolds and minimal modular extensions. Let V be a simple, rational, C 2 cofinite vertex operator algebra and G a finite group acting faithfully on V as automorphisms, which is simply called a rational vertex operator algebra with a G action. It is shown that the category cal E V G generated by the V G submodules of V is a symmetric fusion category braided equivalent to the G module category cal E rm Rep (G) . If V is holomorphic, then the V G module category cal C V G is a minimal modular extension of cal E , and is equivalent to the Drinfeld center cal Z ( rm Vec G alpha ) as modular tensor categories for some alpha in H 3(G,S 1) with a canonical embedding of cal E . Moreover, the collection cal M v( cal E ) of equivalence classes of the minimal modular extensions cal C V G of cal E for holomorphic vertex operator algebras V with a G action form a group, which is isomorphic to a subgroup of H 3(G,S 1). Furthermore, any pointed modular category cal Z ( rm Vec G alpha ) is equivalent to cal C V L G for some positive definite even unimodular lattice L. In general, for any rational vertex operator algebra U with a G action, cal C U G is a minimal modular extension of the braided fusion subcategory cal F generated by the U G submodules of U modules. Furthermore, the group cal M v( cal E ) acts freely on the set of equivalence classes cal M v( cal F ) of the minimal modular extensions cal C W G of cal F for any rational vertex operators algebra W with a G action.


On ill posedness of nonlinear stochastic wave equations driven by rough noise. We highlight a fundamental ill posedness issue for nonlinear stochastic wave equations driven by a fractional noise. Namely, if the noise becomes too rough (i.e., the sum of its Hurst indexes becomes too small), then there is essentially no hope to provide an interpretation of the model, whether directly or through a Wick type renormalization procedure. This phenomenon can be compared with the situation of a general SDE driven by a two dimensional fractional noise of index H leq frac 1 4 . Our results clarify and extend previous similar properties exhibited in 3 or in 14 .


Sparsifying Priors for Bayesian Uncertainty Quantification in Model Discovery. We propose a probabilistic model discovery method for identifying ordinary differential equations (ODEs) governing the dynamics of observed multivariate data. Our method is based on the sparse identification of nonlinear dynamics (SINDy) framework, in which target ODE models are expressed as a sparse linear combinations of pre specified candidate functions. Promoting parsimony through sparsity in SINDy leads to interpretable models that generalize to unknown data. Instead of targeting point estimates of the SINDy (linear combination) coefficients, in this work we estimate these coefficients via sparse Bayesian inference. The resulting method, uncertainty quantification SINDy (UQ SINDy), quantifies not only the uncertainty in the values of the SINDy coefficients due to observation errors and limited data, but also the probability of inclusion of each candidate function in the linear combination. UQ SINDy promotes robustness against observation noise and limited data, interpretability (in terms of model selection and inclusion probabilities), and generalization capacity for out of sample forecast. Sparse inference for UQ SINDy employs Markov Chain Monte Carlo, and we explore two sparsifying priors the spike and slab prior, and the regularized horseshoe prior. We apply UQ SINDy to synthetic nonlinear data sets from a Lotka Volterra model and a nonlinear oscillator, and to a real world data set of lynx and hare populations. We find that UQ SINDy is able to discover accurate and meaningful models even in the presence of noise and limited data samples.


Dense anomaly detection by robust learning on synthetic negative data. Standard machine learning is unable to accommodate inputs which do not belong to the training distribution. The resulting models often give rise to confident incorrect predictions which may lead to devastating consequences. This problem is especially demanding in the context of dense prediction since input images may be partially anomalous. Previous work has addressed dense anomaly detection by discriminative training on mixed content images. We extend this approach with synthetic negative patches which simultaneously achieve high inlier likelihood and uniform discriminative prediction. We generate synthetic negatives with normalizing flows due to their outstanding distribution coverage and capability to generate samples at different resolutions. We also propose to detect anomalies according to a principled information theoretic criterion which can be consistently applied through training and inference. The resulting models set the new state of the art on standard benchmarks and datasets in spite of minimal computational overhead and refraining from auxiliary negative data.


Bridging the Distribution Gap of Visible Infrared Person Re identification with Modality Batch Normalization. Visible infrared cross modality person re identification (VI ReID), whose aim is to match person images between visible and infrared modality, is a challenging cross modality image retrieval task. Most existing works integrate batch normalization layers into their neural network, but we found out that batch normalization layers would lead to two types of distribution gap 1) inter mini batch distribution gap the distribution gap of the same modality between each mini batch 2) intra mini batch modality distribution gap the distribution gap of different modality within the same mini batch. To address these problems, we propose a new batch normalization layer called Modality Batch Normalization (MBN), which normalizes each modality sub mini batch respectively instead of the whole mini batch, and can reduce these distribution gap significantly. Extensive experiments show that our MBN is able to boost the performance of VI ReID models, even with different datasets, backbones and losses.


MisConv Convolutional Neural Networks for Missing Data. Processing of missing data by modern neural networks, such as CNNs, remains a fundamental, yet unsolved challenge, which naturally arises in many practical applications, like image inpainting or autonomous vehicles and robots. While imputation based techniques are still one of the most popular solutions, they frequently introduce unreliable information to the data and do not take into account the uncertainty of estimation, which may be destructive for a machine learning model. In this paper, we present MisConv, a general mechanism, for adapting various CNN architectures to process incomplete images. By modeling the distribution of missing values by the Mixture of Factor Analyzers, we cover the spectrum of possible replacements and find an analytical formula for the expected value of convolution operator applied to the incomplete image. The whole framework is realized by matrix operations, which makes MisConv extremely efficient in practice. Experiments performed on various image processing tasks demonstrate that MisConv achieves superior or comparable performance to the state of the art methods.


Graph distances in scale free percolation the logarithmic case. Scale free percolation is a stochastic model for complex networks. In this spatial random graph model, vertices x,y in mathbb Z d are linked by an edge with probability depending on i.i.d. vertex weights and the Euclidean distance x y . Depending on the various parameters involved, we get a rich phase diagram. We study graph distances and compare it to the Euclidean distance of the vertices. Our main attention is on a regime where graph distances are (poly )logarithmic in the Euclidean distance. We obtain improved bounds on the logarithmic exponents. In the light tail regime, the correct exponent is identified.


SurvODE Extrapolating Gene Expression Distribution for Early Cancer Identification. With the increasingly available large scale cancer genomics datasets, machine learning approaches have played an important role in revealing novel insights into cancer development. Existing methods have shown encouraging performance in identifying genes that are predictive for cancer survival, but are still limited in modeling the distribution over genes. Here, we proposed a novel method that can simulate the gene expression distribution at any given time point, including those that are out of the range of the observed time points. In order to model the irregular time series where each patient is one observation, we integrated a neural ordinary differential equation (neural ODE) with cox regression into our framework. We evaluated our method on eight cancer types on TCGA and observed a substantial improvement over existing approaches. Our visualization results and further analysis indicate how our method can be used to simulate expression at the early cancer stage, offering the possibility for early cancer identification.


ICT Use and Livelihoods of Women Microenterprises in Malawi. This study aimed to assess the impacts of ICTs on livelihoods of women microenterprises in Malawi. The study was an interpretive qualitative approach in which semi structured interviews, observation and field notes were used to collect data. About 25 women involved in various microenterprises in three rural areas of Karonga district in Malawi were purposively selected to participate in the study. The framework for the study was based on Serrat's sustainable livelihood approach. The study noted that the use of ICTs potentially enabled women microenterprises to build their financial, human, social, and informational capital assets. The study found that ICTs to some extent contributed to the livelihoods of women microenterprises such as improved access to information diversification of business opportunities, improved communication, improved marketing, and reduced transport costs. As a result, this led to sustainable use of resources, improved well being, and empowerment for women. However, unreliable electricity lack of affordable ICT devices lack of awareness to utilise ICTs in businesses and lack of ICT literacy skills were major concerns that affected women microenterprises. The study offers insights to research practioners, policy makers and other stakeholders on the role of ICTs in fostering women microentrepreneurship in rural communities of Malawi.


Bound States in the Continuum in Compact Acoustic Resonators. We reveal that finite size solid acoustic resonators can support genuine bound states in the continuum (BICs) completely localized inside the resonator. The developed theory provides the multipole classification of such BICs in the resonators of various shapes. It is shown how breaking of the resonator's symmetry turns BICs into quasi BICs manifesting themselves in the scattering spectra as high Q Fano resonances. We believe that the revealed novel states will push the performance limits of acoustic devices and will serve as high Q building blocks for acoustic sensors, antennas, and topological acoustic structures.


Pullback formula for vector valued Siegel modular forms and its applications. Let E n be the Siegel Eisenstein series of degree n and weight k with a complex parameter s . In this paper, using a differential operator D by Ibukiyama which sends a scalar valued Siegel modular form to the tensor product of two vector valued Siegel modular forms, under a certain condition, we give a formula of DE p q on H p times H q , where H n is the Siegel upper half space of degree n . Furthermore, we give some applications of this formula, i.e., analytic properies of standard L functions and the Klingen Eisenstein series and algebraicity results for Siegel modular forms and standard L functions.


Sparsity of Integral Points on Moduli Spaces of Varieties. Let X be a quasi projective variety over a number field, admitting (after passage to mathbb C ) a geometric variation of Hodge structure whose period mapping has zero dimensional fibers. Then the integral points of X are sparse the number of such points of height leq B grows slower than any positive power of B . For example, homogeneous integral polynomials in a fixed number of variables and degree, with discriminant divisible only by a fixed set of primes, are sparse when considered up to integral linear substitutions.


Learning grounded word meaning representations on similarity graphs. This paper introduces a novel approach to learn visually grounded meaning representations of words as low dimensional node embeddings on an underlying graph hierarchy. The lower level of the hierarchy models modality specific word representations through dedicated but communicating graphs, while the higher level puts these representations together on a single graph to learn a representation jointly from both modalities. The topology of each graph models similarity relations among words, and is estimated jointly with the graph embedding. The assumption underlying this model is that words sharing similar meaning correspond to communities in an underlying similarity graph in a low dimensional space. We named this model Hierarchical Multi Modal Similarity Graph Embedding (HM SGE). Experimental results validate the ability of HM SGE to simulate human similarity judgements and concept categorization, outperforming the state of the art.


The Signed Cumulative Distribution Transform for 1 D Signal Analysis and Classification. This paper presents a new mathematical signal transform that is especially suitable for decoding information related to non rigid signal displacements. We provide a measure theoretic framework to extend the existing Cumulative Distribution Transform ACHA 45 (2018), no. 3, 616 641 to arbitrary (signed) signals on overline mathbb R . We present both forward (analysis) and inverse (synthesis) formulas for the transform, and describe several of its properties including translation, scaling, convexity, linear separability and others. Finally, we describe a metric in transform space, and demonstrate the application of the transform in classifying (detecting) signals under random displacements.


AR Based Half Duplex Attack in Beyond 5G networks. With the evolution of WCN (Wireless communication networks), the absolute fulfillment of security occupies the fundamental concern. In view of security, we have identified another research direction based on the attenuation impact of rain in WCN. An approach is initiated by an eavesdropper in which a secure communication environment is degraded by generating Artificial Rain (AR), which creates an abatement in the secrecy rate, and the cybersecurity gets compromised. By doing so, an attacking scenario is perceived, in which an intruder models a Half Duplex (HD) attack. Half Duplex specifies the attack on the downlink instead of targeting both uplink and downlink. This allows the attacker to alleviate the miss rate of the attacking attempts. The layout for the HD attack is explained using RRC (Radio Resource Control) setup. Further, we have determined and examined the performance parameters such as secrecy rate, energy efficiency, miss rate, sensitivity in the presence of AR. Further comparison of rural and urban scenarios in the presence and absence of AR is carried out concerning the variation in secrecy rate with respect to the millimeter wave frequencies and distance. Lastly, the methodology of the HD attack is simulated, revealing that the HD attack maintains a low miss rate with improved performance as compared to the performance and miss rate attained by the full duplex attack


Question Answering over Knowledge Graphs with Neural Machine Translation and Entity Linking. The goal of Question Answering over Knowledge Graphs (KGQA) is to find answers for natural language questions over a knowledge graph. Recent KGQA approaches adopt a neural machine translation (NMT) approach, where the natural language question is translated into a structured query language. However, NMT suffers from the out of vocabulary problem, where terms in a question may not have been seen during training, impeding their translation. This issue is particularly problematic for the millions of entities that large knowledge graphs describe. We rather propose a KGQA approach that delegates the processing of entities to entity linking (EL) systems. NMT is then used to create a query template with placeholders that are filled by entities identified in an EL phase. Slot filling is used to decide which entity fills which placeholder. Experiments for QA over Wikidata show that our approach outperforms pure NMT while there remains a strong dependence on having seen similar query templates during training, errors relating to entities are greatly reduced.


Stochastic Wasserstein Hamiltonian Flows. In this paper, we study the stochastic Hamiltonian flow in Wasserstein manifold, the probability density space equipped with L 2 Wasserstein metric tensor, via the Wong Zakai approximation. We begin our investigation by showing that the stochastic Euler Lagrange equation, regardless it is deduced from either variational principle or particle dynamics, can be interpreted as the stochastic kinetic Hamiltonian flows in Wasserstein manifold. We further propose a novel variational formulation to derive more general stochastic Wassersetin Hamiltonian flows, and demonstrate that this new formulation is applicable to various systems including the stochastic Schr "odinger equation, Schr "odinger equation with random dispersion, and Schr "odinger bridge problem with common noise.


Fully non linear elliptic equations on compact almost Hermitian manifolds. In this paper, we establish a priori estimates for solutions of a general class of fully non linear equations on compact almost Hermitian manifolds. As an application, we solve the complex Hessian equation and the Monge Amp ere equation for (n 1) plurisubharmonic equations in the almost Hermitian setting.


Improving Neural Ranking via Lossless Knowledge Distillation. We explore a novel perspective of knowledge distillation (KD) for learning to rank (LTR), and introduce Self Distilled neural Rankers (SDR), where student rankers are parameterized identically to their teachers. Unlike the existing ranking distillation work which pursues a good trade off between performance and efficiency, SDR is able to significantly improve ranking performance of students over the teacher rankers without increasing model capacity. The key success factors of SDR, which differs from common distillation techniques for classification are (1) an appropriate teacher score transformation function, and (2) a novel listwise distillation framework. Both techniques are specifically designed for ranking problems and are rarely studied in the existing knowledge distillation literature. Building upon the state of the art neural ranking structure, SDR is able to push the limits of neural ranking performance above a recent rigorous benchmark study and significantly outperforms traditionally strong gradient boosted decision tree based models on 7 out of 9 key metrics, the first time in the literature. In addition to the strong empirical results, we give theoretical explanations on why listwise distillation is effective for neural rankers, and provide ablation studies to verify the necessity of the key factors in the SDR framework.


Neural Network Repair with Reachability Analysis. Safety is a critical concern for the next generation of autonomy that is likely to rely heavily on deep neural networks for perception and control. Formally verifying the safety and robustness of well trained DNNs and learning enabled systems under attacks, model uncertainties, and sensing errors is essential for safe autonomy. This research proposes a framework to repair unsafe DNNs in safety critical systems with reachability analysis. The repair process is inspired by adversarial training which has demonstrated high effectiveness in improving the safety and robustness of DNNs. Different from traditional adversarial training approaches where adversarial examples are utilized from random attacks and may not be representative of all unsafe behaviors, our repair process uses reachability analysis to compute the exact unsafe regions and identify sufficiently representative examples to enhance the efficacy and efficiency of the adversarial training. The performance of our framework is evaluated on two types of benchmarks without safe models as references. One is a DNN controller for aircraft collision avoidance with access to training data. The other is a rocket lander where our framework can be seamlessly integrated with the well known deep deterministic policy gradient (DDPG) reinforcement learning algorithm. The experimental results show that our framework can successfully repair all instances on multiple safety specifications with negligible performance degradation. In addition, to increase the computational and memory efficiency of the reachability analysis algorithm, we propose a depth first search algorithm that combines an existing exact analysis method with an over approximation approach based on a new set representation. Experimental results show that our method achieves a five fold improvement in runtime and a two fold improvement in memory usage compared to exact analysis.


Multimodal datasets misogyny, pornography, and malignant stereotypes. We have now entered the era of trillion parameter machine learning models trained on billion sized datasets scraped from the internet. The rise of these gargantuan datasets has given rise to formidable bodies of critical work that has called for caution while generating these large datasets. These address concerns surrounding the dubious curation practices used to generate these datasets, the sordid quality of alt text data available on the world wide web, the problematic content of the CommonCrawl dataset often used as a source for training large language models, and the entrenched biases in large scale visio linguistic models (such as OpenAI's CLIP model) trained on opaque datasets (WebImageText). In the backdrop of these specific calls of caution, we examine the recently released LAION 400M dataset, which is a CLIP filtered dataset of Image Alt text pairs parsed from the Common Crawl dataset. We found that the dataset contains, troublesome and explicit images and text pairs of rape, pornography, malign stereotypes, racist and ethnic slurs, and other extremely problematic content. We outline numerous implications, concerns and downstream harms regarding the current state of large scale datasets while raising open questions for various stakeholders including the AI community, regulators, policy makers and data subjects.


Non destructive Characterization of Anti Reflective Coatings on PV Modules. Anti reflective coatings (ARCs) are used on the vast majority of solar photovoltaic (PV) modules to increase power production. However, ARC longevity can vary from less than 1 year to over 15 years depending on coating quality and deployment conditions. A technique that can quantify ARC degradation non destructively on commercial modules would be useful both for in field diagnostics and accelerated aging tests. In this paper, we demonstrate that accurate measurements of ARC spectral reflectance can be performed using a modified commercially available integrating sphere probe. The measurement is fast, accurate, non destructive and can be performed outdoors in full sun conditions. We develop an interferometric model that estimates coating porosity, thickness and fractional area coverage from the measured reflectance spectrum for a uniform single layer coating. We demonstrate the measurement outdoors on an active PV installation, identify the presence of an ARC and estimate the properties of the coating.


Visual Tactile Cross Modal Data Generation using Residue Fusion GAN with Feature Matching and Perceptual Losses. Existing psychophysical studies have revealed that the cross modal visual tactile perception is common for humans performing daily activities. However, it is still challenging to build the algorithmic mapping from one modality space to another, namely the cross modal visual tactile data translation generation, which could be potentially important for robotic operation. In this paper, we propose a deep learning based approach for cross modal visual tactile data generation by leveraging the framework of the generative adversarial networks (GANs). Our approach takes the visual image of a material surface as the visual data, and the accelerometer signal induced by the pen sliding movement on the surface as the tactile data. We adopt the conditional GAN (cGAN) structure together with the residue fusion (RF) module, and train the model with the additional feature matching (FM) and perceptual losses to achieve the cross modal data generation. The experimental results show that the inclusion of the RF module, and the FM and the perceptual losses significantly improves cross modal data generation performance in terms of the classification accuracy upon the generated data and the visual similarity between the ground truth and the generated data.


Algebraic versus geometric categorification of the Alexander polynomial a spectral sequence. We construct a bigraded spectral sequence from the gl(0) homology to knot Floer homology. This spectral sequence is of Bockstein type and comes from a subtle manipulation of coefficients. The main tools are quantum traces of foams and of singular Soergel bimodules.


Uncertainty Aware Credit Card Fraud Detection Using Deep Learning. Countless research works of deep neural networks (DNNs) in the task of credit card fraud detection have focused on improving the accuracy of point predictions and mitigating unwanted biases by building different network architectures or learning models. Quantifying uncertainty accompanied by point estimation is essential because it mitigates model unfairness and permits practitioners to develop trustworthy systems which abstain from suboptimal decisions due to low confidence. Explicitly, assessing uncertainties associated with DNNs predictions is critical in real world card fraud detection settings for characteristic reasons, including (a) fraudsters constantly change their strategies, and accordingly, DNNs encounter observations that are not generated by the same process as the training distribution, (b) owing to the time consuming process, very few transactions are timely checked by professional experts to update DNNs. Therefore, this study proposes three uncertainty quantification (UQ) techniques named Monte Carlo dropout, ensemble, and ensemble Monte Carlo dropout for card fraud detection applied on transaction data. Moreover, to evaluate the predictive uncertainty estimates, UQ confusion matrix and several performance metrics are utilized. Through experimental results, we show that the ensemble is more effective in capturing uncertainty corresponding to generated predictions. Additionally, we demonstrate that the proposed UQ methods provide extra insight to the point predictions, leading to elevate the fraud prevention process.


Unsupervised Matching of Data and Text. Entity resolution is a widely studied problem with several proposals to match records across relations. Matching textual content is a widespread task in many applications, such as question answering and search. While recent methods achieve promising results for these two tasks, there is no clear solution for the more general problem of matching textual content and structured data. We introduce a framework that supports this new task in an unsupervised setting for any pair of corpora, being relational tables or text documents. Our method builds a fine grained graph over the content of the corpora and derives word embeddings to represent the objects to match in a low dimensional space. The learned representation enables effective and efficient matching at different granularity, from relational tuples to text sentences and paragraphs. Our flexible framework can exploit pre trained resources, but it does not depends on their existence and achieves better quality performance in matching content when the vocabulary is domain specific. We also introduce optimizations in the graph creation process with an "expand and compress" approach that first identifies new valid relationships across elements, to improve matching, and then prunes nodes and edges, to reduce the graph size. Experiments on real use cases and public datasets show that our framework produces embeddings that outperform word embeddings and fine tuned language models both in results' quality and in execution times.


Nordhaus Gaddum type inequality for the fractional matching number of a graph. The fractional matching number of a graph G, is the maximum size of a fractional matching of G. The following sharp lower bounds for a graph G of order n are proved, and all extremal graphs are characterized in this paper. (1)The sum of the fractional matching number of a graph G and the fractional matching number of its complement is not less than n 2 , where n is not less than 2. (2) If G and its complement are non empty, then the sum of the fractional matching number of a graph G and the fractional matching number of its complement is not less than (n 1) 2, where n is not less than 28. (3) If G and its complement have no isolated vertices, then the sum of the fractional matching number of a graph G and the fractional matching number of its complement is not less than (n 4) 2, where n is not less than 28.


Advanced Phase Change Materials for Enhanced Meta Displays. Structural colors generated due to light scattering from static all dielectric metasurfaces have successfully enabled high resolution, high saturation and wide gamut color printing applications. Despite recent advances, most demonstrations of these structure dependent colors lack post fabrication tunability that hinders their applicability for front end dynamic display technologies. Phase change materials (PCMs), with significant contrast of their optical properties between their amorphous and crystalline states, have demonstrated promising potentials in reconfigurable nanophotonics. Herein, we leverage a tunable all dielectric reflective metasurface made of a newly emerged class of low loss optical PCMs with superb characteristics, i.e., antimony trisulphide (Sb 2 S 3 ), antimony triselenide (Sb 2 Se 3 ), and binary germanium doped selenide (GeSe 3 ), to realize switchable, high saturation, high efficiency and high resolution structural colors. Having polarization sensitive building blocks, the presented metasurface can generate two different colors when illuminated by two orthogonally polarized incident beams. Such degrees of freedom (i.e., structural state and polarization) enable a single reconfigurable metasurface with fixed geometrical parameters to generate four distinct wide gamut colors suitable for a wide range of applications, including tunable full color printing and displays, information encryption, and anti counterfeiting.


Efficient and Accurate Multi scale Topological Network for Single Image Dehazing. Single image dehazing is a challenging ill posed problem that has drawn significant attention in the last few years. Recently, convolutional neural networks have achieved great success in image dehazing. However, it is still difficult for these increasingly complex models to recover accurate details from the hazy image. In this paper, we pay attention to the feature extraction and utilization of the input image itself. To achieve this, we propose a Multi scale Topological Network (MSTN) to fully explore the features at different scales. Meanwhile, we design a Multi scale Feature Fusion Module (MFFM) and an Adaptive Feature Selection Module (AFSM) to achieve the selection and fusion of features at different scales, so as to achieve progressive image dehazing. This topological network provides a large number of search paths that enable the network to extract abundant image features as well as strong fault tolerance and robustness. In addition, ASFM and MFFM can adaptively select important features and ignore interference information when fusing different scale representations. Extensive experiments are conducted to demonstrate the superiority of our method compared with state of the art methods.


Partner Assisted Learning for Few Shot Image Classification. Few shot Learning has been studied to mimic human visual capabilities and learn effective models without the need of exhaustive human annotation. Even though the idea of meta learning for adaptation has dominated the few shot learning methods, how to train a feature extractor is still a challenge. In this paper, we focus on the design of training strategy to obtain an elemental representation such that the prototype of each novel class can be estimated from a few labeled samples. We propose a two stage training scheme, Partner Assisted Learning (PAL), which first trains a partner encoder to model pair wise similarities and extract features serving as soft anchors, and then trains a main encoder by aligning its outputs with soft anchors while attempting to maximize classification performance. Two alignment constraints from logit level and feature level are designed individually. For each few shot task, we perform prototype classification. Our method consistently outperforms the state of the art method on four benchmarks. Detailed ablation studies of PAL are provided to justify the selection of each component involved in training.


Building Stable Off chain Payment Networks. Payment channel is a protocol which allows cryptocurrency users to route multiple transactions through network without committing them to the main blockchain network (mainnet). This ability makes them the most prominent solution to blockchains' scalability problem. Each modification of payment channels requires a transaction on the mainnet and therefore, big transaction fees. In this paper, we assume that a set of payment transactions are given (batch or online) and we study the problem of scheduling modificiations on payment channels to route all of the transactions with minimum modification cost. We investigate two cost models for aforementioned problem the step cost function in which every channel modification has a constant cost and the linear cost function in which modification costs are proportional to the amount of change. For the step cost function model, we prove impossibility results for both batch and online case. Moreover, some heuristic methods for the batch case are presented and compared. For the linear cost we propose a polynomial time algorithm using linear programming for the batch case.


Asymptotic spectra of large (grid) graphs with a uniform local structure (part II) numerical applications. In the current work we are concerned with sequences of graphs having a grid geometry, with a uniform local structure in a bounded domain Omega subset mathbb R d , d ge 1 . When Omega 0,1 , such graphs include the standard Toeplitz graphs and, for Omega 0,1 d , the considered class includes d level Toeplitz graphs. In the general case, the underlying sequence of adjacency matrices has a canonical eigenvalue distribution, in the Weyl sense, and it has been shown in the theoretical part of this work that we can associate to it a symbol boldsymbol mathfrak f . The knowledge of the symbol and of its basic analytical features provides key information on the eigenvalue structure in terms of localization, spectral gap, clustering, and global distribution. In the present paper, many different applications are discussed and various numerical examples are presented in order to underline the practical use of the developed theory. Tests and applications are mainly obtained from the approximation of differential operators via numerical schemes such as Finite Differences (FDs), Finite Elements (FEs), and Isogeometric Analysis (IgA). Moreover, we show that more applications can be taken into account, since the results presented here can be applied as well to study the spectral properties of adjacency matrices and Laplacian operators of general large graphs and networks, whenever the involved matrices enjoy a uniform local structure.


Part aware Panoptic Segmentation. In this work, we introduce the new scene understanding task of Part aware Panoptic Segmentation (PPS), which aims to understand a scene at multiple levels of abstraction, and unifies the tasks of scene parsing and part parsing. For this novel task, we provide consistent annotations on two commonly used datasets Cityscapes and Pascal VOC. Moreover, we present a single metric to evaluate PPS, called Part aware Panoptic Quality (PartPQ). For this new task, using the metric and annotations, we set multiple baselines by merging results of existing state of the art methods for panoptic segmentation and part segmentation. Finally, we conduct several experiments that evaluate the importance of the different levels of abstraction in this single task.


Cutoff for permuted Markov chains. Let P be a bistochastic matrix of size n , and let Pi be a permutation matrix of size n . In this paper, we are interested in the mixing time of the Markov chain whose transition matrix is given by Q P Pi . In other words, the chain alternates between random steps governed by P and deterministic steps governed by Pi . We show that if the permutation Pi is chosen uniformly at random, then under mild assumptions on P , with high probability, the chain Q exhibits cutoff at time frac log n mathbf h , where mathbf h is the entropic rate of P . Moreover, for deterministic permutations, we improve the upper bound on the mixing time obtained by Chatterjee and Diaconis (2020).


Self conditioning pre trained language models. In this paper we aim to investigate the mechanisms that guide text generation with pre trained Transformer based Language Models (TLMs). Grounded on the Product of Experts formulation by Hinton (1999), we describe a generative mechanism that exploits expert units which naturally exist in TLMs. Such units are responsible for detecting concepts in the input and conditioning text generation on such concepts. We describe how to identify expert units and how to activate them during inference in order to induce any desired concept in the generated output. We find that the activation of a surprisingly small amount of units is sufficient to steer text generation (as little as 3 units in a model with 345M parameters). While the objective of this work is to learn more about how TLMs work, we show that our method is effective for conditioning without fine tuning or using extra parameters, even on fine grained homograph concepts. Additionally, we show that our method can be used to correct gender bias present in the output of TLMs and achieves gender parity for all evaluated contexts. We compare our method with FUDGE and PPLM BoW, and show that our approach is able to achieve gender parity at a lower perplexity. The proposed method is accessible to a wide audience thanks to its simplicity and minimal compute needs. The findings in this paper are a step forward in understanding the generative mechanisms of TLMs.


Correlation scenarios and correlation stress testing. We develop a general approach for stress testing correlations of financial asset portfolios. The correlation matrix of asset returns is specified in a parametric form, where correlations are represented as a function of risk factors, such as country and industry factors. A sparse factor structure linking assets and risk factors is built using Bayesian variable selection methods. Regular calibration yields a joint distribution of economically meaningful stress scenarios of the factors. As such, the method also lends itself as a reverse stress testing framework using the Mahalanobis distance or highest density regions (HDR) on the joint risk factor distribution allows to infer worst case correlation scenarios. We give examples of stress tests on a large portfolio of European and North American stocks.


Signal level Fusion for Indexing and Retrieval of Facial Biometric Data. The growing scope, scale, and number of biometric deployments around the world emphasise the need for research into technologies facilitating efficient and reliable biometric identification queries. This work presents a method of indexing biometric databases, which relies on signal level fusion of facial images (morphing) to create a multi stage data structure and retrieval protocol. By successively pre filtering the list of potential candidate identities, the proposed method makes it possible to reduce the necessary number of biometric template comparisons to complete a biometric identification transaction. The proposed method is extensively evaluated on publicly available databases using open source and commercial off the shelf recognition systems. The results show that using the proposed method, the computational workload can be reduced down to around 30 , while the biometric performance of a baseline exhaustive search based retrieval is fully maintained, both in closed set and open set identification scenarios.


The top order energy of quasilinear wave equations in two space dimensions is uniformly bounded. Alinhac solved a long standing open problem in 2001 and established that quasilinear wave equations in two space dimensions with quadratic null nonlinearities admit global in time solutions, provided that the initial data are compactly supported and sufficiently small in Sobolev norm. In this work, Alinhac obtained an upper bound with polynomial growth in time for the top order energy of the solutions. A natural question then arises whether the time growth is a true phenomena, despite the possible conservation of basic energy. Analogous problems are also of central importance for Schr "odinger equations and the incompressible Euler equations in two space dimensions, as studied by Bourgain, Colliander Keel Staffilani Takaoka Tao, Kiselev Sverak, and others. In the present paper, we establish that the top order energy of the solutions in Alinhac theorem remains globally bounded in time, which is opposite to Alinhac's blowup at infinity conjecture.


A Unified Generative Framework for Various NER Subtasks. Named Entity Recognition (NER) is the task of identifying spans that represent entities in sentences. Whether the entity spans are nested or discontinuous, the NER task can be categorized into the flat NER, nested NER, and discontinuous NER subtasks. These subtasks have been mainly solved by the token level sequence labelling or span level classification. However, these solutions can hardly tackle the three kinds of NER subtasks concurrently. To that end, we propose to formulate the NER subtasks as an entity span sequence generation task, which can be solved by a unified sequence to sequence (Seq2Seq) framework. Based on our unified framework, we can leverage the pre trained Seq2Seq model to solve all three kinds of NER subtasks without the special design of the tagging schema or ways to enumerate spans. We exploit three types of entity representations to linearize entities into a sequence. Our proposed framework is easy to implement and achieves state of the art (SoTA) or near SoTA performance on eight English NER datasets, including two flat NER datasets, three nested NER datasets, and three discontinuous NER datasets.


physics.app-ph physics.flu-dyn physics.pop-ph physics.soc-ph

Technoeconomic Analysis of Thermal Energy Grid Storage Using Graphite and Tin. Energy storage is needed to enable dispatchable renewable energy supply and thereby full decarbonization of the grid. However, this can only occur with drastic cost reductions compared to current battery technology, with predicted targets for the cost per unit energy (CPE) below 20 kWh. Notably, for full decarbonization, long duration storage up to 100 hrs will be needed at such low costs, and prior analyses have shown that in such high renewable penetration scenarios, CPE is more critical than other parameters such as round trip efficiency or cost per unit power when comparing the costs of different technologies. Here, we introduce an electricity storage concept that stores electricity as sensible heat in graphite storage blocks and uses multi junction thermophotovoltaics (TPV) as a heat engine to convert it back to electricity on demand. This design is an outgrowth of the system proposed by Amy et al. in 2019, which has been modified here to use a solid graphite medium and molten tin as a heat transfer fluid rather than silicon as both. The reason for this is two fold (1) the CPE of graphite is almost 10X lower than that of silicon, which derives from the lower cost per unit mass (i.e., 0.5 kg vs. 1.5 kg) and the higher heat capacity per unit mass (2000 J kg K vs. 950 J kg K) and (2) the melting point tin and solubility of tin in graphite are much lower than that of silicon, which lessens the number of issues that have to overcome along the research and development pathway. The usage of graphite also eliminates the need for a second tank, but the main disadvantage of using a solid medium is that one cannot easily provide a steady discharge rate, as the power output from the storage will change with time, as the graphite cools during discharge. Thus, the objective of this work is to examine how these changes in the system design effect the overall technoeconomics.


An Efficient Learning Framework For Federated XGBoost Using Secret Sharing And Distributed Optimization. XGBoost is one of the most widely used machine learning models in the industry due to its superior learning accuracy and efficiency. Targeting at data isolation issues in the big data problems, it is crucial to deploy a secure and efficient federated XGBoost (FedXGB) model. Existing FedXGB models either have data leakage issues or are only applicable to the two party setting with heavy communication and computation overheads. In this paper, a lossless multi party federated XGB learning framework is proposed with a security guarantee, which reshapes the XGBoost's split criterion calculation process under a secret sharing setting and solves the leaf weight calculation problem by leveraging distributed optimization. Remarkably, a thorough analysis of model security is provided as well, and multiple numerical results showcase the superiority of the proposed FedXGB compared with the state of the art models on benchmark datasets.


AligNART Non autoregressive Neural Machine Translation by Jointly Learning to Estimate Alignment and Translate. Non autoregressive neural machine translation (NART) models suffer from the multi modality problem which causes translation inconsistency such as token repetition. Most recent approaches have attempted to solve this problem by implicitly modeling dependencies between outputs. In this paper, we introduce AligNART, which leverages full alignment information to explicitly reduce the modality of the target distribution. AligNART divides the machine translation task into (i) alignment estimation and (ii) translation with aligned decoder inputs, guiding the decoder to focus on simplified one to one translation. To alleviate the alignment estimation problem, we further propose a novel alignment decomposition method. Our experiments show that AligNART outperforms previous non iterative NART models that focus on explicit modality reduction on WMT14 En leftrightarrow De and WMT16 Ro rightarrow En. Furthermore, AligNART achieves BLEU scores comparable to those of the state of the art connectionist temporal classification based models on WMT14 En leftrightarrow De. We also observe that AligNART effectively addresses the token repetition problem even without sequence level knowledge distillation.


Characterizing Affine Toric Varieties via the Automorphism Group. Based on results about commuting automorphisms of affine varieties due to Cantat, Xie and the first author, we show that every affine toric variety that is non isomorphic to an algebraic torus is determined by its group of regular automorphisms in the category of irreducible normal affine varieties. The same conclusion holds for the algebraic torus as well, if we additionally assume that all varieties under consideration have the same dimension. This generalizes results obtained by Cantat, Kraft, Liendo, Urech, Xie and the authors.


Simulation of Derivatives Post Trade Services using an Authoritative Data Store and the ISDA Common Domain Model. In this paper, we present a summary of the design and implementation of a simulation of post trade services for interest rate swaps, from execution to maturity. We use an authoritative data store (ADS) and the International Swaps and Derivatives Association (ISDA) Common Domain Model (CDM) to simulate a potential future architecture. We start by providing a brief overview of the CDM and the lifecycle of an interest rate swap. We then compare our simulated future state architecture with a typical current state architecture. Next, we present the key requirements of the simulated system, several suitable design patterns, and a summary of the implementation. The simulation uses the CDM to address the industry problems of inconsistent processes and inconsistent data, and an authoritative data store to address the industry problem of duplicated data.


Efficient Visual Pretraining with Contrastive Detection. Self supervised pretraining has been shown to yield powerful representations for transfer learning. These performance gains come at a large computational cost however, with state of the art methods requiring an order of magnitude more computation than supervised pretraining. We tackle this computational bottleneck by introducing a new self supervised objective, contrastive detection, which tasks representations with identifying object level features across augmentations. This objective extracts a rich learning signal per image, leading to state of the art transfer accuracy on a variety of downstream tasks, while requiring up to 10x less pretraining. In particular, our strongest ImageNet pretrained model performs on par with SEER, one of the largest self supervised systems to date, which uses 1000x more pretraining data. Finally, our objective seamlessly handles pretraining on more complex images such as those in COCO, closing the gap with supervised transfer learning from COCO to PASCAL.


Instance Level Task Parameters A Robust Multi task Weighting Framework. Recent works have shown that deep neural networks benefit from multi task learning by learning a shared representation across several related tasks. However, performance of such systems depend on relative weighting between various losses involved during training. Prior works on loss weighting schemes assume that instances are equally easy or hard for all tasks. In order to break this assumption, we let the training process dictate the optimal weighting of tasks for every instance in the dataset. More specifically, we equip every instance in the dataset with a set of learnable parameters (instance level task parameters) where the cardinality is equal to the number of tasks learned by the model. These parameters model the weighting of each task for an instance. They are updated by gradient descent and do not require hand crafted rules. We conduct extensive experiments on SURREAL and CityScapes datasets, for human shape and pose estimation, depth estimation and semantic segmentation tasks. In these tasks, our approach outperforms recent dynamic loss weighting approaches, e.g. reducing surface estimation errors by 8.97 on SURREAL. When applied to datasets where one or more tasks can have noisy annotations, the proposed method learns to prioritize learning from clean labels for a given task, e.g. reducing surface estimation errors by up to 60 . We also show that we can reliably detect corrupt labels for a given task as a by product from learned instance level task parameters.


Inference for Generative Capsule Models. Capsule networks (see e.g. Hinton et al., 2018) aim to encode knowledge and reason about the relationship between an object and its parts. In this paper we specify a emph generative model for such data, and derive a variational algorithm for inferring the transformation of each object and the assignments of observed parts to the objects. We apply this model to (i) data generated from multiple geometric objects like squares and triangles ("constellations"), and (ii) data from a parts based model of faces. Recent work by Kosiorek et al. 2019 has used amortized inference via stacked capsule autoencoders (SCAEs) to tackle this problem our results show that we significantly outperform them where we can make comparisons (on the constellations data).


MoDL QSM Model based Deep Learning for Quantitative Susceptibility Mapping. Quantitative susceptibility mapping (QSM) has demonstrated great potential in quantifying tissue susceptibility in various brain diseases. However, the intrinsic ill posed inverse problem relating the tissue phase to the underlying susceptibility distribution affects the accuracy for quantifying tissue susceptibility. Recently, deep learning has shown promising results to improve accuracy by reducing the streaking artifacts. However, there exists a mismatch between the observed phase and the theoretical forward phase estimated by the susceptibility label. In this study, we proposed a model based deep learning architecture that followed the STI (susceptibility tensor imaging) physical model, referred to as MoDL QSM. Specifically, MoDL QSM accounts for the relationship between STI derived phase contrast induced by the susceptibility tensor terms (ki13,ki23,ki33) and the acquired single orientation phase. The convolution neural networks are embedded into the physical model to learn a regularization term containing prior information. ki33 and phase induced by ki13 and ki23 terms were used as the labels for network training. Quantitative evaluation metrics (RSME, SSIM, and HFEN) were compared with recently developed deep learning QSM methods. The results showed that MoDL QSM achieved superior performance, demonstrating its potential for future applications.


ProGS Property Graph Shapes Language (Extended Version). Property graphs constitute data models for representing knowledge graphs. They allow for the convenient representation of facts, including facts about facts, represented by triples in subject or object position of other triples. Knowledge graphs such as Wikidata are created by a diversity of contributors and a range of sources leaving them prone to two types of errors. The first type of error, falsity of facts, is addressed by property graphs through the representation of provenance and validity, making triples occur as first order objects in subject position of metadata triples. The second type of error, violation of domain constraints, has not been addressed with regard to property graphs so far. In RDF representations, this error can be addressed by shape languages such as SHACL or ShEx, which allow for checking whether graphs are valid with respect to a set of domain constraints. Borrowing ideas from the syntax and semantics definitions of SHACL, we design a shape language for property graphs, ProGS, which allows for formulating shape constraints on property graphs including their specific constructs, such as edges with identities and key value annotations to both nodes and edges. We define a formal semantics of ProGS, investigate the resulting complexity of validating property graphs against sets of ProGS shapes, compare with corresponding results for SHACL, and implement a prototypical validator that utilizes answer set programming.