A Zariski like topology on the ideal spectrum of a ring. The purpose of this paper is to introduce a Zariski like topology on the spectrum of all proper ideals of a ring. We show that the space is T 0, quasi compact, and every irreducible closed subset has a unique generic point. Furthermore, this space is weaker than a spectral space and if the ring has a non trivial idempotent element then the space has a closed disconnected subspace.


Cloud Removal from Satellite Images. In this report, we have analyzed available cloud detection technique using sentinel hub. We have also implemented spatial attention generative adversarial network and improved quality of generated image compared to previous solution 7 .


City scale Scene Change Detection using Point Clouds. We propose a method for detecting structural changes in a city using images captured from vehicular mounted cameras over traversals at two different times. We first generate 3D point clouds for each traversal from the images and approximate GNSS INS readings using Structure from Motion (SfM). A direct comparison of the two point clouds for change detection is not ideal due to inaccurate geo location information and possible drifts in the SfM. To circumvent this problem, we propose a deep learning based non rigid registration on the point clouds which allows us to compare the point clouds for structural change detection in the scene. Furthermore, we introduce a dual thresholding check and post processing step to enhance the robustness of our method. We collect two datasets for the evaluation of our approach. Experiments show that our method is able to detect scene changes effectively, even in the presence of viewpoint and illumination differences.


In Defense of Liquid Democracy. Fluid democracy is a voting paradigm that allows voters to choose between directly voting and transitively delegating their votes to other voters. While fluid democracy has been viewed as a system that can combine the best aspects of direct and representative democracy, it can also result in situations where few voters amass a large amount of influence. To analyze the impact of this shortcoming, we consider what has been called an epistemic setting, where voters decide on a binary issue for which there is a ground truth. Previous work has shown that under certain assumptions on the delegation mechanism, the concentration of power is so severe that fluid democracy is less likely to identify the ground truth than direct voting. We examine different, arguably more realistic, classes of mechanisms, and prove they behave well by ensuring that (with high probability) there is a limit on concentration of power. Our proofs demonstrate that delegations can be treated as stochastic processes and that they can be compared to well known processes from the literature such as preferential attachment and multi types branching process that are sufficiently bounded for our purposes. Our results suggest that the concerns raised about fluid democracy can be overcome, thereby bolstering the case for this emerging paradigm.


Spiking Neural Networks for Visual Place Recognition via Weighted Neuronal Assignments. Spiking neural networks (SNNs) offer both compelling potential advantages, including energy efficiency and low latencies and challenges including the non differentiable nature of event spikes. Much of the initial research in this area has converted deep neural networks to equivalent SNNs, but this conversion approach potentially negates some of the advantages of SNN based approaches developed from scratch. One promising area for high performance SNNs is template matching and image recognition. This research introduces the first high performance SNN for the Visual Place Recognition (VPR) task given a query image, the SNN has to find the closest match out of a list of reference images. At the core of this new system is a novel assignment scheme that implements a form of ambiguity informed salience, by up weighting single place encoding neurons and down weighting "ambiguous" neurons that respond to multiple different reference places. In a range of experiments on the challenging Nordland, Oxford RobotCar, SPEDTest, Synthia, and St Lucia datasets, we show that our SNN achieves comparable VPR performance to state of the art and classical techniques, and degrades gracefully in performance with an increasing number of reference places. Our results provide a significant milestone towards SNNs that can provide robust, energy efficient, and low latency robot localization.


Quantum and Semiquantum Pseudometrics and Applications. We establish a Kantorovich duality for the pseudometric mathcal E hbar introduced in F. Golse, T. Paul, Arch. Rational Mech. Anal. 223 (2017), 57 94 , obtained from the usual Monge Kantorovich distance d MK,2 between classical densities by quantization of one of the two densities involved. We show several type of inequalities comparing d MK,2 , mathcal E hbar and MK hbar , a full quantum analogue of d MK,2 introduced in F. Golse, C. Mouhot, T. Paul, Commun. Math. Phys. 343 (2016), 165 205 , including an up to hbar triangle inequality for MK hbar . Finally, we show that, when nice optimal Kantorovich potentials exist for mathcal E hbar , optimal couplings induce classical quantum optimal transports and the potentials are linked by a semiquantum Legendre type transform.


Causal Consistent Replication in Reversible Concurrent Calculi. Reversible computation is key in developing new, energy efficient paradigms, but also in providing forward only concepts with broader definitions and finer frames of study.Among other fields, the algebraic specification and representation of networks of agents have been greatly impacted by the study of reversible phenomena reversible declensions of the calculus of communicating systems (CCSK and RCCS) offer new semantic models, finer congruence relations, original properties, and revisits existing theories and results in a finer light.However, much remains to be done concurrency, a central notion in establishing causal consistency a crucial property for reversibly systems , was never given a clear and syntactical definition in CCSK.While recursion was mentioned as a possible mechanism to inject infinite behaviors into the systems, replication was never studied.This work offers a solution to both problems, by leveraging a definition of concurrency developed for forward only calculi using proved transition systems, by endowing CCSK with a replication operator, and by studying the interplay of both notions.The system we obtain is the first reversible system capable of representing infinite behaviors that enjoys causal consistency, for our simple and purely syntactical notion of reversible concurrency.


Stock Portfolio Optimization Using a Deep Learning LSTM Model. Predicting future stock prices and their movement patterns is a complex problem. Hence, building a portfolio of capital assets using the predicted prices to achieve the optimization between its return and risk is an even more difficult task. This work has carried out an analysis of the time series of the historical prices of the top five stocks from the nine different sectors of the Indian stock market from January 1, 2016, to December 31, 2020. Optimum portfolios are built for each of these sectors. For predicting future stock prices, a long and short term memory (LSTM) model is also designed and fine tuned. After five months of the portfolio construction, the actual and the predicted returns and risks of each portfolio are computed. The predicted and the actual returns of each portfolio are found to be high, indicating the high precision of the LSTM model.


Feature Analyses and Modelling of Lithium ion Batteries Manufacturing based on Random Forest Classification. Lithium ion battery manufacturing is a highly complicated process with strongly coupled feature interdependencies, a feasible solution that can analyse feature variables within manufacturing chain and achieve reliable classification is thus urgently needed. This article proposes a random forest (RF) based classification framework, through using the out of bag (OOB) predictions, Gini changes as well as predictive measure of association (PMOA), for effectively quantifying the importance and correlations of battery manufacturing features and their effects on the classification of electrode properties. Battery manufacturing data containing three intermediate product features from the mixing stage and one product parameter from the coating stage are analysed by the designed RF framework to investigate their effects on both the battery electrode active material mass load and porosity. Illustrative results demonstrate that the proposed RF framework not only achieves the reliable classification of electrode properties but also leads to the effective quantification of both manufacturing feature importance and correlations. This is the first time to design a systematic RF framework for simultaneously quantifying battery production feature importance and correlations by three various quantitative indicators including the unbiased feature importance (FI), gain improvement FI and PMOA, paving a promising solution to reduce model dimension and conduct efficient sensitivity analysis of battery manufacturing.


Explainable Deep Reinforcement Learning for Portfolio Management An Empirical Approach. Deep reinforcement learning (DRL) has been widely studied in the portfolio management task. However, it is challenging to understand a DRL based trading strategy because of the black box nature of deep neural networks. In this paper, we propose an empirical approach to explain the strategies of DRL agents for the portfolio management task. First, we use a linear model in hindsight as the reference model, which finds the best portfolio weights by assuming knowing actual stock returns in foresight. In particular, we use the coefficients of a linear model in hindsight as the reference feature weights. Secondly, for DRL agents, we use integrated gradients to define the feature weights, which are the coefficients between reward and features under a linear regression model. Thirdly, we study the prediction power in two cases, single step prediction and multi step prediction. In particular, we quantify the prediction power by calculating the linear correlations between the feature weights of a DRL agent and the reference feature weights, and similarly for machine learning methods. Finally, we evaluate a portfolio management task on Dow Jones 30 constituent stocks during 01 01 2009 to 09 01 2021. Our approach empirically reveals that a DRL agent exhibits a stronger multi step prediction power than machine learning methods.


Pattern Formation in Slot Die Coating. We experimentally study the occurrence of pattern formation during the slot die coating of low viscosity nearly Newtonian liquids onto Polyethylenterephthalat (PET) substrates. In particular, it is demonstrated that with increase of the coating speed a homogeneous coating becomes unstable with respect to periodic stripe patterns. Thereby, depending on the liquid viscosity, the stripes can be oriented parallel or perpendicular with respect to the coating direction. Mixed states do also occur. The spatial period of perpendicular parallel stripes increases decrease with the coating speed. The dependence of the effect on various control parameters of slot die coating is investigated. Finally, a simple theoretical model based on the hydrodynamics of thin films of partially wetting liquids is analysed. Comparing the results to the experiments, conclusions are drawn regarding the acting instability and pattern formation mechanisms.


Conditions for Eliminating Cusps in One Phase Free Boundary Problems with Degeneracy. In this paper, we continue the study local minimizers of a degenerate version of the Alt Caffarelli functional. Specifically, we consider local minimizers of the functional J Q (u, Omega) int Omega nabla u 2 Q(x) 2 chi u 0 dx where Q(x) dist(x, Gamma) gamma for gamma 0 and Gamma a submanifold of dimension 0 le k le n 1 . Previously, it was shown that on Gamma , the free boundary partial u 0 may be decomposed into a rectifiable set mathcal S , which satisfies effective estimates, and a cusp set Sigma . In this note, we prove that under mild assumptions, in the case n 2 and Gamma a line, the cusp set Sigma does not exist. Building upon the work of Arama and Leoni cite AramaLeoni12 , our results apply to the physical case of a variational formulation of the Stokes' wave and provide a complete characterization of the singular portion of the free boundary in complete generality in this context.


Review of Video Predictive Understanding Early Action Recognition and Future Action Prediction. Video predictive understanding encompasses a wide range of efforts that are concerned with the anticipation of the unobserved future from the current as well as historical video observations. Action prediction is a major sub area of video predictive understanding and is the focus of this review. This sub area has two major subdivisions early action recognition and future action prediction. Early action recognition is concerned with recognizing an ongoing action as soon as possible. Future action prediction is concerned with the anticipation of actions that follow those previously observed. In either case, the textbf textit causal relationship between the past, current, and potential future information is the main focus. Various mathematical tools such as Markov Chains, Gaussian Processes, Auto Regressive modeling, and Bayesian recursive filtering are widely adopted jointly with computer vision techniques for these two tasks. However, these approaches face challenges such as the curse of dimensionality, poor generalization, and constraints from domain specific knowledge. Recently, structures that rely on deep convolutional neural networks and recurrent neural networks have been extensively proposed for improving the performance of existing vision tasks, in general, and action prediction tasks, in particular. However, they have their own shortcomings, eg reliance on massive training data and lack of strong theoretical underpinnings. In this survey, we start by introducing the major sub areas of the broad area of video predictive understanding, which recently have received intensive attention and proven to have practical value. Next, a thorough review of various early action recognition and future action prediction algorithms are provided with suitably organized divisions. Finally, we conclude our discussion with future research directions.


Towards a Complexity Classification of LCL Problems in Massively Parallel Computation. In this work, we develop the low space Massively Parallel Computation (MPC) complexity landscape for a family of fundamental graph problems on trees. We present a general method that solves most locally checkable labeling (LCL) problems exponentially faster in the low space MPC model than in the LOCAL message passing model. In particular, we show that all solvable LCL problems on trees can be solved in O( log n) time (high complexity regime) and that all LCL problems on trees with deterministic complexity n o(1) in the LOCAL model can be solved in O( log log n) time (mid complexity regime). We observe that obtaining a greater speed up than from n o(1) to Theta( log log n) is conditionally impossible, since the problem of 3 coloring trees, which is a LCL problem with LOCAL time complexity n o(1) , has a conditional MPC lower bound of Omega( log log n) Linial, FOCS'87 Ghaffari, Kuhn and Uitto, FOCS'19 . We emphasize that we solve LCL problems on constant degree trees, and that our algorithms are deterministic, component stable, and work in the low space MPC model, where local memory is O(n delta) for delta in (0,1) and global memory is O(m) . For the high complexity regime, there are two key ingredients. One is a novel O( log n) time tree rooting algorithm, which may be of independent interest. The other is a novel pointer chain technique and analysis that allows us to solve any solvable LCL problem on trees in O( log n) time. For the mid complexity regime, we adapt the approach by Chang and Pettie FOCS'17 , who gave a canonical LOCAL algorithm for solving LCL problems on trees.


An Accelerated Variance Reduced Conditional Gradient Sliding Algorithm for First order and Zeroth order Optimization. The conditional gradient algorithm (also known as the Frank Wolfe algorithm) has recently regained popularity in the machine learning community due to its projection free property to solve constrained problems. Although many variants of the conditional gradient algorithm have been proposed to improve performance, they depend on first order information (gradient) to optimize. Naturally, these algorithms are unable to function properly in the field of increasingly popular zeroth order optimization, where only zeroth order information (function value) is available. To fill in this gap, we propose a novel Accelerated variance Reduced Conditional gradient Sliding (ARCS) algorithm for finite sum problems, which can use either first order or zeroth order information to optimize. To the best of our knowledge, ARCS is the first zeroth order conditional gradient sliding type algorithms solving convex problems in zeroth order optimization. In first order optimization, the convergence results of ARCS substantially outperform previous algorithms in terms of the number of gradient query oracle. Finally we validated the superiority of ARCS by experiments on real world datasets.


GNeRF GAN based Neural Radiance Field without Posed Camera. We introduce GNeRF, a framework to marry Generative Adversarial Networks (GAN) with Neural Radiance Field (NeRF) reconstruction for the complex scenarios with unknown and even randomly initialized camera poses. Recent NeRF based advances have gained popularity for remarkable realistic novel view synthesis. However, most of them heavily rely on accurate camera poses estimation, while few recent methods can only optimize the unknown camera poses in roughly forward facing scenes with relatively short camera trajectories and require rough camera poses initialization. Differently, our GNeRF only utilizes randomly initialized poses for complex outside in scenarios. We propose a novel two phases end to end framework. The first phase takes the use of GANs into the new realm for optimizing coarse camera poses and radiance fields jointly, while the second phase refines them with additional photometric loss. We overcome local minima using a hybrid and iterative optimization scheme. Extensive experiments on a variety of synthetic and natural scenes demonstrate the effectiveness of GNeRF. More impressively, our approach outperforms the baselines favorably in those scenes with repeated patterns or even low textures that are regarded as extremely challenging before.


TeraPipe Token Level Pipeline Parallelism for Training Large Scale Language Models. Model parallelism has become a necessity for training modern large scale deep language models. In this work, we identify a new and orthogonal dimension from existing model parallel approaches it is possible to perform pipeline parallelism within a single training sequence for Transformer based language models thanks to its autoregressive property. This enables a more fine grained pipeline compared with previous work. With this key idea, we design TeraPipe, a high performance token level pipeline parallel algorithm for synchronous model parallel training of Transformer based language models. We develop a novel dynamic programming based algorithm to calculate the optimal pipelining execution scheme given a specific model and cluster configuration. We show that TeraPipe can speed up the training by 5.0x for the largest GPT 3 model with 175 billion parameters on an AWS cluster with 48 p3.16xlarge instances compared with state of the art model parallel methods. The code for reproduction can be found at https github.com zhuohan123 terapipe


WAIT Protecting the Integrity of Web Applications with Binary Equivalent Transparency. Modern single page web applications require client side executions of application logic, including critical functionality such as client side cryptography. Existing mechanisms such as TLS and Subresource Integrity secure the communication and provide external resource integrity. However, the browser is unaware of modifications to the client side application as provided by the server and the user remains vulnerable against malicious modifications carried out on the server side. Our solution makes such modifications transparent and empowers the browser to validate the integrity of a web application based on a publicly verifiable log. Our Web Application Integrity Transparency (WAIT) approach requires (1) an extension for browsers for local integrity validations, (2) a custom HTTP header for web servers that host the application, and (3) public log servers that serve the verifiable logs. With WAIT, the browser can disallow the execution of undisclosed application changes. Also, web application providers cannot dispute their authorship for published modifications anymore. Although our approach cannot prevent every conceivable attack on client side web application integrity, it introduces a novel sense of transparency for users and an increased level of accountability for application providers particularly effective against targeted insider attacks.


Model Adaptive Interface Generation for Data Driven Discovery. Discovery of new knowledge is increasingly data driven, predicated on a team's ability to collaboratively create, find, analyze, retrieve, and share pertinent datasets over the duration of an investigation. This is especially true in the domain of scientific discovery where generation, analysis, and interpretation of data are the fundamental mechanisms by which research teams collaborate to achieve their shared scientific goal. Data driven discovery in general, and scientific discovery in particular, is distinguished by complex and diverse data models and formats that evolve over the lifetime of an investigation. While databases and related information systems have the potential to be valuable tools in the discovery process, developing effective interfaces for data driven discovery remains a roadblock to the application of database technology as an essential tool in scientific investigations. In this paper, we present a model adaptive approach to creating interaction environments for data driven discovery of scientific data that automatically generates interactive user interfaces for editing, searching, and viewing scientific data based entirely on introspection of an extended relational data model. We have applied model adaptive interface generation to many active scientific investigations spanning domains of proteomics, bioinformatics, neuroscience, occupational therapy, stem cells, genitourinary, craniofacial development, and others. We present the approach, its implementation, and its evaluation through analysis of its usage in diverse scientific settings.


Moving outside the lab The viability of conducting sensorimotor learning studies online. Collecting data online via crowdsourcing platforms has proven to be a very efficient way to recruit a large and diverse sample. Studies of motor learning, however, have been largely confined to the lab due to the need for special equipment to record movement kinematics and, as such, are typically only accessible to specific participants (e.g., college students). As a first foray to make motor learning studies accessible to a larger and more diverse audience, we developed an online, web based platform (OnPoint) to collect kinematic data, serving as a template for researchers to create their own online sensorimotor control and learning experiments. As a proof of concept, we asked if fundamental motor learning phenomena discovered in the lab could be replicated online. In a series of three experiments, we observed a close correspondence between the results obtained online with those previously reported from research conducted in the laboratory. This web based platform paired with online crowdsourcing can serve as a powerful new method for the study of motor control and learning.


Multigrid solvers for isogeometric discretizations of the second biharmonic problem. We develop a multigrid solver for the second biharmonic problem in the context of Isogeometric Analysis (IgA), where we also allow a zero order term. In a previous paper, the authors have developed an analysis for the first biharmonic problem based on Hackbusch's framework. This analysis can only be extended to the second biharmonic problem if one assumes uniform grids. In this paper, we prove a multigrid convergence estimate using Bramble's framework for multigrid analysis without regularity assumptions. We show that the bound for the convergence rate is independent of the scaling of the zero order term and the spline degree. It only depends linearly on the number of levels, thus logarithmically on the grid size. Numerical experiments are provided which illustrate the convergence theory and the efficiency of the proposed multigrid approaches.


Here and There with Arithmetic. In the theory of answer set programming, two groups of rules are called strongly equivalent if, informally speaking, they have the same meaning in any context. The relationship between strong equivalence and the propositional logic of here and there allows us to establish strong equivalence by deriving rules of each group from rules of the other. In the process, rules are rewritten as propositional formulas. We extend this method of proving strong equivalence to an answer set programming language that includes operations on integers. The formula representing a rule in this language is a first order formula that may contain comparison symbols among its predicate constants, and symbols for arithmetic operations among its function constants. The paper is under consideration for acceptance in TPLP.


CondenseNet V2 Sparse Feature Reactivation for Deep Networks. Reusing features in deep networks through dense connectivity is an effective way to achieve high computational efficiency. The recent proposed CondenseNet has shown that this mechanism can be further improved if redundant features are removed. In this paper, we propose an alternative approach named sparse feature reactivation (SFR), aiming at actively increasing the utility of features for reusing. In the proposed network, named CondenseNetV2, each layer can simultaneously learn to 1) selectively reuse a set of most important features from preceding layers and 2) actively update a set of preceding features to increase their utility for later layers. Our experiments show that the proposed models achieve promising performance on image classification (ImageNet and CIFAR) and object detection (MS COCO) in terms of both theoretical efficiency and practical speed.


Structural Information Preserving for Graph to Text Generation. The task of graph to text generation aims at producing sentences that preserve the meaning of input graphs. As a crucial defect, the current state of the art models may mess up or even drop the core structural information of input graphs when generating outputs. We propose to tackle this problem by leveraging richer training signals that can guide our model for preserving input information. In particular, we introduce two types of autoencoding losses, each individually focusing on different aspects (a.k.a. views) of input graphs. The losses are then back propagated to better calibrate our model via multi task training. Experiments on two benchmarks for graph to text generation show the effectiveness of our approach over a state of the art baseline. Our code is available at url http github.com Soistesimmer AMR multiview .


Safety Oriented Pedestrian Motion and Scene Occupancy Forecasting. In this paper, we address the important problem in self driving of forecasting multi pedestrian motion and their shared scene occupancy map, critical for safe navigation. Our contributions are two fold. First, we advocate for predicting both the individual motions as well as the scene occupancy map in order to effectively deal with missing detections caused by postprocessing, e.g., confidence thresholding and non maximum suppression. Second, we propose a Scene Actor Graph Neural Network (SA GNN) which preserves the relative spatial information of pedestrians via 2D convolution, and captures the interactions among pedestrians within the same scene, including those that have not been detected, via message passing. On two large scale real world datasets, nuScenes and ATG4D, we showcase that our scene occupancy predictions are more accurate and better calibrated than those from state of the art motion forecasting methods, while also matching their performance in pedestrian motion forecasting metrics.


Graph of uv paths in 2 connected graphs. For a 2 connected graph G and vertices u,v of G we define an abstract graph mathcal P (G uv ) whose vertices are the paths joining u and v in G , where paths S and T are adjacent if T is obtained from S by replacing a subpath S xy of S with an internally disjoint subpath T xy of T . We prove that mathcal P (G uv ) is always connected and give a necessary and a sufficient condition for connectedness in cases where the cycles formed by the replacing subpaths are restricted to a specific family of cycles of G .


Fourier series representation of Ferrers function P. In this note we consider the Fourier expansion of the Ferrers function P of the first kind. We determine its mode of convergence.


Twin width and permutations. Inspired by a width invariant defined on permutations by Guillemot and Marx, the twin width invariant has been recently introduced by Bonnet, Kim, Thomass 'e, and Watrigant. We prove that a class of binary relational structures (that is edge colored partially directed graphs) has bounded twin width if and only if it is a first order transduction of a proper permutation class. As a by product, it shows that every class with bounded twin width contains at most 2 O(n) pairwise non isomorphic n vertex graphs.


Does QA based intermediate training help fine tuning language models for text classification?. Fine tuning pre trained language models for downstream tasks has become a norm for NLP. Recently it is found that intermediate training based on high level inference tasks such as Question Answering (QA) can improve the performance of some language models for target tasks. However it is not clear if intermediate training generally benefits various language models. In this paper, using the SQuAD 2.0 QA task for intermediate training for target text classification tasks, we experimented on eight tasks for single sequence classification and eight tasks for sequence pair classification using two base and two compact language models. Our experiments show that QA based intermediate training generates varying transfer performance across different language models, except for similar QA tasks.


Exact solutions of the problem of dynamics of a fluid with a free surface located between two approaching vertical walls. Exact solutions of a classical problem of a plane unsteady potential flow of an ideal incompressible fluid with a free boundary are presented. The fluid occupies a semi infinite strip bounded by the free surface (from above) and (from the sides) by two solid vertical walls approaching each other with a constant velocity. The solutions are obtained for a situation where the capillary and gravity forces are absent, and the fluid motion is completely determined by the motion of the walls. The solutions contain an arbitrary function, which allows one to describe the nonlinear evolution of perturbations of an arbitrary shape for an initially flat horizontal surface of the fluid. Examples of exact solutions corresponding to the formation of bubbles, cuspidal points, and droplets are considered.


WhiteningBERT An Easy Unsupervised Sentence Embedding Approach. Producing the embedding of a sentence in an unsupervised way is valuable to natural language matching and retrieval problems in practice. In this work, we conduct a thorough examination of pretrained model based unsupervised sentence embeddings. We study on four pretrained models and conduct massive experiments on seven datasets regarding sentence semantics. We have there main findings. First, averaging all tokens is better than only using CLS vector. Second, combining both top andbottom layers is better than only using top layers. Lastly, an easy whitening based vector normalization strategy with less than 10 lines of code consistently boosts the performance.


RadBERT CL Factually Aware Contrastive Learning For Radiology Report Classification. Radiology reports are unstructured and contain the imaging findings and corresponding diagnoses transcribed by radiologists which include clinical facts and negated and or uncertain statements. Extracting pathologic findings and diagnoses from radiology reports is important for quality control, population health, and monitoring of disease progress. Existing works, primarily rely either on rule based systems or transformer based pre trained model fine tuning, but could not take the factual and uncertain information into consideration, and therefore generate false positive outputs. In this work, we introduce three sedulous augmentation techniques which retain factual and critical information while generating augmentations for contrastive learning. We introduce RadBERT CL, which fuses these information into BlueBert via a self supervised contrastive loss. Our experiments on MIMIC CXR show superior performance of RadBERT CL on fine tuning for multi class, multi label report classification. We illustrate that when few labeled data are available, RadBERT CL outperforms conventional SOTA transformers (BERT BlueBert) by significantly larger margins (6 11 ). We also show that the representations learned by RadBERT CL can capture critical medical information in the latent space.


CE Dedup Cost Effective Convolutional Neural Nets Training based on Image Deduplication. Attributed to the ever increasing large image datasets, Convolutional Neural Networks (CNNs) have become popular for vision based tasks. It is generally admirable to have larger sized datasets for higher network training accuracies. However, the impact of dataset quality has not to be involved. It is reasonable to assume the near duplicate images exist in the datasets. For instance, the Street View House Numbers (SVHN) dataset having cropped house plate digits from 0 to 9 are likely to have repetitive digits from the same similar house plates. Redundant images may take up a certain portion of the dataset without consciousness. While contributing little to no accuracy improvement for the CNNs training, these duplicated images unnecessarily pose extra resource and computation consumption. To this end, this paper proposes a framework to assess the impact of the near duplicate images on CNN training performance, called CE Dedup. Specifically, CE Dedup associates a hashing based image deduplication approach with downstream CNNs based image classification tasks. CE Dedup balances the tradeoff between a large deduplication ratio and a stable accuracy by adjusting the deduplication threshold. The effectiveness of CE Dedup is validated through extensive experiments on well known CNN benchmarks. On one hand, while maintaining the same validation accuracy, CE Dedup can reduce the dataset size by 23 . On the other hand, when allowing a small validation accuracy drop (by 5 ), CE Dedup can trim the dataset size by 75 .


On the reciprocal sum of lcm of k tuples. We prove that the reciprocal sum S k(x) of the least common multiple of k geq 3 positive integers in N cap 1,x satisfies S k(x) P 2 k 1 ( log x) O(x theta k epsilon ) where P is a polynomial of degree 2 k 1 and theta k frac 2 k (k 1) frac k 1 2 cdot frac3 2 k 6k 5 . This was conjectured in Hilberdink, Luca, and T ' o th cite Remark 2.4 HLT . We also prove asymptotic formulas for similar sums conjectured there.


Topological models for stable motivic invariants of regular number rings. For an infinity of number rings we express stable motivic invariants in terms of topological data determined by the complex numbers, the real numbers, and finite fields. We use this to extend Morel's identification of the endomorphism ring of the motivic sphere with the Grothendieck Witt ring of quadratic forms to deeper base schemes.


Unsupervised feature selection via self paced learning and low redundant regularization. Much more attention has been paid to unsupervised feature selection nowadays due to the emergence of massive unlabeled data. The distribution of samples and the latent effect of training a learning method using samples in more effective order need to be considered so as to improve the robustness of the method. Self paced learning is an effective method considering the training order of samples. In this study, an unsupervised feature selection is proposed by integrating the framework of self paced learning and subspace learning. Moreover, the local manifold structure is preserved and the redundancy of features is constrained by two regularization terms. L 2,1 2 norm is applied to the projection matrix, which aims to retain discriminative features and further alleviate the effect of noise in the data. Then, an iterative method is presented to solve the optimization problem. The convergence of the method is proved theoretically and experimentally. The proposed method is compared with other state of the art algorithms on nine real world datasets. The experimental results show that the proposed method can improve the performance of clustering methods and outperform other compared algorithms.


Frequent Pattern Mining in Continuous time Temporal Networks. Networks are used as highly expressive tools in different disciplines. In recent years, the analysis and mining of temporal networks have attracted substantial attention. Frequent pattern mining is considered an essential task in the network science literature. In addition to the numerous applications, the investigation of frequent pattern mining in networks directly impacts other analytical approaches, such as clustering, quasi clique and clique mining, and link prediction. In nearly all the algorithms proposed for frequent pattern mining in temporal networks, the networks are represented as sequences of static networks. Then, the inter or intra network patterns are mined. This type of representation imposes a computation expressiveness trade off to the mining problem. In this paper, we propose a novel representation that can preserve the temporal aspects of the network losslessly. Then, we introduce the concept of constrained interval graphs (CIGs). Next, we develop a series of algorithms for mining the complete set of frequent temporal patterns in a temporal network data set. We also consider four different definitions of isomorphism to allow noise tolerance in temporal data collection. Implementing the algorithm for three real world data sets proves the practicality of the proposed algorithm and its capability to discover unknown patterns in various settings.


Cauchy spinors on 3 manifolds. Let mathcal Z be a spin 4 manifold carrying a parallel spinor and M hookrightarrow mathcal Z a hypersurface. The second fundamental form of the embedding induces a flat metric connection on TM . Such flat connections satisfy a non elliptic, non linear equation in terms of a symmetric 2 tensor on M . When M is compact and has positive scalar curvature, the linearized equation has finite dimensional kernel. Four families of solutions are known on the round 3 sphere mathbb S 3 . We study the linearized equation in the vicinity of these solutions and we construct as a byproduct an incomplete hyperk "ahler metric on mathbb S 3 times mathbb R closely related to the Euclidean Taub NUT metric on mathbb R 4 . On mathbb S 3 there do not exist other solutions which either are constant in a left (or right) invariant frame, have three distinct constant eigenvalues, or are invariant in the direction of a left (or right) invariant eigenvector. We deduce from this last result an extension of Liebmann's sphere rigidity theorem.


COMBO State of the Art Morphosyntactic Analysis. We introduce COMBO a fully neural NLP system for accurate part of speech tagging, morphological analysis, lemmatisation, and (enhanced) dependency parsing. It predicts categorical morphosyntactic features whilst also exposes their vector representations, extracted from hidden layers. COMBO is an easy to install Python package with automatically downloadable pre trained models for over 40 languages. It maintains a balance between efficiency and quality. As it is an end to end system and its modules are jointly trained, its training is competitively fast. As its models are optimised for accuracy, they achieve often better prediction quality than SOTA. The COMBO library is available at https gitlab.clarin pl.eu syntactic tools combo.


Learn Quasi stationary Distributions of Finite State Markov Chain. We propose a reinforcement learning (RL) approach to compute the expression of quasi stationary distribution. Based on the fixed point formulation of quasi stationary distribution, we minimize the KL divergence of two Markovian path distributions induced by the candidate distribution and the true target distribution. To solve this challenging minimization problem by gradient descent, we apply the reinforcement learning technique by introducing the reward and value functions. We derive the corresponding policy gradient theorem and design an actor critic algorithm to learn the optimal solution and the value function. The numerical examples of finite state Markov chain are tested to demonstrate the new method.


Improving 360 Monocular Depth Estimation via Non local Dense Prediction Transformer and Joint Supervised and Self supervised Learning. Due to difficulties in acquiring ground truth depth of equirectangular (360) images, the quality and quantity of equirectangular depth data today is insufficient to represent the various scenes in the world. Therefore, 360 depth estimation studies, which relied solely on supervised learning, are destined to produce unsatisfactory results. Although self supervised learning methods focusing on equirectangular images (EIs) are introduced, they often have incorrect or non unique solutions, causing unstable performance. In this paper, we propose 360 monocular depth estimation methods which improve on the areas that limited previous studies. First, we introduce a self supervised 360 depth learning method that only utilizes gravity aligned videos, which has the potential to eliminate the needs for depth data during the training procedure. Second, we propose a joint learning scheme realized by combining supervised and self supervised learning. The weakness of each learning is compensated, thus leading to more accurate depth estimation. Third, we propose a non local fusion block, which can further retain the global information encoded by vision transformer when reconstructing the depths. With the proposed methods, we successfully apply the transformer to 360 depth estimations, to the best of our knowledge, which has not been tried before. On several benchmarks, our approach achieves significant improvements over previous works and establishes a state of the art.


Single query Path Planning Using Sample efficient Probability Informed Trees. In this work, we present a novel sampling based path planning method, called SPRINT. The method finds solutions for high dimensional path planning problems quickly and robustly. Its efficiency comes from minimizing the number of collision check samples. This reduction in sampling relies on heuristics that predict the likelihood that samples will be useful in the search process. Specifically, heuristics (1) prioritize more promising search regions (2) cull samples from local minima regions and (3) steer the search away from previously observed collision states. Empirical evaluations show that our method finds shorter or comparable length solution paths in significantly less time than commonly used methods. We demonstrate that these performance gains can be largely attributed to our approach to achieve sample efficiency.


Variational Weakly Supervised Sentiment Analysis with Posterior Regularization. Sentiment analysis is an important task in natural language processing (NLP). Most of existing state of the art methods are under the supervised learning paradigm. However, human annotations can be scarce. Thus, we should leverage more weak supervision for sentiment analysis. In this paper, we propose a posterior regularization framework for the variational approach to the weakly supervised sentiment analysis to better control the posterior distribution of the label assignment. The intuition behind the posterior regularization is that if extracted opinion words from two documents are semantically similar, the posterior distributions of two documents should be similar. Our experimental results show that the posterior regularization can improve the original variational approach to the weakly supervised sentiment analysis and the performance is more stable with smaller prediction variance.


ContactOpt Optimizing Contact to Improve Grasps. Physical contact between hands and objects plays a critical role in human grasps. We show that optimizing the pose of a hand to achieve expected contact with an object can improve hand poses inferred via image based methods. Given a hand mesh and an object mesh, a deep model trained on ground truth contact data infers desirable contact across the surfaces of the meshes. Then, ContactOpt efficiently optimizes the pose of the hand to achieve desirable contact using a differentiable contact model. Notably, our contact model encourages mesh interpenetration to approximate deformable soft tissue in the hand. In our evaluations, our methods result in grasps that better match ground truth contact, have lower kinematic error, and are significantly preferred by human participants. Code and models are available online.


Convergence rates for gradient descent in the training of overparameterized artificial neural networks with biases. In recent years, artificial neural networks have developed into a powerful tool for dealing with a multitude of problems for which classical solution approaches reach their limits. However, it is still unclear why randomly initialized gradient descent optimization algorithms, such as the well known batch gradient descent, are able to achieve zero training loss in many situations even though the objective function is non convex and non smooth. One of the most promising approaches to solving this problem in the field of supervised learning is the analysis of gradient descent optimization in the so called overparameterized regime. In this article we provide a further contribution to this area of research by considering overparameterized fully connected rectified artificial neural networks with biases. Specifically, we show that for a fixed number of training data the mean squared error using batch gradient descent optimization applied to such a randomly initialized artificial neural network converges to zero at a linear convergence rate as long as the width of the artificial neural network is large enough, the learning rate is small enough, and the training input data are pairwise linearly independent.


Using Motion History Images with 3D Convolutional Networks in Isolated Sign Language Recognition. Sign language recognition using computational models is a challenging problem that requires simultaneous spatio temporal modeling of the multiple sources, i.e. faces, hands, body, etc. In this paper, we propose an isolated sign language recognition model based on a model trained using Motion History Images (MHI) that are generated from RGB video frames. RGB MHI images represent spatio temporal summary of each sign video effectively in a single RGB image. We propose two different approaches using this RGB MHI model. In the first approach, we use the RGB MHI model as a motion based spatial attention module integrated into a 3D CNN architecture. In the second approach, we use RGB MHI model features directly with the features of a 3D CNN model using a late fusion technique. We perform extensive experiments on two recently released large scale isolated sign language datasets, namely AUTSL and BosphorusSign22k. Our experiments show that our models, which use only RGB data, can compete with the state of the art models in the literature that use multi modal data.


Parallel Algorithms for Finding Large Cliques in Sparse Graphs. We present a parallel k clique listing algorithm with improved work bounds (for the same depth) in sparse graphs with low degeneracy or arboricity. We achieve this by introducing and analyzing a new pruning criterion for a backtracking search. Our algorithm has better asymptotic performance, especially for larger cliques (when k is not constant), where we avoid the straightforwardly exponential runtime growth with respect to the clique size. In particular, for cliques that are a constant factor smaller than the graph's degeneracy, the work improvement is an exponential factor in the clique size compared to previous results. Moreover, we present a low depth approximation to the community degeneracy (which can be arbitrarily smaller than the degeneracy). This approximation enables a low depth clique listing algorithm whose runtime is parameterized by the community degeneracy.


Pruning and Slicing Neural Networks using Formal Verification. Deep neural networks (DNNs) play an increasingly important role in various computer systems. In order to create these networks, engineers typically specify a desired topology, and then use an automated training algorithm to select the network's weights. While training algorithms have been studied extensively and are well understood, the selection of topology remains a form of art, and can often result in networks that are unnecessarily large and consequently are incompatible with end devices that have limited memory, battery or computational power. Here, we propose to address this challenge by harnessing recent advances in DNN verification. We present a framework and a methodology for discovering redundancies in DNNs i.e., for finding neurons that are not needed, and can be removed in order to reduce the size of the DNN. By using sound verification techniques, we can formally guarantee that our simplified network is equivalent to the original, either completely, or up to a prescribed tolerance. Further, we show how to combine our technique with slicing, which results in a family of very small DNNs, which are together equivalent to the original. Our approach can produce DNNs that are significantly smaller than the original, rendering them suitable for deployment on additional kinds of systems, and even more amenable to subsequent formal verification. We provide a proof of concept implementation of our approach, and use it to evaluate our techniques on several real world DNNs.


Event Argument Extraction using Causal Knowledge Structures. Event Argument extraction refers to the task of extracting structured information from unstructured text for a particular event of interest. The existing works exhibit poor capabilities to extract causal event arguments like Reason and After Effects. Furthermore, most of the existing works model this task at a sentence level, restricting the context to a local scope. While it may be effective for short spans of text, for longer bodies of text such as news articles, it has often been observed that the arguments for an event do not necessarily occur in the same sentence as that containing an event trigger. To tackle the issue of argument scattering across sentences, the use of global context becomes imperative in this task. In our work, we propose an external knowledge aided approach to infuse document level event information to aid the extraction of complex event arguments. We develop a causal network for our event annotated dataset by extracting relevant event causal structures from ConceptNet and phrases from Wikipedia. We use the extracted event causal features in a bi directional transformer encoder to effectively capture long range inter sentence dependencies. We report the effectiveness of our proposed approach through both qualitative and quantitative analysis. In this task, we establish our findings on an event annotated dataset in 5 Indian languages. This dataset adds further complexity to the task by labelling arguments of entity type (like Time, Place) as well as more complex argument types (like Reason, After Effect). Our approach achieves state of the art performance across all the five languages. Since our work does not rely on any language specific features, it can be easily extended to other languages.


Subconvexity of Shintani's zeta functions. Enumerating integral orbits in prehomogeneous vector spaces plays an important role in arithmetic statistics. We describe a method of proving subconvexity of the zeta function enumerating the integral orbits, illustrated by proving a subconvex estimate for the Shintani zeta function enumerating class numbers of binary cubic forms. The method also obtains a subconvex estimate for the zeta function twisted by a Maass cusp form.


A conservative implicit PIC scheme for the hybrid kinetic ion fluid electron plasma model on curvilinear meshes. The hybrid kinetic ion fluid electron plasma model is widely used to study challenging multi scale problems in space and laboratory plasma physics. Here, a novel conservative scheme for this model employing implicit particle in cell techniques is extended to arbitrary coordinate systems via curvilinear maps from logical to physical space. The scheme features a fully non linear electromagnetic formulation with a multi rate time advance including sub cycling and orbit averaging for the kinetic ions. By careful choice of compatible particle based kinetic ion and mesh based fluid electron discretizations in curvilinear coordinates, as well as particle mesh interpolations and implicit midpoint time advance, the scheme is proven to conserve total energy for arbitrary curvilinear meshes. In the electrostatic limit, the method is also proven to conserve total momentum for arbitrary curvilinear meshes. Although momentum is not conserved for arbitrary curvilinear meshes in the electromagnetic case, it is for an important subset of Cartesian tensor packed meshes. The scheme and its novel conservation properties are demonstrated for several challenging numerical problems using different curvilinear meshes, including a merging flux rope simulation for a space weather application, and a helical m 1 mode simulation for magnetic fusion energy application.


Boundary Distribution Estimation to Precise Object Detection. In principal modern detectors, the task of object localization is implemented by the box subnet which concentrates on bounding box regression. The box subnet customarily predicts the position of the object by regressing box center position and scaling factors. Although this approach is frequently adopted, we observe that the result of localization remains defective, which makes the performance of the detector unsatisfactory. In this paper, we prove the flaws in the previous method through theoretical analysis and experimental verification and propose a novel solution to detect objects precisely. Rather than plainly focusing on center and size, our approach refines the edges of the bounding box on previous localization results by estimating the distribution at the boundary of the object. Experimental results have shown the potentiality and generalization of our proposed method.


OMPQ Orthogonal Mixed Precision Quantization. To bridge the ever increasing gap between deep neural networks' complexity and hardware capability, network quantization has attracted more and more research attention. The latest trend of mixed precision quantization takes advantage of hardware's multiple bit width arithmetic operations to unleash the full potential of network quantization. However, this also results in a difficult integer programming formulation, and forces most existing approaches to use an extremely time consuming search process even with various relaxations. Instead of solving a problem of the original integer programming, we propose to optimize a proxy metric, the concept of network orthogonality, which is highly correlated with the loss of the integer programming but also easy to optimize with linear programming. This approach reduces the search time and required data amount by orders of magnitude, with little compromise on quantization accuracy. Specifically, we achieve 72.08 Top 1 accuracy on ResNet 18 with 6.7Mb, which does not require any searching iterations. Given the high efficiency and low data dependency of our algorithm, we used it for the post training quantization, which achieve 71.27 Top 1 accuracy on MobileNetV2 with only 1.5Mb. Our code is available at https github.com MAC AutoML OMPQ.


BERT based classification system for detecting rumours on Twitter. The role of social media in opinion formation has far reaching implications in all spheres of society. Though social media provide platforms for expressing news and views, it is hard to control the quality of posts due to the sheer volumes of posts on platforms like Twitter and Facebook. Misinformation and rumours have lasting effects on society, as they tend to influence people's opinions and also may motivate people to act irrationally. It is therefore very important to detect and remove rumours from these platforms. The only way to prevent the spread of rumours is through automatic detection and classification of social media posts. Our focus in this paper is the Twitter social medium, as it is relatively easy to collect data from Twitter. The majority of previous studies used supervised learning approaches to classify rumours on Twitter. These approaches rely on feature extraction to obtain both content and context features from the text of tweets to distinguish rumours and non rumours. Manually extracting features however is time consuming considering the volume of tweets. We propose a novel approach to deal with this problem by utilising sentence embedding using BERT to identify rumours on Twitter, rather than the usual feature extraction techniques. We use sentence embedding using BERT to represent each tweet's sentences into a vector according to the contextual meaning of the tweet. We classify those vectors into rumours or non rumours by using various supervised learning techniques. Our BERT based models improved the accuracy by approximately 10 as compared to previous methods.


COSMic A Coherence Aware Generation Metric for Image Descriptions. Developers of text generation models rely on automated evaluation metrics as a stand in for slow and expensive manual evaluations. However, image captioning metrics have struggled to give accurate learned estimates of the semantic and pragmatic success of output text. We address this weakness by introducing the first discourse aware learned generation metric for evaluating image descriptions. Our approach is inspired by computational theories of discourse for capturing information goals using coherence. We present a dataset of image unicode x2013 description pairs annotated with coherence relations. We then train a coherence aware metric on a subset of the Conceptual Captions dataset and measure its effectiveness unicode x2014 its ability to predict human ratings of output captions unicode x2014 on a test set composed of out of domain images. We demonstrate a higher Kendall Correlation Coefficient for our proposed metric with the human judgments for the results of a number of state of the art coherence aware caption generation models when compared to several other metrics including recently proposed learned metrics such as BLEURT and BERTScore.


An Efficient Method for the Classification of Croplands in Scarce Label Regions. Two of the main challenges for cropland classification by satellite time series images are insufficient ground truth data and inaccessibility of high quality hyperspectral images for under developed areas. Unlabeled medium resolution satellite images are abundant, but how to benefit from them is an open question. We will show how to leverage their potential for cropland classification using self supervised tasks. Self supervision is an approach where we provide simple training signals for the samples, which are apparent from the data's structure. Hence, they are cheap to acquire and explain a simple concept about the data. We introduce three self supervised tasks for cropland classification. They reduce epistemic uncertainty, and the resulting model shows superior accuracy in a wide range of settings compared to SVM and Random Forest. Subsequently, we use the self supervised tasks to perform unsupervised domain adaptation and benefit from the labeled samples in other regions. It is crucial to know what information to transfer to avoid degrading the performance. We show how to automate the information selection and transfer process in cropland classification even when the source and target areas have a very different feature distribution. We improved the model by about 24 compared to a baseline architecture without any labeled sample in the target domain. Our method is amenable to gradual improvement, works with medium resolution satellite images, and does not require complicated models. Code and data are available.


Streaming Local Community Detection through Approximate Conductance. Community is a universal structure in various complex networks, and community detection is a fundamental task for network analysis. With the rapid growth of network scale, networks are massive, changing rapidly and could naturally be modeled as graph streams. Due to the limited memory and access constraint in graph streams, existing non streaming community detection methods are no longer applicable. This raises an emerging need for online approaches. In this work, we consider the problem of uncovering the local community containing a few query nodes in graph streams, termed streaming local community detection. This is a new problem raised recently that is more challenging for community detection and only a few works address this online setting. Correspondingly, we design an online single pass streaming local community detection approach. Inspired by the "local" property of communities, our method samples the local structure around the query nodes in graph streams, and extracts the target community on the sampled subgraph using our proposed metric called the approximate conductance. Comprehensive experiments show that our method remarkably outperforms the streaming baseline on both effectiveness and efficiency, and even achieves similar accuracy comparing to the state of the art non streaming local community detection methods that use static and complete graphs.


User Scheduling for Precoded Satellite Systems with Individual Quality of Service Constraints. Multibeam high throughput satellite (MB HTS) systems will play a key role in delivering broadband services to a large number of users with diverse Quality of Service (QoS) requirements. This paper focuses on MB HTS where the same spectrum is re used by all user links and, in particular, we propose a novel user scheduling design capable to provide guarantees in terms of individual QoS requirements while maximizing the system throughput. This is achieved by precoding to mitigate mutual interference. The combinatorial optimization structure requires an extremely high cost to obtain the global optimum even with a reduced number of users. We, therefore, propose a heuristic algorithm yielding a good local solution and tolerable computational complexity, applicable for large scale networks. Numerical results demonstrate the effectiveness of our proposed algorithm on scheduling many users with better sum throughput than the other benchmarks. Besides, the QoS requirements for all scheduled users are guaranteed.


Demystifying speckle field quantitative phase microscopy. Quantitative phase microscopy (QPM) has found significant applications in the field of biomedical imaging which works on the principle of interferometry. The theory behind achieving interference in QPM with conventional light sources such as white light and lasers is very well developed. Recently, the use of dynamic speckle illumination (DSI) in QPM has attracted attention due to its advantages over conventional light sources such as high spatial phase sensitivity, single shot, scalable field of view (FOV) and resolution. However, the understanding behind obtaining interference fringes in QPM with DSI has not been convincingly covered previously. This imposes a constraint on obtaining interference fringes in QPM using DSI and limits its widespread penetration in the field of biomedical imaging. The present article provides the basic understanding of DSI through both simulation and experiments that is essential to build interference optical microscopy systems such as QPM, digital holographic microscopy and optical coherence tomography. Using the developed theory of DSI we demonstrate its capabilities of using non identical objective lenses in both arms of the interference microscopy without degrading the interference fringe contrast and providing the flexibility to use user defined microscope objective lens. It is also demonstrated that the interference fringes are not washed out over a large range of optical path difference (OPD) between the object and the reference arm providing competitive edge over low temporal coherence light sources. The theory and explanation developed here would enable wider penetration of DSI based QPM for applications in biology and material sciences.


Hellinger and total variation distance in approximating L 'e vy driven SDEs. In this paper, we get some convergence rates in total variation distance in approximating discretized paths of L 'e vy driven stochastic differential equations, assuming that the driving process is locally stable. The particular case of the Euler approximation is studied. Our results are based on sharp local estimates in Hellinger distance obtained using Malliavin calculus for jump processes.


PR CIM a Variation Aware Binary Neural Network Framework for Process Resilient Computation in memory. Binary neural networks (BNNs) that use 1 bit weights and activations have garnered interest as extreme quantization provides low power dissipation. By implementing BNNs as computing in memory (CIM), which computes multiplication and accumulations on memory arrays in an analog fashion, namely analog CIM, we can further improve the energy efficiency to process neural networks. However, analog CIMs suffer from the potential problem that process variation degrades the accuracy of BNNs. Our Monte Carlo simulations show that in an SRAM based analog CIM of VGG 9, the classification accuracy of CIFAR 10 is degraded even below 20 under process variations of 65nm CMOS. To overcome this problem, we present a variation aware BNN framework. The proposed framework is developed for SRAM based BNN CIMs since SRAM is most widely used as on chip memory, however easily extensible to BNN CIMs based on other memories. Our extensive experimental results show that under process variation of 65nm CMOS, our framework significantly improves the CIFAR 10 accuracies of SRAM based BNN CIMs, from 10 and 10.1 to 87.76 and 77.74 for VGG 9 and RESNET 18 respectively.


Well posedness for the dispersive Hunter Saxton equation. This article represents a first step towards understanding the well posedness for the dispersive Hunter Saxton equation. This problem arises in the study of nematic liquid crystals, and although the equation has formal similarities with the KdV equation, the lack of L 2 control gives it a quasilinear character, with only continuous dependence on initial data. Here, we prove the local and global well posedness of the Cauchy problem using a normal form approach to construct modified energies, and frequency envelopes in order to prove the continuous dependence with respect to the initial data.


Broad Infinity and Generation Principles. This paper introduces Broad Infinity, a new and arguably intuitive axiom scheme. It states that "broad numbers", which are three dimensional trees whose growth is controlled, form a set. If the Axiom of Choice is assumed, then Broad Infinity is equivalent to the Ord is Mahlo scheme every closed unbounded class of ordinals contains a regular ordinal. Whereas the axiom of Infinity leads to generation principles for sets and families and ordinals, Broad Infinity leads to more advanced versions of these principles. The paper relates these principles under various prior assumptions the Axiom of Choice, the Law of Excluded Middle, and weaker assumptions.


A Joint Reinforcement Learning Enabled Caching and Cross Layer Network Code for Sum Rate Maximization in F RAN with D2D Communications. In this paper, we leverage reinforcement learning (RL) and cross layer network coding (CLNC) for efficiently pre fetching users' contents to the local caches and delivering these contents to users in a downlink fog radio access network (F RAN) with device to device (D2D) communications. In the considered system, fog access points (F APs) and cache enabled D2D (CE D2D) users are equipped with local caches for alleviating traffic burden at the fronthaul, while users' contents can be easily and quickly accommodated. In CLNC, the coding decisions take users' contents, their rates, and power levels of F APs and CE D2D users into account, and RL optimizes caching strategy. Towards this goal, a joint content placement and delivery problem is formulated as an optimization problem with a goal to maximize system sum rate. For this NP hard problem, we first develop an innovative decentralized CLNC coalition formation (CLNC CF) algorithm to obtain a stable solution for the content delivery problem, where F APs and CE D2D users utilize CLNC resource allocation. By taking the behavior of F APs and CE D2D users into account, we then develop a multi agent RL (MARL) algorithm for optimizing the content placements at both F APs and CE D2D users. Simulation results show that the proposed joint CLNC CF and RL framework can effectively improve the sum rate by up to 30 , 60 , and 150 , respectively, compared to 1) an optimal uncoded algorithm, 2) a standard rate aware NC algorithm, and 3) a benchmark classical NC with network layer optimization.


On the Exploitation of Neuroevolutionary Information Analyzing the Past for a More Efficient Future. Neuroevolutionary algorithms, automatic searches of neural network structures by means of evolutionary techniques, are computationally costly procedures. In spite of this, due to the great performance provided by the architectures which are found, these methods are widely applied. The final outcome of neuroevolutionary processes is the best structure found during the search, and the rest of the procedure is commonly omitted in the literature. However, a good amount of residual information consisting of valuable knowledge that can be extracted is also produced during these searches. In this paper, we propose an approach that extracts this information from neuroevolutionary runs, and use it to build a metamodel that could positively impact future neural architecture searches. More specifically, by inspecting the best structures found during neuroevolutionary searches of generative adversarial networks with varying characteristics (e.g., based on dense or convolutional layers), we propose a Bayesian network based model which can be used to either find strong neural structures right away, conveniently initialize different structural searches for different problems, or help future optimization of structures of any type to keep finding increasingly better structures where uninformed methods get stuck into local optima.


Partial hyperbolicity and pseudo Anosov dynamics. We show that if a hyperbolic 3 manifold admits a partially hyperbolic diffeomorphism then it also admits an Anosov flow. Moreover, we give a complete classification of partially hyperbolic diffeomorphism in hyperbolic 3 manifolds as well as partially hyperbolic diffeomorphisms in Seifert manifolds inducing pseudo Anosov dynamics in the base. This classification is given in terms of the structure of their center (branching) foliations and the notion of collapsed Anosov flows.


Liquidity Stress Testing using Optimal Portfolio Liquidation. We build an optimal portfolio liquidation model for OTC markets, aiming at minimizing the trading costs via the choice of the liquidation time. We work in the Locally Linear Order Book framework of cite toth2011anomalous to obtain the market impact as a function of the traded volume. We find that the optimal terminal time for a linear execution of a small order is proportional to the square root of the ratio between the amount being bought or sold and the average daily volume. Numerical experiments on real market data illustrate the method on a portfolio of corporate bonds.


IDOL Inertial Deep Orientation Estimation and Localization. Many smartphone applications use inertial measurement units (IMUs) to sense movement, but the use of these sensors for pedestrian localization can be challenging due to their noise characteristics. Recent data driven inertial odometry approaches have demonstrated the increasing feasibility of inertial navigation. However, they still rely upon conventional smartphone orientation estimates that they assume to be accurate, while in fact these orientation estimates can be a significant source of error. To address the problem of inaccurate orientation estimates, we present a two stage, data driven pipeline using a commodity smartphone that first estimates device orientations and then estimates device position. The orientation module relies on a recurrent neural network and Extended Kalman Filter to obtain orientation estimates that are used to then rotate raw IMU measurements into the appropriate reference frame. The position module then passes those measurements through another recurrent network architecture to perform localization. Our proposed method outperforms state of the art methods in both orientation and position error on a large dataset we constructed that contains 20 hours of pedestrian motion across 3 buildings and 15 subjects.


Conjugator length in Thompson's groups. We prove Thompson's group F has quadratic conjugator length function. That is, for any two conjugate elements of F of length n or less, there exists an element of F of length O(n 2) that conjugates one to the other. Moreover, there exist conjugate pairs of elements of F of length at most n such that the shortest conjugator between them has length Omega(n 2) . This latter statement holds for T and V as well.


ABD Net Attention Based Decomposition Network for 3D Point Cloud Decomposition. In this paper, we propose Attention Based Decomposition Network (ABD Net), for point cloud decomposition into basic geometric shapes namely, plane, sphere, cone and cylinder. We show improved performance of 3D object classification using attention features based on primitive shapes in point clouds. Point clouds, being the simple and compact representation of 3D objects have gained increasing popularity. They demand robust methods for feature extraction due to unorderness in point sets. In ABD Net the proposed Local Proximity Encapsulator captures the local geometric variations along with spatial encoding around each point from the input point sets. The encapsulated local features are further passed to proposed Attention Feature Encoder to learn basic shapes in point cloud. Attention Feature Encoder models geometric relationship between the neighborhoods of all the points resulting in capturing global point cloud information. We demonstrate the results of our proposed ABD Net on ANSI mechanical component and ModelNet40 datasets. We also demonstrate the effectiveness of ABD Net over the acquired attention features by improving the performance of 3D object classification on ModelNet40 benchmark dataset and compare them with state of the art techniques.


iLQR for Piecewise Smooth Hybrid Dynamical Systems. Trajectory optimization is a popular strategy for planning trajectories for robotic systems. However, many robotic tasks require changing contact conditions, which is difficult due to the hybrid nature of the dynamics. The optimal sequence and timing of these modes are typically not known ahead of time. In this work, we extend the Iterative Linear Quadratic Regulator (iLQR) method to a class of piecewise smooth hybrid dynamical systems by allowing for changing hybrid modes in the forward pass, using the saltation matrix to update the gradient information in the backwards pass, and using a reference extension to account for mode mismatch. We demonstrate these changes on a variety of hybrid systems and compare the different strategies for computing the gradients.


Training Neural Networks is exists mathbb R complete. Given a neural network, training data, and a threshold, it was known that it is NP hard to find weights for the neural network such that the total error is below the threshold. We determine the algorithmic complexity of this fundamental problem precisely, by showing that it is exists mathbb R complete. This means that the problem is equivalent, up to polynomial time reductions, to deciding whether a system of polynomial equations and inequalities with integer coefficients and real unknowns has a solution. If, as widely expected, exists mathbb R is strictly larger than NP, our work implies that the problem of training neural networks is not even in NP. Neural networks are usually trained using some variation of backpropagation. The result of this paper offers an explanation why techniques commonly used to solve big instances of NP complete problems seem not to be of use for this task. Examples of such techniques are SAT solvers, IP solvers, local search, dynamic programming, to name a few general ones.


Unsupervised Multi source Domain Adaptation Without Access to Source Data. Unsupervised Domain Adaptation (UDA) aims to learn a predictor model for an unlabeled domain by transferring knowledge from a separate labeled source domain. However, most of these conventional UDA approaches make the strong assumption of having access to the source data during training, which may not be very practical due to privacy, security and storage concerns. A recent line of work addressed this problem and proposed an algorithm that transfers knowledge to the unlabeled target domain from a single source model without requiring access to the source data. However, for adaptation purposes, if there are multiple trained source models available to choose from, this method has to go through adapting each and every model individually, to check for the best source. Thus, we ask the question can we find the optimal combination of source models, with no source data and without target labels, whose performance is no worse than the single best source? To answer this, we propose a novel and efficient algorithm which automatically combines the source models with suitable weights in such a way that it performs at least as good as the best source model. We provide intuitive theoretical insights to justify our claim. Furthermore, extensive experiments are conducted on several benchmark datasets to show the effectiveness of our algorithm, where in most cases, our method not only reaches best source accuracy but also outperforms it.


"Short is the Road that Leads from Fear to Hate" Fear Speech in Indian WhatsApp Groups. WhatsApp is the most popular messaging app in the world. Due to its popularity, WhatsApp has become a powerful and cheap tool for political campaigning being widely used during the 2019 Indian general election, where it was used to connect to the voters on a large scale. Along with the campaigning, there have been reports that WhatsApp has also become a breeding ground for harmful speech against various protected groups and religious minorities. Many such messages attempt to instil fear among the population about a specific (minority) community. According to research on inter group conflict, such fear speech' messages could have a lasting impact and might lead to real offline violence. In this paper, we perform the first large scale study on fear speech across thousands of public WhatsApp groups discussing politics in India. We curate a new dataset and try to characterize fear speech from this dataset. We observe that users writing fear speech messages use various events and symbols to create the illusion of fear among the reader about a target community. We build models to classify fear speech and observe that current state of the art NLP models do not perform well at this task. Fear speech messages tend to spread faster and could potentially go undetected by classifiers built to detect traditional toxic speech due to their low toxic nature. Finally, using a novel methodology to target users with Facebook ads, we conduct a survey among the users of these WhatsApp groups to understand the types of users who consume and share fear speech. We believe that this work opens up new research questions that are very different from tackling hate speech which the research community has been traditionally involved in.


Applications of Artificial Neural Networks in Microorganism Image Analysis A Comprehensive Review from Conventional Multilayer Perceptron to Popular Convolutional Neural Network and Potential Visual Transformer. Microorganisms are widely distributed in the human daily living environment. They play an essential role in environmental pollution control, disease prevention and treatment, and food and drug production. The analysis of microorganisms is essential for making full use of different microorganisms. The conventional analysis methods are laborious and time consuming. Therefore, the automatic image analysis based on artificial neural networks is introduced to optimize it. However, the automatic microorganism image analysis faces many challenges, such as the requirement of a robust algorithm caused by various application occasions, insignificant features and easy under segmentation caused by the image characteristic, and various analysis tasks. Therefore, we conduct this review to comprehensively discuss the characteristics of microorganism image analysis based on artificial neural networks. In this review, the background and motivation are introduced first. Then, the development of artificial neural networks and representative networks are presented. After that, the papers related to microorganism image analysis based on classical and deep neural networks are reviewed from the perspectives of different tasks. In the end, the methodology analysis and potential direction are discussed.


The obstacle problem for stochastic porous media equations. We prove the existence and uniqueness of non negative entropy solutions of the obstacle problem for stochastic porous media equations. The core of the method is to combine the entropy formulation with the penalization method.


Seismic Inversion by Multi dimensional Newtonian Machine. Newtonian machine learning (NML) is a wave equation inversion method that inverts single dimensional latent space (LS) features of the seismic data for retrieving the subsurface background velocity model. The single dimensional LS features mainly contain the kinematic information of the seismic data, which are automatically extracted from the seismic signal by using an autoencoder network. Because its LS feature dimension is too small to preserve the dynamic information, such as the waveform variations, of the seismic data. Therefore the NML inversion is not able to recover the high wavenumber velocity details. To mitigate this problem, we propose to invert multi dimensional LS features, which can fully represent the entire characters of the seismic data. We denote this method as multi dimensional Newtonian machine learning (MNML). In MNML, we define a new multi variable connective function that works together with the multi variable implicit function theorem to connect the velocity perturbations to the multi dimensional LS feature perturbations. Numerical tests show that (1) the multi dimensional LS features can preserve more data information than the single dimensional LS features (2) a higher resolution velocity model can be recovered by inverting the multi dimensional LS features, and the inversion quality is comparable to that of FWI (3) the MNML method requires a much smaller storage space than conventional FWI because only the low dimensional representations of the high dimensional seismic data are needed to be stored. The disadvantage of MNML is that it can more easily get stuck in local minima compared to the NML method. So we suggest a multiscale inversion approach that inverts for higher dimensional LS features as the iteration count increase.


Analysis of Busy Time Scheduling on Heterogeneous Machines. This paper studies a generalized busy time scheduling model on heterogeneous machines. The input to the model includes a set of jobs and a set of machine types. Each job has a size and a time interval during which it should be processed. Each job is to be placed on a machine for execution. Different types of machines have distinct capacities and cost rates. The total size of the jobs running on a machine must always be kept within the machine's capacity, giving rise to placement restrictions for jobs of various sizes among the machine types. Each machine used is charged according to the time duration in which it is busy, i.e., it is processing jobs. The objective is to schedule the jobs onto machines to minimize the total cost of all the machines used. We develop an O(1) approximation algorithm in the offline setting and an O( mu) competitive algorithm in the online setting (where mu is the max min job length ratio), both of which are asymptotically optimal.


Civil Rephrases Of Toxic Texts With Self Supervised Transformers. Platforms that support online commentary, from social networks to news sites, are increasingly leveraging machine learning to assist their moderation efforts. But this process does not typically provide feedback to the author that would help them contribute according to the community guidelines. This is prohibitively time consuming for human moderators to do, and computational approaches are still nascent. This work focuses on models that can help suggest rephrasings of toxic comments in a more civil manner. Inspired by recent progress in unpaired sequence to sequence tasks, a self supervised learning model is introduced, called CAE T5. CAE T5 employs a pre trained text to text transformer, which is fine tuned with a denoising and cyclic auto encoder loss. Experimenting with the largest toxicity detection dataset to date (Civil Comments) our model generates sentences that are more fluent and better at preserving the initial content compared to earlier text style transfer systems which we compare with using several scoring systems and human evaluation.


ECACL A Holistic Framework for Semi Supervised Domain Adaptation. This paper studies Semi Supervised Domain Adaptation (SSDA), a practical yet under investigated research topic that aims to learn a model of good performance using unlabeled samples and a few labeled samples in the target domain, with the help of labeled samples from a source domain. Several SSDA methods have been proposed recently, which however fail to fully exploit the value of the few labeled target samples. In this paper, we propose Enhanced Categorical Alignment and Consistency Learning (ECACL), a holistic SSDA framework that incorporates multiple mutually complementary domain alignment techniques. ECACL includes two categorical domain alignment techniques that achieve class level alignment, a strong data augmentation based technique that enhances the model's generalizability and a consistency learning based technique that forces the model to be robust with image perturbations. These techniques are applied on one or multiple of the three inputs (labeled source, unlabeled target, and labeled target) and align the domains from different perspectives. ECACL unifies them together and achieves fairly comprehensive domain alignments that are much better than the existing methods For example, ECACL raises the state of the art accuracy from 68.4 to 81.1 on VisDA2017 and from 45.5 to 53.4 on DomainNet for the 1 shot setting. Our code is available at url https github.com kailigo pacl .


XCloud VIP Virtual Peak Enables Highly Accelerated NMR Spectroscopy and Faithful Quantitative Measures. Background Nuclear Magnetic Resonance (NMR) spectroscopy is an important bio engineering tool to determine the metabolic concentrations, molecule structures and so on. The data acquisition time, however, is very long in multi dimensional NMR. To accelerate data acquisition, non uniformly sampling is an effective way but may encounter severe spectral distortions and unfaithful quantitative measures when the acceleration factor is high. Objective To reconstruct high fidelity spectra from highly accelerated NMR and achieve much better quantitative measures. Methods A virtual peak (VIP) approach is proposed to self learn the prior spectral information, such as the central frequency and peak lineshape, and then feed these information into the reconstruction. The proposed method is further implemented with cloud computing to facilitate online, open, and easy access. Results Results on synthetic and experimental data demonstrate that, compared with the state of the art method, the new approach provides much better reconstruction of low intensity peaks and significantly improves the quantitative measures, including the regression of peak intensity, the distances between nuclear pairs, and concentrations of metabolics in mixtures. Conclusion Self learning prior peak information can improve the reconstruction and quantitative measures of spectra. Significance This approach enables highly accelerated NMR and may promote time consuming applications such as quantitative and time resolved NMR experiments.


Acoustic Sensing based Hand Gesture Detection for Wearable Device Interaction. Hand gesture recognition attracts great attention for interaction since it is intuitive and natural to perform. In this paper, we explore a novel method for interaction by using bone conducted sound generated by finger movements while performing gestures. We design a set of gestures that generate unique sound features, and capture the resulting sound from the wrist using a commodity microphone. Next, we design a sound event detector and a recognition model to classify the gestures. Our system achieves an overall accuracy of 90.13 in quiet environments and 85.79 under noisy conditions. This promising technology can be deployed on existing smartwatches as a low power service at no additional cost, and can be used for interaction in augmented and virtual reality applications.


Teaching Continuity in Robotics Labs in the Age of Covid and Beyond. This paper argues that training of future Roboticists and Robotics Engineers in Computer Science departments, requires the extensive direct work with real robots, and that this educational mission will be negatively impacted when access to robotics learning laboratories is curtailed. This is exactly the problem that Robotics Labs encountered in early 2020, at the start of the Covid pandemic. The paper then turns to the description of a remote virtual robotics teaching laboratory and examines in detail what that would mean, what the benefits would be, and how it may be used. Part of this vision was implemented at our institution during 2020 and has been in constant use since then. The specific architecture and implementation, as far as it has been built, is described. The exciting insight in the conclusion is that the work that was encouraged and triggered by a pandemic seems to have very positive longer term benefits of increasing access to robotics education, increasing the ability of any one institution to scale their robotics education greatly, and potentially do this while reducing costs.


Subdivergence free gluings of trees. A gluing of two rooted trees is an identification of their leaves and un subdivision of the resulting 2 valent vertices. A gluing of two rooted trees is subdivergence free if it has no 2 edge cuts with both roots on the same side of the cut. The problem and language is motivated by quantum field theory. We enumerate subdivergence free gluings for certain families of trees, showing a connection with connected permutations, and we give algorithms to compute subdivergence free gluings.


Designing a Real Time IoT Data Streaming Testbed for Horizontally Scalable Analytical Platforms Czech Post Case Study. There is a growing trend for enterprise level Internet of Things (IoT) applications requiring real time horizontally scalable data processing platforms. Real time processing platforms receiving data streams from sensor networks (e.g., autonomous and connected vehicles, smart security for businesses and homes, smartwatches, fitness trackers, and other wearables) require distributed MQTT brokers. This case study presents an IoT data streaming testbed platform prepared for the Czech Post. The presented platform has met the throughput requirement of 2 million messages per 24 hours (comprising SMS and emails). The tested MQTT broker runs on a single virtual node of a horizontally scalable testbed platform. Soon the Czech Post will modernise its eServices to increase package deliveries aligned with eCommerce and eGovernment demands. The presented testbed platform fulfils all requirements, and it is also capable of processing thousands of messages per second. The presented platform and concepts are transferable to healthcare systems, transport operations, the automotive industry, and other domains such as smart cities.


FedRAD Federated Robust Adaptive Distillation. The robustness of federated learning (FL) is vital for the distributed training of an accurate global model that is shared among large number of clients. The collaborative learning framework by typically aggregating model updates is vulnerable to model poisoning attacks from adversarial clients. Since the shared information between the global server and participants are only limited to model parameters, it is challenging to detect bad model updates. Moreover, real world datasets are usually heterogeneous and not independent and identically distributed (Non IID) among participants, which makes the design of such robust FL pipeline more difficult. In this work, we propose a novel robust aggregation method, Federated Robust Adaptive Distillation (FedRAD), to detect adversaries and robustly aggregate local models based on properties of the median statistic, and then performing an adapted version of ensemble Knowledge Distillation. We run extensive experiments to evaluate the proposed method against recently published works. The results show that FedRAD outperforms all other aggregators in the presence of adversaries, as well as in heterogeneous data distributions.


A Neural Approach for Detecting Morphological Analogies. Analogical proportions are statements of the form "A is to B as C is to D" that are used for several reasoning and classification tasks in artificial intelligence and natural language processing (NLP). For instance, there are analogy based approaches to semantics as well as to morphology. In fact, symbolic approaches were developed to solve or to detect analogies between character strings, e.g., the axiomatic approach as well as that based on Kolmogorov complexity. In this paper, we propose a deep learning approach to detect morphological analogies, for instance, with reinflexion or conjugation. We present empirical results that show that our framework is competitive with the above mentioned state of the art symbolic approaches. We also explore empirically its transferability capacity across languages, which highlights interesting similarities between them.


Automated Urban Planning for Reimagining City Configuration via Adversarial Learning Quantification, Generation, and Evaluation. Urban planning refers to the efforts of designing land use configurations given a region. However, to obtain effective urban plans, urban experts have to spend much time and effort analyzing sophisticated planning constraints based on domain knowledge and personal experiences. To alleviate the heavy burden of them and produce consistent urban plans, we want to ask that can AI accelerate the urban planning process, so that human planners only adjust generated configurations for specific needs? The recent advance of deep generative models provides a possible answer, which inspires us to automate urban planning from an adversarial learning perspective. However, three major challenges arise 1) how to define a quantitative land use configuration? 2) how to automate configuration planning? 3) how to evaluate the quality of a generated configuration? In this paper, we systematically address the three challenges. Specifically, 1) We define a land use configuration as a longitude latitude channel tensor. 2) We formulate the automated urban planning problem into a task of deep generative learning. The objective is to generate a configuration tensor given the surrounding contexts of a target region. 3) We provide quantitative evaluation metrics and conduct extensive experiments to demonstrate the effectiveness of our framework.