Poisoning and Backdooring Contrastive Learning. Multimodal contrastive learning methods like CLIP train on noisy and uncurated training datasets. This is cheaper than labeling datasets manually, and even improves out of distribution robustness. We show that this practice makes backdoor and poisoning attacks a significant threat. By poisoning just 0.01 of a dataset (e.g., just 300 images of the 3 million example Conceptual Captions dataset), we can cause the model to misclassify test images by overlaying a small patch. Targeted poisoning attacks, whereby the model misclassifies a particular test input with an adversarially desired label, are even easier requiring control of 0.0001 of the dataset (e.g., just three out of the 3 million images). Our attacks call into question whether training on noisy and uncurated Internet scrapes is desirable.


Efficient Representations of Signals in Nonlinear Signal Processing with Applications to Inverse Problems. The focus of this thesis is the construction and analysis of efficient representations in nonlinear signal processing, and the applications of these structures to inverse problems in a variety of fields. The work is composed of three major sections, each associated with a different form of data Regression and Distance Estimation on Graphs and Riemannian Manifolds. Instantaneous Time Frequency Analysis via Synchrosqueezing. Multiscale Dictionaries of Slepian Functions on the Sphere.


Rigidity and almost rigidity of Sobolev inequalities on compact spaces with lower Ricci curvature bounds. We prove that if M is a closed n dimensional Riemannian manifold, n ge 3 , with rm Ric ge n 1 and for which the optimal constant in the critical Sobolev inequality equals the one of the n dimensional sphere mathbb S n , then M is isometric to mathbb S n . An almost rigidity result is also established, saying that if equality is almost achieved, then M is close in the measure Gromov Hausdorff sense to a spherical suspension. These statements are obtained in the rm RCD setting of (possibly non smooth) metric measure spaces satisfying synthetic lower Ricci curvature bounds. An independent result of our analysis is the characterization of the best constant in the Sobolev inequality on any compact rm CD space, extending to the non smooth setting a classical result by Aubin. Our arguments are based on a new concentration compactness result for mGH converging sequences of rm RCD spaces and on a Polya Szego inequality of Euclidean type in rm CD spaces. As an application of the technical tools developed we prove both an existence result for the Yamabe equation and the continuity of the generalized Yamabe constant under measure Gromov Hausdorff convergence, in the rm RCD setting.


An organized review of key factors for fake news detection. Fake news in social media has quickly become one of the most discussed topics in today's society. With false information proliferating and causing a significant impact in the political, economical, and social domains, research efforts to analyze and automatically identify this type of content have being conducted in the past few years. In this paper, we attempt to summarize the principal findings on the topic of fake news in social media, highlighting the main research path taken and giving a particular focus on the detection of fake news and bot accounts.


OSCAR Net Object centric Scene Graph Attention for Image Attribution. Images tell powerful stories but cannot always be trusted. Matching images back to trusted sources (attribution) enables users to make a more informed judgment of the images they encounter online. We propose a robust image hashing algorithm to perform such matching. Our hash is sensitive to manipulation of subtle, salient visual details that can substantially change the story told by an image. Yet the hash is invariant to benign transformations (changes in quality, codecs, sizes, shapes, etc.) experienced by images during online redistribution. Our key contribution is OSCAR Net (Object centric Scene Graph Attention for Image Attribution Network) a robust image hashing model inspired by recent successes of Transformers in the visual domain. OSCAR Net constructs a scene graph representation that attends to fine grained changes of every object's visual appearance and their spatial relationships. The network is trained via contrastive learning on a dataset of original and manipulated images yielding a state of the art image hash for content fingerprinting that scales to millions of images.


Go Wider Instead of Deeper. More transformer blocks with residual connections have recently achieved impressive results on various tasks. To achieve better performance with fewer trainable parameters, recent methods are proposed to go shallower by parameter sharing or model compressing along with the depth. However, weak modeling capacity limits their performance. Contrastively, going wider by inducing more trainable matrixes and parameters would produce a huge model requiring advanced parallelism to train and inference. In this paper, we propose a parameter efficient framework, going wider instead of deeper. Specially, following existing works, we adapt parameter sharing to compress along depth. But, such deployment would limit the performance. To maximize modeling capacity, we scale along model width by replacing feed forward network (FFN) with mixture of experts (MoE). Across transformer blocks, instead of sharing normalization layers, we propose to use individual layernorms to transform various semantic representations in a more parameter efficient way. To evaluate our plug and run framework, we design WideNet and conduct comprehensive experiments on popular computer vision and natural language processing benchmarks. On ImageNet 1K, our best model outperforms Vision Transformer (ViT) by 1.5 with 0.72 times trainable parameters. Using 0.46 times and 0.13 times parameters, our WideNet can still surpass ViT and ViT MoE by 0.8 and 2.1 , respectively. On four natural language processing datasets, WideNet outperforms ALBERT by 1.8 on average and surpass BERT using factorized embedding parameterization by 0.8 with fewer parameters.


Generating Synthetic Text Data to Evaluate Causal Inference Methods. Drawing causal conclusions from observational data requires making assumptions about the true data generating process. Causal inference research typically considers low dimensional data, such as categorical or numerical fields in structured medical records. High dimensional and unstructured data such as natural language complicates the evaluation of causal inference methods such evaluations rely on synthetic datasets with known causal effects. Models for natural language generation have been widely studied and perform well empirically. However, existing methods not immediately applicable to producing synthetic datasets for causal evaluations, as they do not allow for quantifying a causal effect on the text itself. In this work, we develop a framework for adapting existing generation models to produce synthetic text datasets with known causal effects. We use this framework to perform an empirical comparison of four recently proposed methods for estimating causal effects from text data. We release our code and synthetic datasets.


Neural Descriptor Fields SE(3) Equivariant Object Representations for Manipulation. We present Neural Descriptor Fields (NDFs), an object representation that encodes both points and relative poses between an object and a target (such as a robot gripper or a rack used for hanging) via category level descriptors. We employ this representation for object manipulation, where given a task demonstration, we want to repeat the same task on a new object instance from the same category. We propose to achieve this objective by searching (via optimization) for the pose whose descriptor matches that observed in the demonstration. NDFs are conveniently trained in a self supervised fashion via a 3D auto encoding task that does not rely on expert labeled keypoints. Further, NDFs are SE(3) equivariant, guaranteeing performance that generalizes across all possible 3D object translations and rotations. We demonstrate learning of manipulation tasks from few (5 10) demonstrations both in simulation and on a real robot. Our performance generalizes across both object instances and 6 DoF object poses, and significantly outperforms a recent baseline that relies on 2D descriptors. Project website https yilundu.github.io ndf .


An analysis of gamma ray data collected at traffic intersections in Northern Virginia. Gamma ray spectral data were collected from sensors mounted to traffic signals around Northern Virginia. The data were collected over a span of approximately fifteen months. A subset of the data were analyzed manually and subsequently used to train machine learning models to facilitate the evaluation of the remaining 50k anomalous events identified in the dataset. We describe the analysis approach used here and discuss the results in terms of radioisotope classes and frequency patterns over day of week and time of day spans. Data from this work has been archived and is available for future and ongoing research applications.


Software Engineering for Responsible AI An Empirical Study and Operationalised Patterns. Although artificial intelligence (AI) is solving real world challenges and transforming industries, there are serious concerns about its ability to behave and make decisions in a responsible way. Many AI ethics principles and guidelines for responsible AI have been recently issued by governments, organisations, and enterprises. However, these AI ethics principles and guidelines are typically high level and do not provide concrete guidance on how to design and develop responsible AI systems. To address this shortcoming, we first present an empirical study where we interviewed 21 scientists and engineers to understand the practitioners' perceptions on AI ethics principles and their implementation. We then propose a template that enables AI ethics principles to be operationalised in the form of concrete patterns and suggest a list of patterns using the newly created template. These patterns provide concrete, operationalised guidance that facilitate the development of responsible AI systems.


PFH spectral invariants and C infty closing lemmas. We develop the theory of spectral invariants in periodic Floer homology (PFH) of area preserving surface diffeomorphisms. We use this theory to prove C infty closing lemmas for certain Hamiltonian isotopy classes of area preserving surface diffeomorphisms. In particular, we show that for a C infty generic area preserving diffeomorphism of the torus, the set of periodic points is dense. Our closing lemmas are quantitative, asserting roughly speaking that for a given Hamiltonian isotopy, within time delta a periodic orbit must appear of period O( delta 1 ) . We also prove a "Weyl law" describing the asymptotic behavior of PFH spectral invariants.


Sampling discretization and related problems. This survey addresses sampling discretization and its connections with other areas of mathematics. The survey concentrates on sampling discretization of norms of elements of finite dimensional subspaces. We present here known results on sampling discretization of both integral norms and the uniform norm beginning with classical results and ending with very recent achievements. We also show how sampling discretization connects to spectral properties and operator norms of submatrices, embedding of finite dimensional subspaces, moments of marginals of high dimensional distributions, and learning theory. Along with the corresponding results, important techniques for proving those results are discussed as well.


Doubly Trained Adversarial Data Augmentation for Neural Machine Translation. Neural Machine Translation (NMT) models are known to suffer from noisy inputs. To make models robust, we generate adversarial augmentation samples that attack the model and preserve the source side semantic meaning at the same time. To generate such samples, we propose a doubly trained architecture that pairs two NMT models of opposite translation directions with a joint loss function, which combines the target side attack and the source side semantic similarity constraint. The results from our experiments across three different language pairs and two evaluation metrics show that these adversarial samples improve the model robustness.


A Constraint Programming Approach to Weighted Isomorphic Mapping of Fragment based Shape Signatures. Fragment based shape signature techniques have proven to be powerful tools for computer aided drug design. They allow scientists to search for target molecules with some similarity to a known active compound. They do not require reference to the full underlying chemical structure, which is essential to deal with chemical databases containing millions of compounds. However, finding the optimal match of a part of the fragmented compound can be time consuming. In this paper, we use constraint programming to solve this specific problem. It involves finding a weighted assignment of fragments subject to connectivity constraints. Our experiments demonstrate the practical relevance of our approach and open new perspectives, including generating multiple, diverse solutions. Our approach constitutes an original use of a constraint solver in a real time setting, where propagation allows to avoid an enumeration of weighted paths. The model must remain robust to the addition of constraints making some instances not tractable. This particular context requires the use of unusual criteria for the choice of the model lightweight, standard propagation algorithms, data structures without prohibitive constant cost. The objective is not to design new, complex algorithms to solve difficult instances.


Co advise Cross Inductive Bias Distillation. Transformers recently are adapted from the community of natural language processing as a promising substitute of convolution based neural networks for visual learning tasks. However, its supremacy degenerates given an insufficient amount of training data (e.g., ImageNet). To make it into practical utility, we propose a novel distillation based method to train vision transformers. Unlike previous works, where merely heavy convolution based teachers are provided, we introduce lightweight teachers with different architectural inductive biases (e.g., convolution and involution) to co advise the student transformer. The key is that teachers with different inductive biases attain different knowledge despite that they are trained on the same dataset, and such different knowledge compounds and boosts the student's performance during distillation. Equipped with this cross inductive bias distillation method, our vision transformers (termed as CivT) outperform all previous transformers of the same architecture on ImageNet.


Sexism detection The first corpus in Algerian dialect with a code switching in Arabic French and English. In this paper, an approach for hate speech detection against women in Arabic community on social media (e.g. Youtube) is proposed. In the literature, similar works have been presented for other languages such as English. However, to the best of our knowledge, not much work has been conducted in the Arabic language. A new hate speech corpus (Arabic fr en) is developed using three different annotators. For corpus validation, three different machine learning algorithms are used, including deep Convolutional Neural Network (CNN), long short term memory (LSTM) network and Bi directional LSTM (Bi LSTM) network. Simulation results demonstrate the best performance of the CNN model, which achieved F1 score up to 86 for the unbalanced corpus as compared to LSTM and Bi LSTM.


HNN extension of involutive multiplicative Hom Lie algebras. The construction of HNN extensions of involutive Hom associative algebras and involutive Hom Lie algebras is described. Then, as an application of HNN extension, by using the validity of Poincar 'e Birkhoff Witt theorem for involutive Hom Lie algebras, we provide an embedding theorem.


Squash root microbiome transplants and metagenomic inspection for in situ arid adaptations. Arid zones contain a diverse set of microbes capable of survival under dry conditions, some of which can form relationships with plants under drought stress conditions to improve plant health. We studied squash (Cucurbita pepo L.) root microbiome under historically arid and humid sites, both in situ and performing a common garden experiment. Plants were grown in soils from sites with different drought levels, using in situ collected soils as the microbial source. We described and analyzed bacterial diversity by 16S rRNA gene sequencing (N 48) from the soil, rhizosphere, and endosphere. Proteobacteria were the most abundant phylum present in humid and arid samples, while Actinobacteriota abundance was higher in arid ones. The Beta diversity analyses showed split microbiomes between arid and humid microbiomes, and aridity and soil pH levels could explain it. These differences between humid and arid microbiomes were maintained in the common garden experiment, showing that it is possible to transplant in situ diversity to the greenhouse. We detected a total of 1009 bacterial genera 199 exclusively associated with roots under arid conditions. With shotgun metagenomic sequencing of rhizospheres (N 6), we identified 2969 protein families in the squash core metagenome and found an increased number of exclusively protein families from arid (924) than humid samples (158). We found arid conditions enriched genes involved in protein degradation and folding, oxidative stress, compatible solute synthesis, and ion pumps associated with osmotic regulation. Plant phenotyping allowed us to correlate bacterial communities with plant growth. Our study revealed that it is possible to evaluate microbiome diversity ex situ and identify critical species and genes involved in plant microbe interactions in historically arid locations.


Lipschitz Bandits with Batched Feedback. In this paper, we study Lipschitz bandit problems with batched feedback, where the expected reward is Lipschitz and the reward observations are communicated to the player in batches. We introduce a novel landscape aware algorithm, called Batched Lipschitz Narrowing (BLiN), that optimally solves this problem. Specifically, we show that for a T step problem with Lipschitz reward of zooming dimension d z , our algorithm achieves theoretically optimal regret rate of widetilde mathcal O left( T frac d z 1 d z 2 right) using only mathcal O left( log log T right) batches. We also provide complexity analysis for this problem. Our theoretical lower bound implies that widetilde Omega ( log log T) batches are necessary for any algorithm to achieve the optimal regret. Thus, up to logarithmic factors, BLiN achieves optimal regret rate using minimal communication.


A Neurorobotics Approach to Behaviour Selection based on Human Activity Recognition. Behaviour selection has been an active research topic for robotics, in particular in the field of human robot interaction. For a robot to interact effectively and autonomously with humans, the coupling between techniques for human activity recognition, based on sensing information, and robot behaviour selection, based on decision making mechanisms, is of paramount importance. However, most approaches to date consist of deterministic associations between the recognised activities and the robot behaviours, neglecting the uncertainty inherent to sequential predictions in real time applications. In this paper, we address this gap by presenting a neurorobotics approach based on computational models that resemble neurophysiological aspects of living beings. This neurorobotics approach was compared to a non bioinspired, heuristics based approach. To evaluate both approaches, a robot simulation is developed, in which a mobile robot has to accomplish tasks according to the activity being performed by the inhabitant of an intelligent home. The outcomes of each approach were evaluated according to the number of correct outcomes provided by the robot. Results revealed that the neurorobotics approach is advantageous, especially considering the computational models based on more complex animals.


Enhancing Knowledge Tracing via Adversarial Training. We study the problem of knowledge tracing (KT) where the goal is to trace the students' knowledge mastery over time so as to make predictions on their future performance. Owing to the good representation capacity of deep neural networks (DNNs), recent advances on KT have increasingly concentrated on exploring DNNs to improve the performance of KT. However, we empirically reveal that the DNNs based KT models may run the risk of overfitting, especially on small datasets, leading to limited generalization. In this paper, by leveraging the current advances in adversarial training (AT), we propose an efficient AT based KT method (ATKT) to enhance KT model's generalization and thus push the limit of KT. Specifically, we first construct adversarial perturbations and add them on the original interaction embeddings as adversarial examples. The original and adversarial examples are further used to jointly train the KT model, forcing it is not only to be robust to the adversarial examples, but also to enhance the generalization over the original ones. To better implement AT, we then present an efficient attentive LSTM model as KT backbone, where the key is a proposed knowledge hidden state attention module that adaptively aggregates information from previous knowledge hidden states while simultaneously highlighting the importance of current knowledge hidden state to make a more accurate prediction. Extensive experiments on four public benchmark datasets demonstrate that our ATKT achieves new state of the art performance. Code is available at color blue url https github.com xiaopengguo ATKT .


Resilient Path Planning of UAVs against Covert Attacks on UWB Sensors. In this letter, a resilient path planning scheme is proposed to navigate a UAV to the planned (nominal) destination with minimum energy consumption in the presence of a smart attacker. The UAV is equipped with two sensors, a GPS sensor, which is vulnerable to the spoofing attacker, and a well functioning Ultra Wideband (UWB) sensor, which is possible to be fooled. We show that a covert attacker can significantly deviate the UAV's path by simultaneously corrupting the GPS signals and forging control inputs without being detected by the UWB sensor. The prerequisite for the attack occurrence is first discussed. Based on this prerequisite, the optimal attack scheme is proposed, which maximizes the deviation between the nominal destination and the real one. Correspondingly, an energy efficient and resilient navigation scheme based on Pontryagin's maximum principle cite gelfand2000calculus is formulated, which depresses the above covert attacker effectively. To sum up, this problem can be seen as a Stackelberg game cite bacsar1998dynamic between a secure path planner (defender) and a covert attacker. The effectiveness and practicality of our theoretical results are illustrated via two series of simulation examples and a UAV experiment.


Theoretical simulation and design of AlSb thin films solar cells. The effects of thickness, doping concentration and recombination of AlSb films on the performance of CdS AlSb cells are simulated by one dimensional simulation program called analysis of microelectronic and photonic structures(AMPS1D) soft ware to understand the influence of material characteristic (such as carrier concentration and thickness) on the solar cells. The methods to improve the performance of CdS AlSb cells by optimizing the properties of AlSb have been found. The results show that the thicker AlSb film can improve the long wave response for the higher short circuit current density (Jsc ) of CdS AlSb solar cells and the higher carrier concentration of the film can improve open circuit voltage (Voc ) and fill factor (FF), and its optical thickness for CdS AlSb solar cells is in the range of 500nm 2000nm. The conversion efficiency can be improved from 10.6 to15.3 for introducing AlSb Te, AlSb Cu and ZnTe Cu thin films to CdS AlSb structure. Furthermore, the thicker AlSb Te film can improve the short wave response for the higher Jsc of the cells, and its optical thickness CdS AlSb Te AlSb ZnTe Cu solar cells is in the range of 100nm 200nm. And the lower doping concentration can promote Voc and FF to improve the characteristic of the cells.


Lithium ion battery degradation how to model it. Predicting lithium ion battery degradation is worth billions to the global automotive, aviation and energy storage industries, to improve performance and safety and reduce warranty liabilities. However, very few published models of battery degradation explicitly consider the interactions between more than two degradation mechanisms, and none do so within a single electrode. In this paper, the first published attempt to directly couple more than two degradation mechanisms in the negative electrode is reported. The results are used to map different pathways through the complicated path dependent and non linear degradation space. Four degradation mechanisms are coupled in PyBaMM, an open source modelling environment uniquely developed to allow new physics to be implemented and explored quickly and easily. Crucially it is possible to see 'inside' the model and observe the consequences of the different patterns of degradation, such as loss of lithium inventory and loss of active material. For the same cell, five different pathways that can result in end of life have already been found, depending on how the cell is used. Such information would enable a product designer to either extend life or predict life based upon the usage pattern. However, parameterization of the degradation models remains as a major challenge, and requires the attention of the international battery community.


Identifying negativity factors from social media text corpus using sentiment analysis method. Automatic sentiment analysis play vital role in decision making. Many organizations spend a lot of budget to understand their customer satisfaction by manually going over their feedback comments or tweets. Automatic sentiment analysis can give overall picture of the comments received against any event, product, or activity. Usually, the comments tweets are classified into two main classes that are negative or positive. However, the negative comments are too abstract to understand the basic reason or the context. organizations are interested to identify the exact reason for the negativity. In this research study, we hierarchically goes down into negative comments, and link them with more classes. Tweets are extracted from social media sites such as Twitter and Facebook. If the sentiment analysis classifies any tweet into negative class, then we further try to associates that negative comments with more possible negative classes. Based on expert opinions, the negative comments tweets are further classified into 8 classes. Different machine learning algorithms are evaluated and their accuracy are reported.


Routing Approach for P2P Systems Over MANET Network. Thanks to the great progress in mobile and wireless technologies, Internet distributed applications like P2P file sharing are nowadays deployed over MANET (i.e., P2P mobile systems). These applications allow users to search and share diverse multimedia resources over MANET. Due the nature of MANET, P2P mobile systems brought up many new thriving challenges regarding the query routing issue. To tackle this problem, we introduce a novel context aware query routing protocol for unstructured P2P mobile file sharing systems. Our protocol (i) locates relevant peers sharing pertinent resources for user's query and (ii) ensures that those peers would be reached by considering different MANET constraints (e.g., query content, peer mobility, battery energy, peer load). In order to consider all these constraints for choosing the relevant peers, we are based on the technique for order preferences by similarity to ideal solution (TOPSIS). We implemented the proposed protocol and compared its routing efficiency and retrieval effectiveness with another protocol taken from the literature. Experimental results show that our scheme carries out better than the baseline protocol with respect to accuracy


Well defined forward operators in dynamic diffractive tensor tomography using viscosity solutions of transport equations. We consider a general setting for dynamic tensor field tomography in an inhomogeneous refracting and absorbing medium as inverse source problem for the associated transport equation. Following Fermat's principle the Riemannian metric in the considered domain is generated by the refractive index of the medium. There is wealth of results for the inverse problem of recovering a tensor field from its longitudinal ray transform in a static euclidean setting, whereas there are only few inversion formulas and algorithms existing for general Riemannian metrics and time dependent tensor fields. It is a well known fact that tensor field tomography is equivalent to an inverse source problem for a transport equation where the ray transform serves as given boundary data. We prove that this result extends to the dynamic case. Interpreting dynamic tensor tomography as inverse source problem represents a holistic approach in this field. To guarantee that the forward mappings are well defined, it is necessary to prove existence and uniqueness for the underlying transport equations. Unfortunately, the bilinear forms of the associated weak formulations do not satisfy the coercivity condition. To this end we transfer to viscosity solutions and prove their unique existence in appropriate Sobolev (static case) and Sobolev Bochner (dynamic case) spaces under a certain assumption that allows only small variations of the refractive index. Numerical evidence is given that the viscosity solution solves the original transport equation if the viscosity term turns to zero.


In Defense of Scene Graphs for Image Captioning. The mainstream image captioning models rely on Convolutional Neural Network (CNN) image features to generate captions via recurrent models. Recently, image scene graphs have been used to augment captioning models so as to leverage their structural semantics, such as object entities, relationships and attributes. Several studies have noted that the naive use of scene graphs from a black box scene graph generator harms image captioning performance and that scene graph based captioning models have to incur the overhead of explicit use of image features to generate decent captions. Addressing these challenges, we propose textbf SG2Caps , a framework that utilizes only the scene graph labels for competitive image captioning performance. The basic idea is to close the semantic gap between the two scene graphs one derived from the input image and the other from its caption. In order to achieve this, we leverage the spatial location of objects and the Human Object Interaction (HOI) labels as an additional HOI graph. SG2Caps outperforms existing scene graph only captioning models by a large margin, indicating scene graphs as a promising representation for image captioning. Direct utilization of scene graph labels avoids expensive graph convolutions over high dimensional CNN features resulting in 49 fewer trainable parameters. Our code is available at https github.com Kien085 SG2Caps


Resilient Virtualized Systems Using ReHype. System level virtualization introduces critical vulnerabilities to failures of the software components that implement virtualization the virtualization infrastructure (VI). To mitigate the impact of such failures, we introduce a resilient VI (RVI) that can recover individual VI components from failure, caused by hardware or software faults, transparently to the hosted virtual machines (VMs). Much of the focus is on the ReHype mechanism for recovery from hypervisor failures, that can lead to state corruption and to inconsistencies among the states of system components. ReHype's implementation for the Xen hypervisor was done incrementally, using fault injection results to identify sources of critical corruption and inconsistencies. This implementation involved 900 LOC, with memory space overhead of 2.1MB. Fault injection campaigns, with a variety of fault types, show that ReHype can successfully recover, in less than 750ms, from over 88 of detected hypervisor failures. In addition to ReHype, recovery mechanisms for the other VI components are described. The overall effectiveness of our RVI is evaluated hosting a Web service application, on a cluster of VMs. With faults in any VI component, for over 87 of detected failures, our recovery mechanisms allow services provided by the application to be continuously maintained despite the resulting failures of VI components.


Linear block and convolutional MDS codes to required rate, distance and type. Algebraic methods for the design of series of maximum distance separable (MDS) linear block and convolutional codes to required specifications and types are presented. Algorithms are given to design codes to required rate and required error correcting capability and required types. Infinite series of block codes with rate approaching a given rational R with 0 R 1 and relative distance over length approaching (1 R) are designed. These can be designed over fields of given characteristic p or over fields of prime order and can be specified to be of a particular type such as (i) dual containing under Euclidean inner product, (ii) dual containing under Hermitian inner product, (iii) quantum error correcting, (iv) linear complementary dual (LCD). Convolutional codes to required rate and distance and infinite series of convolutional codes with rate approaching a given rational R and distance over length approaching 2(1 R) are designed. The designs are algebraic and properties, including distances, are shown algebraically. Algebraic explicit efficient decoding methods are referenced.


Intelligent Bearing Fault Diagnosis Method Combining Mixed Input and Hybrid CNN MLP model. Rolling bearings are one of the most widely used bearings in industrial machines. Deterioration in the condition of rolling bearings can result in the total failure of rotating machinery. AI based methods are widely applied in the diagnosis of rolling bearings. Hybrid NN based methods have been shown to achieve the best diagnosis results. Typically, raw data is generated from accelerometers mounted on the machine housing. However, the diagnostic utility of each signal is highly dependent on the location of the corresponding accelerometer. This paper proposes a novel hybrid CNN MLP model based diagnostic method which combines mixed input to perform rolling bearing diagnostics. The method successfully detects and localizes bearing defects using acceleration data from a shaft mounted wireless acceleration sensor. The experimental results show that the hybrid model is superior to the CNN and MLP models operating separately, and can deliver a high detection accuracy of 99,6 for the bearing faults compared to 98 for CNN and 81 for MLP models.


Algorithmic Solution for Non Square, Dense Systems of Linear Equations, with applications in Feature Selection. We present a novel algorithm attaining excessively fast, the sought solution of linear systems of equations. The algorithm is short in its basic formulation and by definition vectorised, while the memory allocation demands trivial, because for each iteration only one dimension of the given input matrix mathbf x is utilized. The execution time is very short compared with state of the art methods, exhibiting up to mathcal O (10 3) speed up and low memory allocation demands, especially for non square Systems of Linear Equations, with ratio of equations versus features high (tall systems), or low (wide systems) accordingly. The accuracy is high and straightforwardly controlled, and the numerical results highlight the efficiency of the proposed algorithm, in terms of computation time, solution accuracy and memory allocations demands. The parallelisation of the algorithm is also presented in multi threaded and GPU accelerators' setting. The paper also comprises a theoretical proof for the algorithmic convergence. Finally, we extend the implementation of the proposed algorithmic rationale to feature selection tasks.


Boundary integral formulations for acoustic modelling of high contrast media. The boundary element method is an efficient algorithm for simulating acoustic propagation through homogeneous objects embedded in free space. The conditioning of the system matrix strongly depends on physical parameters such as density, wavespeed and frequency. In particular, high contrast in density and wavespeed across a material interface leads to an ill conditioned discretisation matrix. Therefore, the convergence of Krylov methods to solve the linear system is slow. Here, specialised boundary integral formulations are designed for the case of acoustic scattering at high contrast media. The eigenvalues of the resulting system matrix accumulate at two points in the complex plane that depend on the density ratio and stay away from zero. The spectral analysis of the Calder 'on preconditioned PMCHWT formulation yields a single accumulation point. Benchmark simulations demonstrate the computational efficiency of the high contrast Neumann formulation for scattering at high contrast media.


Cryptocurrency Solutions to Enable Micro payments in Consumer IoT. The successful amalgamation of cryptocurrency and consumer Internet of Things (IoT) devices can pave the way for novel applications in machine to machine economy. However, the lack of scalability and heavy resource requirements of initial blockchain designs hinders the integration as they prioritized decentralization and security. Numerous solutions have been proposed since the emergence of Bitcoin to achieve this goal. However, none of them seem to dominate and thus it is unclear how consumer devices will be adopting these approaches. Therefore, in this paper, we critically review the existing integration approaches and cryptocurrency designs that strive to enable micro payments among consumer devices. We identify and discuss solutions under three main categories direct integration, payment channel network and new cryptocurrency design. The first approach utilizes a full node to interact with the payment system. Offline channel payment is suggested as a second layer solution to solve the scalability issue and enable instant payment with low fee. New designs converge to semi centralized scheme and focuson lightweight consensus protocol that does not require highcomputation power which might mean loosening the initial designchoices in favor of scalability. We evaluate the pros and cons ofeach of these approaches and then point out future researchchallenges. Our goal is to help researchers and practitioners tobetter focus their efforts to facilitate micro payment adoptions.


Generalizations of a terminating summation formula of basic hypergeometric series and their applications. We generalize a terminating summation formula to a unilateral nonterminating, and further, a bilateral summation formula by a property of analytic functions. The unilateral one is proved to be a q analogue of a 4F 3 summation formula. And, an identity unifying Jacobi's triple product identity and the quintuple product identity is obtained as a special case of the bilateral one.


Analysis of Activity Dependent Development of Topographic Maps in Neural Field Theory with Short Time Scale Dependent Plasticity. Topographic maps are a brain structure connecting pre synpatic and post synaptic brain regions. Topographic development is dependent on Hebbian based plasticity mechanisms working in conjunction with spontaneous patterns of neural activity generated in the pre synaptic regions. Studies performed in mouse have shown that these spontaneous patterns can exhibit complex spatial temporal structures which existing models cannot incorporate. Neural field theories are appropriate modelling paradigms for topographic systems due to the dense nature of the connections between regions and can be augmented with a plasticity rule general enough to capture complex time varying structures. We propose a theoretical framework for studying the development of topography in the context of complex spatial temporal activity fed forward from the pre synaptic to post synaptic regions. Analysis of the model leads to an analytic solution corroborating the conclusion that activity can drive the refinement of topographic projections. The analysis also suggests that biological noise is used in the development of topography to stabilise the dynamics. MCMC simulations are used to analyse and understand the differences in topographic refinement between wild type and the beta2 knock out mutant in mice. The time scale of the synaptic plasticity window is estimated as 0.56 seconds in this context with a model fit of R 2 0.81 .


HBReID Harder Batch for Re identification. Triplet loss is a widely adopted loss function in ReID task which pulls the hardest positive pairs close and pushes the hardest negative pairs far away. However, the selected samples are not the hardest globally, but the hardest only in a mini batch, which will affect the performance. In this report, a hard batch mining method is proposed to mine the hardest samples globally to make triplet harder. More specifically, the most similar classes are selected into a same mini batch so that the similar classes could be pushed further away. Besides, an adversarial scene removal module composed of a scene classifier and an adversarial loss is used to learn scene invariant feature representations. Experiments are conducted on dataset MSMT17 to prove the effectiveness, and our method surpasses all of the previous methods and sets state of the art result.


AR BERT Aspect relation enhanced Aspect level Sentiment Classification with Multi modal Explanations. Aspect level sentiment classification (ALSC) is a difficult problem with state of the art models showing less than 80 macro F1 score on benchmark datasets. Existing models do not incorporate information on aspect aspect relations in knowledge graphs (KGs), e.g. DBpedia. Two main challenges stem from inaccurate disambiguation of aspects to KG entities, and the inability to learn aspect representations from the large KGs in joint training with ALSC models. We propose AR BERT, a novel two level global local entity embedding scheme that allows efficient joint training of KG based aspect embeddings and ALSC models. A novel incorrect disambiguation detection technique addresses the problem of inaccuracy in aspect disambiguation. We also introduce the problem of determining mode significance in multi modal explanation generation, and propose a two step solution. The proposed methods show a consistent improvement of 2.5 4.1 percentage points, over the recent BERT based baselines on benchmark datasets. The code is available at https github.com mainuliitkgp AR BERT.git.


A second order expansion in the local limit theorem for a branching system of symmetric irreducible random walks. Consider a branching random walk, where the branching mechanism is governed by a Galton Watson process, and the migration by a finite range symmetric irreducible random walk on the integer lattice mathbb Z d . Let Z n(z) be the number of the particles in the n th generation at the point z in mathbb Z d . Under the mild moment conditions for offspring distribution of the underlying Galton Watson, we derive a second order expansion in the local limit theorem for Z n(z) for each given z in mathbb Z d . That generalizes the results for simple branching random walks obtained by Gao 2018, SPA .


Multilingual Multi Aspect Explainability Analyses on Machine Reading Comprehension Models. Achieving human level performance on some of the Machine Reading Comprehension (MRC) datasets is no longer challenging with the help of powerful Pre trained Language Models (PLMs). However, the internal mechanism of these artifacts remains unclear, placing an obstacle for further understanding these models. This paper focuses on conducting a series of analytical experiments to examine the relations between the multi head self attention and the final MRC system performance, revealing the potential explainability in PLM based MRC models. To ensure the robustness of the analyses, we perform our experiments in a multilingual way on top of various PLMs. We discover that passage to question and passage understanding attentions are the most important ones in the question answering process, showing strong correlations to the final performance than other parts. Through comprehensive visualizations and case studies, we also observe several general findings on the attention maps, which can be helpful to understand how these models solve the questions.


Playing Codenames with Language Graphs and Word Embeddings. Although board games and video games have been studied for decades in artificial intelligence research, challenging word games remain relatively unexplored. Word games are not as constrained as games like chess or poker. Instead, word game strategy is defined by the players' understanding of the way words relate to each other. The word game Codenames provides a unique opportunity to investigate common sense understanding of relationships between words, an important open challenge. We propose an algorithm that can generate Codenames clues from the language graph BabelNet or from any of several embedding methods word2vec, GloVe, fastText or BERT. We introduce a new scoring function that measures the quality of clues, and we propose a weighting term called DETECT that incorporates dictionary based word representations and document frequency to improve clue selection. We develop BabelNet Word Selection Framework (BabelNet WSF) to improve BabelNet clue quality and overcome the computational barriers that previously prevented leveraging language graphs for Codenames. Extensive experiments with human evaluators demonstrate that our proposed innovations yield state of the art performance, with up to 102.8 improvement in precision 2 in some cases. Overall, this work advances the formal study of word games and approaches for common sense language understanding.


Penn Helsinki Parsed Corpus of Early Modern English First Parsing Results and Analysis. We present the first parsing results on the Penn Helsinki Parsed Corpus of Early Modern English (PPCEME), a 1.9 million word treebank that is an important resource for research in syntactic change. We describe key features of PPCEME that make it challenging for parsing, including a larger and more varied set of function tags than in the Penn Treebank. We present results for this corpus using a modified version of the Berkeley Neural Parser and the approach to function tag recovery of Gabbard et al (2006). Despite its simplicity, this approach works surprisingly well, suggesting it is possible to recover the original structure with sufficient accuracy to support linguistic applications (e.g., searching for syntactic structures of interest). However, for a subset of function tags (e.g., the tag indicating direct speech), additional work is needed, and we discuss some further limits of this approach. The resulting parser will be used to parse Early English Books Online, a 1.1 billion word corpus whose utility for the study of syntactic change will be greatly increased with the addition of accurate parse trees.


Everything Has a Cause Leveraging Causal Inference in Legal Text Analysis. Causal inference is the process of capturing cause effect relationship among variables. Most existing works focus on dealing with structured data, while mining causal relationship among factors from unstructured data, like text, has been less examined, but is of great importance, especially in the legal domain. In this paper, we propose a novel Graph based Causal Inference (GCI) framework, which builds causal graphs from fact descriptions without much human involvement and enables causal inference to facilitate legal practitioners to make proper decisions. We evaluate the framework on a challenging similar charge disambiguation task. Experimental results show that GCI can capture the nuance from fact descriptions among multiple confusing charges and provide explainable discrimination, especially in few shot settings. We also observe that the causal knowledge contained in GCI can be effectively injected into powerful neural networks for better performance and interpretability.


Text Classification Using Hybrid Machine Learning Algorithms on Big Data. Recently, there are unprecedented data growth originating from different online platforms which contribute to big data in terms of volume, velocity, variety and veracity (4Vs). Given this nature of big data which is unstructured, performing analytics to extract meaningful information is currently a great challenge to big data analytics. Collecting and analyzing unstructured textual data allows decision makers to study the escalation of comments posts on our social media platforms. Hence, there is need for automatic big data analysis to overcome the noise and the non reliability of these unstructured dataset from the digital media platforms. However, current machine learning algorithms used are performance driven focusing on the classification prediction accuracy based on known properties learned from the training samples. With the learning task in a large dataset, most machine learning models are known to require high computational cost which eventually leads to computational complexity. In this work, two supervised machine learning algorithms are combined with text mining techniques to produce a hybrid model which consists of Na "ive Bayes and support vector machines (SVM). This is to increase the efficiency and accuracy of the results obtained and also to reduce the computational cost and complexity. The system also provides an open platform where a group of persons with a common interest can share their comments messages and these comments classified automatically as legal or illegal. This improves the quality of conversation among users. The hybrid model was developed using WEKA tools and Java programming language. The result shows that the hybrid model gave 96.76 accuracy as against the 61.45 and 69.21 of the Na "ive Bayes and SVM models respectively.


Methods for estimating of continuants, corrected version. The purpose of 1 was as follows. ?We consider special sets of continuants which occur in applications. For these sets we solve the problem of finding maximal and minimal continuants. There are several methods for finding extremum such as thee mathod of basic substitutions (section 4.1), method of quadratic irrationalities (section 4.2), majorizing inequalities (section 4.3) and unit variation (section 4.4).? Unfortunately, the text of 1 was littered with numerous typos, making it difficult to read. This defect is corrected in the present work. In addition, some of the statements are now provided with more detailed evidence.


Low resource Learning with Knowledge Graphs A Comprehensive Survey. Machine learning methods especially deep neural networks have achieved great success but many of them often rely on a number of labeled samples for training. In real world applications, we often need to address sample shortage due to e.g., dynamic contexts with emerging prediction targets and costly sample annotation. Therefore, low resource learning, which aims to learn robust prediction models with no enough resources (especially training samples), is now being widely investigated. Among all the low resource learning studies, many prefer to utilize some auxiliary information in the form of Knowledge Graph (KG), which is becoming more and more popular for knowledge representation, to reduce the reliance on labeled samples. In this survey, we very comprehensively reviewed over 90 papers about KG aware research for two major low resource learning settings zero shot learning (ZSL) where new classes for prediction have never appeared in training, and few shot learning (FSL) where new classes for prediction have only a small number of labeled samples that are available. We first introduced the KGs used in ZSL and FSL studies as well as the existing and potential KG construction solutions, and then systematically categorized and summarized KG aware ZSL and FSL methods, dividing them into different paradigms such as the mapping based, the data augmentation, the propagation based and the optimization based. We next presented different applications, including not only KG augmented tasks in Computer Vision and Natural Language Processing (e.g., image classification, text classification and knowledge extraction), but also tasks for KG curation (e.g., inductive KG completion), and some typical evaluation resources for each task. We eventually discussed some challenges and future directions on aspects such as new learning and reasoning paradigms, and the construction of high quality KGs.


Some Remarks on Small Values of tau(n) . A natural variant of Lehmer's conjecture that the Ramanujan tau function never vanishes asks whether, for any given integer alpha , there exist any n in mathbb Z such that tau(n) alpha . A series of recent papers excludes many integers as possible values of the tau function using the theory of primitive divisors of Lucas numbers, computations of integer points on curves, and congruences for tau(n) . We synthesize these results and methods to prove that if 0 alpha 100 and alpha notin T 2 k, 24, 48, 70, 90, 92, 96 , then tau(n) neq alpha for all n 1 . Moreover, if alpha in T and tau(n) alpha , then n is square free with prescribed prime factorization. Finally, we show that a strong form of the Atkin Serre conjecture implies that tau(n) 100 for all n 2 .


ContractNLI A Dataset for Document level Natural Language Inference for Contracts. Reviewing contracts is a time consuming procedure that incurs large expenses to companies and social inequality to those who cannot afford it. In this work, we propose "document level natural language inference (NLI) for contracts", a novel, real world application of NLI that addresses such problems. In this task, a system is given a set of hypotheses (such as "Some obligations of Agreement may survive termination.") and a contract, and it is asked to classify whether each hypothesis is "entailed by", "contradicting to" or "not mentioned by" (neutral to) the contract as well as identifying "evidence" for the decision as spans in the contract. We annotated and release the largest corpus to date consisting of 607 annotated contracts. We then show that existing models fail badly on our task and introduce a strong baseline, which (1) models evidence identification as multi label classification over spans instead of trying to predict start and end tokens, and (2) employs more sophisticated context segmentation for dealing with long documents. We also show that linguistic characteristics of contracts, such as negations by exceptions, are contributing to the difficulty of this task and that there is much room for improvement.


A Smart and Defensive Human Machine Approach to Code Analysis. Static analysis remains one of the most popular approaches for detecting and correcting poor or vulnerable program code. It involves the examination of code listings, test results, or other documentation to identify errors, violations of development standards, or other problems, with the ultimate goal of fixing these errors so that systems and software are as secure as possible. There exists a plethora of static analysis tools, which makes it challenging for businesses and programmers to select a tool to analyze their program code. It is imperative to find ways to improve code analysis so that it can be employed by cyber defenders to mitigate security risks. In this research, we propose a method that employs the use of virtual assistants to work with programmers to ensure that software are as safe as possible in order to protect safety critical systems from data breaches and other attacks. The proposed method employs a recommender system that uses various metrics to help programmers select the most appropriate code analysis tool for their project and guides them through the analysis process. The system further tracks the user's behavior regarding the adoption of the recommended practices.


A New Clustering Based Technique for the Acceleration of Deep Convolutional Networks. Deep learning and especially the use of Deep Neural Networks (DNNs) provides impressive results in various regression and classification tasks. However, to achieve these results, there is a high demand for computing and storing resources. This becomes problematic when, for instance, real time, mobile applications are considered, in which the involved (embedded) devices have limited resources. A common way of addressing this problem is to transform the original large pre trained networks into new smaller models, by utilizing Model Compression and Acceleration (MCA) techniques. Within the MCA framework, we propose a clustering based approach that is able to increase the number of employed centroids representatives, while at the same time, have an acceleration gain compared to conventional, k means based approaches. This is achieved by imposing a special structure to the employed representatives, which is enabled by the particularities of the problem at hand. Moreover, the theoretical acceleration gains are presented and the key system hyper parameters that affect that gain, are identified. Extensive evaluation studies carried out using various state of the art DNN models trained in image classification, validate the superiority of the proposed method as compared for its use in MCA tasks.


Musicological, computational and conceptual aspects of first species counterpoint theory. We re create the essential results of a 1989 unpublished article by Mazzola and Muzzulini that contains musicological aspects of a first species counterpoint model. We include a summary of the mathematical counterpoint theory and several variations of the model that offer different perspectives on Mazzola's original principles.


Learning suction graspability considering grasp quality and robot reachability for bin picking. Deep learning has been widely used for inferring robust grasps. Although human labeled RGB D datasets were initially used to learn grasp configurations, preparation of this kind of large dataset is expensive. To address this problem, images were generated by a physical simulator, and a physically inspired model (e.g., a contact model between a suction vacuum cup and object) was used as a grasp quality evaluation metric to annotate the synthesized images. However, this kind of contact model is complicated and requires parameter identification by experiments to ensure real world performance. In addition, previous studies have not considered manipulator reachability such as when a grasp configuration with high grasp quality is unable to reach the target due to collisions or the physical limitations of the robot. In this study, we propose an intuitive geometric analytic based grasp quality evaluation metric. We further incorporate a reachability evaluation metric. We annotate the pixel wise grasp quality and reachability by the proposed evaluation metric on synthesized images in a simulator to train an auto encoder decoder called suction graspability U Net (SG U Net ). Experiment results show that our intuitive grasp quality evaluation metric is competitive with a physically inspired metric. Learning the reachability helps to reduce motion planning computation time by removing obviously unreachable candidates. The system achieves an overall picking speed of 560 PPH (pieces per hour).


Solar cell patent classification method based on keyword extraction and deep neural network. With the growing impact of ESG on businesses, research related to renewable energy is receiving great attention. Solar cells are one of them, and accordingly, it can be said that the research value of solar cell patent analysis is very high. Patent documents have high research value. Being able to accurately analyze and classify patent documents can reveal several important technical relationships. It can also describe the business trends in that technology. And when it comes to investment, new industrial solutions will also be inspired and proposed to make important decisions. Therefore, we must carefully analyze patent documents and utilize the value of patents. To solve the solar cell patent classification problem, we propose a keyword extraction method and a deep neural network based solar cell patent classification method. First, solar cell patents are analyzed for pretreatment. It then uses the KeyBERT algorithm to extract keywords and key phrases from the patent abstract to construct a lexical dictionary. We then build a solar cell patent classification model according to the deep neural network. Finally, we use a deep neural network based solar cell patent classification model to classify power patents, and the training accuracy is greater than 95 . Also, the validation accuracy is about 87.5 . It can be seen that the deep neural network method can not only realize the classification of complex and difficult solar cell patents, but also have a good classification effect.


Graded Specht modules as Bernstein Zelevinsky derivatives of the RSK model. We clarify the links between the graded Specht construction of modules over cyclotomic Hecke algebras and the RSK construction for quiver Hecke algebras of type A, that was recently imported from the setting of representations of p adic groups. For that goal we develop a theory of crystal derivative operators on quiver Hecke algebra modules, that categorifies the Berenstein Zelevinsky strings framework on quantum groups, and generalizes a graded variant of the classical Bernstein Zelevinsky derivatives from the p adic setting. Graded cyclotomic decomposition numbers are shown to be a special subfamily of the wider concept of RSK decomposition numbers.


Using Predicted Weights for Ad Delivery. We study the performance of a proportional weights algorithm for online capacitated bipartite matching modeling the delivery of impression ads. The algorithm uses predictions on the advertiser nodes to match arriving impression nodes fractionally in proportion to the weights of its neighbors. This paper gives a thorough empirical study of the performance of the algorithm on a data set of ad impressions from Yahoo! and shows its superior performance compared to natural baselines such as a greedy water filling algorithm and the ranking algorithm. The proportional weights algorithm has recently received interest in the theoretical literature where it was shown to have strong guarantees beyond the worst case model of algorithms augmented with predictions. We extend these results to the case where the advertisers' capacities are no longer stationary over time. Additionally, we show the algorithm has near optimal performance in the random order arrival model when the number of impressions and the optimal matching are sufficiently large.


A deep potential model with long range electrostatic interactions. Machine learning models for the potential energy of multi atomic systems, such as the deep potential (DP) model, make possible molecular simulations with the accuracy of quantum mechanical density functional theory, at a cost only moderately higher than that of empirical force fields. However, the majority of these models lack explicit long range interactions and fail to describe properties that derive from the Coulombic tail of the forces. To overcome this limitation we extend the DP model by approximating the long range electrostatic interaction between ions (nuclei core electrons) and valence electrons with that of distributions of spherical Gaussian charges located at ionic and electronic sites. The latter are rigorously defined in terms of the centers of the maximally localized Wannier distributions, whose dependence on the local atomic environment is modeled accurately by a deep neural network. In the deep potential long range (DPLR) model, the electrostatic energy of the Gaussian charge system is added to short range interactions that are represented as in the standard DP model. The resulting potential energy surface is smooth and possesses analytical forces and virial. Missing effects in the standard DP scheme are recovered, improving on accuracy and predictive power. By including long range electrostatics, DPLR correctly extrapolates to large systems the potential energy surface learned from quantum mechanical calculations on smaller systems. We illustrate the approach with three examples, the potential energy profile of the water dimer, the free energy of interaction of a water molecule with a liquid water slab, and the phonon dispersion curves of the NaCl crystal.


cs60075 team2 at SemEval 2021 Task 1 Lexical Complexity Prediction using Transformer based Language Models pre trained on various text corpora. This paper describes the performance of the team cs60075 team2 at SemEval 2021 Task 1 Lexical Complexity Prediction. The main contribution of this paper is to fine tune transformer based language models pre trained on several text corpora, some being general (E.g., Wikipedia, BooksCorpus), some being the corpora from which the CompLex Dataset was extracted, and others being from other specific domains such as Finance, Law, etc. We perform ablation studies on selecting the transformer models and how their individual complexity scores are aggregated to get the resulting complexity scores. Our method achieves a best Pearson Correlation of 0.784 in sub task 1 (single word) and 0.836 in sub task 2 (multiple word expressions).


On convergence rates of adaptive ensemble Kalman inversion for linear ill posed problems. In this paper we discuss a deterministic form of ensemble Kalman inversion as a regularization method for linear inverse problems. By interpreting ensemble Kalman inversion as a low rank approximation of Tikhonov regularization, we are able to introduce a new sampling scheme based on the Nystr "om method that improves practical performance. Furthermore, we formulate an adaptive version of ensemble Kalman inversion where the sample size is coupled with the regularization parameter. We prove that the proposed scheme yields an order optimal regularization method under standard assumptions if the discrepancy principle is used as a stopping criterion. The paper concludes with a numerical comparison of the discussed methods for an inverse problem of the Radon transform.


How Knowledge Graph and Attention Help? A Quantitative Analysis into Bag level Relation Extraction. Knowledge Graph (KG) and attention mechanism have been demonstrated effective in introducing and selecting useful information for weakly supervised methods. However, only qualitative analysis and ablation study are provided as evidence. In this paper, we contribute a dataset and propose a paradigm to quantitatively evaluate the effect of attention and KG on bag level relation extraction (RE). We find that (1) higher attention accuracy may lead to worse performance as it may harm the model's ability to extract entity mention features (2) the performance of attention is largely influenced by various noise distribution patterns, which is closely related to real world datasets (3) KG enhanced attention indeed improves RE performance, while not through enhanced attention but by incorporating entity prior and (4) attention mechanism may exacerbate the issue of insufficient training data. Based on these findings, we show that a straightforward variant of RE model can achieve significant improvements (6 AUC on average) on two real world datasets as compared with three state of the art baselines. Our codes and datasets are available at https github.com zig kwin hu how KG ATT help.


On Strominger space forms. In this article, we propose the following conjecture if the Strominger connection of a compact Hermitian manifold has constant non zero holomorphic sectional curvature, then the Hermitian metric must be K "ahler. The main result of this article is to confirm the conjecture in dimension 2 . We also verify the conjecture in higher dimensions in a couple of special situations.


Contextual Skipgram Training Word Representation Using Context Information. The skip gram (SG) model learns word representation by predicting the words surrounding a center word from unstructured text data. However, not all words in the context window contribute to the meaning of the center word. For example, less relevant words could be in the context window, hindering the SG model from learning a better quality representation. In this paper, we propose an enhanced version of the SG that leverages context information to produce word representation. The proposed model, Contextual Skip gram, is designed to predict contextual words with both the center words and the context information. This simple idea helps to reduce the impact of irrelevant words on the training process, thus enhancing the final performance


A State Transfer Method That Adapts to Network Bandwidth Variations in Geographic State Machine Replication. We present a new state transfer method for geographic State Machine Replication (SMR) that dynamically allocates the state to be transferred among replicas according to changes in communication bandwidths. SMR is a method that improves fault tolerance by replicating a service to multiple replicas. When a replica is newly added or is recovered from a failure, the other replicas transfer the current state of the service to it. However, in geographic SMR, the communication bandwidths of replicas are different and constantly changing. Therefore, existing state transfer methods cannot fully utilize the available bandwidth, and their state transfer time becomes long. To overcome this problem, our method divides the state into multiple chunks and assigns them to replicas based on each replica's bandwidth so that the broader a replica's bandwidth is, the more chunks it transfers. The number of assigned chunks is dynamically updated based on the currently estimated bandwidth. The performance evaluation on Amazon EC2 shows that the proposed method reduces the state transfer time by up to 47 compared with the existing one.


Quantifying the mobility of chromatin during embryogenesis Nuclear size matters. Chromatin moves dynamically inside the cell nucleus, and its motion is often correlated with gene functions such as DNA recombination and transcription. A recent study has shown that during early embryogenesis of the nematode, Caenorhabiditis elegans, the chromatin motion markedly decreases. However, the underlying mechanism for this transition has yet to be elucidated. We systematically investigated the impact of nuclear size to demonstrate that it is indeed a decisive factor in chromatin mobility. To this end, we established a method to quantify chromatin motion inside the nucleus, while excluding the contribution of the movement of the nucleus itself, which allowed us to extract the intrinsic mean squared displacement (iMSD) of individual chromosomal loci in moving nuclei from the correlated motion of two loci. We show that a simple theoretical description, which takes into account the topological constraints of chromatin polymers, can quantitatively describe the relationship between the nucleus size and the chromatin motion in vivo. Our results emphasize a regulatory role of nuclear size in restricting chromatin motion, and a generic polymer physics model plays a guiding role in capturing this essential feature.


Multi kV class beta Ga 2 O 3 MESFETs with a Lateral Figure of Merit up to 355 MW cm 2 . We demonstrate over 3 kV gate pad connected field plated (GPFP) beta Ga 2 O 3 lateral MESFETs with high lateral figure of merit (LFOM) using metalorganic vapor phase epitaxy (MOVPE) grown channel layers and regrown ohmic contact layers. Using an improved low temperature MOVPE selective area epitaxy process, we show that a total contact resistance to the channel as low as 1.4 Omega .mm can be achieved.The GPFP design adopted here using PECVD (plasma enhanced chemical vapor deposition) deposited SiN x dielectric and SiN x SiO 2 wrap around passivation exhibits up to 14 improved R ON , up to 70 improved breakdown voltage (V BR V DS V GS ) resulting in up to sim 3 times higher LFOM compared to non FP beta Ga 2 O 3 lateral MESFETs. The V BR ( 2.5 kV) and LFOM (355 MW cm 2 ) measured simultaneously in our GPFP beta Ga 2 O 3 lateral MESFET (with L GD 10 mu m) is the highest value achieved in any depletion mode beta Ga 2 O 3 lateral device.


A Quantitative Analysis of Activities of Daily Living Insights into Improving Functional Independence with Assistive Robotics. Human assistive robotics have the potential to help the elderly and individuals living with disabilities with their Activities of Daily Living (ADL). Robotics researchers focus on assistive tasks from the perspective of various control schemes and motion types. Health research on the other hand focuses on clinical assessment and rehabilitation, arguably leaving important differences between the two domains. In particular, little is known quantitatively on which ADLs are typically carried out in a persons everyday environment at home, work, etc. Understanding what activities are frequently carried out during the day can help guide the development and prioritization of robotic technology for in home assistive robotic deployment. This study targets several lifelogging databases, where we compute (i) ADL task frequency from long term low sampling frequency video and Internet of Things (IoT) sensor data, and (ii) short term arm and hand movement data from 30 fps video data of domestic tasks. Robotics and health care communities have differing terms and taxonomies for representing tasks and motions. In this work, we derive and discuss a robotics relevant taxonomy from quantitative ADL task and motion data in attempt to ameliorate taxonomic differences between the two communities. Our quantitative results provide direction for the development of better assistive robots to support the true demands of the healthcare community.


Real Time Egocentric Object Segmentation THU READ Labeling and Benchmarking Results. Egocentric segmentation has attracted recent interest in the computer vision community due to their potential in Mixed Reality (MR) applications. While most previous works have been focused on segmenting egocentric human body parts (mainly hands), little attention has been given to egocentric objects. Due to the lack of datasets of pixel wise annotations of egocentric objects, in this paper we contribute with a semantic wise labeling of a subset of 2124 images from the RGB D THU READ Dataset. We also report benchmarking results using Thundernet, a real time semantic segmentation network, that could allow future integration with end to end MR applications.


A Learning Based Computational Impact Time Guidance. This paper investigates the problem of impact time control and proposes a learning based computational guidance algorithm to solve this problem. The proposed guidance algorithm is developed based on a general prediction correction concept the exact time to go under proportional navigation guidance with realistic aerodynamic characteristics is estimated by a deep neural network and a biased command to nullify the impact time error is developed by utilizing the emerging reinforcement learning techniques. The deep neural network is augmented into the reinforcement learning block to resolve the issue of sparse reward that has been observed in typical reinforcement learning formulation. Extensive numerical simulations are conducted to support the proposed algorithm.


A new orbit closure in genus 8. We prove that the orbit closure of the unfoldings of the (3,4,13) triangle is a previously unknown 4 dimensional variety, by using a Hurwitz space of covers with monodromy in a certain solvable group of order 400.


Hessian Aware Pruning and Optimal Neural Implant. Pruning is an effective method to reduce the memory footprint and FLOPs associated with neural network models. However, existing structured pruning methods often result in significant accuracy degradation for moderate pruning levels. To address this problem, we introduce a new Hessian Aware Pruning (HAP) method coupled with a Neural Implant approach that uses second order sensitivity as a metric for structured pruning. The basic idea is to prune insensitive components and to use a Neural Implant for moderately sensitive components, instead of completely pruning them. For the latter approach, the moderately sensitive components are replaced with with a low rank implant that is smaller and less computationally expensive than the original component. We use the relative Hessian trace to measure sensitivity, as opposed to the magnitude based sensitivity metric commonly used in the literature. We test HAP for both computer vision tasks and natural language tasks, and we achieve new state of the art results. Specifically, HAP achieves less than 0.1 0.5 degradation on PreResNet29 ResNet50 (CIFAR 10 ImageNet) with more than 70 50 of parameters pruned. Meanwhile, HAP also achieves significantly better performance (up to 0.8 with 60 of parameters pruned) as compared to gradient based method for head pruning on transformer based models. The framework has been open sourced and available online.


AutoNovel Automatically Discovering and Learning Novel Visual Categories. We tackle the problem of discovering novel classes in an image collection given labelled examples of other classes. We present a new approach called AutoNovel to address this problem by combining three ideas (1) we suggest that the common approach of bootstrapping an image representation using the labelled data only introduces an unwanted bias, and that this can be avoided by using self supervised learning to train the representation from scratch on the union of labelled and unlabelled data (2) we use ranking statistics to transfer the model's knowledge of the labelled classes to the problem of clustering the unlabelled images and, (3) we train the data representation by optimizing a joint objective function on the labelled and unlabelled subsets of the data, improving both the supervised classification of the labelled data, and the clustering of the unlabelled data. Moreover, we propose a method to estimate the number of classes for the case where the number of new categories is not known a priori. We evaluate AutoNovel on standard classification benchmarks and substantially outperform current methods for novel category discovery. In addition, we also show that AutoNovel can be used for fully unsupervised image clustering, achieving promising results.


Video Salient Object Detection via Contrastive Features and Attention Modules. Video salient object detection aims to find the most visually distinctive objects in a video. To explore the temporal dependencies, existing methods usually resort to recurrent neural networks or optical flow. However, these approaches require high computational cost, and tend to accumulate inaccuracies over time. In this paper, we propose a network with attention modules to learn contrastive features for video salient object detection without the high computational temporal modeling techniques. We develop a non local self attention scheme to capture the global information in the video frame. A co attention formulation is utilized to combine the low level and high level features. We further apply the contrastive learning to improve the feature representations, where foreground region pairs from the same video are pulled together, and foreground background region pairs are pushed away in the latent space. The intra frame contrastive loss helps separate the foreground and background features, and the inter frame contrastive loss improves the temporal consistency. We conduct extensive experiments on several benchmark datasets for video salient object detection and unsupervised video object segmentation, and show that the proposed method requires less computation, and performs favorably against the state of the art approaches.


A Complete Axiomatisation for Divergence Preserving Branching Congruence of Finite State Behaviours. We present an equational inference system for finite state expressions, and prove that the system is sound and complete with respect to divergence preserving branching congruence, closing a problem that has been open since 1993. The inference system refines Rob van Glabbeek's simple and elegant complete axiomatisation for branching bisimulation congruence of finite state behaviours by joining four simple axioms after dropping one axiom which is unsound under the more refined divergence sensitive semantics.


A note on circular wirelength for hypercubes. We study embeddings of the n dimensional hypercube into the circuit with 2 n vertices. We prove that the circular wirelength attains minimum by gray coding, which is called the CT conjecture by Chavez and Trapp (Discrete Applied Mathematics, 1998). This problem had claimed to be settled by Ching Jung Guu in her doctor dissertation "The circular wirelength problem for hypercubes" (University of California, Riverside, 1997). Many people argue there are gaps in her proof. We eliminate gaps in her dissertation.


Differentiable Dynamic Quantization with Mixed Precision and Adaptive Resolution. Model quantization is challenging due to many tedious hyper parameters such as precision (bitwidth), dynamic range (minimum and maximum discrete values) and stepsize (interval between discrete values). Unlike prior arts that carefully tune these values, we present a fully differentiable approach to learn all of them, named Differentiable Dynamic Quantization (DDQ), which has several benefits. (1) DDQ is able to quantize challenging lightweight architectures like MobileNets, where different layers prefer different quantization parameters. (2) DDQ is hardware friendly and can be easily implemented using low precision matrix vector multiplication, making it capable in many hardware such as ARM. (3) Extensive experiments show that DDQ outperforms prior arts on many networks and benchmarks, especially when models are already efficient and compact. e.g., DDQ is the first approach that achieves lossless 4 bit quantization for MobileNetV2 on ImageNet.


Connecting two types of representations of a permutation of F q . In this paper, we connect two types of representations of a permutation sigma of the finite field F q . One type is algebraic, in which the permutation is represented as the composition of degree one polynomials and k copies of x q 2 , for some prescribed value of k . The other type is combinatorial, in which the permutation is represented as the composition of a degree one rational function followed by the product of k 2 cycles on bP 1( F q) F q cup infty , where each 2 cycle moves infty . We show that, after modding out by obvious equivalences amongst the algebraic representations, then for each k there is a bijection between the algebraic representations of sigma and the combinatorial representations of sigma . We also prove analogous results for permutations of bP 1( F q) . One consequence is a new characterization of the notion of Carlitz rank of a permutation on F q , which we use elsewhere to provide an explicit formula for the Carlitz rank. Another consequence involves a classical theorem of Carlitz, which says that if q 2 then the group of permutations of F q is generated by the permutations induced by degree one polynomials and x q 2 . Our bijection provides a new perspective from which the two proofs of this result in the literature can be seen to arise naturally, without requiring the clever tricks that previously appeared to be needed in order to discover those proofs.


Essential finite generation of extensions of valuation rings. Given a generically finite local extension of valuation rings V subset W , the question of whether W is the localization of a finitely generated V algebra is significant for approaches to the problem of local uniformization of valuations using ramification theory. Hagen Knaf proposed a characterization of when W is essentially of finite type over V in terms of classical invariants of the extension of associated valuations. Knaf's conjecture has been verified in important special cases by Cutkosky and Novacoski using local uniformization of Abhyankar valuations and resolution of singularities of excellent surfaces in arbitrary characteristic, and by Cutkosky for valuation rings of function fields of characteristic 0 using embedded resolution of singularities. In this paper we prove Knaf's conjecture in full generality.


The similarity index of mathematical and other scientific publications with equations and formulas and the problem of self plagiarism identification. The problems of estimating the similarity index of inhomogeneous scientific publications containing equations and formulas are discussed for the first time. It is shown that the presence of equations and formulas (as well as figures, drawings, and tables) is a complicating factor that significantly complicates the study of such texts. It has been proved that the method for determining the similarity index of publications, based on taking into account individual mathematical symbols and parts of equations and formulas, is ineffective and can lead to erroneous and even completely absurd conclusions. Possibilities of the most popular software systems Antiplagiat and iThenticate, currently used in scientific journals, are investigated for detecting plagiarism and self plagiarism. The results of processing by the iThenticate system of specific examples and specific test problems containing equations and formulas are presented. It has been established that this software system, when analyzing heterogeneous texts, is often unable to distinguish self plagiarism from pseudo self plagiarism, seeming real (but false and imaginary) self plagiarism. A model complex situation is considered, in which the identification of self plagiarism requires the involvement of highly qualified specialists of a narrow profile. Various ways to improve the work of software systems for comparing inhomogeneous texts are proposed. This article will be useful to researchers and university teachers in physics, mathematics, and engineering, programmers dealing with problems in image recognition and research topics of digital image processing, as well as a wide range of readers who are interested in issues of plagiarism and self plagiarism.


Distributed optimal control problems for a class of elliptic hemivariational inequalities with a parameter and its asymptotic behavior. In this paper, we study optimal control problems on the internal energy for a system governed by a class of elliptic boundary hemivariational inequalities with a parameter. The system has been originated by a steady state heat conduction problem with non monotone multivalued subdifferential boundary condition on a portion of the boundary of the domain described by the Clarke generalized gradient of a locally Lipschitz function. We prove an existence result for the optimal controls and we show an asymptotic result for the optimal controls and the system states, when the parameter, like a heat transfer coefficient, tends to infinity on a portion of the boundary.


Differential Algebras on Digraphs and Generalized Path Homology. The theory of path homology for digraphs was developed by Alexander Grigor'yan, Yong Lin, Yuri Muranov, and Shing Tung Yau. In this paper, we generalize the path homology for digraphs. We prove that for any digraph G , any t geq 0 , any 0 leq q leq 2t , and any (2t 1) dimensional element alpha in the differential algebra on the set of the vertices, we always have an ( alpha,q) path homology for G . In particular, if t 0 , then the ( alpha,0) path homology gives the weighted path homology for vertex weighted digraphs.


Online Learning for Unknown Partially Observable MDPs. Solving Partially Observable Markov Decision Processes (POMDPs) is hard. Learning optimal controllers for POMDPs when the model is unknown is harder. Online learning of optimal controllers for unknown POMDPs, which requires efficient learning using regret minimizing algorithms that effectively tradeoff exploration and exploitation, is even harder, and no solution exists currently. In this paper, we consider infinite horizon average cost POMDPs with unknown transition model, though a known observation model. We propose a natural posterior sampling based reinforcement learning algorithm (PSRL POMDP) and show that it achieves a regret bound of O( log T) , where T is the time horizon, when the parameter set is finite. In the general case (continuous parameter set), we show that the algorithm achieves O (T 2 3 ) regret under two technical assumptions. To the best of our knowledge, this is the first online RL algorithm for POMDPs and has sub linear regret.


An enhanced parametric nonlinear reduced order model for imperfect structures using Neumann expansion. We present an enhanced version of the parametric nonlinear reduced order model for shape imperfections in structural dynamics we studied in a previous work 1 . The model is computed intrusively and with no training using information about the nominal geometry of the structure and some user defined displacement fields representing shape defects, i.e. small deviations from the nominal geometry parametrized by their respective amplitudes. The linear superposition of these artificial displacements describe the defected geometry and can be embedded in the strain formulation in such a way that, in the end, nonlinear internal elastic forces can be expressed as a polynomial function of both these defect fields and the actual displacement field. This way, a tensorial representation of the internal forces can be obtained and, owning the reduction in size of the model given by a Galerkin projection, high simulation speed ups can be achieved. We show that by adopting a rigorous deformation framework we are able to achieve better accuracy as compared to the previous work. In particular, exploiting Neumann expansion in the definition of the Green Lagrange strain tensor, we show that our previous model is a lower order approximation with respect to the one we present now. Two numerical examples of a clamped beam and a MEMS gyroscope finally demonstrate the benefits of the method in terms of speed and increased accuracy.


Approximate Nearest Neighbor Search for Line Segments. Approximate nearest neighbor search is a fundamental algorithmic problem that continues to inspire study due its essential role in numerous contexts. In contrast to most prior work, which has focused on point sets, we consider nearest neighbor queries against a set of line segments in mathbb R d , for constant dimension d . Given a set S of n disjoint line segments in mathbb R d and an error parameter varepsilon 0 , the objective is to build a data structure such that for any query point q , it is possible to return a line segment whose Euclidean distance from q is at most (1 varepsilon) times the distance from q to its nearest line segment. We present a data structure for this problem with storage O((n 2 varepsilon d ) log ( Delta varepsilon)) and query time O( log ( max(n, Delta) varepsilon)) , where Delta is the spread of the set of segments S . Our approach is based on a covering of space by anisotropic elements, which align themselves according to the orientations of nearby segments.


Cost Capability Compromises in STEM Instrumentation for Low Voltage Imaging. Low voltage transmission electron microscopy ( 80 kV) has many applications in imaging beam sensitive samples, such as metallic nanoparticles, which may become damaged at higher voltages. To improve resolution, spherical aberration can be corrected for in a Scanning Transmission Electron Microscope (STEM), however chromatic aberration may then dominate, limiting the ultimate resolution of the microscope. Using image simulations, we examine how a chromatic aberration corrector, different objective lenses, and different beam energy spreads each affect the image quality of a gold nanoparticle imaged at low voltages in a spherical aberration corrected STEM. Quantitative analysis of the simulated examples can inform the choice of instrumentation for low voltage imaging. We here demonstrate a methodology whereby the optimum energy spread to operate a specific STEM can be deduced. This methodology can then be adapted to the specific sample and instrument of the reader, enabling them to make an informed economical choice as to what would be most beneficial for their STEM in the cost conscious landscape of scientific infrastructure.


The Temporal Opportunist Self Supervised Multi Frame Monocular Depth. Self supervised monocular depth estimation networks are trained to predict scene depth using nearby frames as a supervision signal during training. However, for many applications, sequence information in the form of video frames is also available at test time. The vast majority of monocular networks do not make use of this extra signal, thus ignoring valuable information that could be used to improve the predicted depth. Those that do, either use computationally expensive test time refinement techniques or off the shelf recurrent networks, which only indirectly make use of the geometric information that is inherently available. We propose ManyDepth, an adaptive approach to dense depth estimation that can make use of sequence information at test time, when it is available. Taking inspiration from multi view stereo, we propose a deep end to end cost volume based approach that is trained using self supervision only. We present a novel consistency loss that encourages the network to ignore the cost volume when it is deemed unreliable, e.g. in the case of moving objects, and an augmentation scheme to cope with static cameras. Our detailed experiments on both KITTI and Cityscapes show that we outperform all published self supervised baselines, including those that use single or multiple frames at test time.


Physics informed neural network applied to surface tension driven liquid film flows. A physics informed neural network (PINN), which has been recently proposed by Raissi et al J. Comp. Phys. 378, pp. 686 707 (2019) , is applied to the partial differential equation (PDE) of liquid film flows. The PDE considered is the time evolution of the thickness distribution h(x,t) owing to the Laplace pressure, which involves 4th order spatial derivative and 4th order nonlinear term. Even for such a PDE, it is confirmed that the PINN can predict the solutions with sufficient accuracy. Nevertheless, some improvements are needed in training convergence and accuracy of the solutions. The precision of floating point numbers is a critical issue for the present PDE. When the calculation is executed with a single precision floating point number, the optimization is terminated due to the loss of significant digits. Calculation of the automatic differentiation (AD) dominates the computational time required for training, and becomes exponentially longer with increasing order of derivatives. By splitting the original 4th order one variable PDE into 2nd order two variable PDEs, the computational time for each training iteration is greatly reduced. The sampling density of training data also significantly affects training convergence. For the problem considered in this study, mproved convergence was obtained by allowing the sampling density of training data to be greater in earlier time ranges, where the rapid flattening of the thickness occurs.


Adjoint based methods to compute higher order topological derivatives with an application to elasticity. The goal of this paper is to give a comprehensive and short review on how to compute the first and second order topological derivative and potentially higher order topological derivatives for PDE constrained shape functionals. We employ the adjoint and averaged adjoint variable within the Lagrangian framework and compare three different adjoint based methods to compute higher order topological derivatives. To illustrate the methodology proposed in this paper, we then apply the methods to a linear elasticity model. We compute the first and second order topological derivative of the linear elasticity model for various shape functionals in dimension 2 and 3 using Amstutz' method, the averaged adjoint method and Delfour's method. In contrast to other contributions regarding this subject, we not only compute the first and second order topological derivative, but additionally give some insight on various methods and compared their applicability and efficiency with respect to the underlying problem formulation.


A use case of Content Delivery Network raw logfile analysis. The growth of video streaming has stretched the Internet to its limitation. In other words, the Internet was originally devised to connect a limited number of computers so that they can share network resources, so the Internet cannot handle a large amount of traffic at a time, which leads to network congestion. To overcome this, CDNs are built on top of the Internet as an overlay to efficiently store and swiftly disseminate contents to users by placing many servers and data centers around the globe. The topic of CDNs has been extensively studied in the last several decades. However, there is still a certain gap between theories in academia and current technologies in industry. In this paper, we take a close look at the design, implementation, solution, and performance of a CDN system by analyzing its raw log files. Specifically, its infrastructure and system design are first presented, and then we conduct a trace based study to understand user access patterns, the sources of requests, system performance, and how such information can be used to improve the whole CDN system.


Comparison results for solutions of Poisson equations with Robin boundary on complete Riemannian manifolds. In this paper, by using Schwarz rearrangement and isoperimetric inequalities, we prove comparison results for the solutions of Poisson equations on complete Riemannian manifolds with Ric geq (n 1) kappa , , kappa geq 0 , which extends the results in cite ANT Talenti a . Furthermore, as applications of our comparison results, we obtain the Saint Venant inequality and Bossel Daners inequality for Robin Laplacian.