Sparse Feature Factorization for Recommender Systems with Knowledge Graphs. Deep Learning and factorization based collaborative filtering recommendation models have undoubtedly dominated the scene of recommender systems in recent years. However, despite their outstanding performance, these methods require a training time proportional to the size of the embeddings and it further increases when also side information is considered for the computation of the recommendation list. In fact, in these cases we have that with a large number of high quality features, the resulting models are more complex and difficult to train. This paper addresses this problem by presenting KGFlex a sparse factorization approach that grants an even greater degree of expressiveness. To achieve this result, KGFlex analyzes the historical data to understand the dimensions the user decisions depend on (e.g., movie direction, musical genre, nationality of book writer). KGFlex represents each item feature as an embedding and it models user item interactions as a factorized entropy driven combination of the item attributes relevant to the user. KGFlex facilitates the training process by letting users update only those relevant features on which they base their decisions. In other words, the user item prediction is mediated by the user's personal view that considers only relevant features. An extensive experimental evaluation shows the approach's effectiveness, considering the recommendation results' accuracy, diversity, and induced bias. The public implementation of KGFlex is available at https split.to kgflex.


A fast and light tool for partially coherent beamline simulations in fourth generation storage rings based on coherent mode decomposition. A new algorithm to perform coherent mode decomposition of the undulator radiation is proposed. It is based in separating the horizontal and vertical directions, reducing the problem by working with one dimension wavefronts. The validity conditions of this approximation are discussed. Simulations require low computer resources, and run interactively in a laptop. We study the focusing with lenses of the radiation emitted by an undulator in a fourth generation storage ring (EBS ESRF). Results are compared against multiple optics packages implementing a variety of methods for dealing with partial coherence full 2D coherent mode decomposition, Monte Carlo combination of wavefronts from electrons entering the undulator with different initial conditions, and hybrid ray tracing correcting geometrical optics with wave optics.


Binet's factorial series and extensions to Laplace transforms. We investigate a generalization of Binet's factorial series in the parameter alpha mu left( z right) sum m 1 infty frac b m left( alpha right) prod k 0 m 1 (z alpha k) for the Binet function mu left( z right) log Gamma left( z right) left( z frac 1 2 right) log z z frac 1 2 log left( 2 pi right) After a brief review of the Binet function mu left( z right) , several properties of the Binet polynomials b m left( alpha right) are presented. We compute the corresponding factorial series for the derivatives of the Binet function and apply those series to the digamma and polygamma functions. We compare Binet's generalized factorial series with Stirling's emph asymptotic expansion and demonstrate by a numerical example that, with a same number of terms evaluated, the Binet generalized factorial series with an optimized value of alpha can beat the best possible accuracy of Stirling's expansion. Finally, we extend Binet's method to factorial series of Laplace transforms.


Applied Exoskeleton Technology A Comprehensive Review of Physical and Cognitive Human Robot Interfac. Exoskeletons and orthoses are wearable mobile systems providing mechanical benefits to the users. Despite significant improvements in the last decades, the technology is not fully mature to be adopted for strenuous and non programmed tasks. To accommodate this insufficiency, different aspects of this technology need to be analysed and improved. Numerous studies have been trying to address some aspects of exoskeletons, e.g. mechanism design, intent prediction, and control scheme. However, most works have focused on a specific element of design or application without providing a comprehensive review framework. This study aims to analyse and survey the contributing aspects to the improvement and broad adoption of this technology. To address this, after introducing assistive devices and exoskeletons, the main design criteria will be investigated from a physical Human Robot Interface (HRI) perspective. The study will be further developed by outlining several examples of known assistive devices in different categories. In order to establish an intelligent HRI strategy and enabling intuitive control for users, cognitive HRI will be investigated. Various approaches to this strategy will be reviewed, and a model for intent prediction will be proposed. This model is utilised to predict the gate phase from a single Electromyography (EMG) channel input. The outcomes of modelling show the potential use of single channel input in low power assistive devices. Furthermore, the proposed model can provide redundancy in devices with a complex control strategy.


Deep grey matter quantitative susceptibility mapping from small spatial coverages using deep learning. Introduction Quantitative Susceptibility Mapping (QSM) is generally acquired with full brain coverage, even though many QSM brain iron studies focus on the deep grey matter (DGM) region only. Reducing the spatial coverage to the DGM vicinity can substantially shorten the scan time or enhance the spatial resolution without increasing scan time however, this may lead to significant DGM susceptibility underestimation. Method A recently proposed deep learning based QSM method, namely xQSM, is investigated to assess the accuracy of dipole inversion on reduced brain coverages. Pre processed magnetic field maps are extended symmetrically from the centre of globus pallidus in the coronal plane to simulate QSM acquisitions of difference spatial coverages.Results The proposed xQSM network led to the lowest DGM contrast lost with the smallest susceptibility variation range across all spatial coverages. For the digital brain phantom simulation, xQSM improved the DGM susceptibility underestimation more than 20 in small spatial coverages. For the in vivo acquisition, less than 5 DGM susceptibility error was achieved in 48 mm axial slabs using the xQSM network, while a minimum of 112 mm coverage was required for conventional methods. It is also shown that the background field removal process performed worse in reduced brain coverages, which further deteriorated the subsequent dipole inversion. Conclusion The recently proposed deep learning based xQSM method significantly improves the accuracy of DGM QSM from small spatial coverages as compared with conventional QSM algorithms, which can shorten DGM QSM acquisition time substantially.


Comparative analysis of quality of service scheduling classes in mobile ad hoc networks. Quality of Service (QoS) is now regarded as a requirement for all networks in managing resources like bandwidth and avoidance of network impairments like packet loss, jitter, and delay. Media transfer or streaming would be virtually impossible if QoS parameters were not used even if the streaming protocols were perfectly designed. QoS Scheduling classes help in network traffic optimization and the priority management of packets. This paper presents an analysis of QoS scheduling classes using video traffic in a MANET. The main objective was to identify a scheduling class that provides better QoS for video streaming. A simulation was conducted using NetSim and results were analyzed according to throughput, jitter, and delay. The overall results showed that extended real time Polling Service (ertPS) outperformed the other classes. ertPS has hybrid features of both real time Polling Service (rtPS) and Unsolicited Grant Service(UGS) hence the enhanced performance. It is recommended that ertPS scheduling class should be used in MANET where QoS consideration is utmost particularly in multimedia streaming applications.


Strong Morita equivalences for completely positive linear maps and GNS C correspondences. We will consider the set of all completely positive linear maps from a unital C algebra to the C algebra of all (bounded) adjointable right Hilbert C module maps, which are automatically bounded, on a right Hilbert C module and we will introduce strong Morita equivalence for elements in this set. In this paper, we will give the following result If two classes of two unital C algebras are strongly Morita equivalent, respectively, then we can construct a bijective correspondence between two sets of all strong Morita equivalence classes of completely positive linear maps given as above. Furthermore, we will discuss the relation between strong Morita equivalence for completely positive linear maps and strong Morita equivalence for GNS C correspondences.


Top N Recommendation with Counterfactual User Preference Simulation. Top N recommendation, which aims to learn user ranking based preference, has long been a fundamental problem in a wide range of applications. Traditional models usually motivate themselves by designing complex or tailored architectures based on different assumptions. However, the training data of recommender system can be extremely sparse and imbalanced, which poses great challenges for boosting the recommendation performance. To alleviate this problem, in this paper, we propose to reformulate the recommendation task within the causal inference framework, which enables us to counterfactually simulate user ranking based preferences to handle the data scarce problem. The core of our model lies in the counterfactual question "what would be the user's decision if the recommended items had been different?". To answer this question, we firstly formulate the recommendation process with a series of structural equation models (SEMs), whose parameters are optimized based on the observed data. Then, we actively indicate many recommendation lists (called intervention in the causal inference terminology) which are not recorded in the dataset, and simulate user feedback according to the learned SEMs for generating new training samples. Instead of randomly intervening on the recommendation list, we design a learning based method to discover more informative training samples. Considering that the learned SEMs can be not perfect, we, at last, theoretically analyze the relation between the number of generated samples and the model prediction error, based on which a heuristic method is designed to control the negative effect brought by the prediction error. Extensive experiments are conducted based on both synthetic and real world datasets to demonstrate the effectiveness of our framework.


Maximum principle for optimal control of stochastic evolution equations with recursive utilities. We consider the optimal control problem of stochastic evolution equations in a Hilbert space under a recursive utility, which is described as the solution of a backward stochastic differential equation (BSDE). A very general maximum principle is given for the optimal control, allowing the control domain not to be convex and the generator of the BSDE to vary with the second unknown variable z . The associated second order adjoint process is characterized as a unique solution of a conditionally expected operator valued backward stochastic integral equation.


Optical trapping with structured light. Optical trapping describes the interaction between light and matter to manipulate micro objects through momentum transfer. In the case of 3D trapping with a single beam, this is termed optical tweezers. Optical tweezers are a powerful and non invasive tool for manipulating small objects, which have become indispensable in many fields, including physics, biology, soft condensed matter, amongst others. In the early days, optical trapping were typically used with a single Gaussian beam. In recent years, we have witnessed the rapid progress in the use of structured light beams with customized phase, amplitude and polarization in optical trapping. Unusual beam properties, such as phase singularities on axis, propagation invariant nature, have opened up novel capabilities to the study of micromanipulation in liquid, air and vacuum. In this review, we summarize the recent advances in the field of optical trapping using structured light beams.


Optimal order error estimates for Euler discretization of high index saddle dynamics. High index saddle dynamics provides an effective means to compute the any index saddle points and construct the solution landscape. In this paper we prove the optimal order error estimates for Euler discretization of high index saddle dynamics with respect to the time step size, which remains untreated in the literature. We overcome the main difficulties that lie in the strong nonlinearity of the saddle dynamics and the orthonormalization procedure in the numerical scheme that is uncommon in standard discretization of differential equations. The derived methods are further extended to study the generalized high index saddle dynamics for non gradient systems and provide theoretical support for the accuracy of numerical implementations.


Parameterized (Modular) Counting and Cayley Graph Expanders. We study the problem mathrm EdgeSub ( Phi) of counting k edge subgraphs satisfying a given graph property Phi in a large host graph G . Building upon the breakthrough result of Curticapean, Dell and Marx (STOC 17), we express the number of such subgraphs as a finite linear combination of graph homomorphism counts and derive the complexity of computing this number by studying its coefficients. Our approach relies on novel constructions of low degree Cayley graph expanders of p groups, which might be of independent interest. The properties of those expanders allow us to analyse the coefficients in the aforementioned linear combinations over the field mathbb F p which gives us significantly more control over the cancellation behaviour of the coefficients. Our main result is an exhaustive and fine grained complexity classification of mathrm EdgeSub ( Phi) for minor closed properties Phi , closing the missing gap in previous work by Roth, Schmitt and Wellnitz (ICALP 21). Additionally, we observe that our methods also apply to modular counting. Among others, we investigate the problems of modular counting of paths, cycles, forests and matroid bases. In the course of our investigations we also provide an exhaustive parameterized complexity classification for the problem of counting graph homomorphisms modulo a prime p .


Attributing Fair Decisions with Attention Interventions. The widespread use of Artificial Intelligence (AI) in consequential domains, such as healthcare and parole decision making systems, has drawn intense scrutiny on the fairness of these methods. However, ensuring fairness is often insufficient as the rationale for a contentious decision needs to be audited, understood, and defended. We propose that the attention mechanism can be used to ensure fair outcomes while simultaneously providing feature attributions to account for how a decision was made. Toward this goal, we design an attention based model that can be leveraged as an attribution framework. It can identify features responsible for both performance and fairness of the model through attention interventions and attention weight manipulation. Using this attribution framework, we then design a post processing bias mitigation strategy and compare it with a suite of baselines. We demonstrate the versatility of our approach by conducting experiments on two distinct data types, tabular and textual.


In plane subwavelength near field optical capsule for lab on a chip optical nano tweezer. In this letter, we propose a new proof of concept of optical nano tweezer on the basis of a pair of dielectric rectangular rods capable of generating a novel class of controlled finite volume near field light capsules. The finite difference time domain simulations of light spatial structure and optical trapping forces of the gold nanoparticle immersed in water demonstrate the physical concept of an in plane subwavelength optical capsule, integrated with the microfluidic mesoscale device. It is shown that refractive index and distance between dielectric rectangular rods can control the shape and axial position of the optical capsule. Such an in plane wavelength scaled structure provides a new path for manipulating absorbing nano particles including bio particles in a compact planar architecture and should thus open promising perspectives in lab on a chip domains.


On pseudoholomorphic map between almost Hermitian manifolds. In this paper, we use the canonical connection instead of Levi Civita connection to study the smooth maps between almost Hermitian manifolds, especially, the pseudoholomorphic ones. By using the Bochner formulas, we obtian the C 2 estimate of canonical second fundamental form, Liouville type theorems of pseudoholomorphic maps, pseudoholomorphicity of pluriharmonic maps, and Simons integral inequality of pseudoholomorphic isometric immersion.


FedMe Federated Learning via Model Exchange. Federated learning is a distributed machine learning method in which a single server and multiple clients collaboratively build machine learning models without sharing datasets on clients. Numerous methods have been proposed to cope with the data heterogeneity issue in federated learning. Existing solutions require a model architecture tuned by the central server, yet a major technical challenge is that it is difficult to tune the model architecture due to the absence of local data on the central server. In this paper, we propose Federated learning via Model exchange (FedMe), which personalizes models with automatic model architecture tuning during the learning process. The novelty of FedMe lies in its learning process clients exchange their models for model architecture tuning and model training. First, to optimize the model architectures for local data, clients tune their own personalized models by comparing to exchanged models and picking the one that yields the best performance. Second, clients train both personalized models and exchanged models by using deep mutual learning, in spite of different model architectures across the clients. We perform experiments on three real datasets and show that FedMe outperforms state of the art federated learning methods while tuning model architectures automatically.


Gaussian Mixture Variational Autoencoder with Contrastive Learning for Multi Label Classification. Multi label classification (MLC) is a prediction task where each sample can have more than one label. We propose a novel contrastive learning boosted multi label prediction model based on a Gaussian mixture variational autoencoder (C GMVAE), which learns a multimodal prior space and employs a contrastive loss. Many existing methods introduce extra complex neural modules to capture the label correlations, in addition to the prediction modules. We found that by using contrastive learning in the supervised setting, we can exploit label information effectively, and learn meaningful feature and label embeddings capturing both the label correlations and predictive power, without extra neural modules. Our method also adopts the idea of learning and aligning latent spaces for both features and labels. C GMVAE imposes a Gaussian mixture structure on the latent space, to alleviate posterior collapse and over regularization issues, in contrast to previous works based on a unimodal prior. C GMVAE outperforms existing methods on multiple public datasets and can often match other models' full performance with only 50 of the training data. Furthermore, we show that the learnt embeddings provide insights into the interpretation of label label interactions.


A Projection Stable Grammatical Model for the Distributed Execution of Administrative Processes with Emphasis on Actors' Views. During the last two decades, the decentralized execution of business processes has been one of the main research topics in Business Process Management. Several models (languages) for processes' specification in order to facilitate their distributed execution, have been proposed. LSAWfP is among the most recent in this area it helps to specify administrative processes with grammatical models indicating, in addition to their fundamental elements, the permissions (reading, writing and execution) of each actor in relation to each of their tasks. In this paper, we present a model for a completely decentralized and artifact centric execution of administrative processes specified using LSAWfP. The presented model puts particular emphasis on actors' views it then allows the confidential execution of certain tasks by ensuring that, each actor potentially has only a partial perception of the processes' global execution states. The model thus solves a very important problem in business process execution, which is often sidelined in existing approaches. To accomplish this, the model rely on three projection algorithms allowing to partially replicate the processes' global execution states at a given moment, to consistently update the obtained partial states and to deduce new coherent global states. The proposal of these three algorithms, the proof of underlying mathematical tools' stability and a proposal of their implementation, are this paper's main contributions.


1 GHz dual comb spectrometer for fast and broadband measurements. Dual frequency comb spectroscopy permits broadband precision spectroscopic measurements with short acquisition time. A dramatic improvement of the maximal spectral bandwidth and the minimal measurement time can be expected when the lasers' pulse repetition rate is increased, owing to a quadratic dependence (Nyquist criterion). Here, we demonstrate a dual comb system operating at a high repetition rate of 1 GHz based on mature, digitally controlled, low noise mode locked lasers. Compared to conventional lower repetition rate ( sim 100 MHz) oscillators, this represents a 100 fold improvement in terms of the Nyquist criterion, while still providing adequate spectral sampling even for trace gas absorption fingerprints. Two spectroscopy experiments are performed with acquisition parameters not attainable in a 100 MHz system detection of water vapor absorption around 1375 nm, demonstrating the potential for fast and ambiguity free broadband operation, and real time acquisition of narrow gas absorption features across a spectral span of 0.6 THz (600 comb lines) in only 5 mu s. Additionally, we show high mutual coherence of the lasers below the Hz level, generating opportunities for broadband spectroscopy even with low bandwidth detectors such as mid infrared, imaging or photo acoustic detectors.


Composition of analytic paraproducts. For a fixed analytic function g on the unit disc mathbb D , we consider the analytic paraproducts induced by g , which are defined by T gf(z) int 0 z f( zeta)g'( zeta) ,d zeta , S gf(z) int 0 z f'( zeta)g( zeta) ,d zeta , and M gf(z) f(z)g(z) . The boundedness of these operators on various spaces of analytic functions on mathbb D is well understood. The original motivation for this work is to understand the boundedness of compositions of two of these operators, for example T g 2, ,T gS g, , M gT g , etc. Our methods yield a characterization of the boundedness of a large class of operators contained in the algebra generated by these analytic paraproducts acting on the classical weighted Bergman and Hardy spaces in terms of the symbol g . In some cases it turns out that this property is not affected by cancellation, while in others it requires stronger and more subtle restrictions on the oscillation of the symbol g than the case of a single paraproduct.


Disturbing Target Values for Neural Network Regularization. Diverse regularization techniques have been developed such as L2 regularization, Dropout, DisturbLabel (DL) to prevent overfitting. DL, a newcomer on the scene, regularizes the loss layer by flipping a small share of the target labels at random and training the neural network on this distorted data so as to not learn the training data. It is observed that high confidence labels during training cause the overfitting problem and DL selects disturb labels at random regardless of the confidence of labels. To solve this shortcoming of DL, we propose Directional DisturbLabel (DDL) a novel regularization technique that makes use of the class probabilities to infer the confident labels and using these labels to regularize the model. This active regularization makes use of the model behavior during training to regularize it in a more directed manner. To address regression problems, we also propose DisturbValue (DV), and DisturbError (DE). DE uses only predefined confident labels to disturb target values. DV injects noise into a portion of target values at random similar to DL. In this paper, 6 and 8 datasets are used to validate the robustness of our methods in classification and regression tasks respectively. Finally, we demonstrate that our methods are either comparable to or outperform DisturbLabel, L2 regularization, and Dropout. Also, we achieve the best performance in more than half the datasets by combining our methods with either L2 regularization or Dropout.


Supporting search engines with knowledge and context. Search engines leverage knowledge to improve information access. In order to effectively leverage knowledge, search engines should account for context, i.e., information about the user and query. In this thesis, we aim to support search engines in leveraging knowledge while accounting for context. In the first part of this thesis, we study how to make structured knowledge more accessible to the user when the search engine proactively provides such knowledge as context to enrich search results. As a first task, we study how to retrieve descriptions of knowledge facts from a text corpus. Next, we study how to automatically generate knowledge fact descriptions. And finally, we study how to contextualize knowledge facts, that is, to automatically find facts related to a query fact. In the second part of this thesis, we study how to improve interactive knowledge gathering. We focus on conversational search, where the user interacts with the search engine to gather knowledge over large unstructured knowledge repositories. We focus on multi turn passage retrieval as an instance of conversational search. We propose to model query resolution as a term classification task and propose a method to address it. In the final part of this thesis, we focus on search engine support for professional writers in the news domain. We study how to support such writers create event narratives by exploring knowledge from a corpus of news articles. We propose a dataset construction procedure for this task that relies on existing news articles to simulate incomplete narratives and relevant articles. We study the performance of multiple rankers, lexical and semantic, and provide insights into the characteristics of this task.


Diversified Multiscale Graph Learning with Graph Self Correction. Though the multiscale graph learning techniques have enabled advanced feature extraction frameworks, the classic ensemble strategy may show inferior performance while encountering the high homogeneity of the learnt representation, which is caused by the nature of existing graph pooling methods. To cope with this issue, we propose a diversified multiscale graph learning model equipped with two core ingredients a graph self correction (GSC) mechanism to generate informative embedded graphs, and a diversity boosting regularizer (DBR) to achieve a comprehensive characterization of the input graph. The proposed GSC mechanism compensates the pooled graph with the lost information during the graph pooling process by feeding back the estimated residual graph, which serves as a plug in component for popular graph pooling methods. Meanwhile, pooling methods enhanced with the GSC procedure encourage the discrepancy of node embeddings, and thus it contributes to the success of ensemble learning strategy. The proposed DBR instead enhances the ensemble diversity at the graph level embeddings by leveraging the interaction among individual classifiers. Extensive experiments on popular graph classification benchmarks show that the proposed GSC mechanism leads to significant improvements over state of the art graph pooling methods. Moreover, the ensemble multiscale graph learning models achieve superior enhancement by combining both GSC and DBR.


Finding Triangles or Independent Sets. (I) We revisit the algorithmic problem of finding all triangles in a graph G (V,E) with n vertices and m edges. According to a result of Chiba and Nishizeki (1985), this task can be achieved by a combinatorial algorithm running in O(m 3 2 ) time. We derive this worst case bound from first principles and with a simple proof. We then provide a combinatorial algorithm for finding all triangles in a graph and show that is amenable to the same running time analysis. We also show that the running time of such an algorithm cannot be improved in the worst case and for the entire range of parameters m and n . Our arguments are extended to the problem of finding all small complete subgraphs of a given fixed size. (II) Given a graph G (V,E) with n vertices and m edges, we present a randomized algorithm that computes a (1 pm varepsilon) approximation of the number of triangles in G and finds a witness with high probability in O left( n omega(1 delta) right) or O left( left( m n 2 delta right) frac 2 omega omega 1 right) expected time, provided G has a suitable superlinear number of edges and triangles (where omega 2.376 is the exponent of matrix multiplication, and varepsilon leq 0.5 , delta leq 0.25 are positive constants). This limits the range of a conjecture of P v a tra c s cu (2010) regarding the triangle detection problem. (III) We present an algorithm which given a graph G (V,E) performs one of the following tasks in O(m n) (i.e., linear) time (i) compute a Omega(1 sqrt n ) approximation of a maximum independent set in G (a well known NP hard problem), or (ii) find a triangle in G . The run time is faster than that for any previous method for each of these tasks. A series of trade off results is also available.


High dimensional robustly singular maps that are C 1 robustly transitive. We improve previous results by exhibiting a construction that contains all known examples. A suficient condition for the existence of robustly transitive maps displaying singularities on a certain large class of compact manifolds is given.


Revisiting Sanity Checks for Saliency Maps. Saliency methods are a popular approach for model debugging and explainability. However, in the absence of ground truth data for what the correct maps should be, evaluating and comparing different approaches remains a long standing challenge. The sanity checks methodology of Adebayo et al Neurips 2018 has sought to address this challenge. They argue that some popular saliency methods should not be used for explainability purposes since the maps they produce are not sensitive to the underlying model that is to be explained. Through a causal re framing of their objective, we argue that their empirical evaluation does not fully establish these conclusions, due to a form of confounding introduced by the tasks they evaluate on. Through various experiments on simple custom tasks we demonstrate that some of their conclusions may indeed be artifacts of the tasks more than a criticism of the saliency methods themselves. More broadly, our work challenges the utility of the sanity check methodology, and further highlights that saliency map evaluation beyond ad hoc visual examination remains a fundamental challenge.


Sequence based deep learning antibody design for in silico antibody affinity maturation. Antibody therapeutics has been extensively studied in drug discovery and development within the past decades. One increasingly popular focus in the antibody discovery pipeline is the optimization step for therapeutic leads. Both traditional methods and in silico approaches aim to generate candidates with high binding affinity against specific target antigens. Traditional in vitro approaches use hybridoma or phage display for candidate selection, and surface plasmon resonance (SPR) for evaluation, while in silico computational approaches aim to reduce the high cost and improve efficiency by incorporating mathematical algorithms and computational processing power in the design process. In the present study, we investigated different graph based designs for depicting antibody antigen interactions in terms of antibody affinity prediction using deep learning techniques. While other in silico computations require experimentally determined crystal structures, our study took interest in the capability of sequence based models for in silico antibody maturation. Our preliminary studies achieved satisfying prediction accuracy on binding affinities comparing to conventional approaches and other deep learning approaches. To further study the antibody antigen binding specificity, and to simulate the optimization process in real world scenario, we introduced pairwise prediction strategy. We performed analysis based on both baseline and pairwise prediction results. The resulting prediction and efficiency prove the feasibility and computational efficiency of sequence based method to be adapted as a scalable industry practice.


ProtGNN Towards Self Explaining Graph Neural Networks. Despite the recent progress in Graph Neural Networks (GNNs), it remains challenging to explain the predictions made by GNNs. Existing explanation methods mainly focus on post hoc explanations where another explanatory model is employed to provide explanations for a trained GNN. The fact that post hoc methods fail to reveal the original reasoning process of GNNs raises the need of building GNNs with built in interpretability. In this work, we propose Prototype Graph Neural Network (ProtGNN), which combines prototype learning with GNNs and provides a new perspective on the explanations of GNNs. In ProtGNN, the explanations are naturally derived from the case based reasoning process and are actually used during classification. The prediction of ProtGNN is obtained by comparing the inputs to a few learned prototypes in the latent space. Furthermore, for better interpretability and higher efficiency, a novel conditional subgraph sampling module is incorporated to indicate which part of the input graph is most similar to each prototype in ProtGNN . Finally, we evaluate our method on a wide range of datasets and perform concrete case studies. Extensive results show that ProtGNN and ProtGNN can provide inherent interpretability while achieving accuracy on par with the non interpretable counterparts.


Higher Auslander correspondence for exact categories. Inspired by the recent work of Henrard, Kvamme and van Roosmalen 17 , we prove a categorified version of higher Auslander correspondence in the context of exact categories. We define n Auslander exact categories and show that there is a bijection between the equivalence classes of n cluster tilting subcategories of exact categories and the equivalence classes of n Auslander exact categories.


Power dependent Reflective Metasurface with Self induced Bandgap. A metallic ring based, diode integrated, low profile, power dependent, reflective metasurface working from 3 GHz to 3.6 GHz is proposed in this letter. Unlike the previous study which shifts a band up and down to change the impedance of the surface, the triggering of the diodes directly transforms the structure from a surface wave supportive state to a self induced bandgap topology if exposed to high power RF illumination. We demonstrate the concept by conducting the EM circuit co simulation and measurements for a 6 by 8 unit 2D prototype. Near field scan experiments verify that the proposed topology works in two distinct states, the ON and OFF state, and high power measurements prove that the reflection varies with the incident signal power. The highest 10 dB decrement in transmission occurs at 3.3 GHz with 52 dBm illumination. This structure can be used to protect sensitive devices from large signals while otherwise supporting a communication channel for small signals.


An artificial intelligence natural language processing pipeline for information extraction in neuroradiology. The use of electronic health records in medical research is difficult because of the unstructured format. Extracting information within reports and summarising patient presentations in a way amenable to downstream analysis would be enormously beneficial for operational and clinical research. In this work we present a natural language processing pipeline for information extraction of radiological reports in neurology. Our pipeline uses a hybrid sequence of rule based and artificial intelligence models to accurately extract and summarise neurological reports. We train and evaluate a custom language model on a corpus of 150000 radiological reports from National Hospital for Neurology and Neurosurgery, London MRI imaging. We also present results for standard NLP tasks on domain specific neuroradiology datasets. We show our pipeline, called neuroNLP', can reliably extract clinically relevant information from these reports, enabling downstream modelling of reports and associated imaging on a heretofore unprecedented scale.


Optimizing Pulsed Laser Ablation Production of AlCl Molecules for Laser Cooling. Aluminum monochloride (AlCl) has been proposed as a promising candidate for laser cooling to ultracold temperatures, and recent spectroscopy results support this prediction. It is challenging to produce large numbers of AlCl molecules because it is a highly reactive open shell molecule and must be generated in situ. Here we show that pulsed laser ablation of stable, non toxic mixtures of Al with an alkali or alkaline earth chlorides, denoted XCln, can provide a robust and reliable source of cold AlCl molecules. Both the chemical identity of XCln and the Al XCln molar ratio are varied, and the yield of AlCl is monitored using absorption spectroscopy in a cryogenic gas. For KCl, the production of Al and K atoms was also monitored. We model the AlCl production in the limits of nonequilibrium recombination dominated by first encounter events. The non equilibrium model is in agreement with the data and also reproduces the observed trend with different XCln precursors. We find that AlCl production is limited by the solid state densities of Al and Cl atoms and the recondensation of Al atoms in the ablation plume. We suggest future directions for optimizing the production of cold AlCl molecules using laser ablation.


Transitioning from Blackboard to Moodle amidst Natural Disaster Faculty and Students Perceptions. Higher educational institutions continuously look for ways to improve the quality of their eLearning services and adapt learning solutions to suit the needs of the institution. During the 2016 Fall Semester, a university located in the Southern part of United States decided to transition from the Blackboard learning management system (LMS) to the Moodle learning management system. Typically such a transition presents a huge challenge for the University staff, faculty, and students. Additionally, on August 2016, what CNN themedthe worst natural disaster, to strike the United States since Hurricane Sandy, occurred in Louisiana during the transition. This led to massive disruptions in activities throughout the state. This paper examines the perceptions of both faculty and student on the transition from one LMS to another and also what impact, if any, the natural disaster had on the process. Faculty and students were surveyed to gain understanding of how they perceived the transitioning process, their perception of both systems, their preferences, and why. Furthermore, we identified issues peculiar to transitioning during a natural disaster. The results of this study can be used to anticipate issues that may be associated with transitioning from one LMS to the other and issues peculiar to transitioning amidst a natural disaster. It can also be used to identify areas for improvement.


On the Curvature of Metric Triples. In this article, we introduce a notion of curvature, denoted by k X(T) , for a metric triple T inside a (possibly discrete) metric space X . Such a notion enables us to consider curvature information of any metric space, including discrete metric spaces such as those generated by scientific data. To define the notion, we employ the information consisting of side lengths of the triple as well as the minimum total distance from vertices of the triple to points of the metric space. Those information provides us a unique number k X(T) such that the triple T can be isometrically embedded into the model space M k 2 up to k le k X(T) . The value k X(T) agrees with the usual curvature when X is a convex subset of a model space. We also show that the curvature k X(T) of any metric triple T inside a CAT(k) space is bounded above by k .


Detecting Adversarial Examples from Sensitivity Inconsistency of Spatial Transform Domain. Deep neural networks (DNNs) have been shown to be vulnerable against adversarial examples (AEs), which are maliciously designed to cause dramatic model output errors. In this work, we reveal that normal examples (NEs) are insensitive to the fluctuations occurring at the highly curved region of the decision boundary, while AEs typically designed over one single domain (mostly spatial domain) exhibit exorbitant sensitivity on such fluctuations. This phenomenon motivates us to design another classifier (called dual classifier) with transformed decision boundary, which can be collaboratively used with the original classifier (called primal classifier) to detect AEs, by virtue of the sensitivity inconsistency. When comparing with the state of the art algorithms based on Local Intrinsic Dimensionality (LID), Mahalanobis Distance (MD), and Feature Squeezing (FS), our proposed Sensitivity Inconsistency Detector (SID) achieves improved AE detection performance and superior generalization capabilities, especially in the challenging cases where the adversarial perturbation levels are small. Intensive experimental results on ResNet and VGG validate the superiority of the proposed SID.


Low complexity Rank Efficient Tensor Completion For Prediction And Online Wireless Edge Caching. Wireless edge caching is a popular strategy to avoid backhaul congestion in the next generation networks, where the content is cached in advance at base stations to serve redundant requests during peak congestion periods. In the edge caching data, the missing observations are inevitable due to dynamic selective popularity. Among the completion methods, the tensor based models have been shown to be the most advantageous for missing data imputation. Also, since the observations are correlated across time, files, and base stations, in this paper, we formulate the cooperative caching with recommendations as a fourth order tensor completion and prediction problem. Since the content library can be large leading to a large dimension tensor, we modify the latent norm based Frank Wolfe (FW) algorithm with towards a much lower time complexity using multi rank updates, rather than rank 1 updates in literature. This significantly lower time computational overhead leads in developing an online caching algorithm. With MovieLens dataset, simulations show lower reconstruction errors for the proposed algorithm as compared to that of the recent FW algorithm, albeit with lower computation overhead. It is also demonstrated that the completed tensor improves normalized cache hit rates for linear prediction schemes.


Towards Universal GAN Image Detection. The ever higher quality and wide diffusion of fake images have spawn a quest for reliable forensic tools. Many GAN image detectors have been proposed, recently. In real world scenarios, however, most of them show limited robustness and generalization ability. Moreover, they often rely on side information not available at test time, that is, they are not universal. We investigate these problems and propose a new GAN image detector based on a limited sub sampling architecture and a suitable contrastive learning paradigm. Experiments carried out in challenging conditions prove the proposed method to be a first step towards universal GAN image detection, ensuring also good robustness to common image impairments, and good generalization to unseen architectures.


Using Hyperbolic Geometry for FG NET over Distantly Supervised data. Fine Grained Named Entity Typing ( FGNET ) classifies an entity mention into a fine range of entity types. A large number of entity types make it difficult to manually label the training data, thus distant supervision is used to automatically acquire the training data. Distant supervision incurs a lot of training noise which hinders the performance improvement of the FG NET systems. In this paper, we propose to use hyperbolic geometry for FG NET with the hope that it can help overcoming the noise incurred by distant supervision.


Exploiting Record Similarity for Practical Vertical Federated Learning. As the privacy of machine learning has drawn increasing attention, federated learning is introduced to enable collaborative learning without revealing raw data. Notably, textit vertical federated learning (VFL), where parties share the same set of samples but only hold partial features, has a wide range of real world applications. However, existing studies in VFL rarely study the record linkage'' process. They either design algorithms assuming the data from different parties have been linked or use simple linkage methods like exact linkage or top1 linkage. These approaches are unsuitable for many applications, such as the GPS location and noisy titles requiring fuzzy matching. In this paper, we design a novel similarity based VFL framework, FedSim, which is suitable for more real world applications and achieves higher performance on traditional VFL tasks. Moreover, we theoretically analyze the privacy risk caused by sharing similarities. Our experiments on three synthetic datasets and five real world datasets with various similarity metrics show that FedSim consistently outperforms other state of the art baselines.


A Single Layer Dual Mesh Boundary Element Method for Multiscale Electromagnetic Modeling of Penetrable Objects in Layered Media. A surface integral representation of Maxwell's equations allows the efficient electromagnetic (EM) modeling of three dimensional structures with a two dimensional discretization, via the boundary element method (BEM). However, existing BEM formulations either lead to a poorly conditioned system matrix for multiscale problems, or are computationally expensive for objects embedded in layered substrates. This article presents a new BEM formulation which leverages the surface equivalence principle and Buffa Christiansen basis functions defined on a dual mesh, to obtain a well conditioned system matrix suitable for multiscale EM modeling. Unlike existing methods involving dual meshes, the proposed formulation avoids the double layer potential operator for the surrounding medium, which may be a stratified substrate requiring the use of an advanced Green's function. This feature greatly alleviates the computational expense associated with the use of Buffa Christiansen functions. Numerical examples drawn from several applications, including remote sensing, chip level EM analysis, and metasurface modeling, demonstrate speed ups ranging from 3x to 7x compared to state of the art formulations.


Model selection of chaotic systems from data with hidden variables using sparse data assimilation. Many natural systems exhibit chaotic behaviour such as the weather, hydrology, neuroscience and population dynamics. Although many chaotic systems can be described by relatively simple dynamical equations, characterizing these systems can be challenging, due to sensitivity to initial conditions and difficulties in differentiating chaotic behavior from noise. Ideally, one wishes to find a parsimonious set of equations that describe a dynamical system. However, model selection is more challenging when only a subset of the variables are experimentally accessible. Manifold learning methods using time delay embeddings can successfully reconstruct the underlying structure of the system from data with hidden variables, but not the equations. Recent work in sparse optimization based model selection has enabled model discovery given a library of possible terms, but regression based methods require measurements of all state variables. We present a method combining variational annealing a technique previously used for parameter estimation in chaotic systems with hidden variables with sparse optimization methods to perform model identification for chaotic systems with unmeasured variables. We applied the method to experimental data from an electrical circuit with Lorenz system like behavior to successfully recover the circuit equations with two measured and one hidden variable. We discuss the robustness of our method to varying noise and manifold sampling using ground truth time series simulated from the classic Lorenz system.


HL ideals and Sacks indestructible ultrafilters. We study ultrafilters on countable sets and reaping families which are indestructible by Sacks forcing. We deal with the combinatorial characterization of such families and we prove that every reaping family of size smaller than the continuum is Sacks indestructible. We prove that complements of many definable ideals are Sacks reaping indestructible, with one notable exception, the complement of the ideal mathcal Z of sets of asymptotic density zero. We investigate the existence of Sacks indestructible ultrafilters and prove that every Sacks indestructible ultrafilter is a mathcal Z ultrafilter.


Experimental demonstration of the short bunch extraction by bunch rotation in a high intensity rapid cycling proton synchrotron. Short bunch proton beams are of great significance for the applications of white neutron beams and muon beams. The accelerator complex of China Spallation Neutron Source (CSNS) was designed to support the applications mainly based on neutron scattering techniques where the proton pulse length is not very sensitive. Some theoretical and experimental studies have been performed to see if one can extract a short bunch proton beam by bunch rotation from the rapid cycling synchrotron (RCS) at CSNS. The experimental results at RCS have evidently displayed the bunch lengthening and rotation process, which demonstrates the effectiveness of this method even with a very short available time for the RF gymnastic processes and a high intensity beam. With a beam power of 50 kW and normal longitudinal emittance at the injection, the proton beam with a bunch length of about 53 with respect to the one in the normal operation mode was obtained and transported to the spallation target. With a reduced longitudinal emittance at injection and the beam power of 30 kW, the shortest extraction bunch length obtained is about 26 of the one in the normal operation mode. Different machine settings have also been tested to show the impact of the desynchronization between the RF and magnetic fields, the influence of the non adiabatic risetime and the adiabatic decay time of the RF voltage on the extraction bunch length. The experimental results are well consistent with the theoretical and simulated ones. It is interesting to observe that space charge has a beneficial effect on the bunch lengthening which will result in a shorter bunch at the extraction with the later bunch rotation. The controlled desynchronization method between the RF and magnetic fields in an RCS was also proven successful.


Crawling political communities in Twitter and extracting political affiliations. In theory, a major advantage to the big data approach in studying online communities is that it should be possible to collect a representative random sample from a broadly defined population. However, in practice, data collection processes are not formalized, even for famous social media platforms such as Twitter and Facebook. As a result, there is ambiguity left on questions such as "how much data is enough?" and how representative are the samples of the broader population being studied in online social networks. In this paper, I propose a focused back and forth crawl approach and a validated seed choice method for collecting network level data from Twitter. The proposed crawl method can extract community structures without needing a complete network graph for the Twitter network and validate its size using "reference score". It also takes care of the sampling size problem in Twitter by tracking the percentage of known nodes that have been included in the data. Thus, solving most major problems in Twitter data collection procedures and moving a step further to formalizing data collection methods for the platform. Once the communities are crawled, and the network graph is clean and complete it is then possible to train Machine Learning classifiers using communities as features to predict political affiliations of users on a larger scale. As a case, I used the proposed method for separating French political communities on Twitter from the global Twitter community and knowing the political affiliations of users on a continuous scale.


Large scale hydropower models in StochasticPrograms.jl. We present three large scale hydropower planning models implemented in our open source software framework StochasticPrograms.jl developed using the Julia programming langugage. The framework provides an expressive syntax for formulating stochastic programming models and has distributed capabilities that can handle large scale instances. The three models describe different case studies of the hydroelectric power plants in the Swedish river Skellefte "alven. The models are two stage stochastic programs with sampled scenarios that describe uncertain electricity prices and local water inflows. The first model is a day ahead planning problem that concerns how to determine optimal order strategies in a day ahead energy market. We pose this problem from the perspective of a hydropower producer, who participates in the Nordic day ahead market and operates in the Swedish river Skellefte "alven. We implement the day ahead model using our computational tools and then solve large scale instances of the problem in a distributed environment. A statistically significant value of running stochastic planning is obtained using a sample based algorithm. Next, we consider a variation of the day ahead problem that includes preventive maintenance scheduling. We show how intricate coordination between the submitteed market orders and the maintenance schedule results in a larger value of the stochastic solution than the day ahead problem. The final model is a capacity expansion problem with a long planning horizon. The same methodology is applied as when solving the first two hydropower problems. However, the planning horizon is considerably longer, from one year up to 20 years compared to a 24 hour horizon. We note that the relative significance of the value of the stochastic solution is much greater when comparing to the extra profits incurred from the capacity expansion instead of the total profit.


BLEnD Improving NDN Performance Over Wireless Links Using Interest Bundling. Named Data Networking (NDN) employs small sized Interest packets to retrieve large sized Data packets. Given the half duplex nature of wireless links, Interest packets frequently contend for the channel with Data packets, leading to throughput degradation. In this work, we present a novel idea called BLEnD, an Interest bundling technique that encodes multiple Interests into one at the sender and decodes at the receiver. The major design challenges are to reduce the number of Interest transmissions without impacting the one Interest one Data principle embedded everywhere in NDN architecture and implementation, and support flow congestion control mechanisms that usually use Interest packets as signals. BLEnD achieves these by bundling unbundling Interests at the link adaptation layer, keeping all NDN components unaware and unaffected. Over a one hop WiFi link, BLEnD improves application throughput by 30 . It may also be used over multiple hops and be improved in a number of ways.


Projecting onto rectangular matrices with prescribed row and column sums. In 1990, Romero presented a beautiful formula for the projection onto the set of rectangular matrices with prescribed row and column sums. Variants of Romero's formula have been rediscovered by Khoury and by Glunt, Hayden, and Reams, for bistochastic (square) matrices in 1998. These results have found various generalizations and applications. In this paper, we provide a formula for the more general problem of finding the projection onto the set of rectangular matrices with prescribed scaled row and column sums. Our approach is based on computing the Moore Penrose inverse of a certain linear operator associated with the problem. In fact, our analysis holds even for Hilbert Schmidt operators and we do not have to assume consistency. We also perform numerical experiments featuring the new projection operator.


Bird's Eye Probing for Linguistic Graph Structures with a Simple Information Theoretic Approach. NLP has a rich history of representing our prior understanding of language in the form of graphs. Recent work on analyzing contextualized text representations has focused on hand designed probe models to understand how and to what extent do these representations encode a particular linguistic phenomenon. However, due to the inter dependence of various phenomena and randomness of training probe models, detecting how these representations encode the rich information in these linguistic graphs remains a challenging problem. In this paper, we propose a new information theoretic probe, Bird's Eye, which is a fairly simple probe method for detecting if and how these representations encode the information in these linguistic graphs. Instead of using classifier performance, our probe takes an information theoretic view of probing and estimates the mutual information between the linguistic graph embedded in a continuous space and the contextualized word representations. Furthermore, we also propose an approach to use our probe to investigate localized linguistic information in the linguistic graphs using perturbation analysis. We call this probing setup Worm's Eye. Using these probes, we analyze BERT models on their ability to encode a syntactic and a semantic graph structure, and find that these models encode to some degree both syntactic as well as semantic information albeit syntactic information to a greater extent.


A Meta Learned Neuron model for Continual Learning. Continual learning is the ability to acquire new knowledge without forgetting the previously learned one, assuming no further access to past training data. Neural network approximators trained with gradient descent are known to fail in this setting as they must learn from a stream of data points sampled from a stationary distribution to converge. In this work, we replace the standard neuron by a meta learned neuron model whom inference and update rules are optimized to minimize catastrophic interference. Our approach can memorize dataset length sequences of training samples, and its learning capabilities generalize to any domain. Unlike previous continual learning methods, our method does not make any assumption about how tasks are constructed, delivered and how they relate to each other it simply absorbs and retains training samples one by one, whether the stream of input data is time correlated or not.


SOPE Spectrum of Off Policy Estimators. Many sequential decision making problems are high stakes and require off policy evaluation (OPE) of a new policy using historical data collected using some other policy. One of the most common OPE techniques that provides unbiased estimates is trajectory based importance sampling (IS). However, due to the high variance of trajectory IS estimates, importance sampling methods based on state action visitation distributions (SIS) have recently been adopted. Unfortunately, while SIS often provides lower variance estimates for long horizons, estimating the state action distribution ratios can be challenging and lead to biased estimates. In this paper, we present a new perspective on this bias variance trade off and show the existence of a spectrum of estimators whose endpoints are SIS and IS. Additionally, we also establish a spectrum for doubly robust and weighted version of these estimators. We provide empirical evidence that estimators in this spectrum can be used to trade off between the bias and variance of IS and SIS and can achieve lower mean squared error than both IS and SIS.


Moduli spaces of stable sheaves over quasi polarized surfaces, and the relative Strange Duality morphism. The main result of the present paper is a construction of relative moduli spaces of stable sheaves over the stack of quasipolarized projective surfaces. For this, we use the theory of good moduli spaces, whose study was initiated by Alper. As a corollary, we extend the relative Strange Duality morphism to the locus of quasipolarized K3 surfaces.


Accelerating Large Scale Real Time GNN Inference using Channel Pruning. Graph Neural Networks (GNNs) are proven to be powerful models to generate node embedding for downstream applications. However, due to the high computation complexity of GNN inference, it is hard to deploy GNNs for large scale or real time applications. In this paper, we propose to accelerate GNN inference by pruning the dimensions in each layer with negligible accuracy loss. Our pruning framework uses a novel LASSO regression formulation for GNNs to identify feature dimensions (channels) that have high influence on the output activation. We identify two inference scenarios and design pruning schemes based on their computation and memory usage for each. To further reduce the inference complexity, we effectively store and reuse hidden features of visited nodes, which significantly reduces the number of supporting nodes needed to compute the target embedding. We evaluate the proposed method with the node classification problem on five popular datasets and a real time spam detection application. We demonstrate that the pruned GNN models greatly reduce computation and memory usage with little accuracy loss. For full inference, the proposed method achieves an average of 3.27x speedup with only 0.002 drop in F1 Micro on GPU. For batched inference, the proposed method achieves an average of 6.67x speedup with only 0.003 drop in F1 Micro on CPU. To the best of our knowledge, we are the first to accelerate large scale real time GNN inference through channel pruning.


Semantic Based Explainable AI Leveraging Semantic Scene Graphs and Pairwise Ranking to Explain Robot Failures. When interacting in unstructured human environments, occasional robot failures are inevitable. When such failures occur, everyday people, rather than trained technicians, will be the first to respond. Existing natural language explanations hand annotate contextual information from an environment to help everyday people understand robot failures. However, this methodology lacks generalizability and scalability. In our work, we introduce a more generalizable semantic explanation framework. Our framework autonomously captures the semantic information in a scene to produce semantically descriptive explanations for everyday users. To generate failure focused explanations that are semantically grounded, we leverages both semantic scene graphs to extract spatial relations and object attributes from an environment, as well as pairwise ranking. Our results show that these semantically descriptive explanations significantly improve everyday users' ability to both identify failures and provide assistance for recovery than the existing state of the art context based explanations.


Thermodynamical and spectral phase transition for local diffeomorphisms in the circle. It is known that all uniformly expanding dynamics have no phase transition with respect to H "older continuous potentials. In this paper we show that given a local diffeomorphism f on the circle, that is neither a uniformly expanding dynamics nor invertible, the topological pressure function mathbb R ni t mapsto P top (f , t log Df ) is not analytical. In other words, f has a thermodynamic phase transition with respect to geometric potential. Assuming that f is transitive and that Df is H "older continuous, we show that there exists t 0 in (0 , 1 such that the transfer operator mathcal L f, t log Df , acting on the space of H "older continuous functions, has the spectral gap property for all t t 0 and has not the spectral gap property for all t geq t 0 . Similar results are also obtained when the transfer operator acts on the space of bounded variations functions and smooth functions. In particular, we show that in the transitive case f has a unique thermodynamic phase transition and it occurs in t 0 . In addition, if the loss of expansion of the dynamics occurs because of an indifferent fixed point or the dynamics admits an absolutely continuous invariant probability with positive Lyapunov exponent then t 0 1.


Uniqueness theorems for almost periodic objects. Uniqueness theorems are considered for various types of almost periodic objects functions, measures, distributions, multisets, holomorphic and meromorphic functions.


On radius of convergence of q deformed real numbers. We study analytic properties of " q deformed real numbers", a notion recently introduced by two of us. A q deformed positive real number is a power series with integer coefficients in one formal variable q . We study the radius of convergence of these power series assuming that q in C. Our main conjecture, which can be viewed as a q analogue of Hurwitz's Irrational Number Theorem, provides a lower bound for these radii, given by the radius of convergence of the q deformed golden ratio. The conjecture is proved in several particular cases and confirmed by a number of computer experiments. For an interesting sequence of "Pell polynomials", we obtain stronger bounds.


Event Driven News Stream Clustering using Entity Aware Contextual Embeddings. We propose a method for online news stream clustering that is a variant of the non parametric streaming K means algorithm. Our model uses a combination of sparse and dense document representations, aggregates document cluster similarity along these multiple representations and makes the clustering decision using a neural classifier. The weighted document cluster similarity model is learned using a novel adaptation of the triplet loss into a linear classification objective. We show that the use of a suitable fine tuning objective and external knowledge in pre trained transformer models yields significant improvements in the effectiveness of contextual embeddings for clustering. Our model achieves a new state of the art on a standard stream clustering dataset of English documents.


Accumulation points of the edit distance function. Given a hereditary property mathcal H of graphs and some p in 0,1 , the edit distance function operatorname ed mathcal H (p) is (asymptotically) the maximum proportion of "edits" (edge additions plus edge deletions) necessary to transform any graph of density p into a member of mathcal H . For any fixed p in 0,1 , operatorname ed mathcal H (p) can be computed from an object known as a colored regularity graph (CRG). This paper is concerned with those points p in 0,1 for which infinitely many CRGs are required to compute operatorname ed mathcal H on any open interval containing p such a p is called an accumulation point. We show that, as expected, p 0 and p 1 are indeed accumulation points for some hereditary properties we additionally determine the slope of operatorname ed mathcal H at these two extreme points. Unexpectedly, we construct a hereditary property with an accumulation point at p 1 4 . Finally, we derive a significant structural property about those CRGs which occur at accumulation points.


Rooted Clusters for Graph LP Algebras. LP algebras, introduced by Lam and Pylyavskyy, are a generalization of cluster algebras. These algebras are known to have the Laurent phenomenon, but positivity remains conjectural. Graph LP algebras are finite LP algebras encoded by a graph. For the graph LP algebra defined by a tree, we define a family of clusters called rooted clusters. We prove positivity for these clusters by giving explicit formulas for each cluster variable. We also give a combinatorial interpretation for these expansions using a generalization of T paths.


Dimension estimates for C 1 iterated function systems and repellers. Part II. This is the second part of our study of the dimension theory of C 1 iterated function systems (IFSs) and repellers on Bbb R d . In the first part we proved that the upper box counting dimension of the attractor of any C 1 IFS on Bbb R d is bounded above by its singularity dimension, and the upper packing dimension of any ergodic invariant measure associated with this IFS is bounded above by its Lyapunov dimension. Here we introduce a generalized transversality condition (GTC) for parametrized families of C 1 IFSs, and show that these upper bounds give actually the dimensions for "typical" C 1 IFSs under this transversality condition. Moreover we verify the GTC for some parametrized families of C 1 IFSs on Bbb R d .


Preintegration is not smoothing when monotonicity fails. Preintegration is a technique for high dimensional integration over d dimensional Euclidean space, which is designed to reduce an integral whose integrand contains kinks or jumps to a (d 1) dimensional integral of a smooth function. The resulting smoothness allows efficient evaluation of the (d 1) dimensional integral by a Quasi Monte Carlo or Sparse Grid method. The technique is similar to conditional sampling in statistical contexts, but the intention is different in conditional sampling the aim is to reduce the variance, rather than to achieve smoothness. Preintegration involves an initial integration with respect to one well chosen real valued variable. Griebel, Kuo, Sloan Math. Comp. 82 (2013), 383 400 and Griewank, Kuo, Le "ovey, Sloan J. Comput. Appl. Maths. 344 (2018), 259 274 showed that the resulting (d 1) dimensional integrand is indeed smooth under appropriate conditions, including a key assumption the integrand of the smooth function underlying the kink or jump is strictly monotone with respect to the chosen special variable when all other variables are held fixed. The question addressed in this paper is whether this monotonicity property with respect to one well chosen variable is necessary. We show here that the answer is essentially yes, in the sense that without this property the resulting (d 1) dimensional integrand is generally not smooth, having square root or other singularities.


Infinite Dimensional Linear Algebra and Solvability of Partial Differential Equations. We discuss linear algebra of infinite dimensional vector spaces in terms of algebraic (Hamel) bases. As an application we prove the surjectivity of a large class of linear partial differential operators with smooth ( mathcal C infty coefficients) coefficients, called in the article emph regular , acting on the algebraic dual mathcal D ( Omega) of the space of test functions mathcal D( Omega) . The surjectivity of the partial differential operators guarantees solvability of the corresponding partial differential equations within mathcal D ( Omega) . We discuss our result in contrast to and comparison with similar results about the restrictions of the regular operators on the space of Schwartz distribution mathcal D prime( Omega) , where these operators are often non surjective.


A note on torsion modules with pure embeddings. We study Martsinkovsky Russell torsion modules MaRu20 with pure embeddings as an abstract elementary class. We give a model theoretic characterization of the pure injective and the Sigma pure injective modules relative to the class of torsion modules assuming that the ring is right semihereditary. Our characterization of relative Sigma pure injective modules strictly extends the classical charactetization of GrJe76 and Zim, 3.6 . We study the limit models of the class and determine when the class is superstable assuming that the ring is right semihereditary. As a corollary, we show that the class of torsion abelian groups with pure embeddings is strictly stable, i.e., stable not superstable.


Fabrication of photonic resonators in bulk 4H SiC. The design and engineering of photonic architectures, suitable to enhance, collect and guide light on chip is needed for applications in quantum photonics and quantum optomechanics. In this work we apply a Faraday cage based oblique angle etch method to fabricate various functional photonic devices from 4H Silicon Carbide a material that has attracted attention in recent years, due to its potential in optomechanics, nonlinear optics and quantum information. We detail the processing conditions and thoroughly address the geometrical and optical characteristics of the fabricated devices. Employing photoluminescence measurements we demonstrate high quality factors for suspended microring resonators of up to 3500 in the visible range. Such devices will be applicable in the future to augment the properties of SiC in integrated on chip quantum photonics.


Pseudo Supervised Monocular Depth Estimation with Teacher Student Network. Despite recent improvement of supervised monocular depth estimation, the lack of high quality pixel wise ground truth annotations has become a major hurdle for further progress. In this work, we propose a new unsupervised depth estimation method based on pseudo supervision mechanism by training a teacher student network with knowledge distillation. It strategically integrates the advantages of supervised and unsupervised monocular depth estimation, as well as unsupervised binocular depth estimation. Specifically, the teacher network takes advantage of the effectiveness of binocular depth estimation to produce accurate disparity maps, which are then used as the pseudo ground truth to train the student network for monocular depth estimation. This effectively converts the problem of unsupervised learning to supervised learning. Our extensive experimental results demonstrate that the proposed method outperforms the state of the art on the KITTI benchmark.


Trisections with Kirby Thompson length 2. Kirby and Thompson introduced a length of a trisection. They also defined the length of a 4 manifold as the minimum of length among all lengths of trisection of a 4 manifold. In this paper, we consider trisections whose Kirby Thompson length is 2. Kirby and Thompson conjectured that length 2 trisection is a trisection of 4 manifold with length 0. We shall prove this conjecture in this paper.


DIRECTGO A new DIRECT type MATLAB toolbox for derivative free global optimization. In this work, we introduce DIRECTGO, a new MATLAB toolbox for derivative free global optimization. DIRECTGO collects various deterministic derivative free DIRECT type algorithms for box constrained, generally constrained, and problems with hidden constraints. Each sequential algorithm is implemented in two ways using static and dynamic data structures for more efficient information storage and organization. Furthermore, parallel schemes are applied to some promising algorithms within DIRECTGO. The toolbox is equipped with a graphical user interface (GUI), ensuring the user friendly use of all functionalities available in DIRECTGO. Available features are demonstrated in detailed computational studies using a comprehensive DIRECTGOLib v1.0 library of global optimization test problems. Additionally, eleven classical engineering design problems illustrate the potential of DIRECTGO to solve challenging real world problems. Finally, the appendix gives examples of accompanying MATLAB programs and provides a synopsis of its use on the test problems with box and general constraints.


Magnetized ICF Implosions Scaling of Temperature and Yield Enhancement. This paper investigates the impact of an applied magnetic field on the yield and hot spot temperature of inertial confinement fusion implosions. A scaling of temperature amplification due to magnetization is shown to be in agreement with unperturbed 2 D extended magnetohydrodynamic simulations. A perfectly spherical hot spot with an axial magnetic field is predicted to have a maximum temperature amplification of 37 . However, elongation of the hot spot along field lines raises this value by decreasing the hot spot surface area along magnetic field lines. A scaling for yield amplification predicts that a magnetic field has the greatest benefit for low temperature implosions this is in agreement with simplified 1 D simulations, but not 2 D simulations where the hot spot pressure can be significantly reduced by heat flow anisotropy. Simulations including a P2 drive asymmetry then show that the magnetized yield is a maximum when the capsule drive corrects the hot spot shape to be round at neutron bang time. The benefit of an applied field increases when the implosion is more highly perturbed. Increasing the magnetic field strength past the value required to magnetize the electrons is beneficial due to additional suppression of perturbations by magnetic tension.


HRKD Hierarchical Relational Knowledge Distillation for Cross domain Language Model Compression. On many natural language processing tasks, large pre trained language models (PLMs) have shown overwhelming performances compared with traditional neural network methods. Nevertheless, their huge model size and low inference speed have hindered the deployment on resource limited devices in practice. In this paper, we target to compress PLMs with knowledge distillation, and propose a hierarchical relational knowledge distillation (HRKD) method to capture both hierarchical and domain relational information. Specifically, to enhance the model capability and transferability, we leverage the idea of meta learning and set up domain relational graphs to capture the relational information across different domains. And to dynamically select the most representative prototypes for each domain, we propose a hierarchical compare aggregate mechanism to capture hierarchical relationships. Extensive experiments on public multi domain datasets demonstrate the superior performance of our HRKD method as well as its strong few shot learning ability. For reproducibility, we release the code at https github.com cheneydon hrkd.


Flattening Sharpness for Dynamic Gradient Projection Memory Benefits Continual Learning. The backpropagation networks are notably susceptible to catastrophic forgetting, where networks tend to forget previously learned skills upon learning new ones. To address such the 'sensitivity stability' dilemma, most previous efforts have been contributed to minimizing the empirical risk with different parameter regularization terms and episodic memory, but rarely exploring the usages of the weight loss landscape. In this paper, we investigate the relationship between the weight loss landscape and sensitivity stability in the continual learning scenario, based on which, we propose a novel method, Flattening Sharpness for Dynamic Gradient Projection Memory (FS DGPM). In particular, we introduce a soft weight to represent the importance of each basis representing past tasks in GPM, which can be adaptively learned during the learning process, so that less important bases can be dynamically released to improve the sensitivity of new skill learning. We further introduce Flattening Sharpness (FS) to reduce the generalization gap by explicitly regulating the flatness of the weight loss landscape of all seen tasks. As demonstrated empirically, our proposed method consistently outperforms baselines with the superior ability to learn new skills while alleviating forgetting effectively.


MPI Multi receptive and Parallel Integration for Salient Object Detection. The semantic representation of deep features is essential for image context understanding, and effective fusion of features with different semantic representations can significantly improve the model's performance on salient object detection. In this paper, a novel method called MPI is proposed for salient object detection. Firstly, a multi receptive enhancement module (MRE) is designed to effectively expand the receptive fields of features from different layers and generate features with different receptive fields. MRE can enhance the semantic representation and improve the model's perception of the image context, which enables the model to locate the salient object accurately. Secondly, in order to reduce the reuse of redundant information in the complex top down fusion method and weaken the differences between semantic features, a relatively simple but effective parallel fusion strategy (PFS) is proposed. It allows multi scale features to better interact with each other, thus improving the overall performance of the model. Experimental results on multiple datasets demonstrate that the proposed method outperforms state of the art methods under different evaluation metrics.


Statistical privacy preserving message dissemination for peer to peer networks. Concerns for the privacy of communication is widely discussed in research and overall society. For the public financial infrastructure of blockchains, this discussion encompasses the privacy of transaction data and its broadcasting throughout the network. To tackle this problem, we transform a discrete time protocol for contact networks over infinite trees into a computer network protocol for peer to peer networks. Peer to peer networks are modeled as organically growing graphs. We show that the distribution of shortest paths in such a network can be modeled using a normal distribution mathcal N ( mu, sigma 2). We determine statistical estimators for mu, sigma via multivariate models. The model behaves logarithmic over the number of nodes n and proportional to an inverse exponential over the number of added edges k. These results facilitate the computation of optimal forwarding probabilities during the dissemination phase for optimal privacy in a limited information environment.


Evaluating Fairness in Argument Retrieval. Existing commercial search engines often struggle to represent different perspectives of a search query. Argument retrieval systems address this limitation of search engines and provide both positive (PRO) and negative (CON) perspectives about a user's information need on a controversial topic (e.g., climate change). The effectiveness of such argument retrieval systems is typically evaluated based on topical relevance and argument quality, without taking into account the often differing number of documents shown for the argument stances (PRO or CON). Therefore, systems may retrieve relevant passages, but with a biased exposure of arguments. In this work, we analyze a range of non stochastic fairness aware ranking and diversity metrics to evaluate the extent to which argument stances are fairly exposed in argument retrieval systems. Using the official runs of the argument retrieval task Touch 'e at CLEF 2020, as well as synthetic data to control the amount and order of argument stances in the rankings, we show that systems with the best effectiveness in terms of topical relevance are not necessarily the most fair or the most diverse in terms of argument stance. The relationships we found between (un)fairness and diversity metrics shed light on how to evaluate group fairness in addition to topical relevance in argument retrieval settings.


Training Data Reduction for Performance Models of Data Analytics Jobs in the Cloud. Distributed dataflow systems like Apache Flink and Apache Spark simplify processing large amounts of data on clusters in a data parallel manner. However, choosing suitable cluster resources for distributed dataflow jobs in both type and number is difficult, especially for users who do not have access to previous performance metrics. One approach to overcoming this issue is to have users share runtime metrics to train context aware performance models that help find a suitable configuration for the job at hand. A problem when sharing runtime data instead of trained models or model parameters is that the data size can grow substantially over time. This paper examines several clustering techniques to minimize training data size while keeping the associated performance models accurate. Our results indicate that efficiency gains in data transfer, storage, and model training can be achieved through training data reduction. In the evaluation of our solution on a dataset of runtime data from 930 unique distributed dataflow jobs, we observed that, on average, a 75 data reduction only increases prediction errors by one percentage point.


Feature level Attentive ICF for Recommendation. Item based collaborative filtering (ICF) enjoys the advantages of high recommendation accuracy and ease in online penalization and thus is favored by the industrial recommender systems. ICF recommends items to a target user based on their similarities to the previously interacted items of the user. Great progresses have been achieved for ICF in recent years by applying advanced machine learning techniques (e.g., deep neural networks) to learn the item similarity from data. The early methods simply treat all the historical items equally and recently proposed methods attempt to distinguish the different importance of historical items when recommending a target item. Despite the progress, we argue that those ICF models neglect the diverse intents of users on adopting items (e.g., watching a movie because of the director, leading actors, or the visual effects). As a result, they fail to estimate the item similarity on a finer grained level to predict the user's preference to an item, resulting in sub optimal recommendation. In this work, we propose a general feature level attention method for ICF models. The key of our method is to distinguish the importance of different factors when computing the item similarity for a prediction. To demonstrate the effectiveness of our method, we design a light attention neural network to integrate both item level and feature level attention for neural ICF models. It is model agnostic and easy to implement. We apply it to two baseline ICF models and evaluate its effectiveness on six public datasets. Extensive experiments show the feature level attention enhanced models consistently outperform their counterparts, demonstrating the potential of differentiating user intents on the feature level for ICF recommendation models.


On Decentralized Minimax Control with Nested Subsystems. In this paper, we investigate a decentralized control problem with nested subsystems, which is a general model for one directional communication amongst many subsystems. The noises in our dynamics are modelled as uncertain variables which take values in finite sets. The objective is to minimize a worst case shared cost. We demonstrate how the prescription approach can simplify the information structure and derive a structural form for optimal control strategies. The structural form allows us to restrict attention to control strategies whose domains do not grow in size with time, and thus, this form can be utilized in systems with long time horizons. Finally, we present a dynamic program to derive the optimal control strategies and validate our results with a numerical example.


Transferring Semantic Knowledge Into Language Encoders. We introduce semantic form mid tuning, an approach for transferring semantic knowledge from semantic meaning representations into transformer based language encoders. In mid tuning, we learn to align the text of general sentences not tied to any particular inference task and structured semantic representations of those sentences. Our approach does not require gold annotated semantic representations. Instead, it makes use of automatically generated semantic representations, such as from off the shelf PropBank and FrameNet semantic parsers. We show that this alignment can be learned implicitly via classification or directly via triplet loss. Our method yields language encoders that demonstrate improved predictive performance across inference, reading comprehension, textual similarity, and other semantic tasks drawn from the GLUE, SuperGLUE, and SentEval benchmarks. We evaluate our approach on three popular baseline models, where our experimental results and analysis concludes that current pre trained language models can further benefit from structured semantic frames with the proposed mid tuning method, as they inject additional task agnostic knowledge to the encoder, improving the generated embeddings as well as the linguistic properties of the given model, as evident from improvements on a popular sentence embedding toolkit and a variety of probing tasks.


Syntactic Minimization of Nondeterministic Finite Automata. Nondeterministic automata may be viewed as succinct programs implementing deterministic automata, i.e. complete specifications. Converting a given deterministic automaton into a small nondeterministic one is known to be computationally very hard in fact, the ensuing decision problem is PSPACE complete. This paper stands in stark contrast to the status quo. We restrict attention to subatomic nondeterministic automata, whose individual states accept unions of syntactic congruence classes. They are general enough to cover almost all structural results concerning nondeterministic state minimality. We prove that converting a monoid recognizing a regular language into a small subatomic acceptor corresponds to an NP complete problem. The NP certificates are solutions of simple equations involving relations over the syntactic monoid. We also consider the subclass of atomic nondeterministic automata introduced by Brzozowski and Tamm. Given a deterministic automaton and another one for the reversed language, computing small atomic acceptors is shown to be NP complete with analogous certificates. Our complexity results emerge from an algebraic characterization of (sub)atomic acceptors in terms of deterministic automata with semilattice structure, combined with an equivalence of categories leading to succinct representations.


BRDS An FPGA based LSTM Accelerator with Row Balanced Dual Ratio Sparsification. In this paper, first, a hardware friendly pruning algorithm for reducing energy consumption and improving the speed of Long Short Term Memory (LSTM) neural network accelerators is presented. Next, an FPGA based platform for efficient execution of the pruned networks based on the proposed algorithm is introduced. By considering the sensitivity of two weight matrices of the LSTM models in pruning, different sparsity ratios (i.e., dual ratio sparsity) are applied to these weight matrices. To reduce memory accesses, a row wise sparsity pattern is adopted. The proposed hardware architecture makes use of computation overlapping and pipelining to achieve low power and high speed. The effectiveness of the proposed pruning algorithm and accelerator is assessed under some benchmarks for natural language processing, binary sentiment classification, and speech recognition. Results show that, e.g., compared to a recently published work in this field, the proposed accelerator could provide up to 272 higher effective GOPS W and the perplexity error is reduced by up to 1.4 for the PTB dataset.


The Contextual Appointment Scheduling Problem. This study is concerned with the determination of optimal appointment times for a sequence of jobs with uncertain duration. We investigate the data driven Appointment Scheduling Problem (ASP) when one has n observations of p features (covariates) related to the jobs as well as historical data. We formulate ASP as an Integrated Estimation and Optimization problem using a task based loss function. We justify the use of contexts by showing that not including the them yields to inconsistent decisions, which translates to sub optimal appointments. We validate our approach through two numerical experiments.


GCD of sums of k consecutive Fibonacci, Lucas, and generalized Fibonacci numbers. We explore the sums of k consecutive terms in the generalized Fibonacci sequence left(G n right) n geq 0 given by the recurrence G n G n 1 G n 2 for all n geq 2 with integral initial conditions G 0 and G 1 . In particular, we give precise values for the greatest common divisor (GCD) of all sums of k consecutive terms of left(G n right) n geq 0 . When G 0 0 and G 1 1 , we yield the GCD of all sums of k consecutive Fibonacci numbers, and when G 0 2 and G 1 1 , we yield the GCD of all sums of k consecutive Lucas numbers. Denoting the GCD of all sums of k consecutive generalized Fibonacci numbers by the symbol mathcal G G 0, G 1 !(k) , we give two tantalizing characterizations for these values, one involving a simple formula in k and another involving generalized Pisano periods mathcal G G 0, G 1 !(k) gcd(G k 1 G 1, , G k 2 G 2) mbox and mathcal G G 0, G 1 !(k) mathrm lcm m mid pi G 0,G 1 !(m) text divides k , where pi G 0,G 1 !(m) denotes the generalized Pisano period of the generalized Fibonacci sequence modulo m . The fact that these vastly different looking formulas coincide leads to some surprising and delightful new understandings of the Fibonacci and Lucas numbers.


Biases in Moments of Dirichlet Coefficients of Elliptic Curve Families. Elliptic curves arise in many important areas of modern number theory. One way to study them is take local data, the number of solutions modulo p , and create an L function. The behavior of this global object is related to two of the seven Clay Millennial Problems the Birch and Swinnerton Dyer Conjecture and the Generalized Riemann Hypothesis. We study one parameter families over mathbb Q (T) , which are of the form y 2 x 3 A(T)x B(T) , with non constant j invariant. We define the r th moment of an elliptic curve to be A r,E (p) frac1 p sum t bmod p a t(p) r , where a t(p) is p minus the number of solutions to y 2 x 3 A(t)x B(t) bmod p . Rosen and Silverman showed biases in the first moment equal the rank of the Mordell Weil group of rational solutions. Michel proved that pA 2,E (p) p 2 O(p 3 2 ) . Based on several special families where computations can be done in closed form, Miller in his thesis conjectured that the largest lower order term in the second moment that does not average to 0 is on average negative. He further showed that such a negative bias has implications in the distribution of zeros of the elliptic curve L function near the central point. To date, evidence for this conjecture is limited to special families. In addition to studying some additional families where the calculations can be done in closed form, we also systematically investigate families of various rank. These are the first general tests of the conjecture while we cannot in general obtain closed form solutions, we discuss computations which support or contradict the conjecture. We then generalize to higher moments, and see evidence that the bias continues in the even moments.


Global Local Dynamic Feature Alignment Network for Person Re Identification. The misalignment of human images caused by bounding box detection errors or partial occlusions is one of the main challenges in person Re Identification (Re ID) tasks. Previous local based methods mainly focus on learning local features in predefined semantic regions of pedestrians. These methods usually use local hard alignment methods or introduce auxiliary information such as key human pose points to match local features, which are often not applicable when large scene differences are encountered. To solve these problems, we propose a simple and efficient Local Sliding Alignment (LSA) strategy to dynamically align the local features of two images by setting a sliding window on the local stripes of the pedestrian. LSA can effectively suppress spatial misalignment and does not need to introduce extra supervision information. Then, we design a Global Local Dynamic Feature Alignment Network (GLDFA Net) framework, which contains both global and local branches. We introduce LSA into the local branch of GLDFA Net to guide the computation of distance metrics, which can further improve the accuracy of the testing phase. Evaluation experiments on several mainstream evaluation datasets including Market 1501, DukeMTMC reID, CUHK03 and MSMT17 show that our method has competitive accuracy over the several state of the art person Re ID methods. Specifically, it achieves 86.1 mAP and 94.8 Rank 1 accuracy on Market1501.


Fine spectral estimates with applications to the optimally fast solution of large FDE linear systems. In the present note we consider a type of matrices stemming in the context of the numerical approximation of distributed order fractional differential equations (FDEs) from one side they could look standard, since they are, real, symmetric and positive definite. On the other hand they present specific difficulties which prevent the successful use of classical tools. In particular the associated matrix sequence, with respect to the matrix size, is ill conditioned and it is such that a generating function does not exists, but we face the problem of dealing with a sequence of generating functions with an intricate expression. Nevertheless, we obtain a real interval where the smallest eigenvalue belongs, showing also its asymptotic behavior. We observe that the new bounds improve those already present in the literature and give a more accurate spectral information, which are in fact used in the design of fast numerical algorithms for the associated large linear systems, approximating the given distributed order FDEs. Very satisfactory numerical results are presented and critically discussed, while a section with conclusions and open problems ends the current note.