Small Errors in Random Zeroth Order Optimization are Imaginary. The vast majority of zeroth order optimization methods try to imitate first order methods via some smooth approximation of the gradient. Here, the smaller the smoothing parameter, the smaller the gradient approximation error. We show that for the majority of zeroth order methods this smoothing parameter can however not be chosen arbitrarily small as numerical cancellation errors will dominate. As such, theoretical and numerical performance could differ significantly. Using classical tools from numerical differentiation we will propose a new smoothed approximation of the gradient that can be integrated into general zeroth order algorithmic frameworks. Since the proposed smoothed approximation does not suffer from cancellation errors, the smoothing parameter (and hence the approximation error) can be made arbitrarily small. Sublinear convergence rates for algorithms based on our smoothed approximation are proved. Numerical experiments are also presented to demonstrate the superiority of algorithms based on the proposed approximation.


Relevance Aware Anomalous Users Detection in Social Network via Graph Neural Network. Anomalous users detection in social network is an imperative task for security problems. Motivated by the great power of Graph Neural Networks(GNNs), many current researches adopt GNN based detectors to reveal the anomalous users. However, the increasing scale of social activities, explosive growth of users and manifold technical disguise render the user detection a difficult task. In this paper, we propose an innovate Relevance aware Anomalous Users Detection model (RAU GNN) to obtain a fine grained detection result. RAU GNN first extracts multiple relations of all types of users in social network, including both benign and anomalous users, and accordingly constructs the multiple user relation graph. Secondly, we employ relevance aware GNN framework to learn the hidden features of users, and discriminate the anomalous users after discriminating. Concretely, by integrating Graph Convolution Network(GCN) and Graph Attention Network(GAT), we design a GCN based relation fusion layer to aggregate initial information from different relations, and a GAT based embedding layer to obtain the high level embeddings. Lastly, we feed the learned representations to the following GNN layer in order to consolidate the node embedding by aggregating the final users' embeddings. We conduct extensive experiment on real world datasets. The experimental results show that our approach can achieve high accuracy for anomalous users detection.


Opinions are Made to be Changed Temporally Adaptive Stance Classification. Given the rapidly evolving nature of social media and people's views, word usage changes over time. Consequently, the performance of a classifier trained on old textual data can drop dramatically when tested on newer data. While research in stance classification has advanced in recent years, no effort has been invested in making these classifiers have persistent performance over time. To study this phenomenon we introduce two novel large scale, longitudinal stance datasets. We then evaluate the performance persistence of stance classifiers over time and demonstrate how it decays as the temporal gap between training and testing data increases. We propose a novel approach to mitigate this performance drop, which is based on temporal adaptation of the word embeddings used for training the stance classifier. This enables us to make use of readily available unlabelled data from the current time period instead of expensive annotation efforts. We propose and compare several approaches to embedding adaptation and find that the Incremental Temporal Alignment (ITA) model leads to the best results in reducing performance drop over time.


Migration of CMSWEB Cluster at CERN to Kubernetes. The CMS experiment heavily relies on the CMSWEB cluster to host critical services for its operational needs. The cluster is deployed on virtual machines (VMs) from the CERN OpenStack cloud and is manually maintained by operators and developers. The release cycle is composed of several steps, from building RPMs, their deployment to perform validation, and integration tests. To enhance the sustainability of the CMSWEB cluster, CMS decided to migrate its cluster to a containerized solution such as Docker, orchestrated with Kubernetes (k8s). This allows us to significantly reduce the release upgrade cycle, follow the end to end deployment procedure, and reduce operational cost. This paper gives an overview of the current CMSWEB cluster and its issues. We describe the new architecture of the CMSWEB cluster in Kubernetes. We also provide a comparison of VM and Kubernetes deployment approaches and report on lessons learned during the migration process.


GP S3Net Graph based Panoptic Sparse Semantic Segmentation Network. Panoptic segmentation as an integrated task of both static environmental understanding and dynamic object identification, has recently begun to receive broad research interest. In this paper, we propose a new computationally efficient LiDAR based panoptic segmentation framework, called GP S3Net. GP S3Net is a proposal free approach in which no object proposals are needed to identify the objects in contrast to conventional two stage panoptic systems, where a detection network is incorporated for capturing instance information. Our new design consists of a novel instance level network to process the semantic results by constructing a graph convolutional network to identify objects (foreground), which later on are fused with the background classes. Through the fine grained clusters of the foreground objects from the semantic segmentation backbone, over segmentation priors are generated and subsequently processed by 3D sparse convolution to embed each cluster. Each cluster is treated as a node in the graph and its corresponding embedding is used as its node feature. Then a GCNN predicts whether edges exist between each cluster pair. We utilize the instance label to generate ground truth edge labels for each constructed graph in order to supervise the learning. Extensive experiments demonstrate that GP S3Net outperforms the current state of the art approaches, by a significant margin across available datasets such as, nuScenes and SemanticPOSS, ranking first on the competitive public SemanticKITTI leaderboard upon publication.


Decay properties of Axially Symmetric D solutions to the Steady Incompressible Magnetohydrodynamic Equations. In this paper, we investigate the decay properties of axially symmetric solutions to the steady incompressible magnetohydrodynamic equations in mathbb R 3 with finite Dirichlet integral. We first derive the decay rates of general D solutions to the axisymmetric MHD equations. In the special case where the magnetic field only has the swirl component textbf h (r,z) h theta (r,z) mathbf e theta , we obtain better decay rates. The last result examines the decay rates along the axis Oz also within the special class of D solutions with only swirl magnetic field. The main tool in this paper is the combination of the scaling argument, the textit Brezis Gallouet inequality and the weighted energy estimate.


A Latent space solver for PDE generalization. In this work we propose a hybrid solver to solve partial differential equation (PDE)s in the latent space. The solver uses an iterative inferencing strategy combined with solution initialization to improve generalization of PDE solutions. The solver is tested on an engineering case and the results show that it can generalize well to several PDE conditions.


BERTnesia Investigating the capture and forgetting of knowledge in BERT. Probing complex language models has recently revealed several insights into linguistic and semantic patterns found in the learned representations. In this article, we probe BERT specifically to understand and measure the relational knowledge it captures in its parametric memory. While probing for linguistic understanding is commonly applied to all layers of BERT as well as fine tuned models, this has not been done for factual knowledge. We utilize existing knowledge base completion tasks (LAMA) to probe every layer of pre trained as well as fine tuned BERT models(ranking, question answering, NER). Our findings show that knowledge is not just contained in BERT's final layers. Intermediate layers contribute a significant amount (17 60 ) to the total knowledge found. Probing intermediate layers also reveals how different types of knowledge emerge at varying rates. When BERT is fine tuned, relational knowledge is forgotten. The extent of forgetting is impacted by the fine tuning objective and the training data. We found that ranking models forget the least and retain more knowledge in their final layer compared to masked language modeling and question answering. However, masked language modeling performed the best at acquiring new knowledge from the training data. When it comes to learning facts, we found that capacity and fact density are key factors. We hope this initial work will spur further research into understanding the parametric memory of language models and the effect of training objectives on factual knowledge. The code to repeat the experiments is publicly available on GitHub.


A multidimensional principal component analysis via the c product Golub Kahan SVD for classification and face recognition. Face recognition and identification is a very important application in machine learning. Due to the increasing amount of available data, traditional approaches based on matricization and matrix PCA methods can be difficult to implement. Moreover, the tensorial approaches are a natural choice, due to the mere structure of the databases, for example in the case of color images. Nevertheless, even though various authors proposed factorization strategies for tensors, the size of the considered tensors can pose some serious issues. When only a few features are needed to construct the projection space, there is no need to compute a SVD on the whole data. Two versions of the tensor Golub Kahan algorithm are considered in this manuscript, as an alternative to the classical use of the tensor SVD which is based on truncated strategies. In this paper, we consider the Tensor Tubal Golub Kahan Principal Component Analysis method which purpose is to extract the main features of images using the tensor singular value decomposition (SVD) based on the tensor cosine product that uses the discrete cosine transform. This approach is applied for classification and face recognition and numerical tests show its effectiveness.


When stable Cohen Macaulay Auslander algebra is semisimple. Let text Gprj mbox Lambda denote the category of Gorenstein projective modules over an Artin algebra Lambda and the category text mod mbox ( underline text Gprj mbox Lambda) of finitely presented functors over the stable category underline text Gprj mbox Lambda . In this paper, we study those algebras Lambda with text mod mbox ( underline text Gprj mbox Lambda) to be a semisimple abelian category, and called Omega mathcal G algebras. The class of Omega mathcal G algebras contains important classes of algebras, including gentle algebras. Over an Omega mathcal G algebra Lambda , the structure of the almost split sequences in the morphism categories text H ( text Gprj mbox Lambda) and the monomorphism categories mathcal S ( text Gprj mbox Lambda) of text Gprj mbox Lambda is investigated. Among other applications, we provide some results for the Cohen Macaulay Auslander algebras of Omega mathcal G algebras.


Efficient Feature Representations for Cricket Data Analysis using Deep Learning based Multi Modal Fusion Model. Data analysis has become a necessity in the modern era of cricket. Everything from effective team management to match win predictions use some form of analytics. Meaningful data representations are necessary for efficient analysis of data. In this study we investigate the use of adaptive (learnable) embeddings to represent inter related features (such as players, teams, etc). The data used for this study is collected from a classical T20 tournament IPL (Indian Premier League). To naturally facilitate the learning of meaningful representations of features for accurate data analysis, we formulate a deep representation learning framework which jointly learns a custom set of embeddings (which represents our features of interest) through the minimization of a contrastive loss. We base our objective on a set of classes obtained as a result of hierarchical clustering on the overall run rate of an innings. It's been assessed that the framework ensures greater generality in the obtained embeddings, on top of which a task based analysis of overall run rate prediction was done to show the reliability of the framework.


The asymptotic expansion of a sum appearing in an approximate functional equation for the riemann zeta function. A representation for the Riemann zeta function valid for arbitrary complex s sigma it is zeta(s) sum n 0 infty A(n,s) , where A(n,s) frac 2 n 1 1 2 1 s sum k 0 n left( ! begin array c n k end array ! right) frac ( ) k (k 1) s . In this note we examine the asymptotics of A(n,s) as n to infty when t an , where a 0 is a fixed parameter, by application of the method of steepest descents to an integral representation. Numerical results are presented to illustrate the accuracy of the expansion obtained.


Categorical blow up formula for Hilbert schemes of points. Let S be a smooth projective surface, and hat S be its blow up at a point. In this paper, we study the derived category of the Hilbert scheme of points on the blow up hat S . We obtain a semi orthogonal decomposition consisting of the derived categories of the Hilbert schemes on the original surface S , which recovers the blow up formula for the Euler characteristics obtained by G "ottsche and Nakajima Yoshioka. The proof uses the Quot formula, which was conjectured by Jiang and recently proved by Toda.


Machine vision detection to daily facial fatigue with a nonlocal 3D attention network. Fatigue detection is valued for people to keep mental health and prevent safety accidents. However, detecting facial fatigue, especially mild fatigue in the real world via machine vision is still a challenging issue due to lack of non lab dataset and well defined algorithms. In order to improve the detection capability on facial fatigue that can be used widely in daily life, this paper provided an audiovisual dataset named DLFD (daily life fatigue dataset) which reflected people's facial fatigue state in the wild. A framework using 3D ResNet along with non local attention mechanism was training for extraction of local and long range features in spatial and temporal dimensions. Then, a compacted loss function combining mean squared error and cross entropy was designed to predict both continuous and categorical fatigue degrees. Our proposed framework has reached an average accuracy of 90.8 on validation set and 72.5 on test set for binary classification, standing a good position compared to other state of the art methods. The analysis of feature map visualization revealed that our framework captured facial dynamics and attempted to build a connection with fatigue state. Our experimental results in multiple metrics proved that our framework captured some typical, micro and dynamic facial features along spatiotemporal dimensions, contributing to the mild fatigue detection in the wild.


Patterns of Routes of Administration and Drug Tampering for Nonmedical Opioid Consumption Data Mining and Content Analysis of Reddit Discussions. The complex unfolding of the US opioid epidemic in the last 20 years has been the subject of a large body of medical and pharmacological research, and it has sparked a multidisciplinary discussion on how to implement interventions and policies to effectively control its impact on public health. This study leverages Reddit as the primary data source to investigate the opioid crisis. We aimed to find a large cohort of Reddit users interested in discussing the use of opioids, trace the temporal evolution of their interest, and extensively characterize patterns of the nonmedical consumption of opioids, with a focus on routes of administration and drug tampering. We used a semiautomatic information retrieval algorithm to identify subreddits discussing nonmedical opioid consumption, finding over 86,000 Reddit users potentially involved in firsthand opioid usage. We developed a methodology based on word embedding to select alternative colloquial and nonmedical terms referring to opioid substances, routes of administration, and drug tampering methods. We modeled the preferences of adoption of substances and routes of administration, estimating their prevalence and temporal unfolding, observing relevant trends such as the surge in synthetic opioids like fentanyl and an increasing interest in rectal administration. Ultimately, through the evaluation of odds ratios based on co mentions, we measured the strength of association between opioid substances, routes of administration, and drug tampering, finding evidence of understudied abusive behaviors like chewing fentanyl patches and dissolving buprenorphine sublingually. We believe that our approach may provide a novel perspective for a more comprehensive understanding of nonmedical abuse of opioids substances and inform the prevention, treatment, and control of the public health effects.


Semi supervised Optimal Transport with Self paced Ensemble for Cross hospital Sepsis Early Detection. The utilization of computer technology to solve problems in medical scenarios has attracted considerable attention in recent years, which still has great potential and space for exploration. Among them, machine learning has been widely used in the prediction, diagnosis and even treatment of Sepsis. However, state of the art methods require large amounts of labeled medical data for supervised learning. In real world applications, the lack of labeled data will cause enormous obstacles if one hospital wants to deploy a new Sepsis detection system. Different from the supervised learning setting, we need to use known information (e.g., from another hospital with rich labeled data) to help build a model with acceptable performance, i.e., transfer learning. In this paper, we propose a semi supervised optimal transport with self paced ensemble framework for Sepsis early detection, called SPSSOT, to transfer knowledge from the other that has rich labeled data. In SPSSOT, we first extract the same clinical indicators from the source domain (e.g., hospital with rich labeled data) and the target domain (e.g., hospital with little labeled data), then we combine the semi supervised domain adaptation based on optimal transport theory with self paced under sampling to avoid a negative transfer possibly caused by covariate shift and class imbalance. On the whole, SPSSOT is an end to end transfer learning method for Sepsis early detection which can automatically select suitable samples from two domains respectively according to the number of iterations and align feature space of two domains. Extensive experiments on two open clinical datasets demonstrate that comparing with other methods, our proposed SPSSOT, can significantly improve the AUC values with only 1 labeled data in the target domain in two transfer learning scenarios, MIMIC rightarrow Challenge and Challenge rightarrow MIMIC.


Are Transformers More Robust Than CNNs?. Transformer emerges as a powerful tool for visual recognition. In addition to demonstrating competitive performance on a broad range of visual benchmarks, recent works also argue that Transformers are much more robust than Convolutions Neural Networks (CNNs). Nonetheless, surprisingly, we find these conclusions are drawn from unfair experimental settings, where Transformers and CNNs are compared at different scales and are applied with distinct training frameworks. In this paper, we aim to provide the first fair in depth comparisons between Transformers and CNNs, focusing on robustness evaluations. With our unified training setup, we first challenge the previous belief that Transformers outshine CNNs when measuring adversarial robustness. More surprisingly, we find CNNs can easily be as robust as Transformers on defending against adversarial attacks, if they properly adopt Transformers' training recipes. While regarding generalization on out of distribution samples, we show pre training on (external) large scale datasets is not a fundamental request for enabling Transformers to achieve better performance than CNNs. Moreover, our ablations suggest such stronger generalization is largely benefited by the Transformer's self attention like architectures per se, rather than by other training setups. We hope this work can help the community better understand and benchmark the robustness of Transformers and CNNs. The code and models are publicly available at https github.com ytongbai ViTs vs CNNs.


Vertex numbers of simplicial complexes with free abelian fundamental group. We show that the minimum number of vertices of a simplicial complex with fundamental group mathbb Z n is at most O(n) and at least Omega(n 3 4 ) . For the upper bound, we use a result on orthogonal 1 factorizations of K 2n . For the lower bound, we use a fractional Sylvester Gallai result. We also prove that any group presentation langle S R rangle cong mathbb Z n whose relations are of the form g a h b i c for g, h, i in S has at least Omega(n 3 2 ) generators.


Learning Single Multi Attribute of Object with Symmetry and Group. Attributes and objects can compose diverse compositions. To model the compositional nature of these concepts, it is a good choice to learn them as transformations, e.g., coupling and decoupling. However, complex transformations need to satisfy specific principles to guarantee rationality. Here, we first propose a previously ignored principle of attribute object transformation Symmetry. For example, coupling peeled apple with attribute peeled should result in peeled apple, and decoupling peeled from apple should still output apple. Incorporating the symmetry, we propose a transformation framework inspired by group theory, i.e., SymNet. It consists of two modules Coupling Network and Decoupling Network. We adopt deep neural networks to implement SymNet and train it in an end to end paradigm with the group axioms and symmetry as objectives. Then, we propose a Relative Moving Distance (RMD) based method to utilize the attribute change instead of the attribute pattern itself to classify attributes. Besides the compositions of single attribute and object, our RMD is also suitable for complex compositions of multiple attributes and objects when incorporating attribute correlations. SymNet can be utilized for attribute learning, compositional zero shot learning and outperforms the state of the art on four widely used benchmarks. Code is at https github.com DirtyHarryLYL SymNet.


A Mattila Sj " o lin theorem for triangles. We show for a compact set E subset mathbb R d , d geq 4 , that if the Hausdorff dimension of E is larger than frac 2 3 d 1 , then the set of congruence classes of triangles formed by triples of points of E has nonempty interior. Here we understand the set of congruence classes of triangles formed by triples of points of E as the set Delta text tri (E) left (t,r, alpha) x z t, y z r , text and , alpha alpha(x,z,y), x,y,z in E right , where alpha (x,z,y) denotes the angle formed by x , y and z , centered at z . This extends the Mattila Sj " o lin theorem that establishes a non empty interior for the distance set instead of the set of congruence classes of triangles. These theorems can be thought of as refinements and extensions of the statements in the well known Falconer distance problem.


Differentiable solutions of an equation with product of iterates. In the previous work 2 (i.e., arXiv 2105.03385), we considered continuous solutions of an iterative equation involving the multiplication of iterates. In this paper, we continue to investigate this equation for differentiable solutions. Similar to continuous solutions until 2 , there is no obtained result on differentiable solutions of such an equation on non compact intervals of mathbb R . Although our strategy here is to use conjugation to reduce the equation to the well known polynomial like iterative equation as in 2 , all known results on differentiable solutions of the latter are given on compact intervals. We re explore polynomial like iterative equation on the whole of R and prove the existence and uniqueness of differentiable solutions of our equation on mathbb R and mathbb R .


Generic density of geodesic nets. We prove that for a generic metric on a closed smooth manifold, the union of the images of all stationary geodesic nets forms a dense set.


Large time behavior of strong solutions for stochastic Burgers equation. We consider the large time behavior of strong solutions to a kind of stochastic Burgers equation, where the position x is perturbed by a Brownian noise. It is well known that both the rarefaction wave and viscous shock wave are time asymptotically stable for deterministic Burgers equation since the pioneer work of A. Ilin and O. Oleinik 21 in 1964. However, the stability of these wave patterns under stochastic perturbation is not known until now. In this paper, we give a definite answer to the stability problem of the rarefaction and viscous shock waves for the 1 d stochastic Burgers equation. That is, the rarefaction wave is still stable under white noise perturbation and the viscous shock is not stable yet. Moreover, a time convergence rate toward the rarefaction wave is obtained. To get the desired decay rate, an important inequality (denoted by Area Inequality) is derived. This inequality plays essential role in the proof, and may have applications in the related problems for both the stochastic and deterministic PDEs.


Wavelet Denoised ResNet CNN and LightGBM Method to Predict Forex Rate of Change. Foreign Exchange (Forex) is the largest financial market in the world. The daily trading volume of the Forex market is much higher than that of stock and futures markets. Therefore, it is of great significance for investors to establish a foreign exchange forecast model. In this paper, we propose a Wavelet Denoised ResNet with LightGBM model to predict the rate of change of Forex price after five time intervals to allow enough time to execute trades. All the prices are denoised by wavelet transform, and a matrix of 30 time intervals is formed by calculating technical indicators. Image features are obtained by feeding the maxtrix into a ResNet. Finally, the technical indicators and image features are fed to LightGBM. Our experiments on 5 minutes USDJPY demonstrate that the model outperforms baseline modles with MAE 0.240977x10EXP 3 MSE 0.156x10EXP 6 and RMSE 0.395185x10EXP 3. An accurate price prediction after 25 minutes in future provides a window of opportunity for hedge funds algorithm trading. The code is available from https mkhushi.github.io


The Brauer group of a valuation ring. Let V be a valuation ring and K be its field of fraction. We show that the canonical map Br(V) to Br(K) is injective.


A Decomposition Model for Stereo Matching. In this paper, we present a decomposition model for stereo matching to solve the problem of excessive growth in computational cost (time and memory cost) as the resolution increases. In order to reduce the huge cost of stereo matching at the original resolution, our model only runs dense matching at a very low resolution and uses sparse matching at different higher resolutions to recover the disparity of lost details scale by scale. After the decomposition of stereo matching, our model iteratively fuses the sparse and dense disparity maps from adjacent scales with an occlusion aware mask. A refinement network is also applied to improving the fusion result. Compared with high performance methods like PSMNet and GANet, our method achieves 10 100 times speed increase while obtaining comparable disparity estimation results.


Theoretical study and finite element simulation of ZnO GaAs higher order Lamb waves for microsensor application in liquid media. Lamb waves with dominantly longitudinal displacement component have been reported for sensor application in liquid media. However, they are still limited for the fundamental symmetry mode with low plate thickness to wavelength (h l) ratio, resulting in large wavelength and low resonance frequency. Here we show the finite element simulation study of higher order Lamb waves on GaAs and ZnO GaAs plates. In plane polarised quasi longitudinal higher order Lamb waves are obtained at higher h l ratio resulting in higher resonance frequencies. The solid liquid contact simulation shows the confinement of acoustic energy inside the plate. The result demonstrates the suitability of ZnO GaAs higher order quasi longitudinal modes for microsensor in liquid media.


Cutoff for the Fredrickson Andersen one spin facilitated model. The Fredrickson Andersen one spin facilitated model belongs to the class of Kinetically Constrained Spin Models. It is a non attractive process with positive spectral gap. In this paper we give a precise result on the relaxation for this process on an interval 1,L starting from any initial configuration. A consequence of this result is that this process exhibits cutoff at time L (2v) with window O( sqrt L ) for a certain positive constant v . The key ingredient is the study of the evolution of the leftmost empty site in a filled infinite half line called the front. In the process of the proof, we improve recent results about the front motion by showing that it evolves at speed v according to a uniform central limit theorem.


A Deep Learning Approach To Multi Context Socially Aware Navigation. We present a context classification pipeline to allow a robot to change its navigation strategy based on the observed social scenario. Socially Aware Navigation considers social behavior in order to improve navigation around people. Most of the existing research uses different techniques to incorporate social norms into robot path planning for a single context. Methods that work for hallway behavior might not work for approaching people, and so on. We developed a high level decision making subsystem, a model based context classifier, and a multi objective optimization based local planner to achieve socially aware trajectories for autonomously sensed contexts. Using a context classification system, the robot can select social objectives that are later used by Pareto Concavity Elimination Transformation (PaCcET) based local planner to generate safe, comfortable, and socially appropriate trajectories for its environment. This was tested and validated in multiple environments on a Pioneer mobile robot platform results show that the robot was able to select and account for social objectives related to navigation autonomously.


A fast converging scheme for the Phonon Boltzmann equation with dual relaxation times. Callaway's dual relaxation times model, which takes into account the normal and resistive scatterings of phonon, is used to describe the heat conduction in materials like graphene. For steady state problems, the Callaway model is usually solved by the conventional iterative scheme (CIS), which is efficient in the ballistic regime, but inefficient in the diffusive hydrodynamic regime. In this paper, a general synthetic iterative scheme (GSIS) is proposed to expedite the convergence to steady state solutions. First, macroscopic synthetic equations are designed to guide the evolution of equilibrium distribution functions for normal and resistive scatterings, so that fast convergence can be achieved even in the diffusive hydrodynamic regime. Second, the Fourier stability analysis is conducted to find the convergence rate for both CIS and GSIS, which rigorously proves the efficiency of GSIS over CIS. Finally, several numerical simulations are carried out to demonstrate the accuracy and efficiency of GSIS, where up to three orders of magnitude of convergence acceleration is achieved.


An Unsupervised Sampling Approach for Image Sentence Matching Using Document Level Structural Information. In this paper, we focus on the problem of unsupervised image sentence matching. Existing research explores to utilize document level structural information to sample positive and negative instances for model training. Although the approach achieves positive results, it introduces a sampling bias and fails to distinguish instances with high semantic similarity. To alleviate the bias, we propose a new sampling strategy to select additional intra document image sentence pairs as positive or negative samples. Furthermore, to recognize the complex pattern in intra document samples, we propose a Transformer based model to capture fine grained features and implicitly construct a graph for each document, where concepts in a document are introduced to bridge the representation learning of images and sentences in the context of a document. Experimental results show the effectiveness of our approach to alleviate the bias and learn well aligned multimodal representations.


Spectral multipliers in group algebras and noncommutative Calder 'on Zygmund theory. In this paper we solve three problems in noncommutative harmonic analysis which are related to endpoint inequalities for singular integrals. In first place, we prove that an L 2 form of H "ormander's kernel condition suffices for the weak type (1,1) of Calder 'on Zygmund operators acting on matrix valued functions. To that end, we introduce an improved CZ decomposition for martingale filtrations in von Neumann algebras, and apply a very simple unconventional argument which notably avoids pseudolocalization. In second place, we establish as well the weak L 1 endpoint for matrix valued CZ operators over nondoubling measures of polynomial growth, in the line of the work of Tolsa and Nazarov Treil Volberg. The above results are valid for other von Neumann algebras and solve in the positive two open problems formulated in 2009. An even more interesting problem is the lack of L 1 endpoint inequalities for singular Fourier and Schur multipliers over nonabelian groups. Given a locally compact group G equipped with a conditionally negative length psi mathrm G to mathbb R , we prove that Herz Schur multipliers with symbol m circ psi satisfying a Mikhlin condition in terms of the psi cocycle dimension are of weak type (1,1) . Our result extends to Fourier multipliers for amenable groups and impose sharp regularity conditions on the symbol. The proof crucially combines our new CZ methods with novel forms of recent transference techniques. This L 1 endpoint gives a very much expected inequality which complements the L infty to mathrm BMO estimates proved in 2014 by Junge, Mei and Parcet.


Convolutions for Stirling numbers, Lah numbers, and binomial coefficients. We present various identities in the form of convolutions involving Stirling numbers of both kinds, Lah numbers, and binomial coefficients. Certain convolution polynomials are discussed also. The proofs are based on several series transformation formulas.


MeetDot Videoconferencing with Live Translation Captions. We present MeetDot, a videoconferencing system with live translation captions overlaid on screen. The system aims to facilitate conversation between people who speak different languages, thereby reducing communication barriers between multilingual participants. Currently, our system supports speech and captions in 4 languages and combines automatic speech recognition (ASR) and machine translation (MT) in a cascade. We use the re translation strategy to translate the streamed speech, resulting in caption flicker. Additionally, our system has very strict latency requirements to have acceptable call quality. We implement several features to enhance user experience and reduce their cognitive load, such as smooth scrolling captions and reducing caption flicker. The modular architecture allows us to integrate different ASR and MT services in our backend. Our system provides an integrated evaluation suite to optimize key intrinsic evaluation metrics such as accuracy, latency and erasure. Finally, we present an innovative cross lingual word guessing game as an extrinsic evaluation metric to measure end to end system performance. We plan to make our system open source for research purposes.


On the connected components of Shimura varieties for CM unitary groups in odd variables. We study the prime to p Hecke action on the projective limit of the sets of connected components of Shimura varieties with fixed parahoric or Bruhat Tits level at p . In particular, we construct infinitely many Shimura varieties for CM unitary groups in odd variables for which the considering actions are not transitive. We prove this result by giving negative examples on the question of Bruhat Colliot Th 'el ene Sansuc Tits or its variant, which is related to the weak approximation on tori over mathbb Q .


Time resolution and efficiency of SPADs and SiPMs for photons and charged particles. We give an analytic treatment of the time resolution and efficiency of Single Photon Avalanche Diodes (SPADs) and Silicon Photomultipliers (SiPMs). We provide closed form expressions for structures with uniform electric fields and efficient numerical prescriptions for arbitrary electric field configurations. We discuss the sensor performance for single photon detection and also for charged particle detection.


Algebraic differential equations of periods integrals. We explain that in the study of the asymptotic expansion at the origin of a period integral like gamma z omega df or of a hermitian period like f s rho . omega df land omega df the computation of the Bernstein polynomial of the "fresco" (filtered differential equation) associated to the pair of germs (f, omega ) gives a better control than the computation of the Bernstein polynomial of the full Brieskorn module of the germ of f at the origin. Moreover, it is easier to compute as it has a better functoriality and smaller degree. We illustrate this in the case where f in C x 0 ,. .. , x n has n 2 monomials and is not quasi homogeneous, by giving an explicite simple algorithm to produce a multiple of the Bernstein polynomial when omega is a monomial holomorphic volume form. Several concrete examples are given.


Bipartite 3 Regular Counting Problems with Mixed Signs. We prove a complexity dichotomy for a class of counting problems expressible as bipartite 3 regular Holant problems. For every problem of the form operatorname Holant left(f mid 3 right) , where f is any integer valued ternary symmetric constraint function on Boolean variables, we prove that it is either P time computable or P hard, depending on an explicit criterion of f . The constraint function can take both positive and negative values, allowing for cancellations. The dichotomy extends easily to rational valued functions of the same type. In addition, we discover a new phenomenon there is a set mathcal F with the property that for every f in mathcal F the problem operatorname Holant left(f mid 3 right) is planar P time computable but P hard in general, yet its planar tractability is by a combination of a holographic transformation by left begin smallmatrix 1 1 1 1 end smallmatrix right to FKT together with an independent global argument.


Being ahead Benchmarking and Exploring Accelerators for Hardware Efficient AI Deployment. Customized hardware accelerators have been developed to provide improved performance and efficiency for DNN inference and training. However, the existing hardware accelerators may not always be suitable for handling various DNN models as their architecture paradigms and configuration tradeoffs are highly application specific. It is important to benchmark the accelerator candidates in the earliest stage to gather comprehensive performance metrics and locate the potential bottlenecks. Further demands also emerge after benchmarking, which require adequate solutions to address the bottlenecks and improve the current designs for targeted workloads. To achieve these goals, in this paper, we leverage an automation tool called DNNExplorer for benchmarking customized DNN hardware accelerators and exploring novel accelerator designs with improved performance and efficiency. Key features include (1) direct support to popular machine learning frameworks for DNN workload analysis and accurate analytical models for fast accelerator benchmarking (2) a novel accelerator design paradigm with high dimensional design space support and fine grained adjustability to overcome the existing design drawbacks and (3) a design space exploration (DSE) engine to generate optimized accelerators by considering targeted AI workloads and available hardware resources. Results show that accelerators adopting the proposed novel paradigm can deliver up to 4.2X higher throughput (GOP s) than the state of the art pipeline design in DNNBuilder and up to 2.0X improved efficiency than the recently published generic design in HybridDNN given the same DNN model and resource budgets. With DNNExplorer's benchmarking and exploration features, we can be ahead at building and optimizing customized AI accelerators and enable more efficient AI applications.


Average case complexity of the Whitehead problem for a free group. The worst case complexity of group theoretic algorithms has been studied for a long time. Generic case complexity, or complexity on random inputs, was introduced and studied relatively recently. In this paper, we address the average case complexity of algorithms that solve a well known problem, the Whitehead problem in a free group, which is given two elements of a free group, find out whether there is an automorphism that takes one element to the other.. First we address a special case of the Whitehead problem, namely deciding if a given element of a free group is part of a free basis. We show that there is an algorithm that solves this problem and has sublinear average case complexity. For the general Whitehead problem, we show that the classical Whitehead algorithm has linear average case complexity if the rank of a free group is 2. We argue that the same should be true in a free group of any rank but point out obstacles to establishing this general result.


Randic index, radius, and diameter for cactus graphs. We study the Randic index for cactus graphs. It is conjectured to be bounded below by radius (for other than an even path), and it is known to obey several bounds based on diameter. We study radius and diameter for cacti then verify the radius bound and strengthen two diameter bounds for cacti. Along the way, we produce several other bounds for the Randic index in terms of graph size, order, and valency for several special classes of graphs, including chemical nontrivial cacti and cacti with starlike BC trees.


Weakly Supervised Saliency Detection via Salient Object Subitizing. Salient object detection aims at detecting the most visually distinct objects and producing the corresponding masks. As the cost of pixel level annotations is high, image tags are usually used as weak supervisions. However, an image tag can only be used to annotate one class of objects. In this paper, we introduce saliency subitizing as the weak supervision since it is class agnostic. This allows the supervision to be aligned with the property of saliency detection, where the salient objects of an image could be from more than one class. To this end, we propose a model with two modules, Saliency Subitizing Module (SSM) and Saliency Updating Module (SUM). While SSM learns to generate the initial saliency masks using the subitizing information, without the need for any unsupervised methods or some random seeds, SUM helps iteratively refine the generated saliency masks. We conduct extensive experiments on five benchmark datasets. The experimental results show that our method outperforms other weakly supervised methods and even performs comparably to some fully supervised methods.


Flow Plugin Network for conditional generation. Generative models have gained many researchers' attention in the last years resulting in models such as StyleGAN for human face generation or PointFlow for the 3D point cloud generation. However, by default, we cannot control its sampling process, i.e., we cannot generate a sample with a specific set of attributes. The current approach is model retraining with additional inputs and different architecture, which requires time and computational resources. We propose a novel approach that enables to a generation of objects with a given set of attributes without retraining the base model. For this purpose, we utilize the normalizing flow models Conditional Masked Autoregressive Flow and Conditional Real NVP, as a Flow Plugin Network (FPN).


SmartArm Suturing Feasibility of a Surgical Robotic System on a Neonatal Chest Model. Commercially available surgical robot technology currently addresses many surgical scenarios for adult patients. This same technology cannot be used to the benefit of neonate patients given the considerably smaller workspace. Medically relevant procedures regarding neonate patients include minimally invasive surgery to repair congenital esophagus disorders, which entail the suturing of the fragile esophagus within the narrow neonate cavity. In this work, we explore the use of the SmartArm robotic system in a feasibility study using a neonate chest and esophagus model. We show that a medically inexperienced operator can perform two throw knots inside the neonate chest model using the robotic system.


Multi Class Cell Detection Using Spatial Context Representation. In digital pathology, both detection and classification of cells are important for automatic diagnostic and prognostic tasks. Classifying cells into subtypes, such as tumor cells, lymphocytes or stromal cells is particularly challenging. Existing methods focus on morphological appearance of individual cells, whereas in practice pathologists often infer cell classes through their spatial context. In this paper, we propose a novel method for both detection and classification that explicitly incorporates spatial contextual information. We use the spatial statistical function to describe local density in both a multi class and a multi scale manner. Through representation learning and deep clustering techniques, we learn advanced cell representation with both appearance and spatial context. On various benchmarks, our method achieves better performance than state of the arts, especially on the classification task.


An Alternative Practice of Tropical Convolution to Traditional Convolutional Neural Networks. Convolutional neural networks (CNNs) have been used in many machine learning fields. In practical applications, the computational cost of convolutional neural networks is often high with the deepening of the network and the growth of data volume, mostly due to a large amount of multiplication operations of floating point numbers in convolution operations. To reduce the amount of multiplications, we propose a new type of CNNs called Tropical Convolutional Neural Networks (TCNNs) which are built on tropical convolutions in which the multiplications and additions in conventional convolutional layers are replaced by additions and min max operations respectively. In addition, since tropical convolution operators are essentially nonlinear operators, we expect TCNNs to have higher nonlinear fitting ability than conventional CNNs. In the experiments, we test and analyze several different architectures of TCNNs for image classification tasks in comparison with similar sized conventional CNNs. The results show that TCNN can achieve higher expressive power than ordinary convolutional layers on the MNIST and CIFAR10 image data set. In different noise environments, there are wins and losses in the robustness of TCNN and ordinary CNNs.


When does loss based prioritization fail?. Not all examples are created equal, but standard deep neural network training protocols treat each training point uniformly. Each example is propagated forward and backward through the network the same amount of times, independent of how much the example contributes to the learning protocol. Recent work has proposed ways to accelerate training by deviating from this uniform treatment. Popular methods entail up weighting examples that contribute more to the loss with the intuition that examples with low loss have already been learned by the model, so their marginal value to the training procedure should be lower. This view assumes that updating the model with high loss examples will be beneficial to the model. However, this may not hold for noisy, real world data. In this paper, we theorize and then empirically demonstrate that loss based acceleration methods degrade in scenarios with noisy and corrupted data. Our work suggests measures of example difficulty need to correctly separate out noise from other types of challenging examples.


Ultra wideband electrostrictive mechanical antenna. Conventional mechanical antennas provide a strategy in long wave communication with a surprisingly compact size below 1 1,000 of the wavelength. However, the narrow bandwidth and weak field intensity seriously hamper its practical applications. Here, we present a mechanical antenna based on the electrostrictive effect of PMN PT based relaxor ferroelectric ceramic to improve radiation capacity and achieve ultra wideband characteristics (10 kHz 1 MHz, the relative bandwidth is beyond 196 ). Determined by the different underlying mechanism, the mechanical antenna based on the electrostrictive effect exhibits excellent communication properties from traditional mechanical antennas. The functions of signal coding, transmitting, receiving, and decoding were experimentally demonstrated. This approach offers a promising way of constructing mechanical antennas for long wave communication.


Using Interaction Data to Predict Engagement with Interactive Media. Media is evolving from traditional linear narratives to personalised experiences, where control over information (or how it is presented) is given to individual audience members. Measuring and understanding audience engagement with this media is important in at least two ways (1) a post hoc understanding of how engaged audiences are with the content will help production teams learn from experience and improve future productions (2), this type of media has potential for real time measures of engagement to be used to enhance the user experience by adapting content on the fly. Engagement is typically measured by asking samples of users to self report, which is time consuming and expensive. In some domains, however, interaction data have been used to infer engagement. Fortuitously, the nature of interactive media facilitates a much richer set of interaction data than traditional media our research aims to understand if these data can be used to infer audience engagement. In this paper, we report a study using data captured from audience interactions with an interactive TV show to model and predict engagement. We find that temporal metrics, including overall time spent on the experience and the interval between events, are predictive of engagement. The results demonstrate that interaction data can be used to infer users' engagement during and after an experience, and the proposed techniques are relevant to better understand audience preference and responses.


Encoding Weights of Irregular Sparsity for Fixed to Fixed Model Compression. Even though fine grained pruning techniques achieve a high compression ratio, conventional sparsity representations (such as CSR) associated with irregular sparsity degrade parallelism significantly. Practical pruning methods, thus, usually lower pruning rates (by structured pruning) to improve parallelism. In this paper, we study fixed to fixed (lossless) encoding architecture algorithm to support fine grained pruning methods such that sparse neural networks can be stored in a highly regular structure. We first estimate the maximum compression ratio of encoding based compression using entropy. Then, as an effort to push the compression ratio to the theoretical maximum (by entropy), we propose a sequential fixed to fixed encoding scheme. We demonstrate that our proposed compression scheme achieves almost the maximum compression ratio for the Transformer and ResNet 50 pruned by various fine grained pruning methods.


The distance function in the presence of an obstacle. We study the Riemannian distance function from a fixed point (a point wise target) of Euclidean space in the presence of a compact obstacle bounded by a smooth hypersurface. First, we show that such a function is locally semiconcave with a fractional modulus of order one half and that, near the obstacle, this regularity is optimal. Then, in the Euclidean setting, we prove that the singularities of the distance function propagate, in the sense that each singular point belongs to a nontrivial singular continuum. Finally, we investigate the lack of differentiability of the distance function when a convex obstacle is present.


Understanding Worldwide Private Information Collection on Android. Mobile phones enable the collection of a wealth of private information, from unique identifiers (e.g., email addresses), to a user's location, to their text messages. This information can be harvested by apps and sent to third parties, which can use it for a variety of purposes. In this paper we perform the largest study of private information collection (PIC) on Android to date. Leveraging an anonymized dataset collected from the customers of a popular mobile security product, we analyze the flows of sensitive information generated by 2.1M unique apps installed by 17.3M users over a period of 21 months between 2018 and 2019. We find that 87.2 of all devices send private information to at least five different domains, and that actors active in different regions (e.g., Asia compared to Europe) are interested in collecting different types of information. The United States (62 of the total) and China (7 of total flows) are the countries that collect most private information. Our findings raise issues regarding data regulation, and would encourage policymakers to further regulate how private information is used by and shared among the companies and how accountability can be truly guaranteed.


Invariant Risk Minimisation for Cross Organism Inference Substituting Mouse Data for Human Data in Human Risk Factor Discovery. Human medical data can be challenging to obtain due to data privacy concerns, difficulties conducting certain types of experiments, or prohibitive associated costs. In many settings, data from animal models or in vitro cell lines are available to help augment our understanding of human data. However, this data is known for having low etiological validity in comparison to human data. In this work, we augment small human medical datasets with in vitro data and animal models. We use Invariant Risk Minimisation (IRM) to elucidate invariant features by considering cross organism data as belonging to different data generating environments. Our models identify genes of relevance to human cancer development. We observe a degree of consistency between varying the amounts of human and mouse data used, however, further work is required to obtain conclusive insights. As a secondary contribution, we enhance existing open source datasets and provide two uniformly processed, cross organism, homologue gene matched datasets to the community.


Contrastive Semi supervised Learning for ASR. Pseudo labeling is the most adopted method for pre training automatic speech recognition (ASR) models. However, its performance suffers from the supervised teacher model's degrading quality in low resource setups and under domain transfer. Inspired by the successes of contrastive representation learning for computer vision and speech applications, and more recently for supervised learning of visual objects, we propose Contrastive Semi supervised Learning (CSL). CSL eschews directly predicting teacher generated pseudo labels in favor of utilizing them to select positive and negative examples. In the challenging task of transcribing public social media videos, using CSL reduces the WER by 8 compared to the standard Cross Entropy pseudo labeling (CE PL) when 10hr of supervised data is used to annotate 75,000hr of videos. The WER reduction jumps to 19 under the ultra low resource condition of using 1hr labels for teacher supervision. CSL generalizes much better in out of domain conditions, showing up to 17 WER reduction compared to the best CE PL pre trained model.


Constructing Neural Network Based Models for Simulating Dynamical Systems. Dynamical systems see widespread use in natural sciences like physics, biology, chemistry, as well as engineering disciplines such as circuit analysis, computational fluid dynamics, and control. For simple systems, the differential equations governing the dynamics can be derived by applying fundamental physical laws. However, for more complex systems, this approach becomes exceedingly difficult. Data driven modeling is an alternative paradigm that seeks to learn an approximation of the dynamics of a system using observations of the true system. In recent years, there has been an increased interest in data driven modeling techniques, in particular neural networks have proven to provide an effective framework for solving a wide range of tasks. This paper provides a survey of the different ways to construct models of dynamical systems using neural networks. In addition to the basic overview, we review the related literature and outline the most significant challenges from numerical simulations that this modeling paradigm must overcome. Based on the reviewed literature and identified challenges, we provide a discussion on promising research areas.


Circular arc polygons, numerical conformal mappings, and moduli of quadrilaterals. We study numerical conformal mappings of planar Jordan domains with boundaries consisting of finitely many circular arcs and compute the moduli of quadrilaterals for these domains. Experimental error estimates are provided and, when possible, comparison to exact values or other methods are given. The main ingredients of the computation are boundary integral equations combined with the fast multipole method.


CASTing a Net Supporting Teachers with Search Technology. Past and current research has typically focused on ensuring that search technology for the classroom serves children. In this paper, we argue for the need to broaden the research focus to include teachers and how search technology can aid them. In particular, we share how furnishing a behind the scenes portal for teachers can empower them by providing a window into the spelling, writing, and concept connection skills of their students.


On the Importance and Shortcomings of Code Readability Metrics A Case Study on Reactive Programming. Well structured and readable source code is a pre requisite for maintainable software and successful collaboration among developers. Static analysis enables the automated extraction of code complexity and readability metrics which can be leveraged to highlight potential improvements in code to both attain software of high quality and reinforce good practices for developers as an educational tool. This assumes reliable readability metrics which are not trivial to obtain since code readability is somewhat subjective. Recent research has resulted in increasingly sophisticated models for predicting readability as perceived by humans primarily with a procedural and object oriented focus, while functional and declarative languages and language extensions advance as they often are said to lead to more concise and readable code. In this paper, we investigate whether the existing complexity and readability metrics reflect that wisdom or whether the notion of readability and its constituents requires overhaul in the light of programming language changes. We therefore compare traditional object oriented and reactive programming in terms of code complexity and readability in a case study. Reactive programming is claimed to increase code quality but few studies have substantiated these claims empirically. We refactored an object oriented open source project into a reactive candidate and compare readability with the original using cyclomatic complexity and two state of the art readability metrics. More elaborate investigations are required, but our findings suggest that both cyclomatic complexity and readability decrease significantly at the same time in the reactive candidate, which seems counter intuitive. We exemplify and substantiate why readability metrics may require adjustment to better suit popular programming styles other than imperative and object oriented to better match human expectations.


Trees and stationary reflection at double successors of regular cardinals. We obtain an array of consistency results concerning trees and stationary reflection at double successors of regular cardinals kappa , updating some classical constructions in the process. This includes models of mathsf CSR ( kappa ) wedge mathsf TP ( kappa ) (both with and without mathsf AP ( kappa ) ) and models of the conjunctions mathsf SR ( kappa ) wedge mathsf wTP ( kappa ) wedge mathsf AP ( kappa ) and neg mathsf AP ( kappa ) wedge mathsf SR ( kappa ) (the latter was originally obtained in joint work by Krueger and the first author cite GilKru 8fold , and is here given using different methods). Analogs of these results with the failure of mathsf SH ( kappa ) are given as well. Finally, we obtain all of our results with an arbitrarily large 2 kappa , applying recent joint work by Honzik and the third author.


Mixed graphs with cut vertices having exactly two positive eigenvalues. A mixed graph is obtained by orienting some edges of a simple graph. The positive inertia index of a mixed graph is defined as the number of positive eigenvalues of its Hermitian adjacency matrix, including multiplicities. This matrix was introduced by Liu and Li, independently by Guo and Mohar, in the study of graph energy. Recently, Yuan et al. characterized the mixed graphs with exactly one positive eigenvalue. In this paper, we study the positive inertia indices of mixed graphs and characterize the mixed graphs with cut vertices having positive inertia index 2.


SysMap A Lightweight Software Visualization Tool to Analyze the Software Evolution of a System. Software visualization helps to comprehend the system by providing a vivid illustration. The developers, as well as the analysts, can have a glance over the total system to understand the basic changes over time from a high level point of view through this technique. In recent years, many tools are proposed to visualize software based on different architectural metaphors, such as as solar system, city or park. Some of the solutions have just worked on system visualization where a few tried to explain the changes in software throughout different versions that still need heavy manual work. Keeping such limitations in mind, this paper proposes a lightweight tool named SysMap that takes the source codes of different versions of software systems, provides 3D illustrations of those systems and a graphical statistic of its evolution. To build the graphical element to represent the system, the source code has been studied to find out different software metrics. For experimentation, several open source java projects were chosen to find out the necessary information. Henceforth, this tool will surely increase the work efficiency of both the developer and analyst by reducing the manual effort and by providing the graphical view to comprehend the software evolution over time.


Some notes on continuity in convex optimization. We provide proof that the optimal value function of a convex parametrized optimization problem in Euclidean spaces is itself a convex function onto the extended real line.


Modelling Lips State Detection Using CNN for Non Verbal Communications. Vision based deep learning models can be promising for speech and hearing impaired and secret communications. While such non verbal communications are primarily investigated with hand gestures and facial expressions, no research endeavour is tracked so far for the lips state (i.e., open close) based interpretation translation system. In support of this development, this paper reports two new Convolutional Neural Network (CNN) models for lips state detection. Building upon two prominent lips landmark detectors, DLIB and MediaPipe, we simplify lips state model with a set of six key landmarks, and use their distances for the lips state classification. Thereby, both the models are developed to count the opening and closing of lips and thus, they can classify a symbol with the total count. Varying frame rates, lips movements and face angles are investigated to determine the effectiveness of the models. Our early experimental results demonstrate that the model with DLIB is relatively slower in terms of an average of 6 frames per second (FPS) and higher average detection accuracy of 95.25 . In contrast, the model with MediaPipe offers faster landmark detection capability with an average FPS of 20 and detection accuracy of 94.4 . Both models thus could effectively interpret the lips state for non verbal semantics into a natural language.


Optimizing FPGA based Accelerator Design for Large Scale Molecular Similarity Search. Molecular similarity search has been widely used in drug discovery to identify structurally similar compounds from large molecular databases rapidly. With the increasing size of chemical libraries, there is growing interest in the efficient acceleration of large scale similarity search. Existing works mainly focus on CPU and GPU to accelerate the computation of the Tanimoto coefficient in measuring the pairwise similarity between different molecular fingerprints. In this paper, we propose and optimize an FPGA based accelerator design on exhaustive and approximate search algorithms. On exhaustive search using BitBound folding, we analyze the similarity cutoff and folding level relationship with search speedup and accuracy, and propose a scalable on the fly query engine on FPGAs to reduce the resource utilization and pipeline interval. We achieve a 450 million compounds per second processing throughput for a single query engine. On approximate search using hierarchical navigable small world (HNSW), a popular algorithm with high recall and query speed. We propose an FPGA based graph traversal engine to utilize a high throughput register array based priority queue and fine grained distance calculation engine to increase the processing capability. Experimental results show that the proposed FPGA based HNSW implementation has a 103385 query per second (QPS) on the Chembl database with 0.92 recall and achieves a 35x speedup than the existing CPU implementation on average. To the best of our knowledge, our FPGA based implementation is the first attempt to accelerate molecular similarity search algorithms on FPGA and has the highest performance among existing approaches.


Strong approximation of Bessel processes. We consider the path approximation of Bessel processes and develop a new and efficient algorithm. This study is based on a recent work by the authors, on the path approximation of the Brownian motion, and on the construction of specific own techniques. It is part of the family of the so called varepsilon strong approximations. More precisely, our approach constructs jointly the sequences of exit times and corresponding exit positions of some well chosen domains, the construction of these domains being an important step. Based on this procedure, we emphasize an algorithm which is easy to implement. Moreover, we can develop the method for any dimension. We treat separately the integer dimension case and the non integer framework, each situation requiring appropriate techniques. In particular, for both situations, we show the convergence of the scheme and provide the control of the efficiency with respect to the small parameter varepsilon . We expand the theoretical part by a series of numerical developments.


FINE Samples for Learning with Noisy Labels. Modern deep neural networks (DNNs) become frail when the datasets contain noisy (incorrect) class labels. Robust techniques in the presence of noisy labels can be categorized into two folds developing noise robust functions or using noise cleansing methods by detecting the noisy data. Recently, noise cleansing methods have been considered as the most competitive noisy label learning algorithms. Despite their success, their noisy label detectors are often based on heuristics more than a theory, requiring a robust classifier to predict the noisy data with loss values. In this paper, we propose a novel detector for filtering label noise. Unlike most existing methods, we focus on each data's latent representation dynamics and measure the alignment between the latent distribution and each representation using the eigendecomposition of the data gram matrix. Our framework, coined as filtering noisy instances via their eigenvectors (FINE), provides a robust detector with derivative free simple methods having theoretical guarantees. Under our framework, we propose three applications of the FINE sample selection approach, semi supervised learning approach, and collaboration with noise robust loss functions. Experimental results show that the proposed methods consistently outperform corresponding baselines for all three applications on various benchmark datasets.


Preserving operators on semiprime f algebras. It is an open problem whether a separating operator acting between semiprime f algebras is a weighted composition operator ( cite AAB cite ). We prove that the answer is positive if and only if the separating operator is almost contractive. As a consequence, we can generalized and strengthen some well known results on separating operators.


A Temporal Logic for Asynchronous Hyperproperties. Hyperproperties are properties of computational systems that require more than one trace to evaluate, e.g., many information flow security and concurrency requirements. Where a trace property defines a set of traces, a hyperproperty defines a set of sets of traces. The temporal logics HyperLTL and HyperCTL have been proposed to express hyperproperties. However, their semantics are synchronous in the sense that all traces proceed at the same speed and are evaluated at the same position. This precludes the use of these logics to analyze systems whose traces can proceed at different speeds and allow that different traces take stuttering steps independently. To solve this problem in this paper, we propose an asynchronous variant of HyperLTL. On the negative side, we show that the model checking problem for this variant is undecidable. On the positive side, we identify a decidable fragment which covers a rich set of formulas with practical applications. We also propose two model checking algorithms that reduce our problem to the HyperLTL model checking problem in the synchronous semantics.


Effective Medium Theory with No Approximation for One Dimensional Metamaterials. Transfer matrix theory (TMT) is used to study the effective medium theory (EMT) of one dimensional metamaterials (1D MMs). 1D MMs with equal diagonal elements of periodic transfer matrix are defined as 1D perfect MMs (1D PMMs), which can be suitable for EMT with no approximation.


MobRecon Mobile Friendly Hand Mesh Reconstruction from Monocular Image. In this work, we propose a framework for single view hand mesh reconstruction, which can simultaneously achieve high reconstruction accuracy, fast inference speed, and temporal coherence. Specifically, for 2D encoding, we propose lightweight yet effective stacked structures. Regarding 3D decoding, we provide an efficient graph operator, namely depth separable spiral convolution. Moreover, we present a novel feature lifting module for bridging the gap between 2D and 3D representations. This module begins with a map based position regression (MapReg) block to integrate the merits of both heatmap encoding and position regression paradigms for improved 2D accuracy and temporal coherence. Furthermore, MapReg is followed by pose pooling and pose to vertex lifting approaches, which transform 2D pose encodings to semantic features of 3D vertices. Overall, our hand reconstruction framework, called MobRecon, comprises affordable computational costs and miniature model size, which reaches a high inference speed of 83FPS on Apple A14 CPU. Extensive experiments on popular datasets such as FreiHAND, RHD, and HO3Dv2 demonstrate that our MobRecon achieves superior performance on reconstruction accuracy and temporal coherence. Our code is publicly available at https github.com SeanChenxy HandMesh.


Multi Modal Mutual Information Maximization A Novel Approach for Unsupervised Deep Cross Modal Hashing. In this paper, we adopt the maximizing mutual information (MI) approach to tackle the problem of unsupervised learning of binary hash codes for efficient cross modal retrieval. We proposed a novel method, dubbed Cross Modal Info Max Hashing (CMIMH). First, to learn informative representations that can preserve both intra and inter modal similarities, we leverage the recent advances in estimating variational lower bound of MI to maximize the MI between the binary representations and input features and between binary representations of different modalities. By jointly maximizing these MIs under the assumption that the binary representations are modelled by multivariate Bernoulli distributions, we can learn binary representations, which can preserve both intra and inter modal similarities, effectively in a mini batch manner with gradient descent. Furthermore, we find out that trying to minimize the modality gap by learning similar binary representations for the same instance from different modalities could result in less informative representations. Hence, balancing between reducing the modality gap and losing modality private information is important for the cross modal retrieval tasks. Quantitative evaluations on standard benchmark datasets demonstrate that the proposed method consistently outperforms other state of the art cross modal retrieval methods.


Row Polymorphic Types for Strategic Rewriting. We present a type system for strategy languages that express program transformations as compositions of rewrite rules. Our row polymorphic type system assists compiler engineers to write correct strategies by statically rejecting non meaningful compositions of rewrites that otherwise would fail during rewriting at runtime. Furthermore, our type system enables reasoning about how rewriting transforms the shape of the computational program. We present a formalization of our language at its type system and demonstrate its practical use for expressing compiler optimization strategies. Our type system builds the foundation for many interesting future applications, including verifying the correctness of program transformations and synthesizing program transformations from specifications encoded as types.


Sharp Waiting Time Bounds for Multiserver Jobs. Multiserver jobs, which are jobs that occupy multiple servers simultaneously during service, are prevalent in today's computing clusters. But little is known about the delay performance of systems with multiserver jobs. We consider queueing models for multiserver jobs in a scaling regime where the total number of servers in the system becomes large and meanwhile both the system load and the number of servers that a job needs scale with the total number of servers. Prior work has derived upper bounds on the queueing probability in this scaling regime. However, without proper lower bounds, the existing results cannot be used to differentiate between policies. In this paper, we study the delay performance by establishing sharp bounds on the mean waiting time of multiserver jobs, where the waiting time of a job is the time spent in queueing rather than in service. We first consider the commonly used First Come First Serve (FCFS) policy and characterize the exact order of its mean waiting time. We then prove a lower bound on the mean waiting time of all policies, and demonstrate that there is an order gap between this lower bound and the mean waiting time under FCFS. We finally complement the lower bound with an achievability result we show that under a priority policy that we call P Priority, the mean waiting time achieves the order of the lower bound. This achievability result implies the tightness of the lower bound, the asymptotic optimality of P Priority, and the strict suboptimality of FCFS.


No More Handshaking How have COVID 19 pushed the expansion of computer mediated communication in Japanese idol culture?. In Japanese idol culture, meet and greet events where fans were allowed to handshake with an idol member for several seconds were regarded as its essential component until the spread of COVID 19. Now, idol groups are struggling in the transition of such events to computer mediated communication because these events had emphasized meeting face to face over communicating, as we can infer from their length of time. I anticipated that investigating this emerging transition would provide implications because their communication has a unique characteristic that is distinct from well studied situations, such as workplace communication and intimate relationships. Therefore, I first conducted a quantitative survey to develop a precise understanding of the transition, and based on its results, had semi structured interviews with idol fans about their perceptions of the transition. The survey revealed distinctive approaches, including one where fans gathered at a venue but were isolated from the idol member by an acrylic plate and talked via a video call. Then the interviews not only provided answers to why such an approach would be reasonable but also suggested the existence of a large gap between conventional offline events and emerging online events in their perceptions. Based on the results, I discussed how we can develop interaction techniques to support this transition and how we can apply it to other situations outside idol culture, such as computer mediated performing arts.


Use of Metamorphic Relations as Knowledge Carriers to Train Deep Neural Networks. Training multiple layered deep neural networks (DNNs) is difficult. The standard practice of using a large number of samples for training often does not improve the performance of a DNN to a satisfactory level. Thus, a systematic training approach is needed. To address this need, we introduce an innovative approach of using metamorphic relations (MRs) as "knowledge carriers" to train DNNs. Based on the concept of metamorphic testing and MRs (which play the role of a test oracle in software testing), we make use of the notion of metamorphic group of inputs as concrete instances of MRs (which are abstractions of knowledge) to train a DNN in a systematic and effective manner. To verify the viability of our training approach, we have conducted a preliminary experiment to compare the performance of two DNNs one trained with MRs and the other trained without MRs. We found that the DNN trained with MRs has delivered a better performance, thereby confirming that our approach of using MRs as knowledge carriers to train DNNs is promising. More work and studies, however, are needed to solidify and leverage this approach to generate widespread impact on effective DNN training.


Hamiltonicity via cohomology of right angled Artin groups. Let Gamma be a finite graph and let A( Gamma) be the corresponding right angled Artin group. We characterize the Hamiltonicity of Gamma via the structure of the cohomology algebra of A( Gamma) . In doing so, we define and develop a new canonical graph associated to a matrix, which as a consequence provides a novel perspective on the matrix determinant.


Subtractive mountain clustering algorithm applied to a chatbot to assist elderly people in medication intake. Errors in medication intake among elderly people are very common. One of the main causes for this is their loss of ability to retain information. The high amount of medicine intake required by the advanced age is another limiting factor. Thence, the design of an interactive aid system, preferably using natural language, to help the older population with medication is in demand. A chatbot based on a subtractive cluster algorithm, included in unsupervised learned models, is the chosen solution since the processing of natural languages is a necessary step in view to construct a chatbot able to answer questions that older people may pose upon themselves concerning a particular drug. In this work, the subtractive mountain clustering algorithm has been adapted to the problem of natural languages processing. This algorithm version allows for the association of a set of words into clusters. After finding the centre of every cluster the most relevant word, all the others are aggregated according to a defined metric adapted to the language processing realm. All the relevant stored information is processed, as well as the questions, by the algorithm. The correct processing of the text enables the chatbot to produce answers that relate to the posed queries. To validate the method, we use the package insert of a drug as the available information and formulate associated questions.


Sur un article de 1954 sign 'e N. Cuesta, une traduction. A translation from Spanish into French of a paper by N. Cuesta published in 1954. The paper deals mainly with partially, and totally, ordered sets. Two subjects are specially dealt with Construction of new ordered sets starting from a family of those. Completion of ordered sets by tools akin to Dedekind cuts. Curiously enough, the so called surreal numbers (later defined by Conway, in 1974) are already there, thirty years before.


SSFG Stochastically Scaling Features and Gradients for Regularizing Graph Convolutional Networks. Graph convolutional networks have been successfully applied in various graph based tasks. In a typical graph convolutional layer, node features are updated by aggregating neighborhood information. Repeatedly applying graph convolutions can cause the oversmoothing issue, i.e., node features at deep layers converge to similar values. Previous studies have suggested that oversmoothing is one of the major issues that restrict the performance of graph convolutional networks. In this paper, we propose a stochastic regularization method to tackle the oversmoothing problem. In the proposed method, we stochastically scale features and gradients (SSFG) by a factor sampled from a probability distribution in the training procedure. By explicitly applying a scaling factor to break feature convergence, the oversmoothing issue is alleviated. We show that applying stochastic scaling at the gradient level is complementary to that applied at the feature level to improve the overall performance. Our method does not increase the number of trainable parameters. When used together with ReLU, our SSFG can be seen as a stochastic ReLU activation function. We experimentally validate our SSFG regularization method on three commonly used types of graph networks. Extensive experimental results on seven benchmark datasets for four graph based tasks demonstrate that our SSFG regularization is effective in improving the overall performance of the baseline graph networks.


ASAT Adaptively Scaled Adversarial Training in Time Series. Adversarial training is a method for enhancing neural networks to improve the robustness against adversarial examples. Besides the security concerns of potential adversarial examples, adversarial training can also improve the performance of the neural networks, train robust neural networks, and provide interpretability for neural networks. In this work, we take the first step to introduce adversarial training in time series analysis by taking the finance field as an example. Rethinking existing researches of adversarial training, we propose the adaptively scaled adversarial training (ASAT) in time series analysis, by treating data at different time slots with time dependent importance weights. Experimental results show that the proposed ASAT can improve both the accuracy and the adversarial robustness of neural networks. Besides enhancing neural networks, we also propose the dimension wise adversarial sensitivity indicator to probe the sensitivities and importance of input dimensions. With the proposed indicator, we can explain the decision bases of black box neural networks.


Dynamic Information Security Management Capability Strategising for Organisational Performance. The increasing frequency, impact, consequence and sophistication of cybersecurity attacks is becoming a strategic concern for boards and executive management of organisations. Consequently, in addition to focusing on productivity and performance, organisations are prioritizing Information Security Management (ISM). However, research has revealed little or no conceptualisation of a dynamic ISM capability and its link to organisational performance. In this research, we set out to 1) define and describe an organisational level dynamic ISM capability, 2) to develop a strategic model that links resources with this dynamic capability, and then 3) empirically demonstrate how dynamic ISM capability contributes to firm performance. By drawing on Resource Based Theory (RBT) and Dynamic Capabilities View (DCV), we have developed the Dynamic ISM Capability model to address the identified gap. As we develop this research, we will empirically test this model to demonstrate causality between ISM capability and organisational performance.


Augmenting Customer Support with an NLP based Receptionist. In this paper, we show how a Portuguese BERT model can be combined with structured data in order to deploy a chatbot based on a finite state machine to create a conversational AI system that helps a real estate company to predict its client's contact motivation. The model achieves human level results in a dataset that contains 235 unbalanced labels. Then, we also show its benefits considering the business impact comparing it against classical NLP methods.


Evaluation of In Person Counseling Strategies To Develop Physical Activity Chatbot for Women. Artificial intelligence chatbots are the vanguard in technology based intervention to change people's behavior. To develop intervention chatbots, the first step is to understand natural language conversation strategies in human conversation. This work introduces an intervention conversation dataset collected from a real world physical activity intervention program for women. We designed comprehensive annotation schemes in four dimensions (domain, strategy, social exchange, and task focused exchange) and annotated a subset of dialogs. We built a strategy classifier with context information to detect strategies from both trainers and participants based on the annotation. To understand how human intervention induces effective behavior changes, we analyzed the relationships between the intervention strategies and the participants' changes in the barrier and social support for physical activity. We also analyzed how participant's baseline weight correlates to the amount of occurrence of the corresponding strategy. This work lays the foundation for developing a personalized physical activity intervention bot. The dataset and code are available at https github.com KaihuiLiang physical activity counseling


A in Seguran c c a dos Sistemas Governamentais Brasileiros Um Estudo de Caso em Sistemas Web e Redes Abertas. Whereas the world relies on computer systems for providing public services, there is a lack of academic work that systematically assess the security of government systems. To partially fill this gap, we conducted a security evaluation of publicly available systems from public institutions. We revisited OWASP top 10 and identified multiple vulnerabilities in deployed services by scanning public government networks. Overall, the unprotected services found have inadequate security level, which must be properly discussed and addressed.


Regret and Cumulative Constraint Violation Analysis for Distributed Online Constrained Convex Optimization. This paper considers the distributed online convex optimization problem with time varying constraints over a network of agents. This is a sequential decision making problem with two sequences of arbitrarily varying convex loss and constraint functions. At each round, each agent selects a decision from the decision set, and then only a portion of the loss function and a coordinate block of the constraint function at this round are privately revealed to this agent. The goal of the network is to minimize the network wide loss accumulated over time. Two distributed online algorithms with full information and bandit feedback are proposed. Both dynamic and static network regret bounds are analyzed for the proposed algorithms, and network cumulative constraint violation is used to measure constraint violation, which excludes the situation that strictly feasible constraints can compensate the effects of violated constraints. In particular, we show that the proposed algorithms achieve mathcal O (T max kappa,1 kappa ) static network regret and mathcal O (T 1 kappa 2 ) network cumulative constraint violation, where T is the time horizon and kappa in(0,1) is a user defined trade off parameter. Moreover, if the loss functions are strongly convex, then the static network regret bound can be reduced to mathcal O (T kappa ) . Finally, numerical simulations are provided to illustrate the effectiveness of the theoretical results.


Separating circles on the sphere by polygonal tilings. We say that a tiling separates discs of a packing in the Euclidean plane, if each tile contains exactly one member of the packing. It is a known elementary geometric problem to show that for each locally finite packing of circular discs, there exists a separating tiling with convex polygons. In this paper we show that this separating property remains true for circle packings on the sphere and in the hyperbolic plane. Moreover, we show that in the Euclidean plane circles are the only convex discs, whose packings with similar copies can be always separated by polygonal tilings. The analogous statement is not true on the sphere and it is not known in the hyperbolic plane.


Point Cloud based Hierarchical Deep Odometry Estimation. Processing point clouds using deep neural networks is still a challenging task. Most existing models focus on object detection and registration with deep neural networks using point clouds. In this paper, we propose a deep model that learns to estimate odometry in driving scenarios using point cloud data. The proposed model consumes raw point clouds in order to extract frame to frame odometry estimation through a hierarchical model architecture. Also, a local bundle adjustment variation of this model using LSTM layers is implemented. These two approaches are comprehensively evaluated and are compared against the state of the art.


Stock price forecast with deep learning. In this paper, we compare various approaches to stock price prediction using neural networks. We analyze the performance fully connected, convolutional, and recurrent architectures in predicting the next day value of S P 500 index based on its previous values. We further expand our analysis by including three different optimization techniques Stochastic Gradient Descent, Root Mean Square Propagation, and Adaptive Moment Estimation. The numerical experiments reveal that a single layer recurrent neural network with RMSprop optimizer produces optimal results with validation and test Mean Absolute Error of 0.0150 and 0.0148 respectively.


A multidimensional combustion model for oblique, wrinkled premixed flames. A new premixed turbulent combustion model is proposed. It is based on one dimensional (1D) filtering of density times progress variable and of the reaction source term of laminar premixed flame profiles using a filter kernel which reflects the variation of the slicing area of planar flame fronts as they move through multidimensional filter volumes. It is shown that these multidimensional effects qualitatively change the relation between the filtered reaction source term and the Favre filtered reaction progress variable compared to 1D filtering, particularly at large filter widths. Analytical results for the filtered quantities are achieved by approximating density times progress variable and reaction source term by suitable Ansatz functions. Filtered data from Direct Numerical Simulations (DNS) of statistically planar turbulent premixed flames at different free stream turbulence levels and heat release parameters is used to develop and validate the model. Two wrinkling factor models as function of filter width and subgrid turbulence level are proposed. For small filter widths up to two times the laminar flame thickness, minor effects from subgrid flame folding are observed. For larger filters, the filtered reaction source term rises linearly with filter width at a rate which increases with subgrid turbulence level. The modelled reaction source term as function of Favre averaged progress variable and filter width shows excellent agreement with filtered DNS data for all investigated free stream turbulence levels, filter widths and heat release parameters.


Analysis of Fractal Dimension of Mixed Riemann Liouville Fractional Integral. In this article, we investigate the fractal dimension of the graph of the mixed Riemann Liouville fractional integral for various choice of continuous functions on a rectangular region. We estimate bounds for the box dimension and the Hausdorff dimension of the graph of the mixed Riemann Liouville fractional integral of the functions which belong to the class of continuous functions and the class of Holder continuous functions. We also show that the box dimension of the graph of the mixed Riemann Liouville fractional integral of two dimensional continuous functions is also two. Furthermore, we give construction of unbounded variational continuous functions. Later, we prove that the box dimension and the Hausdorff dimension of the graph of the mixed Riemann Liouville fractional integral of unbounded variational continuous functions are also two.


Ray tracing Analysis for Cross polarization Scattering Diagnostic on MAST Upgrade Spherical Tokamak. A combined Doppler backscattering cross polarization scattering (DBS CPS) system is being deployed on MAST U, for simultaneous measurements of local density turbulence, turbulence flows, and magnetic turbulence. In this design, CPS shares the probing beam with the DBS and uses a separate parallel viewing receiver system. In this study, we utilize a modified GENRAY 3D ray tracing code, to simulate the propagation of the probing and scattered beams. The contributions of different scattering locations along the entire beam trajectories are considered, and the corresponding local tilde mathbf B wavenumbers are estimated using the wave vector matching criterion. The wavenumber ranges of the local tilde mathbf B that is detectable to the CPS system are explored for simulated L and H mode plasmas.


Localized mixing zone for Muskat bubbles and turned interfaces. We construct mixing solutions to the incompressible porous media equation starting from Muskat type data in the partially unstable regime. In particular, we consider bubble and turned type interfaces with Sobolev regularity. As a by product, we prove the continuation of the evolution of IPM after the Rayleigh Taylor and smoothness breakdown exhibited in 18,17 . At each time slice the space is split into three evolving domains two non mixing zones and a mixing zone which is localized in a neighborhood of the unstable region. In this way, we show the compatibility between the classical Muskat problem and the convex integration method.


When any three solutions are independent. Given an algebraic differential equation of order greater than one, it is shown that if there is any nontrivial algebraic relation amongst any number of distinct nonalgebraic solutions, along with their derivatives, then there is already such a relation between three solutions. In the autonomous situation when the equation is over constant parameters the assumption that the order be greater than one can be dropped, and a nontrivial algebraic relation exists already between two solutions. These theorems are deduced as an application of the following model theoretic result Suppose p is a stationary nonalgebraic type in the theory of differentially closed fields of characteristic zero if any three distinct realisations of p are independent then p is minimal. If the type is over the constants then minimality (and complete disintegratedness) already follow from knowing that any two realisations are independent. An algebro geometric formulation in terms of D varieties is given. The same methods yield also an analogous statement about families of compact K "ahler manifolds.


Actionable Cognitive Twins for Decision Making in Manufacturing. Actionable Cognitive Twins are the next generation Digital Twins enhanced with cognitive capabilities through a knowledge graph and artificial intelligence models that provide insights and decision making options to the users. The knowledge graph describes the domain specific knowledge regarding entities and interrelationships related to a manufacturing setting. It also contains information on possible decision making options that can assist decision makers, such as planners or logisticians. In this paper, we propose a knowledge graph modeling approach to construct actionable cognitive twins for capturing specific knowledge related to demand forecasting and production planning in a manufacturing plant. The knowledge graph provides semantic descriptions and contextualization of the production lines and processes, including data identification and simulation or artificial intelligence algorithms and forecasts used to support them. Such semantics provide ground for inferencing, relating different knowledge types creative, deductive, definitional, and inductive. To develop the knowledge graph models for describing the use case completely, systems thinking approach is proposed to design and verify the ontology, develop a knowledge graph and build an actionable cognitive twin. Finally, we evaluate our approach in two use cases developed for a European original equipment manufacturer related to the automotive industry as part of the European Horizon 2020 project FACTLOG.