Smoothness Matrices Beat Smoothness Constants Better Communication Compression Techniques for Distributed Optimization. Large scale distributed optimization has become the default tool for the training of supervised machine learning models with a large number of parameters and training data. Recent advancements in the field provide several mechanisms for speeding up the training, including em compressed communication , em variance reduction and em acceleration . However, none of these methods is capable of exploiting the inherently rich data dependent smoothness structure of the local losses beyond standard smoothness constants. In this paper, we argue that when training supervised models, em smoothness matrices information rich generalizations of the ubiquitous smoothness constants can and should be exploited for further dramatic gains, both in theory and practice. In order to further alleviate the communication burden inherent in distributed optimization, we propose a novel communication sparsification strategy that can take full advantage of the smoothness matrices associated with local losses. To showcase the power of this tool, we describe how our sparsification technique can be adapted to three distributed optimization algorithms DCGD, DIANA and ADIANA yielding significant savings in terms of communication complexity. The new methods always outperform the baselines, often dramatically so.


Abstract Factorization Theorems with Applications to Idempotent Factorizations. Let preceq be a preorder on a monoid H and s be an integer ge 2 . The preceq height of an x in H is the sup of the integers k ge 1 for which there is a (strictly) preceq decreasing sequence x 1, ldots,x k of preceq non units of H with x 1 x (with sup emptyset 0 ), where u in H is a preceq unit if u preceq 1 H preceq u and a preceq non unit otherwise. We say H is preceq artinian if there exists no preceq decreasing sequence x 1,x 2, ldots of elements of H and strongly preceq artinian if the preceq height of each element is finite. We establish that, if H is preceq artinian, then each preceq non unit x in H factors through the preceq irreducibles of degree s , where a preceq irreducible of degree s is a preceq non unit a in H that cannot be written as a product of s or fewer preceq non units each of which is (strictly) smaller than a with respect to preceq . In addition, we show that, if H is strongly preceq artinian, then x factors through the preceq quarks of H , where a preceq quark is a preceq min preceq non unit. In the process, we also obtain upper bounds for the length of a shortest factorization of x (into either preceq irreducible of degree s or preceq quarks) in terms of its preceq height. Next, we specialize these abstract results to the case in which H is the multiplicative submonoid of a ring R formed by the zero divisors and the identity 1 R , and preceq is the preorder on H defined by a preceq b iff r R(1 R b) subseteq r R(1 R a) , where r R( cdot) denotes a right annihilator. We can thus recover and improve on classical theorems of J.A. Erdos (1967), R.J.H. Dawlings (1981), and J. Fountain (1991) on idempotent factorizations in the endomorphism ring of a free module of finite rank over a skew field or a commutative DVD.


Proof of Concept of High Temperature Superconducting Screens for Magnetic Field Error Cancellation in Accelerator Magnets. Accelerators magnets must have minimal magnetic field imperfections for reducing particle beam instabilities. In the case of coils made of high temperature superconducting (HTS) tapes, the field imperfections from persistent currents need to be carefully evaluated. In this paper we study the use of superconducting screens based on HTS tapes for reducing the magnetic field imperfections in accelerator magnets. The screens exploit the magnetization by persistent currents to cancel out the magnetic field error. The screens are aligned with the main field components, such that only the undesired field components are compensated. The screens are passive, self regulating, and do not require any external source of energy. Measurements in liquid nitrogen at 77 Kelvin show for dipole field configurations a significant reduction of the magnetic field error up to a factor of four. The residual error is explained via numerical simulations, accounting for the geometrical imperfections in the HTS screens, thus achieving satisfactory agreement with experimental results. Simulations show that if screens are increased in width and thickness, and operated at 4.5 Kelvin, field errors may be eliminated almost entirely for the typical excitation cycles of accelerator magnets.


Can Machine Learning Tools Support the Identification of Sustainable Design Leads From Product Reviews? Opportunities and Challenges. The increasing number of product reviews posted online is a gold mine for designers to know better about the products they develop, by capturing the voice of customers, and to improve these products accordingly. In the meantime, product design and development have an essential role in creating a more sustainable future. With the recent advance of artificial intelligence techniques in the field of natural language processing, this research aims to develop an integrated machine learning solution to obtain sustainable design insights from online product reviews automatically. In this paper, the opportunities and challenges offered by existing frameworks including Python libraries, packages, as well as state of the art algorithms like BERT are discussed, illustrated, and positioned along an ad hoc machine learning process. This contribution discusses the opportunities to reach and the challenges to address for building a machine learning pipeline, in order to get insights from product reviews to design more sustainable products, including the five following stages, from the identification of sustainability related reviews to the interpretation of sustainable design leads data collection, data formatting, model training, model evaluation, and model deployment. Examples of sustainable design insights that can be produced out of product review mining and processing are given. Finally, promising lines for future research in the field are provided, including case studies putting in parallel standard products with their sustainable alternatives, to compare the features valued by customers and to generate in fine relevant sustainable design leads.


On the regularized risk of distributionally robust learning over deep neural networks. In this paper we explore the relation between distributionally robust learning and different forms of regularization to enforce robustness of deep neural networks. In particular, starting from a concrete min max distributionally robust problem, and using tools from optimal transport theory, we derive first order and second order approximations to the distributionally robust problem in terms of appropriate regularized risk minimization problems. In the context of deep ResNet models, we identify the structure of the resulting regularization problems as mean field optimal control problems where the number and dimension of state variables is within a dimension free factor of the dimension of the original unrobust problem. Using the Pontryagin maximum principles associated to these problems we motivate a family of scalable algorithms for the training of robust neural networks. Our analysis recovers some results and algorithms known in the literature (in settings explained throughout the paper) and provides many other theoretical and algorithmic insights that to our knowledge are novel. In our analysis we employ tools that we deem useful for a future analysis of more general adversarial learning problems.


MindReader Recommendation over Knowledge Graph Entities with Explicit User Ratings. Knowledge Graphs (KGs) have been integrated in several models of recommendation to augment the informational value of an item by means of its related entities in the graph. Yet, existing datasets only provide explicit ratings on items and no information is provided about user opinions of other (non recommendable) entities. To overcome this limitation, we introduce a new dataset, called the MindReader, providing explicit user ratings both for items and for KG entities. In this first version, the MindReader dataset provides more than 102 thousands explicit ratings collected from 1,174 real users on both items and entities from a KG in the movie domain. This dataset has been collected through an online interview application that we also release open source. As a demonstration of the importance of this new dataset, we present a comparative study of the effect of the inclusion of ratings on non item KG entities in a variety of state of the art recommendation models. In particular, we show that most models, whether designed specifically for graph data or not, see improvements in recommendation quality when trained on explicit non item ratings. Moreover, for some models, we show that non item ratings can effectively replace item ratings without loss of recommendation quality. This finding, thanks also to an observed greater familiarity of users towards common KG entities than towards long tail items, motivates the use of KG entities for both warm and cold start recommendations.


Tutorial on Deep Learning for Human Activity Recognition. Activity recognition systems that are capable of estimating human activities from wearable inertial sensors have come a long way in the past decades. Not only have state of the art methods moved away from feature engineering and have fully adopted end to end deep learning approaches, best practices for setting up experiments, preparing datasets, and validating activity recognition approaches have similarly evolved. This tutorial was first held at the 2021 ACM International Symposium on Wearable Computers (ISWC'21) and International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp'21). The tutorial, after a short introduction in the research field of activity recognition, provides a hands on and interactive walk through of the most important steps in the data pipeline for the deep learning of human activities. All presentation slides shown during the tutorial, which also contain links to all code exercises, as well as the link of the GitHub page of the tutorial can be found on https mariusbock.github.io dl for har


Thermal experiments for fractured rock characterization theoretical analysis and inverse modeling. Field scale properties of fractured rocks play crucial role in many subsurface applications, yet methodologies for identification of the statistical parameters of a discrete fracture network (DFN) are scarce. We present an inversion technique to infer two such parameters, fracture density and fractal dimension, from cross borehole thermal experiments data. It is based on a particle based heat transfer model, whose evaluation is accelerated with a deep neural network (DNN) surrogate that is integrated into a grid search. The DNN is trained on a small number of heat transfer model runs, and predicts the cumulative density function of the thermal field. The latter is used to compute fine posterior distributions of the (to be estimated) parameters. Our synthetic experiments reveal that fracture density is well constrained by data, while fractal dimension is harder to determine. Adding non uniform prior information related to the DFN connectivity improves the inference of this parameter.


On the number of critical points of the second eigenfunction of the Laplacian in convex planar domains. In this paper we consider the second eigenfunction of the Laplacian with Dirichlet boundary conditions in convex domains. If the domain has emph large eccentricity then the eigenfunction has emph exactly two nondegenerate critical points (of course they are one maximum and one minimum). The proof uses some estimates proved by Jerison ( Jer95a ) and Grieser Jerison ( GJ96 ) jointly with a topological degree argument. Analogous results for higher order eigenfunctions are proved in rectangular like domains considered in GJ09 .


Effects of Number of Filters of Convolutional Layers on Speech Recognition Model Accuracy. Inspired by the progress of the End to End approach 1 , this paper systematically studies the effects of Number of Filters of convolutional layers on the model prediction accuracy of CNN RNN (Convolutional Neural Networks adding to Recurrent Neural Networks) for ASR Models (Automatic Speech Recognition). Experimental results show that only when the CNN Number of Filters exceeds a certain threshold value is adding CNN to RNN able to improve the performance of the CNN RNN speech recognition model, otherwise some parameter ranges of CNN can render it useless to add the CNN to the RNN model. Our results show a strong dependency of word accuracy on the Number of Filters of convolutional layers. Based on the experimental results, the paper suggests a possible hypothesis of Sound 2 Vector Embedding (Convolutional Embedding) to explain the above observations. Based on this Embedding hypothesis and the optimization of parameters, the paper develops an End to End speech recognition system which has a high word accuracy but also has a light model weight. The developed LVCSR (Large Vocabulary Continuous Speech Recognition) model has achieved quite a high word accuracy of 90.2 only by its Acoustic Model alone, without any assistance from intermediate phonetic representation and any Language Model. Its acoustic model contains only 4.4 million weight parameters, compared to the 35 68 million acoustic model weight parameters in DeepSpeech2 2 (one of the top state of the art LVCSR models) which can achieve a word accuracy of 91.5 . The light weighted model is good for improving the transcribing computing efficiency and also useful for mobile devices, Driverless Vehicles, etc. Our model weight is reduced to 10 the size of DeepSpeech2, but our model accuracy remains close to that of DeepSpeech2. If combined with a Language Model, our LVCSR system is able to achieve 91.5 word accuracy.


Harmonic Persistent Homology. We introduce harmonic persistent homology spaces for filtrations of finite simplicial complexes. As a result we can associate concrete subspaces of cycles to each bar of the barcode of the filtration. We prove stability of the harmonic persistent homology subspaces under small perturbations of functions defining them. We relate the notion of "essential simplices" introduced in an earlier work to identify simplices which play a significant role in the birth of a bar, with that of harmonic persistent homology. We prove that the harmonic representatives of simple bars maximizes the "relative essential content" amongst all representatives of the bar, where the relative essential content is the weight a particular cycle puts on the set of essential simplices.


PerceMon Online Monitoring for Perception Systems. Perception algorithms in autonomous vehicles are vital for the vehicle to understand the semantics of its surroundings, including detection and tracking of objects in the environment. The outputs of these algorithms are in turn used for decision making in safety critical scenarios like collision avoidance, and automated emergency braking. Thus, it is crucial to monitor such perception systems at runtime. However, due to the high level, complex representations of the outputs of perception systems, it is a challenge to test and verify these systems, especially at runtime. In this paper, we present a runtime monitoring tool, PerceMon that can monitor arbitrary specifications in Timed Quality Temporal Logic (TQTL) and its extensions with spatial operators. We integrate the tool with the CARLA autonomous vehicle simulation environment and the ROS middleware platform while monitoring properties on state of the art object detection and tracking algorithms.


Families of almost complex structures and transverse (p,p) forms. An em almost p K "ahler manifold is a triple (M,J, Omega) , where (M,J) is an almost complex manifold of real dimension 2n and Omega is a closed real tranverse (p,p) form on (M,J) , where 1 leq p leq n . When J is integrable, almost p K "ahler manifolds are called p em K "ahler manifolds . We produce families of almost p K "ahler structures (J t, Omega t) on C 3 , C 4 , and on the real torus mathbb T 6 , arising as deformations of K "ahler structures (J 0,g 0, omega 0) , such that the almost complex structures J t cannot be locally compatible with any symplectic form for t neq 0 . Furthermore, examples of special compact nilmanifolds with and without almost p K "ahler structures are presented.


IID GAN an IID Sampling Perspective for Regularizing Mode Collapse. Despite its success, generative adversarial networks (GANs) still suffer from mode collapse, namely the generator can only map latent variables to a partial set of modes of the target distribution. In this paper, we analyze and try to regularize this issue with an independent and identically distributed (IID) sampling perspective and emphasize that holding the IID property for generation for target distribution (i.e. real distribution) can naturally avoid mode collapse. This is based on the basic IID assumption for real data in machine learning. However, though the source samples mathbf z obey IID, the generations G( mathbf z ) may not necessarily be IID from the target distribution. Based on this observation, we propose a necessary condition of IID generation and provide a new loss to encourage the closeness between the inverse source of real data and the Gaussian source in the latent space to regularize the generation to be IID from the target distribution. The logic is that the inverse samples from target data should also be IID in the source distribution. Experiments on both synthetic and real world data show the effectiveness of our model.


An Experimental Study of Permanently Stored Learned Clauses. Modern CDCL SAT solvers learn clauses rapidly, and an important heuristic is the clause deletion scheme. Most current solvers have two (or more) stores of clauses. One has valuable'' clauses which are never deleted. Most learned clauses are added to the other, with an aggressive deletion strategy to restrict its size. Recent solvers in the MapleSAT family, have comparatively complex deletion scheme, and perform well. Many solvers store only binary clauses permanently, but MapleLCMDistChronoBT stores clauses with small LBD permanently. We report an experimental study of the permanent clause store in MapleLCMDistChronoBT. We observe that this store can get quite large, but several methods for limiting its size reduced performance. We also show that alternate size and LBD based criteria improve performance, while still having large permanent stores. In particular, saving clauses up to size 8, and adding small numbers of high centrality clauses, both improved performance, with the best improvement using both methods.


Finite dimensional representations of map superalgebras. We obtain a complete classification of all finite dimensional irreducible modules over classical map superalgebras, provide formulas for their (super)characters and a description of their extension groups. Furthermore, we describe the block decomposition of the category of finite dimensional modules for such map superalgebras. As an application, we specialize our results to the case of loop superalgebras in order to obtain a classification of finite dimensional irreducible modules and block decomposition of the category of finite dimensional modules over affine Lie superalgebras.


A New Sentence Ordering Method Using BERT Pretrained Model. Building systems with capability of natural language understanding (NLU) has been one of the oldest areas of AI. An essential component of NLU is to detect logical succession of events contained in a text. The task of sentence ordering is proposed to learn succession of events with applications in AI tasks. The performance of previous works employing statistical methods is poor, while the neural networks based approaches are in serious need of large corpora for model learning. In this paper, we propose a method for sentence ordering which does not need a training phase and consequently a large corpus for learning. To this end, we generate sentence embedding using BERT pre trained model and measure sentence similarity using cosine similarity score. We suggest this score as an indicator of sequential events' level of coherence. We finally sort the sentences through brute force search to maximize overall similarities of the sequenced sentences. Our proposed method outperformed other baselines on ROCStories, a corpus of 5 sentence human made stories. The method is specifically more efficient than neural network based methods when no huge corpus is available. Among other advantages of this method are its interpretability and needlessness to linguistic knowledge.


Multimodal Adversarially Learned Inference with Factorized Discriminators. Learning from multimodal data is an important research topic in machine learning, which has the potential to obtain better representations. In this work, we propose a novel approach to generative modeling of multimodal data based on generative adversarial networks. To learn a coherent multimodal generative model, we show that it is necessary to align different encoder distributions with the joint decoder distribution simultaneously. To this end, we construct a specific form of the discriminator to enable our model to utilize data efficiently, which can be trained constrastively. By taking advantage of contrastive learning through factorizing the discriminator, we train our model on unimodal data. We have conducted experiments on the benchmark datasets, whose promising results show that our proposed approach outperforms the state of the art methods on a variety of metrics. The source code will be made publicly available.


Frontiers in Evolutionary Computation A Workshop Report. In July of 2021, the Santa Fe Institute hosted a workshop on evolutionary computation as part of its Foundations of Intelligence in Natural and Artificial Systems project. This project seeks to advance the field of artificial intelligence by promoting interdisciplinary research on the nature of intelligence. The workshop brought together computer scientists and biologists to share their insights about the nature of evolution and the future of evolutionary computation. In this report, we summarize each of the talks and the subsequent discussions. We also draw out a number of key themes and identify important frontiers for future research.


Optimal Low Thrust Orbit Transfers Made Easy A Direct Approach. The optimization of low thrust, multi revolution orbit transfer trajectories is often regarded as a difficult problem in modern astrodynamics. In this paper, a flexible and computationally efficient approach is presented for the optimization of low thrust orbit transfers under eclipse constraints. The proposed approach leverages a new dynamic model of the orbital motion and a Lyapunov based initial guess generation scheme that is very easy to tune. A multi objective, single phase formulation of the optimal control problem is devised, which provides a convenient way to trade off fuel consumption and time of flight. A distinctive feature of such a formulation is that it requires no prior information about the structure of the optimal solution. Simulation results for two benchmark orbit transfer scenarios indicate that minimum time, minimum fuel and mixed time fuel optimal instances of the control problem can be readily solved via direct collocation, while incurring a significantly lower computational demand with respect to existing techniques.


Probabilistic Contrastive Loss for Self Supervised Learning. This paper proposes a probabilistic contrastive loss function for self supervised learning. The well known contrastive loss is deterministic and involves a temperature hyperparameter that scales the inner product between two normed feature embeddings. By reinterpreting the temperature hyperparameter as a quantity related to the radius of the hypersphere, we derive a new loss function that involves a confidence measure which quantifies uncertainty in a mathematically grounding manner. Some intriguing properties of the proposed loss function are empirically demonstrated, which agree with human like predictions. We believe the present work brings up a new prospective to the area of contrastive learning.


Time profile of temperature rise in assemblies of nanomagnets. We compute the heat generated by (non interacting) nanomagnets subjected to an alternating magnetic field (AMF) and study its transfer to the hosting medium and environment. For the first task, we compute the heat generated by the nanomagnets (or the specific absorption rate mathfrak S ) using the ac susceptibility in the linear regime. For the second task, the loss of heat to the environment is modeled with the help of a balance (macroscopic) equation based on Newton's law of cooling. This equation is solved both numerically and analytically for a generic ferrofluid and the analytical solution renders a very good approximation to the general balance equation. Then, we investigate the effects of AMF frequency and amplitude on the temperature elevation during its temporal evolution. Finally, using the available experimental data for maghemite and magnetite ferrofluids, we discuss the behavior of Newton's heat transfer coefficient in terms of the AMF amplitude and frequency. These results could trigger experimental investigations of this coefficient which characterizes the rate of heating in a ferrofluid, with the aim to build more refined models for the mechanisms of heat generation and its diffusion in ferrofluids used in magnetic hyperthermia.


Zero Sum Games for Continuous time Markov Decision Processes with Risk Sensitive Average Cost Criterion. We consider zero sum stochastic games for continuous time Markov decision processes with risk sensitive average cost criterion. Here the transition and cost rates may be unbounded. We prove the existence of the value of the game and a saddle point equilibrium in the class of all stationary strategies under a Lyapunov stability condition. This is accomplished by establishing the existence of a principal eigenpair for the corresponding Hamilton Jacobi Isaacs (HJI) equation. This in turn is established by using the nonlinear version of Krein Rutman theorem. We then obtain a characterization of the saddle point equilibrium in terms of the corresponding HJI equation. Finally, we use a controlled population system to illustrate results.


A Proposition Level Clustering Approach for Multi Document Summarization. Text clustering methods were traditionally incorporated into multi document summarization (MDS) as a means for coping with considerable information repetition. Clusters were leveraged to indicate information saliency and to avoid redundancy. These methods focused on clustering sentences, even though closely related sentences also usually contain non aligning information. In this work, we revisit the clustering approach, grouping together propositions for more precise information alignment. Specifically, our method detects salient propositions, clusters them into paraphrastic clusters, and generates a representative sentence for each cluster by fusing its propositions. Our summarization method improves over the previous state of the art MDS method in the DUC 2004 and TAC 2011 datasets, both in automatic ROUGE scores and human preference.


Minimal pairs, inertia degrees, ramification degrees and implicit constant fields. An extension (K(X) K, v) of valued fields is said to be valuation transcendental if we have equality in the Abhyankar inequality. Minimal pairs of definition are fundamental objects in the investigation of valuation transcendental extensions. In this article, we associate a uniquely determined positive integer with a valuation transcendental extension. This integer is defined via a chosen minimal pair of definition, but it is later shown to be independent of the choice. Further, we show that this integer encodes important information regarding the implicit constant field of the extension (K(X) K, v).


Parameter sensitivity analysis for mean field games of production. We study a mean field game system introduced by Chan and Sircar (AMO, 2015) to model production of an exhaustible resource. In particular, we study the sensitivity of the solution with respect to a parameter varepsilon , which measures the degree to which producers are interchangeable. We prove that on some interval 0, varepsilon 0 , where varepsilon 0 0 , the solution is infinitely differentiable with respect to varepsilon . The result is based on a set of new a priori estimates for forward backward systems of linear partial differential equations.


Identification of Fine Grained Location Mentions in Crisis Tweets. Identification of fine grained location mentions in crisis tweets is central in transforming situational awareness information extracted from social media into actionable information. Most prior works have focused on identifying generic locations, without considering their specific types. To facilitate progress on the fine grained location identification task, we assemble two tweet crisis datasets and manually annotate them with specific location types. The first dataset contains tweets from a mixed set of crisis events, while the second dataset contains tweets from the global COVID 19 pandemic. We investigate the performance of state of the art deep learning models for sequence tagging on these datasets, in both in domain and cross domain settings.


Foureye Defensive Deception based on Hypergame Theory Against Advanced Persistent Threats. Defensive deception techniques have emerged as a promising proactive defense mechanism to mislead an attacker and thereby achieve attack failure. However, most game theoretic defensive deception approaches have assumed that players maintain consistent views under uncertainty. They do not consider players' possible, subjective beliefs formed due to asymmetric information given to them. In this work, we formulate a hypergame between an attacker and a defender where they can interpret the same game differently and accordingly choose their best strategy based on their respective beliefs. This gives a chance for defensive deception strategies to manipulate an attacker's belief, which is the key to the attacker's decision making. We consider advanced persistent threat (APT) attacks, which perform multiple attacks in the stages of the cyber kill chain where both the attacker and the defender aim to select optimal strategies based on their beliefs. Through extensive simulation experiments, we demonstrated how effectively the defender can leverage defensive deception techniques while dealing with multi staged APT attacks in a hypergame in which the imperfect information is reflected based on perceived uncertainty, cost, and expected utilities of both attacker and defender, the system lifetime (i.e., mean time to security failure), and improved false positive rates in detecting attackers.


Unsupervised Sentiment Analysis by Transferring Multi source Knowledge. Sentiment analysis (SA) is an important research area in cognitive computation thus in depth studies of patterns of sentiment analysis are necessary. At present, rich resource data based SA has been well developed, while the more challenging and practical multi source unsupervised SA (i.e. a target domain SA by transferring from multiple source domains) is seldom studied. The challenges behind this problem mainly locate in the lack of supervision information, the semantic gaps among domains (i.e., domain shifts), and the loss of knowledge. However, existing methods either lack the distinguishable capacity of the semantic gaps among domains or lose private knowledge. To alleviate these problems, we propose a two stage domain adaptation framework. In the first stage, a multi task methodology based shared private architecture is employed to explicitly model the domain common features and the domain specific features for the labeled source domains. In the second stage, two elaborate mechanisms are embedded in the shared private architecture to transfer knowledge from multiple source domains. The first mechanism is a selective domain adaptation (SDA) method, which transfers knowledge from the closest source domain. And the second mechanism is a target oriented ensemble (TOE) method, in which knowledge is transferred through a well designed ensemble method. Extensive experiment evaluations verify that the performance of the proposed framework outperforms unsupervised state of the art competitors. What can be concluded from the experiments is that transferring from very different distributed source domains may degrade the target domain performance, and it is crucial to choose the proper source domains to transfer from.


Maximal characterisation of local Hardy spaces on locally doubling manifolds. We prove a radial maximal function characterisation of the local atomic Hardy space h 1(M) on a Riemannian manifold M with positive injectivity radius and Ricci curvature bounded from below. As a consequence, we show that an integrable function belongs to h 1(M) if and only if either its local heat maximal function or its local Poisson maximal function are integrable. A key ingredient is a decomposition of H "older cut offs in terms of an appropriate class of approximations of the identity, which we obtain on arbitrary Ahlfors regular metric measure spaces and generalises a previous result of A. Uchiyama.


Turing universality of the incompressible Euler equations and a conjecture of Moore. In this article we construct a compact Riemannian manifold of high dimension on which the time dependent Euler equations are Turing complete. More precisely, the halting of any Turing machine with a given input is equivalent to a certain global solution of the Euler equations entering a certain open set in the space of divergence free vector fields. In particular, this implies the undecidability of whether a solution to the Euler equations with an initial datum will reach a certain open set or not in the space of divergence free fields. This result goes one step further in Tao's programme to study the blow up problem for the Euler and Navier Stokes equations using fluid computers. As a remarkable spin off, our method of proof allows us to give a counterexample to a conjecture of Moore dating back to 1998 on the non existence of analytic maps on compact manifolds that are Turing complete.


Multiple Graph Learning for Scalable Multi view Clustering. Graph based multi view clustering has become an active topic due to the efficiency in characterizing both the complex structure and relationship between multimedia data. However, existing methods have the following shortcomings (1) They are inefficient or even fail for graph learning in large scale due to the graph construction and eigen decomposition. (2) They cannot well exploit both the complementary information and spatial structure embedded in graphs of different views. To well exploit complementary information and tackle the scalability issue plaguing graph based multi view clustering, we propose an efficient multiple graph learning model via a small number of anchor points and tensor Schatten p norm minimization. Specifically, we construct a hidden and tractable large graph by anchor graph for each view and well exploit complementary information embedded in anchor graphs of different views by tensor Schatten p norm regularizer. Finally, we develop an efficient algorithm, which scales linearly with the data size, to solve our proposed model. Extensive experimental results on several datasets indicate that our proposed method outperforms some state of the art multi view clustering algorithms.


Connectionism, Complexity, and Living Systems a comparison of Artificial and Biological Neural Networks. While Artificial Neural Networks (ANNs) have yielded impressive results in the realm of simulated intelligent behavior, it is important to remember that they are but sparse approximations of Biological Neural Networks (BNNs). We go beyond comparison of ANNs and BNNs to introduce principles from BNNs that might guide the further development of ANNs as embodied neural models. These principles include representational complexity, complex network structure energetics, and robust function. We then consider these principles in ways that might be implemented in the future development of ANNs. In conclusion, we consider the utility of this comparison, particularly in terms of building more robust and dynamic ANNs. This even includes constructing a morphology and sensory apparatus to create an embodied ANN, which when complemented with the organizational and functional advantages of BNNs unlocks the adaptive potential of lifelike networks.


Mechanism of Indirect Photon induced Desorption at the Water Ice Surface. Electronic excitations near the surface of water ice lead to the desorption of adsorbed molecules, through a so far debated mechanism. A systematic study of photon induced indirect desorption, revealed by the spectral dependence of the desorption (7 to 13 eV), is conducted for Ar, Kr, N2, and CO adsorbed on H2O or D2O amorphous ices. The mass and isotopic dependence and the increase of intrinsic desorption efficiency with photon energy all point to a mechanism of desorption induced by collisions between adsorbates and energetic H or D atoms, produced by photodissociation of water. This constitutes a direct and unambiguous experimental demonstration of the mechanism of indirect desorption of weakly adsorbed species on water ice, and sheds new light on the possibility of this mechanism in other systems. It also has implications for the description of photon induced desorption in astrochemical models.


Remarks on the Pair Correlation Statistic of Kronecker Sequences and Lattice Point Counting. In this short note, we reformulate the task of calculating the pair correlation statistics of a Kronecker sequence as a lattice point counting problem. This can be done analogously to the lattice based approach which was used to (re )prove the famous three gap property for Kronecker sequences. We show that recently developed lattice point counting techniques can then be applied to derive that a certain class of Kronecker sequences have beta pair correlations for all 0 beta 1 .


RLTutor Reinforcement Learning Based Adaptive Tutoring System by Modeling Virtual Student with Fewer Interactions. A major challenge in the field of education is providing review schedules that present learned items at appropriate intervals to each student so that memory is retained over time. In recent years, attempts have been made to formulate item reviews as sequential decision making problems to realize adaptive instruction based on the knowledge state of students. It has been reported previously that reinforcement learning can help realize mathematical models of students learning strategies to maintain a high memory rate. However, optimization using reinforcement learning requires a large number of interactions, and thus it cannot be applied directly to actual students. In this study, we propose a framework for optimizing teaching strategies by constructing a virtual model of the student while minimizing the interaction with the actual teaching target. In addition, we conducted an experiment considering actual instructions using the mathematical model and confirmed that the model performance is comparable to that of conventional teaching methods. Our framework can directly substitute mathematical models used in experiments with human students, and our results can serve as a buffer between theoretical instructional optimization and practical applications in e learning systems.


Detecting Potentially Harmful and Protective Suicide related Content on Twitter A Machine Learning Approach. Research shows that exposure to suicide related news media content is associated with suicide rates, with some content characteristics likely having harmful and others potentially protective effects. Although good evidence exists for a few selected characteristics, systematic large scale investigations are missing in general, and in particular for social media data. We apply machine learning methods to automatically label large quantities of Twitter data. We developed a novel annotation scheme that classifies suicide related tweets into different message types and problem vs. solution focused perspectives. We then trained a benchmark of machine learning models including a majority classifier, an approach based on word frequency (TF IDF with a linear SVM) and two state of the art deep learning models (BERT, XLNet). The two deep learning models achieved the best performance in two classification tasks First, we classified six main content categories, including personal stories about either suicidal ideation and attempts or coping, calls for action intending to spread either problem awareness or prevention related information, reportings of suicide cases, and other suicide related and off topic tweets. The deep learning models reach accuracy scores above 73 on average across the six categories, and F1 scores in between 69 and 85 for all but the suicidal ideation and attempts category (55 ). Second, in separating postings referring to actual suicide from off topic tweets, they correctly labelled around 88 of tweets, with BERT achieving F1 scores of 93 and 74 for the two categories. These classification performances are comparable to the state of the art on similar tasks. By making data labeling more efficient, this work enables future large scale investigations on harmful and protective effects of various kinds of social media content on suicide rates and on help seeking behavior.


Two Class (r,k) Coloring Coloring with Service Guarantees. This paper introduces the Two Class ( r , k ) Coloring problem Given a fixed number of k colors, such that only r of these k colors allow conflicts, what is the minimal number of conflicts incurred by an optimal coloring of the graph? We establish that the family of Two Class ( r , k ) Coloring problems is NP complete for any k geq 2 when (r, k) neq (0,2) . Furthermore, we show that Two Class ( r , k ) Coloring for k geq 2 colors with one ( r 1 ) relaxed color cannot be approximated to any constant factor ( notin APX). Finally, we show that Two Class ( r , k ) Coloring with k geq r geq 2 colors is APX complete.


Topological Entropy of Surface Braids and Maximally Efficient Mixing. The deep connections between braids and dynamics by way of the Nielsen Thurston classification theorem have led to a wide range of practical applications. Braids have been used to detect coherent structures and mixing regions in oceanic flows, drive the design of industrial mixing machines, contextualize the evolution of taffy pullers, and characterize the chaotic motion of topological defects in active nematics. Mixing plays a central role in each of these examples, and the braids naturally associated with each system come equipped with a useful measure of mixing efficiency, the topological entropy per operation (TEPO). This motivates the following questions. What is the maximum mixing efficiency for braids, and what braids realize this? The answer depends on how we define braids. For the standard Artin presentation, well known braids with mixing efficiencies related to the golden and silver ratios have been proven to be maximal. However, it is fruitful to consider surface braids, a natural generalization of braids, with presentations constructed from Artin like braid generators on embedded graphs. In this work, we introduce an efficient and elegant algorithm for finding the topological entropy and TEPO of surface braids on any pairing of orientable surface and planar embeddable graph. Of the myriad possible graphs and surfaces, graphs that can be embedded in mathbb R 2 as a lattice are a simple, highly symmetric choice, and the braids that result more naturally model the motion of points on the plane. We extensively search for a maximum mixing efficiency braid on planar lattice graphs and examine a novel candidate braid, which we conjecture to have this maximal property.


Deep Active Learning for Text Classification with Diverse Interpretations. Recently, Deep Neural Networks (DNNs) have made remarkable progress for text classification, which, however, still require a large number of labeled data. To train high performing models with the minimal annotation cost, active learning is proposed to select and label the most informative samples, yet it is still challenging to measure informativeness of samples used in DNNs. In this paper, inspired by piece wise linear interpretability of DNNs, we propose a novel Active Learning with DivErse iNterpretations (ALDEN) approach. With local interpretations in DNNs, ALDEN identifies linearly separable regions of samples. Then, it selects samples according to their diversity of local interpretations and queries their labels. To tackle the text classification problem, we choose the word with the most diverse interpretations to represent the whole sentence. Extensive experiments demonstrate that ALDEN consistently outperforms several state of the art deep active learning methods.


EILEEN A recommendation system for scientific publications and grants. Finding relevant scientific articles is crucial for advancing knowledge. Recommendation systems are helpful for such purpose, although they have only been applied to science recently. This article describes EILEEN (Exploratory Innovator of LitEraturE Networks), a recommendation system for scientific publications and grants with open source code and datasets. We describe EILEEN's architecture for ingesting and processing documents and modeling the recommendation system and keyphrase estimator. Using a unique dataset of log in user behavior, we validate our recommendation system against Latent Semantic Analysis (LSA) and the standard ranking from Elasticsearch (Lucene scoring). We find that a learning to rank with Random Forest achieves an AUC of 0.9, significantly outperforming both baselines. Our results suggest that we can substantially improve science recommendations and learn about scientists' behavior through their search behavior. We make our system available through eileen.io


Dynamics of the Volterra type integral and differentiation operators on generalized Fock spaces. Various dynamical properties of the differentiation and Volterra type integral operators on generalized Fock spaces are studied. We show that the differentiation operator is always supercyclic on these spaces. We further characterize when it is hypercyclic, power bounded and uniformly mean ergodic. We prove that the operator satisfies the Ritt's resolvent condition if and only if it is power bounded and uniformly mean ergodic. Some similar results are obtained for the Volterra type and Hardy integral operators.


Multilingual Counter Narrative Type Classification. The growing interest in employing counter narratives for hatred intervention brings with it a focus on dataset creation and automation strategies. In this scenario, learning to recognize counter narrative types from natural text is expected to be useful for applications such as hate speech countering, where operators from non governmental organizations are supposed to answer to hate with several and diverse arguments that can be mined from online sources. This paper presents the first multilingual work on counter narrative type classification, evaluating SoTA pre trained language models in monolingual, multilingual and cross lingual settings. When considering a fine grained annotation of counter narrative classes, we report strong baseline classification results for the majority of the counter narrative types, especially if we translate every language to English before cross lingual prediction. This suggests that knowledge about counter narratives can be successfully transferred across languages.


Exponential bounds for random walks on hyperbolic spaces without moment conditions. We consider nonelementary random walks on general hyperbolic spaces. Without any moment condition on the walk, we show that it escapes linearly to infinity, with exponential error bounds. We even get such exponential bounds up to the rate of escape of the walk. Our proof relies on an inductive decomposition of the walk, recording times at which it could go to infinity in several independent directions, and using these times to control further backtracking.


Oracle Complexity in Nonsmooth Nonconvex Optimization. It is well known that given a smooth, bounded from below, and possibly nonconvex function, standard gradient based methods can find epsilon stationary points (with gradient norm less than epsilon ) in mathcal O (1 epsilon 2) iterations. However, many important nonconvex optimization problems, such as those associated with training modern neural networks, are inherently not smooth, making these results inapplicable. In this paper, we study nonsmooth nonconvex optimization from an oracle complexity viewpoint, where the algorithm is assumed to be given access only to local information about the function at various points. We provide two main results First, we consider the problem of getting near epsilon stationary points. This is perhaps the most natural relaxation of finding epsilon stationary points, which is impossible in the nonsmooth nonconvex case. We prove that this relaxed goal cannot be achieved efficiently, for any distance and epsilon smaller than some constants. Our second result deals with the possibility of tackling nonsmooth nonconvex optimization by reduction to smooth optimization Namely, applying smooth optimization methods on a smooth approximation of the objective function. For this approach, we prove under a mild assumption an inherent trade off between oracle complexity and smoothness On the one hand, smoothing a nonsmooth nonconvex function can be done very efficiently (e.g., by randomized smoothing), but with dimension dependent factors in the smoothness parameter, which can strongly affect iteration complexity when plugging into standard smooth optimization methods. On the other hand, these dimension factors can be eliminated with suitable smoothing methods, but only by making the oracle complexity of the smoothing process exponentially large.


Explainable Point Based Document Visualizations. Two dimensional data maps can visually reveal information about the relations between data instances. Popular techniques to construct data maps are t SNE and UMAP. The resulting point based visualizations, though, provide information only through their interpretation. We here consider a set of abstracts from the articles on longevity to argue for using keyword extraction methods to label clusters of documents in the map. Among the considered approaches, the best results were obtained by recently proposed YAKE!. Surprisingly, a classical TF IDF term ranking outperformed graph and embedding based techniques.


Continuous Learning in Neural Machine Translation using Bilingual Dictionaries. While recent advances in deep learning led to significant improvements in machine translation, neural machine translation is often still not able to continuously adapt to the environment. For humans, as well as for machine translation, bilingual dictionaries are a promising knowledge source to continuously integrate new knowledge. However, their exploitation poses several challenges The system needs to be able to perform one shot learning as well as model the morphology of source and target language. In this work, we proposed an evaluation framework to assess the ability of neural machine translation to continuously learn new phrases. We integrate one shot learning methods for neural machine translation with different word representations and show that it is important to address both in order to successfully make use of bilingual dictionaries. By addressing both challenges we are able to improve the ability to translate new, rare words and phrases from 30 to up to 70 . The correct lemma is even generated by more than 90 .


Tropical tangents for complete intersection curves. We consider the tropicalization of tangent lines to a complete intersection curve X in mathbb P n . Under mild hypotheses, we describe the tropicalization of the image of the Gauss map of X in terms of the tropicalizations of the hypersurfaces cutting out X . We apply this to obtain descriptions of the tropicalization of the dual variety X and tangential variety tau(X) of X . In particular, we are able to compute the degrees of X and tau(X) and the Newton polytope of tau(X) without using any elimination theory.


Tackling the Imbalance for GNNs. Different from deep neural networks for non graph data classification, graph neural networks (GNNs) leverage the information exchange between nodes (or samples) when representing nodes. The category distribution shows an imbalance or even a highly skewed trend on nearly all existing benchmark GNN data sets. The imbalanced distribution will cause misclassification of nodes in the minority classes, and even cause the classification performance on the entire data set to decrease. This study explores the effects of the imbalance problem on the performances of GNNs and proposes new methodologies to solve it. First, a node level index, namely, the label difference index ( LDI ), is defined to quantitatively analyze the relationship between imbalance and misclassification. The less samples in a class, the higher the value of its average LDI the higher the LDI of a sample, the more likely the sample will be misclassified. We define a new loss and propose four new methods based on LDI . Experimental results indicate that the classification accuracies of the three among our proposed four new methods are better in both transductive and inductive settings. The LDI can be applied to other GNNs.


Multi Scale Cost Volumes Cascade Network for Stereo Matching. Stereo matching is essential for robot navigation. However, the accuracy of current widely used traditional methods is low, while methods based on CNN need expensive computational cost and running time. This is because different cost volumes play a crucial role in balancing speed and accuracy. Thus we propose MSCVNet, which combines traditional methods and neural networks to improve the quality of cost volume. Concretely, our network first generates multiple 3D cost volumes with different resolutions and then uses 2D convolutions to construct a novel cascade hourglass network for cost aggregation. Meanwhile, we design an algorithm to distinguish and calculate the loss for discontinuous areas of disparity result. According to the KITTI official website, our network is much faster than most top performing methods (24 times than CSPN, 44 times than GANet, etc.). Meanwhile, compared to traditional methods (SPS St, SGM) and other real time stereo matching networks (Fast DS CS, DispNetC, and RTSNet, etc.), our network achieves a big improvement in accuracy, demonstrating the feasibility and capability of the proposed method.


Image Animation with Keypoint Mask. Motion transfer is the task of synthesizing future video frames of a single source image according to the motion from a given driving video. In order to solve it, we face the challenging complexity of motion representation and the unknown relations between the driving video and the source image. Despite its difficulty, this problem attracted great interests from researches at the recent years, with gradual improvements. The goal is often thought as the decoupling of motion and appearance, which is may be solved by extracting the motion from keypoint movement. We chose to tackle the generic, unsupervised setting, where we need to apply animation to any arbitrary object, without any domain specific model for the structure of the input. In this work, we extract the structure from a keypoint heatmap, without an explicit motion representation. Then, the structures from the image and the video are extracted to warp the image according to the video, by a deep generator. We suggest two variants of the structure from different steps in the keypoint module, and show superior qualitative pose and quantitative scores.


Computing Goldstein ( epsilon, delta) Stationary Points of Lipschitz Functions in widetilde O ( epsilon 3 delta 1 ) Iterations via Random Conic Perturbation. In this note, we report a practical algorithm to compute Goldstein ( epsilon, delta) stationary points for L Lipschitz function in finite time without subgradient selection procedure in every oracle calls. The new algorithm removes the impractical first order oracle assumption in Zhang et al. 2020 with a random conic perturbation.


Universal coupled theory for metamaterial Bound states in the continuum. In this paper, we present a novel universal coupled theory for metamaterial Bound states in the continuum (BIC) or quasi Bound states in the continuum (quasi BIC) which provides ultra high Q resonance for metamaterial devices. Our theory analytically calculates the coupling of two bright modes with phase. Our method has much more accuracy for ultra strong coupling comparing with the previous theory (the coupling of one bright mode and one dark mode). Therefore, our theory is much more suitable for BIC or quasi BIC and we can accurately predict the transmission spectrum of metamaterial BIC or quasi BIC for the first time.


Exploiting Isomorphic Subgraphs in SAT (Long version). While static symmetry breaking has been explored in the SAT community for decades, only as of 2010 research has focused on exploiting the same discovered symmetry dynamically, during the run of the SAT solver, by learning extra clauses. The two methods are distinct and not compatible. The former prunes solutions, whereas the latter does not it only prunes areas of the search that do not have solutions, like standard conflict clauses. Both approaches, however, require what we call emph full symmetry , namely a propositionally consistent mapping sigma between the literals, such that sigma( varphi) equiv varphi , where here equiv means syntactic equivalence modulo clause ordering and literal ordering within the clauses. In this article we show that such full symmetry is not a necessary condition for adding extra clauses isomorphism between possibly overlapping subgraphs of the colored incidence graph is sufficient. While finding such subgraphs is a computationally hard problem, there are many cases in which they can be detected a priory by analyzing the high level structure of the problem from which the CNF was derived. We demonstrate this principle with several well known problems, including Van der Waerden numbers, bounded model checking and Boolean Pythagorean triples.


Local Navigation and Docking of an Autonomous Robot Mower using Reinforcement Learning and Computer Vision. We demonstrate a successful navigation and docking control system for the John Deere Tango autonomous mower, using only a single camera as the input. This vision only system is of interest because it is inexpensive, simple for production, and requires no external sensing. This is in contrast to existing systems that rely on integrated position sensors and global positioning system (GPS) technologies. To produce our system we combined a state of the art object detection architecture, You Only Look Once (YOLO), with a reinforcement learning (RL) architecture, Double Deep QNetworks (Double DQN). The object detection network identifies features on the mower and passes its output to the RL network, providing it with a low dimensional representation that enables rapid and robust training. Finally, the RL network learns how to navigate the machine to the desired spot in a custom simulation environment. When tested on mower hardware, the system is able to dock with centimeter level accuracy from arbitrary initial locations and orientations.


Time Series Prediction using Deep Learning Methods in Healthcare. Traditional machine learning methods face two main challenges in dealing with healthcare predictive analytics tasks. First, the high dimensional nature of healthcare data needs labor intensive and time consuming processes to select an appropriate set of features for each new task. Second, these methods depend on feature engineering to capture the sequential nature of patient data, which may not adequately leverage the temporal patterns of the medical events and their dependencies. Recent deep learning methods have shown promising performance for various healthcare prediction tasks by addressing the high dimensional and temporal challenges of medical data. These methods can learn useful representations of key factors (e.g., medical concepts or patients) and their interactions from high dimensional raw or minimally processed healthcare data. In this paper we systematically reviewed studies focused on advancing and using deep neural networks to leverage patients structured time series data for healthcare prediction tasks. To identify relevant studies, MEDLINE, IEEE, Scopus and ACM digital library were searched for studies published up to February 7th 2021. We found that researchers have contributed to deep time series prediction literature in ten research streams deep learning models, missing value handling, irregularity handling, patient representation, static data inclusion, attention mechanisms, interpretation, incorporating medical ontologies, learning strategies, and scalability. This study summarizes research insights from these literature streams, identifies several critical research gaps, and suggests future research opportunities for deep learning in patient time series data.


Machine learning enabled vectorial opto magnetization orientation. Manipulation of light induced magnetization has become a fundamentally hot topic with a potentially high impact for atom trapping, confocal and magnetic resonance microscopy, and data storage. The control of the magnetization orientation mainly relies on the direct methods composed of amplitude, phase and polarization modulations of the incident light under the tight focusing condition, leaving the achievement of arbitrary desirable three dimensional (3D) magnetization orientation complicated, inflexible and inefficient. Here, we propose a facile approach called machine learning inverse design to achieve expected vectorial opto magnetization orientation. This pathway is time efficient and accurate to produce the demanded incident beam for arbitrary prescribed 3D magnetization orientation. It is highlighted that the machine learning method is not only applied for magnetization orientations, but also widely used in the control of magnetization structures.


Formalizing Generalization and Robustness of Neural Networks to Weight Perturbations. Studying the sensitivity of weight perturbation in neural networks and its impacts on model performance, including generalization and robustness, is an active research topic due to its implications on a wide range of machine learning tasks such as model compression, generalization gap assessment, and adversarial attacks. In this paper, we provide the first integral study and analysis for feed forward neural networks in terms of the robustness in pairwise class margin and its generalization behavior under weight perturbation. We further design a new theory driven loss function for training generalizable and robust neural networks against weight perturbations. Empirical experiments are conducted to validate our theoretical analysis. Our results offer fundamental insights for characterizing the generalization and robustness of neural networks against weight perturbations.


Towards General Deep Leakage in Federated Learning. Unlike traditional central training, federated learning (FL) improves the performance of the global model by sharing and aggregating local models rather than local data to protect the users' privacy. Although this training approach appears secure, some research has demonstrated that an attacker can still recover private data based on the shared gradient information. This on the fly reconstruction attack deserves to be studied in depth because it can occur at any stage of training, whether at the beginning or at the end of model training no relevant dataset is required and no additional models need to be trained. We break through some unrealistic assumptions and limitations to apply this reconstruction attack in a broader range of scenarios. We propose methods that can reconstruct the training data from shared gradients or weights, corresponding to the FedSGD and FedAvg usage scenarios, respectively. We propose a zero shot approach to restore labels even if there are duplicate labels in the batch. We study the relationship between the label and image restoration. We find that image restoration fails even if there is only one incorrectly inferred label in the batch we also find that when batch images have the same label, the corresponding image is restored as a fusion of that class of images. Our approaches are evaluated on classic image benchmarks, including CIFAR 10 and ImageNet. The batch size, image quality, and the adaptability of the label distribution of our approach exceed those of GradInversion, the state of the art.


Capacity Results for MIMO Optical Wireless Communication With Per Antenna Intensity Constraints. In this paper, we investigate capacities of two types of the multiple input multiple output (MIMO) optical intensity channel (OIC) under per antenna peak and average intensity constraints, called the equal cost constrained OIC (EC OIC) and the bounded cost constrained OIC (BC OIC). The average intensities of input in the EC OIC are required to be equal to preassigned constants, while in the BC OIC those intensities are no larger than preassigned constants. We first consider a general vector Gaussian channel under moment constraints and prove that its high SNR capacity is determined by the maximum differential entropy with some mild conditions. Then three capacity expressions are derived for the rank one EC OIC, the rank one BC OIC and the EC OIC of rank being the number of transmit antennas minus one, respectively, based on which we obtain the results that 1) either a rank one EC OIC and a rank one BC OIC is equivalent to some SISO OIC with an amplitude constraint and several moment constraints 2) by asymptotic results on the moment constrained vector Gaussian channel, both high SNR asymptotic capacities of the EC OIC and the BC OIC of rank being the number of transmit antennas minus one are characterized. Furthermore, we focus on low SNR capacity slopes for the general MIMO BC OIC, and prove properties of the optimal intensity allocation, which simplify the involved nonsmooth optimization problem.


Fine grained Semantics aware Representation Enhancement for Self supervised Monocular Depth Estimation. Self supervised monocular depth estimation has been widely studied, owing to its practical importance and recent promising improvements. However, most works suffer from limited supervision of photometric consistency, especially in weak texture regions and at object boundaries. To overcome this weakness, we propose novel ideas to improve self supervised monocular depth estimation by leveraging cross domain information, especially scene semantics. We focus on incorporating implicit semantic knowledge into geometric representation enhancement and suggest two ideas a metric learning approach that exploits the semantics guided local geometry to optimize intermediate depth representations and a novel feature fusion module that judiciously utilizes cross modality between two heterogeneous feature representations. We comprehensively evaluate our methods on the KITTI dataset and demonstrate that our method outperforms state of the art methods. The source code is available at https github.com hyBlue FSRE Depth.


Relativistic BGK model for massless particles in the FLRW spacetime. In this paper, we address the Cauchy problem for the relativistic BGK model proposed by Anderson and Witting for massless particles in the Friedmann Lemaitre Robertson Walker (FLRW) spacetime.


Joint Radar and Communication A Survey. Joint radar and communication (JRC) technology has become important for civil and military applications for decades. This paper introduces the concepts, characteristics and advantages of JRC technology, presenting the typical applications that have benefited from JRC technology currently and in the future. This paper explores the state of the art of JRC in the levels of coexistence, cooperation, co design and collaboration. Compared to previous surveys, this paper reviews the entire trends that drive the development of radar sensing and wireless communication using JRC. Specifically, we explore an open research issue on radar and communication operating with mutual benefits based on collaboration, which represents the fourth stage of JRC evolution. This paper provides useful perspectives for future researches of JRC technology.


Clairaut Riemannian maps. In this paper, first we define Clairaut Riemannian map between Riemannian manifolds by using geodesic curve on the base space and find necessary and sufficient conditions for Riemannian map to be Clairaut with a non trivial example. We also obtain necessary and sufficient condition for Clairaut Riemannian map to be harmonic. Thereafter, we study Clairaut Riemannian map whose base manifold admits a Ricci soliton with a non trivial example. We obtain scalar curvatures of rangeF ast and (rangeF ast) bot by using Ricci soliton. Further, we obtain necessary conditions for the rangeF ast to be almost Ricci soliton and Einstein. We also obtain necessary condition for vector field dot beta to be conformal on rangeF ast and necessary and sufficient condition for vector field dot beta to be Killing on (rangeF ast) bot , where beta is a geodesic curve on the base space of Clairaut Riemannian map. Also, we obtain necessary condition for the mean curvature vector field of rangeF ast to be constant. Finally, we introduce Clairaut anti invariant Riemannian map from Riemannian manifold to K "ahler manifold, and obtain necessary and sufficient condition for an anti invariant Riemannian map to be Clairaut with a non trivial example. Further, we find necessary condition for rangeF ast to be minimal and totally geodesic. We also obtain necessary and sufficient condition for Clairaut anti invariant Riemannian maps to be harmonic. Moreover, we calculated Ricci tensor and obtained curvature relations for Clairaut anti invariant Riemannian map.


Constant Q curvature metrics with a singularity. For dimensions n geq 3 , we classify singular solutions to the generalized Liouville equation ( Delta) n 2 u e nu on mathbb R n setminus 0 with the finite integral condition int mathbb R n e nu infty in terms of their behavior at 0 and infty . These solutions correspond to metrics of constant Q curvature which are singular in the origin. Conversely, we give an optimal existence result for radial solutions. This extends some recent results on solutions with singularities of logarithmic type to allow for singularities of arbitrary order. As a key tool to the existence result, we derive a new weighted Moser Trudinger inequality for radial functions.


Localized smoothing and concentration for the Navier Stokes equations in the half space. We establish a local in space short time smoothing effect for the Navier Stokes equations in the half space. The whole space analogue, due to Jia and v S ver 'ak J v S 14 , is a central tool in two of the authors' recent work on quantitative L 3 x blow up criteria BP21 . The main difficulty is that the non local effects of the pressure in the half space are much stronger than in the whole space. As an application, we demonstrate that the critical L 3 x norm must concentrate at scales sim sqrt T t in the presence of a Type I blow up.


Lower Perplexity is Not Always Human Like. In computational psycholinguistics, various language models have been evaluated against human reading behavior (e.g., eye movement) to build human like computational models. However, most previous efforts have focused almost exclusively on English, despite the recent trend towards linguistic universal within the general community. In order to fill the gap, this paper investigates whether the established results in computational psycholinguistics can be generalized across languages. Specifically, we re examine an established generalization the lower perplexity a language model has, the more human like the language model is in Japanese with typologically different structures from English. Our experiments demonstrate that this established generalization exhibits a surprising lack of universality namely, lower perplexity is not always human like. Moreover, this discrepancy between English and Japanese is further explored from the perspective of (non )uniform information density. Overall, our results suggest that a cross lingual evaluation will be necessary to construct human like computational models.


Ramsey theory of homogeneous structures current trends and open problems. This article highlights historical achievements in the partition theory of countable homogeneous relational structures, and presents recent work, current trends, and open problems. Exciting recent developments include new methods involving logic, topological Ramsey spaces, and category theory. The paper concentrates on big Ramsey degrees, presenting their essential structure where known and outlining areas for further development. Cognate areas, including infinite dimensional Ramsey theory of homogeneous structures and partition theory of uncountable structures, are also discussed.


Towards Fine grained 3D Face Dense Registration An Optimal Dividing and Diffusing Method. Dense vertex to vertex correspondence between 3D faces is a fundamental and challenging issue for 3D 2D face analysis. While the sparse landmarks have anatomically ground truth correspondence, the dense vertex correspondences on most facial regions are unknown. In this view, the current literatures commonly result in reasonable but diverse solutions, which deviate from the optimum to the 3D face dense registration problem. In this paper, we revisit dense registration by a dimension degraded problem, i.e. proportional segmentation of a line, and employ an iterative dividing and diffusing method to reach the final solution uniquely. This method is then extended to 3D surface by formulating a local registration problem for dividing and a linear least square problem for diffusing, with constraints on fixed features. On this basis, we further propose a multi resolution algorithm to accelerate the computational process. The proposed method is linked to a novel local scaling metric, where we illustrate the physical meaning as smooth rearrangement for local cells of 3D facial shapes. Extensive experiments on public datasets demonstrate the effectiveness of the proposed method in various aspects. Generally, the proposed method leads to coherent local registrations and elegant mesh grid routines for fine grained 3D face dense registrations, which benefits many downstream applications significantly. It can also be applied to dense correspondence for other format of data which are not limited to face. The core code will be publicly available at https github.com NaughtyZZ 3D face dense registration.


Risk sensitive discounted cost criterion for Continuous time Markov decision processes on a general state space. In this paper, we consider risk sensitive discounted control problem for continuous time jump Markov processes taking values in general state space. The transition rates of underlying continuous time jump Markov processes and the cost rates are allowed to be unbounded. Under certain Lyapunov condition, we establish the existence and uniqueness of the solution to the Hamilton Jacobi Bellman (HJB) equation. Also we prove the existence of optimal risk sensitive control in the class of Markov control.


Representations of Deligne Mostow lattices into PGL(3, C). We classify representations of a class of Deligne Mostow lattices into PGL(3, C). In particular, we show local rigidity for the representations (of Deligne Mostow lattices with 3 fold symmetry and of type one) where the generators we chose are in the same conjugacy class as the generators of Deligne Mostow lattices. We use formal computations in SAGE to obtain the results. The code files are available on GitHub ( FPUP21 ).


Observation of intermolecular Coulombic decay in liquid water. We report the first observation of intermolecular Coulombic decay (ICD) in liquid water following inner valence ionization. By combining a monochromatized table top high harmonic source with a liquid micro jet, we recorded electron electron coincidence spectra at two photon energies that identify the ICD electrons, together with the photoelectrons originating from the 2a1 inner valence band of liquid water. Our results confirm the importance of ICD as a source of low energy electrons in bulk liquid water and provide quantitative results for modeling the velocity distribution of the slow electrons that are thought to dominate radiation damage in aqueous environments.


Recurrence relations of poly Cauchy numbers by the r Stirling transform. We give some formulas of poly Cauchy numbers by the r Stirling transform. In the case of the classical or poly Bernoulli numbers, the formulas are with Stirling numbers of the first kind. In our case of the classical or poly Cauchy numbers, the formulas are with Stirling numbers of the second kind. We also discuss annihilation formulas for poly Cauchy number with negative indices.


Demand Response Analogues for Residential Loads in Natural Gas Networks. Demand response for electrical power networks is a mature field that has yielded numerous efficiency and resilience benefits like managing the peaks and valleys of electricity usage, reduction in peak electricity usage, to name a few. However, only recently has the study of the counterpart to demand response for electric power in natural gas started to receive similar levels of attention. Natural gas systems are increasingly operating at or near capacity, which challenges these systems to meet all the needs for gas, especially during severe winter weather. However, unlike demand response programs in electrical networks, demand response in gas networks cannot shift peak usage. Here, we develop analogues to demand response that can help improve the resilience of natural gas systems by reducing peak consumption and thereby limiting potential disruptions such events can cause. This paper develops a mathematical formulation to support a residential level demand response analogue for natural gas based on current and anticipated smart thermostat technologies. The mathematical formulation takes the form of an optimal control problem (OCP) that leverages physics constraints to model temperature changes, balance equitable service, and optimize the gas consumption for collections of houses. On test problems, the formulation demonstrates significant benefits, including the ability to cut peak demand by 15 while still ensuring equitable service to customers.


The Human Evaluation Datasheet 1.0 A Template for Recording Details of Human Evaluation Experiments in NLP. This paper introduces the Human Evaluation Datasheet, a template for recording the details of individual human evaluation experiments in Natural Language Processing (NLP). Originally taking inspiration from seminal papers by Bender and Friedman (2018), Mitchell et al. (2019), and Gebru et al. (2020), the Human Evaluation Datasheet is intended to facilitate the recording of properties of human evaluations in sufficient detail, and with sufficient standardisation, to support comparability, meta evaluation, and reproducibility tests.


Temporal Feature Networks for CNN based Object Detection. For reliable environment perception, the use of temporal information is essential in some situations. Especially for object detection, sometimes a situation can only be understood in the right perspective through temporal information. Since image based object detectors are currently based almost exclusively on CNN architectures, an extension of their feature extraction with temporal features seems promising. Within this work we investigate different architectural components for a CNN based temporal information extraction. We present a Temporal Feature Network which is based on the insights gained from our architectural investigations. This network is trained from scratch without any ImageNet information based pre training as these images are not available with temporal information. The object detector based on this network is evaluated against the non temporal counterpart as baseline and achieves competitive results in an evaluation on the KITTI object detection dataset.


Learning Task Agnostic Skills with Data driven Guidance. To increase autonomy in reinforcement learning, agents need to learn useful behaviours without reliance on manually designed reward functions. To that end, skill discovery methods have been used to learn the intrinsic options available to an agent using task agnostic objectives. However, without the guidance of task specific rewards, emergent behaviours are generally useless due to the under constrained problem of skill discovery in complex and high dimensional spaces. This paper proposes a framework for guiding the skill discovery towards the subset of expert visited states using a learned state projection. We apply our method in various reinforcement learning (RL) tasks and show that such a projection results in more useful behaviours.


High yield and high angular flux neutron generation from deuterons accelerated by laser driven collisionless shock. A bright collimated neutron source is an essential tool for global security missions and fundamental scientific research. In this paper, we study a compact high yield and high angular flux neutron source utilizing the break up reaction of laser driven deuterons in a 9 text Be converter. The neutron generation scaling from such a reaction is used to guide the choice and optimization of the acceleration process for the bulk ions in a low density text CD 2 foam. In particular, the collisionless shock acceleration mechanism is exploited with proper choice in the laser and target parameter space to accelerate these ions towards energies above the temperature of the distribution. Particle In Cell and Monte Carlo simulations are coupled to investigate this concept and possible adverse effects, as well as the contribution from the surface ions accelerated and the optimal converter design. The simulation results indicated that our design can be a practical approach to increase both the neutron yield and forward flux of laser driven neutron sources, reaching peak angular neutron flux 10 11 neutron sr and yield 10 11 neutron pulse with present day kJ class high power lasers. Such developments will advance fundamental neutron science, high precision radiography and other global security applications with the laser driven sources.


BoGraph Structured Bayesian Optimization From Logs for Expensive Systems with Many Parameters. Current auto tuning frameworks struggle with tuning computer systems configurations due to their large parameter space, complex interdependencies, and high evaluation cost. Utilizing probabilistic models, Structured Bayesian Optimization (SBO) has recently overcome these difficulties. SBO decomposes the parameter space by utilizing contextual information provided by system experts leading to fast convergence. However, the complexity of building probabilistic models has hindered its wider adoption. We propose BoAnon, a SBO framework that learns the system structure from its logs. BoAnon provides an API enabling experts to encode knowledge of the system as performance models or components dependency. BoAnon takes in the learned structure and transforms it into a probabilistic graph model. Then it applies the expert provided knowledge to the graph to further contextualize the system behavior. BoAnon probabilistic graph allows the optimizer to find efficient configurations faster than other methods. We evaluate BoAnon via a hardware architecture search problem, achieving an improvement in energy latency objectives ranging from 5 7 x factors improvement over the default architecture. With its novel contextual structure learning pipeline, BoAnon makes using SBO accessible for a wide range of other computer systems such as databases and stream processors.


Neural View Synthesis and Matching for Semi Supervised Few Shot Learning of 3D Pose. We study the problem of learning to estimate the 3D object pose from a few labelled examples and a collection of unlabelled data. Our main contribution is a learning framework, neural view synthesis and matching, that can transfer the 3D pose annotation from the labelled to unlabelled images reliably, despite unseen 3D views and nuisance variations such as the object shape, texture, illumination or scene context. In our approach, objects are represented as 3D cuboid meshes composed of feature vectors at each mesh vertex. The model is initialized from a few labelled images and is subsequently used to synthesize feature representations of unseen 3D views. The synthesized views are matched with the feature representations of unlabelled images to generate pseudo labels of the 3D pose. The pseudo labelled data is, in turn, used to train the feature extractor such that the features at each mesh vertex are more invariant across varying 3D views of the object. Our model is trained in an EM type manner alternating between increasing the 3D pose invariance of the feature extractor and annotating unlabelled data through neural view synthesis and matching. We demonstrate the effectiveness of the proposed semi supervised learning framework for 3D pose estimation on the PASCAL3D and KITTI datasets. We find that our approach outperforms all baselines by a wide margin, particularly in an extreme few shot setting where only 7 annotated images are given. Remarkably, we observe that our model also achieves an exceptional robustness in out of distribution scenarios that involve partial occlusion.


Conditional Bures Metric for Domain Adaptation. As a vital problem in classification oriented transfer, unsupervised domain adaptation (UDA) has attracted widespread attention in recent years. Previous UDA methods assume the marginal distributions of different domains are shifted while ignoring the discriminant information in the label distributions. This leads to classification performance degeneration in real applications. In this work, we focus on the conditional distribution shift problem which is of great concern to current conditional invariant models. We aim to seek a kernel covariance embedding for conditional distribution which remains yet unexplored. Theoretically, we propose the Conditional Kernel Bures (CKB) metric for characterizing conditional distribution discrepancy, and derive an empirical estimation for the CKB metric without introducing the implicit kernel feature map. It provides an interpretable approach to understand the knowledge transfer mechanism. The established consistency theory of the empirical estimation provides a theoretical guarantee for convergence. A conditional distribution matching network is proposed to learn the conditional invariant and discriminative features for UDA. Extensive experiments and analysis show the superiority of our proposed model.


An Upper Bound on the Complexity of Tablut. Tablut is a complete knowledge, deterministic, and asymmetric board game, which has not been solved nor properly studied yet. In this work, its rules and characteristics are presented, then a study on its complexity is reported. An upper bound to its complexity is found eventually by dividing the state space of the game into subspaces according to specific conditions. This upper bound is comparable to the one found for Draughts, therefore, it would seem that the open challenge of solving this game requires a considerable computational effort.


An upper bound on the dimension of the Rauzy gasket. In this note, we give an elementary proof that the Hausdorff dimension of the Rauzy gasket is at most 1.7407, improving upon results of Avila et al. and Fougeron.


Average Case Subset Balancing Problems. Given a set of n input integers, the Equal Subset Sum problem asks us to find two distinct subsets with the same sum. In this paper we present an algorithm that runs in time O (3 0.387n ) in the average case, significantly improving over the O (3 0.488n ) running time of the best known worst case algorithm and the Meet in the Middle benchmark of O (3 0.5n ) . Our algorithm generalizes to a number of related problems, such as the Generalized Equal Subset Sum'' problem, which asks us to assign a coefficient c i from a set C to each input number x i such that sum i c i x i 0 . Our algorithm for the average case version of this problem runs in time C (0.5 c 0 C )n for some positive constant c 0 , whenever C 0, pm 1, dots, pm d or pm 1, dots, pm d for some positive integer d (with O ( C 0.45n ) when C 10 ). Our results extend to the problem of finding nearly balanced'' solutions in which the target is a not too large nonzero offset tau . Our approach relies on new structural results that characterize the probability that sum i c i x i tau has a solution c in C n when x i 's are chosen randomly these results may be of independent interest. Our algorithm is inspired by the representation technique'' introduced by Howgrave Graham and Joux. This requires several new ideas to overcome preprocessing hurdles that arise in the representation framework, as well as a novel application of dynamic programming in the solution recovery phase of the algorithm.


Manifold Repairing, Reconstruction and Denoising from Scattered Data in High Dimension. We consider a problem of great practical interest the repairing and recovery of a low dimensional manifold embedded in high dimensional space from noisy scattered data. Suppose that we observe a point cloud sampled from the low dimensional manifold, with noise, and let us assume that there are holes in the data. Can we recover missing information inside the holes? While in low dimension the problem was extensively studied, manifold repairing in high dimension is still an open problem. We introduce a new approach, called Repairing Manifold Locally Optimal Projection (R MLOP), that expands the MLOP method introduced by Faigenbaum Golovin et al. in 2020, to cope with manifold repairing in low and high dimensional cases. The proposed method can deal with multiple holes in a manifold. We prove the validity of the proposed method, and demonstrate the effectiveness of our approach by considering different manifold topologies, for single and multiple holes repairing, in low and high dimensions.


Introduction to Neural Network Verification. Deep learning has transformed the way we think of software and what it can do. But deep neural networks are fragile and their behaviors are often surprising. In many settings, we need to provide formal guarantees on the safety, security, correctness, or robustness of neural networks. This book covers foundational ideas from formal verification and their adaptation to reasoning about neural networks and deep learning.


Kippenhahn Varieties and the Weyl Calculus for Several Matrices I. The paper reviews properties of the Weyl functional calculus for several operators and its relation to the generalised numerical range of n hermitian matrices. The support and singular support of the Weyl functional calculus for n hermitian matrices are determined by Kippenhahn varieties in algebraic geometry.


Goal Conditioned Reinforcement Learning with Imagined Subgoals. Goal conditioned reinforcement learning endows an agent with a large variety of skills, but it often struggles to solve tasks that require more temporally extended reasoning. In this work, we propose to incorporate imagined subgoals into policy learning to facilitate learning of complex tasks. Imagined subgoals are predicted by a separate high level policy, which is trained simultaneously with the policy and its critic. This high level policy predicts intermediate states halfway to the goal using the value function as a reachability metric. We don't require the policy to reach these subgoals explicitly. Instead, we use them to define a prior policy, and incorporate this prior into a KL constrained policy iteration scheme to speed up and regularize learning. Imagined subgoals are used during policy learning, but not during test time, where we only apply the learned policy. We evaluate our approach on complex robotic navigation and manipulation tasks and show that it outperforms existing methods by a large margin.


A Brief Introduction to Generative Models. We introduce and motivate generative modeling as a central task for machine learning and provide a critical view of the algorithms which have been proposed for solving this task. We overview how generative modeling can be defined mathematically as trying to make an estimating distribution the same as an unknown ground truth distribution. This can then be quantified in terms of the value of a statistical divergence between the two distributions. We outline the maximum likelihood approach and how it can be interpreted as minimizing KL divergence. We explore a number of approaches in the maximum likelihood family, while discussing their limitations. Finally, we explore the alternative adversarial approach which involves studying the differences between an estimating distribution and a real data distribution. We discuss how this approach can give rise to new divergences and methods that are necessary to make adversarial learning successful. We also discuss new evaluation metrics which are required by the adversarial approach.


Negative Selection Algorithm Research and Applications in the last decade A Review. The Negative selection Algorithm (NSA) is one of the important methods in the field of Immunological Computation (or Artificial Immune Systems). Over the years, some progress was made which turns this algorithm (NSA) into an efficient approach to solve problems in different domain. This review takes into account these signs of progress during the last decade and categorizes those based on different characteristics and performances. Our study shows that NSA's evolution can be labeled in four ways highlighting the most notable NSA variations and their limitations in different application domains. We also present alternative approaches to NSA for comparison and analysis. It is evident that NSA performs better for nonlinear representation than most of the other methods, and it can outperform neural based models in computation time. We summarize NSA's development and highlight challenges in NSA research in comparison with other similar models.


On the Hardy Littlewood Chowla conjecture on average. There has been recent interest in a hybrid form of the celebrated conjectures of Hardy Littlewood and of Chowla. We prove that for any k, ell ge1 and distinct integers h 2, ldots,h k,a 1, ldots,a ell , we have sum n leq X mu(n h 1) cdots mu(n h k) Lambda(n a 1) cdots Lambda(n a ell ) o(X) for all except o(H) values of h 1 leq H , so long as H geq ( log X) ell varepsilon . This improves on the range H ge ( log X) psi(X) , psi(X) to infty , obtained in previous work of the first author. Our results also generalize from the M "obius function mu to arbitrary (non pretentious) multiplicative functions.


Robust Explanations for Private Support Vector Machines. We consider counterfactual explanations for private support vector machines (SVM), where the privacy mechanism that publicly releases the classifier guarantees differential privacy. While privacy preservation is essential when dealing with sensitive data, there is a consequent degradation in the classification accuracy due to the introduced perturbations in the classifier weights. For such classifiers, counterfactual explanations need to be robust against the uncertainties in the SVM weights in order to ensure, with high confidence, that the classification of the data instance to be explained is different than its explanation. We model the uncertainties in the SVM weights through a random vector, and formulate the explanation problem as an optimization problem with probabilistic constraint. Subsequently, we characterize the problem's deterministic equivalent and study its solution. For linear SVMs, the problem is a convex second order cone program. For non linear SVMs, the problem is non convex. Thus, we propose a sub optimal solution that is based on the bisection method. The results show that, contrary to non robust explanations, the quality of explanations from the robust solution degrades with increasing privacy in order to guarantee a prespecified confidence level for correct classifications.


Challenges and approaches to time series forecasting in data center telemetry A Survey. Time series forecasting has been an important research domain for so many years. Its applications include ECG predictions, sales forecasting, weather conditions, even COVID 19 spread predictions. These applications have motivated many researchers to figure out an optimal forecasting approach, but the modeling approach also changes as the application domain changes. This work has focused on reviewing different forecasting approaches for telemetry data predictions collected at data centers. Forecasting of telemetry data is a critical feature of network and data center management products. However, there are multiple options of forecasting approaches that range from a simple linear statistical model to high capacity deep learning architectures. In this paper, we attempted to summarize and evaluate the performance of well known time series forecasting techniques. We hope that this evaluation provides a comprehensive summary to innovate in forecasting approaches for telemetry data.


Architecting and Visualizing Deep Reinforcement Learning Models. To meet the growing interest in Deep Reinforcement Learning (DRL), we sought to construct a DRL driven Atari Pong agent and accompanying visualization tool. Existing approaches do not support the flexibility required to create an interactive exhibit with easily configurable physics and a human controlled player. Therefore, we constructed a new Pong game environment, discovered and addressed a number of unique data deficiencies that arise when applying DRL to a new environment, architected and tuned a policy gradient based DRL model, developed a real time network visualization, and combined these elements into an interactive display to help build intuition and awareness of the mechanics of DRL inference.