SAI BA IoMT Secure AI Based Blockchain Assisted Internet of Medical Things Tool to Moderate the Outbreak of COVID 19 Crisis. Recently, an infectious disease, coronavirus disease 2019 (COVID 19), has been reported in Wuhan, China, and subsequently spread worldwide within a couple of months. On May 16, 2020, the COVID 19 pandemic has affected several countries in the world. In this article, we present how the amalgamation of blockchain, cryptography, internet of medical things (IoMT), and artificial intelligence (AI) technologies can address such an issue in the event of the COVID 19 pandemic. Further, we propose a secure AI based blockchain assisted IoMT (SAI BA IoMT) model for the healthcare system in the COVID 19 crisis. The paper also examines the post corona crisis that the world could be experienced after the pandemic. Additionally, we exhibit the potential applications of the proposed model to resolve the difficulties originated from coronavirus.


Chiral Multi layer Waveguides Incorporating Graphene Sheets An Analytical Approach. This paper presents a novel analytical model for chiral multi layer waveguides incorporating graphene sheets. The general structure is composed of various chiral layers, where a graphene sheet has been sandwiched between two adjacent chiral layers. We have chosen and studied this general structure to suggest a new platform for novel emerging sciences such as optical sensing. An analytical model has been proposed for the general structure to find its dispersion relation and other modal properties such as the effective index and the propagation length. To show the richness of our general structure, two novel chiral waveguides containing graphene sheets have been introduced and investigated. Our analytical results show that high values of effective indices, e.g. neff 15 at the frequency of 8 THz for the second structure, is achievable. Moreover, it has been shown that the chemical potential of the graphene and the chirality can adjust and control the plasmonic features. Our multi layer structure and its analytical model can be utilized in several potential applications such as switches, absorbers, cloaks, polarization rotators, and directional couplers in the THz frequencies.


Im2Mesh GAN Accurate 3D Hand Mesh Recovery from a Single RGB Image. This work addresses hand mesh recovery from a single RGB image. In contrast to most of the existing approaches where the parametric hand models are employed as the prior, we show that the hand mesh can be learned directly from the input image. We propose a new type of GAN called Im2Mesh GAN to learn the mesh through end to end adversarial training. By interpreting the mesh as a graph, our model is able to capture the topological relationship among the mesh vertices. We also introduce a 3D surface descriptor into the GAN architecture to further capture the 3D features associated. We experiment two approaches where one can reap the benefits of coupled groundtruth data availability of images and the corresponding meshes, while the other combats the more challenging problem of mesh estimations without the corresponding groundtruth. Through extensive evaluations we demonstrate that the proposed method outperforms the state of the art.


On the Feasibility of Learning Finger gaiting In hand Manipulation with Intrinsic Sensing. Finger gaiting manipulation is an important skill to achieve large angle in hand re orientation of objects. However, achieving these gaits with arbitrary orientations of the hand is challenging due to the unstable nature of the task. In this work, we use model free reinforcement learning (RL) to learn finger gaiting only via precision grasps and demonstrate finger gaiting for rotation about an axis purely using on board proprioceptive and tactile feedback. To tackle the inherent instability of precision grasping, we propose the use of initial state distributions that enable effective exploration of the state space. Our method can learn finger gaiting with significantly improved sample complexity than the state of the art. The policies we obtain are robust and also transfer to novel objects. Videos can be found at https roamlab.github.io learnfg


Non Coherent Modulation for RIS Empowered Multi Antenna OFDM Communications. The Reconfigurable Intelligent Surface (RIS) constitutes one of the prominent technologies for the next 6 th Generation (6G) of wireless communications. It is envisioned to enhance signal coverage in cases where obstacles block the direct communication from Base Stations (BSs), and when high carrier frequencies are used that are sensitive to attenuation losses. In the literature, the exploitation of RISs is exclusively based on traditional coherent demodulation, which necessitates the availability of Channel State Information (CSI). Given the CSI, a multi antenna BS or a dedicated controller computes the pre post spatial coders and the RIS configuration. The latter tasks require significant amount of time and resources, which may not be affordable when the channel is time varying or the CSI is not accurate enough. In this paper, we consider the uplink between a single antenna user and a multi antenna BS and present a novel RIS empowered Orthogonal Frequency Division Multiplexing (OFDM) communication system based on the differential phase shift keying, which is suitable for high noise and or mobility scenarios. Considering both an idealistic and a realistic channel model, analytical expressions for the Signal to Interference and Noise Ratio (SINR) and the Symbol Error Probability (SEP) of the proposed non coherent RIS empowered system are presented. Our extensive computer simulation results verify the accuracy of the presented analysis and showcase the proposed system's performance and superiority over coherent demodulation in different mobility and spatial correlation scenarios.


Only finitely many s Cullen numbers are repunits for a fixed s ge 2 . We show that for any integer s geq 2 , there are only finitely many s Cullen numbers that are repunits. More precisely, for fixed s ge 2 , there are only finitely many integers n , b , and q with n geq 2 , b geq 2 and q geq 3 such that C n,s ns n 1 frac b q 1 b 1 . The proof is elementary and effective, and it is used to show that there are no s Cullen repunits, other than explicitly known ones, for all s in 2,8896 .


The Real Schwarz Lemma The Sequel. A decade ago, when teaching complex analysis, the third named author posed the question on whether or not there is an analogue to the Schwarz lemma for real analytic functions. This led to the note MT , indicating that it is possible to have a real analytic automorphism f of ( 1,1) with f'(0) arbitrarily large. In this note we provide other families with this property, and moreover show that we can always find such a function so that f'(0) equals any desired real number. We end with some questions on related problems.


Cyclically Equivariant Neural Decoders for Cyclic Codes. Neural decoders were introduced as a generalization of the classic Belief Propagation (BP) decoding algorithms, where the Trellis graph in the BP algorithm is viewed as a neural network, and the weights in the Trellis graph are optimized by training the neural network. In this work, we propose a novel neural decoder for cyclic codes by exploiting their cyclically invariant property. More precisely, we impose a shift invariant structure on the weights of our neural decoder so that any cyclic shift of inputs results in the same cyclic shift of outputs. Extensive simulations with BCH codes and punctured Reed Muller (RM) codes show that our new decoder consistently outperforms previous neural decoders when decoding cyclic codes. Finally, we propose a list decoding procedure that can significantly reduce the decoding error probability for BCH codes and punctured RM codes. For certain high rate codes, the gap between our list decoder and the Maximum Likelihood decoder is less than 0.1 dB. Code available at https github.com cyclicallyneuraldecoder CyclicallyEquivariantNeuralDecoders


Relational Neural Markov Random Fields. Statistical Relational Learning (SRL) models have attracted significant attention due to their ability to model complex data while handling uncertainty. However, most of these models have been limited to discrete domains due to their limited potential functions. We introduce Relational Neural Markov Random Fields (RN MRFs) which allow for handling of complex relational hybrid domains. The key advantage of our model is that it makes minimal data distributional assumptions and can seamlessly allow for human knowledge through potentials or relational rules. We propose a maximum pseudolikelihood estimation based learning algorithm with importance sampling for training the neural potential parameters. Our empirical evaluations across diverse domains such as image processing and relational object mapping, clearly demonstrate its effectiveness against non neural counterparts.


Certifying solution geometry in random CSPs counts, clusters and balance. An active topic in the study of random constraint satisfaction problems (CSPs) is the geometry of the space of satisfying or almost satisfying assignments as the function of the density, for which a precise landscape of predictions has been made via statistical physics based heuristics. In parallel, there has been a recent flurry of work on refuting random constraint satisfaction problems, via nailing refutation thresholds for spectral and semidefinite programming based algorithms, and also on counting solutions to CSPs. Inspired by this, the starting point for our work is the following question what does the solution space for a random CSP look like to an efficient algorithm? In pursuit of this inquiry, we focus on the following problems about random Boolean CSPs at the densities where they are unsatisfiable but no refutation algorithm is known. 1. Counts. For every Boolean CSP we give algorithms that with high probability certify a subexponential upper bound on the number of solutions. We also give algorithms to certify a bound on the number of large cuts in a Gaussian weighted graph, and the number of large independent sets in a random d regular graph. 2. Clusters. For Boolean 3 CSPs we give algorithms that with high probability certify an upper bound on the number of clusters of solutions. 3. Balance. We also give algorithms that with high probability certify that there are no "unbalanced" solutions, i.e., solutions where the fraction of 1 s deviates significantly from 50 . Finally, we also provide hardness evidence suggesting that our algorithms for counting are optimal.


Preferences Relations and Consumer Theory. In this paper an interesting application of mathematics in economics is presented the formulation of the theory of consumer basic problem, grounded on the concept of preferences relation and operationalized with optimization tools.


a posteriori stabilized sixth order finite volume scheme with adaptive stencil construction Basics for the 1D steady state hyperbolic equations. We propose an adaptive stencil construction for high order accurate finite volume schemes aposteriori stabilized devoted to solve one dimensional steady state hyperbolic equations. High accuracy (up to the sixth order presently) is achieved thanks to polynomial reconstructions while stability is provided with an aposteriori MOOD method which controls the cell polynomial degree for eliminating non physical oscillations in the vicinity of discontinuities. We supplemented this scheme with a stencil construction allowing to reduce even further the numerical dissipation. The stencil is shifted away from troubles (shocks, discontinuities, etc.) leading to less oscillating polynomial reconstructions. Experimented on linear, B "urgers', and Euler equations, we demonstrate that the adaptive stencil technique manages to retrieve smooth solutions with optimal order of accuracy but also irregular ones without spurious oscillations. Moreover we numerically show that the approach allows to reduce the dissipation still maintaining the essentially non oscillatory behavior.


Polite Emotional Dialogue Acts for Conversational Analysis in Daily Dialog Data. Many socio linguistic cues are used in the conversational analysis, such as emotion, sentiment, and dialogue acts. One of the fundamental social cues is politeness, which linguistically possesses properties useful in conversational analysis. This short article presents some of the brief findings of polite emotional dialogue acts, where we can correlate the relational bonds between these socio linguistics cues. We found that the utterances with emotion classes Anger and Disgust are more likely to be impolite while Happiness and Sadness to be polite. Similar phenomenon occurs with dialogue acts, Inform and Commissive contain many polite utterances than Question and Directive. Finally, we will conclude on the future work of these findings.


Unifying Relational Sentence Generation and Retrieval for Medical Image Report Composition. Beyond generating long and topic coherent paragraphs in traditional captioning tasks, the medical image report composition task poses more task oriented challenges by requiring both the highly accurate medical term diagnosis and multiple heterogeneous forms of information including impression and findings. Current methods often generate the most common sentences due to dataset bias for individual case, regardless of whether the sentences properly capture key entities and relationships. Such limitations severely hinder their applicability and generalization capability in medical report composition where the most critical sentences lie in the descriptions of abnormal diseases that are relatively rare. Moreover, some medical terms appearing in one report are often entangled with each other and co occurred, e.g. symptoms associated with a specific disease. To enforce the semantic consistency of medical terms to be incorporated into the final reports and encourage the sentence generation for rare abnormal descriptions, we propose a novel framework that unifies template retrieval and sentence generation to handle both common and rare abnormality while ensuring the semantic coherency among the detected medical terms. Specifically, our approach exploits hybrid knowledge co reasoning i) explicit relationships among all abnormal medical terms to induce the visual attention learning and topic representation encoding for better topic oriented symptoms descriptions ii) adaptive generation mode that changes between the template retrieval and sentence generation according to a contextual topic encoder. Experimental results on two medical report benchmarks demonstrate the superiority of the proposed framework in terms of both human and metrics evaluation.


The MultiBERTs BERT Reproductions for Robustness Analysis. Experiments with pre trained models such as BERT are often based on a single checkpoint. While the conclusions drawn apply to the artifact tested in the experiment (i.e., the particular instance of the model), it is not always clear whether they hold for the more general procedure which includes the architecture, training data, initialization scheme, and loss function. Recent work has shown that repeating the pre training process can lead to substantially different performance, suggesting that an alternate strategy is needed to make principled statements about procedures. To enable researchers to draw more robust conclusions, we introduce the MultiBERTs, a set of 25 BERT Base checkpoints, trained with similar hyper parameters as the original BERT model but differing in random weight initialization and shuffling of training data. We also define the Multi Bootstrap, a non parametric bootstrap method for statistical inference designed for settings where there are multiple pre trained models and limited test data. To illustrate our approach, we present a case study of gender bias in coreference resolution, in which the Multi Bootstrap lets us measure effects that may not be detected with a single checkpoint. We release our models and statistical library along with an additional set of 140 intermediate checkpoints captured during pre training to facilitate research on learning dynamics.


Construction and convergence results for stable webs. We introduce a new metric for collections of aged paths and a robust set of criteria for compactness for a set of collection of aged paths in the topology corresponding to this metric. We show that the distribution of stable webs ( 1 alpha leq 2 ) made up of collections of stable paths is tight in this topology. We then show the weak convergence of appropriately normalized systems of coalescing random walks in the domain of attraction of stable laws for 1 alpha leq 2 under this metric to the corresponding stable web. We obtain some path results in the brownian case.


Reprogrammable Surfaces Through Star Graph Metamaterials. The ability to change a surface's profile allows biological systems to effectively manipulate and blend into their surroundings. Current surface morphing techniques rely either on having a small number of fixed states or on directly driving the entire system. We discovered a subset of scale independent auxetic metamaterials have a state trajectory with a star graph structure. At the central node, small nudges can move the material between trajectories, allowing us to locally shift Poisson's ratio, causing the material to take on different shapes under loading. While the number of possible shapes grows exponentially with the size of the material, the probability of finding one at random is vanishingly small. By actively guiding the material through the node points, we produce a reprogrammable surface that does not require inputs to maintain shape and can display arbitrary 2D information and take on complex 3D shapes. Our work opens new opportunities in micro devices, tactile displays, manufacturing, and robotic systems.


Weisfeiler Leman in the BAMBOO Novel AMR Graph Metrics and a Benchmark for AMR Graph Similarity. Several metrics have been proposed for assessing the similarity of (abstract) meaning representations (AMRs), but little is known about how they relate to human similarity ratings. Moreover, the current metrics have complementary strengths and weaknesses some emphasize speed, while others make the alignment of graph structures explicit, at the price of a costly alignment step. In this work we propose new Weisfeiler Leman AMR similarity metrics that unify the strengths of previous metrics, while mitigating their weaknesses. Specifically, our new metrics are able to match contextualized substructures and induce n m alignments between their nodes. Furthermore, we introduce a Benchmark for AMR Metrics based on Overt Objectives (BAMBOO), the first benchmark to support empirical assessment of graph based MR similarity metrics. BAMBOO maximizes the interpretability of results by defining multiple overt objectives that range from sentence similarity objectives to stress tests that probe a metric's robustness against meaning altering and meaning preserving graph transformations. We show the benefits of BAMBOO by profiling previous metrics and our own metrics. Results indicate that our novel metrics may serve as a strong baseline for future work.


On the number of roots for harmonic trinomials. In this manuscript we study the counting problem for harmonic trinomials of the form a zeta n b overline zeta m c , where n,m in mathbb N , n m , and a , b and c are non zero complex numbers. As a consequence, we obtain the Fundamental Theorem of Algebra and the Wilmshurst conjecture for harmonic trinomials. The proof of the counting problem relies on the Bohl method introduced in Bohl (1908).


Broadband switchable terahertz half quarter wave plate based on VO2 metal hybrid metasurface with over underdamped transition. Dynamically switchable half quarter wave plates have recently been the focus in the terahertz regime. Conventional design philosophy leads to multilayer metamaterials or narrowband metasurfaces. Here we propose a novel design philosophy and a VO2 metal hybrid metasurface for achieving broadband dynamically switchable half quarter wave plate (HWP QWP) based on the transition from the overdamped to the underdamped resonance. Results show that, by varying the VO2 conductivity by three orders of magnitude, the proposed metasurface's function can be switched between an HWP with polarization conversion ratio larger than 96 and a QWP with ellipticity close to 1 over the broad working band of 0.8 1.2 THz. We expect that the proposed design philosophy will advance the engineering of metasurfaces for dynamically switchable functionalities beyond the terahertz regime.


Self Supervised Learning Disentangled Group Representation as Feature. A good visual representation is an inference map from observations (images) to features (vectors) that faithfully reflects the hidden modularized generative factors (semantics). In this paper, we formulate the notion of "good" representation from a group theoretic view using Higgins' definition of disentangled representation, and show that existing Self Supervised Learning (SSL) only disentangles simple augmentation features such as rotation and colorization, thus unable to modularize the remaining semantics. To break the limitation, we propose an iterative SSL algorithm Iterative Partition based Invariant Risk Minimization (IP IRM), which successfully grounds the abstract semantics and the group acting on them into concrete contrastive learning. At each iteration, IP IRM first partitions the training samples into two subsets that correspond to an entangled group element. Then, it minimizes a subset invariant contrastive loss, where the invariance guarantees to disentangle the group element. We prove that IP IRM converges to a fully disentangled representation and show its effectiveness on various benchmarks. Codes are available at https github.com Wangt CN IP IRM.


Lower bound on size of branch and bound trees for solving lot sizing problem. We show that there exists a family of instances of the lot sizing problem, such that any branch and bound tree that solves them requires an exponential number of nodes, even in the case when the branchings are performed on general split disjunctions.


physics.ao-ph math.OC physics.comp-ph physics.flu-dyn physics.geo-ph

An augmented Lagrangian algorithm for recovery of ice thickness in unidirectional flow using the Shallow Ice Approximation. A key parameter in ice flow modelling is basal slipping at the ice bed interface as it can have a large effect on the resultant ice thickness. Unfortunately, its contribution to surface observations can be hard to distinguish from that of bed undulations. Therefore, inferring the ice thickness from surface measurements is an interesting and non trivial inverse problem. This paper presents a method for recovering dually the ice thickness and the basal slip using only surface elevation and speed measurements. The unidirectional shallow ice approximation is first implemented to model steady state ice flow for given bedrock and basal slip profiles. This surface is then taken as synthetic observed data. An augmented Lagrangian algorithm is then used to find the diffusion coefficient which gives the best fit to observations. Combining this recovered diffusion with observed surface velocity, a simple Newton's method is used to recover both the ice thickness and basal slip. The method was successful in each test case and this implies that it should be possible to recover both of these parameters in two dimensional cases also.


A Multilevel Method for Self Concordant Minimization. The analysis of second order optimization methods based either on sampling, randomization or sketching has two serious shortcomings compared to the conventional Newton method. The first shortcoming is that the analysis of the iterates is not scale invariant, and even if it is, restrictive assumptions are required on the problem structure. The second shortfall is that the fast convergence rates of second order methods have only been established by making assumptions regarding the input data. In this paper, we close the theoretical gap between the theoretical analysis of the conventional Newton method and randomization based second order methods. We propose a Self concordant Iterative minimization Galerkin based Multilevel Algorithm (SIGMA) and establish its super linear convergence rate using the well established theory of self concordant functions. Our analysis is global and entirely independent of unknown constants such as Lipschitz constants and strong convexity parameters. Our analysis is based on the connections between multigrid optimization methods, and the role of coarse grained or reduced order models in the computation of search directions. We take advantage of the insights from the analysis to significantly improve the performance of second order methods in machine learning applications. We report encouraging initial experiments that suggest SIGMA significantly outperforms the state of the art sub sampled sketched Newton methods for both medium and large scale problems.


Asymptotic behavior of the multilevel type error for SDEs driven by a pure jump L 'evy process. Motivated by the multilevel Monte Carlo method introduced by Giles 5 , we study the asymptotic behavior of the normalized error process u n,m (X n X nm ) where X n and X nm are respectively Euler approximations with time steps 1 n and 1 nm of a given stochastic differential equation X driven by a pure jump L 'evy process. In this paper, we prove that this normalized multilevel error converges to different non trivial limiting processes with various sharp rates u n,m depending on the behavior of the L 'evy measure around zero. Our results are consistent with those of Jacod 9 obtained for the normalized error u n(X n X) , as when letting m tends to infinity, we recover the same limiting processes. For the multilevel error, the proofs of the current paper are challenging since unlike 9 we need to deal with m dependent triangular arrays instead of one.


How do the finite size particles modify the drag in Taylor Couette turbulent flow. We experimentally investigate the drag modification by neutrally buoyant finite size particles with various aspect ratios in a Taylor Couette (TC) turbulent flow. The current Reynolds number, Re , ranges from 6.5 times10 3 to 2.6 times10 4 , and the particle volume fraction, Phi , is up to 10 . Particles with three kinds of aspect ratio, lambda , are used lambda 1 3 (oblate), lambda 1 (spherical) and lambda 3 (prolate). Unlike the case of bubbly TC flow, we find that the suspended finite size particles increase the drag of the TC system regardless of their aspect ratios. In addition, the normalized friction coefficient, c f, Phi c f, Phi 0 , decreases with increasing Re , the reason could be that in the current low volume fractions the turbulent stress becomes dominant at higher Re . As Re increases, the particles distribute more evenly in the entire system, which results from both the greater turbulence intensity and the more pronounced finite size effects of the particles at higher Re . Moreover, it is found that the variation of the particle aspect ratios leads to different particle collective effects. The suspended spherical particles, which tend to cluster near the walls and form a particle layer, significantly affect the boundary layer and result in maximum drag modification. The minimal drag modification is found in the oblate case, where the particles preferentially cluster in the bulk region, and thus the particle layer is absent. Based on the optical measurement results, it can be concluded that, in the low volume fraction ranges ( Phi 0.5 and Phi 2 here), the larger drag modification is connected to the near wall particle clustering. The present findings suggest that the particle shape plays a significant role in drag modification.


A submetric characterization of Rolewicz's property ( beta ). The main result is a submetric characterization of the class of Banach spaces admitting an equivalent norm with Rolewicz's property ( beta ). As applications we prove that up to renorming, property ( beta ) is stable under coarse Lipschitz embeddings and coarse quotients.


Degrees of Kalman varieties of tensors. Kalman varieties of tensors are algebraic varieties consisting of tensors whose singular vector k tuples lay on prescribed subvarieties. They were first studied by Ottaviani and Sturmfels in the context of matrices. We extend recent results of Ottaviani and the first author to the partially symmetric setting. We describe a generating function whose coefficients are the degrees of these varieties and we analyze its asymptotics, providing analytic results a la Zeilberger and Pantone. We emphasize the special role of isotropic vectors in the spectral theory of tensors and describe the totally isotropic Kalman variety as a dual variety.


ZARTS On Zero order Optimization for Neural Architecture Search. Differentiable architecture search (DARTS) has been a popular one shot paradigm for NAS due to its high efficiency. It introduces trainable architecture parameters to represent the importance of candidate operations and proposes first second order approximation to estimate their gradients, making it possible to solve NAS by gradient descent algorithm. However, our in depth empirical results show that the approximation will often distort the loss landscape, leading to the biased objective to optimize and in turn inaccurate gradient estimation for architecture parameters. This work turns to zero order optimization and proposes a novel NAS scheme, called ZARTS, to search without enforcing the above approximation. Specifically, three representative zero order optimization methods are introduced RS, MGS, and GLD, among which MGS performs best by balancing the accuracy and speed. Moreover, we explore the connections between RS MGS and gradient descent algorithm and show that our ZARTS can be seen as a robust gradient free counterpart to DARTS. Extensive experiments on multiple datasets and search spaces show the remarkable performance of our method. In particular, results on 12 benchmarks verify the outstanding robustness of ZARTS, where the performance of DARTS collapses due to its known instability issue. Also, we search on the search space of DARTS to compare with peer methods, and our discovered architecture achieves 97.54 accuracy on CIFAR 10 and 75.7 top 1 accuracy on ImageNet, which are state of the art performance.


Volta at SemEval 2021 Task 9 Statement Verification and Evidence Finding with Tables using TAPAS and Transfer Learning. Tables are widely used in various kinds of documents to present information concisely. Understanding tables is a challenging problem that requires an understanding of language and table structure, along with numerical and logical reasoning. In this paper, we present our systems to solve Task 9 of SemEval 2021 Statement Verification and Evidence Finding with Tables (SEM TAB FACTS). The task consists of two subtasks (A) Given a table and a statement, predicting whether the table supports the statement and (B) Predicting which cells in the table provide evidence for against the statement. We fine tune TAPAS (a model which extends BERT's architecture to capture tabular structure) for both the subtasks as it has shown state of the art performance in various table understanding tasks. In subtask A, we evaluate how transfer learning and standardizing tables to have a single header row improves TAPAS' performance. In subtask B, we evaluate how different fine tuning strategies can improve TAPAS' performance. Our systems achieve an F1 score of 67.34 in subtask A three way classification, 72.89 in subtask A two way classification, and 62.95 in subtask B.


Outline to Story Fine grained Controllable Story Generation from Cascaded Events. Large scale pretrained language models have shown thrilling generation capabilities, especially when they generate consistent long text in thousands of words with ease. However, users of these models can only control the prefix of sentences or certain global aspects of generated text. It is challenging to simultaneously achieve fine grained controllability and preserve the state of the art unconditional text generation capability. In this paper, we first propose a new task named "Outline to Story" (O2S) as a test bed for fine grained controllable generation of long text, which generates a multi paragraph story from cascaded events, i.e. a sequence of outline events that guide subsequent paragraph generation. We then create dedicate datasets for future benchmarks, built by state of the art keyword extraction techniques. Finally, we propose an extremely simple yet strong baseline method for the O2S task, which fine tunes pre trained language models on augmented sequences of outline story pairs with simple language modeling objective. Our method does not introduce any new parameters or perform any architecture modification, except several special tokens as delimiters to build augmented sequences. Extensive experiments on various datasets demonstrate state of the art conditional story generation performance with our model, achieving better fine grained controllability and user flexibility. Our paper is among the first ones by our knowledge to propose a model and to create datasets for the task of "outline to story". Our work also instantiates research interest of fine grained controllable generation of open domain long text, where controlling inputs are represented by short text.


Learn from Syntax Improving Pair wise Aspect and Opinion Terms Extractionwith Rich Syntactic Knowledge. In this paper, we propose to enhance the pair wise aspect and opinion terms extraction (PAOTE) task by incorporating rich syntactic knowledge. We first build a syntax fusion encoder for encoding syntactic features, including a label aware graph convolutional network (LAGCN) for modeling the dependency edges and labels, as well as the POS tags unifiedly, and a local attention module encoding POS tags for better term boundary detection. During pairing, we then adopt Biaffine and Triaffine scoring for high order aspect opinion term pairing, in the meantime re harnessing the syntax enriched representations in LAGCN for syntactic aware scoring. Experimental results on four benchmark datasets demonstrate that our model outperforms current state of the art baselines, meanwhile yielding explainable predictions with syntactic knowledge.


Hierarchical Aspect guided Explanation Generation for Explainable Recommendation. Explainable recommendation systems provide explanations for recommendation results to improve their transparency and persuasiveness. The existing explainable recommendation methods generate textual explanations without explicitly considering the user's preferences on different aspects of the item. In this paper, we propose a novel explanation generation framework, named Hierarchical Aspect guided explanation Generation (HAG), for explainable recommendation. Specifically, HAG employs a review based syntax graph to provide a unified view of the user item details. An aspect guided graph pooling operator is proposed to extract the aspect relevant information from the review based syntax graphs to model the user's preferences on an item at the aspect level. Then, a hierarchical explanation decoder is developed to generate aspects and aspect relevant explanations based on the attention mechanism. The experimental results on three real datasets indicate that HAG outperforms state of the art explanation generation methods in both single aspect and multi aspect explanation generation tasks, and also achieves comparable or even better preference prediction accuracy than strong baseline methods.


Predicting the Behavior of Dealers in Over The Counter Corporate Bond Markets. Trading in Over The Counter (OTC) markets is facilitated by broker dealers, in comparison to public exchanges, e.g., the New York Stock Exchange (NYSE). Dealers play an important role in stabilizing prices and providing liquidity in OTC markets. We apply machine learning methods to model and predict the trading behavior of OTC dealers for US corporate bonds. We create sequences of daily historical transaction reports for each dealer over a vocabulary of US corporate bonds. Using this history of dealer activity, we predict the future trading decisions of the dealer. We consider a range of neural network based prediction models. We propose an extension, the Pointwise Product ReZero (PPRZ) Transformer model, and demonstrate the improved performance of our model. We show that individual history provides the best predictive model for the most active dealers. For less active dealers, a collective model provides improved performance. Further, clustering dealers based on their similarity can improve performance. Finally, prediction accuracy varies based on the activity level of both the bond and the dealer.


A Polyhedral Abstraction for Petri nets and its Application to SMT Based Model Checking. We define a new method for taking advantage of net reductions in combination with a SMT based model checker. Our approach consists in transforming a reachability problem about some Petri net, into the verification of an updated reachability property on a reduced version of this net. This method relies on a new state space abstraction based on systems of constraints, called polyhedral abstraction. We prove the correctness of this method using a new notion of equivalence between nets. We provide a complete framework to define and check the correctness of equivalence judgements prove that this relation is a congruence and give examples of basic equivalence relations that derive from structural reductions. Our approach has been implemented in a tool, named SMPT, that provides two main procedures Bounded Model Checking (BMC) and Property Directed Reachability (PDR). Each procedure has been adapted in order to use reductions and to work with arbitrary Petri nets. We tested SMPT on a large collection of queries used in the Model Checking Contest. Our experimental results show that our approach works well, even when we only have a moderate amount of reductions.


Adaptive Sampling for Heterogeneous Rank Aggregation from Noisy Pairwise Comparisons. In heterogeneous rank aggregation problems, users often exhibit various accuracy levels when comparing pairs of items. Thus a uniform querying strategy over users may not be optimal. To address this issue, we propose an elimination based active sampling strategy, which estimates the ranking of items via noisy pairwise comparisons from users and improves the users' average accuracy by maintaining an active set of users. We prove that our algorithm can return the true ranking of items with high probability. We also provide a sample complexity bound for the proposed algorithm which is better than that of non active strategies in the literature. Experiments are provided to show the empirical advantage of the proposed methods over the state of the art baselines.


Stereotype and Skew Quantifying Gender Bias in Pre trained and Fine tuned Language Models. This paper proposes two intuitive metrics, skew and stereotype, that quantify and analyse the gender bias present in contextual language models when tackling the WinoBias pronoun resolution task. We find evidence that gender stereotype correlates approximately negatively with gender skew in out of the box models, suggesting that there is a trade off between these two forms of bias. We investigate two methods to mitigate bias. The first approach is an online method which is effective at removing skew at the expense of stereotype. The second, inspired by previous work on ELMo, involves the fine tuning of BERT using an augmented gender balanced dataset. We show that this reduces both skew and stereotype relative to its unaugmented fine tuned counterpart. However, we find that existing gender bias benchmarks do not fully probe professional bias as pronoun resolution may be obfuscated by cross correlations from other manifestations of gender prejudice. Our code is available online, at https github.com 12kleingordon34 NLP masters project.


Multipliers and Unicentral Diassociative Algebras. The objective of this paper is to develop diassociative analogues of Lie theoretic results from Peggy Batten's 1993 dissertation. We first prove that covers of diassociative algebras are unique. Second, we show that the multiplier of a diassociative algebra is characterized by the second cohomology group with coefficients in the field. Third, we establish criteria for when the center of a cover maps onto the center of the algebra. Along the way, we obtain a collection of exact sequences, characterizations, and a brief theory of unicentral diassociative algebras and stem extensions. This paper is part of an ongoing project to advance extension theory in the context of several Loday algebras.


Cutoff phenomenon of the Glauber dynamics for the Ising model on complete multipartite graphs in the high temperature regime. In this paper, the Glauber dynamics for the Ising model on the complete multipartite graph K np 1, dots,np m is investigated where 0 p i 1 is the proportion of the vertices in the i th component. We show that the dynamics exhibits the cutoff phenomena at t n frac 1 2(1 beta beta cr ) n ln n with window size O(n) in the high temperature regime beta beta cr where beta cr is a constant only depending on p 1, dots,p m . Exponentially slow mixing is shown in the low temperature regime beta beta cr .


Predicting peer to peer and collective social contagion from individual behaviour. Understanding the heterogeneous role of individuals for large scale information spreading is essential to manage online behaviour as well as its potential offline consequences. To this end, most existing studies from diverse research domains focus on the disproportionate role played by highly connected "hub" individuals. However, we demonstrate here that information spreading in online social media is best understood and predicted by simultaneously uncovering two individual level behavioural traits influence and susceptibility. Specifically, we derive a nonlinear network based algorithm to quantify individuals' influence and susceptibility from multiple spreading event data. By applying the algorithm to large scale data from Twitter and Weibo, we demonstrate that both individuals' influence and susceptibility are key determinants of peer to peer information propagation neglecting one of the two properties leads to sub optimal propagation predictions. Besides, we show that both properties are needed to accurately identify superspreaders of information, which challenges common assumptions in studies on influential individuals in social networks.


Creating and Implementing a Smart Speaker. We have seen significant advancements in Artificial Intelligence and Machine Learning in the 21st century. It has enabled a new technology where we can have a human like conversation with the machines. The most significant use of this speech recognition and contextual understanding technology exists in the form of a Smart Speaker. We have a wide variety of Smart Speaker products available to us. This paper aims to decode its creation and explain the technology that makes these Speakers, "Smart."


Horospherical random graphs. Expanders are sparse graph that are strongly connected, where it connectivity is quantified using eigenvalues of the adjacency matrix, and it sparsity in terms of vertex valency. We give a model of random graphs and study their connectivity and sparsity. This model is a particular case of soft geometric random graphs, and allows to construct sparse graphs with good expansion properties, as well as highly clustered ones. On those graphs, we study the speed at which random walks spread in the graph, and visit all vertices. As an illustration, we build a model for mainland France and study the spread of random walks under several types of lockdown. Our experiments show that completely closing medium and long distance travel to slow down the spread of a random walk is more efficient than than local restrictions.


Representation and Invariance in Reinforcement Learning. If we changed the rules, would the wise trade places with the fools? Different groups formalize reinforcement learning (RL) in different ways. If an agent in one RL formalization is to run within another RL formalization's environment, the agent must first be converted, or mapped. A criterion of adequacy for any such mapping is that it preserves relative intelligence. This paper investigates the formulation and properties of this criterion of adequacy. However, prior to the problem of formulation is, we argue, the problem of comparative intelligence. We compare intelligence using ultrafilters, motivated by viewing agents as candidates in intelligence elections where voters are environments. These comparators are counterintuitive, but we prove an impossibility theorem about RL intelligence measurement, suggesting such counterintuitions are unavoidable. Given a mapping between RL frameworks, we establish sufficient conditions to ensure that, for any ultrafilter based intelligence comparator in the destination framework, there exists an ultrafilter based intelligence comparator in the source framework such that the mapping preserves relative intelligence. We consider three concrete mappings between various RL frameworks and show that they satisfy these sufficient conditions and therefore preserve suitably measured relative intelligence.


ast2vec Utilizing Recursive Neural Encodings of Python Programs. Educational datamining involves the application of datamining techniques to student activity. However, in the context of computer programming, many datamining techniques can not be applied because they expect vector shaped input whereas computer programs have the form of syntax trees. In this paper, we present ast2vec, a neural network that maps Python syntax trees to vectors and back, thereby facilitating datamining on computer programs as well as the interpretation of datamining results. Ast2vec has been trained on almost half a million programs of novice programmers and is designed to be applied across learning tasks without re training, meaning that users can apply it without any need for (additional) deep learning. We demonstrate the generality of ast2vec in three settings First, we provide example analyses using ast2vec on a classroom sized dataset, involving visualization, student motion analysis, clustering, and outlier detection, including two novel analyses, namely a progress variance projection and a dynamical systems analysis. Second, we consider the ability of ast2vec to recover the original syntax tree from its vector representation on the training data and two further large scale programming datasets. Finally, we evaluate the predictive capability of a simple linear regression on top of ast2vec, obtaining similar results to techniques that work directly on syntax trees. We hope ast2vec can augment the educational datamining toolbelt by making analyses of computer programs easier, richer, and more efficient.


Counting Baxter Matrices. Donald Knuth recently introduced the notion of a Baxter matrix, generalizing Baxter permutations. We show that for fixed number of rows, r , the number of Baxter matrices with r rows and k columns eventually satisfies a polynomial in k of degree 2r 2 . We also give a proof of Knuth's conjecture that the number of 1's in a r times k Baxter matrix is less than r k .


On Gradient Coding with Partial Recovery. We consider a generalization of the gradient coding framework where a dataset is divided across n workers and each worker transmits to a master node one or more linear combinations of the gradients over its assigned data subsets. Unlike the conventional framework which requires the master node to recover the sum of the gradients over all the data subsets in the presence of straggler workers, we relax the goal to computing the sum of at least some alpha fraction of the gradients. We begin by deriving a lower bound on the computation load of any scheme and also propose two strategies which achieve this lower bound, albeit at the cost of high communication load and a number of data partitions which can be polynomial in n . We then propose schemes based on cyclic assignment which utilize n data partitions and have a lower communication load. When each worker transmits a single linear combination, we prove lower bounds on the computation load of any scheme using n data partitions. Finally, we describe a class of schemes which achieve different intermediate operating points for the computation and communication load and provide simulation results to demonstrate the empirical performance of our schemes.


Anticipating Safety Issues in E2E Conversational AI Framework and Tooling. Over the last several years, end to end neural conversational agents have vastly improved in their ability to carry a chit chat conversation with humans. However, these models are often trained on large datasets from the internet, and as a result, may learn undesirable behaviors from this data, such as toxic or otherwise harmful language. Researchers must thus wrestle with the issue of how and when to release these models. In this paper, we survey the problem landscape for safety for end to end conversational AI and discuss recent and related work. We highlight tensions between values, potential positive impact and potential harms, and provide a framework for making decisions about whether and how to release these models, following the tenets of value sensitive design. We additionally provide a suite of tools to enable researchers to make better informed decisions about training and releasing end to end conversational AI models.


Moment preserving Fourier Galerkin spectral methods and application to the Boltzmann equation. Spectral methods, thanks to the high accuracy and the possibility of using fast algorithms, represent an effective way to approximate collisional kinetic equations in kinetic theory. On the other hand, the loss of some local invariants can lead to the wrong long time behavior of the numerical solution. We introduce in this paper a novel Fourier Galerkin spectral method that improves the classical spectral method by making it conservative on the moments of the approximated distribution, without sacrificing its spectral accuracy or the possibility of using fast algorithms. The method is derived directly using a constrained best approximation in the space of trigonometric polynomials and can be applied to a wide class of problems where preservation of moments is essential. We then apply the new spectral method to the evaluation of the Boltzmann collision term, and prove spectral consistency and stability of the resulting Fourier Galerkin approximation scheme. Various numerical experiments illustrate the theoretical findings.


Why large time stepping methods for the Cahn Hilliard equation is stable. We consider the Cahn Hilliard equation with standard double well potential. We employ a prototypical class of first order in time semi implicit methods with implicit treatment of the linear dissipation term and explicit extrapolation of the nonlinear term. When the dissipation coefficient is held small, a conventional wisdom is to add a judiciously chosen stabilization term in order to afford relatively large time stepping and speed up the simulation. In practical numerical implementations it has been long observed that the resulting system exhibits remarkable stability properties in the regime where the stabilization parameter is mathcal O(1) , the dissipation coefficient is vanishingly small and the size of the time step is moderately large. In this work we develop a new stability theory to address this perplexing phenomenon.


Transversely symplectic Dirac operators on a transversely symplectic foliation. We study the transversely metaplectic structure and the transversely symplectic Dirac operator on a transversely symplectic foliation. Moreover, we give the Weitzenbock type formula for transversely symplectic Dirac operators and we estimate the lower bound of the eigenvalues of the transversely symplectic Dirac operator on special spinors space on transverse K ahler foliations.


The Variational Geometry of Surfaces in the Conformally Flat Space. In this paper, close surfaces are considered in 3 dimensional harmonic conformally flat space in point of the variation. It is shown that if the conformal vector field be tangent to surface and the sign of the mean curvature does not change then surface is minimal. Also, it is determined that the critical point of mean curvature functional is homeomorphic to the sphere. Furthermore, the Euler Lagrange equations associated to the mean curvature and Willmore functionals are determined.


Approximation algorithms for priority Steiner tree problems. In the Priority Steiner Tree (PST) problem, we are given an undirected graph G (V,E) with a source s in V and terminals T subseteq V setminus s , where each terminal v in T requires a nonnegative priority P(v) . The goal is to compute a minimum weight Steiner tree containing edges of varying rates such that the path from s to each terminal v consists of edges of rate greater than or equal to P(v) . The PST problem with k priorities admits a min 2 ln T 2, k rho approximation Charikar et al., 2004 , and is hard to approximate with ratio c log log n for some constant c Chuzhoy et al., 2008 . In this paper, we first strengthen the analysis provided by Charikar et al., 2004 for the (2 ln T 2) approximation to show an approximation ratio of lceil log 2 T rceil 1 le 1.443 ln T 2 , then provide a very simple, parallelizable algorithm which achieves the same approximation ratio. We then consider a more difficult node weighted version of the PST problem, and provide a (2 ln T 2) approximation using extensions of the spider decomposition by Klein Ravi, 1995 . This is the first result for the PST problem in node weighted graphs. Moreover, the approximation ratios for all above algorithms are tight.


Extreme sparsity gives rise to functional specialization. Modularity of neural networks both biological and artificial can be thought of either structurally or functionally, and the relationship between these is an open question. We show that enforcing structural modularity via sparse connectivity between two dense sub networks which need to communicate to solve the task leads to functional specialization of the sub networks, but only at extreme levels of sparsity. With even a moderate number of interconnections, the sub networks become functionally entangled. Defining functional specialization is in itself a challenging problem without a universally agreed solution. To address this, we designed three different measures of specialization (based on weight masks, retraining and correlation) and found them to qualitatively agree. Our results have implications in both neuroscience and machine learning. For neuroscience, it shows that we cannot conclude that there is functional modularity simply by observing moderate levels of structural modularity knowing the brain's connectome is not sufficient for understanding how it breaks down into functional modules. For machine learning, using structure to promote functional modularity which may be important for robustness and generalization may require extremely narrow bottlenecks between modules.


Anomaly Detection in Smart Manufacturing with an Application Focus on Robotic Finishing Systems A Review. As systems in smart manufacturing become increasingly complex, producing an abundance of data, the potential for production failures becomes increasingly more likely. There arises the need to minimize or eradicate production failures, one of which is by means of anomaly detection. However, with the deployment of anomaly detection systems, there are many aspects to be considered. In this paper, an overview of the components, benefits, challenges, methods, and open problems of anomaly detection in smart manufacturing and robotic finishing systems are discussed.


A Shape Newton approach to the problem of covering with identical balls. The problem of covering a region of the plane with a fixed number of minimum radius identical balls is studied in the present work. An explicit construction of bi Lipschitz mappings is provided to model small perturbations of the union of balls. This allows us to obtain analytical expressions for first and second order derivatives using nonsmooth shape optimization techniques under appropriate regularity assumptions. Singular cases are also studied using asymptotic analysis. For the case of regions given by the union of disjoint convex polygons, algorithms based on Voronoi diagrams that do not rely on approximations are given to compute the derivatives. Extensive numerical experiments illustrate the capabilities and limitations of the introduced approach.


Data driven stability analysis of switched linear systems with Sum of Squares guarantees. We present a new data driven method to provide probabilistic stability guarantees for black box switched linear systems. By sampling a finite number of observations of trajectories, we construct approximate Lyapunov functions and deduce the stability of the underlying system with a user defined confidence. The number of observations required to attain this confidence level on the guarantee is explicitly characterized. Our contribution is twofold first, we propose a novel approach for common quadratic Lyapunov functions, relying on sensitivity analysis of a quasi convex optimization program. By doing so, we improve a recently proposed bound. Then, we show that our new approach allows for extension of the method to Sum of Squares Lyapunov functions, providing further improvement for the technique. We demonstrate these improvements on a numerical example.


Effective Resistance for Pandemics Mobility Network Sparsification for High Fidelity Epidemic Simulation. Network science has increasingly become central to the field of epidemiology and our ability to respond to infectious disease threats. However, many networks derived from modern datasets are not just large, but dense, with a high ratio of edges to nodes. This includes human mobility networks where each location has links to a large number of others. Simulating large scale epidemics on these data driven networks requires substantial computational resources and in many cases is practically infeasible. One way to reduce the computational cost of simulating epidemics on these networks is sparsification, where a representative subset of edges is selected based on some measure of their importance. We test several sparsification strategies, ranging from naive thresholding to random sampling of edges, on mobility data from the U.S. Following recent work in computer science, we find that the most accurate approach uses the effective resistances of edges, which prioritizes edges that are the only efficient way to travel between their endpoints. The resulting sparse network preserves many aspects of the behavior of an SIR model, including global quantities, like the epidemic size, and details of stochastic events at individual nodes, including the probability each node becomes infected and its distribution of arrival times. This holds even when the sparse network preserves fewer than 10 of the edges of the original network. In addition to its practical utility, this method helps illuminate which links of a weighted, undirected network are most important to disease spread.


Simulated dose painting of hypoxic sub volumes in pancreatic cancer stereotactic body radiotherapy. Dose painting of hypoxic tumour sub volumes using positron emission tomography (PET) has been shown to improve tumour control in silico in several sites. Pancreatic cancer presents a more stringent challenge, given its proximity to critical organs at risk (OARs) and anatomic motion. A radiobiological model was developed to estimate clonogen survival fraction (SF), using 18F fluoroazomycin arabinoside PET (FAZA PET) images from ten patients with pancreatic cancer to quantify oxygen enhancement effects. For each patient, four simulated five fraction stereotactic body radiotherapy (SBRT) plans were generated 1) a standard SBRT plan aiming to cover the planning target volume with 40 Gy, 2) dose painting plans delivering escalated doses to FAZA avid hypoxic sub volumes, 3) dose painting plans with simulated spacer separating the duodenum and pancreatic head, and 4), plans with integrated boosts to geometric contractions of the tumour (GTV). All plans saturated at least one OAR dose limit. SF was calculated for each plan and sensitivity of SF to simulated hypoxia quantification errors was evaluated. Dose painting resulted in a 55 reduction in SF as compared to standard SBRT 78 with spacer. Integrated boosts to hypoxia blind geometric contractions resulted in a 41 reduction in SF. The reduction in SF for dose painting plans persisted for all hypoxia quantification parameters studied, including registration and rigid motion errors that resulted in shifts and rotations of the GTV and hypoxic sub volumes by as much as 1 cm and 10 degrees. Although proximity to OARs ultimately limited dose escalation, with estimated SFs ( 10 5) well above levels required to completely ablate a 10 cm 3 tumour, dose painting robustly reduced clonogen survival when accounting for expected treatment and imaging uncertainties and thus, may improve local response and associated morbidity.


Optimal Retirement Time and Consumption with the Variation in Habitual Persistence. In this paper,we study the individual's optimal retirement time and optimal consumption under habitual persistence. Because the individual feels equally satisfied with a lower habitual level and is more reluctant to change the habitual level after retirement, we assume that both the level and the sensitivity of the habitual consumption decline at the time of retirement. We establish the concise form of the habitual evolutions, and obtain the optimal retirement time and consumption policy based on martingale and duality methods. The optimal consumption experiences a sharp decline at retirement, but the excess consumption raises because of the reduced sensitivity of the habitual level. This result contributes to explain the "retirement consumption puzzle". Particularly, the optimal retirement and consumption policies are balanced between the wealth effect and the habitual effect. Larger wealth increases consumption, and larger growth inertia (sensitivity) of the habitual level decreases consumption and brings forward the retirement time.


Connecting Images through Time and Sources Introducing Low data, Heterogeneous Instance Retrieval. With impressive results in applications relying on feature learning, deep learning has also blurred the line between algorithm and data. Pick a training dataset, pick a backbone network for feature extraction, and voil a this usually works for a variety of use cases. But the underlying hypothesis that there exists a training dataset matching the use case is not always met. Moreover, the demand for interconnections regardless of the variations of the content calls for increasing generalization and robustness in features. An interesting application characterized by these problematics is the connection of historical and cultural databases of images. Through the seemingly simple task of instance retrieval, we propose to show that it is not trivial to pick features responding well to a panel of variations and semantic content. Introducing a new enhanced version of the Alegoria benchmark, we compare descriptors using the detailed annotations. We further give insights about the core problems in instance retrieval, testing four state of the art additional techniques to increase performance.


NPT Loss A Metric Loss with Implicit Mining for Face Recognition. Face recognition (FR) using deep convolutional neural networks (DCNNs) has seen remarkable success in recent years. One key ingredient of DCNN based FR is the appropriate design of a loss function that ensures discrimination between various identities. The state of the art (SOTA) solutions utilise normalised Softmax loss with additive and or multiplicative margins. Despite being popular, these Softmax margin based losses are not theoretically motivated and the effectiveness of a margin is justified only intuitively. In this work, we utilise an alternative framework that offers a more direct mechanism of achieving discrimination among the features of various identities. We propose a novel loss that is equivalent to a triplet loss with proxies and an implicit mechanism of hard negative mining. We give theoretical justification that minimising the proposed loss ensures a minimum separability between all identities. The proposed loss is simple to implement and does not require heavy hyper parameter tuning as in the SOTA solutions. We give empirical evidence that despite its simplicity, the proposed loss consistently achieves SOTA performance in various benchmarks for both high resolution and low resolution FR tasks.


Accelerating Distributed K FAC with Smart Parallelism of Computing and Communication Tasks. Distributed training with synchronous stochastic gradient descent (SGD) on GPU clusters has been widely used to accelerate the training process of deep models. However, SGD only utilizes the first order gradient in model parameter updates, which may take days or weeks. Recent studies have successfully exploited approximate second order information to speed up the training process, in which the Kronecker Factored Approximate Curvature (KFAC) emerges as one of the most efficient approximation algorithms for training deep models. Yet, when leveraging GPU clusters to train models with distributed KFAC (D KFAC), it incurs extensive computation as well as introduces extra communications during each iteration. In this work, we propose D KFAC (SPD KFAC) with smart parallelism of computing and communication tasks to reduce the iteration time. Specifically, 1) we first characterize the performance bottlenecks of D KFAC, 2) we design and implement a pipelining mechanism for Kronecker factors computation and communication with dynamic tensor fusion, and 3) we develop a load balancing placement for inverting multiple matrices on GPU clusters. We conduct real world experiments on a 64 GPU cluster with 100Gb s InfiniBand interconnect. Experimental results show that our proposed SPD KFAC training scheme can achieve 10 35 improvement over state of the art algorithms.


Measurement of ion displacement via RF power variation for excess micromotion compensation. We demonstrate a method of micromotion minimization of a trapped ion in a linear Paul trap based on the precision measurement of the ion trapping position displacement due to a stray electric field in the radial plane by ion fluorescence imaging. The amount of displacement in the radial plane is proportional to the strength of a stray electric field. Therefore, we evaluated the micromotion compensation condition by measuring the ion displacements from the ion equilibrium position using two different radial trap frequencies with various combinations of the compensation voltage. The residual electric field uncertainty of this technique reached a few volts per meter. This compensation technique does not depend on the orientation of the incident cooling laser or the detuning and imaging direction. Therefore, this method is suitable for a planar ion trap, a stylus ion trap, which limits the propagation angle of lasers, or miniaturized ion trap systems for sensing and metrological applications.


An Automated Data Engineering Pipeline for Anomaly Detection of IoT Sensor Data. The rapid development in the field of System of Chip (SoC) technology, Internet of Things (IoT), cloud computing, and artificial intelligence has brought more possibilities of improving and solving the current problems. With data analytics and the use of machine learning deep learning, it is made possible to learn the underlying patterns and make decisions based on what was learned from massive data generated from IoT sensors. When combined with cloud computing, the whole pipeline can be automated, and free of manual controls and operations. In this paper, an implementation of an automated data engineering pipeline for anomaly detection of IoT sensor data is studied and proposed. The process involves the use of IoT sensors, Raspberry Pis, Amazon Web Services (AWS) and multiple machine learning techniques with the intent to identify anomalous cases for the smart home security system.


Stopping Rules for Algebraic Iterative Reconstruction Methods in Computed Tomography. Algebraic models for the reconstruction problem in X ray computed tomography (CT) provide a flexible framework that applies to many measurement geometries. For large scale problems we need to use iterative solvers, and we need stopping rules for these methods that terminate the iterations when we have computed a satisfactory reconstruction that balances the reconstruction error and the influence of noise from the measurements. Many such stopping rules are developed in the inverse problems communities, but they have not attained much attention in the CT world. The goal of this paper is to describe and illustrate four stopping rules that are relevant for CT reconstructions.


Joint equidistribution on the product of the circle and the unit tangent bundle of the modular surface. We use spectral method to prove a joint equidistribution of primitive rational points and the same along expanding horocycle orbits in the products of the circle and the unit cotangent bundle of the modular surface. This result explicates the error bound in a recent work of Einsiedler, Luethi, and Shah cite Theorem 1.1 ELS . The error is sharp upon the best known progress towards the Ramanujan conjecture at the finite places for the modular surface.


Tameness of definably complete locally o minimal structures and definable bounded multiplication. We first show that the projection image of a discrete definable set is again discrete for an arbitrary definably complete locally o minimal structure. This fact together with the results in a previous paper implies tame dimension theory and decomposition theorem into good shaped definable subsets called quasi special submanifolds. Using this fact, in the latter part of this paper, we investigate definably complete locally o minimal expansions of ordered groups when the restriction of multiplication to an arbitrary bounded open box is definable. Similarly to o minimal expansions of ordered fields, L ojasiewicz's inequality, Tietze extension theorem and affiness of psudo definable spaces hold true for such structures under the extra assumption that the domains of definition and the psudo definable spaces are definably compact. Here, a pseudo definable space is a topological spaces having finite definable atlases. We also demonstrate Michael's selection theorem for definable set valued functions with definably compact domains of definition.


Searching to Learn with Instructional Scaffolding. Search engines are considered the primary tool to assist and empower learners in finding information relevant to their learning goals be it learning something new, improving their existing skills, or just fulfilling a curiosity. While several approaches for improving search engines for the learning scenario have been proposed, instructional scaffolding has not been studied in the context of search as learning, despite being shown to be effective for improving learning in both digital and traditional learning contexts. When scaffolding is employed, instructors provide learners with support throughout their autonomous learning process. We hypothesize that the usage of scaffolding techniques within a search system can be an effective way to help learners achieve their learning objectives whilst searching. As such, this paper investigates the incorporation of scaffolding into a search system employing three different strategies (as well as a control condition) (I) AQE SC , the automatic expansion of user queries with relevant subtopics (ii) CURATED SC , the presenting of a manually curated static list of relevant subtopics on the search engine result page and (iii) FEEDBACK SC , which projects real time feedback about a user's exploration of the topic space on top of the CURATED SC visualization. To investigate the effectiveness of these approaches with respect to human learning, we conduct a user study (N 126) where participants were tasked with searching and learning about topics such as genetically modified organisms'. We find that (I) the introduction of the proposed scaffolding methods does not significantly improve learning gains. However, (ii) it does significantly impact search behavior. Furthermore, (iii) immediate feedback of the participants' learning leads to undesirable user behavior, with participants focusing on the feedback gauges instead of learning.


Asymmetric four grating compressor for ultrafast high power lasers. The peak power improvement and running safety of petawatt (PW) lasers are limited by laser induced damage of optical components with limited sizes and damage thresholds. Diffraction gratings in pulse compressors have been the shortest stave of PW lasers up to now, as to manufacture a high quality meter sized grating remains particularly challenging. Here, the asymmetric four grating compressor (AFGC) with asymmetric configuration is proposed for PW lasers to increase the maximum bearable output pulse energy and running safety without neither additional optical component nor extra control in comparison to a traditional Treacy four grating compressor (TFGC) with symmetric configuration. In AFGC, suitable spatial dispersion can be introduced in the output laser beam which is able to decrease the laser spatial intensity modulation (LSIM) of the output beam on the final grating. The introduced spatial dispersion can be automatically compensated at the focal plane by using the spatiotemporal focusing technique. Based on this simple AFGC design, not only the damage risk of the final grating can be reduced, but also the maximum output pulse energy can be improved by about 1.8 times theoretically. As an example, 100 PW output power can be achieved theoretically by using the AFGC with an input beam size of 550 700 mm2.


Learning Human Objectives from Sequences of Physical Corrections. When personal, assistive, and interactive robots make mistakes, humans naturally and intuitively correct those mistakes through physical interaction. In simple situations, one correction is sufficient to convey what the human wants. But when humans are working with multiple robots or the robot is performing an intricate task often the human must make several corrections to fix the robot's behavior. Prior research assumes each of these physical corrections are independent events, and learns from them one at a time. However, this misses out on crucial information each of these interactions are interconnected, and may only make sense if viewed together. Alternatively, other work reasons over the final trajectory produced by all of the human's corrections. But this method must wait until the end of the task to learn from corrections, as opposed to inferring from the corrections in an online fashion. In this paper we formalize an approach for learning from sequences of physical corrections during the current task. To do this we introduce an auxiliary reward that captures the human's trade off between making corrections which improve the robot's immediate reward and long term performance. We evaluate the resulting algorithm in remote and in person human robot experiments, and compare to both independent and final baselines. Our results indicate that users are best able to convey their objective when the robot reasons over their sequence of corrections.


Field canalization using anisotropic 2D plasmonics. Optical devices capable of suppressing diffraction nature of light are of great technological importance to many nanophotonic applications. One important technique to achieve diffractionless optics is to exploit field canalization effect. However, current technological platforms based on metamaterial structures typically suffer from strict loss confinement tradeoff, or lack dynamic reconfigurability over device operations. Here we report an integrated canalization platform that can alleviate this performance tradeoff. It is found that by leveraging material absorption of anisotropic 2D materials, the dispersion of this class of materials can flatten without increasing propagation losses and compromising confinement. The realization of such plasmon canalization can be considered using black phosphorus (BP), where topological transition from elliptic to hyperbolic curves can be induced by dynamically leveraging material absorption of BP. At the transition point, BP film can support long range, deeply subwavelength, near diffractionless field propagation, exhibiting diffraction angle of 5.5o, propagation distance of 10 lambda spp, and lambda spp lambda 0 300.


Is math useful?. "Is math useful?" might sound as a trick question. And it is. Of course math is useful, we live in a data filled world and every aspect of life is totally entwined with math applications, both trivial and subtle applications, of both basic and advanced math. But we need to ask once again that question, in order to truly understand what is math useful for and what being useful means. Moreover, is it knowledge of math useful for a class of specialists, or for political leaders or for all people at large? Being more on a concrete level, why does math need to have a central role in education? Each section will be titled by a question. And each section will not give an answer, but at least I hope provide some food for tought to the reader, in order to try to come up with his or her own answers. I feel that these kind of questions are at home in a book devoted to the interplays between mathematics and culture what is the space we should give to math in culture and what is math's role in becoming a complete citizen?


Radiative Transfer in a Fluid. We study the Radiative Transfer equations coupled with the time dependent temperature equation of a fluid existence, uniqueness, a maximum principle are established. A short numerical section illustrates the pros and cons of the method.


Longitudinal patient stratification of electronic health records with flexible adjustment for clinical outcomes. The increase in availability of longitudinal electronic health record (EHR) data is leading to improved understanding of diseases and discovery of novel phenotypes. The majority of clustering algorithms focus only on patient trajectories, yet patients with similar trajectories may have different outcomes. Finding subgroups of patients with different trajectories and outcomes can guide future drug development and improve recruitment to clinical trials. We develop a recurrent neural network autoencoder to cluster EHR data using reconstruction, outcome, and clustering losses which can be weighted to find different types of patient clusters. We show our model is able to discover known clusters from both data biases and outcome differences, outperforming baseline models. We demonstrate the model performance on 29,229 diabetes patients, showing it finds clusters of patients with both different trajectories and different outcomes which can be utilized to aid clinical decision making.


LightMBERT A Simple Yet Effective Method for Multilingual BERT Distillation. The multilingual pre trained language models (e.g, mBERT, XLM and XLM R) have shown impressive performance on cross lingual natural language understanding tasks. However, these models are computationally intensive and difficult to be deployed on resource restricted devices. In this paper, we propose a simple yet effective distillation method (LightMBERT) for transferring the cross lingual generalization ability of the multilingual BERT to a small student model. The experiment results empirically demonstrate the efficiency and effectiveness of LightMBERT, which is significantly better than the baselines and performs comparable to the teacher mBERT.


Sonodynamic effect in A375 melanoma cells with Chlorin e6 induced by 20 kHz ultrasound. In this research, combined effect of chlorin e6 and noninvasive 20 kHz, low powered ultrasound on melanoma cells is proposed for the first time. The proposed combination incorporates the application of safe low frequency ultrasound that is used for transdermal drug delivery. We demonstrate that impact of the 20 kHz ultrasound on chlorin e6 leads to strong sonodynamic effect in melanoma cells. We also show that chlorin e6 keeps its monomeric form in the cells and does not aggregate after exposure to ultrasound. We propose a two step model where sonoluminescence serves as an internal light source for the chlorin e6 photoactivation.


The Chern class for K 3 and the cyclic quantum dilogarithm. In this note we confirm the guess of Calegari, Garoufalidis and Zagier in Arxiv 1712.04887 that R zeta c zeta 2 , where R zeta is their map on K 3 defined using the cyclic quantum dilogarithm and c zeta is the Chern class map on K 3 .


The Interplay of Variant, Size, and Task Type in Arabic Pre trained Language Models. In this paper, we explore the effects of language variants, data sizes, and fine tuning task types in Arabic pre trained language models. To do so, we build three pre trained language models across three variants of Arabic Modern Standard Arabic (MSA), dialectal Arabic, and classical Arabic, in addition to a fourth language model which is pre trained on a mix of the three. We also examine the importance of pre training data size by building additional models that are pre trained on a scaled down set of the MSA variant. We compare our different models to each other, as well as to eight publicly available models by fine tuning them on five NLP tasks spanning 12 datasets. Our results suggest that the variant proximity of pre training data to fine tuning data is more important than the pre training data size. We exploit this insight in defining an optimized system selection model for the studied tasks.


Beamforming Design for IRS Aided Decode and Forward Relay Wireless Network. As a low cost and low power consumption passive reflector, intelligent reflecting surface (IRS) can make a significant rate improvement by building a programmable wireless environment. To improve the rate performance and coverage range of wireless networks, an IRS aided decode and forward (DF) relay network is proposed with multiple antennas at relay station (RS). To achieve a high rate, an alternately iterative structure (AIS) of maximizing receive power (Max RP) at RS is proposed to jointly optimize the beamforming vectors at RS and phase shifts at IRS. Considering its high complexity, two low complexity Max RP schemes of null space projection (NSP) plus maximum ratio combining (MRC) and IRS element selection (IRSES) plus MRC are presented to reduce this complexity, respectively. For the former, NSP is used to separate the reflected signal from IRS and the direct transmitted signal from source and MRC is adopted to combine the two signals at RS. For the latter, the basic concept of IRSES is as follows IRS is partitioned into M subsets of elements and adjusting the phases of all elements per subset make all reflected signals and the direct signal from source phase alignment (PA) at the corresponding antenna of relay. Simulation results show that the proposed three methods perform much better than the existing network with single antenna relay in terms of rate performance. In particular, a 85 rate gain over existing scheme is achieved in the high signal to noise ratio region. Moreover, it is verified that the positions of RS and IRS have a substantial impact on rate performance, and there exists an optimal positions of RS and IRS.


Coupling local and nonlocal equations with Neumann boundary conditions. We introduce two different ways of coupling local and nonlocal equations with Neumann boundary conditions in such a way that the resulting model is naturally associated with an energy functional. For these two models we prove that there is a minimizer of the resulting energy that is unique modulo adding a constant.


Homogeneous sub Riemannian geodesics on the group of motions of the plane. We describe homogeneous sub Riemannian geodesics for the standard sub Riemannian structure on the group of proper motions of the plane SE(2). We show that this structure is not geodesically orbital, although the cut time is invariant w.r.t. shift of the initial point along geodesic.


A further generalization of the Glauberman Thompson p nilpotency criterion in fusion systems. Let p be a prime and mathcal F be a saturated fusion system over a finite p group P . The fusion system mathcal F is said to be nilpotent if mathcal F mathcal F P (P) . We provide new criteria for a saturated fusion system mathcal F to be nilpotent, which may be viewed as extending the Glauberman Thompson p nilpotency criterion to fusion systems.


The average number of integral points on the congruent number curves. We show that the total number of non torsion integral points on the curves y 2 x 3 D 2x , where D ranges over positive squarefree integers less than N , is ll N( log N) 1 8 .


Internet of Things (IoT) Connectivity in 6G An Interplay of Time, Space, Intelligence, and Value. Internet of Things (IoT) connectivity has a prominent presence in the 5G wireless communication systems. As these systems are being deployed, there is a surge of research efforts and visions towards 6G wireless systems. In order to position the evolution of IoT within the 6G systems, this paper first takes a critical view on the way IoT connectivity is supported within 5G. Following that, the wireless IoT evolution is discussed through multiple dimensions time, space, intelligence, and value. We also conjecture that the focus will broaden from IoT devices and their connections towards the emergence of complex IoT environments, seen as building blocks of the overall IoT ecosystem.


Position Information in Transformers An Overview. Transformers are arguably the main workhorse in recent Natural Language Processing research. By definition a Transformer is invariant with respect to reordering of the input. However, language is inherently sequential and word order is essential to the semantics and syntax of an utterance. In this article, we provide an overview and theoretical comparison of existing methods to incorporate position information into Transformer models. The objectives of this survey are to (1) showcase that position information in Transformer is a vibrant and extensive research area (2) enable the reader to compare existing methods by providing a unified notation and systematization of different approaches along important model dimensions (3) indicate what characteristics of an application should be taken into account when selecting a position encoding (4) provide stimuli for future research.


HCDG A Hierarchical Consistency Framework for Domain Generalization on Medical Image Segmentation. Modern deep neural networks struggle to transfer knowledge and generalize across domains when deploying to real world applications. Domain generalization (DG) aims to learn a universal representation from multiple source domains to improve the network generalization ability on unseen target domains. Previous DG methods mostly focus on the data level consistency scheme to advance the generalization capability of deep networks, without considering the synergistic regularization of different consistency schemes. In this paper, we present a novel Hierarchical Consistency framework for Domain Generalization (HCDG) by ensembling Extrinsic Consistency and Intrinsic Consistency. Particularly, for Extrinsic Consistency, we leverage the knowledge across multiple source domains to enforce data level consistency. Also, we design a novel Amplitude Gaussian mixing strategy for Fourier based data augmentation to enhance such consistency. For Intrinsic Consistency, we perform task level consistency for the same instance under the dual task form. We evaluate the proposed HCDG framework on two medical image segmentation tasks, i.e., optic cup disc segmentation on fundus images and prostate MRI segmentation. Extensive experimental results manifest the effectiveness and versatility of our HCDG framework. Code will be available once accept.


Biomarker Gene Identification for Breast Cancer Classification. BACKGROUND Breast cancer has emerged as one of the most prevalent cancers among women leading to a high mortality rate. Due to the heterogeneous nature of breast cancer, there is a need to identify differentially expressed genes associated with breast cancer subtypes for its timely diagnosis and treatment. OBJECTIVE To identify a small gene set for each of the four breast cancer subtypes that could act as its signature, the paper proposes a novel algorithm for gene signature identification. METHODS The present work uses interpretable AI methods to investigate the predictions made by the deep neural network employed for subtype classification to identify biomarkers using the TCGA breast cancer RNA Sequence data. RESULTS The proposed algorithm led to the discovery of a set of 43 differentially expressed gene signatures. We achieved a competitive average 10 fold accuracy of 0.91, using neural network classifier. Further, gene set analysis revealed several relevant pathways, such as GRB7 events in ERBB2 and p53 signaling pathway. Using the Pearson correlation matrix, we noted that the subtype specific genes are correlated within each subtype. CONCLUSIONS The proposed technique enables us to find a concise and clinically relevant gene signature set.


Degree preserving graph dynamics a versatile process to construct random networks. Real world networks evolve over time via additions or removals of nodes and edges. In current network evolution models, node degree varies or grows arbitrarily. A recently introduced degree preserving network growth (DPG) family of models preserves node degree, resulting in structures significantly different from and more diverse than previous models (Nature Physics 2021, DOI 10.1038 s41567 021 01417 7). Here we present a rigorous mathematical theory underlying the DPG family of network growth models. We prove that the general problem of deciding whether a simple graph can be obtained via the DPG process from a small "kernel" graph (DPG feasibility) is NP complete, in contrast with the surprising numerical observation that most real world networks are actually easily constructible by this process a dichotomy that still needs to be understood. We demonstrate how some of the well known network models can be constructed via the DPG process, using proper parametrization.


A massively parallel explicit solver for elasto dynamic problems exploiting octree meshes. Typical areas of application of explicit dynamics are impact, crash test, and most importantly, wave propagation simulations. Due to the numerically highly demanding nature of these problems, efficient automatic mesh generators and transient solvers are required. To this end, a parallel explicit solver exploiting the advantages of balanced octree meshes is introduced. To avoid the hanging nodes problem encountered in standard finite element analysis (FEA), the scaled boundary finite element method (SBFEM) is deployed as a spatial discretization scheme. Consequently, arbitrarily shaped star convex polyhedral elements are straightforwardly generated. Considering the scaling and transformation of octree cells, the stiffness and mass matrices of a limited number of unique cell patterns are pre computed. A recently proposed mass lumping technique is extended to 3D yielding a well conditioned diagonal mass matrix. This enables us to leverage the advantages of explicit time integrator, i.e., it is possible to efficiently compute the nodal displacements without the need for solving a system of linear equations. We implement the proposed scheme together with a central difference method (CDM) in a distributed computing environment. The performance of our parallel explicit solver is evaluated by means of several numerical benchmark examples, including complex geometries and various practical applications. A significant speedup is observed for these examples with up to one billion of degrees of freedom and running on up to 16,384 computing cores.


Score Based Explanations in Data Management and Machine Learning An Answer Set Programming Approach to Counterfactual Analysis. We describe some recent approaches to score based explanations for query answers in databases and outcomes from classification models in machine learning. The focus is on work done by the author and collaborators. Special emphasis is placed on declarative approaches based on answer set programming to the use of counterfactual reasoning for score specification and computation. Several examples that illustrate the flexibility of these methods are shown.


Long Term Object Detection and Tracking in Collaborative Learning Environments. Human activity recognition in videos is a challenging problem that has drawn a lot of interest, particularly when the goal requires the analysis of a large video database. AOLME project provides a collaborative learning environment for middle school students to explore mathematics, computer science, and engineering by processing digital images and videos. As part of this project, around 2200 hours of video data was collected for analysis. Because of the size of the dataset, it is hard to analyze all the videos of the dataset manually. Thus, there is a huge need for reliable computer based methods that can detect activities of interest. My thesis is focused on the development of accurate methods for detecting and tracking objects in long videos. All the models are validated on videos from 7 different sessions, ranging from 45 minutes to 90 minutes. The keyboard detector achieved a very high average precision (AP) of 92 at 0.5 intersection over union (IoU). Furthermore, a combined system of the detector with a fast tracker KCF (159fps) was developed so that the algorithm runs significantly faster without sacrificing accuracy. For a video of 23 minutes having resolution 858X480 30 fps, the detection alone runs at 4.7Xthe real time, and the combined algorithm runs at 21Xthe real time for an average IoU of 0.84 and 0.82, respectively. The hand detector achieved average precision (AP) of 72 at 0.5 IoU. The detection results were improved to 81 using optimal data augmentation parameters. The hand detector runs at 4.7Xthe real time with AP of 81 at 0.5 IoU. The hand detection method was integrated with projections and clustering for accurate proposal generation. This approach reduced the number of false positive hand detections by 80 . The overall hand detection system runs at 4Xthe real time, capturing all the activity regions of the current collaborative group.


Finite Sample Analysis of Two Time Scale Natural Actor Critic Algorithm. Actor critic style two time scale algorithms are one of the most popular methods in reinforcement learning, and have seen great empirical success. However, their performance is not completely understood theoretically. In this paper, we characterize the emph global convergence of an online natural actor critic algorithm in the tabular setting using a single trajectory of samples. Our analysis applies to very general settings, as we only assume ergodicity of the underlying Markov decision process. In order to ensure enough exploration, we employ an epsilon greedy sampling of the trajectory. For a fixed and small enough exploration parameter epsilon , we show that the two time scale natural actor critic algorithm has a rate of convergence of tilde mathcal O (1 T 1 4 ) , where T is the number of samples, and this leads to a sample complexity of Tilde mathcal O (1 delta 8 ) samples to find a policy that is within an error of delta from the emph global optimum . Moreover, by carefully decreasing the exploration parameter epsilon as the iterations proceed, we present an improved sample complexity of Tilde mathcal O (1 delta 6 ) for convergence to the global optimum.


A case study of non Fourier heat conduction using Internal Variables and GENERIC. Applying simultaneously the methodology of Non Equilibrium Thermodynamics with Internal Variables (NET IV) and the framework of General Equation for the Non Equilibrium Reversible Irreversible Coupling (GENERIC), we demonstrate that, in heat conduction theories, entropy current multipliers can be interpreted as relaxed state variables. Fourier's law and its various extensions the Maxwell Cattaneo Vernotte, Guyer Krumhansl, Jeffreys type, Ginzburg Landau (Allen Cahn) type and ballistic diffusive heat conduction equations are derived in both formulations. Along these lines, a comparison of NET IV and GENERIC is also performed. Our results may pave the way for microscopic multiscale understanding of beyond Fourier heat conduction, and open new ways for numerical simulations of heat conduction problems.


Fixed point theorems for asymptotically T regular mappings in preordered modular G metric spaces applied to solving nonlinear integral equations. Our aim in this paper is to prove some interesting fixed point theorems for the class of asymptotically T regular mappings in the framework of preordered modular G metric spaces. Our results are novel and generalizes several know results. Furthermore we apply our results in solving nonlinear integral equations.