Time Frequency Localization Using Deep Convolutional Maxout Neural Network in Persian Speech Recognition. In this paper, a CNN based structure for the time frequency localization of information is proposed for Persian speech recognition. Research has shown that the receptive fields' spectrotemporal plasticity of some neurons in mammals' primary auditory cortex and midbrain makes localization facilities improve recognition performance. Over the past few years, much work has been done to localize time frequency information in ASR systems, using the spatial or temporal immutability properties of methods such as HMMs, TDNNs, CNNs, and LSTM RNNs. However, most of these models have large parameter volumes and are challenging to train. For this purpose, we have presented a structure called Time Frequency Convolutional Maxout Neural Network (TFCMNN) in which parallel time domain and frequency domain 1D CMNNs are applied simultaneously and independently to the spectrogram, and then their outputs are concatenated and applied jointly to a fully connected Maxout network for classification. To improve the performance of this structure, we have used newly developed methods and models such as Dropout, maxout, and weight normalization. Two sets of experiments were designed and implemented on the FARSDAT dataset to evaluate the performance of this model compared to conventional 1D CMNN models. According to the experimental results, the average recognition score of TFCMNN models is about 1.6 higher than the average of conventional 1D CMNN models. In addition, the average training time of the TFCMNN models is about 17 hours lower than the average training time of traditional models. Therefore, as proven in other sources, time frequency localization in ASR systems increases system accuracy and speeds up the training process.


Probing BERT in Hyperbolic Spaces. Recently, a variety of probing tasks are proposed to discover linguistic properties learned in contextualized word embeddings. Many of these works implicitly assume these embeddings lay in certain metric spaces, typically the Euclidean space. This work considers a family of geometrically special spaces, the hyperbolic spaces, that exhibit better inductive biases for hierarchical structures and may better reveal linguistic hierarchies encoded in contextualized representations. We introduce a Poincare probe, a structural probe projecting these embeddings into a Poincare subspace with explicitly defined hierarchies. We focus on two probing objectives (a) dependency trees where the hierarchy is defined as head dependent structures (b) lexical sentiments where the hierarchy is defined as the polarity of words (positivity and negativity). We argue that a key desideratum of a probe is its sensitivity to the existence of linguistic structures. We apply our probes on BERT, a typical contextualized embedding model. In a syntactic subspace, our probe better recovers tree structures than Euclidean probes, revealing the possibility that the geometry of BERT syntax may not necessarily be Euclidean. In a sentiment subspace, we reveal two possible meta embeddings for positive and negative sentiments and show how lexically controlled contextualization would change the geometric localization of embeddings. We demonstrate the findings with our Poincare probe via extensive experiments and visualization. Our results can be reproduced at https github.com FranxYao PoincareProbe.


Language Invariant Properties in Natural Language Processing. Meaning is context dependent, but many properties of language (should) remain the same even if we transform the context. For example, sentiment, entailment, or speaker properties should be the same in a translation and original of a text. We introduce language invariant properties i.e., properties that should not change when we transform text, and how they can be used to quantitatively evaluate the robustness of transformation algorithms. We use translation and paraphrasing as transformation examples, but our findings apply more broadly to any transformation. Our results indicate that many NLP transformations change properties like author characteristics, i.e., make them sound more male. We believe that studying these properties will allow NLP to address both social factors and pragmatic aspects of language. We also release an application suite that can be used to evaluate the invariance of transformation applications.


Scalar Gaussian Wiretap Channel Properties of the Support Size of the Secrecy Capacity Achieving Distribution. This work studies the secrecy capacity of a scalar Gaussian wiretap channel with an amplitude constraint on the input. It is known that for this channel, the secrecy capacity achieving distribution is discrete with finitely many points. This work improves such result by showing an upper bound of the order frac mathsf A sigma 1 2 where mathsf A is the amplitude constraint and sigma 1 2 is the variance of the Gaussian noise over the legitimate channel.


Quality Evaluation of the Low Resource Synthetically Generated Code Mixed Hinglish Text. In this shared task, we seek the participating teams to investigate the factors influencing the quality of the code mixed text generation systems. We synthetically generate code mixed Hinglish sentences using two distinct approaches and employ human annotators to rate the generation quality. We propose two subtasks, quality rating prediction and annotators' disagreement prediction of the synthetic Hinglish dataset. The proposed subtasks will put forward the reasoning and explanation of the factors influencing the quality and human perception of the code mixed text.


Solar System Dynamics and Multiyear Droughts of the Western USA. The recent addition of orbit spin coupling torques to atmospheric global circulation models has enabled successful years in advance forecasts of global and regional scale dust storms on Mars. Here we explore the applicability of the orbit spin coupling mechanism for understanding and forecasting anomalous weather and climate events on Earth. We calculate the time history of orbit spin coupling torques on the Earth system for the interval from 1860 2040. The torque exhibits substantial variability on decadal to bidecadal timescales. Deep minima recur at intervals from 15 26 years eight such episodes are documented within the study period prior to 2020. Each of the identified torque minima corresponds in time to an episode of widespread drought in the Western USA extending over several years. The multiyear droughts of the 1930s, the 1950s, the mid 1970s, the early 1990s, and of 2011 2015 were each coincident in time with orbit spin coupling torque minima. The upcoming torque minimum of 2030 is the deepest such minimum of the 180 yr study interval. A multiyear episode of widespread drought in the Western USA is likely to be underway by 2028 plus or minus 4 years (2 standard deviations). The potential benefits to societies of improved drought predictions justify an immediate high priority effort to include forcing by orbit spin coupling within state of the art Earth system GCMs. Future targeted numerical modeling investigations are likely to yield forecasts with considerably lower uncertainties and with much improved temporal resolution in comparison to that obtained here.


Graph Drawing and Network Visualization GD2020. Proceedings of GD2020 This volume contains the papers presented at GD 2020, the 28th International Symposium on Graph Drawing and Network Visualization, held on September 18 20, 2020 online. Graph drawing is concerned with the geometric representation of graphs and constitutes the algorithmic core of network visualization. Graph drawing and network visualization are motivated by applications where it is crucial to visually analyse and interact with relational datasets. Information about the conference series and past symposia is maintained at http www.graphdrawing.org. The 2020 edition of the conference was hosted by University Of British Columbia, with Will Evans as chair of the Organizing Committee. A total of 251 participants attended the conference.


CARMI A Cache Aware Learned Index with a Cost based Construction Algorithm. Learned indexes, which use machine learning models to replace traditional index structures, have shown promising results in recent studies. Existing learned indexes use heuristic rules to construct index structures, which are often suboptimal and sensitive to data distribution. In this paper, we argue that upper level RMI nodes should focus on data partitioning instead of model fitting, and show that it leads to much better results in real world datasets. We introduce entropy as a metric to quantify and characterize the models in learned indexes, which provides a new theoretical basis for subsequent works. Moreover, we propose a new memory layout design with a fixed node size throughout the tree structure, which allows the type of each node to be flexibly chosen at runtime. We propose CARMI, a new efficient and updatable cache aware RMI framework. To reduce reliance on the expertise of database administrators, CARMI uses a hybrid construction algorithm to automatically construct the index structures under various datasets and workloads without any manual tuning. Our experimental study shows that CARMI performs better and is more robust compared to baselines, achieving an average of 2.37x 1.98x speedup compared to B Tree ALEX, while using only about 0.70x memory space of B Tree. In the SOSD platform, CARMI outperforms all the baselines over the real world datasets, with an average speedup of 1.21x over the nearest competitor. We believe that our theoretical analysis and proposed framework can help learned indexes to get closer to practical deployment.


Appearance Editing with Free viewpoint Neural Rendering. We present a neural rendering framework for simultaneous view synthesis and appearance editing of a scene from multi view images captured under known environment illumination. Existing approaches either achieve view synthesis alone or view synthesis along with relighting, without direct control over the scene's appearance. Our approach explicitly disentangles the appearance and learns a lighting representation that is independent of it. Specifically, we independently estimate the BRDF and use it to learn a lighting only representation of the scene. Such disentanglement allows our approach to generalize to arbitrary changes in appearance while performing view synthesis. We show results of editing the appearance of a real scene, demonstrating that our approach produces plausible appearance editing. The performance of our view synthesis approach is demonstrated to be at par with state of the art approaches on both real and synthetic data.


Design of Robust and Efficient Edge Server Placement and Server Scheduling Policies Extended Version. We study how to design edge server placement and server scheduling policies under workload uncertainty for 5G networks. We introduce a new metric called resource pooling factor to handle unexpected workload bursts. Maximizing this metric offers a strong enhancement on top of robust optimization against workload uncertainty. Using both real traces and synthetic traces, we show that the proposed server placement and server scheduling policies not only demonstrate better robustness against workload uncertainty than existing approaches, but also significantly reduce the cost of service providers. Specifically, in order to achieve close to zero workload rejection rate, the proposed server placement policy reduces the number of required edge servers by about 25 compared with the state of the art approach the proposed server scheduling policy reduces the energy consumption of edge servers by about 13 without causing much impact on the service quality.


Mixture Outlier Exposure Towards Out of Distribution Detection in Fine grained Environments. Many real world scenarios in which DNN based recognition systems are deployed have inherently fine grained attributes (e.g., bird species recognition, medical image classification). In addition to achieving reliable accuracy, a critical subtask for these models is to detect Out of distribution (OOD) inputs. Given the nature of the deployment environment, one may expect such OOD inputs to also be fine grained w.r.t. the known classes (e.g., a novel bird species), which are thus extremely difficult to identify. Unfortunately, OOD detection in fine grained scenarios remains largely underexplored. In this work, we aim to fill this gap by first carefully constructing four large scale fine grained test environments, in which existing methods are shown to have difficulties. Particularly, we find that even explicitly incorporating a diverse set of auxiliary outlier data during training does not provide sufficient coverage over the broad region where fine grained OOD samples locate. We then propose Mixture Outlier Exposure (MixOE), which mixes ID data and training outliers to expand the coverage of different OOD granularities, and trains the model such that the prediction confidence linearly decays as the input transitions from ID to OOD. Extensive experiments and analyses demonstrate the effectiveness of MixOE for building up OOD detector in fine grained environments. Code is available at https github.com zjysteven MixOE.


Robotic Waste Sorter with Agile Manipulation and Quickly Trainable Detector. Owing to human labor shortages, the automation of labor intensive manual waste sorting is needed. The goal of automating waste sorting is to replace the human role of robust detection and agile manipulation of waste items with robots. To achieve this, we propose three methods. First, we provide a combined manipulation method using graspless push and drop and pick and release manipulation. Second, we provide a robotic system that can automatically collect object images to quickly train a deep neural network model. Third, we provide a method to mitigate the differences in the appearance of target objects from two scenes one for dataset collection and the other for waste sorting in a recycling factory. If differences exist, the performance of a trained waste detector may decrease. We address differences in illumination and background by applying object scaling, histogram matching with histogram equalization, and background synthesis to the source target object images. Via experiments in an indoor experimental workplace for waste sorting, we confirm that the proposed methods enable quick collection of the training image sets for three classes of waste items (i.e., aluminum can, glass bottle, and plastic bottle) and detection with higher performance than the methods that do not consider the differences. We also confirm that the proposed method enables the robot quickly manipulate the objects.


RailLoMer Rail Vehicle Localization and Mapping with LiDAR IMU Odometer GNSS Data Fusion. We present RailLoMer in this article, to achieve real time accurate and robust odometry and mapping for rail vehicles. RailLoMer receives measurements from two LiDARs, an IMU, train odometer, and a global navigation satellite system (GNSS) receiver. As frontend, the estimated motion from IMU odometer preintegration de skews the denoised point clouds and produces initial guess for frame to frame LiDAR odometry. As backend, a sliding window based factor graph is formulated to jointly optimize multi modal information. In addition, we leverage the plane constraints from extracted rail tracks and the structure appearance descriptor to further improve the system robustness against repetitive structures. To ensure a globally consistent and less blurry mapping result, we develop a two stage mapping method that first performs scan to map in local scale, then utilizes the GNSS information to register the submaps. The proposed method is extensively evaluated on datasets gathered for a long time range over numerous scales and scenarios, and show that RailLoMer delivers decimeter grade localization accuracy even in large or degenerated environments. We also integrate RailLoMer into an interactive train state and railway monitoring system prototype design, which has already been deployed to an experimental freight traffic railroad.


How Does Knowledge Graph Embedding Extrapolate to Unseen Data a Semantic Evidence View. Knowledge Graph Embedding (KGE) aims to learn representations for entities and relations. Most KGE models have gained great success, especially on extrapolation scenarios. Specifically, given an unseen triple (h, r, t), a trained model can still correctly predict t from (h, r, ?), or h from (?, r, t), such extrapolation ability is impressive. However, most existing KGE works focus on the design of delicate triple modeling function, which mainly tells us how to measure the plausibility of observed triples, but offers limited explanation of why the methods can extrapolate to unseen data, and what are the important factors to help KGE extrapolate. Therefore in this work, we attempt to study the KGE extrapolation of two problems 1. How does KGE extrapolate to unseen data? 2. How to design the KGE model with better extrapolation ability? For the problem 1, we first discuss the impact factors for extrapolation and from relation, entity and triple level respectively, propose three Semantic Evidences (SEs), which can be observed from train set and provide important semantic information for extrapolation. Then we verify the effectiveness of SEs through extensive experiments on several typical KGE methods. For the problem 2, to make better use of the three levels of SE, we propose a novel GNN based KGE model, called Semantic Evidence aware Graph Neural Network (SE GNN). In SE GNN, each level of SE is modeled explicitly by the corresponding neighbor pattern, and merged sufficiently by the multi layer aggregation, which contributes to obtaining more extrapolative knowledge representation. Finally, through extensive experiments on FB15k 237 and WN18RR datasets, we show that SE GNN achieves state of the art performance on Knowledge Graph Completion task and performs a better extrapolation ability.


Obscuring digital route choice information prevents delay induced congestion. Although routing applications increasingly affect individual mobility choices, their impact on collective traffic dynamics remains largely unknown. Smart communication technologies provide accurate traffic data for choosing one route over other alternatives, yet inherent delays undermine the potential usefulness of such information. Here we introduce and analyze a simple model of collective traffic dynamics which result from route choice relying on outdated traffic information. We find for sufficiently small information delays that traffic flows are stable against perturbations. However, delays beyond a bifurcation point induce self organized flow oscillations of increasing amplitude congestion arises. Providing delayed information averaged over sufficiently long periods of time or, more intriguingly, reducing the number of vehicles adhering to the route recommendations may prevent such delay induced congestion. We reveal the fundamental mechanisms underlying these phenomena in a minimal two road model and demonstrate their generality in microscopic, agent based simulations of a road network system. Our findings provide a way to conceptually understand system wide traffic dynamics caused by broadly used non instantaneous routing information and suggest how resulting unintended collective traffic states could be avoided.


Multiple Borel Cantelli Lemma in dynamics and MultiLog law for recurrence. A classical Borel Cantelli Lemma gives conditions for deciding whether an infinite number of rare events will almost surely happen. In this article, we propose an extension of Borel Cantelli Lemma to characterize the multiple occurrence of events on the same time scale. Our results imply multiple Logarithm Laws for recurrence and hitting times, as well as Poisson Limit Laws for systems which are exponentially mixing of all orders. The applications include geodesic flows on compact negatively curved manifolds, geodesic excursions, Diophantine approximations and extreme value theory for dynamical systems.


Multi Level Fine Tuning Closing Generalization Gaps in Approximation of Solution Maps under a Limited Budget for Training Data. In scientific machine learning, regression networks have been recently applied to approximate solution maps (e.g., potential ground state map of Schr "odinger equation). In this paper, we aim to reduce the generalization error without spending more time in generating training samples. However, to reduce the generalization error, the regression network needs to be fit on a large number of training samples (e.g., a collection of potential ground state pairs). The training samples can be produced by running numerical solvers, which takes much time in many applications. In this paper, we aim to reduce the generalization error without spending more time in generating training samples. Inspired by few shot learning techniques, we develop the Multi Level Fine Tuning algorithm by introducing levels of training we first train the regression network on samples generated at the coarsest grid and then successively fine tune the network on samples generated at finer grids. Within the same amount of time, numerical solvers generate more samples on coarse grids than on fine grids. We demonstrate a significant reduction of generalization error in numerical experiments on challenging problems with oscillations, discontinuities, or rough coefficients. Further analysis can be conducted in the Neural Tangent Kernel regime and we provide practical estimators to the generalization error. The number of training samples at different levels can be optimized for the smallest estimated generalization error under the constraint of budget for training data. The optimized distribution of budget over levels provides practical guidance with theoretical insight as in the celebrated Multi Level Monte Carlo algorithm.


SEDAT Security Enhanced Device Attestation with TPM2.0. Remote attestation is one of the ways to verify the state of an untrusted device. Earlier research has attempted remote verification of a devices' state using hardware, software, or hybrid approaches. Majority of them have used Attestation Key as a hardware root of trust, which does not detect hardware modification or counterfeit issues. In addition, they do not have a secure communication channel between verifier and prover, which makes them susceptible to modern security attacks. This paper presents SEDAT, a novel methodology for remote attestation of the device via a security enhanced communication channel. SEDAT performs hardware, firmware, and software attestation. SEDAT enhances the communication protocol security between verifier and prover by using the Single Packet Authorization (SPA) technique, which provides replay and Denial of Service (DoS) protection. SEDAT provides a way for verifier to get on demand device integrity and authenticity status via a secure channel. It also enables the verifier to detect counterfeit hardware, change in firmware, and software code on the device. SEDAT validates the manufacturers root CA certificate, platform certificate, endorsement certificate (EK), and attributes certificates to perform platform hardware attestation. SEDAT is the first known tool that represents firmware, and Integrity Measurement Authority (IMA) event logs in the Canonical Event Logs (CEL) format (recommended by Trusted Computing Group). SEDAT is the first implementation, to the best of our knowledge, that showcases end to end hardware, firmware, and software remote attestation using Trusted Platform Module (TPM2.0) which is resilient to DoS and replay attacks. SEDAT is the first remote verifier that is capable of retrieving a TPM2.0 quote from prover and validate it after regeneration, using a software TPM2.0 quote check.


Fight Detection from Still Images in the Wild. Detecting fights from still images shared on social media is an important task required to limit the distribution of violent scenes in order to prevent their negative effects. For this reason, in this study, we address the problem of fight detection from still images collected from the web and social media. We explore how well one can detect fights from just a single still image. We also propose a new dataset, named Social Media Fight Images (SMFI), comprising real world images of fight actions. Results of the extensive experiments on the proposed dataset show that fight actions can be recognized successfully from still images. That is, even without exploiting the temporal information, it is possible to detect fights with high accuracy by utilizing appearance only. We also perform cross dataset experiments to evaluate the representation capacity of the collected dataset. These experiments indicate that, as in the other computer vision problems, there exists a dataset bias for the fight recognition problem. Although the methods achieve close to 100 accuracy when trained and tested on the same fight dataset, the cross dataset accuracies are significantly lower, i.e., around 70 when more representative datasets are used for training. SMFI dataset is found to be one of the two most representative datasets among the utilized five fight datasets.


Effects of Plasticity Functions on Neural Assemblies. We explore the effects of various plasticity functions on assemblies of neurons. To bridge the gap between experimental and computational theories we make use of a conceptual framework, the Assembly Calculus, which is a formal system for the description of brain function based on assemblies of neurons. The Assembly Calculus includes operations for projecting, associating, and merging assemblies of neurons. Our research is focused on simulating different plasticity functions with Assembly Calculus. Our main contribution is the modification and evaluation of the projection operation. We experiment with Oja's and Spike Time Dependent Plasticity (STDP) rules and test the effect of various hyper parameters.


A New Approach for Solving the Linear Complementarity Problem using Smoothing Functions. Based on smoothing techniques, we propose two new methods to solve linear complementarity problems (LCP) called TLCP and Soft Max. The idea of these two new methods takes inspiration from interior point methods in optimization. The technique that we propose avoids any parameter management while ensuring good theoretical convergence results. In our approach we do not need any complicated strategy to update the smoothing parameter r since we will consider it as a new variable. Our methods are validated by extensive numerical tests, in which we compare our methods to several other classical methods.


On the hollow enclosed by convex sets. For n leq d , a family cal F C 0,C 1, ldots, C n of compact convex sets in R d is called an n critical family provided any n members of cal F have a non empty intersection, but bigcap i 0 n C i varnothing . If n d then a lemma on the intersection of convex sets due to Klee implies that the d 1 members of the d critical family enclose a hollow' in R d , a bounded connected component of R d setminus bigcup i 0 n C i. Here we prove that the closure of the convex hull of a hollow in R d is a d simplex.


Spectral theory, the holomorphic functional calculus, and frames. We may need for some applications to construct one or more frames. To do so, we need to think of some family of vectors that satisfies the frame condition. In the present article, we will discuss a spectral criterion allowing us to check this condition for sequences constructed from an orthonormal basis, a bounded operator, and a holomorphic function.


Modeling Ground to Air Path Loss for Millimeter Wave UAV Networks. Path loss is a significant component of wireless communication channel design and analysis and reflects the reduction in a transmitted signal's power density. Due to the differences in the propagation conditions, wireless aerial channels' features differ from those of terrestrial wireless channels therefore, unmanned aerial vehicle path loss models are often different from conventional terrestrial wireless channel path loss models. A mathematical propagation model is proposed in this paper to estimate the Ground to Air path loss between a wireless device and a low altitude platform using the frequency bands of the millimeter wave. The suggested model of Ground to Air path loss will assist academic researchers in formulating several vital problems.


Incorporating Connections Beyond Knowledge Embeddings A Plug and Play Module to Enhance Commonsense Reasoning in Machine Reading Comprehension. Conventional Machine Reading Comprehension (MRC) has been well addressed by pattern matching, but the ability of commonsense reasoning remains a gap between humans and machines. Previous methods tackle this problem by enriching word representations via pre trained Knowledge Graph Embeddings (KGE). However, they make limited use of a large number of connections between nodes in Knowledge Graphs (KG), which could be pivotal cues to build the commonsense reasoning chains. In this paper, we propose a Plug and play module to IncorporatE Connection information for commonsEnse Reasoning (PIECER). Beyond enriching word representations with knowledge embeddings, PIECER constructs a joint query passage graph to explicitly guide commonsense reasoning by the knowledge oriented connections between words. Further, PIECER has high generalizability since it can be plugged into suitable positions in any MRC model. Experimental results on ReCoRD, a large scale public MRC dataset requiring commonsense reasoning, show that PIECER introduces stable performance improvements for four representative base MRC models, especially in low resource settings.


What We Measure in Mixed Reality Experiments. There are many potential measures that one might use when evaluating mixed reality experiences. In this position paper I will argue that there are various stances to take for evaluation, depending on the framing of the experience within a larger body of work. I will draw upon various types of work that my team has been involved with in order to illustrate these different stances. I will then sketch out some directions for developing more robust measures that can help the field move forward.


Iterative Hierarchical Attention for Answering Complex Questions over Long Documents. We propose a new model, DocHopper, that iteratively attends to different parts of long, hierarchically structured documents to answer complex questions. Similar to multi hop question answering (QA) systems, at each step, DocHopper uses a query q to attend to information from a document, combines this retrieved'' information with q to produce the next query. However, in contrast to most previous multi hop QA systems, DocHopper is able to retrieve'' either short passages or long sections of the document, thus emulating a multi step process of navigating'' through a long document to answer a question. To enable this novel behavior, DocHopper does not combine document information with q by concatenating text to the text of q , but by combining a compact neural representation of q with a compact neural representation of a hierarchical part of the document, which can potentially be quite large. We experiment with DocHopper on four different QA tasks that require reading long and complex documents to answer multi hop questions, and show that DocHopper achieves state of the art results on three of the datasets. Additionally, DocHopper is efficient at inference time, being 3 10 times faster than the baselines.


Some weighted Hardy type inequalities and applications. We study the two weighted estimate bigg sum k 0 na k(x) int 0 xt kf(t)dt L q,v (0, infty) bigg leq c f L p,u (0, infty) , tag where the functions a k(x) are not assumed to be positive. It is shown that for 1 p leq q leq infty , provided that the weight u satisfies the certain conditions, the estimate ( ) holds if and only if the estimate sum k 0 n bigg a k(x) int 0 xt kf(t)dt L q,v (0, infty) bigg leq c f L p,u (0, infty) . tag is fulfilled. The necessary and sufficient conditions for ( ) to be valid are well known. The obtained result can be applied to the estimates of differential operators with variable coefficients in some weighted Sobolev spaces.


Learning 6DoF Grasping Using Reward Consistent Demonstration. As the number of the robot's degrees of freedom increases, the implementation of robot motion becomes more complex and difficult. In this study, we focus on learning 6DOF grasping motion and consider dividing the grasping motion into multiple tasks. We propose to combine imitation and reinforcement learning in order to facilitate a more efficient learning of the desired motion. In order to collect demonstration data as teacher data for the imitation learning, we created a virtual reality (VR) interface that allows humans to operate the robot intuitively. Moreover, by dividing the motion into simpler tasks, we simplify the design of reward functions for reinforcement learning and show in our experiments a reduction in the steps required to learn the grasping motion.


Large Scale Differentially Private BERT. In this work, we study the large scale pretraining of BERT Large with differentially private SGD (DP SGD). We show that combined with a careful implementation, scaling up the batch size to millions (i.e., mega batches) improves the utility of the DP SGD step for BERT we also enhance its efficiency by using an increasing batch size schedule. Our implementation builds on the recent work of SVK20 , who demonstrated that the overhead of a DP SGD step is minimized with effective use of JAX BFH 18, FJL18 primitives in conjunction with the XLA compiler XLA17 . Our implementation achieves a masked language model accuracy of 60.5 at a batch size of 2M, for epsilon 5.36 . To put this number in perspective, non private BERT models achieve an accuracy of sim 70 .


Benchmark of visual and 3D lidar SLAM systems in simulation environment for vineyards. In this work, we present a comparative analysis of the trajectories estimated from various Simultaneous Localization and Mapping (SLAM) systems in a simulation environment for vineyards. Vineyard environment is challenging for SLAM methods, due to visual appearance changes over time, uneven terrain, and repeated visual patterns. For this reason, we created a simulation environment specifically for vineyards to help studying SLAM systems in such a challenging environment. We evaluated the following SLAM systems LIO SAM, StaticMapping, ORB SLAM2, and RTAB MAP in four different scenarios. The mobile robot used in this study equipped with 2D and 3D lidars, IMU, and RGB D camera (Kinect v2). The results show good and encouraging performance of RTAB MAP in such an environment.


DropAttack A Masked Weight Adversarial Training Method to Improve Generalization of Neural Networks. Adversarial training has been proven to be a powerful regularization method to improve the generalization of models. However, current adversarial training methods only attack the original input sample or the embedding vectors, and their attacks lack coverage and diversity. To further enhance the breadth and depth of attack, we propose a novel masked weight adversarial training method called DropAttack, which enhances generalization of model by adding intentionally worst case adversarial perturbations to both the input and hidden layers in different dimensions and minimize the adversarial risks generated by each layer. DropAttack is a general technique and can be adopt to a wide variety of neural networks with different architectures. To validate the effectiveness of the proposed method, we used five public datasets in the fields of natural language processing (NLP) and computer vision (CV) for experimental evaluating. We compare the proposed method with other adversarial training methods and regularization methods, and our method achieves state of the art on all datasets. In addition, Dropattack can achieve the same performance when it use only a half training data compared to other standard training method. Theoretical analysis reveals that DropAttack can perform gradient regularization at random on some of the input and wight parameters of the model. Further visualization experiments show that DropAttack can push the minimum risk of the model to a lower and flatter loss landscapes. Our source code is publicly available on https github.com nishiwen1214 DropAttack.


Global Complexity Bound of a Proximal ADMM for Linearly Constrained Nonseperable Nonconvex Composite Programming. This paper proposes and analyzes a dampened proximal alternating direction method of multipliers (DP.ADMM) for solving linearly constrained nonconvex optimization problems where the smooth part of the objective function is nonseparable. Each iteration of DP.ADMM consists of (i) a sequence of partial proximal augmented Lagrangian (AL) updates, (ii) an under relaxed Lagrange multiplier update, and (iii) a novel test to check whether the penalty parameter of the AL function should be updated. Under a basic Slater condition and some requirements related to the dampening factor and under relaxation parameter, it is shown that DP.ADMM obtains a first order stationary point of the constrained problem in cal O ( varepsilon 3 ) iterations for a given numerical tolerance varepsilon 0 . One of the main novelties of the paper is that convergence of the method is obtained without requiring any rank assumptions on the constraint matrices.


The Number of Optimal Strategies in the Penney Ante Game. In the Penney Ante game, Player I chooses a head tail string of a predetermined length n ge3 . Player II, upon seeing Player I's choice, chooses another head tail string of the same length. A coin is then tossed repeatedly and the player whose string appears first in the resulting head tail sequence wins the game. The Penney Ante game has gained notoriety as a source of counterintuitive probabilities and nontransitivity phenomena. For example, Player II can always choose a string that beats the choice of Player I in the sense of being more likely to appear first in a random head tail sequence. It is known that Player II has a unique optimal strategy that maximizes her winning chances in this game. On the other hand, for Player I there exist multiple equivalent optimal strategies. In this paper we investigate the number, c n , of optimal strategies for Player I, i.e., the number of head tail strings of length n that maximize the winning probability for Player I assuming optimal play by Player II. We derive a recurrence relation for c n and use this to obtain a sharp asymptotic estimate for c n . In particular, we show that, as n to infty , a fixed proportion alpha approx 0.04062 dots of the 2 n head tail strings of length n are optimal from Player I's perspective.


Ground SLAM Ground Constrained LiDAR SLAM for Structured Multi Floor Environments. This paper proposes a 3D LiDAR SLAM algorithm named Ground SLAM, which exploits grounds in structured multi floor environments to compress the pose drift mainly caused by LiDAR measurement bias. Ground SLAM is developed based on the well known pose graph optimization framework. In the front end, motion estimation is conducted using LiDAR Odometry (LO) with a novel sensor centric sliding map introduced, which is maintained by filtering out expired features based on the model of error propagation. At each key frame, the sliding map is recorded as a local map. The ground nearby is extracted and modelled as an infinite planar landmark in the form of Closest Point (CP) parameterization. Then, ground planes observed at different key frames are associated, and the ground constraints are fused into the pose graph optimization framework to compress the pose drift of LO. Finally, loop closure detection is carried out, and the residual error is jointly minimized, which could lead to a globally consistent map. Experimental results demonstrate superior performances in the accuracy of the proposed approach.


Interplay between physics self efficacy, calculus transfer ability, and gender. We present our initial work to develop an assessment of calculus proficiency in the context of introductory physics (i.e., calculus transfer to physics), including a comparison of calculus transfer ability with physics self efficacy and how these attributes intersect with student gender. Although students demonstrated an improvement in their calculus transfer ability, most nevertheless displayed a decrease in their physics self efficacy. Similarly, while women consistently exhibited lower physics self efficacy than men, women demonstrated larger gains in their proficiency with calculus transfer to physics. We also discuss future work to better evaluate the interdependence of mathematics self efficacy, physics self efficacy, and ability to transfer calculus to physics.


Towards Automatic Comparison of Data Privacy Documents A Preliminary Experiment on GDPR like Laws. General Data Protection Regulation (GDPR) becomes a standard law for data protection in many countries. Currently, twelve countries adopt the regulation and establish their GDPR like regulation. However, to evaluate the differences and similarities of these GDPR like regulations is time consuming and needs a lot of manual effort from legal experts. Moreover, GDPR like regulations from different countries are written in their languages leading to a more difficult task since legal experts who know both languages are essential. In this paper, we investigate a simple natural language processing (NLP) approach to tackle the problem. We first extract chunks of information from GDPR like documents and form structured data from natural language. Next, we use NLP methods to compare documents to measure their similarity. Finally, we manually label a small set of data to evaluate our approach. The empirical result shows that the BERT model with cosine similarity outperforms other baselines. Our data and code are publicly available.


CausalX Causal Explanations and Block Multilinear Factor Analysis. By adhering to the dictum, "No causation without manipulation (treatment, intervention)", cause and effect data analysis represents changes in observed data in terms of changes in the causal factors. When causal factors are not amenable for active manipulation in the real world due to current technological limitations or ethical considerations, a counterfactual approach performs an intervention on the model of data formation. In the case of object representation or activity (temporal object) representation, varying object parts is generally unfeasible whether they be spatial and or temporal. Multilinear algebra, the algebra of higher order tensors, is a suitable and transparent framework for disentangling the causal factors of data formation. Learning a part based intrinsic causal factor representations in a multilinear framework requires applying a set of interventions on a part based multilinear model. We propose a unified multilinear model of wholes and parts. We derive a hierarchical block multilinear factorization, the M mode Block SVD, that computes a disentangled representation of the causal factors by optimizing simultaneously across the entire object hierarchy. Given computational efficiency considerations, we introduce an incremental bottom up computational alternative, the Incremental M mode Block SVD, that employs the lower level abstractions, the part representations, to represent the higher level of abstractions, the parent wholes. This incremental computational approach may also be employed to update the causal model parameters when data becomes available incrementally. The resulting object representation is an interpretable combinatorial choice of intrinsic causal factor representations related to an object's recursive hierarchy of wholes and parts that renders object recognition robust to occlusion and reduces training data requirements.


Wavelet Code based postquantum signatures with fast verification on microcontrollers. This work presents the first full implementation of Wave, a postquantum code based signature scheme. We define Wavelet, a concrete Wave scheme at the 128 bit classical security level (or NIST postquantum security Level 1) equipped with a fast verification algorithm targeting embedded devices. Wavelet offers 930 byte signatures, with a public key of 3161 kB. We include implementation details using AVX instructions, and on ARM Cortex M4, including a solution to deal with Wavelet's large public keys, which do not fit in the SRAM of a typical embedded device. Our verification algorithm is approx 4.65 times faster then the original, and verifies in 1 087 538 cycles using AVX instructions, or 13 172 ticks in an ARM Cortex M4.


Reasoning Chain Based Adversarial Attack for Multi hop Question Answering. Recent years have witnessed impressive advances in challenging multi hop QA tasks. However, these QA models may fail when faced with some disturbance in the input text and their interpretability for conducting multi hop reasoning remains uncertain. Previous adversarial attack works usually edit the whole question sentence, which has limited effect on testing the entity based multi hop inference ability. In this paper, we propose a multi hop reasoning chain based adversarial attack method. We formulate the multi hop reasoning chains starting from the query entity to the answer entity in the constructed graph, which allows us to align the question to each reasoning hop and thus attack any hop. We categorize the questions into different reasoning types and adversarially modify part of the question corresponding to the selected reasoning hop to generate the distracting sentence. We test our adversarial scheme on three QA models on HotpotQA dataset. The results demonstrate significant performance reduction on both answer and supporting facts prediction, verifying the effectiveness of our reasoning chain based attack method for multi hop reasoning models and the vulnerability of them. Our adversarial re training further improves the performance and robustness of these models.


Multi stage Stochastic Alternating Current Optimal Power Flow with Storage Bounding the Relaxation Gap. We propose a generic multistage stochastic model for the Alternating Current Optimal Power Flow (AC OPF) problem for radial distribution networks, to account for the random electricity production of renewable energy sources and dynamic constraints of storage systems. We consider single phase radial networks. Radial three phase balanced networks (medium voltage distribution networks typically have this structure) reduce to the former case. This induces a large scale optimization problem, which, given the non convex nature of the AC OPF, is generally challenging to solve to global optimality. We derive a priori conditions guaranteeing a vanishing relaxation gap for the multi stage AC OPF problem, which can thus be solved using convex optimization algorithms. We also give an a posteriori upper bound on the relaxation gap. In particular, we show that a null or low relaxation gap may be expected for applications with light reverse power flows or if sufficient storage capacities with low cost are available. Then, we discuss the validity of our results when incorporating voltage regulation devices. Finally, we illustrate our results on problems of planning of a realistic distribution feeder with distributed solar production and storage systems. Scenario trees for solar production are constructed from a stochastic model, by a quantile based algorithm.


Feedback Vertex Set on Hamiltonian Graphs. We study the computational complexity of Feedback Vertex Set on subclasses of Hamiltonian graphs. In particular, we consider Hamiltonian graphs that are regular or are planar and regular. Moreover, we study the less known class of p Hamiltonian ordered graphs, which are graphs that admit for any p tuple of vertices a Hamiltonian cycle visiting them in the order given by the tuple. We prove that Feedback Vertex Set remains NP hard in these restricted cases, even if a Hamiltonian cycle is additionally given as part of the input.


The Impacts of Low No Code Development on Digital Transformation and Software Development. Low No code development is a software development method that provides users with a platform for visually creating applications with little or no coding. Companies and organizations need software applications and information systems for various business purposes like management in the technology era. Low No code development gives non IT professionals a convenient tool for rapidly building simple business applications they need without or with little coding. In this paper, we explored the benefits limitations of Low No Code development and modern Low No Code development platforms in the industry. In addition, we analyzed how it can be improved and prospected the impacts of Low No Code development on society and related industries in the future. In conclusion, we find that Low No code development is a promising trend that can significantly impact future software development and digital transformation.


LiDARTouch Monocular metric depth estimation with a few beam LiDAR. Vision based depth estimation is a key feature in autonomous systems, which often relies on a single camera or several independent ones. In such a monocular setup, dense depth is obtained with either additional input from one or several expensive LiDARs, e.g., with 64 beams, or camera only methods, which suffer from scale ambiguity and infinite depth problems. In this paper, we propose a new alternative of densely estimating metric depth by combining a monocular camera with a light weight LiDAR, e.g., with 4 beams, typical of today's automotive grade mass produced laser scanners. Inspired by recent self supervised methods, we introduce a novel framework, called LiDARTouch, to estimate dense depth maps from monocular images with the help of touches'' of LiDAR, i.e., without the need for dense ground truth depth. In our setup, the minimal LiDAR input contributes on three different levels as an additional model's input, in a self supervised LiDAR reconstruction objective function, and to estimate changes of pose (a key component of self supervised depth estimation architectures). Our LiDARTouch framework achieves new state of the art in self supervised depth estimation on the KITTI dataset, thus supporting our choices of integrating the very sparse LiDAR signal with other visual features. Moreover, we show that the use of a few beam LiDAR alleviates scale ambiguity and infinite depth issues that camera only methods suffer from. We also demonstrate that methods from the fully supervised depth completion literature can be adapted to a self supervised regime with a minimal LiDAR signal.


Some products in fusion systems and localities. The theory of saturated fusion systems resembles in many parts the theory of finite groups. However, some concepts from finite group theory are difficult to translate to fusion systems. For example, products of normal subsystems with other subsystems are only defined in special cases. In this paper the theory of localities is used to prove the following result Suppose mathcal F is a saturated fusion system over a p group S . If mathcal E is a normal subsystem of mathcal F over T leq S , and mathcal D is a normal subsystem of N mathcal F (T) over R leq S , then there is a normal subsystem mathcal E mathcal D of mathcal F over TR , which plays the role of a product of mathcal E and mathcal D in mathcal F . It is shown along the way that the subsystem mathcal E mathcal D is closely related to a naturally arising product in certain localities attached to mathcal F .


Isotopy equivalence of analytic branches in ( mathbb C n,0) . We prove that two analytic branches in ( mathbb C n,0) whose dual resolution graph is the same admit an ambient isotopy which is smooth outside the origin. A weaker version of the converse is also proved.


Vectorial Parameterizations of Pose. Robotics and computer vision problems commonly require handling rigid body motions comprising translation and rotation together referred to as pose. In some situations, a vectorial parameterization of pose can be useful, where elements of a vector space are surjectively mapped to a matrix Lie group. For example, these vectorial representations can be employed for optimization as well as uncertainty representation on groups. The most common mapping is the matrix exponential, which maps elements of a Lie algebra onto the associated Lie group. However, this choice is not unique. It has been previously shown how to characterize all such vectorial parameterizations for SO(3), the group of rotations. Some results are also known for the group of poses, where it is possible to build a family of vectorial mappings that includes the matrix exponential as well as the Cayley transformation. We extend what is known for these pose mappings to the 4 x 4 representation common in robotics, and also demonstrate three different examples of the proposed pose mappings (i) pose interpolation, (ii) pose servoing control, and (iii) pose estimation in a pointcloud alignment problem. In the pointcloud alignment problem our results lead to a new algorithm based on the Cayley transformation, which we call CayPer.


YMIR A Rapid Data centric Development Platform for Vision Applications. This paper introduces an open source platform to support the rapid development of computer vision applications at scale. The platform puts the efficient data development at the center of the machine learning development process, integrates active learning methods, data and model version control, and uses concepts such as projects to enable fast iterations of multiple task specific datasets in parallel. This platform abstracts the development process into core states and operations, and integrates third party tools via open APIs as implementations of the operations. This open design reduces the development cost and adoption cost for ML teams with existing tools. At the same time, the platform supports recording project development histories, through which successful projects can be shared to further boost model production efficiency on similar tasks. The platform is open source and is already used internally to meet the increasing demand for different real world computer vision applications.


Yet another Proof of an old Hat. Every odd prime number p can be written in exactly (p 1) 2 ways as a sum ab cd with min(a, b) max(c, d) of two ordered products. This gives a new proof Fermat's Theorem expressing primes of the form 1 4N as sums of two squares 1 .


Bernstein and Markov type inequalities. This survey discusses the classical Bernstein and Markov inequalities for the derivatives of polynomials, as well as some of their extensions to general sets.


DomiKnowS A Library for Integration of Symbolic Domain Knowledge in Deep Learning. We demonstrate a library for the integration of domain knowledge in deep learning architectures. Using this library, the structure of the data is expressed symbolically via graph declarations and the logical constraints over outputs or latent variables can be seamlessly added to the deep models. The domain knowledge can be defined explicitly, which improves the models' explainability in addition to the performance and generalizability in the low data regime. Several approaches for such an integration of symbolic and sub symbolic models have been introduced however, there is no library to facilitate the programming for such an integration in a generic way while various underlying algorithms can be used. Our library aims to simplify programming for such an integration in both training and inference phases while separating the knowledge representation from learning algorithms. We showcase various NLP benchmark tasks and beyond. The framework is publicly available at Github(https github.com HLR DomiKnowS).


Embedded Hardware Appropriate Fast 3D Trajectory Optimization for Fixed Wing Aerial Vehicles by Leveraging Hidden Convex Structures. Most commercially available fixed wing aerial vehicles (FWV) can carry only small, lightweight computing hardware such as Jetson TX2 onboard. Solving non linear trajectory optimization on these computing resources is computationally challenging even while considering only the kinematic motion model. Most importantly, the computation time increases sharply as the environment becomes more cluttered. In this paper, we take a step towards overcoming this bottleneck and propose a trajectory optimizer that achieves online performance on both conventional laptops desktops and Jetson TX2 in a typical urban environment setting. Our optimizer builds on the novel insight that the seemingly non linear trajectory optimization problem for FWV has an implicit multi convex structure. Our optimizer exploits these computational structures by bringing together diverse concepts from Alternating Minimization, Bregman iteration, and Alternating Direction Method of Multipliers. We show that our optimizer outperforms the state of the art implementation of sequential quadratic programming approach in optimal control solver ACADO in computation time and solution quality measured in terms of control and goal reaching cost.


Geometry of canonical genus four curves. We apply Bridgeland stability conditions machinery to describe the geometry of some classical moduli spaces associated with canonical genus four curves in mathbb P 3 via an effective control over its wall crossing. These moduli spaces with several irreducible components include the moduli space of Pandharipande Thomas stable pairs, and the Hilbert scheme, which are not previously studied. We give a full list of irreducible components of the space of stable pairs, along with a birational description of each component, and a partial list for the Hilbert scheme.


PIM DRAM Accelerating Machine Learning Workloads using Processing in Commodity DRAM. Deep Neural Networks (DNNs) have transformed the field of machine learning and are widely deployed in many applications involving image, video, speech and natural language processing. The increasing compute demands of DNNs have been widely addressed through Graphics Processing Units (GPUs) and specialized accelerators. However, as model sizes grow, these von Neumann architectures require very high memory bandwidth to keep the processing elements utilized as a majority of the data resides in the main memory. Processing in memory has been proposed as a promising solution for the memory wall bottleneck for ML workloads. In this work, we propose a new DRAM based processing in memory (PIM) multiplication primitive coupled with intra bank accumulation to accelerate matrix vector operations in ML workloads. The proposed multiplication primitive adds 1 area overhead and does not require any change in the DRAM peripherals. Therefore, the proposed multiplication can be easily adopted in commodity DRAM chips. Subsequently, we design a DRAM based PIM architecture, data mapping scheme and dataflow for executing DNNs within DRAM. System evaluations performed on networks like AlexNet, VGG16 and ResNet18 show that the proposed architecture, mapping, and data flow can provide up to 19.5x speedup over an NVIDIA Titan Xp GPU highlighting the need to overcome the memory bottleneck in future generations of DNN hardware.


A uniform quantitative Manin Mumford theorem for curves over function fields. We prove that any smooth projective geometrically connected non isotrivial curve of genus g ge 2 over a one dimensional function field of any characteristic has at most 16g 2 32g 124 torsion points for any Abel Jacobi embedding of the curve into its Jacobian. The proof uses Zhang's admissible pairing on curves, the arithmetic Hodge index theorem over function fields, and the metrized graph analogue of Elkies' lower bound for the Green function. More generally, we prove an explicit Bogomolov type result bounding the number of geometric points of small N 'eron Tate height on the curve embedded into its Jacobian.


On the Convergence Rate of Off Policy Policy Optimization Methods with Density Ratio Correction. In this paper, we study the convergence properties of off policy policy improvement algorithms with state action density ratio correction under function approximation setting, where the objective function is formulated as a max max min optimization problem. We characterize the bias of the learning objective and present two strategies with finite time convergence guarantees. In our first strategy, we present algorithm P SREDA with convergence rate O( epsilon 3 ) , whose dependency on epsilon is optimal. In our second strategy, we propose a new off policy actor critic style algorithm named O SPIM. We prove that O SPIM converges to a stationary point with total complexity O( epsilon 4 ) , which matches the convergence rate of some recent actor critic algorithms in the on policy setting.


Extreme vorticity events in turbulent Rayleigh B 'enard convection from stereoscopic measurements and reservoir computing. High amplitude events of the out of plane vorticity component omega z are analyzed by stereoscopic particle image velocimetry (PIV) in the bulk region of turbulent Rayleigh B ' e nard convection in air. The Rayleigh numbers rm Ra vary from 1.7 times 10 4 to 5.1 times 10 5 . The experimental investigation is connected with a comprehensive statistical analysis of long term time series of omega z and individual velocity derivatives partial u i partial x j . A statistical convergence for derivative moments up to an order of 6 is demonstrated. Our results are found to agree well with existing high resolution direct numerical simulation data in the same range of parameters, including the extreme vorticity events which appear in the far exponential tails of the corresponding probability density functions. The transition from a Gaussian to a non Gaussian velocity derivative statistics in the bulk of a convection flow is confirmed experimentally. The experimental data are used to train a reservoir computing model, one implementation of a recurrent neural network, to reproduce highly intermittent experimental time series of the vorticity and thus reconstruct extreme out of plane vorticity events. After training the model with high resolution PIV data, the machine learning model is run with sparsely seeded, continually available, and unseen measurement data in the reconstruction phase. The dependence of the reconstruction quality on the sparsity of the partial observations is also documented. Our latter result paves the way to machine learning assisted experimental analyses of small scale turbulence for which time series of missing velocity derivatives can be provided by generative algorithms.


Explainable AI (XAI) A Systematic Meta Survey of Current Challenges and Future Opportunities. The past decade has seen significant progress in artificial intelligence (AI), which has resulted in algorithms being adopted for resolving a variety of problems. However, this success has been met by increasing model complexity and employing black box AI models that lack transparency. In response to this need, Explainable AI (XAI) has been proposed to make AI more transparent and thus advance the adoption of AI in critical domains. Although there are several reviews of XAI topics in the literature that identified challenges and potential research directions in XAI, these challenges and research directions are scattered. This study, hence, presents a systematic meta survey for challenges and future research directions in XAI organized in two themes (1) general challenges and research directions in XAI and (2) challenges and research directions in XAI based on machine learning life cycle's phases design, development, and deployment. We believe that our meta survey contributes to XAI literature by providing a guide for future exploration in the XAI area.


Equivariant Functors and Sheaves. In this thesis we study two main topics which culminate in a proof that four distinct definitions of the equivariant derived category of a smooth algebraic group G acting on a variety X are in fact equivalent. In the first part of this thesis we introduce and study equivariant categories on a quasi projective variety X . These are a generalization of the equivariant derived category of Lusztig and are indexed by certain pseudofunctors that take values in the 2 category of categories. This 2 categorical generalization allow us to prove rigorously and carefully when such categories are additive, monoidal, triangulated, admit t structures, among and more. We also define equivariant functors and natural transformations before using these to prove how to lift adjoints to the equivariant setting. We also give a careful foundation of how to manipulate t structures on these equivariant categories for future use and with an eye towards future applications. In the final part of this thesis we prove a four way equivalence between the different formulations of the equivariant derived category of ell adic sheaves on a quasi projective variety X . We show that the equivariant derived category of Lusztig is equivalent to the equivariant derived category of Bernstein Lunts and the simplicial equivariant derived category. We then show that these equivariant derived categories are equivalent to the derived ell adic category of Behrend on the algebraic stack G backslash X . We also provide an isomorphism of the simplicial equivariant derived category on the variety X with the simplicial equivariant derived category on the simplicial presentation of G backslash X , as well as prove explicit equivalences between the categories of equivariant ell adic sheaves, local systems, and perverse sheaves with the classical incarnations of such categories of equivariant sheaves.


UCNN A Convolutional Strategy on Unstructured Mesh. In machine learning for fluid mechanics, fully connected neural network (FNN) only uses the local features for modelling, while the convolutional neural network (CNN) cannot be applied to data on structured unstructured mesh. In order to overcome the limitations of FNN and CNN, the unstructured convolutional neural network (UCNN) is proposed, which aggregates and effectively exploits the features of neighbour nodes through the weight function. Adjoint vector modelling is taken as the task to study the performance of UCNN. The mapping function from flow field features to adjoint vector is constructed through efficient parallel implementation on GPU. The modelling capability of UCNN is compared with that of FNN on validation set and in aerodynamic shape optimization at test case. The influence of mesh changing on the modelling capability of UCNN is further studied. The results indicate that UCNN is more accurate in modelling process.


Technical Report Disentangled Action Parsing Networks for Accurate Part level Action Parsing. Part level Action Parsing aims at part state parsing for boosting action recognition in videos. Despite of dramatic progresses in the area of video classification research, a severe problem faced by the community is that the detailed understanding of human actions is ignored. Our motivation is that parsing human actions needs to build models that focus on the specific problem. We present a simple yet effective approach, named disentangled action parsing (DAP). Specifically, we divided the part level action parsing into three stages 1) person detection, where a person detector is adopted to detect all persons from videos as well as performs instance level action recognition 2) Part parsing, where a part parsing model is proposed to recognize human parts from detected person images and 3) Action parsing, where a multi modal action parsing network is used to parse action category conditioning on all detection results that are obtained from previous stages. With these three major models applied, our approach of DAP records a global mean of 0.605 score in 2021 Kinetics TPS Challenge.


Real time Optimal Landing Control of the MIT Mini Cheetah. Quadrupedal landing is a complex process involving large impacts, elaborate contact transitions, and is a crucial recovery behavior observed in many biological animals. This work presents a real time, optimal landing controller that is free of pre specified contact schedules. The controller determines optimal touchdown postures and reaction force profiles and is able to recover from a variety of falling configurations. The quadrupedal platform used, the MIT Mini Cheetah, recovered safely from drops of up to 8 m in simulation, as well as from a range of orientations and planar velocities. The controller is also tested on hardware, successfully recovering from drops of up to 2 m.


A Hinge Loss based Codebook Transfer for Cross Domain Recommendation with Nonoverlapping Data. Recommender systems(RS), especially collaborative filtering(CF) based RS, has been playing an important role in many e commerce applications. As the information being searched over the internet is rapidly increasing, users often face the difficulty of finding items of his her own interest and RS often provides help in such tasks. Recent studies show that, as the item space increases, and the number of items rated by the users become very less, issues like sparsity arise. To mitigate the sparsity problem, transfer learning techniques are being used wherein the data from dense domain(source) is considered in order to predict the missing entries in the sparse domain(target). In this paper, we propose a transfer learning approach for cross domain recommendation when both domains have no overlap of users and items. In our approach the transferring of knowledge from source to target domain is done in a novel way. We make use of co clustering technique to obtain the codebook (cluster level rating pattern) of source domain. By making use of hinge loss function we transfer the learnt codebook of the source domain to target. The use of hinge loss as a loss function is novel and has not been tried before in transfer learning. We demonstrate that our technique improves the approximation of the target matrix on benchmark datasets.


Probabilistic Regression with Huber Distributions. In this paper we describe a probabilistic method for estimating the position of an object along with its covariance matrix using neural networks. Our method is designed to be robust to outliers, have bounded gradients with respect to the network outputs, among other desirable properties. To achieve this we introduce a novel probability distribution inspired by the Huber loss. We also introduce a new way to parameterize positive definite matrices to ensure invariance to the choice of orientation for the coordinate system we regress over. We evaluate our method on popular body pose and facial landmark datasets and get performance on par or exceeding the performance of non heatmap methods. Our code is available at github.com Davmo049 Public prob regression with huber distributions


Offline reinforcement learning with uncertainty for treatment strategies in sepsis. Guideline based treatment for sepsis and septic shock is difficult because sepsis is a disparate range of life threatening organ dysfunctions whose pathophysiology is not fully understood. Early intervention in sepsis is crucial for patient outcome, yet those interventions have adverse effects and are frequently overadministered. Greater personalization is necessary, as no single action is suitable for all patients. We present a novel application of reinforcement learning in which we identify optimal recommendations for sepsis treatment from data, estimate their confidence level, and identify treatment options infrequently observed in training data. Rather than a single recommendation, our method can present several treatment options. We examine learned policies and discover that reinforcement learning is biased against aggressive intervention due to the confounding relationship between mortality and level of treatment received. We mitigate this bias using subspace learning, and develop methodology that can yield more accurate learning policies across healthcare applications.


PRECODE A Generic Model Extension to Prevent Deep Gradient Leakage. Collaborative training of neural networks leverages distributed data by exchanging gradient information between different clients. Although training data entirely resides with the clients, recent work shows that training data can be reconstructed from such exchanged gradient information. To enhance privacy, gradient perturbation techniques have been proposed. However, they come at the cost of reduced model performance, increased convergence time, or increased data demand. In this paper, we introduce PRECODE, a PRivacy EnhanCing mODulE that can be used as generic extension for arbitrary model architectures. We propose a simple yet effective realization of PRECODE using variational modeling. The stochastic sampling induced by variational modeling effectively prevents privacy leakage from gradients and in turn preserves privacy of data owners. We evaluate PRECODE using state of the art gradient inversion attacks on two different model architectures trained on three datasets. In contrast to commonly used defense mechanisms, we find that our proposed modification consistently reduces the attack success rate to 0 while having almost no negative impact on model training and final performance. As a result, PRECODE reveals a promising path towards privacy enhancing model extensions.


Asymptotics for the critical level and a strong invariance principle for high intensity shot noise fields. It is known that the global connectivity of the excursion sets of shot noise fields undergoes a percolation phase transition at a certain critical level, which depends on the intensity of the underlying Poisson point process. In this paper we study the asymptotic expansion of the critical level in the high intensity limit. A key input is a strong invariance principle for shot noise fields, i.e. a quantitative coupling between a high intensity shot noise field and the Gaussian field with the same covariance, which is of independent interest.


One Curious Identity Counting Graceful Labelings. Let a and b be positive integers with prime factorisations a p 1 np 2 n and b q 1 nq 2 n . We prove that the number of essentially distinct alpha graceful labelings of the complete bipartite graph K a, b equals the alternating sum of fourth powers of binomial coefficients ( 1) n binom 2n 0 4 binom 2n 1 4 binom 2n 2 4 binom 2n 3 4 cdots binom 2n 2n 4 .


RAMA A Rapid Multicut Algorithm on GPU. We propose a highly parallel primal dual algorithm for the multicut (a.k.a. correlation clustering) problem, a classical graph clustering problem widely used in machine learning and computer vision. Our algorithm consists of three steps executed recursively (1) Finding conflicted cycles that correspond to violated inequalities of the underlying multicut relaxation, (2) Performing message passing between the edges and cycles to optimize the Lagrange relaxation coming from the found violated cycles producing reduced costs and (3) Contracting edges with high reduced costs through matrix matrix multiplications. Our algorithm produces primal solutions and lower bounds that estimate the distance to optimum. We implement our algorithm on GPUs and show resulting one to two orders of magnitudes improvements in execution speed without sacrificing solution quality compared to traditional sequential algorithms that run on CPUs. We can solve very large scale benchmark problems with up to mathcal O (10 8) variables in a few seconds with small primal dual gaps. Our code is available at https github.com pawelswoboda RAMA.


Learning Graph Neural Networks with Positive and Unlabeled Nodes. Graph neural networks (GNNs) are important tools for transductive learning tasks, such as node classification in graphs, due to their expressive power in capturing complex interdependency between nodes. To enable graph neural network learning, existing works typically assume that labeled nodes, from two or multiple classes, are provided, so that a discriminative classifier can be learned from the labeled data. In reality, this assumption might be too restrictive for applications, as users may only provide labels of interest in a single class for a small number of nodes. In addition, most GNN models only aggregate information from short distances (e.g., 1 hop neighbors) in each round, and fail to capture long distance relationship in graphs. In this paper, we propose a novel graph neural network framework, long short distance aggregation networks (LSDAN), to overcome these limitations. By generating multiple graphs at different distance levels, based on the adjacency matrix, we develop a long short distance attention model to model these graphs. The direct neighbors are captured via a short distance attention mechanism, and neighbors with long distance are captured by a long distance attention mechanism. Two novel risk estimators are further employed to aggregate long short distance networks, for PU learning and the loss is back propagated for model learning. Experimental results on real world datasets demonstrate the effectiveness of our algorithm.


LatticeNet Fast Spatio Temporal Point Cloud Segmentation Using Permutohedral Lattices. Deep convolutional neural networks (CNNs) have shown outstanding performance in the task of semantically segmenting images. Applying the same methods on 3D data still poses challenges due to the heavy memory requirements and the lack of structured data. Here, we propose LatticeNet, a novel approach for 3D semantic segmentation, which takes raw point clouds as input. A PointNet describes the local geometry which we embed into a sparse permutohedral lattice. The lattice allows for fast convolutions while keeping a low memory footprint. Further, we introduce DeformSlice, a novel learned data dependent interpolation for projecting lattice features back onto the point cloud. We present results of 3D segmentation on multiple datasets where our method achieves state of the art performance. We also extend and evaluate our network for instance and dynamic object segmentation.


Fully Spiking Variational Autoencoder. Spiking neural networks (SNNs) can be run on neuromorphic devices with ultra high speed and ultra low energy consumption because of their binary and event driven nature. Therefore, SNNs are expected to have various applications, including as generative models being running on edge devices to create high quality images. In this study, we build a variational autoencoder (VAE) with SNN to enable image generation. VAE is known for its stability among generative models recently, its quality advanced. In vanilla VAE, the latent space is represented as a normal distribution, and floating point calculations are required in sampling. However, this is not possible in SNNs because all features must be binary time series data. Therefore, we constructed the latent space with an autoregressive SNN model, and randomly selected samples from its output to sample the latent variables. This allows the latent variables to follow the Bernoulli process and allows variational learning. Thus, we build the Fully Spiking Variational Autoencoder where all modules are constructed with SNN. To the best of our knowledge, we are the first to build a VAE only with SNN layers. We experimented with several datasets, and confirmed that it can generate images with the same or better quality compared to conventional ANNs. The code is available at https github.com kamata1729 FullySpikingVAE


Proof Complexity of Symbolic QBF Reasoning. We introduce and investigate symbolic proof systems for Quantified Boolean Formulas (QBF) operating on Ordered Binary Decision Diagrams (OBDDs). These systems capture QBF solvers that perform symbolic quantifier elimination, and as such admit short proofs of formulas of bounded path width and quantifier complexity. As a consequence, we obtain exponential separations from standard clausal proof systems, specifically (long distance) QU Resolution and IR Calc. We further develop a lower bound technique for symbolic QBF proof systems based on strategy extraction that lifts known lower bounds from communication complexity. This allows us to derive strong lower bounds against symbolic QBF proof systems that are independent of the variable ordering of the underlying OBDDs, and that hold even if the proof system is allowed access to an NP oracle.


Survey and Perspective on Social Emotions in Robotics. This study reviews research on social emotions in robotics. In robotics, the study of emotions has been pursued for a long time, including the study of their recognition, expression, and computational modeling of the basic mechanisms which underlie them. Research has advanced according to well known psychological findings, such as category and dimension theories. Many studies have been based on these basic theories, addressing only basic emotions. However, social emotions, also referred to as higher level emotions, have been studied in psychology. We believe that these higher level emotions are worth pursuing in robotics for next generation, socially aware robots. In this review paper, we summarize the findings on social emotions in psychology and neuroscience, along with a survey of the studies on social emotions in robotics that have been conducted to date. Thereafter, research directions toward the implementation of social emotions in robots are discussed.


Synthesising Reinforcement Learning Policies through Set Valued Inductive Rule Learning. Today's advanced Reinforcement Learning algorithms produce black box policies, that are often difficult to interpret and trust for a person. We introduce a policy distilling algorithm, building on the CN2 rule mining algorithm, that distills the policy into a rule based decision system. At the core of our approach is the fact that an RL process does not just learn a policy, a mapping from states to actions, but also produces extra meta information, such as action values indicating the quality of alternative actions. This meta information can indicate whether more than one action is near optimal for a certain state. We extend CN2 to make it able to leverage knowledge about equally good actions to distill the policy into fewer rules, increasing its interpretability by a person. Then, to ensure that the rules explain a valid, non degenerate policy, we introduce a refinement algorithm that fine tunes the rules to obtain good performance when executed in the environment. We demonstrate the applicability of our algorithm on the Mario AI benchmark, a complex task that requires modern reinforcement learning algorithms including neural networks. The explanations we produce capture the learned policy in only a few rules, that allow a person to understand what the black box agent learned. Source code https gitlab.ai.vub.ac.be yocoppen svcn2


Space Efficient Huffman Codes Revisited. Canonical Huffman code is an optimal prefix free compression code whose codewords enumerated in the lexicographical order form a list of binary words in non decreasing lengths. Gagie et al. (2015) gave a representation of this coding capable to encode or decode a symbol in constant worst case time. It uses sigma lg ell text max o( sigma) O( ell text max 2) bits of space, where sigma and ell text max are the alphabet size and maximum codeword length, respectively. We refine their representation to reduce the space complexity to sigma lg ell text max (1 o(1)) bits while preserving the constant encode and decode times. Our algorithmic idea can be applied to any canonical code.


Cyclic and well rounded lattices. We focus on two important classes of lattices, the well rounded and the cyclic. We show that every well rounded lattice in the plane is similar to a cyclic lattice, and use this cyclic parameterization to count planar well rounded similarity classes defined over a fixed number field with respect to height. We then investigate cyclic properties of the irreducible root lattices in arbitrary dimensions, in particular classifying those that are simple cyclic, i.e. generated by rotation shifts of a single vector. Finally, we classify cyclic, simple cyclic and well rounded cyclic lattices coming from rings of integers of Galois algebraic number fields.


Split link detection for sl(P) link homology in characteristic P. If P is a prime number, we show that reduced mathfrak sl (P) link homology with coefficients in mathbf Z P detects split links. The argument uses Dowlin's spectral sequence and sutured Floer homology with twisted coefficients. When P 2 , we recover Lipshitz Sarkar's split link detection result for Khovanov homology with mathbf Z 2 coefficients.


The Privatization of AI Research( ers) Causes and Potential Consequences From university industry interaction to public research brain drain?. The private sector is playing an increasingly important role in basic Artificial Intelligence (AI) R D. This phenomenon, which is reflected in the perception of a brain drain of researchers from academia to industry, is raising concerns about a privatisation of AI research which could constrain its societal benefits. We contribute to the evidence base by quantifying transition flows between industry and academia and studying its drivers and potential consequences. We find a growing net flow of researchers from academia to industry, particularly from elite institutions into technology companies such as Google, Microsoft and Facebook. Our survival regression analysis reveals that researchers working in the field of deep learning as well as those with higher average impact are more likely to transition into industry. A difference in differences analysis of the effect of switching into industry on a researcher's influence proxied by citations indicates that an initial increase in impact declines as researchers spend more time in industry. This points at a privatisation of AI knowledge compared to a counterfactual where those high impact researchers had remained in academia. Our findings highlight the importance of strengthening the public AI research sphere in order to ensure that the future of this powerful technology is not dominated by private interests.


Adaptive Massively Parallel Constant round Tree Contraction. Miller and Reif's FOCS'85 classic and fundamental tree contraction algorithm is a broadly applicable technique for the parallel solution of a large number of tree problems. Additionally it is also used as an algorithmic design technique for a large number of parallel graph algorithms. In all previously explored models of computation, however, tree contractions have only been achieved in Omega( log n) rounds of parallel run time. In this work, we not only introduce a generalized tree contraction method but also show it can be computed highly efficiently in O(1 epsilon 3) rounds in the Adaptive Massively Parallel Computing (AMPC) setting, where each machine has O(n epsilon) local memory for some 0 epsilon 1 . AMPC is a practical extension of Massively Parallel Computing (MPC) which utilizes distributed hash tables. In general, MPC is an abstract model for MapReduce, Hadoop, Spark, and Flume which are currently widely used across industry and has been studied extensively in the theory community in recent years. Last but not least, we show that our results extend to multiple problems on trees, including but not limited to maximum and maximal matching, maximum and maximal independent set, tree isomorphism testing, and more.


Mobile 3D Printing Robot Simulation with Viscoelastic Fluids. The system design and algorithm development of mobile 3D printing robots need a realistic simulation. They require a mobile robot simulation platform to interoperate with a physics based material simulation for handling interactions between the time variant deformable 3D printing materials and other simulated rigid bodies in the environment, which is not available for roboticists yet. To bridge this gap and enable the real time simulation of mobile 3D printing processes, we develop a simulation framework that includes particle based viscoelastic fluid simulation and particle to mesh conversion in the widely adopted Gazebo robotics simulator, avoiding the bottlenecks of traditional additive manufacturing simulation approaches. This framework is the first of its kind that enables the simulation of robot arms or mobile manipulators together with viscoelastic fluids. The method is tested using various material properties and multiple collaborating robots to demonstrate its simulation ability for the robots to plan and control the printhead trajectories and to visually sense at the same time the printed fluid materials as a free form mesh. The scalability as a function of available material particles in the simulation was also studied. A simulation with an average of 5 FPS was achieved on a regular desktop computer.


Shadow Neural Radiance Fields for Multi view Satellite Photogrammetry. We present a new generic method for shadow aware multi view satellite photogrammetry of Earth Observation scenes. Our proposed method, the Shadow Neural Radiance Field (S NeRF) follows recent advances in implicit volumetric representation learning. For each scene, we train S NeRF using very high spatial resolution optical images taken from known viewing angles. The learning requires no labels or shape priors it is self supervised by an image reconstruction loss. To accommodate for changing light source conditions both from a directional light source (the Sun) and a diffuse light source (the sky), we extend the NeRF approach in two ways. First, direct illumination from the Sun is modeled via a local light source visibility field. Second, indirect illumination from a diffuse light source is learned as a non local color field as a function of the position of the Sun. Quantitatively, the combination of these factors reduces the altitude and color errors in shaded areas, compared to NeRF. The S NeRF methodology not only performs novel view synthesis and full 3D shape estimation, it also enables shadow detection, albedo synthesis, and transient object filtering, without any explicit shape supervision.


An Effective Bernstein type Bound on Shannon Entropy over Countably Infinite Alphabets. We prove a Bernstein type bound for the difference between the average of negative log likelihoods of independent discrete random variables and the Shannon entropy, both defined on a countably infinite alphabet. The result holds for the class of discrete random variables with tails lighter than or on the same order of a discrete power law distribution. Most commonly used discrete distributions such as the Poisson distribution, the negative binomial distribution, and the power law distribution itself belong to this class. The bound is effective in the sense that we provide a method to compute the constants in it.


The EpiBench Platform to Propel AI ML based Epidemic Forecasting A Prototype Demonstration Reaching Human Expert level Performance. During the COVID 19 pandemic, a significant effort has gone into developing ML driven epidemic forecasting techniques. However, benchmarks do not exist to claim if a new AI ML technique is better than the existing ones. The "covid forecast hub" is a collection of more than 30 teams, including us, that submit their forecasts weekly to the CDC. It is not possible to declare whether one method is better than the other using those forecasts because each team's submission may correspond to different techniques over the period and involve human interventions as the teams are continuously changing tuning their approach. Such forecasts may be considered "human expert" forecasts and do not qualify as AI ML approaches, although they can be used as an indicator of human expert performance. We are interested in supporting AI ML research in epidemic forecasting which can lead to scalable forecasting without human intervention. Which modeling technique, learning strategy, and data pre processing technique work well for epidemic forecasting is still an open problem. To help advance the state of the art AI ML applied to epidemiology, a benchmark with a collection of performance points is needed and the current "state of the art" techniques need to be identified. We propose EpiBench a platform consisting of community driven benchmarks for AI ML applied to epidemic forecasting to standardize the challenge with a uniform evaluation protocol. In this paper, we introduce a prototype of EpiBench which is currently running and accepting submissions for the task of forecasting COVID 19 cases and deaths in the US states and We demonstrate that we can utilize the prototype to develop an ensemble relying on fully automated epidemic forecasts (no human intervention) that reaches human expert level ensemble currently being used by the CDC.


Responsibility and verification Importance value in temporal logics. We aim at measuring the influence of the nondeterministic choices of a part of a system on its ability to satisfy a specification. For this purpose, we apply the concept of Shapley values to verification as a means to evaluate how important a part of a system is. The importance of a component is measured by giving its control to an adversary, alone or along with other components, and testing whether the system can still fulfill the specification. We study this idea in the framework of model checking with various classical types of linear time specification, and propose several ways to transpose it to branching ones. We also provide tight complexity bounds in almost every case.


RSN Range Sparse Net for Efficient, Accurate LiDAR 3D Object Detection. The detection of 3D objects from LiDAR data is a critical component in most autonomous driving systems. Safe, high speed driving needs larger detection ranges, which are enabled by new LiDARs. These larger detection ranges require more efficient and accurate detection models. Towards this goal, we propose Range Sparse Net (RSN), a simple, efficient, and accurate 3D object detector in order to tackle real time 3D object detection in this extended detection regime. RSN predicts foreground points from range images and applies sparse convolutions on the selected foreground points to detect objects. The lightweight 2D convolutions on dense range images results in significantly fewer selected foreground points, thus enabling the later sparse convolutions in RSN to efficiently operate. Combining features from the range image further enhance detection accuracy. RSN runs at more than 60 frames per second on a 150m x 150m detection region on Waymo Open Dataset (WOD) while being more accurate than previously published detectors. As of 11 2020, RSN is ranked first in the WOD leaderboard based on the APH LEVEL 1 metrics for LiDAR based pedestrian and vehicle detection, while being several times faster than alternatives.


Tarema Adaptive Resource Allocation for Scalable Scientific Workflows in Heterogeneous Clusters. Scientific workflow management systems like Nextflow support large scale data analysis by abstracting away the details of scientific workflows. In these systems, workflows consist of several abstract tasks, of which instances are run in parallel and transform input partitions into output partitions. Resource managers like Kubernetes execute such workflow tasks on cluster infrastructures. However, these resource managers only consider the number of CPUs and the amount of available memory when assigning tasks to resources they do not consider hardware differences beyond these numbers, while computational speed and memory access rates can differ significantly. We propose Tarema, a system for allocating task instances to heterogeneous cluster resources during the execution of scalable scientific workflows. First, Tarema profiles the available infrastructure with a set of benchmark programs and groups cluster nodes with similar performance. Second, Tarema uses online monitoring data of tasks, assigning labels to tasks depending on their resource usage. Third, Tarema uses the node groups and task labels to dynamically assign task instances evenly to resources based on resource demand. Our evaluation of a prototype implementation for Kubernetes, using five real world Nextflow workflows from the popular nf core framework and two 15 node clusters consisting of different virtual machines, shows a mean reduction of isolated job runtimes by 19.8 compared to popular schedulers in widely used resource managers and 4.54 compared to the heuristic SJFN, while providing a better cluster usage. Moreover, executing two long running workflows in parallel and on restricted resources shows that Tarema is able to reduce the runtimes even more while providing a fair cluster usage.


PYInfer Deep Learning Semantic Type Inference for Python Variables. Python type inference is challenging in practice. Due to its dynamic properties and extensive dependencies on third party libraries without type annotations, the performance of traditional static analysis techniques is limited. Although semantics in source code can help manifest intended usage for variables (thus help infer types), they are usually ignored by existing tools. In this paper, we propose PYInfer, an end to end learning based type inference tool that automatically generates type annotations for Python variables. The key insight is that contextual code semantics is critical in inferring the type for a variable. For each use of a variable, we collect a few tokens within its contextual scope, and design a neural network to predict its type. One challenge is that it is difficult to collect a high quality human labeled training dataset for this purpose. To address this issue, we apply an existing static analyzer to generate the ground truth for variables in source code. Our main contribution is a novel approach to statically infer variable types effectively and efficiently. Formulating the type inference as a classification problem, we can handle user defined types and predict type probabilities for each variable. Our model achieves 91.2 accuracy on classifying 11 basic types in Python and 81.2 accuracy on classifying 500 most common types. Our results substantially outperform the state of the art type annotators. Moreover, PYInfer achieves 5.2X more code coverage and is 187X faster than a state of the art learning based tool. With similar time consumption, our model annotates 5X more variables than a state of the art static analysis tool. Our model also outperforms a learning based function level annotator on annotating types for variables and function arguments. All our tools and datasets are publicly available to facilitate future research in this direction.