Mutant reduction evaluation what is there and what is missing?. Background. Many mutation reduction strategies, which aim to reduce the number of mutants, have been proposed. Problem. It is important to measure the ability of a mutation reduction strategy to maintain test suite effectiveness evaluation. However, existing evaluation indicators are unable to measure the "order preserving ability". Objective. We aim to propose evaluation indicators to measure the "order preserving ability" of a mutation reduction strategy, which is important but missing in our community. Method. Given a test suite on a Software Under Test (SUT) with a set of original mutants, we leverage the test suite to generate a group of test suites that have a partial order relationship in fault detecting potential. When evaluating a reduction strategy, we first construct two partial order relationships among the generated test suites in terms of mutation score, one with the original mutants and another with the reduced mutants. Then, we measure the extent to which the two partial order relationships are consistent. The more consistent the two partial order relationships are, the stronger the Order Preservation (OP) of the mutation reduction strategy is, and the more effective the reduction strategy is. Furthermore, we propose Effort aware Relative Order Preservation (EROP) to measure how much gain a mutation reduction strategy can provide compared with a random reduction strategy. Result. The experimental results show that OP and EROP are able to efficiently measure the "order preserving ability" of a mutation reduction strategy. As a result, they have a better ability to distinguish various mutation reduction strategies compared with the existing evaluation indicators. Conclusion. We suggest, for the researchers, that OP and EROP should be used to measure the effectiveness of a mutant reduction strategy.


Doubly isogenous genus 2 curves with D 4 action. We study the zeta functions of curves over finite fields. Suppose C and C' are curves over a finite field K, with K rational base points P and P', and let D and D' be the pullbacks (via the Abel Jacobi map) of the multiplication by 2 maps on their Jacobians. We say that (C,P) and (C',P') are doubly isogenous if Jac(C) and Jac(C') are isogenous over K and Jac(D) and Jac(D') are isogenous over K. For curves of genus 2 whose automorphism groups contain the dihedral group of order eight, we show that the number of pairs of doubly isogenous curves is larger than naive heuristics predict, and we provide an explanation for this phenomenon.


Multi Label Classification Neural Networks with Hard Logical Constraints. Multi label classification (MC) is a standard machine learning problem in which a data point can be associated with a set of classes. A more challenging scenario is given by hierarchical multi label classification (HMC) problems, in which every prediction must satisfy a given set of hard constraints expressing subclass relationships between classes. In this paper, we propose C HMCNN(h), a novel approach for solving HMC problems, which, given a network h for the underlying MC problem, exploits the hierarchy information in order to produce predictions coherent with the constraints and to improve performance. Furthermore, we extend the logic used to express HMC constraints in order to be able to specify more complex relations among the classes and propose a new model CCN(h), which extends C HMCNN(h) and is again able to satisfy and exploit the constraints to improve performance. We conduct an extensive experimental analysis showing the superior performance of both C HMCNN(h) and CCN(h) when compared to state of the art models in both the HMC and the general MC setting with hard logical constraints.


A (2 epsilon) Approximation Algorithm for Maximum Independent Set of Rectangles. We study the Maximum Independent Set of Rectangles (MISR) problem, where we are given a set of axis parallel rectangles in the plane and the goal is to select a subset of non overlapping rectangles of maximum cardinality. In a recent breakthrough, Mitchell 2021 obtained the first constant factor approximation algorithm for MISR. His algorithm achieves an approximation ratio of 10 and it is based on a dynamic program that intuitively recursively partitions the input plane into special polygons called corner clipped rectangles (CCRs), without intersecting certain special horizontal line segments called fences. In this paper, we present a (2 epsilon) approximation algorithm for MISR which is also based on a recursive partitioning scheme. First, we use a partition into a class of axis parallel polygons with constant complexity each that are more general than CCRs. This allows us to provide an arguably simpler analysis and at the same time already improves the approximation ratio to 6. Then, using a more elaborate charging scheme and a recursive partitioning into general axis parallel polygons with constant complexity, we improve our approximation ratio to 2 epsilon . In particular, we construct a recursive partitioning based on more general fences which can be sequences of up to O(1 epsilon) line segments each. This partitioning routine and our other new ideas may be useful for future work towards a PTAS for MISR.


Spatial Constraint Generation for Motion Planning in Dynamic Environments. This paper presents a novel method to generate spatial constraints for motion planning in dynamic environments. Motion planning methods for autonomous driving and mobile robots typically need to rely on the spatial constraints imposed by a map based global planner to generate a collision free trajectory. These methods may fail without an offline map or where the map is invalid due to dynamic changes in the environment such as road obstruction, construction, and traffic congestion. To address this problem, triangulation based methods can be used to obtain a spatial constraint. However, the existing methods fall short when dealing with dynamic environments and may lead the motion planner to an unrecoverable state. In this paper, we propose a new method to generate a sequence of channels across different triangulation mesh topologies to serve as the spatial constraints. This can be applied to motion planning of autonomous vehicles or robots in cluttered, unstructured environments. The proposed method is evaluated and compared with other triangulation based methods in synthetic and complex scenarios collected from a real world autonomous driving dataset. We have shown that the proposed method results in a more stable, long term plan with a higher task completion rate, faster arrival time, a higher rate of successful plans, and fewer collisions compared to existing methods.


Marriage is a Peach and a Chalice Modelling Cultural Symbolism on the SemanticWeb. In this work, we fill the gap in the Semantic Web in the context of Cultural Symbolism. Building upon earlier work in, we introduce the Simulation Ontology, an ontology that models the background knowledge of symbolic meanings, developed by combining the concepts taken from the authoritative theory of Simulacra and Simulations of Jean Baudrillard with symbolic structures and content taken from "Symbolism a Comprehensive Dictionary" by Steven Olderr. We re engineered the symbolic knowledge already present in heterogeneous resources by converting it into our ontology schema to create HyperReal, the first knowledge graph completely dedicated to cultural symbolism. A first experiment run on the knowledge graph is presented to show the potential of quantitative research on symbolism.


On Extensions of Bilinear Maps. The paper deals with extension of bounded bilinear maps . It gives a necessary and sufficient condition for extending a bounded bilinear map on the Cartesian product of subspaces of Banach spaces . This leads to a full characterization for extension of bounded bilinear maps on the Cartesian product of arbitrary subspaces of Hilbert spaces . Applications concerning projective tensor products are also investigated.


Video Corpus Moment Retrieval with Contrastive Learning. Given a collection of untrimmed and unsegmented videos, video corpus moment retrieval (VCMR) is to retrieve a temporal moment (i.e., a fraction of a video) that semantically corresponds to a given text query. As video and text are from two distinct feature spaces, there are two general approaches to address VCMR (i) to separately encode each modality representations, then align the two modality representations for query processing, and (ii) to adopt fine grained cross modal interaction to learn multi modal representations for query processing. While the second approach often leads to better retrieval accuracy, the first approach is far more efficient. In this paper, we propose a Retrieval and Localization Network with Contrastive Learning (ReLoCLNet) for VCMR. We adopt the first approach and introduce two contrastive learning objectives to refine video encoder and text encoder to learn video and text representations separately but with better alignment for VCMR. The video contrastive learning (VideoCL) is to maximize mutual information between query and candidate video at video level. The frame contrastive learning (FrameCL) aims to highlight the moment region corresponds to the query at frame level, within a video. Experimental results show that, although ReLoCLNet encodes text and video separately for efficiency, its retrieval accuracy is comparable with baselines adopting cross modal interaction learning.


Learning Reward Functions from Scale Feedback. Today's robots are increasingly interacting with people and need to efficiently learn inexperienced user's preferences. A common framework is to iteratively query the user about which of two presented robot trajectories they prefer. While this minimizes the users effort, a strict choice does not yield any information on how much one trajectory is preferred. We propose scale feedback, where the user utilizes a slider to give more nuanced information. We introduce a probabilistic model on how users would provide feedback and derive a learning framework for the robot. We demonstrate the performance benefit of slider feedback in simulations, and validate our approach in two user studies suggesting that scale feedback enables more effective learning in practice.


On reducing the order of arm passes bandit streaming algorithms under memory bottleneck. In this work we explore multi arm bandit streaming model, especially in cases where the model faces resource bottleneck. We build over existing algorithms conditioned by limited arm memory at any instance of time. Specifically, we improve the amount of streaming passes it takes for a bandit algorithm to incur a O( sqrt T log(T) ) regret by a logarithmic factor, and also provide 2 pass algorithms with some initial conditions to incur a similar order of regret.


Tensor Network for Supervised Learning at Finite Temperature. The large variation of datasets is a huge barrier for image classification tasks. In this paper, we embraced this observation and introduce the finite temperature tensor network (FTTN), which imports the thermal perturbation into the matrix product states framework by placing all images in an environment with constant temperature, in analog to energy based learning. Tensor network is chosen since it is the best platform to introduce thermal fluctuation. Different from traditional network structure which directly takes the summation of individual losses as its loss function, FTTN regards it as thermal average loss computed from the entanglement with the environment. The temperature like parameter can be automatically optimized, which gives each database an individual temperature. FTTN obtains improvement in both test accuracy and convergence speed in several datasets. The non zero temperature automatically separates similar features, avoiding the wrong classification in previous architecture. The thermal fluctuation may give a better improvement in other frameworks, and we may also implement the temperature of database to improve the training effect.


Reinforcement Learning to Optimize Lifetime Value in Cold Start Recommendation. Recommender system plays a crucial role in modern E commerce platform. Due to the lack of historical interactions between users and items, cold start recommendation is a challenging problem. In order to alleviate the cold start issue, most existing methods introduce content and contextual information as the auxiliary information. Nevertheless, these methods assume the recommended items behave steadily over time, while in a typical E commerce scenario, items generally have very different performances throughout their life period. In such a situation, it would be beneficial to consider the long term return from the item perspective, which is usually ignored in conventional methods. Reinforcement learning (RL) naturally fits such a long term optimization problem, in which the recommender could identify high potential items, proactively allocate more user impressions to boost their growth, therefore improve the multi period cumulative gains. Inspired by this idea, we model the process as a Partially Observable and Controllable Markov Decision Process (POC MDP), and propose an actor critic RL framework (RL LTV) to incorporate the item lifetime values (LTV) into the recommendation. In RL LTV, the critic studies historical trajectories of items and predict the future LTV of fresh item, while the actor suggests a score based policy which maximizes the future LTV expectation. Scores suggested by the actor are then combined with classical ranking scores in a dual rank framework, therefore the recommendation is balanced with the LTV consideration. Our method outperforms the strong live baseline with a relative improvement of 8.67 and 18.03 on IPV and GMV of cold start items, on one of the largest E commerce platform.


Spherical CR uniformization of the "magic" 3 manifold. We show the 3 manifold at infinity of the complex hyperbolic triangle group Delta 3, infty, infty infty is the three cusped "magic" 3 manifold 6 1 3 . We also show the 3 manifold at infinity of the complex hyperbolic triangle group Delta 3,4, infty infty is the two cusped 3 manifold m295 in the Snappy Census, which is a 3 manifold obtained by Dehn filling on one cusp of 6 1 3 . In particular, hyperbolic 3 manifolds 6 1 3 and m295 admit spherical CR uniformizations. These results support our conjecture that the 3 manifold at infinity of the complex hyperbolic triangle group Delta 3,n,m infty is the one cusped hyperbolic 3 manifold from the "magic" 6 1 3 via Dehn fillings with filling slopes (n 2) and (m 2) on the first two cusps of it.


Rubrics for Charge Conserving Current Mapping in Finite Element Particle in Cell Methods. Modeling of kinetic plasmas using electromagnetic particle in cell methods (EM PIC) is a problem that is well worn, in that methods developed have been used extensively both understanding physics and exploiting them for device design. EM PIC tools have largely relied on finite difference methods coupled with particle representations of the distribution function. Refinements to ensure consistency and charge conservation have largely been an ad hoc efforts specific to finite difference methods. Meanwhile, solution methods for field solver have grown by leaps and bounds with significant performance metrics compared to finite difference methods. Developing new EM PIC computational schemes that leverage modern field solver technology means re examining analysis framework necessary for self consistent EM PIC solution. In this paper, we prescribe general rubrics for charge conservation, demonstrate how these are satisfied in conventional finite difference PIC as well as finite element PIC, and prescribe a novel charge conserving finite element PIC. Our effort leverages proper mappings on to de Rham sequences and lays a groundwork for understanding conditions that must be satisfied for consistency. Several numerical results demonstrate the applicability of these rubrics.


Outdoor inverse rendering from a single image using multiview self supervision. In this paper we show how to perform scene level inverse rendering to recover shape, reflectance and lighting from a single, uncontrolled image using a fully convolutional neural network. The network takes an RGB image as input, regresses albedo, shadow and normal maps from which we infer least squares optimal spherical harmonic lighting coefficients. Our network is trained using large uncontrolled multiview and timelapse image collections without ground truth. By incorporating a differentiable renderer, our network can learn from self supervision. Since the problem is ill posed we introduce additional supervision. Our key insight is to perform offline multiview stereo (MVS) on images containing rich illumination variation. From the MVS pose and depth maps, we can cross project between overlapping views such that Siamese training can be used to ensure consistent estimation of photometric invariants. MVS depth also provides direct coarse supervision for normal map estimation. We believe this is the first attempt to use MVS supervision for learning inverse rendering. In addition, we learn a statistical natural illumination prior. We evaluate performance on inverse rendering, normal map estimation and intrinsic image decomposition benchmarks.


Revealing topological phase in Pancharatnam Berry metasurfaces using mesoscopic electrodynamics. Relying on the local orientation of nanostructures, Pancharatnam Berry metasurfaces are currently enabling a new generation of polarization sensitive optical devices. A systematical mesoscopic description of topological metasurfaces is developed, providing a deeper understanding of the physical mechanisms leading to the polarization dependent breaking of translational symmetry in contrast with propagation phase effects. These theoretical results, along with interferometric experiments, contribute to the development of a solid theoretical framework for arbitrary polarization dependent metasurfaces.


Efficient Learning of Pinball TWSVM using Privileged Information and its applications. In any learning framework, an expert knowledge always plays a crucial role. But, in the field of machine learning, the knowledge offered by an expert is rarely used. Moreover, machine learning algorithms (SVM based) generally use hinge loss function which is sensitive towards the noise. Thus, in order to get the advantage from an expert knowledge and to reduce the sensitivity towards the noise, in this paper, we propose privileged information based Twin Pinball Support Vector Machine classifier (Pin TWSVMPI) where expert's knowledge is in the form of privileged information. The proposed Pin TWSVMPI incorporates privileged information by using correcting function so as to obtain two nonparallel decision hyperplanes. Further, in order to make computations more efficient and fast, we use Sequential Minimal Optimization (SMO) technique for obtaining the classifier and have also shown its application for Pedestrian detection and Handwritten digit recognition. Further, for UCI datasets, we first implement a procedure which extracts privileged information from the features of the dataset which are then further utilized by Pin TWSVMPI that leads to enhancement in classification accuracy with comparatively lesser computational time.


Fall of Giants How popular text based MLaaS fall against a simple evasion attack. The increased demand for machine learning applications made companies offer Machine Learning as a Service (MLaaS). In MLaaS (a market estimated 8000M USD by 2025), users pay for well performing ML models without dealing with the complicated training procedure. Among MLaaS, text based applications are the most popular ones (e.g., language translators). Given this popularity, MLaaS must provide resiliency to adversarial manipulations. For example, a wrong translation might lead to a misunderstanding between two parties. In the text domain, state of the art attacks mainly focus on strategies that leverage ML models' weaknesses. Unfortunately, not much attention has been given to the other pipeline' stages, such as the indexing stage (i.e., when a sentence is converted from a textual to a numerical representation) that, if manipulated, can significantly affect the final performance of the application. In this paper, we propose a novel text evasion technique called " textit Zero Width attack" (ZeW) that leverages the injection of human non readable characters, affecting indexing stage mechanisms. We demonstrate that our simple yet effective attack deceives MLaaS of "giants" such as Amazon, Google, IBM, and Microsoft. Our case study, based on the manipulation of hateful tweets, shows that out of 12 analyzed services, only one is resistant to our injection strategy. We finally introduce and test a simple textit input validation defense that can prevent our proposed attack.


A new method for constructing linear codes with small hulls. The hull of a linear code over finite fields is the intersection of the code and its dual, which was introduced by Assmus and Key. In this paper, we develop a method to construct linear codes with trivial hull ( LCD codes) and one dimensional hull by employing the positive characteristic analogues of Gauss sums. These codes are quasi abelian, and sometimes doubly circulant. Some sufficient conditions for a linear code to be an LCD code (resp. a linear code with one dimensional hull) are presented. It is worth mentioning that we present a lower bound on the minimum distances of the constructed linear codes. As an application, using these conditions, we obtain some optimal or almost optimal LCD codes (resp. linear codes with one dimensional hull) with respect to the online Database of Grassl.


Improving Zero Shot Entity Retrieval through Effective Dense Representations. Entity Linking (EL) seeks to align entity mentions in text to entries in a knowledge base and is usually comprised of two phases candidate generation and candidate ranking. While most methods focus on the latter, it is the candidate generation phase that sets an upper bound to both time and accuracy performance of the overall EL system. This work's contribution is a significant improvement in candidate generation which thus raises the performance threshold for EL, by generating candidates that include the gold entity in the least candidate set (top K). We propose a simple approach that efficiently embeds mention entity pairs in dense space through a BERT based bi encoder. Specifically, we extend (Wu et al., 2020) by introducing a new pooling function and incorporating entity type side information. We achieve a new state of the art 84.28 accuracy on top 50 candidates on the Zeshel dataset, compared to the previous 82.06 on the top 64 of (Wu et al., 2020). We report the results from extensive experimentation using our proposed model on both seen and unseen entity datasets. Our results suggest that our method could be a useful complement to existing EL approaches.


Competence based Curriculum Learning for Multilingual Machine Translation. Currently, multilingual machine translation is receiving more and more attention since it brings better performance for low resource languages (LRLs) and saves more space. However, existing multilingual machine translation models face a severe challenge imbalance. As a result, the translation performance of different languages in multilingual translation models are quite different. We argue that this imbalance problem stems from the different learning competencies of different languages. Therefore, we focus on balancing the learning competencies of different languages and propose Competence based Curriculum Learning for Multilingual Machine Translation, named CCL M. Specifically, we firstly define two competencies to help schedule the high resource languages (HRLs) and the low resource languages 1) Self evaluated Competence, evaluating how well the language itself has been learned and 2) HRLs evaluated Competence, evaluating whether an LRL is ready to be learned according to HRLs' Self evaluated Competence. Based on the above competencies, we utilize the proposed CCL M algorithm to gradually add new languages into the training set in a curriculum learning manner. Furthermore, we propose a novel competenceaware dynamic balancing sampling strategy for better selecting training samples in multilingual training. Experimental results show that our approach has achieved a steady and significant performance gain compared to the previous state of the art approach on the TED talks dataset.


Riiid! Answer Correctness Prediction Kaggle Challenge 4th Place Solution Summary. This paper presents my solution to the challenge "Riiid! Answer Correctness Prediction" on Kaggle hosted by Riiid Labs (2020), which scores 0.817 (AUC) and ranks 4th on the final private leaderboard. It is a single transformer based model heavily inspired from previous works such as SAKT, SAINT and SAINT . Novel ingredients that I believed to have made a difference are the time aware attention mechanism, the concatenation of the embeddings of the input sequences and the embedding of continuous features.


On absolutely and simply popular rankings. Van Zuylen et al. introduced the notion of a popular ranking in a voting context, where each voter submits a strictly ordered list of all candidates. A popular ranking pi of the candidates is at least as good as any other ranking sigma in the following sense if we compare pi to sigma , at least half of all voters will always weakly prefer pi . Whether a voter prefers one ranking to another is calculated based on the Kendall distance. A more traditional definition of popularity as applied to popular matchings, a well established topic in computational social choice is stricter, because it requires at least half of the voters emph who are not indifferent between pi and sigma to prefer pi . In this paper, we derive structural and algorithmic results in both settings, also improving upon the results by van Zuylen et al. We also point out strong connections to the famous open problem of finding a Kemeny consensus with 3 voters.


Saudi Parents' Privacy Concerns about Their Children's Smart Device Applications. In this paper, we investigate Saudi parents' privacy concerns regarding their children's smart device applications (apps). To this end, we conducted a survey and analysed 119 responses. Our results show that Saudi parents expressed a high level of concern regarding their children's privacy when using smart device apps. However, they expressed higher concerns about apps' content than privacy issues such as apps' requests to access sensitive data. Furthermore, parents' concerns are not in line with most of the children's installed apps, which contain apps inappropriate for their age, require parental guidance, and request access to sensitive data such as location. We also discuss several aspects of Saudi parents' practices and concerns compared to those reported by Western (mainly from the UK) and Chinese parents in previous reports. We found interesting patterns and established new relationships. For example, Saudi and Western parents show higher levels of privacy concerns than Chinese parents. Finally, we tested 14 privacy practices and concerns against high versus low socioeconomic classes (parents' education, technical background, and income) to find whether there are significant differences between high and low classes (we denote these differences by "digital divide"). Out of 42 tests (14 properties x 3 classes) we found significant differences between high and low classes in 7 tests only. While this is a positive trend overall, it is important to work on bridging these gaps. The results of this paper provide key findings to identify areas of improvement and recommendations, especially for Saudis, which can be used by parents, developers, researchers, regulators, and policy makers.


Spectral radius conditions for the existence of all subtrees of diameter at most four. Let mu(G) denote the spectral radius of a graph G . We partly confirm a conjecture due to Nikiforov, which is a spectral radius analogue of the well known Erd H o s S 'os Conjecture that any tree of order t is contained in a graph of average degree greater than t 2 . Let S n,k K k vee overline K n k , and let S n,k be the graph obtained from S n,k by adding a single edge joining two vertices of the independent set of S n,k . In 2010, Nikiforov conjectured that for a given integer k , every graph G of sufficiently large order n with mu(G) geq mu(S n,k ) contains all trees of order 2k 3 , unless G S n,k . We confirm this conjecture for trees with diameter at most four, with one exception. In fact, we prove the following stronger result for k geq 8 . If a graph G with sufficiently large order n satisfies mu(G) geq mu(S n,k ) and G neq S n,k , then G contains all trees of order 2k 3 with diameter at most four, except for the tree obtained from a star K 1,k 1 by subdividing each of its k 1 edges once.


The Karoubi Weibel complexity for groups. Let G be a finitely presented group. A new complexity called textit Karoubi Weibel complexity or textit covering type , is defined for G . The construction is inspired by recent work of Karoubi and Weibel cite KW , initially applied to topological spaces. We introduce a similar notion in combinatorial form in order to apply it to finitely presentable groups. Some properties of this complexity as well as a few examples of calculation estimation for certain classes of finitely presentable groups are considered. Finally we give a few applications of complexity to some geometric problems, namely to the systolic area and the volume entropy of groups.


Not All Voxels Are Equal Semantic Scene Completion from the Point Voxel Perspective. We revisit Semantic Scene Completion (SSC), a useful task to predict the semantic and occupancy representation of 3D scenes, in this paper. A number of methods for this task are always based on voxelized scene representations for keeping local scene structure. However, due to the existence of visible empty voxels, these methods always suffer from heavy computation redundancy when the network goes deeper, and thus limit the completion quality. To address this dilemma, we propose our novel point voxel aggregation network for this task. Firstly, we transfer the voxelized scenes to point clouds by removing these visible empty voxels and adopt a deep point stream to capture semantic information from the scene efficiently. Meanwhile, a light weight voxel stream containing only two 3D convolution layers preserves local structures of the voxelized scenes. Furthermore, we design an anisotropic voxel aggregation operator to fuse the structure details from the voxel stream into the point stream, and a semantic aware propagation module to enhance the up sampling process in the point stream by semantic labels. We demonstrate that our model surpasses state of the arts on two benchmarks by a large margin, with only depth images as the input.


A Nested Cross Decomposition Algorithm for Power System Capacity Expansion with Multiscale Uncertainties. Modern electric power systems have witnessed rapidly increasing penetration of renewable energy, storage, electrical vehicles and various demand response resources. The electric infrastructure planning is thus facing more challenges due to the variability and uncertainties arising from the diverse new resources. This study aims to develop a multistage and multiscale stochastic mixed integer programming (MM SMIP) model to capture both the coarse temporal scale uncertainties, such as investment cost and long run demand stochasticity, and fine temporal scale uncertainties, such as hourly renewable energy output and electricity demand uncertainties, for the power system capacity expansion problem. To be applied to a real power system, the resulting model will lead to extremely large scale mixed integer programming problems, which suffer not only the well known curse of dimensionality, but also computational difficulties with a vast number of integer variables at each stage. In addressing such challenges associated with the MM SMIP model, we propose a nested cross decomposition algorithm that consists of two layers of decomposition, that is, the Dantzig Wolfe decomposition and L shaped decomposition. The algorithm exhibits promising computational performance under our numerical study, and is especially amenable to parallel computing, which will also be demonstrated through the computational results.


Flexibility and rigidity of free boundary MHD equilibria. We study stationary free boundary configurations of an ideal incompressible magnetohydrodynamic fluid possessing nested flux surfaces. In 2D simply connected domains, we prove that if the magnetic field and velocity field are never commensurate, the only possible domain for any such equilibria is a disk, and the velocity and magnetic field are circular. We give examples of non symmetric equilibria occupying a domain of any shape by imposing an external magnetic field generated by a singular current sheet charge distribution (external coils). Some results carry over to 3D axisymmetric solutions. These results highlight the importance of external magnetic fields for the existence of asymmetric equilibria.


KaggleDBQA Realistic Evaluation of Text to SQL Parsers. The goal of database question answering is to enable natural language querying of real life relational databases in diverse application domains. Recently, large scale datasets such as Spider and WikiSQL facilitated novel modeling techniques for text to SQL parsing, improving zero shot generalization to unseen databases. In this work, we examine the challenges that still prevent these techniques from practical deployment. First, we present KaggleDBQA, a new cross domain evaluation dataset of real Web databases, with domain specific data types, original formatting, and unrestricted questions. Second, we re examine the choice of evaluation tasks for text to SQL parsers as applied in real life settings. Finally, we augment our in domain evaluation task with database documentation, a naturally occurring source of implicit domain knowledge. We show that KaggleDBQA presents a challenge to state of the art zero shot parsers but a more realistic evaluation setting and creative use of associated database documentation boosts their accuracy by over 13.2 , doubling their performance.


Scale dependent permeability and formation factor in porous media Applications from percolation theory. Understanding porous media properties and their scale dependence have been an active subject of research in the past several decades in hydrology, geosciences and petroleum engineering. The scale dependence of flow in porous media is attributed to small and large scale heterogeneities, such as pore size distribution, pore connectivity, long range correlations, fractures and faults orientations, and spatial and temporal variations. The main objective of this study was to investigate how permeability (k) and formation factor (F) vary with sample dimension at small scales by means of a combination of pore network modeling and percolation theory. For this purpose, the permeability and formation factor were simulated in twelve three dimensional pore networks with different levels of pore scale heterogeneities. Simulations were carried out at five different network sizes, i.e., 1130, 2250, 3380, 4510 and 6770 microns. Four theoretical models were also developed based on percolation theory to estimate the scale dependence of permeability and formation factor from the pore throat radius distribution. In addition, two other theoretical scale dependent permeability models were proposed to estimate permeability at different scales from the pore throat radius distribution and formation factor. Comparing theoretical estimations with numerical simulations showed that the proposed models estimate the scale dependence of permeability and formation factor reasonably. The calculated relative error (RE) ranged between 3.7 and 3.8 for the permeability and between 0.21 and 4.04 for the formation factor in the studied pore networks.


Automatic Boundedness of Adjointable Operators on Barreled VH Spaces. We consider the space of adjointable operators on barreled VH (Vector Hilbert) spaces and show that such operators are automatically bounded.This generalizes the well known corresponding result for locally Hilbert C modules.We pick a consequence of this result in the dilation theory of VH spaces and show that, when barreled VH spaces are considered, a certain boundedness condition for the existence of VH space linearisations, equivalently, of reproducing kernel VH spaces, is automatically satisfied.


Estimating Presentation Competence using Multimodal Nonverbal Behavioral Cues. Public speaking and presentation competence plays an essential role in many areas of social interaction in our educational, professional, and everyday life. Since our intention during a speech can differ from what is actually understood by the audience, the ability to appropriately convey our message requires a complex set of skills. Presentation competence is cultivated in the early school years and continuously developed over time. One approach that can promote efficient development of presentation competence is the automated analysis of human behavior during a speech based on visual and audio features and machine learning. Furthermore, this analysis can be used to suggest improvements and the development of skills related to presentation competence. In this work, we investigate the contribution of different nonverbal behavioral cues, namely, facial, body pose based, and audio related features, to estimate presentation competence. The analyses were performed on videos of 251 students while the automated assessment is based on manual ratings according to the T "ubingen Instrument for Presentation Competence (TIP). Our classification results reached the best performance with early fusion in the same dataset evaluation (accuracy of 71.25 ) and late fusion of speech, face, and body pose features in the cross dataset evaluation (accuracy of 78.11 ). Similarly, regression results performed the best with fusion strategies.


Stability for an inverse source problem of the damped biharmonic plate equation. This paper is concerned with the stability of the inverse source problem for the damped biharmonic plate equation in three dimensions. The stability estimate consists of the Lipschitz type data discrepancy and the high frequency tail of the source function, where the latter decreases as the upper bound of the frequency increases. The stability also shows exponential dependence on the constant damping coefficient. The analysis employs Carleman estimates and time decay estimates for the damped plate wave equation to obtain an exact observability bound and depends on the study of the resonance free region and an upper bound of the resolvent of the biharmonic operator with respect to the complex wavenumber.


On the number of real zeros of real entire functions with a non decreasing sequence of the second quotients of Taylor coefficients. For an entire function f(z) sum k 0 infty a k z k, a k 0, we define the sequence of the second quotients of Taylor coefficients Q left( frac a k 2 a k 1 a k 1 right) k 1 infty . We find new necessary conditions for a function with a non decreasing sequence Q to belong to the Laguerre P 'olya class of type I. We also estimate the possible number of nonreal zeros for a function with a non decreasing sequence Q.


The local cohomology spectral sequence for topological modular forms. We discuss proofs of local cohomology theorems for topological modular forms, based on Mahowald Rezk duality and on Gorenstein duality, and then make the associated local cohomology spectral sequences explicit, including their differential patterns and hidden extensions.


Danzer's Problem, Effective Constructions of Dense Forests and Digital Sequences. A 1965 problem due to Danzer asks whether there exists a set in Euclidean space with finite density intersecting any convex body of volume one. A recent approach to this problem is concerned with the construction of dense forests and is obtained by a suitable weakening of the volume constraint. A dense forest is a discrete point set of finite density getting uniformly close to long enough line segments. The distribution of points in a dense forest is then quantified in terms of a visibility function. Another way to weaken the assumptions in Danzer's problem is by relaxing the density constraint. In this respect, a new concept is introduced in this paper, namely that of an optical forest. An optical forest in mathbb R d is a point set with optimal visibility but not necessarily with finite density. In the literature, the best constructions of Danzer sets and dense forests lack effectivity. The goal of this paper is to provide constructions of dense and optical forests which yield the best known results in any dimension d ge 2 both in terms of visibility and density bounds and effectiveness. Namely, there are three main results in this work (1) the construction of a dense forest with the best known visibility bound which, furthermore, enjoys the property of being deterministic (2) the deterministic construction of an optical forest with a density failing to be finite only up to a logarithm and (3) the construction of a planar Peres type forest (that is, a dense forest obtained from a construction due to Peres) with the best known visibility bound. This is achieved by constructing a deterministic digital sequence satisfying strong dispersion properties.


Exact Probability Landscapes of Stochastic Phenotype Switching in Feed Forward Loops Phase Diagrams of Multimodality. Feed forward loops (FFLs) are among the most ubiquitously found motifs of reaction networks in nature. However, little is known about their stochastic behavior and the variety of network phenotypes they can exhibit. In this study, we provide full characterizations of the properties of stochastic multimodality of FFLs, and how switching between different network phenotypes are controlled. We have computed the exact steady state probability landscapes of all eight types of coherent and incoherent FFLs using the finite butter ACME algorithm, and quantified the exact topological features of their high dimensional probability landscapes using persistent homology. Through analysis of the degree of multimodality for each of a set of 10,812 probability landscapes, where each landscape resides over 10 5 10 6 microstates, we have constructed comprehensive phase diagrams of all relevant behavior of FFL multimodality over broad ranges of input and regulation intensities, as well as different regimes of promoter binding dynamics. Our results show that with slow binding and unbinding dynamics of transcription factor to promoter, FFLs exhibit strong stochastic behavior that is very different from what would be inferred from deterministic models. In addition, input intensity play major roles in the phenotypes of FFLs At weak input intensity, FFL exhibit monomodality, but strong input intensity may result in up to 6 stable phenotypes. Furthermore, we found that gene duplication can enlarge stable regions of specific multimodalities and enrich the phenotypic diversity of FFL networks, providing means for cells towards better adaptation to changing environment. Our results are directly applicable to analysis of behavior of FFLs in biological processes such as stem cell differentiation and for design of synthetic networks when certain phenotypic behavior is desired.


Global weak solutions to the density dependent Hall magnetohydrodynamics system. We are concerned with the global existence of finite energy weak solutions to 3D density dependent magnetohydrodynamics (MHD) system with Hall effect set in a general smooth bounded domain. The perfectly conducting wall boundary condition is imposed on the magnetic field. Due to the degeneracy of Hall effect term (a tri linear term) in vacuum, we assumed initial density lies in the bounded function space and having a positive lower bound. Particularly, these bounds are preserved by the density transport equation which helps yield a satisfying compactness argument of the magnetic field.


Sharp weak type estimates for a family of Soria bases. Let mathcal B be a collection of rectangular parallelepipeds in mathbb R 3 whose sides are parallel to the coordinate axes and such that mathcal B contains parallelepipeds with side lengths of the form s, frac 2 N s , t , where s, t 0 and N lies in a nonempty subset S of the natural numbers. We show that if S is an infinite set, then the associated geometric maximal operator M mathcal B satisfies the weak type estimate left left x in mathbb R 3 M mathcal B f(x) alpha right right leq C int mathbb R 3 frac f alpha left(1 log frac f alpha right) 2 but does not satisfy an estimate of the form left left x in mathbb R 3 M mathcal B f(x) alpha right right leq C int mathbb R 3 phi left( frac f alpha right) for any convex increasing function phi mathbb 0, infty) rightarrow 0, infty) satisfying the condition lim x rightarrow infty frac phi(x) x ( log(1 x)) 2 0 .


The Uniform Boundedness and Dynamical Lang Conjectures for polynomials. We give a conditional proof of the Uniform Boundedness Conjecture of Morton and Silverman in the case of polynomials over number fields, assuming a standard conjecture in arithmetic geometry. Our technique simultaneously yields a dynamical analogue of Lang's conjecture on minimal canonical heights for these maps. We obtain similar results for non isotrivial polynomials over a function field of characteristic zero. When the latter are unicritical of degree at least 5, the results hold unconditionally.


Empirical Analysis of Machine Learning Configurations for Prediction of Multiple Organ Failure in Trauma Patients. Multiple organ failure (MOF) is a life threatening condition. Due to its urgency and high mortality rate, early detection is critical for clinicians to provide appropriate treatment. In this paper, we perform quantitative analysis on early MOF prediction with comprehensive machine learning (ML) configurations, including data preprocessing (missing value treatment, label balancing, feature scaling), feature selection, classifier choice, and hyperparameter tuning. Results show that classifier choice impacts both the performance improvement and variation most among all the configurations. In general, complex classifiers including ensemble methods can provide better performance than simple classifiers. However, blindly pursuing complex classifiers is unwise as it also brings the risk of greater performance variation.


Learning Transport Processes with Machine Intelligence. We present a machine learning based approach to address the study of transport processes, ubiquitous in continuous mechanics, with particular attention to those phenomena ruled by complex micro physics, impractical to theoretical investigation, yet exhibiting emergent behavior describable by a closed mathematical expression. Our machine learning model, built using simple components and following a few well established practices, is capable of learning latent representations of the transport process substantially closer to the ground truth than expected from the nominal error characterising the data, leading to sound generalisation properties. This is demonstrated through an idealized study of the long standing problem of heat flux suppression relevant to fusion and cosmic plasmas. Our analysis shows that the result applies beyond those case specific assumptions and that, in particular, the accuracy of the learned representation is controllable through knowledge of the data quality (error properties) and a suitable choice of the dataset size. While the learned representation can be used as a plug in for numerical modeling purposes, it can also be leveraged with the above error analysis to obtain reliable mathematical expressions describing the transport mechanism and of great theoretical value.


On the spectral radius of graphs without a star forest. In this paper, we present two sharp upper bounds for the spectral radius of (bipartite) graphs with forbidden a star forest and characterize all extremal graphs. Moreover, the minimum least eigenvalue of the adjacency matrix of graph with forbidden a star forest and all extremal graphs for graphs are obtained.


The Moser isotopy for holomorphic symplectic and C symplectic structures. A C symplectic structure is a complex valued 2 form which is holomorphically symplectic for an appropriate complex structure. We prove an analogue of Moser's isotopy theorem for families of C symplectic structures and list several applications of this result. We prove that the degenerate twistorial deformation associated to a holomorphic Lagrangian fibration is locally trivial over the base of this fibration. This is used to extend several theorems about Lagrangian fibrations, known for projective hyperk "ahler manifolds, to the non projective case. We also exhibit new examples of non compact complex manifolds with infinitely many pairwise non birational algebraic compactifications.


Fast deep learning correspondence for neuron tracking and identification in C.elegans using synthetic training. We present an automated method to track and identify neurons in C. elegans, called "fast Deep Learning Correspondence" or fDLC, based on the transformer network architecture. The model is trained once on empirically derived synthetic data and then predicts neural correspondence across held out real animals via transfer learning. The same pre trained model both tracks neurons across time and identifies corresponding neurons across individuals. Performance is evaluated against hand annotated datasets, including NeuroPAL 1 . Using only position information, the method achieves 80.0 accuracy at tracking neurons within an individual and 65.8 accuracy at identifying neurons across individuals. Accuracy is even higher on a published dataset 2 . Accuracy reaches 76.5 when using color information from NeuroPAL. Unlike previous methods, fDLC does not require straightening or transforming the animal into a canonical coordinate system. The method is fast and predicts correspondence in 10 ms making it suitable for future real time applications.


GANiry Bald to Hairy Translation Using CycleGAN. This work presents our computer vision course project called bald men to hairy men translation using CycleGAN. On top of CycleGAN architecture, we utilize perceptual loss in order to achieve more realistic results. We also integrate conditional constrains to obtain different stylized and colored hairs on bald men. We conducted extensive experiments and present qualitative results in this paper. Our code and models are available at https github.com fidansamet GANiry.


Wave focusing and related multiple dispersion transitions in plane Poiseuille flows. Motivated by the recent discovery of a dispersive to nondispersive transition for linear waves in shear flows, we accurately explored the wavenumber Reynolds number parameter map of the plane Poiseuille flow, in the limit of least damped waves. We have discovered the existence of regions of the map where the dispersion and propagation features vary significantly from their surroundings. These regions are nested in the dispersive, low wavenumber part of the map. This complex dispersion scenario demonstrates the existence of linear dispersive focusing in wave envelopes evolving out of an initial, spatially localized, three dimensional perturbation. An asymptotic wave packet's representation, based on the saddle point method, allows to enlighten the nature of the packet's morphology, in particular the arrow shaped structure and spatial spreading rates. A correlation is also highlighted between the regions of largest dispersive focusing and the regions which are most subject to strong nonlinear coupling in observations.


Measuring Attribution in Natural Language Generation Models. With recent improvements in natural language generation (NLG) models for various applications, it has become imperative to have the means to identify and evaluate whether NLG output is only sharing verifiable information about the external world. In this work, we present a new evaluation framework entitled Attributable to Identified Sources (AIS) for assessing the output of natural language generation models, when such output pertains to the external world. We first define AIS and introduce a two stage annotation pipeline for allowing annotators to appropriately evaluate model output according to AIS guidelines. We empirically validate this approach on three generation datasets (two in the conversational QA domain and one in summarization) via human evaluation studies that suggest that AIS could serve as a common framework for measuring whether model generated statements are supported by underlying sources. We release guidelines for the human evaluation studies.


Improve Learning from Crowds via Generative Augmentation. Crowdsourcing provides an efficient label collection schema for supervised machine learning. However, to control annotation cost, each instance in the crowdsourced data is typically annotated by a small number of annotators. This creates a sparsity issue and limits the quality of machine learning models trained on such data. In this paper, we study how to handle sparsity in crowdsourced data using data augmentation. Specifically, we propose to directly learn a classifier by augmenting the raw sparse annotations. We implement two principles of high quality augmentation using Generative Adversarial Networks 1) the generated annotations should follow the distribution of authentic ones, which is measured by a discriminator 2) the generated annotations should have high mutual information with the ground truth labels, which is measured by an auxiliary network. Extensive experiments and comparisons against an array of state of the art learning from crowds methods on three real world datasets proved the effectiveness of our data augmentation framework. It shows the potential of our algorithm for low budget crowdsourcing in general.


SIRfyN Single Image Relighting from your Neighbors. We show how to relight a scene, depicted in a single image, such that (a) the overall shading has changed and (b) the resulting image looks like a natural image of that scene. Applications for such a procedure include generating training data and building authoring environments. Naive methods for doing this fail. One reason is that shading and albedo are quite strongly related for example, sharp boundaries in shading tend to appear at depth discontinuities, which usually apparent in albedo. The same scene can be lit in different ways, and established theory shows the different lightings form a cone (the illumination cone). Novel theory shows that one can use similar scenes to estimate the different lightings that apply to a given scene, with bounded expected error. Our method exploits this theory to estimate a representation of the available lighting fields in the form of imputed generators of the illumination cone. Our procedure does not require expensive "inverse graphics" datasets, and sees no ground truth data of any kind. Qualitative evaluation suggests the method can erase and restore soft indoor shadows, and can "steer" light around a scene. We offer a summary quantitative evaluation of the method with a novel application of the FID. An extension of the FID allows per generated image evaluation. Furthermore, we offer qualitative evaluation with a user study, and show that our method produces images that can successfully be used for data augmentation.


Vision based Vehicle Speed Estimation A Survey. The need to accurately estimate the speed of road vehicles is becoming increasingly important for at least two main reasons. First, the number of speed cameras installed worldwide has been growing in recent years, as the introduction and enforcement of appropriate speed limits is considered one of the most effective means to increase the road safety. Second, traffic monitoring and forecasting in road networks plays a fundamental role to enhance traffic, emissions and energy consumption in smart cities, being the speed of the vehicles one of the most relevant parameters of the traffic state. Among the technologies available for the accurate detection of vehicle speed, the use of vision based systems brings great challenges to be solved, but also great potential advantages, such as the drastic reduction of costs due to the absence of expensive range sensors, and the possibility of identifying vehicles accurately. This paper provides a review of vision based vehicle speed estimation. We describe the terminology, the application domains, and propose a complete taxonomy of a large selection of works that categorizes all stages involved. An overview of performance evaluation metrics and available datasets is provided. Finally, we discuss current limitations and future directions.


Towards Practical Evaluation of Android ICC Resolution Techniques. Inter component communication (ICC) is a widely used mechanism in mobile apps, which enables message based control flow transferring and data passing between components. Effective ICC resolution requires precisely identifying entry points, analyzing data values of ICC fields, modeling related framework APIs, etc. Due to various control flow and data value tracking related characteristics involved and the lack of oracles for real world apps, the practical evaluation of ICC resolution techniques is challenging. To fill this gap, we collect multiple type benchmark suites with 4,104 apps, covering the characteristic specific, open source, and commercial ones. Considering the differences of benchmarks, we adopt various evaluation metrics for them, which are based on the number count, graph structure, and reliable oracle. As the oracle for real world apps is unavailable, we design a dynamic analysis approach to extract the real ICC links triggered during GUI exploration. Overall, we manually confirm 1,680 ICCs and label 42,000 code characteristic tags to form a reliable oracle set. The evaluation performed on six off the shelf ICC resolution tools show that tools behave inconsistently on multiple benchmarks and with different metrics. Using reliable oracles, we find that up to 39 85 ICCs are missed by the six tools, for their inadequate analysis of specific code characteristics. And with the help of graph metrics, we identify many wrongly reported ICCs caused by conservative analysis or the transitivity of imprecision. Finally, we summarize eight FN FP patterns in ICC resolution for further improvement.


Extrapolating the Arnoldi Algorithm To Improve Eigenvector Convergence. We consider extrapolation of the Arnoldi algorithm to accelerate computation of the dominant eigenvalue eigenvector pair. The basic algorithm uses sequences of Krylov vectors to form a small eigenproblem which is solved exactly. The two dominant eigenvectors output from consecutive Arnoldi steps are then recombined to form an extrapolated iterate, and this accelerated iterate is used to restart the next Arnoldi process. We present numerical results testing the algorithm on a variety of cases and find on most examples it substantially improves the performance of restarted Arnoldi. The extrapolation is a simple post processing step which has minimal computational cost.


Recent advances of MXene saturable absorber for near infrared mode locked fiber laser. To date, MXene has been discovered for its viability as alternatives to conventional saturable absorber such as carbon nanotube and graphene. The characteristics of high nonlinear saturable absorption, astounding modulation depth, flexible bandgap tunability, and high electron density near Fermi level are the fundamentals of the MXene as an excellent saturable absorber candidate. In particular, the research effort contributed to MXene in nonlinear ul trafast optics are extensively growing because MXene comprises one of the largest families in 2D nanomaterials that provides huge combination possibilities by forming a class of metal carbide or metal nitride with 2D layered structure. Herein, this review summarizes the recent development on synthesis and material characterization of the MXene, the studies on its nonlinear saturable absorption and the application of the MXene saturable absorber in near infrared mode locked fiber laser. Finally, some issues and challenges as well as future perspectives of this novel material are discussed.


Adapting Behaviors via Reactive Synthesis. In the emph Adapter Design Pattern , a programmer implements a emph Target interface by constructing an emph Adapter that accesses an existing emph Adaptee code. In this work, we present a reactive synthesis interpretation to the adapter design pattern, wherein an algorithm takes an emph Adaptee and a emph Target transducers, and the aim is to synthesize an emph Adapter transducer that, when composed with the em Adaptee , generates a behavior that is equivalent to the behavior of the em Target . One use of such an algorithm is to synthesize controllers that achieve similar goals on different hardware platforms. While this problem can be solved with existing synthesis algorithms, current state of the art tools fail to scale. To cope with the computational complexity of the problem, we introduce a special form of specification format, called em Separated GR( k ) , which can be solved with a scalable synthesis algorithm but still allows for a large set of realistic specifications. We solve the realizability and the synthesis problems for Separated GR( k ), and show how to exploit the separated nature of our specification to construct better algorithms, in terms of time complexity, than known algorithms for GR( k ) synthesis. We then describe a tool, called SGR( k ), that we have implemented based on the above approach and show, by experimental evaluation, how our tool outperforms current state of the art tools on various benchmarks and test cases.


AGMB Transformer Anatomy Guided Multi Branch Transformer Network for Automated Evaluation of Root Canal Therapy. Accurate evaluation of the treatment result on X ray images is a significant and challenging step in root canal therapy since the incorrect interpretation of the therapy results will hamper timely follow up which is crucial to the patients' treatment outcome. Nowadays, the evaluation is performed in a manual manner, which is time consuming, subjective, and error prone. In this paper, we aim to automate this process by leveraging the advances in computer vision and artificial intelligence, to provide an objective and accurate method for root canal therapy result assessment. A novel anatomy guided multi branch Transformer (AGMB Transformer) network is proposed, which first extracts a set of anatomy features and then uses them to guide a multi branch Transformer network for evaluation. Specifically, we design a polynomial curve fitting segmentation strategy with the help of landmark detection to extract the anatomy features. Moreover, a branch fusion module and a multi branch structure including our progressive Transformer and Group Multi Head Self Attention (GMHSA) are designed to focus on both global and local features for an accurate diagnosis. To facilitate the research, we have collected a large scale root canal therapy evaluation dataset with 245 root canal therapy X ray images, and the experiment results show that our AGMB Transformer can improve the diagnosis accuracy from 57.96 to 90.20 compared with the baseline network. The proposed AGMB Transformer can achieve a highly accurate evaluation of root canal therapy. To our best knowledge, our work is the first to perform automatic root canal therapy evaluation and has important clinical value to reduce the workload of endodontists.


The No Endmarker Theorem for One Way Probabilistic Pushdown Automata. In various models of one way pushdown automata, the explicit use of two designated endmarkers on a read once input tape has proven to be extremely useful for making a conscious, final decision on the acceptance rejection of each input word right after reading the right endmarker. With no endmarkers, by contrast, a machine must constantly stay in either accepting or rejecting states at any moment since it never notices the end of the input instance. This situation, however, helps us analyze the behavior of the machine whose tape head makes the consecutive moves on all prefixes of a given extremely long input word. Since those two machine formulations have their own advantages, it is natural to ask whether the endmarkers are truly necessary to correctly recognize languages. In the deterministic and nondeterministic models, it is well known that the endmarkers are removable without changing the acceptance criteria of each input instance. This paper proves that, for a more general model of one way probabilistic pushdown automata, the endmarkers are also removable. This is proven by employing probabilistic transformations from an "endmarker" machine to an equivalent "no endmarker" machine at the cost of double exponential state complexity without compromising its error probability. By setting this error probability appropriately, our proof also provides an alternative proof to both the deterministic and the nondeterministic models as well.


PoNet Pooling Network for Efficient Token Mixing in Long Sequences. Transformer based models have achieved great success in various NLP, vision, and speech tasks. However, the core of Transformer, the self attention mechanism, has a quadratic time and memory complexity with respect to the sequence length, which hinders applications of Transformer based models to long sequences. Many approaches have been proposed to mitigate this problem, such as sparse attention mechanisms, low rank matrix approximations and scalable kernels, and token mixing alternatives to self attention. We propose a novel Pooling Network (PoNet) for token mixing in long sequences with linear complexity. We design multi granularity pooling and pooling fusion to capture different levels of contextual information and combine their interactions with tokens. On the Long Range Arena benchmark, PoNet significantly outperforms Transformer and achieves competitive accuracy, while being only slightly slower than the fastest model, FNet, across all sequence lengths measured on GPUs. We also conduct systematic studies on the transfer learning capability of PoNet and observe that PoNet achieves 95.7 of the accuracy of BERT on the GLUE benchmark, outperforming FNet by 4.5 relative. Comprehensive ablation analysis demonstrates effectiveness of the designed multi granularity pooling and pooling fusion for token mixing in long sequences and efficacy of the designed pre training tasks for PoNet to learn transferable contextualized language representations.


Guiding Global Placement With Reinforcement Learning. Recent advances in GPU accelerated global and detail placement have reduced the time to solution by an order of magnitude. This advancement allows us to leverage data driven optimization (such as Reinforcement Learning) in an effort to improve the final quality of placement results. In this work we augment state of the art, force based global placement solvers with a reinforcement learning agent trained to improve the final detail placed Half Perimeter Wire Length (HPWL). We propose novel control schemes with either global or localized control of the placement process. We then train reinforcement learning agents to use these controls to guide placement to improved solutions. In both cases, the augmented optimizer finds improved placement solutions. Our trained agents achieve an average 1 improvement in final detail place HPWL across a range of academic benchmarks and more than 1 in global place HPWL on real industry designs.


Simultaneous orthogonalization of inner products in infinite dimensional vector spaces. For an arbitrary field mathbb K and a family of inner products in a mathbb K vector space V of arbitrary dimension, we study necessary and sufficient conditions in order to have an orthogonal basis relative to all the inner products. If the family contains a nondegenerate element plus a compatibility condition, then under mild hypotheses the simultaneous orthogonalization can be achieved. So we investigate several constructions whose purpose is to add a nondegenerate element to a degenerate family and we study under what conditions the enlarged family is nondegenerate.


Towards Interpretable and Reliable Reading Comprehension A Pipeline Model with Unanswerability Prediction. Multi hop QA with annotated supporting facts, which is the task of reading comprehension (RC) considering the interpretability of the answer, has been extensively studied. In this study, we define an interpretable reading comprehension (IRC) model as a pipeline model with the capability of predicting unanswerable queries. The IRC model justifies the answer prediction by establishing consistency between the predicted supporting facts and the actual rationale for interpretability. The IRC model detects unanswerable questions, instead of outputting the answer forcibly based on the insufficient information, to ensure the reliability of the answer. We also propose an end to end training method for the pipeline RC model. To evaluate the interpretability and the reliability, we conducted the experiments considering unanswerability in a multi hop question for a given passage. We show that our end to end trainable pipeline model outperformed a non interpretable model on our modified HotpotQA dataset. Experimental results also show that the IRC model achieves comparable results to the previous non interpretable models in spite of the trade off between prediction performance and interpretability.


Action Based Conversations Dataset A Corpus for Building More In Depth Task Oriented Dialogue Systems. Existing goal oriented dialogue datasets focus mainly on identifying slots and values. However, customer support interactions in reality often involve agents following multi step procedures derived from explicitly defined company policies as well. To study customer service dialogue systems in more realistic settings, we introduce the Action Based Conversations Dataset (ABCD), a fully labeled dataset with over 10K human to human dialogues containing 55 distinct user intents requiring unique sequences of actions constrained by policies to achieve task success. We propose two additional dialog tasks, Action State Tracking and Cascading Dialogue Success, and establish a series of baselines involving large scale, pre trained language models on this dataset. Empirical results demonstrate that while more sophisticated networks outperform simpler models, a considerable gap (50.8 absolute accuracy) still exists to reach human level performance on ABCD.


Plant Disease Detection Using Image Processing and Machine Learning. One of the important and tedious task in agricultural practices is the detection of the disease on crops. It requires huge time as well as skilled labor. This paper proposes a smart and efficient technique for detection of crop disease which uses computer vision and machine learning techniques. The proposed system is able to detect 20 different diseases of 5 common plants with 93 accuracy.


On the joint spectral radius of nonnegative matrices. We give an effective bound of the joint spectral radius rho( mathcal A) for a finite set mathcal A of nonnegative matrices For every n , sqrt n Bigl( frac V UD Bigr) D max C max i,j in C max A 1, dots,A n in mathcal A (A 1 dots A n) i,j le rho( mathcal A) le sqrt n D max C max i,j in C max A 1, dots,A n in mathcal A (A 1 dots A n) i,j , where D times D is the dimension of the matrices, U,V are respectively the largest entry and the smallest entry over all the positive entries of the matrices in mathcal A , and C is taken over all components in the dependency graph. The dependency graph is a directed graph where the vertices are the dimensions and there is an edge from i to j if and only if A i,j ne 0 for some matrix A in mathcal A . Furthermore, a bound on the norm is also given There exists a nonnegative integer r so that for every n , operatorname const n r rho( mathcal A) n le max A 1, dots,A n in mathcal A A 1 dots A n le operatorname const n r rho( mathcal A) n. Corollaries of the approach include a simple proof for the joint spectral theorem for finite sets of nonnegative matrices and the convergence rate of some sequences. The method in use is mostly based on Fekete's lemma.


Revising Berg Purcell for finite receptor kinetics. From nutrient uptake, to chemoreception, to synaptic transmission, many systems in cell biology depend on molecules diffusing and binding to membrane receptors. Mathematical analysis of such systems often neglects the fact that receptors process molecules at finite kinetic rates. A key example is the celebrated formula of Berg and Purcell for the rate that cell surface receptors capture extracellular molecules. Indeed, this influential result is only valid if receptors transport molecules through the cell wall at a rate much faster than molecules arrive at receptors. From a mathematical perspective, ignoring receptor kinetics is convenient because it makes the diffusing molecules independent. In contrast, including receptor kinetics introduces correlations between the diffusing molecules since, for example, bound receptors may be temporarily blocked from binding additional molecules. In this work, we present a modeling framework for coupling bulk diffusion to surface receptors with finite kinetic rates. The framework uses boundary homogenization to couple the diffusion equation to nonlinear ordinary differential equations on the boundary. We use this framework to derive an explicit formula for the cellular uptake rate and show that the analysis of Berg and Purcell significantly overestimates uptake in some typical biophysical scenarios. We confirm our analysis by numerical simulations of a many particle stochastic system.


GraphSense A General Purpose Cryptoasset Analytics Platform. There is currently an increasing demand for cryptoasset analysis tools among cryptoasset service providers, the financial industry in general, as well as across academic fields. At the moment, one can choose between commercial services or low level open source tools providing programmatic access. In this paper, we present the design and implementation of another option the GraphSense Cryptoasset Analytics Platform, which can be used for interactive investigations of monetary flows and, more importantly, for executing advanced analytics tasks using a standard data science tool stack. By providing a growing set of open source components, GraphSense could ultimately become an instrument for scientific investigations in academia and a possible response to emerging compliance and regulation challenges for businesses and organizations dealing with cryptoassets.


The Reidemeister spectrum of split metacyclic groups. Given a group G and an automorphism varphi of G , two elements x, y in G are said to be varphi conjugate if x gy varphi(g) 1 for some g in G . The number of equivalence classes for this relation is the Reidemeister number R( varphi) of varphi . The set R( psi) mid psi in mathrm Aut (G) is called the Reidemeister spectrum of G . We fully determine the Reidemeister spectrum of split metacyclic groups of the form C n rtimes C p where p is a prime and the action is non trivial.


QMugs Quantum Mechanical Properties of Drug like Molecules. Machine learning approaches in drug discovery, as well as in other areas of the chemical sciences, benefit from curated datasets of physical molecular properties. However, there is a lack of sufficiently large data collections that include first principle quantum chemical information on bioactive molecules, such as single point electronic properties, quantum mechanical wave functions and density functional theory (DFT) matrices. The open access QMugs (Quantum Mechanical Properties of Drug like Molecules) dataset fills this void. The QMugs collection comprises quantum mechanical properties of more than 665k biologically and pharmacologically relevant molecules extracted from the ChEMBL database, totaling sim 2M conformers. QMugs contains optimized molecular geometries and thermodynamic data obtained via the semi empirical method GFN2 xTB. Atomic and molecular properties (e.g., partial charges, energies, and rotational constants) are provided on both the GFN2 xTB and on the DFT ( omega B97X D def2 SVP) levels of theory. QMugs also comprises the respective quantum mechanical wave functions, including DFT density and orbital matrices, totaling over 7 terabytes of uncompressed data. This dataset is intended to facilitate the development of models that learn from molecular data on different levels of theory while also providing insight into the corresponding relationships between molecular structure and biological activity.


On the Bergman kernel in weighted monogenic Bargmann Fock spaces. In this paper, we study the Bergman kernel B varphi(x, bar y) of generalized Bargmann Fock spaces in the setting of Clifford algebra. The versions of L 2 estimate method and weighted subharmonic inequality for Clifford algebra are established. Consequently we show the existence of B varphi(x, bar y) and then give some estimates on and off the diagonal.


Poly Cauchy numbers the combinatorics behind. We introduce poly Cauchy permutations that are enumerated by the poly Cauchy numbers. We provide combinatorial proofs for several identities involving poly Cauchy numbers and some of their generalizations. The aim of this work is to demonstrate the power and beauty of the elementary combinatorial approach.


Emergence of synchronization in Kuramoto model with general digraph. In this paper, we study the complete synchronization of the Kuramoto model with general network containing a spanning tree, when the initial phases are distributed in an open half circle. As lack of uniform coercivity in general digraph, in order to capture the dissipation structure on a general network, we apply the node decomposition criteria in cite H L Z20 to yield a hierarchical structure, which leads to the hypo coercivity. This drives the phase diameter into a small region after finite time in a large coupling regime, and the uniform boundedness of the diameter eventually leads to the emergence of exponentially fast synchronization.


Greenberg's force prediction for vertical axis wind turbine blades. We present a method to adapt Greenberg's potential flow model for coupled pitching and surging flow such that it can be applied to predict the loads on a vertical axis wind turbine blade. The model is extended to compute loads on a blade undergoing multi harmonic oscillations in effective angle of attack and incoming flow velocity by formulating the blade kinematics as a sum of simple harmonic motions. Each of these functions is a multiple of the main turbine rotational frequency, associated with an individual amplitude, as suggested by Greenberg. The results of the adapted model are compared with experimental data from a scaled down model of a single bladed H type Darrieus wind turbine. The comparison between the predictions by the Greenberg model and experimentally obtained phase averaged radial force evolutions show that the inviscid Greenberg model predicts well the loads at the start of the upwind portion and the maximum loads during upwind, but fails during the downwind portion when flow separation occurs. The proposed application of Greenberg's model to vertical axis wind turbine kinematics shows a great potential to diagnose regions of separated flow and for quantifying the relative influences of dynamic stall and intrinsic turbine kinematics on the blade loading. Future research can readily extend this method to any airfoil undergoing an arbitrary combination of pitching, surging, and heaving, following a kinematic profile that can be approximated by a Fourier series.


DeepCC Bridging the Gap Between Congestion Control and Applications via Multi Objective Optimization. The increasingly complicated and diverse applications have distinct network performance demands, e.g., some desire high throughput while others require low latency. Traditional congestion controls (CC) have no perception of these demands. Consequently, literatures have explored the objective specific algorithms, which are based on either offline training or online learning, to adapt to certain application demands. However, once generated, such algorithms are tailored to a specific performance objective function. Newly emerged performance demands in a changeable network environment require either expensive retraining (in the case of offline training), or manually redesigning a new objective function (in the case of online learning). To address this problem, we propose a novel architecture, DeepCC. It generates a CC agent that is generically applicable to a wide range of application requirements and network conditions. The key idea of DeepCC is to leverage both offline deep reinforcement learning and online fine tuning. In the offline phase, instead of training towards a specific objective function, DeepCC trains its deep neural network model using multi objective optimization. With the trained model, DeepCC offers near Pareto optimal policies w.r.t different user specified trade offs between throughput, delay, and loss rate without any redesigning or retraining. In addition, a quick online fine tuning phase further helps DeepCC achieve the application specific demands under dynamic network conditions. The simulation and real world experiments show that DeepCC outperforms state of the art schemes in a wide range of settings. DeepCC gains a higher target completion ratio of application requirements up to 67.4 than that of other schemes, even in an untrained environment.


Additive Link Metrics Identification Proof of Selected Lemmas and Propositions. This is a technical report, containing all the lemma and proposition proofs in paper "Topological Constraints on Identifying Additive Link Metrics via End to end Paths Measurements" by Liang Ma, Ting He, Kin K. Leung, Don Towsley, and Ananthram Swami, published in Annual Conference of The International Technology Alliance (ACITA), 2012.


Matrix forms of immanant inequalities. We translate inequalities and conjectures for immanants and generalized matrix functions into inequalities in the L "owner order. These have the form of trace polynomials and generalize the inequalities from FH, J. Math. Phys. 62 (2021), 2, 022203 .


On the Design of Strategic Task Recommendations for Sustainable Crowdsourcing Based Content Moderation. Crowdsourcing based content moderation is a platform that hosts content moderation tasks for crowd workers to review user submissions (e.g. text, images and videos) and make decisions regarding the admissibility of the posted content, along with a gamut of other tasks such as image labeling and speech to text conversion. In an attempt to reduce cognitive overload at the workers and improve system efficiency, these platforms offer personalized task recommendations according to the worker's preferences. However, the current state of the art recommendation systems disregard the effects on worker's mental health, especially when they are repeatedly exposed to content moderation tasks with extreme content (e.g. violent images, hate speech). In this paper, we propose a novel, strategic recommendation system for the crowdsourcing platform that recommends jobs based on worker's mental status. Specifically, this paper models interaction between the crowdsourcing platform's recommendation system (leader) and the worker (follower) as a Bayesian Stackelberg game where the type of the follower corresponds to the worker's cognitive atrophy rate and task preferences. We discuss how rewards and costs should be designed to steer the game towards desired outcomes in terms of maximizing the platform's productivity, while simultaneously improving the working conditions of crowd workers.


Hallmarks of Human Machine Collaboration A framework for assessment in the DARPA Communicating with Computers Program. There is a growing desire to create computer systems that can communicate effectively to collaborate with humans on complex, open ended activities. Assessing these systems presents significant challenges. We describe a framework for evaluating systems engaged in open ended complex scenarios where evaluators do not have the luxury of comparing performance to a single right answer. This framework has been used to evaluate human machine creative collaborations across story and music generation, interactive block building, and exploration of molecular mechanisms in cancer. These activities are fundamentally different from the more constrained tasks performed by most contemporary personal assistants as they are generally open ended, with no single correct solution, and often no obvious completion criteria. We identified the Key Properties that must be exhibited by successful systems. From there we identified "Hallmarks" of success capabilities and features that evaluators can observe that would be indicative of progress toward achieving a Key Property. In addition to being a framework for assessment, the Key Properties and Hallmarks are intended to serve as goals in guiding research direction.


Masked Autoencoders Are Scalable Vision Learners. This paper shows that masked autoencoders (MAE) are scalable self supervised learners for computer vision. Our MAE approach is simple we mask random patches of the input image and reconstruct the missing pixels. It is based on two core designs. First, we develop an asymmetric encoder decoder architecture, with an encoder that operates only on the visible subset of patches (without mask tokens), along with a lightweight decoder that reconstructs the original image from the latent representation and mask tokens. Second, we find that masking a high proportion of the input image, e.g., 75 , yields a nontrivial and meaningful self supervisory task. Coupling these two designs enables us to train large models efficiently and effectively we accelerate training (by 3x or more) and improve accuracy. Our scalable approach allows for learning high capacity models that generalize well e.g., a vanilla ViT Huge model achieves the best accuracy (87.8 ) among methods that use only ImageNet 1K data. Transfer performance in downstream tasks outperforms supervised pre training and shows promising scaling behavior.


Phase memory for optical vortex beams. Optical vortex beams have been under considerable attention recently due to their demonstrated potential for applications ranging from optical communication to particle trapping. Practical problems related to the dependence between their phase structure and the physical size have been addressed by introducing the concept of perfect optical vortex beams. Propagation of these structured beams through different levels of disturbances is critical for their uses. For the first time, we examine quantitatively the degradation of perfect optical vortex beams after their interaction with localized random media. We developed an analytical model that describes how the spatial correlation length and the phase variance of the disturbance affect the phase distribution across the vortex beams. This allows to ascertain the regimes of randomness where the beams maintain the memory of their initial vorticity. Systematic numerical simulations and controlled experiments demonstrate the extent of this memory effect for beams with different vorticity indices.


OmiEmbed a unified multi task deep learning framework for multi omics data. High dimensional omics data contains intrinsic biomedical information that is crucial for personalised medicine. Nevertheless, it is challenging to capture them from the genome wide data due to the large number of molecular features and small number of available samples, which is also called 'the curse of dimensionality' in machine learning. To tackle this problem and pave the way for machine learning aided precision medicine, we proposed a unified multi task deep learning framework named OmiEmbed to capture biomedical information from high dimensional omics data with the deep embedding and downstream task modules. The deep embedding module learnt an omics embedding that mapped multiple omics data types into a latent space with lower dimensionality. Based on the new representation of multi omics data, different downstream task modules were trained simultaneously and efficiently with the multi task strategy to predict the comprehensive phenotype profile of each sample. OmiEmbed support multiple tasks for omics data including dimensionality reduction, tumour type classification, multi omics integration, demographic and clinical feature reconstruction, and survival prediction. The framework outperformed other methods on all three types of downstream tasks and achieved better performance with the multi task strategy comparing to training them individually. OmiEmbed is a powerful and unified framework that can be widely adapted to various application of high dimensional omics data and has a great potential to facilitate more accurate and personalised clinical decision making.


On the Safety Implications of Misordered Events and Commands in IoT Systems. IoT devices, equipped with embedded actuators and sensors, provide custom automation in the form of IoT apps. IoT apps subscribe to events and upon receipt, transmit actuation commands which trigger a set of actuators. Events and actuation commands follow paths in the IoT ecosystem such as sensor to edge, edge to cloud, and cloud to actuator, with different network and processing delays between these connections. Significant delays may occur especially when an IoT system cloud interacts with other clouds. Due to this variation in delays, the cloud may receive events in an incorrect order, and in turn, devices may receive and actuate misordered commands. In this paper, we first study eight major IoT platforms and show that they do not make strong guarantees on event orderings to address these issues. We then analyze the end to end interactions among IoT components, from the creation of an event to the invocation of a command. From this, we identify and formalize the root causes of misorderings in events and commands leading to undesired states. We deploy 23 apps in a simulated smart home containing 35 IoT devices to evaluate the misordering problem. Our experiments demonstrate a high number of misordered events and commands that occur through different interaction paths. Through this effort, we reveal the root and extent of the misordering problem and guide future work to ensure correct ordering in IoT systems.


Infinitely many solutions for Schr " o dinger Newton equations. We prove the existence of infinitely many non radial positive solutions for the Schr " o dinger Newton system left begin array ll Delta u V( x )u Psi u 0, x in mathbb R 3, newline Delta Psi frac12 u 2 0, x in mathbb R 3, end array right. provided that V(r) has the following behavior at infinity V(r) V 0 frac a r m O left( frac 1 r m theta right) quad mbox as r rightarrow infty, where frac12 le m 1 and a, V 0, theta are some positive constants. In particular, for any s large we use a reduction method to construct s bump solutions lying on a circle of radius r sim (s log s) frac 1 1 m .


Batch Curation for Unsupervised Contrastive Representation Learning. The state of the art unsupervised contrastive visual representation learning methods that have emerged recently (SimCLR, MoCo, SwAV) all make use of data augmentations in order to construct a pretext task of instant discrimination consisting of similar and dissimilar pairs of images. Similar pairs are constructed by randomly extracting patches from the same image and applying several other transformations such as color jittering or blurring, while transformed patches from different image instances in a given batch are regarded as dissimilar pairs. We argue that this approach can result similar pairs that are textit semantically dissimilar. In this work, we address this problem by introducing a textit batch curation scheme that selects batches during the training process that are more inline with the underlying contrastive objective. We provide insights into what constitutes beneficial similar and dissimilar pairs as well as validate textit batch curation on CIFAR10 by integrating it in the SimCLR model.


Benefits, Challenges and Contributors to Success for National eHealth Systems Implementation A Scoping Review. Our scoping review aims to assess what legal, ethical, and socio technical factors contribute or inhibit the success of national eHealth system implementations. In addition, our review seeks to describe the characteristics and benefits of eHealth systems. We conducted a scoping review of literature published in English between January 2000 and 2020 using a keyword search on five databases PubMed, Scopus, Web of Science, IEEEXplore, and ProQuest. After removal of duplicates, abstract screening and full text filtering, 86 articles were included from 8276 search results. We identified 17 stakeholder groups, 6 eHealth Systems areas, and 15 types of legal regimes and standards. In depth textual analysis revealed challenges mainly in implementation, followed by ethico legal and data related aspects. Key factors influencing success include promoting trust of the system, ensuring wider acceptance amongst users, reconciling the system with legal requirements and ensuring an adaptable technical platform. Results revealed support for decentralised implementations because they carry less implementation and engagement challenges than centralised ones. Simultaneously, due to decentralised systems interoperability issues, federated implementations (with a set of national standards) might be preferable. This study identifies the primary socio technical, legal and ethical factors that challenge and contribute to the success of eHealth system implementations. This study also describes the complexities and characteristics of existing eHealth implementation programs, and surmises suggested guidance for resolving the identified challenges.


Impact of locus coeruleus and its projections on memory and aging. Introduction Locus coeruleus (LC) is the primary source of norepinephrine to the brain and its efferent projections innervate many brain regions, including the thalamus. LC degrades with normal aging, but not much is known regarding whether its structural connectivity evolves with age or predicts aspects of cognition. Methods Here, we use high resolution diffusion tensor imaging (DTI) based tractography to examine structural connectivity between LC and the thalamus in younger and older adults. Results We found LC projections to be bundled in a fiber tract anatomically consistent with the central tegmental tract (CTT) and branch from this tract into the thalamus. The older cohort exhibited a significant reduction in mean and radial diffusivity within CTT as compared to the young cohort. We also observed a significant correlation between CTT mean, axial, and radial diffusivities and memory performance (delayed recall) in the older adult cohort. Discussion These observations suggest that although LC projections degrade with age, the degree of degradation is associated with cognitive abilities in older adults.


A probabilistic approach to the Erd "os Kac theorem for additive functions. We present a new perspective of assessing the rates of convergence to the Gaussian and Poisson distributions in the Erd "os Kac theorem for additive arithmetic functions psi of a random integer J n uniformly distributed over 1,...,n . Our approach is probabilistic, working directly on spaces of random variables without any use of Fourier analytic methods, and our psi is more general than those considered in the literature. Our main results are (i) bounds on the Kolmogorov distance and Wasserstein distance between the distribution of the normalized psi(J n) and the standard Gaussian distribution, and (ii) bounds on the Kolmogorov distance and total variation distance between the distribution of psi(J n) and a Poisson distribution under mild additional assumptions on psi . Our results generalize the existing ones in the literature.


MonoRUn Monocular 3D Object Detection by Reconstruction and Uncertainty Propagation. Object localization in 3D space is a challenging aspect in monocular 3D object detection. Recent advances in 6DoF pose estimation have shown that predicting dense 2D 3D correspondence maps between image and object 3D model and then estimating object pose via Perspective n Point (PnP) algorithm can achieve remarkable localization accuracy. Yet these methods rely on training with ground truth of object geometry, which is difficult to acquire in real outdoor scenes. To address this issue, we propose MonoRUn, a novel detection framework that learns dense correspondences and geometry in a self supervised manner, with simple 3D bounding box annotations. To regress the pixel related 3D object coordinates, we employ a regional reconstruction network with uncertainty awareness. For self supervised training, the predicted 3D coordinates are projected back to the image plane. A Robust KL loss is proposed to minimize the uncertainty weighted reprojection error. During testing phase, we exploit the network uncertainty by propagating it through all downstream modules. More specifically, the uncertainty driven PnP algorithm is leveraged to estimate object pose and its covariance. Extensive experiments demonstrate that our proposed approach outperforms current state of the art methods on KITTI benchmark.