CodeXGLUE A Machine Learning Benchmark Dataset for Code Understanding and Generation. Benchmark datasets have a significant impact on accelerating research in programming language tasks. In this paper, we introduce CodeXGLUE, a benchmark dataset to foster machine learning research for program understanding and generation. CodeXGLUE includes a collection of 10 tasks across 14 datasets and a platform for model evaluation and comparison. CodeXGLUE also features three baseline systems, including the BERT style, GPT style, and Encoder Decoder models, to make it easy for researchers to use the platform. The availability of such data and baselines can help the development and validation of new methods that can be applied to various program understanding and generation problems.


Unsupervised Law Article Mining based on Deep Pre Trained Language Representation Models with Application to the Italian Civil Code. Modeling law search and retrieval as prediction problems has recently emerged as a predominant approach in law intelligence. Focusing on the law article retrieval task, we present a deep learning framework named LamBERTa, which is designed for civil law codes, and specifically trained on the Italian civil code. To our knowledge, this is the first study proposing an advanced approach to law article prediction for the Italian legal system based on a BERT (Bidirectional Encoder Representations from Transformers) learning framework, which has recently attracted increased attention among deep learning approaches, showing outstanding effectiveness in several natural language processing and learning tasks. We define LamBERTa models by fine tuning an Italian pre trained BERT on the Italian civil code or its portions, for law article retrieval as a classification task. One key aspect of our LamBERTa framework is that we conceived it to address an extreme classification scenario, which is characterized by a high number of classes, the few shot learning problem, and the lack of test query benchmarks for Italian legal prediction tasks. To solve such issues, we define different methods for the unsupervised labeling of the law articles, which can in principle be applied to any law article code system. We provide insights into the explainability and interpretability of our LamBERTa models, and we present an extensive experimental analysis over query sets of different type, for single label as well as multi label evaluation tasks. Empirical evidence has shown the effectiveness of LamBERTa, and also its superiority against widely used deep learning text classifiers and a few shot learner conceived for an attribute aware prediction task.


Multigraded Sylvester forms, Duality and Elimination Matrices. In this paper we study the equations of the elimination ideal associated with n 1 generic multihomogeneous polynomials defined over a product of projective spaces of dimension n . We first prove a duality property and then make this duality explicit by introducing multigraded Sylvester forms. These results provide a partial generalization of similar properties that are known in the setting of homogeneous polynomial systems defined over a single projective space. As an important consequence, we derive a new family of elimination matrices that can be used for solving zero dimensional multiprojective polynomial systems by means of linear algebra methods.


Adaptive Probabilistic Model for Energy Efficient Distance based Clustering in WSNs (Adapt P) A LEACH based Analytical Study. Network lifetime and energy consumption of data transmission have been primary Quality of Service (QoS) obligations in Wireless Sensor Networks (WSNs). The environment of a WSN is often organized into clusters to mitigate the management complexity of such obligations. However, the distance between Sensor Nodes (SNs) and the number of clusters per round are vital factors that affect QoS performance of a WSN. A designer's conundrum resolves around the desire to sustain a balance between the limited residual energy of SNs and the demand for prolonged network lifetime. Any imbalance in controlling such objectives results in either QoS penalties due to draining SN energies, or an over cost environment that is significantly difficult to distribute and operate. Low Energy Adaptive Clustering Hierarchy (LEACH) is a distributed algorithm proposed to tackle such difficulties. Proposed LEACH based algorithms focus on residual energies of SNs to compute a probability function that selects cluster heads and an optimal energy efficient path toward a destination SN. Nevertheless, these algorithms do not consider variations in network's state at run time. Such a state changes in an adaptive manner according to existing network structures and conditions. Thus, cluster heads per round are not elected adaptively depending on the state and distances between SNs. This paper proposes an energy efficient adaptive distance based clustering called Adapt P, in which an adaptive probability function is developed to formulate clusters. A near optimal distance between each cluster head and its cluster members is formulated so that energy consumption of the network is mitigated and network lifetime is maximized. The cluster head selection probability is adapted at the end of each round based on the maximum number of cluster heads permitted per round found a priori and the number of alive SNs in the network.


Feature Generation and Hypothesis Verification for Reliable Face Anti Spoofing. Although existing face anti spoofing (FAS) methods achieve high accuracy in intra domain experiments, their effects drop severely in cross domain scenarios because of poor generalization. Recently, multifarious techniques have been explored, such as domain generalization and representation disentanglement. However, the improvement is still limited by two issues 1) It is difficult to perfectly map all faces to a shared feature space. If faces from unknown domains are not mapped to the known region in the shared feature space, accidentally inaccurate predictions will be obtained. 2) It is hard to completely consider various spoof traces for disentanglement. In this paper, we propose a Feature Generation and Hypothesis Verification framework to alleviate the two issues. Above all, feature generation networks which generate hypotheses of real faces and known attacks are introduced for the first time in the FAS task. Subsequently, two hypothesis verification modules are applied to judge whether the input face comes from the real face space and the real face distribution respectively. Furthermore, some analyses of the relationship between our framework and Bayesian uncertainty estimation are given, which provides theoretical support for reliable defense in unknown domains. Experimental results show our framework achieves promising results and outperforms the state of the art approaches on extensive public datasets.


A note on Dirichlet spectrum. We prove a result related to Dirichlet spectrum for simultaneous approximation to two real numbers in Euclidean norm and badly or very well approximability.


Time Resolved Vibrational Spectroscopy to Measure Lifetime of the E 3 Pi g(v 3) state of Molecular Iodine. The lifetime of the E 3 Pi g(v 3) state of molecular iodine was measured in the gas phase at room temperature. The E 3 Pi g state was selectively populated by two sequential nanosecond pulse laser excitation. Resolved molecular fluorescence for the B 3 Pi u leftarrow E 3 Pi g was analyzed and the lifetime of the E(v 3) state, measured using a time correlated single photon counting technique, is to be tau 21 (2) ns.


Embedding dimensions of simplicial complexes on few vertices. We provide a simple characterization of simplicial complexes on few vertices that embed into the d sphere. Namely, a simplicial complex on d 3 vertices embeds into the d sphere if and only if its non faces do not form an intersecting family. As immediate consequences, we recover the classical van Kampen Flores theorem and provide a topological extension of the Erd H os Ko Rado theorem. By analogy with F 'ary's theorem for planar graphs, we show in addition that such complexes satisfy the rigidity property that continuous and linear embeddability are equivalent.


Studying Test Driven Development and its Retainment Over a Six month Time Span. In this paper, we investigate the effect of TDD, as compared to a non TDD approach, as well as its retainment (or retention) over a time span of (about) six months. To pursue these objectives, we conducted a (quantitative) longitudinal cohort study with 30 novice developers (i.e., third year undergraduate students in Computer Science). We observed that TDD affects neither the external quality of software products nor developers' productivity. However, we observed that the participants applying TDD produced significantly more tests, with a higher fault detection capability than those using a non TDD approach. As for the retainment of TDD, we found that TDD is retained by novice developers for at least six months.


The scaling limit of fair Peano curves. We study random Peano curves on planar square grids that arise from fair random spanning trees. These are trees that are sampled in such a way as to have the same (if possible) edge probabilities. In particular, we are interested in identifying the scaling limit as the mesh size of the grid tends to zero. It is known cite lawler schramm werner2002 that if the trees are sampled uniformly, then the scaling limit exists and equals rm SLE 8 . We show that if we simply follow the same steps as in cite lawler schramm werner2002 , then fair Peano curves have a deterministic scaling limit.


Analysing ocular parameters for web browsing and graph visualization. This paper proposes a set of techniques to investigate eye gaze and fixation patterns while users interact with electronic user interfaces. In particular, two case studies are presented one on analysing eye gaze while interacting with deceptive materials in web pages and another on analysing graphs in standard computer monitor and virtual reality displays. We analysed spatial and temporal distributions of eye gaze fixations and sequence of eye gaze movements. We used this information to propose new design guidelines to avoid deceptive materials in web and user friendly representation of data in 2D graphs. In 2D graph study we identified that area graph has lowest number of clusters for user's gaze fixations and lowest average response time. The results of 2D graph study were implemented in virtual and mixed reality environment. Along with this, it was ob served that the duration while interacting with deceptive materials in web pages is independent of the number of fixations. Furthermore, web based data visualization tool for analysing eye tracking data from single and multiple users was developed.


BV and Sobolev homeomorphisms between metric measure spaces and the plane. We show that given a homeomorphism f G rightarrow Omega where G is a open subset of mathbb R 2 and Omega is a open subset of a 2 Ahlfors regular metric measure space supporting a weak (1,1) Poincar 'e inequality, it holds f in BV operatorname loc (G, Omega) if and only f 1 in BV operatorname loc ( Omega,G) . Further if f satisfies the Luzin N and N 1 conditions then f in W 1,1 operatorname loc (G, Omega) if and only if f 1 in W 1,1 operatorname loc ( Omega,G) .


Public Data Assisted Mirror Descent for Private Model Training. In this paper, we revisit the problem of using in distribution public data to improve the privacy utility trade offs for differentially private (DP) model training. (Here, public data refers to auxiliary data sets that have no privacy concerns.) We design a natural variant of DP mirror descent, where the DP gradients of the private sensitive data act as the linear term, and the loss generated by the public data as the mirror map. We show that, for linear regression with feature vectors drawn from a non isotropic sub Gaussian distribution, our algorithm, PDA DPMD (a variant of mirror descent), provides population risk guarantees that are asymptotically better than the best known guarantees under DP (without having access to public data), when the number of public data samples ( n sf pub ) is sufficiently large. We further show that our algorithm has natural "noise stability" properties that control the variance due to noise added to ensure DP. We demonstrate the efficacy of our algorithm by showing privacy utility trade offs on four benchmark datasets (StackOverflow, WikiText 2, CIFAR 10, and EMNIST). We show that our algorithm not only significantly improves over traditional DP SGD, which does not have access to public data, but to our knowledge is the first to improve over DP SGD on models that have been pre trained with public data.


Adaptive Streaming of 360 Videos with Perfect, Imperfect, and Unknown FoV Viewing Probabilities in Wireless Networks. This paper investigates adaptive streaming of one or multiple tiled 360 videos from a multi antenna base station (BS) to one or multiple single antenna users, respectively, in a multi carrier wireless system. We aim to maximize the video quality while keeping rebuffering time small via encoding rate adaptation at each group of pictures (GOP) and transmission adaptation at each (transmission) slot. To capture the impact of field of view (FoV) prediction, we consider three cases of FoV viewing probability distributions, i.e., perfect, imperfect, and unknown FoV viewing probability distributions, and use the average total utility, worst average total utility, and worst total utility as the respective performance metrics. In the single user scenario, we optimize the encoding rates of the tiles, encoding rates of the FoVs, and transmission beamforming vectors for all subcarriers to maximize the total utility in each case. In the multi user scenario, we adopt rate splitting with successive decoding and optimize the encoding rates of the tiles, encoding rates of the FoVs, rates of the common and private messages, and transmission beamforming vectors for all subcarriers to maximize the total utility in each case. Then, we separate the challenging optimization problem into multiple tractable problems in each scenario. In the single user scenario, we obtain a globally optimal solution of each problem using transformation techniques and the Karush Kuhn Tucker (KKT) conditions. In the multi user scenario, we obtain a KKT point of each problem using the concave convex procedure (CCCP). Finally, numerical results demonstrate that the proposed solutions achieve notable gains over existing schemes in all three cases. To the best of our knowledge, this is the first work revealing the impact of FoV prediction on the performance of adaptive streaming of tiled 360 videos.


A Probabilistic Framework for Lexicon based Keyword Spotting in Handwritten Text Images. Query by String Keyword Spotting (KWS) is here considered as a key technology for indexing large collections of handwritten text images to allow fast textual access to the contents of these collections. Under this perspective, a probabilistic framework for lexicon based KWS in text images is presented. The presentation aims at providing a tutorial view that helps to understand the relations between classical statements of KWS and the relative challenges entailed by these statements. More specifically, the development of the proposed framework makes it self evident that word recognition or classification implicitly or explicitly underlies any formulation of KWS. Moreover, it clearly suggests that the same statistical models and training methods successfully used for handwriting text recognition can advantageously be used also for KWS, even though KWS does not generally require or rely on any kind of previously produced image transcripts. These ideas are developed into a specific, probabilistically sound approach for segmentation free, lexicon based, query by string KWS. Experiments carried out using this approach are presented, which support the consistency and general interest of the proposed framework. Several datasets, traditionally used for KWS benchmarking are considered, with results significantly better than those previously published for these datasets. In addition, results on two new, larger handwritten text image datasets are reported, showing the great potential of the methods proposed in this paper for indexing and textual search in large collections of handwritten documents.


Studying Ad Library Integration Strategies of Top Free to Download Apps. In app advertisements have become a major revenue source for app developers in the mobile app ecosystem. Ad libraries play an integral part in this ecosystem as app developers integrate these libraries into their apps to display ads. In this paper, we study ad library integration practices by analyzing 35,459 updates of 1,837 top free to download apps of the Google Play Store. We observe that ad libraries (e.g., Google AdMob) are not always used for serving ads 22.5 of the apps that integrate Google AdMob do not display ads. They instead depend on Google AdMob for analytical purposes. Among the apps that display ads, we observe that 57.9 of them integrate multiple ad libraries. We observe that such integration of multiple ad libraries occurs commonly in apps with a large number of downloads and ones in app categories with a high proportion of ad displaying apps. We manually analyze a sample of apps and derive a set of rules to automatically identify four common strategies for integrating multiple ad libraries. Our analysis of the apps across the identified strategies shows that app developers prefer to manage their own integrations instead of using off the shelf features of ad libraries for integrating multiple ad libraries. Our findings are valuable for ad library developers who wish to learn first hand about the challenges of integrating ad libraries.


Covariant theory of light in a dispersive medium. The relativistic theory of the time and position dependent energy and momentum densities of light in glasses and other low loss dispersive media, where different wavelengths of light propagate at different phase velocities, has remained a largely unsolved challenge until now. This is astonishing in view of the excellent theoretical understanding of Maxwell's equations and the abundant experimental measurements of optical phenomena in dispersive media. The challenge is related to the complexity of the interference patterns of partial waves and to the coupling of the field and medium dynamics by the optical force. In this work, we use the mass polariton theory of light Phys. Rev. A 96, 063834 (2017) to derive the stress energy momentum (SEM) tensors of the field and the dispersive medium. Our starting point, the fundamental local conservation laws of energy and momentum, is close to that of a recent theoretical work on light in dispersive media by Philbin Phys. Rev. A 83, 013823 (2011) , which however, excludes the power conversion and force density source terms describing the coupling between the field and the medium. In the general inertial frame, we present the SEM tensors in terms of Lorentz scalars, four vectors, and field tensors that reflect in a transparent way the Lorentz covariance of the theory. The SEM tensors of the field and the medium are symmetric, form invariant, and in full accordance with the covariance principle of the special theory of relativity. The SEM tensor of the coupled field medium state of light also has zero four divergence. Therefore, light in a dispersive medium has well defined four momentum and rest frame. The volume integrals of the total energy and momentum densities of light agree with the model of mass polariton quasiparticles having a nonzero rest mass. The predictions of our work are directly accessible to experiments.


A linear Galerkin numerical method for a quasilinear subdiffusion equation. We couple the L1 discretization for Caputo derivative in time with spectral Galerkin method in space to devise a scheme that solves quasilinear subdiffusion equations. Both the diffusivity and the source are allowed to be nonlinear functions of the solution. We prove method's stability and convergence with spectral accuracy in space. The temporal order depends on solution's regularity in time. Further, we support our results with numerical simulations that utilize parallelism for spatial discretization. Moreover, as a side result we find asymptotic exact values of error constants along with their remainders for discretizations of Caputo derivative and fractional integrals. These constants are the smallest possible which improves the previously established results from the literature.


Another criterion for solvability of finite groups. Let G be a finite group. Denote by psi(G) the sum psi(G) sum x in G x where x denotes the order of the element x, and by o(G) the quotient o(G) frac psi(G) G . Confirming a conjecture posed by E.I. Khukhro, A. Moreto and M. Zarrin, we prove that if o(G) o(A 5), then G is solvable.


EDSC An Event Driven Smart Contract Platform. This paper presents EDSC, a novel smart contract platform design based on the event driven execution model as opposed to the traditionally employed transaction driven execution model. We reason that such a design is a better fit for many emerging smart contract applications and is better positioned to address the scalability and performance challenges plaguing the smart contract ecosystem. We propose EDSC's design under the Ethereum framework, and the design can be easily adapted for other existing smart contract platforms. We have conducted implementation using Ethereum client and experiments where performance modeling results show on average 2.2 to 4.6 times reduced total latency of event triggered smart contracts, which demonstrates its effectiveness for supporting contracts that demand timely execution based on events. In addition, we discuss example use cases to demonstrate the design's utility and comment on its potential security dynamics.


Stochastic Maximum Principle for Optimal Liquidation with Control dependent Terminal Time. In this paper we study a general optimal liquidation problem with a control dependent stopping time which is the first time the stock holding becomes zero or a fixed terminal time, whichever comes first. We prove a stochastic maximum principle (SMP) which is markedly different in its Hamiltonian condition from that of the standard SMP with fixed terminal time. We present a simple example in which the optimal solution satisfies the SMP in this paper but fails the standard SMP in the literature.


Verifying Controllers with Convolutional Neural Network based Perception A Case for Intelligible, Safe, and Precise Abstractions. Convolutional Neural Networks (CNN) for object detection, lane detection, and segmentation now sit at the head of most autonomy pipelines, and yet, their safety analysis remains an important challenge. Formal analysis of perception models is fundamentally difficult because their correctness is hard if not impossible to specify. We present a technique for inferring intelligible and safe abstractions for perception models from system level safety requirements, data, and program analysis of the modules that are downstream from perception. The technique can help tradeoff safety, size, and precision, in creating abstractions and the subsequent verification. We apply the method to two significant case studies based on high fidelity simulations (a) a vision based lane keeping controller for an autonomous vehicle and (b) a controller for an agricultural robot. We show how the generated abstractions can be composed with the downstream modules and then the resulting abstract system can be verified using program analysis tools like CBMC. Detailed evaluations of the impacts of size, safety requirements, and the environmental parameters (e.g., lighting, road surface, plant type) on the precision of the generated abstractions suggest that the approach can help guide the search for corner cases and safe operating envelops.


Analytical estimation of maximum fraction of infected individuals with one shot non pharmaceutical intervention in a hybrid epidemic model. Facing a global epidemic of new infectious diseases such as COVID 19, non pharmaceutical interventions (NPIs), which reduce transmission rates without medical actions, are being implemented around the world to mitigate spreads. One of the problems in assessing the effects of NPIs is that different NPIs have been implemented at different times based on the situation of each country therefore, few assumptions can be shared about how the introduction of policies affects the patient population. Mathematical models can contribute to further understanding these phenomena by obtaining analytical solutions as well as numerical simulations. In this study, an NPI was introduced into the SIR model for a conceptual study of infectious diseases under the condition that the transmission rate was reduced to a fixed value only once within a finite time duration, and its effect was analyzed numerically and theoretically. It was analytically shown that the maximum fraction of infected individuals and the final size could be larger if the intervention starts too early. The analytical results also suggested that more individuals may be infected at the peak of the second wave with a stronger intervention. This study provides quantitative relationship between the strength of a one shot intervention and the reduction in the number of patients with no approximation. This suggests the importance of the strength and time of NPIs, although detailed studies are necessary for the implementation of NPIs in complicated real world environments as the model used in this study is based on various simplifications.


Two New Kinds of Interference Alignment Schemes for Cellular K user MIMO Downlink Networks. It is known that interference alignment (IA) plays an important role in improving the degree of freedom (DoF) of multi input and multi output (MIMO) systems. However, most of the traditional IA schemes suffer from the high computational complexity and require the global and instantaneous channel state information (CSI), both of which make them difficult to be extended to cellular MIMO systems. To handle these issues, two new interference alignment schemes, i.e., the retrospective interference regeneration (RIR) scheme and the beamforming based distributed retrospective interference alignment (B DRIA) scheme, are proposed for cellular K user MIMO downlink networks. For the RIR scheme, it adopts interference elimination algorithm to erase redundant symbols in inter cell interference (ICI) signals, and then uses interference regeneration algorithm to avoid secondary interference. The RIR scheme obtains greater DoF gain than the retrospective interference alignment (RIA) scheme, but incurs performance degradation when the transceiver antennas ratio (TAR) approaches 1. Therefore, the B DRIA scheme is further proposed. For the B DRIA scheme, the cellular beamforming matrix is introduced to eliminate the ICI, and meanwhile distributed retrospective interference alignment algorithm is adopted to align inter user interference (IUI). The simulation results show that the B DRIA scheme obtains larger DoF than the RIR scheme locally. Specifically, when TAR approaches 1, two schemes obtain the same DoF. While TAR approaches 2, the DoF of the B DRIA scheme is superior than the RIR scheme.


Multi object Tracking with Tracked Object Bounding Box Association. The CenterTrack tracking algorithm achieves state of the art tracking performance using a simple detection model and single frame spatial offsets to localize objects and predict their associations in a single network. However, this joint detection and tracking method still suffers from high identity switches due to the inferior association method. To reduce the high number of identity switches and improve the tracking accuracy, in this paper, we propose to incorporate a simple tracked object bounding box and overlapping prediction based on the current frame onto the CenterTrack algorithm. Specifically, we propose an Intersection over Union (IOU) distance cost matrix in the association step instead of simple point displacement distance. We evaluate our proposed tracker on the MOT17 test dataset, showing that our proposed method can reduce identity switches significantly by 22.6 and obtain a notable improvement of 1.5 in IDF1 compared to the original CenterTrack's under the same tracklet lifetime. The source code is released at https github.com Nanyangny CenterTrack IOU.


Stress driven two phase integral elasticity for Timoshenko curved beams. In this research, the size dependent static behaviour of elastic curved stubby beams is investigated by Timoshenko kinematics. Stress driven two phase integral elasticity is adopted to model size effects which soften or stiffen classical local responses. The corresponding governing equations of nonlocal elasticity are established and discussed, non classical boundary conditions are detected and an effective coordinate free solution procedure is proposed. The presented mixture approach is elucidated by solving simple curved small scale beams of current interest in Nanotechnology. The contributed results could be useful for design and optimization of modern sensors and actuators.


SelectAugment Hierarchical Deterministic Sample Selection for Data Augmentation. Data augmentation (DA) has been widely investigated to facilitate model optimization in many tasks. However, in most cases, data augmentation is randomly performed for each training sample with a certain probability, which might incur content destruction and visual ambiguities. To eliminate this, in this paper, we propose an effective approach, dubbed SelectAugment, to select samples to be augmented in a deterministic and online manner based on the sample contents and the network training status. Specifically, in each batch, we first determine the augmentation ratio, and then decide whether to augment each training sample under this ratio. We model this process as a two step Markov decision process and adopt Hierarchical Reinforcement Learning (HRL) to learn the augmentation policy. In this way, the negative effects of the randomness in selecting samples to augment can be effectively alleviated and the effectiveness of DA is improved. Extensive experiments demonstrate that our proposed SelectAugment can be adapted upon numerous commonly used DA methods, e.g., Mixup, Cutmix, AutoAugment, etc, and improve their performance on multiple benchmark datasets of image classification and fine grained image recognition.


Temporal Meta path Guided Explainable Recommendation. This paper utilizes well designed item item path modelling between consecutive items with attention mechanisms to sequentially model dynamic user item evolutions on dynamic knowledge graph for explainable recommendations. Compared with existing works that use heavy recurrent neural networks to model temporal information, we propose simple but effective neural networks to capture user historical item features and path based context to characterise next purchased item. Extensive evaluations of TMER on three real world benchmark datasets show state of the art performance compared against recent strong baselines.


Knowledge Rocks Adding Knowledge Assistance to Visualization Systems. We present Knowledge Rocks, an implementation strategy and guideline for augmenting visualization systems to knowledge assisted visualization systems, as defined by the KAVA model. Visualization systems become more and more sophisticated. Hence, it is increasingly important to support users with an integrated knowledge base in making constructive choices and drawing the right conclusions. We support the effective reactivation of visualization software resources by augmenting them with knowledge assistance. To provide a general and yet supportive implementation strategy, we propose an implementation process that bases on an application agnostic architecture. This architecture is derived from existing knowledge assisted visualization systems and the KAVA model. Its centerpiece is an ontology that is able to automatically analyze and classify input data, linked to a database to store classified instances. We discuss design decisions and advantages of the KR framework and illustrate its broad area of application in diverse integration possibilities of this architecture into an existing visualization system. In addition, we provide a detailed case study by augmenting an it security system with knowledge assistance facilities.


On Dynamic Noise Influence in Differentially Private Learning. Protecting privacy in learning while maintaining the model performance has become increasingly critical in many applications that involve sensitive data. Private Gradient Descent (PGD) is a commonly used private learning framework, which noises gradients based on the Differential Privacy protocol. Recent studies show that emph dynamic privacy schedules of decreasing noise magnitudes can improve loss at the final iteration, and yet theoretical understandings of the effectiveness of such schedules and their connections to optimization algorithms remain limited. In this paper, we provide comprehensive analysis of noise influence in dynamic privacy schedules to answer these critical questions. We first present a dynamic noise schedule minimizing the utility upper bound of PGD, and show how the noise influence from each optimization step collectively impacts utility of the final model. Our study also reveals how impacts from dynamic noise influence change when momentum is used. We empirically show the connection exists for general non convex losses, and the influence is greatly impacted by the loss curvature.


Positive solutions of the mathcal A Laplace equation with a potential. In this paper, we study positive solutions of the quasilinear elliptic equation Q' p, mathcal A ,V u triangleq mathrm div , mathcal A (x, nabla u) V(x) u p 2 u 0, in a domain Omega subseteq mathbb R n , where n geq 2 , 1 p infty , the divergence of mathcal A is the well known mathcal A Laplace operator considered in the influential book of Heinonen, Kilpel " a inen, and Martio, and the potential V belongs to a certain local Morrey space. The main aim of the paper is to extend criticality theory to the operator Q' p, mathcal A ,V . In particular, we prove an Agmon Allegretto Piepenbrink (AAP) type theorem, establish the uniqueness and simplicity of the principal eigenvalue of Q' p, mathcal A ,V in a domain omega Subset Omega and various characterizations of criticality. Furthermore, we also study positive solutions of the equation Q' p, mathcal A ,V u 0 of minimal growth at infinity in Omega , the existence of minimal positive Green function, and the minimal decay at infinity of Hardy weights.


Hamiltonian flows for pseudo Anosov mapping classes. For a given pseudo Anosov homeomorphism phi of a closed surface S , the action of phi on the Teichm "uller space mathcal T(S) is an automorphism of the complex structure preserving the Weil Petersson Goldman symplectic form. We give explicit formulae for two invariant functions mathcal T(S) to mathbb R whose symplectic gradients generate autonomous Hamiltonian flows that coincide with the action of phi at time one. We compute the Poisson bracket of these two functions, which essentially amounts to computing the variation of length of a H "older cocyle on one lamination along the shear vector field defined by another. For a measurably generic set of laminations, we prove that the variation of length is expressed as the cosine of the angle between the two laminations integrated against the product H "older distribution, generalizing a result of Kerckhoff. We also obtain rates of convergence for the supports of germs of differentiable paths of measured laminations in the Hausdorff metric on a hyperbolic surface, which may be of independent interest.


Sarnak's conjecture for rank one subshifts. Using techniques developed in cite KLR , we verify Sarnak's conjecture for two classes of rank one subshifts with unbounded cutting parameters. The first class of rank one subshifts we consider are called em almost complete congruency classes ( em accc ), the definition of which is motivated by the main result of cite GS , which implies that, when a rank one subshift carries a unique non atomic invariant probability measure, it is accc if it is measure theoretically isomorphic to an odometer. The second class we consider consists of Katok's map and its generalizations.


Street Map Based Validation of Semantic Segmentation in Autonomous Driving. Artificial intelligence for autonomous driving must meet strict requirements on safety and robustness, which motivates the thorough validation of learned models. However, current validation approaches mostly require ground truth data and are thus both cost intensive and limited in their applicability. We propose to overcome these limitations by a model agnostic validation using a priori knowledge from street maps. In particular, we show how to validate semantic segmentation masks and demonstrate the potential of our approach using OpenStreetMap. We introduce validation metrics that indicate false positive or negative road segments. Besides the validation approach, we present a method to correct the vehicle's GPS position so that a more accurate localization can be used for the street map based validation. Lastly, we present quantitative results on the Cityscapes dataset indicating that our validation approach can indeed uncover errors in semantic segmentation masks.


Terrain prickliness theoretical grounds for low complexity viewsheds. An important task when working with terrain models is computing viewsheds the parts of the terrain visible from a given viewpoint. When the terrain is modeled as a polyhedral terrain, the viewshed is composed of the union of all the triangle parts that are visible from the viewpoint. The complexity of a viewshed can vary significantly, from constant to quadratic in the number of terrain vertices, depending on the terrain topography and the viewpoint position. In this work we study a new topographic attribute, the emph prickliness , that measures the number of local maxima in a terrain from all possible perspectives. We show that the prickliness effectively captures the potential of 2.5D terrains to have high complexity viewsheds, and we present near optimal algorithms to compute the prickliness of 1.5D and 2.5D terrains. We also report on some experiments relating the prickliness of real word 2.5D terrains to the size of the terrains and to their viewshed complexity.


Doping A technique for efficient compression of LSTM models using sparse structured additive matrices. Structured matrices, such as those derived from Kronecker products (KP), are effective at compressing neural networks, but can lead to unacceptable accuracy loss when applied to large models. In this paper, we propose the notion of doping addition of an extremely sparse matrix to a structured matrix. Doping facilitates additional degrees of freedom for a small number of parameters, allowing them to independently diverge from the fixed structure. To train LSTMs with doped structured matrices, we introduce the additional parameter matrix while slowly annealing its sparsity level. However, we find that performance degrades as we slowly sparsify the doping matrix, due to co matrix adaptation (CMA) between the structured and the sparse matrices. We address this over dependence on the sparse matrix using a co matrix dropout regularization (CMR) scheme. We provide empirical evidence to show that doping, CMA and CMR are concepts generally applicable to multiple structured matrices (Kronecker Product, LMF, Hybrid Matrix Decomposition). Additionally, results with doped kronecker product matrices demonstrate state of the art accuracy at large compression factors (10 25x) across 4 natural language processing applications with minor loss in accuracy. Doped KP compression technique outperforms previous state of the art compression results by achieving 1.3 2.4x higher compression factor at a similar accuracy, while also beating strong alternatives like pruning and low rank methods by a large margin (8 or more). Additionally, we show that doped KP can be deployed on commodity hardware using the current software stack and achieve 2.5 5.5x inference run time speed up over baseline.


From Show to Tell A Survey on Deep Learning based Image Captioning. Connecting Vision and Language plays an essential role in Generative Intelligence. For this reason, large research efforts have been devoted to image captioning, i.e. describing images with syntactically and semantically meaningful sentences. Starting from 2015 the task has generally been addressed with pipelines composed of a visual encoder and a language model for text generation. During these years, both components have evolved considerably through the exploitation of object regions, attributes, the introduction of multi modal connections, fully attentive approaches, and BERT like early fusion strategies. However, regardless of the impressive results, research in image captioning has not reached a conclusive answer yet. This work aims at providing a comprehensive overview of image captioning approaches, from visual encoding and text generation to training strategies, datasets, and evaluation metrics. In this respect, we quantitatively compare many relevant state of the art approaches to identify the most impactful technical innovations in architectures and training strategies. Moreover, many variants of the problem and its open challenges are discussed. The final goal of this work is to serve as a tool for understanding the existing literature and highlighting the future directions for a research area where Computer Vision and Natural Language Processing can find an optimal synergy.


Analysis of stochastic Lanczos quadrature for spectrum approximation. The cumulative empirical spectral measure (CESM) Phi mathbf A mathbb R to 0,1 of a n times n symmetric matrix mathbf A is defined as the fraction of eigenvalues of mathbf A less than a given threshold, i.e., Phi mathbf A (x) sum i 1 n frac 1 n large unicode x1D7D9 lambda i mathbf A leq x . Spectral sums operatorname tr (f mathbf A ) can be computed as the Riemann Stieltjes integral of f against Phi mathbf A , so the task of estimating CESM arises frequently in a number of applications, including machine learning. We present an error analysis for stochastic Lanczos quadrature (SLQ). We show that SLQ obtains an approximation to the CESM within a Wasserstein distance of t lambda text max mathbf A lambda text min mathbf A with probability at least 1 eta , by applying the Lanczos algorithm for lceil 12 t 1 frac 1 2 rceil iterations to lceil 4 ( n 2 ) 1 t 2 ln(2n eta 1 ) rceil vectors sampled independently and uniformly from the unit sphere. We additionally provide (matrix dependent) a posteriori error bounds for the Wasserstein and Kolmogorov Smirnov distances between the output of this algorithm and the true CESM. The quality of our bounds is demonstrated using numerical experiments.


Discrepancy of rational points in simple algebraic groups. The present paper analyzes the discrepancy of distribution of rational points on general semisimple algebraic group varieties. The results include mean square, almost sure, and uniform discrepancy estimates with explicit error bounds, which apply to general families of subsets, and are valid at arbitrarily small scales. We also consider an analogue of W. Schmidt's classical theorem, which establishes effective almost sure asymptotic counting of rational solutions to Diophantine inequalities in Euclidean spaces. We formulate and prove a version of it for rational points on the group variety, together with an effective bound which in some instances can be expected to be best possible.


Design and Analysis of a Robotic Lizard using Five Bar Mechanism. Legged robots are being used to explore rough terrains as they are capable of traversing gaps and obstacles. In this paper, a new mechanism is designed to replicate a robotic lizard using integrated five bar mechanisms. There are two five bar mechanisms from which two more are formed by connecting the links in a particular order. The legs are attached to the links of the five bar mechanism such that, when the mechanism is actuated, they move the robot forward. Position analysis using vector loop approach has been done for the mechanism. A prototype has been built and controlled using servo motors to verify the robotic lizard mechanism.


A Unified Benchmark for the Unknown Detection Capability of Deep Neural Networks. Deep neural networks have achieved outstanding performance over various tasks, but they have a critical issue over confident predictions even for completely unknown samples. Many studies have been proposed to successfully filter out these unknown samples, but they only considered narrow and specific tasks, referred to as misclassification detection, open set recognition, or out of distribution detection. In this work, we argue that these tasks should be treated as fundamentally an identical problem because an ideal model should possess detection capability for all those tasks. Therefore, we introduce the unknown detection task, an integration of previous individual tasks, for a rigorous examination of the detection capability of deep neural networks on a wide spectrum of unknown samples. To this end, unified benchmark datasets on different scales were constructed and the unknown detection capabilities of existing popular methods were subject to comparison. We found that Deep Ensemble consistently outperforms the other approaches in detecting unknowns however, all methods are only successful for a specific type of unknown. The reproducible code and benchmark datasets are available at https github.com daintlab unknown detection benchmarks .


Pitch Angle Scattering of Fast Particles by Low Frequency Magnetic Fluctuations. The adiabatic invariance of the magnetic moment during particle motion is of fundamental importance to the dynamics of magnetized plasma. The related rate of pitch angle scattering is investigated here for fast particles that thermally stream through static magnetic perturbations. For a uniform magnetic field with a localized perturbation it is found that the curvature parameter kappa 2 min(R c rho L) does not predict the level of pitch angle scattering. Instead, based on numerical integration of particle orbits in prescribed magnetic fields, we derive predictions for the particle scattering rates, which can be characterized by the relative perturbation amplitude A simeq delta B B , and the particle Larmor radius normalized by the field aligned wavelength of the perturbations, rho L lambda . Particles with rho L lambda simeq1 , are subject to strong pitch angle scattering, while the level of scattering vanishes for both the limits of rho L lambda ll 1 and rho L lambda gg 1 . The results are summarized in terms of a scattering operator, suitable for including the described scattering in basic kinetic models.


Empirical Analysis of Image Caption Generation using Deep Learning. Automated image captioning is one of the applications of Deep Learning which involves fusion of work done in computer vision and natural language processing, and it is typically performed using Encoder Decoder architectures. In this project, we have implemented and experimented with various flavors of multi modal image captioning networks where ResNet101, DenseNet121 and VGG19 based CNN Encoders and Attention based LSTM Decoders were explored. We have studied the effect of beam size and the use of pretrained word embeddings and compared them to baseline CNN encoder and RNN decoder architecture. The goal is to analyze the performance of each approach using various evaluation metrics including BLEU, CIDEr, ROUGE and METEOR. We have also explored model explainability using Visual Attention Maps (VAM) to highlight parts of the images which has maximum contribution for predicting each word of the generated caption.


DEFT Detection Embeddings for Tracking. Most modern multiple object tracking (MOT) systems follow the tracking by detection paradigm, consisting of a detector followed by a method for associating detections into tracks. There is a long history in tracking of combining motion and appearance features to provide robustness to occlusions and other challenges, but typically this comes with the trade off of a more complex and slower implementation. Recent successes on popular 2D tracking benchmarks indicate that top scores can be achieved using a state of the art detector and relatively simple associations relying on single frame spatial offsets notably outperforming contemporary methods that leverage learned appearance features to help re identify lost tracks. In this paper, we propose an efficient joint detection and tracking model named DEFT, or "Detection Embeddings for Tracking." Our approach relies on an appearance based object matching network jointly learned with an underlying object detection network. An LSTM is also added to capture motion constraints. DEFT has comparable accuracy and speed to the top methods on 2D online tracking leaderboards while having significant advantages in robustness when applied to more challenging tracking data. DEFT raises the bar on the nuScenes monocular 3D tracking challenge, more than doubling the performance of the previous top method. Code is publicly available.


Influencing Towards Stable Multi Agent Interactions. Learning in multi agent environments is difficult due to the non stationarity introduced by an opponent's or partner's changing behaviors. Instead of reactively adapting to the other agent's (opponent or partner) behavior, we propose an algorithm to proactively influence the other agent's strategy to stabilize which can restrain the non stationarity caused by the other agent. We learn a low dimensional latent representation of the other agent's strategy and the dynamics of how the latent strategy evolves with respect to our robot's behavior. With this learned dynamics model, we can define an unsupervised stability reward to train our robot to deliberately influence the other agent to stabilize towards a single strategy. We demonstrate the effectiveness of stabilizing in improving efficiency of maximizing the task reward in a variety of simulated environments, including autonomous driving, emergent communication, and robotic manipulation. We show qualitative results on our website https sites.google.com view stable marl .


Towards Trustworthy Cross patient Model Development. Machine learning is used in medicine to support physicians in examination, diagnosis, and predicting outcomes. One of the most dynamic area is the usage of patient generated health data from intensive care units. The goal of this paper is to demonstrate how we advance cross patient ML model development by combining the patient's demographics data with their physiological data. We used a population of patients undergoing Carotid Enderarterectomy (CEA), where we studied differences in model performance and explainability when trained for all patients and one patient at a time. The results show that patients' demographics has a large impact on the performance and explainability and thus trustworthiness. We conclude that we can increase trust in ML models in a cross patient context, by careful selection of models and patients based on their demographics and the surgical procedure.


Converting Nondeterministic Two Way Automata into Small Deterministic Linear Time Machines. In 1978 Sakoda and Sipser raised the question of the cost, in terms of size of representations, of the transformation of two way and one way nondeterministic automata into equivalent two way deterministic automata. Despite all the attempts, the question has been answered only for particular cases (e.g., restrictions of the class of simulated automata or of the class of simulating automata). However the problem remains open in the general case, the best known upper bound being exponential. We present a new approach in which unrestricted nondeterministic finite automata are simulated by deterministic models extending two way deterministic finite automata, paying a polynomial increase of size only. Indeed, we study the costs of the conversions of nondeterministic finite automata into some variants of one tape deterministic Turing machines working in linear time, namely Hennie machines, weight reducing Turing machines, and weight reducing Hennie machines. All these variants are known to share the same computational power they characterize the class of regular languages.


PPL MCTS Constrained Textual Generation Through Discriminator Guided MCTS Decoding. Large language models (LM) based on Transformers allow to generate plausible long texts. In this paper, we explore how this generation can be further controlled at decoding time to satisfy certain constraints (e.g. being non toxic, conveying certain emotions, using a specific writing style, etc.) without fine tuning the LM. Precisely, we formalize constrained generation as a tree exploration process guided by a discriminator that indicates how well the associated sequence respects the constraint. This approach, in addition to being easier and cheaper to train than fine tuning the LM, allows to apply the constraint more finely and dynamically. We propose several original methods to search this generation tree, notably the Monte Carlo Tree Search (MCTS) which provides theoretical guarantees on the search efficiency, but also simpler methods based on re ranking a pool of diverse sequences using the discriminator scores. These methods are evaluated, with automatic and human based metrics, on two types of constraints and languages review polarity and emotion control in French and English. We show that discriminator guided MCTS decoding achieves state of the art results without having to tune the language model, in both tasks and languages. We also demonstrate that other proposed decoding methods based on re ranking can be really effective when diversity among the generated propositions is encouraged.


KGSecConfig A Knowledge Graph Based Approach for Secured Container Orchestrator Configuration. Container Orchestrator (CO) is a vital technology for managing clusters of containers, which may form a virtualized infrastructure for developing and operating software systems. Like any other software system, securing CO is critical, but can be quite challenging task due to large number of configurable options. Manual configuration is not only knowledge intensive and time consuming, but also is error prone. For automating security configuration of CO, we propose a novel Knowledge Graph based Security Configuration, KGSecConfig, approach. Our solution leverages keyword and learning models to systematically capture, link, and correlate heterogeneous and multi vendor configuration space in a unified structure for supporting automation of security configuration of CO. We implement KGSecConfig on Kubernetes, Docker, Azure, and VMWare to build secured configuration knowledge graph. Our evaluation results show 0.98 and 0.94 accuracy for keyword and learning based secured configuration option and concept extraction, respectively. We also demonstrate the utilization of the knowledge graph for automated misconfiguration mitigation in a Kubernetes cluster. We assert that our knowledge graph based approach can help in addressing several challenges, e.g., misconfiguration of security, associated with manually configuring the security of CO.


Understanding and Mitigating the Effect of Outliers in Fair Ranking. Traditional ranking systems are expected to sort items in the order of their relevance and thereby maximize their utility. In fair ranking, utility is complemented with fairness as an optimization goal. Recent work on fair ranking focuses on developing algorithms to optimize for fairness, given position based exposure. In contrast, we identify the potential of outliers in a ranking to influence exposure and thereby negatively impact fairness. An outlier in a list of items can alter the examination probabilities, which can lead to different distributions of attention, compared to position based exposure. We formalize outlierness in a ranking, show that outliers are present in realistic datasets, and present the results of an eye tracking study, showing that users scanning order and the exposure of items are influenced by the presence of outliers. We then introduce OMIT, a method for fair ranking in the presence of outliers. Given an outlier detection method, OMIT improves fair allocation of exposure by suppressing outliers in the top k ranking. Using an academic search dataset, we show that outlierness optimization leads to a fairer policy that displays fewer outliers in the top k, while maintaining a reasonable trade off between fairness and utility.


Probabilistic and Geometric Depth Detecting Objects in Perspective. 3D object detection is an important capability needed in various practical applications such as driver assistance systems. Monocular 3D detection, as a representative general setting among image based approaches, provides a more economical solution than conventional settings relying on LiDARs but still yields unsatisfactory results. This paper first presents a systematic study on this problem. We observe that the current monocular 3D detection can be simplified as an instance depth estimation problem The inaccurate instance depth blocks all the other 3D attribute predictions from improving the overall detection performance. Moreover, recent methods directly estimate the depth based on isolated instances or pixels while ignoring the geometric relations across different objects. To this end, we construct geometric relation graphs across predicted objects and use the graph to facilitate depth estimation. As the preliminary depth estimation of each instance is usually inaccurate in this ill posed setting, we incorporate a probabilistic representation to capture the uncertainty. It provides an important indicator to identify confident predictions and further guide the depth propagation. Despite the simplicity of the basic idea, our method, PGD, obtains significant improvements on KITTI and nuScenes benchmarks, achieving 1st place out of all monocular vision only methods while still maintaining real time efficiency. Code and models will be released at https github.com open mmlab mmdetection3d.


Selectively Delaying Instructions to Prevent Microarchitectural Replay Attacks. MicroScope, and microarchitectural replay attacks in general, take advantage of the characteristics of speculative execution to trap the execution of the victim application in an infinite loop, enabling the attacker to amplify a side channel attack by executing it indefinitely. Due to the nature of the replay, it can be used to effectively attack security critical trusted execution environments (secure enclaves), even under conditions where a side channel attack would not be possible. At the same time, unlike speculative side channel attacks, MicroScope can be used to amplify the correct path of execution, rendering many existing speculative side channel defences ineffective. In this work, we generalize microarchitectural replay attacks beyond MicroScope and present an efficient defence against them. We make the observation that such attacks rely on repeated squashes of so called "replay handles" and that the instructions causing the side channel must reside in the same reorder buffer window as the handles. We propose Delay on Squash, a technique for tracking squashed instructions and preventing them from being replayed by speculative replay handles. Our evaluation shows that it is possible to achieve full security against microarchitectural replay attacks with very modest hardware requirements, while still maintaining 97 of the insecure baseline performance.


Noncommutative conics in Calabi Yau quantum projective planes. In noncommutative algebraic geometry, noncommutative quadric hypersurfaces are major objects of study. In this paper, we focus on studying noncommutative conics operatorname Proj nc A embedded into Calabi Yau quantum projective planes. In particular, we give complete classifications of homogeneous coordinate algebras A of noncommutative conics up to isomorphism of graded algebras, and of noncommutive conics operatorname Proj nc A up to isomorphism of noncommutative schemes.


A Method for Hiding the Increased Non Volatile Cache Read Latency. The increased memory demands of workloads is putting high pressure on Last Level Caches (LLCs). Unfortunately, there is limited opportunity to increase the capacity of LLCs due to the area and power requirements of the underlying SRAM technology. Interestingly, emerging Non Volatile Memory (NVM) technologies promise a feasible alternative to SRAM for LLCs due to their higher area density. However, NVMs have substantially higher read and write latencies, which offset their area density benefit. Although researchers have proposed methods to tolerate NVM's increased write latency, little emphasis has been placed on reducing the critical NVM read latency. To address this problem, this paper proposes Cloak. Cloak exploits data reuse in the LLC at the page level, to hide NVM read latency. Specifically, on certain L1 TLB misses to a page, Cloak transfers LLC resident data belonging to the page from the LLC NVM array to a set of small SRAM Page Buffers that will service subsequent requests to this page. Further, to enable the high bandwidth, low latency transfer of lines of a page to the page buffers, Cloak uses an LLC layout that accelerates the discovery of LLC resident cache lines from the page. We evaluate Cloak with full system simulations of a 4 core processor across 14 workloads. We find that, on average, Cloak outperforms an SRAM LLC by 23.8 and an NVM only LLC by 8.9 in both cases, with negligible additional area. Further, Cloak's ED 2 is 39.9 and 17.5 lower, respectively, than these designs.


Results and Insights from Diagnostic Questions The NeurIPS 2020 Education Challenge. This competition concerns educational diagnostic questions, which are pedagogically effective, multiple choice questions (MCQs) whose distractors embody misconceptions. With a large and ever increasing number of such questions, it becomes overwhelming for teachers to know which questions are the best ones to use for their students. We thus seek to answer the following question how can we use data on hundreds of millions of answers to MCQs to drive automatic personalized learning in large scale learning scenarios where manual personalization is infeasible? Success in using MCQ data at scale helps build more intelligent, personalized learning platforms that ultimately improve the quality of education en masse. To this end, we introduce a new, large scale, real world dataset and formulate 4 data mining tasks on MCQs that mimic real learning scenarios and target various aspects of the above question in a competition setting at NeurIPS 2020. We report on our NeurIPS competition in which nearly 400 teams submitted approximately 4000 submissions, with encouragingly diverse and effective approaches to each of our tasks.


Kluv ' a nek Lewis Henstock integral in a Banach space. We investigate some properties and convergence theorem of Kluv ' a nek Lewis Henstock m integrability for m measurable functions that we introduced in cite ABH . We give a m a.e. convergence version of Dominated (resp. Bounded) Convergence Theorem for m. We introduce Kluv ' a nek Lewis Henstock integrable of scalar valued functions with respect to a set valued measure in a Banach space. Finally we introduce (KL) type Dominated Convergence Theorem for the set valued Kluv ' a nek Lewis Henstock integral.


Unsupervised Discriminative Embedding for Sub Action Learning in Complex Activities. Action recognition and detection in the context of long untrimmed video sequences has seen an increased attention from the research community. However, annotation of complex activities is usually time consuming and challenging in practice. Therefore, recent works started to tackle the problem of unsupervised learning of sub actions in complex activities. This paper proposes a novel approach for unsupervised sub action learning in complex activities. The proposed method maps both visual and temporal representations to a latent space where the sub actions are learnt discriminatively in an end to end fashion. To this end, we propose to learn sub actions as latent concepts and a novel discriminative latent concept learning (DLCL) module aids in learning sub actions. The proposed DLCL module lends on the idea of latent concepts to learn compact representations in the latent embedding space in an unsupervised way. The result is a set of latent vectors that can be interpreted as cluster centers in the embedding space. The latent space itself is formed by a joint visual and temporal embedding capturing the visual similarity and temporal ordering of the data. Our joint learning with discriminative latent concept module is novel which eliminates the need for explicit clustering. We validate our approach on three benchmark datasets and show that the proposed combination of visual temporal embedding and discriminative latent concepts allow to learn robust action representations in an unsupervised setting.


Derivation of Computational Formulas for certain class of finite sums Approach to Generating functions arising from p adic integrals and special functions. The aim of this paper is to construct generating functions for some families of special finite sums with the aid of the Newton Mercator series, hypergeometric series, and p adic integral (the Volkenborn integral). By using these generating functions, their functional equations, and their partial derivative equations, many novel computational formulas involving the special finite sums of (inverse) binomial coefficients, the Bernoulli type polynomials and numbers, Euler polynomials and numbers, the Stirling numbers, the (alternating) harmonic numbers, the Leibnitz polynomials and others. Among these formulas, by considering a computational formula which computes the aforementioned certain class of finite sums with the aid of the Bernoulli numbers and the Stirling numbers of the first kind, we present a computation algorithm and we provide some of their special values. Morover, using the aforementioned special finite sums and combinatorial numbers, we give relations among multiple alternating zeta functions, the Bernoulli polynomials of higher order and the Euler polynomials of higher order. We also give decomposition of the multiple Hurwitz zeta functions with the aid of finite sums. Relationships and comparisons between the main results given in the article and previously known results have been criticized. With the help of the results of this paper, the solution of the problem that Charalambides 8, Exercise 30, p. 273 gave in his book was found and with the help of this solution, we also find very new formulas. In addition, the solutions of some of the problems we have raised in 48 are also given.


Hierarchical Self Attention Based Autoencoder for Open Set Human Activity Recognition. Wearable sensor based human activity recognition is a challenging problem due to difficulty in modeling spatial and temporal dependencies of sensor signals. Recognition models in closed set assumption are forced to yield members of known activity classes as prediction. However, activity recognition models can encounter an unseen activity due to body worn sensor malfunction or disability of the subject performing the activities. This problem can be addressed through modeling solution according to the assumption of open set recognition. Hence, the proposed self attention based approach combines data hierarchically from different sensor placements across time to classify closed set activities and it obtains notable performance improvement over state of the art models on five publicly available datasets. The decoder in this autoencoder architecture incorporates self attention based feature representations from encoder to detect unseen activity classes in open set recognition setting. Furthermore, attention maps generated by the hierarchical model demonstrate explainable selection of features in activity recognition. We conduct extensive leave one subject out validation experiments that indicate significantly improved robustness to noise and subject specific variability in body worn sensor signals. The source code is available at github.com saif mahmud hierarchical attention HAR


Older Adults and Brain Computer Interface An Exploratory Study. In this exploratory study, we examine the possibilities of non invasive Brain Computer Interface (BCI) in the context of Smart Home Technology (SHT) targeted at older adults. During two workshops, one stationary, and one online via Zoom, we researched the insights of the end users concerning the potential of the BCI in the SHT setting. We explored its advantages and drawbacks, and the features older adults see as vital as well as the ones that they would benefit from. Apart from evaluating the participants' perception of such devices during the two workshops we also analyzed some key considerations resulting from the insights gathered during the workshops, such as potential barriers, ways to mitigate them, strengths and opportunities connected to BCI. These may be useful for designing BCI interaction paradigms and pinpointing areas of interest to pursue in further studies.


Deep Rival Penalized Competitive Learning for Low resolution Face Recognition. Current face recognition tasks are usually carried out on high quality face images, but in reality, most face images are captured under unconstrained or poor conditions, e.g., by video surveillance. Existing methods are featured by learning data uncertainty to avoid overfitting the noise, or by adding margins to the angle or cosine space of the normalized softmax loss to penalize the target logit, which enforces intra class compactness and inter class discrepancy. In this paper, we propose a deep Rival Penalized Competitive Learning (RPCL) for deep face recognition in low resolution (LR) images. Inspired by the idea of the RPCL, our method further enforces regulation on the rival logit, which is defined as the largest non target logit for an input image. Different from existing methods that only consider penalization on the target logit, our method not only strengthens the learning towards the target label, but also enforces a reverse direction, i.e., becoming de learning, away from the rival label. Comprehensive experiments demonstrate that our method improves the existing state of the art methods to be very robust for LR face recognition.


Deep unsupervised 3D human body reconstruction from a sparse set of landmarks. In this paper we propose the first deep unsupervised approach in human body reconstruction to estimate body surface from a sparse set of landmarks, so called DeepMurf. We apply a denoising autoencoder to estimate missing landmarks. Then we apply an attention model to estimate body joints from landmarks. Finally, a cascading network is applied to regress parameters of a statistical generative model that reconstructs body. Our set of proposed loss functions allows us to train the network in an unsupervised way. Results on four public datasets show that our approach accurately reconstructs the human body from real world mocap data.


Infinitesimal rigidity and prestress stability for frameworks in normed spaces. A (bar and joint) framework is a set of points in a normed space with a set of fixed distance constraints between them. Determining whether a framework is locally rigid i.e. whether every other suitably close framework with the same distance constraints is an isometric copy is NP hard when the normed space has dimension 2 or greater. We can reduce the complexity by instead considering derivatives of the constraints, which linearises the problem. By applying methods from non smooth analysis, we shall strengthen previous sufficient conditions for framework rigidity that utilise first order derivatives. We shall also introduce the notions of prestress stability and second order rigidity to the topic of normed space rigidity, two weaker sufficient conditions for framework rigidity previously only considered for Euclidean spaces.


Existence and regularity for a system of porous medium equations with small cross diffusion and nonlocal drifts. We prove existence and Sobolev regularity of solutions of a nonlinear system of degenerate parabolic PDEs with self and cross diffusion, transport confinement and nonlocal interaction terms. The macroscopic system of PDEs is formally derived from a large particle system and models the evolution of an arbitrary number of species with quadratic porous medium interactions in a bounded domain Omega in any spatial dimension. The cross interactions between different species are scaled by a parameter delta 1 , with the delta 0 case corresponding to no interactions across species. A smallness condition on delta ensures existence of solutions up to an arbitrary time T 0 in a subspace of L 2(0,T H 1( Omega)) . This is shown via a Schauder fixed point argument for a regularised system combined with a vanishing diffusivity approach. The behaviour of solutions for extreme values of delta is studied numerically.


Reducing ReLU Count for Privacy Preserving CNN Speedup. Privacy Preserving Machine Learning algorithms must balance classification accuracy with data privacy. This can be done using a combination of cryptographic and machine learning tools such as Convolutional Neural Networks (CNN). CNNs typically consist of two types of operations a convolutional or linear layer, followed by a non linear function such as ReLU. Each of these types can be implemented efficiently using a different cryptographic tool. But these tools require different representations and switching between them is time consuming and expensive. Recent research suggests that ReLU is responsible for most of the communication bandwidth. ReLU is usually applied at each pixel (or activation) location, which is quite expensive. We propose to share ReLU operations. Specifically, the ReLU decision of one activation can be used by others, and we explore different ways to group activations and different ways to determine the ReLU for such a group of activations. Experiments on several datasets reveal that we can cut the number of ReLU operations by up to three orders of magnitude and, as a result, cut the communication bandwidth by more than 50 .


Gevrey well posedness for 3 evolution equations with variable coefficients. We study the Cauchy problem for a class of third order linear anisotropic evolution equations with complex valued lower order terms depending both on time and space variables. Under suitable decay assumptions for x to infty on these coefficients, we prove a well posedness result in Gevrey type spaces.


DSXplore Optimizing Convolutional Neural Networks via Sliding Channel Convolutions. As the key advancement of the convolutional neural networks (CNNs), depthwise separable convolutions (DSCs) are becoming one of the most popular techniques to reduce the computations and parameters size of CNNs meanwhile maintaining the model accuracy. It also brings profound impact to improve the applicability of the compute and memory intensive CNNs to a broad range of applications, such as mobile devices, which are generally short of computation power and memory. However, previous research in DSCs are largely focusing on compositing the limited existing DSC designs, thus, missing the opportunities to explore more potential designs that can achieve better accuracy and higher computation parameter reduction. Besides, the off the shelf convolution implementations offer limited computing schemes, therefore, lacking support for DSCs with different convolution patterns. To this end, we introduce, DSXplore, the first optimized design for exploring DSCs on CNNs. Specifically, at the algorithm level, DSXplore incorporates a novel factorized kernel sliding channel convolution (SCC), featured with input channel overlapping to balance the accuracy performance and the reduction of computation and memory cost. SCC also offers enormous space for design exploration by introducing adjustable kernel parameters. Further, at the implementation level, we carry out an optimized GPU implementation tailored for SCC by leveraging several key techniques, such as the input centric backward design and the channel cyclic optimization. Intensive experiments on different datasets across mainstream CNNs show the advantages of DSXplore in balancing accuracy and computation parameter reduction over the standard convolution and the existing DSCs.


Pre Training on Dynamic Graph Neural Networks. The pre training on the graph neural network model can learn the general features of large scale networks or networks of the same type by self supervised methods, which allows the model to work even when node labels are missing. However, the existing pre training methods do not take network evolution into consideration. This paper proposes a pre training method on dynamic graph neural networks (PT DGNN), which uses dynamic attributed graph generation tasks to simultaneously learn the structure, semantics, and evolution features of the graph. The method includes two steps 1) dynamic sub graph sampling, and 2) pre training with dynamic attributed graph generation task. Comparative experiments on three realistic dynamic network datasets show that the proposed method achieves the best results on the link prediction fine tuning task.


Uncovering the Dark Side of Telegram Fakes, Clones, Scams, and Conspiracy Movements. Telegram is one of the most used instant messaging apps worldwide. Some of its success lies in providing high privacy protection and social network features like the channels virtual rooms in which only the admins can post and broadcast messages to all its subscribers. However, these same features contributed to the emergence of borderline activities and, as is common with Online Social Networks, the heavy presence of fake accounts. Telegram started to address these issues by introducing the verified and scam marks for the channels. Unfortunately, the problem is far from being solved. In this work, we perform a large scale analysis of Telegram by collecting 35,382 different channels and over 130,000,000 messages. We study the channels that Telegram marks as verified or scam, highlighting analogies and differences. Then, we move to the unmarked channels. Here, we find some of the infamous activities also present on privacy preserving services of the Dark Web, such as carding, sharing of illegal adult and copyright protected content. In addition, we identify and analyze two other types of channels the clones and the fakes. Clones are channels that publish the exact content of another channel to gain subscribers and promote services. Instead, fakes are channels that attempt to impersonate celebrities or well known services. Fakes are hard to identify even by the most advanced users. To detect the fake channels automatically, we propose a machine learning model that is able to identify them with an accuracy of 86 . Lastly, we study Sabmyk, a conspiracy theory that exploited fakes and clones to spread quickly on the platform reaching over 1,000,000 users.


Three approaches to facilitate DNN generalization to objects in out of distribution orientations and illuminations. The training data distribution is often biased towards objects in certain orientations and illumination conditions. While humans have a remarkable capability of recognizing objects in out of distribution (OoD) orientations and illuminations, Deep Neural Networks (DNNs) severely suffer in this case, even when large amounts of training examples are available. In this paper, we investigate three different approaches to improve DNNs in recognizing objects in OoD orientations and illuminations. Namely, these are (i) training much longer after convergence of the in distribution (InD) validation accuracy, i.e., late stopping, (ii) tuning the momentum parameter of the batch normalization layers, and (iii) enforcing invariance of the neural activity in an intermediate layer to orientation and illumination conditions. Each of these approaches substantially improves the DNN's OoD accuracy (more than 20 in some cases). We report results in four datasets two datasets are modified from the MNIST and iLab datasets, and the other two are novel (one of 3D rendered cars and another of objects taken from various controlled orientations and illumination conditions). These datasets allow to study the effects of different amounts of bias and are challenging as DNNs perform poorly in OoD conditions. Finally, we demonstrate that even though the three approaches focus on different aspects of DNNs, they all tend to lead to the same underlying neural mechanism to enable OoD accuracy gains individual neurons in the intermediate layers become more selective to a category and also invariant to OoD orientations and illuminations. We anticipate this study to be a basis for further improvement of deep neural networks' OoD generalization performance, which is highly demanded to achieve safe and fair AI applications.


A single frequency single resonator laser on erbium doped lithium niobate on insulator. Erbium doped lithium niobate on insulator (Er LNOI) is a promising platform for photonic integrated circuits as it adds gain to the LNOI system and enables on chip lasers and amplifiers. A challenge for Er LNOI laser is to increase its output power while maintaining single frequency and single ( transverse) mode operation. In this work, we demonstrate that single frequency and single mode operation can be achieved even in a single multi mode Er LNOI microring by introducing mode dependent loss and gain competition. In a single microring with a free spectral range of 192 GHz, we have achieved single mode lasing with an output power of 2.1 microwatt, a side mode suppression of 35.5 dB, and a linewidth of 1.27 MHz.


A new matroid lift construction and an application to group labeled graphs. A well known result of Brylawski constructs an elementary lift of a matroid M from a linear class of circuits of M . We generalize this result by showing how to construct a rank k lift of M from a rank k matroid on the set of circuits of M . We conjecture that every lift of M arises via this construction. We then apply this result to group labeled graphs, generalizing a construction of Zaslavsky. Given a graph G with edges labeled by a group, Zaslavsky's lift matroid K is an elementary lift of the graphic matroid M(G) that respects the group labeling specifically, the cycles of G that are circuits of K coincide with the cycles that are balanced with respect to the group labeling. For k ge 2 , when does there exist a rank k lift of M(G) that respects the group labeling in this same sense? For abelian groups, we show that such a matroid exists if and only if the group is isomorphic to the additive group of a non prime finite field.


Additive Polycyclic Codes over mathbb F 4 Induced by Binary Vectors and Some Optimal Codes. In this paper we study the structure and properties of additive right and left polycyclic codes induced by a binary vector a in mathbb F 2 n . We find the generator polynomials and the cardinality of these codes. We also study different duals for these codes. In particular, we show that if C is a right polycyclic code induced by a vector a in mathbb F 2 n , then the Hermitian dual of C is a sequential code induced by a. As an application of these codes, we present examples of additive right polycyclic codes over mathbb F 4 with more codewords than comparable optimal linear codes as well as optimal binary linear codes and optimal quantum codes obtained from additive right polycyclic codes over mathbb F 4 .


The Cuntz semigroup of unital commutative AI algebras. We provide an abstract characterization for the Cuntz semigroup of unital commutative AI algebras, as well as a characterization for abstract Cuntz semigroups of the form text Lsc (X, overline mathbb N ) for some T 1 space X . In our investigations, we also uncover new properties that the Cuntz semigroup of all AI algebras satisfies.


Anomaly Detection on IT Operation Series via Online Matrix Profile. Anomaly detection on time series is a fundamental task in monitoring the Key Performance Indicators (KPIs) of IT systems. Many of the existing approaches in the literature show good performance while requiring a lot of training resources. In this paper, the online matrix profile, which requires no training, is proposed to address this issue. The anomalies are detected by referring to the past subsequence that is the closest to the current one. The distance significance is introduced based on the online matrix profile, which demonstrates a prominent pattern when an anomaly occurs. Another training free approach spectral residual is integrated into our approach to further enhance the detection accuracy. Moreover, the proposed approach is sped up by at least four times for long time series by the introduced cache strategy. In comparison to the existing approaches, the online matrix profile makes a good trade off between accuracy and efficiency. More importantly, it is generic to various types of time series in the sense that it works without the constraint from any trained model.


PrivateSNN Privacy Preserving Spiking Neural Networks. How can we bring both privacy and energy efficiency to a neural system? In this paper, we propose PrivateSNN, which aims to build low power Spiking Neural Networks (SNNs) from a pre trained ANN model without leaking sensitive information contained in a dataset. Here, we tackle two types of leakage problems 1) Data leakage is caused when the networks access real training data during an ANN SNN conversion process. 2) Class leakage is caused when class related features can be reconstructed from network parameters. In order to address the data leakage issue, we generate synthetic images from the pre trained ANNs and convert ANNs to SNNs using the generated images. However, converted SNNs remain vulnerable to class leakage since the weight parameters have the same (or scaled) value with respect to ANN parameters. Therefore, we encrypt SNN weights by training SNNs with a temporal spike based learning rule. Updating weight parameters with temporal data makes SNNs difficult to be interpreted in the spatial domain. We observe that the encrypted PrivateSNN eliminates data and class leakage issues with a slight performance drop (less than 2) and significant energy efficiency gain (about 55x) compared to the standard ANN. We conduct extensive experiments on various datasets including CIFAR10, CIFAR100, and TinyImageNet, highlighting the importance of privacy preserving SNN training.


Persistent Anti Muslim Bias in Large Language Models. It has been observed that large scale language models capture undesirable societal biases, e.g. relating to race and gender yet religious bias has been relatively unexplored. We demonstrate that GPT 3, a state of the art contextual language model, captures persistent Muslim violence bias. We probe GPT 3 in various ways, including prompt completion, analogical reasoning, and story generation, to understand this anti Muslim bias, demonstrating that it appears consistently and creatively in different uses of the model and that it is severe even compared to biases about other religious groups. For instance, "Muslim" is analogized to "terrorist" in 23 of test cases, while "Jewish" is mapped to "money" in 5 of test cases. We quantify the positive distraction needed to overcome this bias with adversarial text prompts, and find that use of the most positive 6 adjectives reduces violent completions for "Muslims" from 66 to 20 , but which is still higher than for other religious groups.


Egalitarian Resource Sharing Over Multiple Rounds. It is often beneficial for agents to pool their resources in order to better accommodate fluctuations in individual demand. Many multi round resource allocation mechanisms operate in an online manner in each round, the agents specify their demands for that round, and the mechanism determines a corresponding allocation. In this paper, we focus instead on the offline setting in which the agents specify their demand for each round at the outset. We formulate a specific resource allocation problem in this setting, and design and analyze an associated mechanism based on the solution concept of lexicographic maximin fairness. We present an efficient implementation of our mechanism, and prove that it is Pareto efficient, envy free, non wasteful, resource monotonic, population monotonic, and group strategyproof. We also prove that our mechanism guarantees each agent at least half of the utility that they can obtain by not sharing their resources. We complement these positive results by proving that no maximin fair mechanism can improve on the aforementioned factor of one half.


Artistic Autonomy in AI Art. The concept of art has transposed meaning and medium across time, with its context being a deciding factor for its evolution. However, human beings' innermost functionality remains the same, and art, to this day, serves as an expression of the subconscious. Accelerated by the conception of GANs in 2014, automation has become a central medium in Artificial Intelligence (AI) Art. However, this raises concern over AI's influence on artistic autonomy within the process of creativity. This paper introduces the ethical responsibility of AI towards maintaining the artist's volition in exercising autonomy and utilizes principles of self determination theory alongside fundamental limits of creativity to do so.


Mastering the Explicit Opinion role Interaction Syntax aided Neural Transition System for Unified Opinion Role Labeling. Unified opinion role labeling (ORL) aims to detect all possible opinion structures of 'opinion holder target' in one shot, given a text. The existing transition based unified method, unfortunately, is subject to longer opinion terms and fails to solve the term overlap issue. Current top performance has been achieved by employing the span based graph model, which however still suffers from both high model complexity and insufficient interaction among opinions and roles. In this work, we investigate a novel solution by revisiting the transition architecture, and augmenting it with a pointer network (PointNet). The framework parses out all opinion structures in linear time complexity, meanwhile breaks through the limitation of any length of terms with PointNet. To achieve the explicit opinion role interactions, we further propose a unified dependency opinion graph (UDOG), co modeling the syntactic dependency structure and the partial opinion role structure. We then devise a relation centered graph aggregator (RCGA) to encode the multi relational UDOG, where the resulting high order representations are used to promote the predictions in the vanilla transition system. Our model achieves new state of the art results on the MPQA benchmark. Analyses further demonstrate the superiority of our methods on both efficacy and efficiency.


Study of the Subtyping Machine of Nominal Subtyping with Variance (full version). This is a study of the computing power of the subtyping machine behind Kennedy and Pierce's nominal subtyping with variance. We depict the lattice of fragments of Kennedy and Pierce's type system and characterize their computing power in terms of regular, context free, deterministic, and non deterministic tree languages. Based on the theory, we present Treetop a generator of C implementations of subtyping machines. The software artifact constitutes the first feasible (yet POC) fluent API generator to support context free API protocols in a decidable type system fragment.


Bounds on heat transport for convection driven by internal heating. The mean vertical heat transport langle wT rangle in convection between isothermal plates driven by uniform internal heating is investigated by means of rigorous bounds. These are obtained as a function of the Rayleigh number R by constructing feasible solutions to a convex variational problem, derived using a formulation of the classical background method in terms of quadratic auxiliary functions. When the fluid's temperature relative to the boundaries is allowed to be positive or negative, numerical solution of the variational problem shows that best previous bound langle wT rangle leq 1 2 can only be improved up to finite R . Indeed, we demonstrate analytically that langle wT rangle leq 2 21 5 R 1 5 and therefore prove that langle wT rangle 1 2 for R 65 ,536 . However, if the minimum principle for temperature is invoked, which asserts that internal temperature is at least as large as the temperature of the isothermal boundaries, then numerically optimised bounds are strictly smaller than 1 2 until at least R 3.4 times 10 5 . While the computational results suggest that the best bound on langle wT rangle approaches 1 2 asymptotically from below as R rightarrow infty , we prove that typical analytical constructions cannot be used to prove this conjecture.


Performance landscape of resource constrained platforms targeting DNNs. Over the recent years, a significant number of complex, deep neural networks have been developed for a variety of applications including speech and face recognition, computer vision in the areas of health care, automatic translation, image classification, etc. Moreover, there is an increasing demand in deploying these networks in resource constrained edge devices. As the computational demands of these models keep increasing, pushing to their limits the targeted devices, the constant development of new hardware systems tailored to those workloads has been observed. Since programmability of these diverse and complex platforms compounded by the rapid development of new DNN models is a major challenge, platform vendors have developed Machine Learning tailored SDKs to maximize the platform's performance. This work investigates the performance achieved on a number of modern commodity embedded platforms coupled with the vendors' provided software support when state of the art DNN models from image classification, object detection and image segmentation are targeted. The work quantifies the relative latency gains of the particular embedded platforms and provides insights on the relationship between the required minimum batch size for achieving maximum throughput, concluding that modern embedded systems reach their maximum performance even for modest batch sizes when a modern state of the art DNN model is targeted. Overall, the presented results provide a guide for the expected performance for a number of state of the art DNNs on popular embedded platforms across the image classification, detection and segmentation domains.


Dominantly Aseismic Nucleation of Laboratory Earthquakes A Quantitative Investigation. Decades of seismological observations have highlighted the variability of foreshock occurrence prior to natural earthquakes, making thus difficult to track how earthquakes start. Here, we report on three stick slip experiments performed on cylindrical samples of Indian metagabbro under upper crustal stress conditions (30 60 MPa ). Acoustic emission activity (AE) was continuously recorded by 8 calibrated acoustic sensors during the experiments, and the seismological parameters (moment magnitude, corner frequency and stress drop) of the detected AEs were estimated. The scaling law between moment magnitude and corner frequency that characterizes natural earthquakes also applies to the detected AEs ( 8.8 leq Mw leq 7 . Precursory AE activity is systemically detected during the pre failure period, increases towards failure and is found to be driven by along fault slip velocity. Consistently for all three experiments, the stacked AE sequences follow an inverse power law of the time to failure. Decreasing fault strength heterogeneity promotes, on average, AEs migration towards zones where stick slip events initiate. Overall, the seismic component of the pre failure phase differs by several orders of magnitude from the aseismic component. Our observations suggest that, in this particular experimental setting, precursory AE activity is predominantly triggered by the larger nucleation phase of the upcoming stick slip event which is an almost fully aseismic process.


Stability properties of dissipative evolution equations with nonautonomous and nonlinear damping. In this paper, we obtain some stability results of (abstract) dissipative evolution equations with a nonautonomous and nonlinear damping using the exponential stability of the retrograde problem with a linear and autonomous feedback and a comparison principle. We then illustrate our abstract statements for different concrete examples, where new results are achieved. In a preliminary step, we prove some well posedness results for some nonlinear and nonautonomous evolution equations.


Identification of a reversible catenary system by the measure of one compartment. The known results on compartment system's identification requires observations and measures on some compartments. However, in many situations some compartments can not be measured, so the minimization of the compartment's number to be measured is a very important problem to solve. In this work, this problem had been solved for a class of compartmental systems. It is proved that all reversible catenary system is identifiable by the measure of one compartment, the principal compartment.


Continuous Time Mixed State Branching Processes and Stochastic Equations. A continuous time mixed state branching process is constructed as the scaling limits of two type Galton Watson processes. The process can also be obtained by the pathwise unique solution to a stochastic equation system. From the stochastic equation system we derive the distribution of local jumps and the exponential ergodicity in Wasserstein type distances of the transition semigroup is given. Meanwhile, we study immigration structures associated with the process and prove the existence of the stationary distribution of the process with immigration.