Narasimhan Simha type metrics on strongly pseudoconvex domains in mathbb C n . For a bounded domain D subset mathbb C n , let K D K D(z) 0 denote the Bergman kernel on the diagonal and consider the reproducing kernel Hilbert space of holomorphic functions on D that are square integrable with respect to the weight K D d , where d geq 0 is an integer. The corresponding weighted kernel K D, d transforms appropriately under biholomorphisms and hence produces an invariant K " a hler metric on D . Thus, there is a hierarchy of such metrics starting with the classical Bergman metric that corresponds to the case d 0 . This note is an attempt to study this class of metrics in much the same way as the Bergman metric has been with a view towards identifying properties that are common to this family. When D is strongly pseudoconvex, the scaling principle is used to obtain the boundary asymptotics of these metrics and several invariants associated to them. It turns out that all these metrics are complete on strongly pseudoconvex domains.


What? Who? Why? Stellify. In his 1981 article, Roberts highlights the term 'stellify' defined as "to transform (a person or thing) into a star or constellation, to place among the stars." Using the case of the Tabwa people of central Africa, not the Democratic Republic of Congo, Roberts presents among other things the sky as a mnemonic for remembering migrations and remembering culture heroes. We do not know the details of the processes of stellification, however we do know what has been stellified in many cultures by examining their names for stars and asterisms and their skylore. Of the many ideas presented in his latest book, Aveni teases out the ideas of the sky stories having connections to celestial motions, as well as being a mnemonic for remembering seasonal activities and a mnemonic for remembering locally embedded moral, ethical, and sociocultural codes, thus overlapping with Roberts' supposition of the sky serving as a mnemonic. I draw on case studies to flesh out three themes 1. celestial motions, 2. moral, ethical, and sociocultural codes, and 3. seasonal activities within African sky stories. As previously stated, though the human process of assigning names and stories to the night sky as well as stellifying aspects of their lives is not fully understood, these three themes hold promise for being foundational if not part of every culture's practice of stellification.


Arrangements of orthogonal circles with many intersections. An arrangement of circles in which circles intersect only in angles of pi 2 is called an emph arrangement of orthogonal circles . We show that in the case that no two circles are nested, the intersection graph of such an arrangement is planar. The same result holds for arrangement of circles that intersect in an angle of at most pi 2 . For the general case we prove that the maximal number of edges in an intersection graph of an arrangement of orthogonal circles lies in between 4n O left( sqrt n right) and left(4 frac 5 11 right)n , for n being the number of circles. Based on the lower bound we can also improve the bound for the number of triangles in arrangements of orthogonal circles to (3 5 9)n O left( sqrt n right) .


MIRA Multihop Relation Prediction in Temporal Knowledge Graphs. In knowledge graph reasoning, we observe a trend to analyze temporal data evolving over time. The additional temporal dimension is attached to facts in a knowledge base resulting in quadruples between entities such as (Nintendo, released, Super Mario, Sep 13 1985), where the relation between two entities is associated to a specific time interval or point in time. Multi hop reasoning on inferred subgraphs connecting entities within a knowledge graph can be formulated as a reinforcement learning task where the agent sequentially performs inference upon the explored subgraph. The task in this work is to infer the predicate between a subject and an object entity, i.e., (subject, ?, object, time), being valid at a certain timestamp or time interval. Given query entities, our agent starts to gather temporal relevant information about the neighborhood of the subject and object. The encoding of information about the explored graph structures is referred to as fingerprints. Subsequently, we use the two fingerprints as input to a Q Network. Our agent decides sequentially which relational type needs to be explored next expanding the local subgraphs of the query entities in order to find promising paths between them. The evaluation shows that the proposed method not only yields results being in line with state of the art embedding algorithms for temporal Knowledge Graphs (tKG), but we also gain information about the relevant structures between subjects and objects.


Solution for Large scale Long tailed Recognition with Noisy Labels. This is a technical report for CVPR 2021 AliProducts Challenge. AliProducts Challenge is a competition proposed for studying the large scale and fine grained commodity image recognition problem encountered by worldleading ecommerce companies. The large scale product recognition simultaneously meets the challenge of noisy annotations, imbalanced (long tailed) data distribution and fine grained classification. In our solution, we adopt stateof the art model architectures of both CNNs and Transformer, including ResNeSt, EfficientNetV2, and DeiT. We found that iterative data cleaning, classifier weight normalization, high resolution finetuning, and test time augmentation are key components to improve the performance of training with the noisy and imbalanced dataset. Finally, we obtain 6.4365 mean class error rate in the leaderboard with our ensemble model.


Widen The Backdoor To Let More Attackers In. As collaborative learning and the outsourcing of data collection become more common, malicious actors (or agents) which attempt to manipulate the learning process face an additional obstacle as they compete with each other. In backdoor attacks, where an adversary attempts to poison a model by introducing malicious samples into the training data, adversaries have to consider that the presence of additional backdoor attackers may hamper the success of their own backdoor. In this paper, we investigate the scenario of a multi agent backdoor attack, where multiple non colluding attackers craft and insert triggered samples in a shared dataset which is used by a model (a defender) to learn a task. We discover a clear backfiring phenomenon increasing the number of attackers shrinks each attacker's attack success rate (ASR). We then exploit this phenomenon to minimize the collective ASR of attackers and maximize defender's robustness accuracy by (i) artificially augmenting the number of attackers, and (ii) indexing to remove the attacker's sub dataset from the model for inference, hence proposing 2 defenses.


Maximal and fractional maximal operators in the Lorentz Morrey spaces and their applications to the Bochner Riesz and Schrodinger type operators. The aim of this paper is to obtain boundedness conditions for the maximal function Mf and to prove the necessary and sufficient conditions for the fractional maximal oparator Ma in the Lorentz Morrey spaces which are a new class of functions. We get our main results by using the obtained sharp rearrangement estimates. The obtained results are applied to the boundedness of particular operators such as the Bochner Riesz operator and the Schrodinger type operators in the Lorentz Morrey spaces, where the nonnegative potential V belongs to the reverse Holder class.


Extension of the operation mode of back streaming white neutron beam at the China spallation neutron source. The back streaming white neutron source (Back n) is a comprehensive neutron experimental platform for nuclear data measurement, nuclear astrophysics, neutron irradiation, detector calibration, etc. In order to meet a variety of experimental requirements, an attempt of various combinations of collimators has been made for the current collimation system. The basic parameters such as beam flux and beam spot characteristics under different beam operation modes have been studied. According to the change of the CSNS proton beam operation mode, the influence of different proton beam spots on the Back n beam is also studied. The study finds that some beam modes with new collimator combinations can meet the experimental requirements and greatly shorten the experimental time. In addition, we have found a parameterized formula for the uniform beam spot of the high power proton beam based on the beam monitor data, which provides a more accurate proton beam source term for future white neutron beam physics and application researches.


Statistical Post Processing for Gridded Temperature Prediction Using Encoder Decoder Based Deep Convolutional Neural Networks. The Japan Meteorological Agency operates gridded temperature guidance to predict two dimensional snowfall amounts and precipitation types, e.g., rain and snow, because surface temperature is one of the key elements to predict them. Operational temperature guidance is based on the Kalman filter, which uses temperature observation and numerical weather prediction (NWP) outputs only around observation sites. Correcting a temperature field when NWP models incorrectly predict a front's location or when observed temperatures are extremely cold or hot has been challenging. In this study, an encoder decoder based convolutional neural network has been proposed to predict gridded temperatures at the surface around the Kanto region in Japan. Verification results showed that the proposed model greatly improves the operational guidance and can correct NWP model biases, such as a positional error of fronts and extreme temperatures.


KGR 4 Retrieval, Retrospect, Refine and Rethink for Commonsense Generation. Generative commonsense reasoning requires machines to generate sentences describing an everyday scenario given several concepts, which has attracted much attention recently. However, existing models cannot perform as well as humans, since sentences they produce are often implausible and grammatically incorrect. In this paper, inspired by the process of humans creating sentences, we propose a novel Knowledge enhanced Commonsense Generation framework, termed KGR 4, consisting of four stages Retrieval, Retrospect, Refine, Rethink. Under this framework, we first perform retrieval to search for relevant sentences from external corpus as the prototypes. Then, we train the generator that either edits or copies these prototypes to generate candidate sentences, of which potential errors will be fixed by an autoencoder based refiner. Finally, we select the output sentence from candidate sentences produced by generators with different hyper parameters. Experimental results and in depth analysis on the CommonGen benchmark strongly demonstrate the effectiveness of our framework. Particularly, KGR 4 obtains 33.56 SPICE points in the official leaderboard, outperforming the previously reported best result by 2.49 SPICE points and achieving state of the art performance.


ParaLaw Nets Cross lingual Sentence level Pretraining for Legal Text Processing. Ambiguity is a characteristic of natural language, which makes expression ideas flexible. However, in a domain that requires accurate statements, it becomes a barrier. Specifically, a single word can have many meanings and multiple words can have the same meaning. When translating a text into a foreign language, the translator needs to determine the exact meaning of each element in the original sentence to produce the correct translation sentence. From that observation, in this paper, we propose ParaLaw Nets, a pretrained model family using sentence level cross lingual information to reduce ambiguity and increase the performance in legal text processing. This approach achieved the best result in the Question Answering task of COLIEE 2021.


CGAN EB A Non parametric Empirical Bayes Method for Crash Hotspot Identification Using Conditional Generative Adversarial Networks A Simulated Crash Data Study. In this paper, a new non parametric empirical Bayes approach called CGAN EB is proposed for approximating empirical Bayes (EB) estimates in traffic locations (e.g., road segments) which benefits from the modeling advantages of deep neural networks, and its performance is compared in a simulation study with the traditional approach based on negative binomial model (NB EB). The NB EB uses negative binomial model in order to model the crash data and is the most common approach in practice. To model the crash data in the proposed CGAN EB, conditional generative adversarial network is used, which is a powerful deep neural network based method that can model any types of distributions. A number of simulation experiments are designed and conducted to evaluate the CGAN EB performance in different conditions and compare it with the NB EB. The results show that CGAN EB performs as well as NB EB when conditions favor the NB EB model (i.e. data conform to the assumptions of the NB model) and outperforms NB EB in experiments reflecting conditions frequently encountered in practice, specifically low sample means, and when crash frequency does not follow a log linear relationship with covariates.


KnowSR Knowledge Sharing among Homogeneous Agents in Multi agent Reinforcement Learning. Recently, deep reinforcement learning (RL) algorithms have made great progress in multi agent domain. However, due to characteristics of RL, training for complex tasks would be resource intensive and time consuming. To meet this challenge, mutual learning strategy between homogeneous agents is essential, which is under explored in previous studies, because most existing methods do not consider to use the knowledge of agent models. In this paper, we present an adaptation method of the majority of multi agent reinforcement learning (MARL) algorithms called KnowSR which takes advantage of the differences in learning between agents. We employ the idea of knowledge distillation (KD) to share knowledge among agents to shorten the training phase. To empirically demonstrate the robustness and effectiveness of KnowSR, we performed extensive experiments on state of the art MARL algorithms in collaborative and competitive scenarios. The results demonstrate that KnowSR outperforms recently reported methodologies, emphasizing the importance of the proposed knowledge sharing for MARL.


i Algebra Towards Interactive Interpretability of Deep Neural Networks. Providing explanations for deep neural networks (DNNs) is essential for their use in domains wherein the interpretability of decisions is a critical prerequisite. Despite the plethora of work on interpreting DNNs, most existing solutions offer interpretability in an ad hoc, one shot, and static manner, without accounting for the perception, understanding, or response of end users, resulting in their poor usability in practice. In this paper, we argue that DNN interpretability should be implemented as the interactions between users and models. We present i Algebra, a first of its kind interactive framework for interpreting DNNs. At its core is a library of atomic, composable operators, which explain model behaviors at varying input granularity, during different inference stages, and from distinct interpretation perspectives. Leveraging a declarative query language, users are enabled to build various analysis tools (e.g., "drill down", "comparative", "what if" analysis) via flexibly composing such operators. We prototype i Algebra and conduct user studies in a set of representative analysis tasks, including inspecting adversarial inputs, resolving model inconsistency, and cleansing contaminated data, all demonstrating its promising usability.


Overconvergent modular symboles and p adic L functions. We come back to the construction of p adic L functions attached to cusp forms of even weight k in the spirit of G. Stevens, R. Pollack 7 and M. Greenberg 3 with a new unified presentation including the non ordinary case. This construction is based on Stevens's modular symbols rather than q developments. We review the proofs in order to obtain an effective algorithm guaranteeing a given p adic accuracy.


Velocimetry in rapidly rotating convection spatial correlations, flow structures and length scales. Rotating Rayleigh B 'enard convection is an oft employed model system to evaluate the interplay of buoyant forcing and Coriolis forces due to rotation, an eminently relevant interaction of dynamical effects found in many geophysical and astrophysical flows. These flows display extreme values of the governing parameters large Rayleigh numbers Ra , quantifying the strength of thermal forcing, and small Ekman numbers E , a parameter inversely proportional to the rotation rate. This leads to the dominant geostrophic balance of forces in the flow between pressure gradient and Coriolis force. The so called geostrophic regime of rotating convection is difficult to study with laboratory experiments and numerical simulations given the requirements to attain simultaneously large Ra values and small values of E . Here, we use flow measurements using stereoscopic particle image velocimetry in a large scale rotating convection apparatus in a horizontal plane at mid height to study the rich flow phenomenology of the geostrophic regime of rotating convection. We quantify the horizontal length scales of the flow using spatial correlations of vertical velocity and vertical vorticity, reproducing features of the convective Taylor columns and plumes flow states both part of the geostrophic regime. Additionally, we find in this horizontal plane an organisation into a quadrupolar vortex at higher Rayleigh numbers starting from the plumes state.


Towards Accurate Alignment in Real time 3D Hand Mesh Reconstruction. 3D hand mesh reconstruction from RGB images facilitates many applications, including augmented reality (AR). However, this requires not only real time speed and accurate hand pose and shape but also plausible mesh image alignment. While existing works already achieve promising results, meeting all three requirements is very challenging. This paper presents a novel pipeline by decoupling the hand mesh reconstruction task into three stages a joint stage to predict hand joints and segmentation a mesh stage to predict a rough hand mesh and a refine stage to fine tune it with an offset mesh for mesh image alignment. With careful design in the network structure and in the loss functions, we can promote high quality finger level mesh image alignment and drive the models together to deliver real time predictions. Extensive quantitative and qualitative results on benchmark datasets demonstrate that the quality of our results outperforms the state of the art methods on hand mesh pose precision and hand image alignment. In the end, we also showcase several real time AR scenarios.


Modeling the COVID 19 pandemic A primer and overview of mathematical epidemiology. Since the start of the still ongoing COVID 19 pandemic, there have been many modeling efforts to assess several issues of importance to public health. In this work, we review the theory behind some important mathematical models that have been used to answer questions raised by the development of the pandemic. We start revisiting the basic properties of simple Kermack McKendrick type models. Then, we discuss extensions of such models and important epidemiological quantities applied to investigate the role of heterogeneity in disease transmission e.g. mixing functions and superspreading events, the impact of non pharmaceutical interventions in the control of the pandemic, vaccine deployment, herd immunity, viral evolution and the possibility of vaccine escape. From the perspective of mathematical epidemiology, we highlight the important properties, findings, and, of course, deficiencies, that all these models have.


Parking functions From combinatorics to probability. Suppose that m drivers each choose a preferred parking space in a linear car park with n spots. In order, each driver goes to their chosen spot and parks there if possible, and otherwise takes the next available spot if it exists. If all drivers park successfully, the sequence of choices is called a parking function. Classical parking functions correspond to the case m n we study here combinatorial and probabilistic aspects of this generalized case. We construct a family of bijections between parking functions text PF (m, n) with m cars and n spots and spanning forests mathscr F (n 1, n 1 m) with n 1 vertices and n 1 m distinct trees having specified roots. This leads to a bijective correspondence between text PF (m, n) and monomial terms in the associated Tutte polynomial of a disjoint union of n m 1 complete graphs. We present an identity between the "inversion enumerator" of spanning forests with fixed roots and the "displacement enumerator" of parking functions. The displacement is then related to the number of graphs on n 1 labeled vertices with a fixed number of edges, where the graph has n 1 m disjoint rooted components with specified roots. We investigate various probabilistic properties of a uniform parking function, giving a formula for the law of a single coordinate. As a side result we obtain a recurrence relation for the displacement enumerator. Adapting known results on random linear probes, we further deduce the covariance between two coordinates when m n .


TITAN T Cell Receptor Specificity Prediction with Bimodal Attention Networks. Motivation The activity of the adaptive immune system is governed by T cells and their specific T cell receptors (TCR), which selectively recognize foreign antigens. Recent advances in experimental techniques have enabled sequencing of TCRs and their antigenic targets (epitopes), allowing to research the missing link between TCR sequence and epitope binding specificity. Scarcity of data and a large sequence space make this task challenging, and to date only models limited to a small set of epitopes have achieved good performance. Here, we establish a k nearest neighbor (K NN) classifier as a strong baseline and then propose TITAN (Tcr epITope bimodal Attention Networks), a bimodal neural network that explicitly encodes both TCR sequences and epitopes to enable the independent study of generalization capabilities to unseen TCRs and or epitopes. Results By encoding epitopes at the atomic level with SMILES sequences, we leverage transfer learning and data augmentation to enrich the input data space and boost performance. TITAN achieves high performance in the prediction of specificity of unseen TCRs (ROC AUC 0.87 in 10 fold CV) and surpasses the results of the current state of the art (ImRex) by a large margin. Notably, our Levenshtein distance based K NN classifier also exhibits competitive performance on unseen TCRs. While the generalization to unseen epitopes remains challenging, we report two major breakthroughs. First, by dissecting the attention heatmaps, we demonstrate that the sparsity of available epitope data favors an implicit treatment of epitopes as classes. This may be a general problem that limits unseen epitope performance for sufficiently complex models. Second, we show that TITAN nevertheless exhibits significantly improved performance on unseen epitopes and is capable of focusing attention on chemically meaningful molecular structures.


Use of mathematical modelling to assess respiratory syncytial virus epidemiology and interventions A literature review. Respiratory syncytial virus (RSV) is a leading cause of acute lower respiratory tract infection worldwide, resulting in approximately sixty thousand annual hospitalizations of 5 year olds in the United States alone and three million annual hospitalizations globally. The development of over 40 vaccines and immunoprophylactic interventions targeting RSV has the potential to significantly reduce the disease burden from RSV infection in the near future. In the context of RSV, a highly contagious pathogen, dynamic transmission models (DTMs) are valuable tools in the evaluation and comparison of the effectiveness of different interventions. This review, the first of its kind for RSV DTMs, provides a valuable foundation for future modelling efforts and highlights important gaps in our understanding of RSV epidemics. Specifically, we have searched the literature using Web of Science, Scopus, Embase, and PubMed to identify all published manuscripts reporting the development of DTMs focused on the population transmission of RSV. We reviewed the resulting studies and summarized the structure, parameterization, and results of the models developed therein. We anticipate that future RSV DTMs, combined with cost effectiveness evaluations, will play a significant role in shaping decision making in the development and implementation of intervention programs.


Sharding and HTTP 2 Connection Reuse Revisited Why Are There Still Redundant Connections?. HTTP 2 and HTTP 3 avoid concurrent connections but instead multiplex requests over a single connection. Besides enabling new features, this reduces overhead and enables fair bandwidth sharing. Redundant connections should hence be a story of the past with HTTP 2. However, they still exist, potentially hindering innovation and performance. Thus, we measure their spread and analyze their causes in this paper. We find that 36 72 of the 6.24M HTTP Archive and 78 of the Alexa Top 100k websites cause Chromium based webbrowsers to open superfluous connections. We mainly attribute these to domain sharding, despite HTTP 2 efforts to revert it, and DNS load balancing, but also the Fetch Standard.


The gamma function and a certain sequence of differences. We study a sequence of differences related to the problem of finding the smallest factorial n! greater than or equal to a n , where a 1 , using the gamma function.


Quasi representations of groups and two homology. The Exel Loring formula asserts that two topological invariants associated to a pair of almost commuting unitary matrices coincide. Such a pair can be viewed as a quasi representation of mathbb Z 2 . We give a generalization of this formula for countable discrete groups. We also show the nontriviality of the corresponding invariants for quasidiagonal groups which are coarsely embeddable in a Hilbert space and have nonvanishing second Betti number.


Proof of a Conjecture on Hankel Determinants for Dyck Paths with Restricted Peak Heights. For any integer m geq 2 and r in 1, dots, m , let f n m,r denote the number of n Dyck paths whose peak's heights are im r for some integer i . We find the generating function of f n m,r satisfies a simple algebraic functional equation of degree 2 . The r m case is particularly nice and we give a combinatorial proof. By using the Sulanke and Xin's continued fraction method, we calculate the Hankel determinants for f n m,r . The special case r m of our result solves a conjecture proposed by Chien, Eu and Fu. We also enriched the class of eventually periodic Hankel determinant sequences.


The Cauchy problem for the energy critical inhomogeneous nonlinear Schr " o dinger equation with inverse square potential. In this paper, we study the Cauchy problem for the energy critical inhomogeneous nonlinear Schr " o dinger equation with inverse square potential iu t Delta u c x 2 u lambda x b u sigma u, u(0) u 0 in H 1 , (t,x) in mathbb R times mathbb R d , where d ge3 , lambda pm1 , 0 b 2 , sigma frac 4 2b d 2 and c c(d) left( frac d 2 2 right) 2 . We first prove the local well posedness as well as small data global well posedness and scattering in H 1 for c frac (d 2 2b) 2 4 (d 2 2b) 2 c(d) and 0 b frac 4 d , by using the contraction mapping principle based on the Strichartz estimates. Based on the local well posedness result, we then establish the blowup criteria for solutions to the equation in the focusing case lambda 1 . To this end, we derive the sharp Hardy Sobolev inequality and virial estimates related to this equation.


Rapid genetic screening with high quality factor metasurfaces. Genetic analysis methods are foundational to advancing personalized and preventative medicine, accelerating disease diagnostics, and monitoring the health of organisms and ecosystems. Current nucleic acid technologies such as polymerase chain reaction (PCR), next generation sequencing (NGS), and DNA microarrays rely on fluorescence and absorbance, necessitating sample amplification or replication and leading to increased processing time and cost. Here, we introduce a label free genetic screening platform based on high quality (high Q) factor silicon nanoantennas functionalized with monolayers of nucleic acid fragments. Each nanoantenna exhibits substantial electromagnetic field enhancements with sufficiently localized fields to ensure isolation from neighboring resonators, enabling dense biosensor integration. Quantitative detection of complementary target sequences via hybridization occurs simultaneously for arrays of sensing elements patterned at densities of 160,000 pixels per cm 2 . In physiological buffer, our nanoantennas exhibit average resonant quality factors of 2,200, allowing detection of purified SARS CoV 2 envelope (E) and open reading frame 1b (ORF1b) gene fragments with high sensitivity and specificity (up to 94 and 96 ) within 5 minutes of nucleic acid introduction. Combined with advances in nucleic acid extraction from complex samples (eg, mucus, blood, or wastewater), our work provides a foundation for rapid, compact, and high throughput multiplexed genetic screening assays spanning medical diagnostics to environmental monitoring.


SentEmojiBot Empathising Conversations Generation with Emojis. The increasing use of dialogue agents makes it extremely desirable for them to understand and acknowledge the implied emotions to respond like humans with empathy. Chatbots using traditional techniques analyze emotions based on the context and meaning of the text and lack the understanding of emotions expressed through face. Emojis representing facial expressions present a promising way to express emotions. However, none of the AI systems utilizes emojis for empathetic conversation generation. We propose, SentEmojiBot, based on the SentEmoji dataset, to generate empathetic conversations with a combination of emojis and text. Evaluation metrics show that the BERT based model outperforms the vanilla transformer model. A user study indicates that the dialogues generated by our model were understandable and adding emojis improved empathetic traits in conversations by 9.8


HASHTAG Hash Signatures for Online Detection of Fault Injection Attacks on Deep Neural Networks. We propose HASHTAG, the first framework that enables high accuracy detection of fault injection attacks on Deep Neural Networks (DNNs) with provable bounds on detection performance. Recent literature in fault injection attacks shows the severe DNN accuracy degradation caused by bit flips. In this scenario, the attacker changes a few weight bits during DNN execution by tampering with the program's DRAM memory. To detect runtime bit flips, HASHTAG extracts a unique signature from the benign DNN prior to deployment. The signature is later used to validate the integrity of the DNN and verify the inference output on the fly. We propose a novel sensitivity analysis scheme that accurately identifies the most vulnerable DNN layers to the fault injection attack. The DNN signature is then constructed by encoding the underlying weights in the vulnerable layers using a low collision hash function. When the DNN is deployed, new hashes are extracted from the target layers during inference and compared against the ground truth signatures. HASHTAG incorporates a lightweight methodology that ensures a low overhead and real time fault detection on embedded platforms. Extensive evaluations with the state of the art bit flip attack on various DNNs demonstrate the competitive advantage of HASHTAG in terms of both attack detection and execution overhead.


A Survey of Event triggered Control for Nonlinear Multiagent Systems with Guaranteed Steady State Performance. With the gradual advancement of a novel idea of the distributed control of the multiagent systems, an event triggered control protocol has received significant research attention, especially in designing the controller for the nonlinear multiagent system. Compared to other widely used control conditions, the event triggered control of the nonlinear system has a significant capability to improve resource utilization in real life scenarios such as using and controlling the intelligent control input of each agent. It is worth mentioning that a group of interconnected agents have a network communication topology to transmit the feedback information state across the networked link. The transmission of information among a group of agents ensures that each agent reaches the consensus agreement cooperatively. The cooperative protocol of the distributed control of nonlinear multiagent system also ensures the proper information flow between each agent, irrespective of communication delays, variability of environment, and switching of the communication topology via the event triggered control protocol. Consequently, event triggered control for nonlinear multi agent systems via steady state performance will be investigated in this paper. The steady state performances of a nonlinear closed loop system demonstrate the stabilization, output regulation, and output synchronization problem of the nonlinear system using proper control protocol to achieve a consensus in a multiagent system will also be discussed. Based on the steady state conditions of the nonlinear system, the consensus agreement among the agents will be realized.


An assessment of Sentinel 1 radar and Sentinel 2 multispectral data for remote archaeological investigation and preservation Qubbet el Hawa, Egypt. Remote sensing for archaeological investigations using surface response is reasonably well established, however, remote subsurface exploration is limited by depth and penetration and ground resolution. Furthermore, the conservation of archaeological sites requires constant monitoring capability, which is often not feasible between annual field seasons, but may be provided by modern satellite coverage. Here we develop an approach using Sentinel 1 C band radar backscatter, and Sentinel 2 multispectral data, to map and characterise the site of Qubbet el Hawa, Egypt. The multispectral bands analysed show similar sensitivity to satellite imagery. However, the radar backscatter is sensitive to exposed known structures, as well as disturbances to soil textural composition profile due to excavation erosion. Sub resolution features such as causeways manifest as a 'radar break' in the backscatter a discontinuity in otherwise continuous radar units. Furthermore, the finite subsurface response in the backscatter under the arid conditions of the site means we are able to delineate some shallow subsurface structures and map their orientation beneath the surface in areas not yet excavated. The sensitivity of Sentinel 1 backscatter to soil disturbance and human activity at Qubbet el Hawa, and the short ( 12 day) recurrence time of the satellites, makes it an important tool in heritage conservation.


Abelian Complex Structures and Generalizations. After a review on the development of deformation theory of abelian complex structures from both the classical and generalized sense, we propose the concept of semi abelian generalized complex structure. We present some observations on such structure and illustrate this new concept with a variety of examples.


BaLeNAS Differentiable Architecture Search via the Bayesian Learning Rule. Differentiable Architecture Search (DARTS) has received massive attention in recent years, mainly because it significantly reduces the computational cost through weight sharing and continuous relaxation. However, more recent works find that existing differentiable NAS techniques struggle to outperform naive baselines, yielding deteriorative architectures as the search proceeds. Rather than directly optimizing the architecture parameters, this paper formulates the neural architecture search as a distribution learning problem through relaxing the architecture weights into Gaussian distributions. By leveraging the natural gradient variational inference (NGVI), the architecture distribution can be easily optimized based on existing codebases without incurring more memory and computational consumption. We demonstrate how the differentiable NAS benefits from Bayesian principles, enhancing exploration and improving stability. The experimental results on NAS Bench 201 and NAS Bench 1shot1 benchmark datasets confirm the significant improvements the proposed framework can make. In addition, instead of simply applying the argmax on the learned parameters, we further leverage the recently proposed training free proxies in NAS to select the optimal architecture from a group architectures drawn from the optimized distribution, where we achieve state of the art results on the NAS Bench 201 and NAS Bench 1shot1 benchmarks. Our best architecture in the DARTS search space also obtains competitive test errors with 2.37 , 15.72 , and 24.2 on CIFAR 10, CIFAR 100, and ImageNet datasets, respectively.


Green's function related to a n order linear differential equation coupled to arbitrary linear non local boundary conditions. In this paper we obtain the explicit expression of the Green's function related to a general n order differential equation coupled to non local linear boundary conditions. In such boundary conditions, a n dimensional parameter dependence is also assumed. Moreover, some comparison principles are obtained. The explicit expression depends on the value of the Green's function related to the two point homogeneous problem, that is, we are assuming that when all the parameters involved on the boundary conditions take the value zero then the problem has a unique solution which is characterized by the corresponding Green's function g . The expression of the Green's function G of the general problem is given as a function of g and the real parameters considered at the boundary conditions. It is important to show that, in order to ensure the uniqueness of solutions of the linear considered problem, we must assume a non resonant additional condition on the considered problem, which depends on the non local conditions and the corresponding parameters. We point out that the assumption of the uniqueness of solutions of the two point homogeneous problem is not a necessary condition to ensure the solution of the general case. Of course, in this situation the expression we are looking for must be obtained in a different manner. To show the applicability of the obtained results, a particular example is given.


Gates are not what you need in RNNs. Recurrent neural networks have flourished in many areas. Consequently, we can see new RNN cells being developed continuously, usually by creating or using gates in a new, original way. But what if we told you that gates in RNNs are redundant? In this paper, we propose a new recurrent cell called Residual Recurrent Unit (RRU) which beats traditional cells and does not employ a single gate. It is based on the residual shortcut connection together with linear transformations, ReLU, and normalization. To evaluate our cell's effectiveness, we compare its performance against the widely used GRU and LSTM cells and the recently proposed Mogrifier LSTM on several tasks including, polyphonic music modeling, language modeling, and sentiment analysis. Our experiments show that RRU outperforms the traditional gated units on most of these tasks. Also, it has better robustness to parameter selection, allowing immediate application in new tasks without much tuning. We have implemented the RRU in TensorFlow, and the code is made available at https github.com LUMII Syslab RRU .


Learning Eye in Hand Camera Calibration from a Single Image. Eye in hand camera calibration is a fundamental and long studied problem in robotics. We present a study on using learning based methods for solving this problem online from a single RGB image, whilst training our models with entirely synthetic data. We study three main approaches one direct regression model that directly predicts the extrinsic matrix from an image, one sparse correspondence model that regresses 2D keypoints and then uses PnP, and one dense correspondence model that uses regressed depth and segmentation maps to enable ICP pose estimation. In our experiments, we benchmark these methods against each other and against well established classical methods, to find the surprising result that direct regression outperforms other approaches, and we perform noise sensitivity analysis to gain further insights into these results.


EENLP Cross lingual Eastern European NLP Index. This report presents the results of the EENLP project, done as a part of EEML 2021 summer school. It presents a broad index of NLP resources for Eastern European languages, which, we hope, could be helpful for the NLP community several new hand crafted cross lingual datasets focused on Eastern European languages, and a sketch evaluation of cross lingual transfer learning abilities of several modern multilingual Transformer based models.


Cohomological Mackey 2 functors. We show that the bicategory of finite groupoids and right free permutation bimodules is a quotient of the bicategory of Mackey 2 motives introduced in arXiv 1808.04902, obtained by modding out the so called cohomological relations. This categorifies Yoshida's Theorem for ordinary cohomological Mackey functors, and provides a direct connection between Mackey 2 motives and the usual blocks of representation theory.


Caching in Heterogeneous Satellite Networks with Fountain Codes. In this paper we investigate the performance of caching schemes based on fountain codes in a heterogeneous satellite network. We consider multiple cache aided hubs which are connected to a geostationary satellite through backhaul links. With the aimof reducing the average number of transmissions over the satellite backhaul link, we propose the use of a caching scheme based on fountain codes. We derive a simple analytical expression of the average backhaul transmission rate and provide a tightupper bound on it. Furthermore, we show how the performance of the fountain code based caching scheme is similar to that of a caching scheme based on maximum distance separable codes.


Efficient Low Latency Dynamic Licensing for Deep Neural Network Deployment on Edge Devices. Along with the rapid development in the field of artificial intelligence, especially deep learning, deep neural network applications are becoming more and more popular in reality. To be able to withstand the heavy load from mainstream users, deployment techniques are essential in bringing neural network models from research to production. Among the two popular computing topologies for deploying neural network models in production are cloud computing and edge computing. Recent advances in communication technologies, along with the great increase in the number of mobile devices, has made edge computing gradually become an inevitable trend. In this paper, we propose an architecture to solve deploying and processing deep neural networks on edge devices by leveraging their synergy with the cloud and the access control mechanisms of the database. Adopting this architecture allows low latency DNN model updates on devices. At the same time, with only one model deployed, we can easily make different versions of it by setting access permissions on the model weights. This method allows for dynamic model licensing, which benefits commercial applications.


VGNMN Video grounded Neural Module Network to Video Grounded Language Tasks. Neural module networks (NMN) have achieved success in image grounded tasks such as Visual Question Answering (VQA) on synthetic images. However, very limited work on NMN has been studied in the video grounded language tasks. These tasks extend the complexity of traditional visual tasks with the additional visual temporal variance. Motivated by recent NMN approaches on image grounded tasks, we introduce Video grounded Neural Module Network (VGNMN) to model the information retrieval process in video grounded language tasks as a pipeline of neural modules. VGNMN first decomposes all language components to explicitly resolve any entity references and detect corresponding action based inputs from the question. The detected entities and actions are used as parameters to instantiate neural module networks and extract visual cues from the video. Our experiments show that VGNMN can achieve promising performance on two video grounded language tasks video QA and video grounded dialogues.


Growth rates of the bipartite Erd H o s Gy ' a rf ' a s function. Given two graphs G, H and a positive integer q , an (H,q) coloring of G is an edge coloring of G such that every copy of H in G receives at least q distinct colors. The bipartite Erd H o s Gy ' a rf ' a s function r(K n,n , K s,t , q) is defined to be the minimum number of colors needed for K n,n to have a (K s,t , q) coloring. For balanced complete bipartite graphs K p,p , the function r(K n,n , K p,p , q) was studied systematically in Axenovich, F " u redi and Mubayi, it J. Combin. Theory Ser. B bf 79 (2000), 66 86 . In this paper, we study the asymptotic behavior of this function for complete bipartite graphs K s,t that are not necessarily balanced. Our main results deal with thresholds and lower and upper bounds for the growth rate of this function, in particular for (sub)linear and (sub)quadratic growth. We also obtain new lower bounds for the balanced bipartite case, and improve several results given by Axenovich, F " u redi and Mubayi. Our proof techniques are based on an extension to bipartite graphs of the recently developed Color Energy Method by Pohoata and Sheffer and its refinements, and a generalization of an old result due to Corr ' a di.


Adaptive Similarity Function with Structural Features of Network Embedding for Missing Link Prediction. Link prediction is a fundamental problem of data science, which usually calls for unfolding the mechanisms that govern the micro dynamics of networks. In this regard, using features obtained from network embedding for predicting links has drawn widespread attention. Though edge features based or node similarity based methods have been proposed to solve the link prediction problem, many technical challenges still exist due to the unique structural properties of networks, especially when the networks are sparse. From the graph mining perspective, we first give empirical evidence of the inconsistency between heuristic and learned edge features. Then we propose a novel link prediction framework, AdaSim, by introducing an Adaptive Similarity function using features obtained from network embedding based on random walks. The node feature representations are obtained by optimizing a graph based objective function. Instead of generating edge features using binary operators, we perform link prediction solely leveraging the node features of the network. We define a flexible similarity function with one tunable parameter, which serves as a penalty of the original similarity measure. The optimal value is learned through supervised learning thus is adaptive to data distribution. To evaluate the performance of our proposed algorithm, we conduct extensive experiments on eleven disparate networks of the real world. Experimental results show that AdaSim achieves better performance than state of the art algorithms and is robust to different sparsities of the networks.


TransMIL Transformer based Correlated Multiple Instance Learning for Whole Slide Image Classification. Multiple instance learning (MIL) is a powerful tool to solve the weakly supervised classification in whole slide image (WSI) based pathology diagnosis. However, the current MIL methods are usually based on independent and identical distribution hypothesis, thus neglect the correlation among different instances. To address this problem, we proposed a new framework, called correlated MIL, and provided a proof for convergence. Based on this framework, we devised a Transformer based MIL (TransMIL), which explored both morphological and spatial information. The proposed TransMIL can effectively deal with unbalanced balanced and binary multiple classification with great visualization and interpretability. We conducted various experiments for three different computational pathology problems and achieved better performance and faster convergence compared with state of the art methods. The test AUC for the binary tumor classification can be up to 93.09 over CAMELYON16 dataset. And the AUC over the cancer subtypes classification can be up to 96.03 and 98.82 over TCGA NSCLC dataset and TCGA RCC dataset, respectively. Implementation is available at https github.com szc19990412 TransMIL.


Wine is Not v i n. On the Compatibility of Tokenizations Across Languages. The size of the vocabulary is a central design choice in large pretrained language models, with respect to both performance and memory requirements. Typically, subword tokenization algorithms such as byte pair encoding and WordPiece are used. In this work, we investigate the compatibility of tokenizations for multilingual static and contextualized embedding spaces and propose a measure that reflects the compatibility of tokenizations across languages. Our goal is to prevent incompatible tokenizations, e.g., "wine" (word level) in English vs. "v i n" (character level) in French, which make it hard to learn good multilingual semantic representations. We show that our compatibility measure allows the system designer to create vocabularies across languages that are compatible a desideratum that so far has been neglected in multilingual models.


Spatial resolution of different discretizations over long time for the Dirac equation with small potentials. We compare the long time error bounds and spatial resolution of finite difference methods with different spatial discretizations for the Dirac equation with small electromagnetic potentials characterized by varepsilon in (0, 1 a dimensionless parameter. We begin with the simple and widely used finite difference time domain (FDTD) methods, and establish rigorous error bounds of them, which are valid up to the time at O(1 varepsilon) . In the error estimates, we pay particular attention to how the errors depend explicitly on the mesh size h and time step tau as well as the small parameter varepsilon . Based on the results, in order to obtain "correct" numerical solutions up to the time at O(1 varepsilon) , the varepsilon scalability (or meshing strategy requirement) of the FDTD methods should be taken as h O( varepsilon 1 2 ) and tau O( varepsilon 1 2 ) . To improve the spatial resolution capacity, we apply the Fourier spectral method to discretize the Dirac equation in space. Error bounds of the resulting finite difference Fourier pseudospectral (FDFP) methods show that they exhibit uniform spatial errors in the long time regime, which are optimal in space as suggested by the Shannon's sampling theorem. Extensive numerical results are reported to confirm the error bounds and demonstrate that they are sharp.


Learning to Generate 3D Shapes with Generative Cellular Automata. We present a probabilistic 3D generative model, named Generative Cellular Automata, which is able to produce diverse and high quality shapes. We formulate the shape generation process as sampling from the transition kernel of a Markov chain, where the sampling chain eventually evolves to the full shape of the learned distribution. The transition kernel employs the local update rules of cellular automata, effectively reducing the search space in a high resolution 3D grid space by exploiting the connectivity and sparsity of 3D shapes. Our progressive generation only focuses on the sparse set of occupied voxels and their neighborhood, thus enabling the utilization of an expressive sparse convolutional network. We propose an effective training scheme to obtain the local homogeneous rule of generative cellular automata with sequences that are slightly different from the sampling chain but converge to the full shapes in the training data. Extensive experiments on probabilistic shape completion and shape generation demonstrate that our method achieves competitive performance against recent methods.


Swimming of a ludion in a stratified sea. We describe and model experimental results on the dynamics of a "ludion" a neutrally buoyant body immersed in a layer of stably stratified salt water. By oscillating a piston inside a cylinder communicating with a narrow (in one of its horizontal dimensions) vessel containing the stably stratified layer of salt water, it is easy to periodically vary the hydrostatic pressure of the fluid. The ludion or Cartesian diver, initially positioned at its equilibrium height and free to move horizontally, can then oscillate vertically when forced by the pressure oscillations. Depending on the ratio of the forcing frequency to the Brunt V " a is " a l " a frequency of the stratified fluid, the ludion can emit its own internal gravity waves that we measure by a classical Particle Image Velocimetry technique. Our experimental results describe first the resonance of the vertical motions of the ludion when excited at different frequencies. A theoretical oscillator model is then derived taking into account added mass and added friction coefficients and its predictions are compared to the experimental data. Then, for the larger oscillation amplitudes, we observe and describe a bifurcation towards free horizontal motions. Although the internal gravity wave frequencies are affected by the Doppler shift induced by the horizontal displacement velocities, it seems that, contrary to surface waves associated with Couder walkers they are not the cause of the horizontal swimming. This does not however, exclude possible interactions between the ludion and internal gravity waves and possible hydrodynamic quantum analogies to be explored in the future.


Local newforms for the general linear groups over a non archimedean local field. In 12 , Jacquet Piatetskii Shapiro Shalika defined a family of compact open subgroups of p adic general linear groups indexed by non negative integers, and established the theory of local newforms for irreducible generic representations. In this paper, we extend their results to all irreducible representations. To do this, we define a new family of compact open subgroups indexed by certain tuples of non negative integers. For the proof, we introduce the Rankin Selberg integrals for Speh representations.


Learning Cloth Folding Tasks with Refined Flow Based Spatio Temporal Graphs. Cloth folding is a widespread domestic task that is seemingly performed by humans but which is highly challenging for autonomous robots to execute due to the highly deformable nature of textiles It is hard to engineer and learn manipulation pipelines to efficiently execute it. In this paper, we propose a new solution for robotic cloth folding (using a standard folding board) via learning from demonstrations. Our demonstration video encoding is based on a high level abstraction, namely, a refined optical flow based spatiotemporal graph, as opposed to a low level encoding such as image pixels. By constructing a new spatiotemporal graph with an advanced visual corresponding descriptor, the policy learning can focus on key points and relations with a 3D spatial configuration, which allows to quickly generalize across different environments. To further boost the policy searching, we combine optical flow and static motion saliency maps to discriminate the dominant motions for better handling the system dynamics in real time, which aligns with the attentional motion mechanism that dominates the human imitation process. To validate the proposed approach, we analyze the manual folding procedure and developed a custom made end effector to efficiently interact with the folding board. Multiple experiments on a real robotic platform were conducted to validate the effectiveness and robustness of the proposed method.


Job Posting Enriched Knowledge Graph for Skills based Matching. The labor market is constantly evolving. Occupations are changing, being added, or disappearing to fit the needs of today's market. In recent years the pace of this change has accelerated, due to factors such as globalization, digitization, and the shift to working from home. Different factors are relevant when selecting employment, e.g., cultural fit, compensation, provided degree of freedom. To successfully fulfill an occupation the gap between required (by the job) and possessed (by the job seeker) skills needs to be as small as possible. Decreasing this skill gap improves the fit between a job candidate and occupation. In this paper we propose a custom built Skills Occupation Knowledge Graph (KG) that fits the above described dynamic nature of the labor market, by leveraging existing skills and occupation taxonomies enriched with external job posting data. We leverage this KG and explore several applications for skills based matching of jobs to job seekers. First, we study link prediction as a means to quantify relevance of skills to occupations, which can help in prioritizing learning and development of employees. Next, we study node similarity methods and shortest path algorithms for career pathfinding. Finally, we leverage a term weighting method for identifying which skills are most "distinctive" for different (types of) occupations.


Super Resolution for Doubly Dispersive Channel Estimation. In this work we consider the problem of identification and reconstruction of doubly dispersive channel operators which are given by finite linear combinations of time frequency shifts. Such operators arise as time varying linear systems for example in radar and wireless communications. In particular, for information transmission in highly non stationary environments the channel needs to be estimated quickly with identification signals of short duration and for vehicular application simultaneous high resolution radar is desired as well. We consider the time continuous setting and prove an exact resampling reformulation of the involved channel operator when applied to a trigonometric polynomial as identifier in terms of sparse linear combinations of real valued atoms. Motivated by recent works of Heckel et al. we present an exact approach for off the grid superresolution which allows to perform the identification with realizable signals having compact support. Then we show how an alternating descent conditional gradient algorithm can be adapted to solve the reformulated problem. Numerical examples demonstrate the performance of this algorithm, in particular in comparison with a simple adaptive grid refinement strategy and an orthogonal matching pursuit algorithm.


Jointly Learning Clinical Entities and Relations with Contextual Language Models and Explicit Context. We hypothesize that explicit integration of contextual information into an Multi task Learning framework would emphasize the significance of context for boosting performance in jointly learning Named Entity Recognition (NER) and Relation Extraction (RE). Our work proves this hypothesis by segmenting entities from their surrounding context and by building contextual representations using each independent segment. This relation representation allows for a joint NER RE system that achieves near state of the art (SOTA) performance on both NER and RE tasks while beating the SOTA RE system at end to end NER RE with a 49.07 F1.


Electromagnetic induction physics, historical breakthroughs, epistemological issues and textbooks. The discovery of Electromagnetism by Oersted (1820) started an 'extraordinary decennium' ended by the discovery of electromagnetic induction by Faraday (1831). During this decennium, in several experiments, the electromagnetic induction was there, but it was not seen or recognized. Faraday built up a local theory of electromagnetic induction based on the idea that there is an induced current when there is an intersection between lines of magnetic force and a conductor in relative motion. In 1873, Maxwell, within a Lagrangian description of electric currents, wrote down a 'general law of electromagnetic induction' in which a fundamental role is played by the vector potential. A modern reformulation of Maxwell's general law is based on the definition of the induced emf as the line integral of the Lorentz force on a unit positive charge and the use of the equation that relates the electric field to the potentials. Maxwell's general law has been rapidly forgotten instead, the "flux rule" has deeply taken root. The "flux rule" not always yields the correct prediction, it does not say where the induced emf is localized, it requires ad hoc choices of the integration paths. It is not a physical law but only a calculation tool. To understand why Maxwell's general law' has been forgotten, also a sample of representative textbooks, distributed over about a century, has been analyzed. In this framework, a part of the paper deals with the idea of rotating lines of magnetic force, falsified by Faraday, but common in the first decades of the Twentieth century and astonishingly resumed recently. It is shown that this hypothesis is incompatible with Maxwell Lorentz Einstein electromagnetism and that it is falsified also by recent experiments. Finally, the electromagnetic induction in some recent research papers is briefly discussed.


Dynamic Convolution for 3D Point Cloud Instance Segmentation. We propose an approach to instance segmentation from 3D point clouds based on dynamic convolution. This enables it to adapt, at inference, to varying feature and object scales. Doing so avoids some pitfalls of bottom up approaches, including a dependence on hyper parameter tuning and heuristic post processing pipelines to compensate for the inevitable variability in object sizes, even within a single scene. The representation capability of the network is greatly improved by gathering homogeneous points that have identical semantic categories and close votes for the geometric centroids. Instances are then decoded via several simple convolution layers, where the parameters are generated conditioned on the input. The proposed approach is proposal free, and instead exploits a convolution process that adapts to the spatial and semantic characteristics of each instance. A light weight transformer, built on the bottleneck layer, allows the model to capture long range dependencies, with limited computational overhead. The result is a simple, efficient, and robust approach that yields strong performance on various datasets ScanNetV2, S3DIS, and PartNet. The consistent improvements on both voxel and point based architectures imply the effectiveness of the proposed method. Code is available at https git.io DyCo3D


The jump of the clique chromatic number of random graphs. The clique chromatic number of a graph is the smallest number of colors in a vertex coloring so that no maximal clique is monochromatic. In 2016 McDiarmid, Mitsche and Pralat noted that around p approx n 1 2 the clique chromatic number of the random graph G n,p changes by n Omega(1) when we increase the edge probability p by n o(1) , but left the details of this surprising phenomenon as an open problem. We settle this problem, i.e., resolve the nature of this polynomial jump' of the clique chromatic number of the random graph G n,p around edge probability p approx n 1 2 . Our proof uses a mix of approximation and concentration arguments, which enables us to (i) go beyond Janson's inequality used in previous work and (ii) determine the clique chromatic number of G n,p up to logarithmic factors for any edge probability p.


On homotopy groups of spaces of embeddings of an arc or a circle the Dax invariant. We compute in many classes of examples the first potentially interesting homotopy group of the space of embeddings of either an arc or a circle into a manifold M of dimension d geq4 . In particular, if M is a simply connected 4 manifold the fundamental group of both of these embedding spaces is isomorphic to the second homology group of M , answering a question posed by Arone and Szymik. The case d 3 gives isotopy invariants of knots in a 3 manifold, that are universal of Vassiliev type leq1 , and reduce to Schneiderman's concordance invariant.


Lipschitz mappings, metric differentiability, and factorization through metric trees. Given a Lipschitz map f from a cube into a metric space, we find several equivalent conditions for f to have a Lipschitz factorization through a metric tree. As an application we prove a recent conjecture of David and Schul. The techniques developed for the proof of the factorization result yield several other new and seemingly unrelated results. We prove that if f is a Lipschitz mapping from an open set in mathbb R n onto a metric space X , then the topological dimension of X equals n if and only if X has positive n dimensional Hausdorff measure. We also prove an area formula for length preserving maps between metric spaces, which gives, in particular, a new formula for integration on countably rectifiable sets in the Heisenberg group.


Exploring open ended gameplay features with Micro RollerCoaster Tycoon. This paper introduces MicroRCT, a novel open source simulator inspired by the theme park sandbox game RollerCoaster Tycoon. The goal in MicroRCT is to place rides and shops in an amusement park to maximize profit earned from park guests. Thus, the challenges for game AI include both selecting high earning attractions and placing them in locations that are convenient to guests. In this paper, the MAP Elites algorithm is used to generate a diversity of park layouts, exploring two theoretical questions about evolutionary algorithms and game design 1) Is there a benefit to starting from a minimal starting point for evolution and complexifying incrementally? and 2) What are the effects of resource limitations on creativity and optimization? Results indicate that building from scratch with no costs results in the widest diversity of high performing designs.


Non local Channel Aggregation Network for Single Image Rain Removal. Rain streaks showing in images or videos would severely degrade the performance of computer vision applications. Thus, it is of vital importance to remove rain streaks and facilitate our vision systems. While recent convolutinal neural network based methods have shown promising results in single image rain removal (SIRR), they fail to effectively capture long range location dependencies or aggregate convolutional channel information simultaneously. However, as SIRR is a highly illposed problem, these spatial and channel information are very important clues to solve SIRR. First, spatial information could help our model to understand the image context by gathering long range dependency location information hidden in the image. Second, aggregating channels could help our model to concentrate on channels more related to image background instead of rain streaks. In this paper, we propose a non local channel aggregation network (NCANet) to address the SIRR problem. NCANet models 2D rainy images as sequences of vectors in three directions, namely vertical direction, transverse direction and channel direction. Recurrently aggregating information from all three directions enables our model to capture the long range dependencies in both channels and spaitials locations. Extensive experiments on both heavy and light rain image data sets demonstrate the effectiveness of the proposed NCANet model.


Learning Chess Blindfolded Evaluating Language Models on State Tracking. Transformer language models have made tremendous strides in natural language understanding tasks. However, the complexity of natural language makes it challenging to ascertain how accurately these models are tracking the world state underlying the text. Motivated by this issue, we consider the task of language modeling for the game of chess. Unlike natural language, chess notations describe a simple, constrained, and deterministic domain. Moreover, we observe that the appropriate choice of chess notation allows for directly probing the world state, without requiring any additional probing related machinery. We find that (a) With enough training data, transformer language models can learn to track pieces and predict legal moves with high accuracy when trained solely on move sequences. (b) For small training sets providing access to board state information during training can yield significant improvements. (c) The success of transformer language models is dependent on access to the entire game history i.e. "full attention". Approximating this full attention results in a significant performance drop. We propose this testbed as a benchmark for future work on the development and analysis of transformer language models.


WorldKG A World Scale Geographic Knowledge Graph. OpenStreetMap is a rich source of openly available geographic information. However, the representation of geographic entities, e.g., buildings, mountains, and cities, within OpenStreetMap is highly heterogeneous, diverse, and incomplete. As a result, this rich data source is hardly usable for real world applications. This paper presents WorldKG a new geographic knowledge graph aiming to provide a comprehensive semantic representation of geographic entities in OpenStreetMap. We describe the WorldKG knowledge graph, including its ontology that builds the semantic dataset backbone, the extraction procedure of the ontology and geographic entities from OpenStreetMap, and the methods to enhance entity annotation. We perform statistical and qualitative dataset assessment, demonstrating the large scale and high precision of the semantic geographic information in WorldKG.


The "Kinesthetic HMD" Enhancing Self Motion Sensations in VR with Head Based Force Feedback. The sensation of self motion is essential in many virtual reality applications, from entertainment to training, such as flying and driving simulators. If the common approach used in amusement parks is to actuate the seats with cumbersome systems, multisensory integration can also be leveraged to get rich effects from lightweight solutions. In this short paper, we introduce a novel approach called the "Kinesthetic HMD" actuating a head mounted display with force feedback in order to provide sensations of self motion. We discuss its design considerations and demonstrate an augmented flight simulator use case with a proof of concept prototype. We conducted a user study assessing our approach's ability to enhance self motion sensations. Taken together, our results show that our Kinesthetic HMD provides significantly stronger and more egocentric sensations than a visual only self motion experience. Thus, by providing congruent vestibular and proprioceptive cues related to balance and self motion, the Kinesthetic HMD represents a promising approach for a variety of virtual reality applications in which motion sensations are prominent.


Arithmetic Progressions in Sumsets of Sparse Sets. A set of positive integers A subset mathbb Z 0 is emph log sparse if there is an absolute constant C so that for any positive integer x the sequence contains at most C elements in the interval x,2x) . In this note we study arithmetic progressions in sums of log sparse subsets of mathbb Z 0 . We prove that for any log sparse subsets S 1, dots, S n of mathbb Z 0 , the sumset S S 1 cdots S n cannot contain an arithmetic progression of size greater than n (1 o(1))n . We also show that this is nearly tight by proving that there exist log sparse sets S 1, dots, S n such that S 1 cdots S n contains an arithmetic progression of size n (1 o(1)) n .


Ubiquity of Fourier Transformation in Optical Sciences (Part II). This paper contains a transcript of my presentation at the Wyant Tribute Symposium on August 2, 2021 at SPIE's Optics Photonics conference in San Diego, California. The technical part of the paper has no overlap with a previous article of mine that was published in Applied Optics last year, bearing the same title as this one.1 The applications of Fourier transformation described in the present paper include the central limit theorem of probability and statistics, the Shannon Nyquist sampling theorem, and computing the electromagnetic field radiated by an oscillating magnetic dipole.


Accurate computational evolution of proteins and its dependence on deep learning. Enzyme is the major workhorse to carry out the diverse cellular functions. It catalyzes the biological reactions with a high specificity, with its topology playing a crucial role. For ecologically safe production of numerous bioproducts including drugs and chemicals, we have been striving to design the industrially useful enzyme molecules with highly improved catalytic capability. As the sequence space is enormous for an enzyme, its quick and effective exploration is quite improbable for the mutagenesis studies whose accuracy is greatly reliant on the prior information of the mutated sites and the extent of rigorous screening of the mutant libraries. Although directed evolution methods significantly aid the construction of a functionally improved molecule, their credibility depends on the successful excavation of the functionally similar sequence space in the available databases, encompassing billions of proteins. As deep learning methods aid us to extensively uncover the underlying network of all the key catalytic positions without any experimental data, their implementation has reliably increased the accuracy of directed evolution. The chapter comprehensively explains data mining and deep learning methods to further showcase their importance in enzyme engineering methods. The key biological and algorithmic limitations of these deep learning methodologies are lastly highlighted.


Semantic Code Search for Smart Contracts. Semantic code search technology allows searching for existing code snippets through natural language, which can greatly improve programming efficiency. Smart contracts, programs that run on the blockchain, have a code reuse rate of more than 90 , which means developers have a great demand for semantic code search tools. However, the existing code search models still have a semantic gap between code and query, and perform poorly on specialized queries of smart contracts. In this paper, we propose a Multi Modal Smart contract Code Search (MM SCS) model. Specifically, we construct a Contract Elements Dependency Graph (CEDG) for MM SCS as an additional modality to capture the data flow and control flow information of the code. To make the model more focused on the key contextual information, we use a multi head attention network to generate embeddings for code features. In addition, we use a fine tuned pretrained model to ensure the model's effectiveness when the training data is small. We compared MM SCS with four state of the art models on a dataset with 470K (code, docstring) pairs collected from Github and Etherscan. Experimental results show that MM SCS achieves an MRR (Mean Reciprocal Rank) of 0.572, outperforming four state of the art models UNIF, DeepCS, CARLCS CNN, and TAB CS by 34.2 , 59.3 , 36.8 , and 14.1 , respectively. Additionally, the search speed of MM SCS is second only to UNIF, reaching 0.34s query.


BBAM Bounding Box Attribution Map for Weakly Supervised Semantic and Instance Segmentation. Weakly supervised segmentation methods using bounding box annotations focus on obtaining a pixel level mask from each box containing an object. Existing methods typically depend on a class agnostic mask generator, which operates on the low level information intrinsic to an image. In this work, we utilize higher level information from the behavior of a trained object detector, by seeking the smallest areas of the image from which the object detector produces almost the same result as it does from the whole image. These areas constitute a bounding box attribution map (BBAM), which identifies the target object in its bounding box and thus serves as pseudo ground truth for weakly supervised semantic and instance segmentation. This approach significantly outperforms recent comparable techniques on both the PASCAL VOC and MS COCO benchmarks in weakly supervised semantic and instance segmentation. In addition, we provide a detailed analysis of our method, offering deeper insight into the behavior of the BBAM.


Lower Bounds for Unambiguous Automata via Communication Complexity. We use results from communication complexity, both new and old ones, to prove lower bounds for unambiguous finite automata (UFAs). We show three results. textit Complement There is a language L recognised by an n state UFA such that the complement language overline L requires NFAs with n tilde Omega ( log n) states. This improves on a lower bound by Raskin. textit Union There are languages L 1 , L 2 recognised by n state UFAs such that the union L 1 cup L 2 requires UFAs with n tilde Omega ( log n) states. textit Separation There is a language L such that both L and overline L are recognised by n state NFAs but such that L requires UFAs with n Omega( log n) states. This refutes a conjecture by Colcombet.


Piezoelectric metastructures for simultaneous broadband energy harvesting and vibration suppression of traveling waves. In this paper, we explore an electromechanical metastructure consisting of a periodic array of piezoelectric bimorphs with resistive inductive loads for simultaneous harvesting and attenuation of traveling wave energy. We develop fully coupled analytical models, i.e., an electroelastic transfer matrix method, and exploit both locally resonant and Bragg band gaps to achieve a multifunctional metastructure which is capable for maximum energy conversion and vibration mitigation in a broadband fashion. Our analytical and numerical results show that the proposed metastructure can achieve energy harvesting efficiency up to 95 at the local resonance frequency of 3.18 kHz, while reaching about 51 at 5.8 kHz near the upper limit of the Bragg band gap. The broadband vibration mitigation performance based on 50 power attenuation is predicted as 1.8 kHz and 1.1 kHz in the vicinity of the band gaps. The theoretical frameworks and the applicability of the proposed metastructure are validated using a full scale experimental setup.


Nonintrusive model order reduction for cross diffusion systems. In this paper, we investigate tensor based nonintrusive reduced order models (ROMs) for parametric cross diffusion equations. The full order model (FOM) consists of ordinary differential equations (ODEs) in matrix or tensor form resulting from finite difference discretization of the differential operators by taking the advantage of Kronecker structure. The matrix tensor differential equations are integrated in time with the implicit explicit (IMEX) Euler method. The reduced bases, relying on a finite sample set of parameter values, are constructed in form of a two level approach by applying higher order singular value decomposition (HOSVD) to the space time snapshots in tensor form, which leads to a large amount of computational and memory savings. The nonintrusive reduced approximations for an arbitrary parameter value are obtained through tensor product of the reduced basis by the parameter dependent core tensor that contains the reduced coefficients. The reduced coefficients for new parameter values are computed using radial basis function (RBF) interpolation. The efficiency of the proposed method is illustrated through numerical experiments for two dimensional Schnakenberg and three dimensional Brusselator cross diffusion equations. The spatiotemporal patterns are accurately predicted by the reduced order models with speed up factors of orders two and three over the full order models.


Optimal Beamwidth and Altitude for Maximal Uplink Coverage in Satellite Networks. Dense satellite constellations recently emerged as a prominent solution to complementing terrestrial networks in attaining true global coverage. As such, analytic optimization techniques can be adopted to rapidly maximize the benefits of such satellite networks. The paper presents a framework that relies on two primary tuning parameters to optimize the uplink performance (i) the constellation altitude and (ii) the satellite antenna beamwidth. The framework leverages tools from stochastic geometry to derive analytical models that formulate a parametric uplink coverage problem which also includes user traffic demand as an input. This allows operators to devise uplink expansion strategies to cater for expanding user demand. The framework demonstrates that fine tuning of these parameters can significantly enhance the network capacity. We show that the optimization of random constellations provides a close match to that of practical satellite constellations such as Walker delta and Walker star.


Are you doing what I say? On modalities alignment in ALFRED. ALFRED is a recently proposed benchmark that requires a model to complete tasks in simulated house environments specified by instructions in natural language. We hypothesize that key to success is accurately aligning the text modality with visual inputs. Motivated by this, we inspect how well existing models can align these modalities using our proposed intrinsic metric, boundary adherence score (BAS). The results show the previous models are indeed failing to perform proper alignment. To address this issue, we introduce approaches aimed at improving model alignment and demonstrate how improved alignment, improves end task performance.


Designing a Combinatorial Financial Options Market. Financial options are contracts that specify the right to buy or sell an underlying asset at a strike price by an expiration date. Standard exchanges offer options of predetermined strike values and trade options of different strikes independently, even for those written on the same underlying asset. Such independent market design can introduce arbitrage opportunities and lead to the thin market problem. The paper first proposes a mechanism that consolidates and matches orders on standard options related to the same underlying asset, while providing agents the flexibility to specify any custom strike value. The mechanism generalizes the classic double auction, runs in time polynomial to the number of orders, and poses no risk to the exchange, regardless of the value of the underlying asset at expiration. Empirical analysis on real market options data shows that the mechanism can find new matches for options of different strike prices and reduce bid ask spreads. Extending standard options written on a single asset, we propose and define a new derivative instrument combinatorial financial options that offer contract holders the right to buy or sell any linear combination of multiple underlying assets. We generalize our single asset mechanism to match options written on different combinations of assets, and prove that optimal clearing of combinatorial financial options is coNP hard. To facilitate market operations, we propose an algorithm that finds the exact optimal match through iterative constraint generation, and evaluate its performance on synthetically generated combinatorial options markets of different scales. As option prices reveal the market's collective belief of an underlying asset's future value, a combinatorial options market enables the expression of aggregate belief about future correlations among assets.


Targeted Muscle Effort Distribution with Exercise Robots Trajectory and Resistance Effects. The objective of this work is to relate muscle effort distributions to the trajectory and resistance settings of a robotic exercise and rehabilitation machine. Muscular effort distribution, representing the participation of each muscle in the training activity, was measured with electromyography sensors (EMG) and defined as the individual activation divided by the total muscle group activation. A four degrees of freedom robot and its impedance control system are used to create advanced exercise protocols whereby the user is asked to follow a path against the machine's neutral path and resistance. In this work, the robot establishes a zero effort circular path, and the subject is asked to follow an elliptical trajectory. The control system produces a user defined stiffness between the deviations from the neutral path and the torque applied by the subject. The trajectory and resistance settings used in the experiments were the orientation of the ellipse and a stiffness parameter. Multiple combinations of these parameters were used to measure their effects on the muscle effort distribution. An artificial neural network (ANN) used part of the data for training the model. Then, the accuracy of the model was evaluated using the rest of the data. The results show how the precision of the model is lost over time. These outcomes show the complexity of the muscle dynamics for long term estimations suggesting the existence of time varying dynamics possibly associated with fatigue.


Algorithmic Symplectic Packing. In this article we explore a symplectic packing problem where the targets and domains are 2n dimensional symplectic manifolds. We work in the context where the manifolds have first homology group equal to mathbb Z n , and we require the embeddings to induce isomorphisms between first homology groups. In this case, Maley, Mastrangeli and Traynor showed that the problem can be reduced to a combinatorial optimization problem, namely packing certain allowable simplices into a given standard simplex. They designed a computer program and presented computational results. In particular, they determined the simplex packing widths in dimension four for up to k 12 simplices, along with lower bounds for higher values of k . We present a modified algorithmic approach that allows us to determine the k simplex packing widths for up to k 13 simplices in dimension four and up to k 8 simplices in dimension six. Moreover, our approach determines all simplex multisets that allow for optimal packings.


Significance of the strain dominated region around a vortex on induced aerodynamic loads. The ability of vortices to induce aerodynamic loads on proximal surfaces plays a significant role in a wide variety of flows. However, most studies of vortex induced effects primarily focus on analyzing the influence of the rotation dominated cores of vortices. In this work, we show that not only are vortices in viscous flows surrounded by strain dominated regions, but that these regions are dynamically important and can sometimes even dictate the induced aerodynamic loads. We demonstrate this for a pitching airfoil, which exhibits dynamic stall and generates several force inducing vortices. Using a data driven force partitioning method, we quantify the influence of vortices as well as vortex associated strain to show that our current understanding of vortex dominated phenomena, such as dynamic stall, is incomplete without considering the substantial effect of strain dominated regions that are associated with vortices.


The Sensitivity of Word Embeddings based Author Detection Models to Semantic preserving Adversarial Perturbations. Authorship analysis is an important subject in the field of natural language processing. It allows the detection of the most likely writer of articles, news, books, or messages. This technique has multiple uses in tasks related to authorship attribution, detection of plagiarism, style analysis, sources of misinformation, etc. The focus of this paper is to explore the limitations and sensitiveness of established approaches to adversarial manipulations of inputs. To this end, and using those established techniques, we first developed an experimental frame work for author detection and input perturbations. Next, we experimentally evaluated the performance of the authorship detection model to a collection of semantic preserving adversarial perturbations of input narratives. Finally, we compare and analyze the effects of different perturbation strategies, input and model configurations, and the effects of these on the author detection model.


Blind Image Deblurring based on Kernel Mixture. Blind Image deblurring tries to estimate blurriness and a latent image out of a blurred image. This estimation, as being an ill posed problem, requires imposing restrictions on the latent image or a blur kernel that represents blurriness. Different from recent studies that impose some priors on the latent image, this paper regulates the structure of the blur kernel. We propose a kernel mixture structure while using the Gaussian kernel as a base kernel. By combining multiple Gaussian kernels structurally enhanced in terms of scales and centers, the kernel mixture becomes capable of modeling nearly non parametric shape of blurriness. A data driven decision for the number of base kernels to combine makes the structure even more flexible. We apply this approach to a remote sensing problem to recover images from blurry images of satellite. This case study shows the superiority of the proposed method regulating the blur kernel in comparison with state of the art methods that regulates the latent image.


Reformulating HOI Detection as Adaptive Set Prediction. Determining which image regions to concentrate on is critical for Human Object Interaction (HOI) detection. Conventional HOI detectors focus on either detected human and object pairs or pre defined interaction locations, which limits learning of the effective features. In this paper, we reformulate HOI detection as an adaptive set prediction problem, with this novel formulation, we propose an Adaptive Set based one stage framework (AS Net) with parallel instances and interaction branches. To attain this, we map a trainable interaction query set to an interaction prediction set with a transformer. Each query adaptively aggregates the interaction relevant features from global contexts through multi head co attention. Besides, the training process is supervised adaptively by matching each ground truth with the interaction prediction. Furthermore, we design an effective instance aware attention module to introduce instructive features from the instance branch into the interaction branch. Our method outperforms previous state of the art methods without any extra human pose and language features on three challenging HOI detection datasets. Especially, we achieve over 31 relative improvement on a large scale HICO DET dataset. Code is available at https github.com yoyomimi AS Net.


Twisted cubic and plane line incidence matrix in mathrm PG (3,q) . We consider the structure of the plane line incidence matrix of the projective space mathrm PG (3,q) with respect to the orbits of planes and lines under the stabilizer group of the twisted cubic. Structures of submatrices with incidences between a union of line orbits and an orbit of planes are investigated. For the unions consisting of two or three line orbits, the original submatrices are split into new ones, in which the incidences are also considered. For each submatrix (apart from the ones corresponding to a special type of lines), the numbers of lines in every plane and planes through every line are obtained. This corresponds to the numbers of ones in columns and rows of the submatrices.


Self intersections on a pair of pants. In this paper1 , we use the coding developed by R. Bowen and C. Series to compute the number of self intersections of a closed geodesic on a pair of pants. We give lower and upper bounds on the number of self intersections of a closed geodesic on a pair of pants. We prove a conjecture of Moira Chas and Anthony Phillips in 4 . We get also bounds for the number of closed geodesics whose self intersection number is very close to the maximal self intersection number on a pair of pants


Concept Drift Detection in Federated Networked Systems. As next generation networks materialize, increasing levels of intelligence are required. Federated Learning has been identified as a key enabling technology of intelligent and distributed networks however, it is prone to concept drift as with any machine learning application. Concept drift directly affects the model's performance and can result in severe consequences considering the critical and emergency services provided by modern networks. To mitigate the adverse effects of drift, this paper proposes a concept drift detection system leveraging the federated learning updates provided at each iteration of the federated training process. Using dimensionality reduction and clustering techniques, a framework that isolates the system's drifted nodes is presented through experiments using an Intelligent Transportation System as a use case. The presented work demonstrates that the proposed framework is able to detect drifted nodes in a variety of non iid scenarios at different stages of drift and different levels of system exposure.


Synthetic Acute Hypotension and Sepsis Datasets Based on MIMIC III and Published as Part of the Health Gym Project. These two synthetic datasets comprise vital signs, laboratory test results, administered fluid boluses and vasopressors for 3,910 patients with acute hypotension and for 2,164 patients with sepsis in the Intensive Care Unit (ICU). The patient cohorts were built using previously published inclusion and exclusion criteria and the data were created using Generative Adversarial Networks (GANs) and the MIMIC III Clinical Database. The risk of identity disclosure associated with the release of these data was estimated to be very low (0.045 ). The datasets were generated and published as part of the Health Gym, a project aiming to publicly distribute synthetic longitudinal health data for developing machine learning algorithms (with a particular focus on offline reinforcement learning) and for educational purposes.


Forecasting VaR and ES using a joint quantile regression and implications in portfolio allocation. In this paper we propose a multivariate quantile regression framework to forecast Value at Risk (VaR) and Expected Shortfall (ES) of multiple financial assets simultaneously, extending Taylor (2019). We generalize the Multivariate Asymmetric Laplace (MAL) joint quantile regression of Petrella and Raponi (2019) to a time varying setting, which allows us to specify a dynamic process for the evolution of both VaR and ES of each asset. The proposed methodology accounts for the dependence structure among asset returns. By exploiting the properties of the MAL distribution, we then propose a new portfolio optimization method that minimizes the portfolio risk and controls for well known characteristics of financial data. We evaluate the advantages of the proposed approach on both simulated and real data, using weekly returns on three major stock market indices. We show that our method outperforms other existing models and provides more accurate risk measure forecasts compared to univariate ones.


Bounded and unbounded cohomology of homeomorphism and diffeomorphism groups. We determine the bounded cohomology of the group of homeomorphisms of certain low dimensional manifolds. In particular, for the group of orientation preserving homeomorphisms of the circle and of the closed 2 disc, it is isomorphic to the polynomial ring generated by the bounded Euler class. These seem to be the first examples of groups for which the entire bounded cohomology can be described without being trivial. We further prove that, contrary to ordinary cohomology, the diffeomorphisms groups of the circle and of the closed 2 disc have the same bounded cohomology as their homeomorphism groups and that both differ from the ordinary cohomology. Finally, we determine the low dimensional bounded cohomology of homeo and diffeomorphism of the spheres S n and of certain 3 manifolds. In particular, we answer a question of Ghys by showing that the Euler class in H 4(Homeo circ(S 3)) is unbounded.


Gromov Hausdorff distance between filtered A infty categories 1 Lagrangian Floer theory. In this paper we introduce and study a distance, Gromov Hausdorff distance, which measures how two filtered A A infty categories are far away each other. In symplectic geometry the author associated a filtered A infty category, Fukaya category, to a finite set of Lagrangian submanifolds. The Gromov Hausdorff distance then gives a new invariant of a finite set of Lagrangian submanifolds. One can estimate it by the Hofer distance of Hamiltonian diffeomorphisms needed to send one Lagrangain submanifold to the other. A motivation to introduce Gromov Hausdorff distance is to obtain a certain completion of Fukaya category. If we have a sequence of sets of Lagrangian submanifolds, which is a Cauchy sequence in the sense of Hofer metric, then the associated filtered A infinity categories also form a Cauchy sequence in Gromov Hausdorff distance. In this paper we develop a theory to obtain an inductive limit of such a sequence of filtered A infty categories. In other words, we give an affirmative answer to Fu5 Conjecture 15.34.


Hybrid Contrastive Learning of Tri Modal Representation for Multimodal Sentiment Analysis. The wide application of smart devices enables the availability of multimodal data, which can be utilized in many tasks. In the field of multimodal sentiment analysis (MSA), most previous works focus on exploring intra and inter modal interactions. However, training a network with cross modal information (language, visual, audio) is still challenging due to the modality gap, and existing methods still cannot ensure to sufficiently learn intra inter modal dynamics. Besides, while learning dynamics within each sample draws great attention, the learning of inter class relationships is neglected. Moreover, the size of datasets limits the generalization ability of existing methods. To address the afore mentioned issues, we propose a novel framework HyCon for hybrid contrastive learning of tri modal representation. Specifically, we simultaneously perform intra inter modal contrastive learning and semi contrastive learning (that is why we call it hybrid contrastive learning), with which the model can fully explore cross modal interactions, preserve inter class relationships and reduce the modality gap. Besides, a refinement term is devised to prevent the model falling into a sub optimal solution. Moreover, HyCon can naturally generate a large amount of training pairs for better generalization and reduce the negative effect of limited datasets. Extensive experiments on public datasets demonstrate that our proposed method outperforms existing works.