Enhancing Human in the Loop Adaptive Systems through Digital Twins and VR Interfaces. Self adaptation approaches usually rely on closed loop controllers that avoid human intervention from adaptation. While such fully automated approaches have proven successful in many application domains, there are situations where human involvement in the adaptation process is beneficial or even necessary. For such "human in the loop" adaptive systems, two major challenges, namely transparency and controllability, have to be addressed to include the human in the self adaptation loop. Transparency means that relevant context information about the adaptive systems and its context is represented based on a digital twin enabling the human an immersive and realistic view. Concerning controllability, the decision making and adaptation operations should be managed in a natural and interactive way. As existing human in the loop adaptation approaches do not fully cover these aspects, we investigate alternative human in the loop strategies by using a combination of digital twins and virtual reality (VR) interfaces. Based on the concept of the digital twin, we represent a self adaptive system and its respective context in a virtual environment. With the help of a VR interface, we support an immersive and realistic human involvement in the self adaptation loop by mirroring the physical entities of the real world to the VR interface. For integrating the human in the decision making and adaptation process, we have implemented and analyzed two different human in the loop strategies in VR a procedural control where the human can control the decision making process and adaptations through VR interactions (human controlled) and a declarative control where the human specifies the goal state and the configuration is delegated to an AI planner (mixed initiative). We illustrate and evaluate our approach based on an autonomic robot system that is accessible and controlled through a VR interface.


WKB based scheme with adaptive step size control for the Schr "odinger equation in the highly oscillatory regime. This paper is concerned with an efficient numerical method for solving the 1D stationary Schr "odinger equation in the highly oscillatory regime. Being a hybrid, analytical numerical approach it does not have to resolve each oscillation, in contrast to standard schemes for ODEs. We build upon the WKB based (named after the physicists Wentzel, Kramers, Brillouin) marching method from 2 and extend it in two ways By comparing the mathcal O (h) and mathcal O (h 2 ) methods from 2 we design an adaptive step size controller for the WKB method. While this WKB method is very efficient in the highly oscillatory regime, it cannot be used close to turning points. Hence, we introduce for such regions an automated methods switching, choosing between the WKB method for the oscillatory region and a standard Runge Kutta Fehlberg 4(5) method in smooth regions. A similar approach was proposed recently in 9, 4 , however, only for an mathcal O (h) method. Hence, we compare our new strategy to their method on two examples (Airy function on the spatial interval 0, ,10 8 with one turning point at x 0 and on a parabolic cylinder function having two turning points), and illustrate the advantages of the new approach w.r.t. accuracy and efficiency.


On approximate operator representations of sequences in Banach spaces. Generalizing results by Halperin et al., Grivaux recently showed that any linearly independent sequence f k k 1 infty in a separable Banach space X can be represented as a suborbit T alpha(k) varphi k 1 infty of some bounded operator T X to X. In general, the operator T and the powers alpha(k) are not known explicitly. In this paper we consider approximate representations f k k 1 infty approx T alpha(k) varphi k 1 infty of certain types of sequences f k k 1 infty. In contrast to the results in the literature we are able to be very explicit about the operator T and suitable powers alpha(k), and we do not need to assume that the sequences are linearly independent. The exact meaning of approximation is defined in a way such that T alpha(k) varphi k 1 infty keeps essential features of f k k 1 infty, e.g., in the setting of atomic decompositions and Banach frames. We will present two different approaches. The first approach is universal, in the sense that it applies in general Banach spaces the technical conditions are typically easy to verify in sequence spaces, but are more complicated in function spaces. For this reason we present a second approach, directly tailored to the setting of Banach function spaces. A number of examples prove that the results apply in arbitrary weighted ell p spaces and L p spaces.


A quantitative stability inequality for fractional capacities. The aim of this work is to show a non sharp quantitative stability version of the fractional isocapacitary inequality. In particular, we provide a lower bound for the isocapacitary deficit in terms of the Fraenkel asymmetry. In addition, we provide the asymptotic behaviour of the s fractional capacity when s goes to 1 and the stability of our estimate with respect to the parameter s .


GasHis Transformer A Multi scale Visual Transformer Approach for Gastric Histopathology Image Classification. Existing deep learning methods for diagnosis of gastric cancer commonly use convolutional neural network. Recently, the Visual Transformer has attracted great attention because of its performance and efficiency, but its applications are mostly in the field of computer vision. In this paper, a multi scale visual transformer model, referred to as GasHis Transformer, is proposed for Gastric Histopathological Image Classification (GHIC), which enables the automatic classification of microscopic gastric images into abnormal and normal cases. The GasHis Transformer model consists of two key modules A global information module and a local information module to extract histopathological features effectively. In our experiments, a public hematoxylin and eosin (H E) stained gastric histopathological dataset with 280 abnormal and normal images are divided into training, validation and test sets by a ratio of 1 1 2. The GasHis Transformer model is applied to estimate precision, recall, F1 score and accuracy on the test set of gastric histopathological dataset as 98.0 , 100.0 , 96.0 and 98.0 , respectively. Furthermore, a critical study is conducted to evaluate the robustness of GasHis Transformer, where ten different noises including four adversarial attack and six conventional image noises are added. In addition, a clinically meaningful study is executed to test the gastrointestinal cancer identification performance of GasHis Transformer with 620 abnormal images and achieves 96.8 accuracy. Finally, a comparative study is performed to test the generalizability with both H E and immunohistochemical stained images on a lymphoma image dataset and a breast cancer dataset, producing comparable F1 scores (85.6 and 82.8 ) and accuracies (83.9 and 89.4 ), respectively. In conclusion, GasHisTransformer demonstrates high classification performance and shows its significant potential in the GHIC task.


Identifying the Limits of Cross Domain Knowledge Transfer for Pretrained Models. There is growing evidence that pretrained language models improve task specific fine tuning not just for the languages seen in pretraining, but also for new languages and even non linguistic data. What is the nature of this surprising cross domain transfer? We offer a partial answer via a systematic exploration of how much transfer occurs when models are denied any information about word identity via random scrambling. In four classification tasks and two sequence labeling tasks, we evaluate baseline models, LSTMs using GloVe embeddings, and BERT. We find that only BERT shows high rates of transfer into our scrambled domains, and for classification but not sequence labeling tasks. Our analyses seek to explain why transfer succeeds for some tasks but not others, to isolate the separate contributions of pretraining versus fine tuning, and to quantify the role of word frequency. These findings help explain where and why cross domain transfer occurs, which can guide future studies and practical fine tuning efforts.


On free semigroups of affine maps on the real line. In this note we generalise some of the work of Klarner on free semigroups of affine maps acting on the real line by using a classical approach from geometric group theory (the Ping Pong lemma). We also investigate the boundaries within which Klarner's necessary condition for a semigroup to be related is applicable.


Unsupervised Action Localization Crop in Video Retargeting for 3D ConvNets. Untrimmed videos on social media or those captured by robots and surveillance cameras are of varied aspect ratios. However, 3D CNNs usually require as input a square shaped video, whose spatial dimension is smaller than the original. Random or center cropping may leave out the video's subject altogether. To address this, we propose an unsupervised video cropping approach by shaping this as a retargeting and video to video synthesis problem. The synthesized video maintains a 1 1 aspect ratio, is smaller in size and is targeted at video subject(s) throughout the entire duration. First, action localization is performed on each frame by identifying patches with homogeneous motion patterns. Thus, a single salient patch is pinpointed per frame. But to avoid viewpoint jitters and flickering, any inter frame scale or position changes among the patches should be performed gradually over time. This issue is addressed with a polyBezier fitting in 3D space that passes through some chosen pivot timestamps and whose shape is influenced by the in between control timestamps. To corroborate the effectiveness of the proposed method, we evaluate the video classification task by comparing our dynamic cropping technique with random cropping on three benchmark datasets, viz. UCF 101, HMDB 51 and ActivityNet v1.3. The clip and top 1 accuracy for video classification after our cropping, outperform 3D CNN performances for same sized random crop inputs, also surpassing some larger random crop sizes.


SGToolkit An Interactive Gesture Authoring Toolkit for Embodied Conversational Agents. Non verbal behavior is essential for embodied agents like social robots, virtual avatars, and digital humans. Existing behavior authoring approaches including keyframe animation and motion capture are too expensive to use when there are numerous utterances requiring gestures. Automatic generation methods show promising results, but their output quality is not satisfactory yet, and it is hard to modify outputs as a gesture designer wants. We introduce a new gesture generation toolkit, named SGToolkit, which gives a higher quality output than automatic methods and is more efficient than manual authoring. For the toolkit, we propose a neural generative model that synthesizes gestures from speech and accommodates fine level pose controls and coarse level style controls from users. The user study with 24 participants showed that the toolkit is favorable over manual authoring, and the generated gestures were also human like and appropriate to input speech. The SGToolkit is platform agnostic, and the code is available at https github.com ai4r SGToolkit.


Blocking Trails for f factors of Multigraphs. Blocking flows, introduced by Dinic 2 for network flow, have been used to speed up many augmenting path type algorithms, especially matching algorithms e.g., 18, 23, 16 . We present an O(m) time algorithm for blocking trails for f factors of general multigraphs. This improves a previous algorithm by a factor of alpha(m,n) . This speeds up a number of efficient algorithms for f factors, e.g., the algorithm of 11 for maximum weight f matching improves by the aforementioned alpha(m,n) factor to running time O( sqrt Phi log Phi m log ( Phi W) ) for Phi sum v in V f(v) , W the maximum edge weight. This time bound is within a factor sqrt log Phi of the bound for bipartite multigraphs. The technical difficulty for this work stems from the fact that previous algorithms for both matching and f matching use vertex contractions to form blossoms, but our dfs based approach necessitates using edge contractions. As an example difficulty, edge contractions introduce a new form of blossoms we call "skew blossoms". These are configurations that must be reorganized in order to become valid blossoms.


Cnoidal Waves for the cubic nonlinear Klein Gordon and Schr "odinger Equations. In this paper, we establish orbital stability results of cnoidal periodic waves for the cubic nonlinear Klein Gordon and Schr "odinger equations. The spectral analysis for the corresponding linearized operator is established by using the Floquet theory and a Morse Index Theorem. First, we prove that the cnoidal waves for the cubic Klein Gordon equation are orbitally unstable as a direct application of Grillakis, Shatah and Strauss' theory. The orbital stability of cnoidal waves for the Schr "odinger equation is established by constructing a suitable Lyapunov functional restricted to the associated zero mean energy space.


Cyber Risk Frequency, Severity and Insurance Viability. In this study an exploration of insurance risk transfer is undertaken for the cyber insurance industry in the United States of America, based on the leading industry dataset of cyber events provided by Advisen. We seek to address two core unresolved questions. First, what factors are the most significant covariates that may explain the frequency and severity of cyber loss events and are they heterogeneous over cyber risk categories? Second, is cyber risk insurable in regards to the required premiums, risk pool sizes and how would this decision vary with the insured companies industry sector and size? We address these questions through a combination of regression models based on the class of Generalised Additive Models for Location Shape and Scale (GAMLSS) and a class of ordinal regressions. These models will then form the basis for our analysis of frequency and severity of cyber risk loss processes. We investigate the viability of insurance for cyber risk using a utility modelling framework with premium calculated by classical certainty equivalence analysis utilising the developed regression models. Our results provide several new key insights into the nature of insurability of cyber risk and rigorously address the two insurance questions posed in a real data driven case study analysis.


A note on the complexity of k Metric Dimension. Two vertices u, v in V of an undirected connected graph G (V,E) are resolved by a vertex w if the distance between u and w and the distance between v and w are different. A set R subseteq V of vertices is a k resolving set for G if for each pair of vertices u, v in V there are at least k distinct vertices w 1, ldots,w k in R such that each of them resolves u and v . The k Metric Dimension of G is the size of a smallest k resolving set for G . The decision problem k Metric Dimension is the question whether G has a k resolving set of size at most r , for a given graph G and a given number r . In this paper, we proof the NP completeness of k Metric Dimension for bipartite graphs and each k geq 2 .


On the String Topology Coproduct for Lie Groups. The free loop space of a Lie group is homeomorphic to the product of the Lie group itself and its based loop space. We show that the coproduct on the homology of the free loop space that was introduced by Goresky and Hingston splits into the diagonal map on the group and a based coproduct on the homology of the based loop space. This result implies that the coproduct is trivial for even dimensional Lie groups. Using results by Bott and Samelson, we show that the coproduct is trivial as well for a large family of simply connected Lie groups.


Fast discovery of multidimensional subsequences for robust trajectory classification. Trajectory classification tasks became more complex as large volumes of mobility data are being generated every day and enriched with new sources of information, such as social networks and IoT sensors. Fast classification algorithms are essential for discovering knowledge in trajectory data for real applications. In this work we propose a method for fast discovery of subtrajectories with the reduction of the search space and the optimization of the MASTERMovelets method, which has proven to be effective for discovering interpretable patterns in classification problems.


EfficientLPS Efficient LiDAR Panoptic Segmentation. Panoptic segmentation of point clouds is a crucial task that enables autonomous vehicles to comprehend their vicinity using their highly accurate and reliable LiDAR sensors. Existing top down approaches tackle this problem by either combining independent task specific networks or translating methods from the image domain ignoring the intricacies of LiDAR data and thus often resulting in sub optimal performance. In this paper, we present the novel top down Efficient LiDAR Panoptic Segmentation (EfficientLPS) architecture that addresses multiple challenges in segmenting LiDAR point clouds including distance dependent sparsity, severe occlusions, large scale variations, and re projection errors. EfficientLPS comprises of a novel shared backbone that encodes with strengthened geometric transformation modeling capacity and aggregates semantically rich range aware multi scale features. It incorporates new scale invariant semantic and instance segmentation heads along with the panoptic fusion module which is supervised by our proposed panoptic periphery loss function. Additionally, we formulate a regularized pseudo labeling framework to further improve the performance of EfficientLPS by training on unlabelled data. We benchmark our proposed model on two large scale LiDAR datasets nuScenes, for which we also provide ground truth annotations, and SemanticKITTI. Notably, EfficientLPS sets the new state of the art on both these datasets.


Continuum Simple Management of Complex Continual Learning Scenarios. Continual learning is a machine learning sub field specialized in settings with non iid data. Hence, the training data distribution is not static and drifts through time. Those drifts might cause interferences in the trained model and knowledge learned on previous states of the data distribution might be forgotten. Continual learning's challenge is to create algorithms able to learn an ever growing amount of knowledge while dealing with data distribution drifts. One implementation difficulty in these field is to create data loaders that simulate non iid scenarios. Indeed, data loaders are a key component for continual algorithms. They should be carefully designed and reproducible. Small errors in data loaders have a critical impact on algorithm results, e.g. with bad preprocessing, wrong order of data or bad test set. Continuum is a simple and efficient framework with numerous data loaders that avoid researcher to spend time on designing data loader and eliminate time consuming errors. Using our proposed framework, it is possible to directly focus on the model design by using the multiple scenarios and evaluation metrics implemented. Furthermore the framework is easily extendable to add novel settings for specific needs.


Resilient Level Ancestor, Bottleneck, and Lowest Common Ancestor Queries in Dynamic Trees. We study the problem of designing a emph resilient data structure maintaining a tree under the Faulty RAM model Finocchi and Italiano, STOC'04 in which up to delta memory words can be corrupted by an adversary. Our data structure stores a rooted dynamic tree that can be updated via the addition of new leaves, requires linear size, and supports emph resilient (weighted) level ancestor queries, lowest common ancestor queries, and bottleneck vertex queries in O( delta) worst case time per operation.


Automated Vision Based Wellness Analysis for Elderly Care Centers. The growth in the aging population requires caregivers to improve both efficiency and quality of healthcare. In this study, we develop an automatic, vision based system for monitoring and analyzing the physical and mental well being of senior citizens. Through collaboration with Haven of Hope Christian Service, we collect video recording data in the care center with surveillance cameras. We then process and extract personalized facial, activity, and interaction features from the video data using deep neural networks. This integrated health information systems can assist caregivers to gain better insights into the seniors they are taking care of. These insights, including wellness metrics and long term health patterns of senior citizens, can help caregivers update their caregiving strategies. We report the findings of our analysis and evaluate the system quantitatively. We also summarize technical challenges and additional functionalities and technologies needed for offering a comprehensive system.


Scalable Surface Reconstruction with Delaunay Graph Neural Networks. We introduce a novel learning based, visibility aware, surface reconstruction method for large scale, defect laden point clouds. Our approach can cope with the scale and variety of point cloud defects encountered in real life Multi View Stereo (MVS) acquisitions. Our method relies on a 3D Delaunay tetrahedralization whose cells are classified as inside or outside the surface by a graph neural network and an energy model solvable with a graph cut. Our model, making use of both local geometric attributes and line of sight visibility information, is able to learn a visibility model from a small amount of synthetic training data and generalizes to real life acquisitions. Combining the efficiency of deep learning methods and the scalability of energy based models, our approach outperforms both learning and non learning based reconstruction algorithms on two publicly available reconstruction benchmarks. Our code and data is available at https github.com raphaelsulzer dgnn.


A Contrastive Learning Approach to Auroral Identification and Classification. Unsupervised learning algorithms are beginning to achieve accuracies comparable to their supervised counterparts on benchmark computer vision tasks, but their utility for practical applications has not yet been demonstrated. In this work, we present a novel application of unsupervised learning to the task of auroral image classification. Specifically, we modify and adapt the Simple framework for Contrastive Learning of Representations (SimCLR) algorithm to learn representations of auroral images in a recently released auroral image dataset constructed using image data from Time History of Events and Macroscale Interactions during Substorms (THEMIS) all sky imagers. We demonstrate that (a) simple linear classifiers fit to the learned representations of the images achieve state of the art classification performance, improving the classification accuracy by almost 10 percentage points over the current benchmark and (b) the learned representations naturally cluster into more clusters than exist manually assigned categories, suggesting that existing categorizations are overly coarse and may obscure important connections between auroral types, near earth solar wind conditions, and geomagnetic disturbances at the earth's surface. Moreover, our model is much lighter than the previous benchmark on this dataset, requiring in the area of fewer than 25 of the number of parameters. Our approach exceeds an established threshold for operational purposes, demonstrating readiness for deployment and utilization.


Neural GIF Neural Generalized Implicit Functions for Animating People in Clothing. We present Neural Generalized Implicit Functions(Neural GIF), to animate people in clothing as a function of the body pose. Given a sequence of scans of a subject in various poses, we learn to animate the character for new poses. Existing methods have relied on template based representations of the human body (or clothing). However such models usually have fixed and limited resolutions, require difficult data pre processing steps and cannot be used with complex clothing. We draw inspiration from template based methods, which factorize motion into articulation and non rigid deformation, but generalize this concept for implicit shape learning to obtain a more flexible model. We learn to map every point in the space to a canonical space, where a learned deformation field is applied to model non rigid effects, before evaluating the signed distance field. Our formulation allows the learning of complex and non rigid deformations of clothing and soft tissue, without computing a template registration as it is common with current approaches. Neural GIF can be trained on raw 3D scans and reconstructs detailed complex surface geometry and deformations. Moreover, the model can generalize to new poses. We evaluate our method on a variety of characters from different public datasets in diverse clothing styles and show significant improvements over baseline methods, quantitatively and qualitatively. We also extend our model to multiple shape setting. To stimulate further research, we will make the model, code and data publicly available at https virtualhumans.mpi inf.mpg.de neuralgif


Man versus Machine AutoML and Human Experts' Role in Phishing Detection. Machine learning (ML) has developed rapidly in the past few years and has successfully been utilized for a broad range of tasks, including phishing detection. However, building an effective ML based detection system is not a trivial task, and requires data scientists with knowledge of the relevant domain. Automated Machine Learning (AutoML) frameworks have received a lot of attention in recent years, enabling non ML experts in building a machine learning model. This brings to an intriguing question of whether AutoML can outperform the results achieved by human data scientists. Our paper compares the performances of six well known, state of the art AutoML frameworks on ten different phishing datasets to see whether AutoML based models can outperform manually crafted machine learning models. Our results indicate that AutoML based models are able to outperform manually developed machine learning models in complex classification tasks, specifically in datasets where the features are not quite discriminative, and datasets with overlapping classes or relatively high degrees of non linearity. Challenges also remain in building a real world phishing detection system using AutoML frameworks due to the current support only on supervised classification problems, leading to the need for labeled data, and the inability to update the AutoML based models incrementally. This indicates that experts with knowledge in the domain of phishing and cybersecurity are still essential in the loop of the phishing detection pipeline.


On the mathbb A 1 invariance of mathrm K 2 modeled on linear and even orthogonal groups. Let k be an arbitrary field. In this paper we show that in the linear case ( Phi mathsf A ell , ell geq 4 ) and even orthogonal case ( Phi mathsf D ell , ell geq 7 , mathrm char (k) neq 2 ) the unstable functor mathrm K 2( Phi, ) possesses the mathbb A 1 invariance property in the geometric case, i. e. mathrm K 2( Phi, R t ) mathrm K 2( Phi, R) for a regular ring R containing k . As a consequence, the unstable mathrm K 2 groups can be represented in the unstable mathbb A 1 homotopy category mathcal H mathbb A 1 k as fundamental groups of the simply connected Chevalley Demazure group schemes mathrm G ( Phi, ) . Our invariance result can be considered as the mathrm K 2 analogue of the geometric case of Bass Quillen conjecture. We also show for a semilocal regular k algebra A that mathrm K 2( Phi, A) embeds as a subgroup into mathrm K mathrm M 2( mathrm Frac (A)) .


Uniform subconvex bounds for Rankin Selberg L functions. Let f be a Maass cusp form for rm SL 2( mathbb Z ) with Laplace eigenvalue 1 4 mu f 2 , mu f 0 . Let g be an arbitrary but fixed holomorphic or Maass cusp form for rm SL 2( mathbb Z ) . In this paper, we establish the following uniform subconvexity bound for the Rankin Selberg L function L(s,f otimes g) L left(1 2 it,f otimes g right) ll ( mu f t ) 9 10 varepsilon , where the implied constant depends only on varepsilon and g .


Joint Graph Learning and Matching for Semantic Feature Correspondence. In recent years, powered by the learned discriminative representation via graph neural network (GNN) models, deep graph matching methods have made great progresses in the task of matching semantic features. However, these methods usually rely on heuristically generated graph patterns, which may introduce unreliable relationships to hurt the matching performance. In this paper, we propose a joint emph graph learning and matching network, named GLAM, to explore reliable graph structures for boosting graph matching. GLAM adopts a pure attention based framework for both graph learning and graph matching. Specifically, it employs two types of attention mechanisms, self attention and cross attention for the task. The self attention discovers the relationships between features and to further update feature representations over the learnt structures and the cross attention computes cross graph correlations between the two feature sets to be matched for feature reconstruction. Moreover, the final matching solution is directly derived from the output of the cross attention layer, without employing a specific matching decision module. The proposed method is evaluated on three popular visual matching benchmarks (Pascal VOC, Willow Object and SPair 71k), and it outperforms previous state of the art graph matching methods by significant margins on all benchmarks. Furthermore, the graph patterns learnt by our model are validated to be able to remarkably enhance previous deep graph matching methods by replacing their handcrafted graph structures with the learnt ones.


Fourvolutions and automorphism groups of orbifold lattice vertex operator algebras. Let L be an even positive definite lattice with no roots, i.e., L(2) x in L mid (x x) 2 emptyset . Let g in O(L) be an isometry of order 4 such that g 2 1 on L . In this article, we determine the full automorphism group of the orbifold vertex operator algebra V L hat g . As our main result, we show that Aut(V L hat g ) is isomorphic to N Aut(V L) ( langle hat g rangle) langle hat g rangle unless L cong sqrt 2 E 8 or BW 16 .


Cyclic subspace codes via the sum of Sidon spaces. Subspace codes, especially cyclic constant subspace codes, are of great use in random network coding. Subspace codes can be constructed by subspaces and subspace polynomials. In particular, many researchers are keen to find special subspaces and subspace polynomials to construct subspace codes with the size and the minimum distance as large as possible. In 14 , Roth, Raviv and Tamo constructed several subspace codes using Sidon spaces, and it is proved that subspace codes constructed by Sidon spaces has the largest size and minimum distance. In 12 , Niu, Yue and Wu extended some results of 14 and obtained several new subspace codes. In this paper, we first provide a sufficient condition for the sum of Sidon spaces is again a Sidon space. Based on this result, we obtain new cyclic constant subspace codes through the sum of two and three Sidon spaces. Our results generalize the results in 14 and 12 .


Large Scale Spatio Temporal Person Re identification Algorithms and Benchmark. Person re identification (re ID) in the scenario with large spatial and temporal spans has not been fully explored. This is partially because that, existing benchmark datasets were mainly collected with limited spatial and temporal ranges, e.g., using videos recorded in a few days by cameras in a specific region of the campus. Such limited spatial and temporal ranges make it hard to simulate the difficulties of person re ID in real scenarios. In this work, we contribute a novel Large scale Spatio Temporal LaST person re ID dataset, including 10,862 identities with more than 228k images. Compared with existing datasets, LaST presents more challenging and high diversity re ID settings, and significantly larger spatial and temporal ranges. For instance, each person can appear in different cities or countries, and in various time slots from daytime to night, and in different seasons from spring to winter. To our best knowledge, LaST is a novel person re ID dataset with the largest spatio temporal ranges. Based on LaST, we verified its challenge by conducting a comprehensive performance evaluation of 14 re ID algorithms. We further propose an easy to implement baseline that works well on such challenging re ID setting. We also verified that models pre trained on LaST can generalize well on existing datasets with short term and cloth changing scenarios. We expect LaST to inspire future works toward more realistic and challenging re ID tasks. More information about the dataset is available at https github.com shuxjweb last.git.


Sub femtosecond wakefield injector and accelerator based on an undulating plasma bubble controlled by a laser phase. We demonstrate that a long propagating plasma bubble executing undulatory motion can be produced in the wake of two co propagating laser pulses a near single cycle injector and a multi cycle driver. When the undulation amplitude exceeds the analytically derived threshold, highly localized injections of plasma electrons into the bubble are followed by their long distance acceleration. While the locations of the injection regions are controlled by the carrier envelope phase (CEP) of the injector pulse, the mono energetic spectrum of the accelerated sub femtosecond high charge electron bunches is shown to be nearly CEP independent


Accuracy Enhancing Interface Treatment Algorithm the Back and Forth Error Compensation and Correction method. The accuracy of information transmission while solving domain decomposed problems is crucial to smooth transition of a solution around the interface overlapping region. This paper describes a systematical study on an accuracy enhancing interface treatment algorithm based on the back and forth error compensation and correction method (BFECC). By repetitively employing a low order interpolation technique (normally local 2 ts nd order) 3 times, this algorithm achieves local 3 ts rd order accuracy. Analytical derivations for 1D 2D cases are made, and the "super convergence" phenomenon (4 ts th order accuracy) is found for specific positioning of the donor and target grids. A set of numerical experiments based on various relative displacements, relative rotations, mesh ratios, and meshes with perturbation have been tested, and the results match the derivations. Different interface treatments are compared with 3D examples corner flow and cavity flow. The component convergence rate analysis shows that the BFECC method positively affects the accuracy of solutions.


Simple but Effective CLIP Embeddings for Embodied AI. Contrastive language image pretraining (CLIP) encoders have been shown to be beneficial for a range of visual tasks from classification and detection to captioning and image manipulation. We investigate the effectiveness of CLIP visual backbones for Embodied AI tasks. We build incredibly simple baselines, named EmbCLIP, with no task specific architectures, inductive biases (such as the use of semantic maps), auxiliary tasks during training, or depth maps yet we find that our improved baselines perform very well across a range of tasks and simulators. EmbCLIP tops the RoboTHOR ObjectNav leaderboard by a huge margin of 20 pts (Success Rate). It tops the iTHOR 1 Phase Rearrangement leaderboard, beating the next best submission, which employs Active Neural Mapping, and more than doubling the Fixed Strict metric (0.08 to 0.17). It also beats the winners of the 2021 Habitat ObjectNav Challenge, which employ auxiliary tasks, depth maps, and human demonstrations, and those of the 2019 Habitat PointNav Challenge. We evaluate the ability of CLIP's visual representations at capturing semantic information about input observations primitives that are useful for navigation heavy embodied tasks and find that CLIP's representations encode these primitives more effectively than ImageNet pretrained backbones. Finally, we extend one of our baselines, producing an agent capable of zero shot object navigation that can navigate to objects that were not used as targets during training. Our code and models are available at https github.com allenai embodied clip


Thin shell theory for rotationally invariant random simplices. For fixed functions G,H 0, infty) to 0, infty) , consider the rotationally invariant probability density on mathbb R n of the form mu n(ds) frac 1 Z n G( s 2) , e n H( s 2) ds. We show that when n is large, the Euclidean norm Y n 2 of a random vector Y n distributed according to mu n satisfies a Gaussian thin shell property the distribution of Y n 2 concentrates around a certain value s 0 , and the fluctuations of Y n 2 are approximately Gaussian with the order 1 sqrt n . We apply this thin shell property to the study of rotationally invariant random simplices, simplices whose vertices consist of the origin as well as independent random vectors Y 1 n, ldots,Y p n distributed according to mu n . We show that the logarithmic volume of the resulting simplex exhibits highly Gaussian behavior, providing a generalizing and unifying setting for the objects considered in Grote Kabluchko Th "ale Limit theorems for random simplices in high dimensions, ALEA, Lat. Am. J. Probab. Math. Stat. 16, 141 177 (2019) . Finally, by relating the volumes of random simplices to random determinants, we show that if A n is an n times n random matrix whose entries are independent standard Gaussian random variables, then there are explicit constants c 0,c 1 in(0, infty) and an absolute constant C in(0, infty) such that sup s in mathbb R left mathbb P left frac log mathrm det (A n) log(n 1)! c 0 sqrt frac 1 2 log n c 1 s right int infty s frac e u 2 2 du sqrt 2 pi right frac C log 3 2 n , sharpening the 1 log 1 3 o(1) n bound in Nguyen and Vu Random matrices Law of the determinant, Ann. Probab. 42 (1) (2014), 146 167 .


An inverse approach to hyperspheres of prescribed mean curvature in Euclidean space. We construct families of smooth functions H colon mathbb R n 1 to mathbb R such that the Euclidean (n 1) space is completely filled by not necessarily round hyperspheres of mean curvature H at every point.


DeepSets and their derivative networks for solving symmetric PDEs. Machine learning methods for solving nonlinear partial differential equations (PDEs) are hot topical issues, and different algorithms proposed in the literature show efficient numerical approximation in high dimension. In this paper, we introduce a class of PDEs that are invariant to permutations, and called symmetric PDEs. Such problems are widespread, ranging from cosmology to quantum mechanics, and option pricing hedging in multi asset market with exchangeable payoff. Our main application comes actually from the particles approximation of mean field control problems. We design deep learning algorithms based on certain types of neural networks, named PointNet and DeepSet (and their associated derivative networks), for computing simultaneously an approximation of the solution and its gradient to symmetric PDEs. We illustrate the performance and accuracy of the PointNet DeepSet networks compared to classical feedforward ones, and provide several numerical results of our algorithm for the examples of a mean field systemic risk, mean variance problem and a min max linear quadratic McKean Vlasov control problem.


Seed Relations for Eichler Shimura congruences and Euler systems. This paper proves that the mathbb U operator attached to a cocharacter is a right root of the corresponding Hecke polynomial. This result is an important ingredient in the proof of (i) the horizontal norm relations in the context of Gross Gan Prasad cycles and of (ii) the generalization of Eichler Shimura relations.


D 2Conv3D Dynamic Dilated Convolutions for Object Segmentation in Videos. Despite receiving significant attention from the research community, the task of segmenting and tracking objects in monocular videos still has much room for improvement. Existing works have simultaneously justified the efficacy of dilated and deformable convolutions for various image level segmentation tasks. This gives reason to believe that 3D extensions of such convolutions should also yield performance improvements for video level segmentation tasks. However, this aspect has not yet been explored thoroughly in existing literature. In this paper, we propose Dynamic Dilated Convolutions (D 2Conv3D) a novel type of convolution which draws inspiration from dilated and deformable convolutions and extends them to the 3D (spatio temporal) domain. We experimentally show that D 2Conv3D can be used to improve the performance of multiple 3D CNN architectures across multiple video segmentation related benchmarks by simply employing D 2Conv3D as a drop in replacement for standard convolutions. We further show that D 2Conv3D out performs trivial extensions of existing dilated and deformable convolutions to 3D. Lastly, we set a new state of the art on the DAVIS 2016 Unsupervised Video Object Segmentation benchmark. Code is made publicly available at https github.com Schmiddo d2conv3d .


Global hypercontractivity and its applications. The hypercontractive inequality on the discrete cube plays a crucial role in many fundamental results in the Analysis of Boolean functions, such as the KKL theorem, Friedgut's junta theorem and the invariance principle. In these results the cube is equipped with the uniform measure, but it is desirable, particularly for applications to the theory of sharp thresholds, to also obtain such results for general p biased measures. However, simple examples show that when p o(1) , there is no hypercontractive inequality that is strong enough. In this paper, we establish an effective hypercontractive inequality for general p that applies to global functions', i.e. functions that are not significantly affected by a restriction of a small set of coordinates. This class of functions appears naturally, e.g. in Bourgain's sharp threshold theorem, which states that such functions exhibit a sharp threshold. We demonstrate the power of our tool by strengthening Bourgain's theorem, thereby making progress on a conjecture of Kahn and Kalai and by establishing a p biased analog of the invariance principle. Our results have significant applications in Extremal Combinatorics. Here we obtain new results on the Tur 'an number of any bounded degree uniform hypergraph obtained as the expansion of a hypergraph of bounded uniformity. These are asymptotically sharp over an essentially optimal regime for both the uniformity and the number of edges and solve a number of open problems in the area. In particular, we give general conditions under which the crosscut parameter asymptotically determines the Tur 'an number, answering a question of Mubayi and Verstra "ete. We also apply the Junta Method to refine our asymptotic results and obtain several exact results, including proofs of the Huang Loh Sudakov conjecture on cross matchings and the F "uredi Jiang Seiver conjecture on path expansions.


Evaluation of Portable Acceleration Solutions for LArTPC Simulation Using Wire Cell Toolkit. The Liquid Argon Time Projection Chamber (LArTPC) technology plays an essential role in many current and future neutrino experiments. Accurate and fast simulation is critical to developing efficient analysis algorithms and precise physics model projections. The speed of simulation becomes more important as Deep Learning algorithms are getting more widely used in LArTPC analysis and their training requires a large simulated dataset. Heterogeneous computing is an efficient way to delegate computing heavy tasks to specialized hardware. However, as the landscape of the compute accelerators is evolving fast, it becomes more and more difficult to manually adapt the code constantly to the latest hardware or software environments. A solution which is portable to multiple hardware architectures while not substantially compromising performance would be very beneficial, especially for long term projects such as the LArTPC simulations. In search of a portable, scalable and maintainable software solution for LArTPC simulations, we have started to explore high level portable programming frameworks that support several hardware backends. In this paper, we will present our experience porting the LArTPC simulation code in the Wire Cell toolkit to NVIDIA GPUs, first with the CUDA programming model and then with a portable library called Kokkos. Preliminary performance results on NVIDIA V100 GPUs and multi core CPUs will be presented, followed by a discussion of the factors affecting the performance and plans for future improvements.


Latent Hatred A Benchmark for Understanding Implicit Hate Speech. Hate speech has grown significantly on social media, causing serious consequences for victims of all demographics. Despite much attention being paid to characterize and detect discriminatory speech, most work has focused on explicit or overt hate speech, failing to address a more pervasive form based on coded or indirect language. To fill this gap, this work introduces a theoretically justified taxonomy of implicit hate speech and a benchmark corpus with fine grained labels for each message and its implication. We present systematic analyses of our dataset using contemporary baselines to detect and explain implicit hate speech, and we discuss key features that challenge existing models. This dataset will continue to serve as a useful benchmark for understanding this multifaceted issue.


Women's Solidarity and Social Media Sisterhood Concept in LasRespondonas, a Facebook group in Peru. Women in Peru are exposed daily to gender violence and exclusion. Several feminist groups have used social media to share information, debate, denounce, organize, and provide help to victims. This contribution analyzes the meaning of female solidarity, sisterhood or sororidad, as a feminist political concept among members of one feminist Facebook group. We reviewed the how various feminist and sisterhood concepts clustered together in the group communications by reviewing group publications and comments as far as interviewing key former and current members. The sisterhood concept was found to be central to feminist practice.


Branch Predicting with Sparse Distributed Memories. Modern processors rely heavily on speculation to keep the pipeline filled and consequently execute and commit instructions as close to maximum capacity as possible. To improve instruction level parallelism, the processor core needs to fetch and decode multiple instructions per cycle and has come to rely on incredibly accurate branch prediction. However, this comes at cost of the increased area and complexity which is needed for modern high accuracy branch predictors. The key idea described in this work is to use hyperdimensional computing and sparse distributed memory principles to create a novel branch predictor that can deliver complex predictions for a fraction of the current area. Sparse distributed memories can store vast amounts of data in a compressed manner, theoretically enabling branch histories larger and more precise than the branch predictors used today to be stored with equal or smaller area footprint. Furthermore, as all the data is in a hashed format and due to the nature of the hashing scheme used, it is inherently harder to manipulate with known side channel attacks. We describe our proof of concept and evaluate it against a state of the art academic TAGE predictor. Our experiments are conducted on realistic synthetic branch predictor patterns and the Championship Branch Prediction traces and show competitive accuracy. Finally, we describe techniques that can be used to solve some of the challenges of processing with hyperdimensional vectors in order to deliver timely predictions.


On the density of eigenvalues on periodic graphs. We consider graphs Gamma with an amenable finitely generated group G acting on the vertices freely and cofinitely and preserving the edges. We study eigenvalues lambda (i.e. elements of the pure point spectrum) of the discrete Laplacian on Gamma (or more generally a discrete periodic operator of finite order). We show that eigenfunctions of lambda with finite support are dense in the l 2 eigenspace of lambda . Moreover, whenever G is virtually polycyclic, there are finitely many finite support eigenfunctions eigenfunctions of lambda up to translations and linear combinations. This property can be used to approximate the density of lambda . When G has subexponential growth, the density of lambda is obtained from a finite resolution by finitely generated free mathbb C G modules (if it exists) of the mathbb C G module of finite support eigenfunctions. Such a resolution always exists when G is abelian.


Transferable Knowledge Based Multi Granularity Aggregation Network for Temporal Action Localization Submission to ActivityNet Challenge 2021. This technical report presents an overview of our solution used in the submission to 2021 HACS Temporal Action Localization Challenge on both Supervised Learning Track and Weakly Supervised Learning Track. Temporal Action Localization (TAL) requires to not only precisely locate the temporal boundaries of action instances, but also accurately classify the untrimmed videos into specific categories. However, Weakly Supervised TAL indicates locating the action instances using only video level class labels. In this paper, to train a supervised temporal action localizer, we adopt Temporal Context Aggregation Network (TCANet) to generate high quality action proposals through local and global" temporal context aggregation and complementary as well as progressive boundary refinement. As for the WSTAL, a novel framework is proposed to handle the poor quality of CAS generated by simple classification network, which can only focus on local discriminative parts, rather than locate the entire interval of target actions. Further inspired by the transfer learning method, we also adopt an additional module to transfer the knowledge from trimmed videos (HACS Clips dataset) to untrimmed videos (HACS Segments dataset), aiming at promoting the classification performance on untrimmed videos. Finally, we employ a boundary regression module embedded with Outer Inner Contrastive (OIC) loss to automatically predict the boundaries based on the enhanced CAS. Our proposed scheme achieves 39.91 and 29.78 average mAP on the challenge testing set of supervised and weakly supervised temporal action localization track respectively.


Speed of extinction for continuous state branching processes in subcritical L 'evy environments the strongly and intermediate regimes. In this paper, we study the speed of extinction of continuous state branching processes in subcritical L 'evy environments. More precisely, when the associated L 'evy process to the environment drifts to infty and, under a suitable exponential martingale change of measure (Esscher transform), the environment either drifts to infty or oscillates. We extend recent results of Palau et al. (2016) and Li and Xu (2018), where the branching term is associated to a spectrally positive stable L 'evy process and complement the recent article of Bansaye et al. (2021) where the critical case was studied. Our methodology combines a path analysis of the branching process together with its L 'evy environment, fluctuation theory for L 'evy processes and the asymptotic behaviour of exponential functionals of L 'evy processes. As an application of the aforementioned results, we characterise the process conditioned to survival also known as the Q process.


Distribution dependent SDEs driven by additive continuous noise. We study distribution dependent stochastic differential equation driven by a continuous process, without any specification on its law, following the approach initiated in 16 . We provide several criteria for existence and uniqueness of solutions which go beyond the classical globally Lipschitz setting. In particular we show well posedness of the equation, as well as almost sure convergence of the associated particle system, for drifts satisfying either Osgood continuity, monotonicity, local Lipschitz or Sobolev differentiability type assumptions.


F CAM Full Resolution Class Activation Maps via Guided Parametric Upscaling. Class Activation Mapping (CAM) methods have recently gained much attention for weakly supervised object localization (WSOL) tasks. They allow for CNN visualization and interpretation without training on fully annotated image datasets. CAM methods are typically integrated within off the shelf CNN backbones, such as ResNet50. Due to convolution and pooling operations, these backbones yield low resolution CAMs with a down scaling factor of up to 32, contributing to inaccurate localizations. Interpolation is required to restore full size CAMs, yet it does not consider the statistical properties of objects, such as color and texture, leading to activations with inconsistent boundaries, and inaccurate localizations. As an alternative, we introduce a generic method for parametric upscaling of CAMs that allows constructing accurate full resolution CAMs (F CAMs). In particular, we propose a trainable decoding architecture that can be connected to any CNN classifier to produce highly accurate CAM localizations. Given an original low resolution CAM, foreground and background pixels are randomly sampled to fine tune the decoder. Additional priors such as image statistics and size constraints are also considered to expand and refine object boundaries. Extensive experiments, over three CNN backbones and six WSOL baselines on the CUB 200 2011 and OpenImages datasets, indicate that our F CAM method yields a significant improvement in CAM localization accuracy. F CAM performance is competitive with state of art WSOL methods, yet it requires fewer computations during inference.


Water lifting and outflow gain of kinetic energy in tropical cyclones. While water lifting plays a recognized role in the global atmospheric power budget, estimates for this role in tropical cyclones vary from no effect to a major reduction in storm intensity. To better assess this impact, here we consider the work output of an infinitely narrow thermodynamic cycle with two streamlines connecting the top of the boundary layer in the vicinity of maximum wind (without assuming gradient wind balance) to an arbitrary level in the inviscid free troposphere. The reduction of a storm's maximum wind speed due to water lifting is found to decline with increasing efficiency of the cycle and is about 5 for maximum observed Carnot efficiencies. In the steady state cycle, there is an extra heat input associated with the warming of precipitating water. The corresponding positive extra work is of an opposite sign and several times smaller than that due to water lifting. We also estimate the gain of kinetic energy in the outflow region. Contrary to previous assessments, this term is found to be large when the outflow radius is small (comparable to the radius of maximum wind). Using our framework, we show that Emanuel's maximum potential intensity (E PI) corresponds to a cycle where total work equals work performed at the top of the boundary layer (net work in the free troposphere is zero). This constrains a dependence between the outflow temperature and heat input at the point of maximum wind, but does not constrain the radial pressure gradient. We outline the implications of the established patterns for assessing real storms.


DeepIC Coding for Interference Channels via Deep Learning. The two user interference channel is a model for multi one to one communications, where two transmitters wish to communicate with their corresponding receivers via a shared wireless medium. Two most common and simple coding schemes are time division (TD) and treating interference as noise (TIN). Interestingly, it is shown that there exists an asymptotic scheme, called Han Kobayashi scheme, that performs better than TD and TIN. However, Han Kobayashi scheme has impractically high complexity and is designed for asymptotic settings, which leads to a gap between information theory and practice. In this paper, we focus on designing practical codes for interference channels. As it is challenging to analytically design practical codes with feasible complexity, we apply deep learning to learn codes for interference channels. We demonstrate that DeepIC, a convolutional neural network based code with an iterative decoder, outperforms TD and TIN by a significant margin for two user additive white Gaussian noise channels with moderate amount of interference.


Verification and Optimization of Cyber Physical Systems Preprint for FedCSIS. Optimizing CPS behavior in terms of energy consumption can have a significant impact on system reliability. The environment influences the system's behavior, and neglecting the environmental behavior has an indirect negative impact on optimizing the system's behavior. In this work, to increase the system's flexibility, the behavior of the environment is modeled dynamically to apply the disorderliness of its behavior. The resulting models are formally verified. By examining the past environmental behavior and predicting its future behavior, energy optimization is done more dynamically. The verification results acquired using a UPPAAL SMC show that the optimization of system behavior by predicting the environmental behavior has been successful. Our approach is demonstrated using a case study within an I4 setting.


Investigating Monolingual and Multilingual BERTModels for Vietnamese Aspect Category Detection. Aspect category detection (ACD) is one of the challenging tasks in the Aspect based sentiment Analysis problem. The purpose of this task is to identify the aspect categories mentioned in user generated reviews from a set of pre defined categories. In this paper, we investigate the performance of various monolingual pre trained language models compared with multilingual models on the Vietnamese aspect category detection problem. We conduct the experiments on two benchmark datasets for the restaurant and hotel domain. The experimental results demonstrated the effectiveness of the monolingual PhoBERT model than others on two datasets. We also evaluate the performance of the multilingual model based on the combination of whole SemEval 2016 datasets in other languages with the Vietnamese dataset. To the best of our knowledge, our research study is the first attempt at performing various available pre trained language models on aspect category detection task and utilize the datasets from other languages based on multilingual models.


Fast and Data Efficient Training of Rainbow an Experimental Study on Atari. Across the Arcade Learning Environment, Rainbow achieves a level of performance competitive with humans and modern RL algorithms. However, attaining this level of performance requires large amounts of data and hardware resources, making research in this area computationally expensive and use in practical applications often infeasible. This paper's contribution is threefold We (1) propose an improved version of Rainbow, seeking to drastically reduce Rainbow's data, training time, and compute requirements while maintaining its competitive performance (2) we empirically demonstrate the effectiveness of our approach through experiments on the Arcade Learning Environment, and (3) we conduct a number of ablation studies to investigate the effect of the individual proposed modifications. Our improved version of Rainbow reaches a median human normalized score close to classic Rainbow's, while using 20 times less data and requiring only 7.5 hours of training time on a single GPU. We also provide our full implementation including pre trained models.


How May I Help You? Using Neural Text Simplification to Improve Downstream NLP Tasks. The general goal of text simplification (TS) is to reduce text complexity for human consumption. This paper investigates another potential use of neural TS assisting machines performing natural language processing (NLP) tasks. We evaluate the use of neural TS in two ways simplifying input texts at prediction time and augmenting data to provide machines with additional information during training. We demonstrate that the latter scenario provides positive effects on machine performance on two separate datasets. In particular, the latter use of TS improves the performances of LSTM (1.82 1.98 ) and SpanBERT (0.7 1.3 ) extractors on TACRED, a complex, large scale, real world relation extraction task. Further, the same setting yields improvements of up to 0.65 matched and 0.62 mismatched accuracies for a BERT text classifier on MNLI, a practical natural language inference dataset.


Training Federated GANs with Theoretical Guarantees A Universal Aggregation Approach. Recently, Generative Adversarial Networks (GANs) have demonstrated their potential in federated learning, i.e., learning a centralized model from data privately hosted by multiple sites. A federatedGAN jointly trains a centralized generator and multiple private discriminators hosted at different sites. A major theoretical challenge for the federated GAN is the heterogeneity of the local data distributions. Traditional approaches cannot guarantee to learn the target distribution, which isa mixture of the highly different local distributions. This paper tackles this theoretical challenge, and for the first time, provides a provably correct framework for federated GAN. We propose a new approach called Universal Aggregation, which simulates a centralized discriminator via carefully aggregating the mixture of all private discriminators. We prove that a generator trained with this simulated centralized discriminator can learn the desired target distribution. Through synthetic and real datasets, we show that our method can learn the mixture of largely different distributions where existing federated GAN methods fail.


Energy Balance Within Thermonuclear Reactors. Thermonuclear reactors hold a great promise for the future of Humankind. Within Tokamak and Stellarator reactors, plasma is confined by twisted magnetic fields. Reactors which produce fusion energy have existed since Princeton Large Torus Tokamak in 1978, nevertheless in all reactors built up to now, energy loss from plasma vastly exceeded fusion energy production. In order for a thermonuclear power plant to run, generated fusion energy must significantly exceed energy loss by the plasma. There are four processes by which plasma looses energy neutron radiation, Bremsstrahlung radiation, synchrotron radiation, and heat conduction to the walls. For a deuterium tritium reactor, 80 of energy produced by fusion is lost to neutron radiation, about 4 to 6 of fusion energy is lost to Bremsstrahlung and synchrotron radiation. For a deuterium 3 He reactor, 5 of energy produced by fusion is lost to neutron radiation, about 50 to 75 of fusion energy is lost to Bremsstrahlung and synchrotron radiation. Increasing reactor operating temperature decreases loss to Bremsstrahlung radiation and increases loss to synchrotron radiation. Power loss to conduction is either independent or weakly dependent on fusion power production. Up to now, no single theoretical or experimental model can accurately predict conduction power loss. For small Tokamaks and Stellarators, conduction power loss vastly exceeds power generated by fusion. For large and powerful thermonuclear reactors which are yet to be built, conduction power loss should be much lower then power produced by fusion.


Programmatic Innovations that Accord with the Retention of Women in STEM Careers. Gender representation in the physical sciences remains inequitable and continues to lag behind other fields. Even though there exists adequate documentation regarding programmatic postures and innovations, difficulties persist within the physics discipline. In this paper, we present innovative, programmatic implementations over an eight year period at an undergraduate, liberal arts physics program. Some of these innovations accord with nationally published, best practices' for undergraduate physics programs, while others are novel to the program's context. Within this eight year period, alterations are separated into curricular, co curricular, and experiential elements. Innovations are introduced in some detail, and data are presented before, during, and after their introduction. While it is currently impossible to say which elements had the greatest impact, the synergistic combination did have a positive effect on the program. Not only did the number of total majors and graduates increase, there was a 200 increase of women degree recipients compared to the previous ten years, which boosted average graduation rates above the national average (30 20 ). Women were retained within the undergraduate physics major at a higher percentage during this time period when compared to men in the program. Lastly, these women physics majors maintained careers in science advancement fields at a rate of 90 after leq5 years post graduation. While this paper presents a singular case study, the purpose is two fold a) to validate quantitatively the work of national physics organizations within the context of a liberal arts institution, and b) to suggest that a multi level approach is most efficacious when considering programmatic innovations.


AnchorGAE General Data Clustering via O(n) Bipartite Graph Convolution. Graph based clustering plays an important role in clustering tasks. As graph convolution network (GCN), a variant of neural networks on graph type data, has achieved impressive performance, it is attractive to find whether GCNs can be used to augment the graph based clustering methods on non graph data, i.e., general data. However, given n samples, the graph based clustering methods usually need at least O(n 2) time to build graphs and the graph convolution requires nearly O(n 2) for a dense graph and O( mathcal E ) for a sparse one with mathcal E edges. In other words, both graph based clustering and GCNs suffer from severe inefficiency problems. To tackle this problem and further employ GCN to promote the capacity of graph based clustering, we propose a novel clustering method, AnchorGAE. As the graph structure is not provided in general clustering scenarios, we first show how to convert a non graph dataset into a graph by introducing the generative graph model, which is used to build GCNs. Anchors are generated from the original data to construct a bipartite graph such that the computational complexity of graph convolution is reduced from O(n 2) and O( mathcal E ) to O(n) . The succeeding steps for clustering can be easily designed as O(n) operations. Interestingly, the anchors naturally lead to a siamese GCN architecture. The bipartite graph constructed by anchors is updated dynamically to exploit the high level information behind data. Eventually, we theoretically prove that the simple update will lead to degeneration and a specific strategy is accordingly designed.


CGEMs A Metric Model for Automatic Code Generation using GPT 3. Today, AI technology is showing its strengths in almost every industry and walks of life. From text generation, text summarization, chatbots, NLP is being used widely. One such paradigm is automatic code generation. An AI could be generating anything hence the output space is unconstrained. A self driving car is driven for 100 million miles to validate its safety, but tests cannot be written to monitor and cover an unconstrained space. One of the solutions to validate AI generated content is to constrain the problem and convert it from abstract to realistic, and this can be accomplished by either validating the unconstrained algorithm using theoretical proofs or by using Monte Carlo simulation methods. In this case, we use the latter approach to test validate a statistically significant number of samples. This hypothesis of validating the AI generated code is the main motive of this work and to know if AI generated code is reliable, a metric model CGEMs is proposed. This is an extremely challenging task as programs can have different logic with different naming conventions, but the metrics must capture the structure and logic of the program. This is similar to the importance grammar carries in AI based text generation, Q A, translations, etc. The various metrics that are garnered in this work to support the evaluation of generated code are as follows Compilation, NL description to logic conversion, number of edits needed, some of the commonly used static code metrics and NLP metrics. These metrics are applied to 80 codes generated using OpenAI's GPT 3. Post which a Neural network is designed for binary classification (acceptable not acceptable quality of the generated code). The inputs to this network are the values of the features obtained from the metrics. The model achieves a classification accuracy of 76.92 and an F1 score of 55.56 . XAI is augmented for model interpretability.


IADA Iterative Adversarial Data Augmentation Using Formal Verification and Expert Guidance. Neural networks (NNs) are widely used for classification tasks for their remarkable performance. However, the robustness and accuracy of NNs heavily depend on the training data. In many applications, massive training data is usually not available. To address the challenge, this paper proposes an iterative adversarial data augmentation (IADA) framework to learn neural network models from an insufficient amount of training data. The method uses formal verification to identify the most "confusing" input samples, and leverages human guidance to safely and iteratively augment the training data with these samples. The proposed framework is applied to an artificial 2D dataset, the MNIST dataset, and a human motion dataset. By applying IADA to fully connected NN classifiers, we show that our training method can improve the robustness and accuracy of the learned model. By comparing to regular supervised training, on the MNIST dataset, the average perturbation bound improved 107.4 . The classification accuracy improved 1.77 , 3.76 , 10.85 on the 2D dataset, the MNIST dataset, and the human motion dataset respectively.


Extremal bipartite independence number and balanced coloring. In this paper, we establish a couple of results on extremal problems in bipartite graphs. Firstly, we show that every sufficiently large bipartite graph with average degree Delta and with n vertices on each side has a balanced independent set containing (1 epsilon) frac log Delta Delta n vertices from each side for small epsilon 0 . Secondly, we prove that the vertex set of every sufficiently large balanced bipartite graph with maximum degree at most Delta can be partitioned into (1 epsilon) frac Delta log Delta balanced independent sets. Both of these results are algorithmic and best possible up to a factor of 2, which might be hard to improve as evidenced by the phenomenon known as algorithmic barrier' in the literature. The first result improves a recent theorem of Axenovich, Sereni, Snyder, and Weber in a slightly more general setting. The second result improves a theorem of Feige and Kogan about coloring balanced bipartite graphs.


TDLS A Top Down Layer Searching Algorithm for Generating Counterfactual Visual Explanation. Explanation of AI, as well as fairness of algorithms' decisions and the transparency of the decision model, are becoming more and more important. And it is crucial to design effective and human friendly techniques when opening the black box model. Counterfactual conforms to the human way of thinking and provides a human friendly explanation, and its corresponding explanation algorithm refers to a strategic alternation of a given data point so that its model output is "counter facted", i.e. the prediction is reverted. In this paper, we adapt counterfactual explanation over fine grained image classification problem. We demonstrated an adaptive method that could give a counterfactual explanation by showing the composed counterfactual feature map using top down layer searching algorithm (TDLS). We have proved that our TDLS algorithm could provide more flexible counterfactual visual explanation in an efficient way using VGG 16 model on Caltech UCSD Birds 200 dataset. At the end, we discussed several applicable scenarios of counterfactual visual explanations.


What Can I Do Here? Learning New Skills by Imagining Visual Affordances. A generalist robot equipped with learned skills must be able to perform many tasks in many different environments. However, zero shot generalization to new settings is not always possible. When the robot encounters a new environment or object, it may need to finetune some of its previously learned skills to accommodate this change. But crucially, previously learned behaviors and models should still be suitable to accelerate this relearning. In this paper, we aim to study how generative models of possible outcomes can allow a robot to learn visual representations of affordances, so that the robot can sample potentially possible outcomes in new situations, and then further train its policy to achieve those outcomes. In effect, prior data is used to learn what kinds of outcomes may be possible, such that when the robot encounters an unfamiliar setting, it can sample potential outcomes from its model, attempt to reach them, and thereby update both its skills and its outcome model. This approach, visuomotor affordance learning (VAL), can be used to train goal conditioned policies that operate on raw image inputs, and can rapidly learn to manipulate new objects via our proposed affordance directed exploration scheme. We show that VAL can utilize prior data to solve real world tasks such drawer opening, grasping, and placing objects in new scenes with only five minutes of online experience in the new scene.


On sifted colimits in the presence of pullbacks. We show that in a category with pullbacks, arbitrary sifted colimits may be constructed as filtered colimits of reflexive coequalizers. This implies that "lex sifted colimits", in the sense of Garner Lack, decompose as Barr exactness plus filtered colimits commuting with finite limits. We also prove generalizations of these results for kappa small sifted and filtered colimits, and their interaction with lambda small limits in place of finite ones, generalizing Garner's characterization of algebraic exactness in the sense of Ad 'amek Lawvere Rosick 'y. Along the way, we prove a general result on classes of colimits, showing that the kappa small restriction of a saturated class of colimits is still "closed under iteration".


Tackling Consistency related Design Challenges of Distributed Data Intensive Systems An Action Research Study. Background Distributed data intensive systems are increasingly designed to be only eventually consistent. Persistent data is no longer processed with serialized and transactional access, exposing applications to a range of potential concurrency anomalies that need to be handled by the application itself. Controlling concurrent data access in monolithic systems is already challenging, but the problem is exacerbated in distributed systems. To make it worse, only little systematic engineering guidance is provided by the software architecture community regarding this issue. Aims In this paper, we report on our study of the effectiveness and applicability of the novel design guidelines we are proposing in this regard. Method We used action research and conducted it in the context of the software architecture design process of a multi site platform development project. Results Our hypotheses regarding effectiveness and applicability have been accepted in the context of the study. The initial design guidelines were refined throughout the study. Thus, we also contribute concrete guidelines for architecting distributed data intensive systems with eventually consistent data. The guidelines are an advancement of Domain Driven Design and provide additional patterns for the tactical design part. Conclusions Based on our results, we recommend using the guidelines to architect safe eventually consistent systems. Because of the relevance of distributed data intensive systems, we will drive this research forward and evaluate it in further domains.


A Wide Temperature Range Unified Undoped Bulk Silicon Electron and Hole Mobility Model. For decades, there is no unified model developed for Silicon carriers from 4K to above room temperature. In this paper, a unified undoped Silicon low field and high field mobility model for both electron in the 100 and 111 directions from 8K to 300K and 430K, respectively, and hole in the 100 direction from 6K to 430K is proposed and calibrated to the experiment. It is found that the Canali high field saturation model is sufficient to fit the 111 experimental data but not the 100 data due to the anisotropy induced negative differential velocity. Therefore, a modified Farahmand model is used. To allow parameter interpolation in anisotropic simulation, the modified Farahmand model is also calibrated for the 111 direction. The model is then used to predict the mobility of electrons and holes in undoped Si at 4K, which can be served as the initial calibration parameters when reliable experimental data is available for the TCAD model development.


Tensor based Subspace Factorization for StyleGAN. In this paper, we propose tau GAN a tensor based method for modeling the latent space of generative models. The objective is to identify semantic directions in latent space. To this end, we propose to fit a multilinear tensor model on a structured facial expression database, which is initially embedded into latent space. We validate our approach on StyleGAN trained on FFHQ using BU 3DFE as a structured facial expression database. We show how the parameters of the multilinear tensor model can be approximated by Alternating Least Squares. Further, we introduce a tacked style separated tensor model, defined as an ensemble of style specific models to integrate our approach with the extended latent space of StyleGAN. We show that taking the individual styles of the extended latent space into account leads to higher model flexibility and lower reconstruction error. Finally, we do several experiments comparing our approach to former work on both GANs and multilinear models. Concretely, we analyze the expression subspace and find that the expression trajectories meet at an apathetic face that is consistent with earlier work. We also show that by changing the pose of a person, the generated image from our approach is closer to the ground truth than results from two competing approaches.


Number of visits in arbitrary sets for phi mixing dynamics. It is well known that, for sufficiently mixing dynamical systems, the number of visits to balls and cylinders of vanishing measure is approximately Poisson compound distributed in the Kac scaling. Here we extend this kind of results when the target set is an arbitrary set with vanishing measure in the case of phi mixing systems. The error of approximation in total variation is derived using Stein Chen method. An important part of the paper is dedicated to examples to illustrate the assumptions, as well as applications to temporal synchronisation of g measures


Applying the Case Difference Heuristic to Learn Adaptations from Deep Network Features. The case difference heuristic (CDH) approach is a knowledge light method for learning case adaptation knowledge from the case base of a case based reasoning system. Given a pair of cases, the CDH approach attributes the difference in their solutions to the difference in the problems they solve, and generates adaptation rules to adjust solutions accordingly when a retrieved case and new query have similar problem differences. As an alternative to learning adaptation rules, several researchers have applied neural networks to learn to predict solution differences from problem differences. Previous work on such approaches has assumed that the feature set describing problems is predefined. This paper investigates a two phase process combining deep learning for feature extraction and neural network based adaptation learning from extracted features. Its performance is demonstrated in a regression task on an image data predicting age given the image of a face. Results show that the combined process can successfully learn adaptation knowledge applicable to nonsymbolic differences in cases. The CBR system achieves slightly lower performance overall than a baseline deep network regressor, but better performance than the baseline on novel queries.


Anisotropic Multi layer Cylindrical Structures Containing Graphene Layers An Analytical Approach. We propose a novel analytical model for anisotropic multi layer cylindrical structures containing graphene layers. The general structure is formed by an aperiodic repetition of a three layer sub structure, where a graphene layer, with an isotropic surface conductivity, has been sandwiched between two adjacent magnetic materials. An external magnetic bias has been applied in the axial direction. General matrix representation is obtained in our proposed analytical model to find the dispersion relation. The relation will be used to find the effective index of the structure and its other propagation parameters. Two special exemplary structures have been introduced and studied to show the richness of the proposed general structure regarding the related specific plasmonic wave phenomena and effects. A series of simulations have been conducted to demonstrate the noticeable wave guiding properties of the structure in the 10 40 THz band. A very good agreement between the analytical and simulation results is observed. The proposed structure can be utilized to design novel plasmonic devices such as absorbers, modulators, plasmonic sensors and tunable antennas in the THz frequencies.


Role Similarity Metric Based on Spanning Rooted Forest. As a fundamental issue in network analysis, structural node similarity has received much attention in academia and is adopted in a wide range of applications. Among these proposed structural node similarity measures, role similarity stands out because of satisfying several axiomatic properties including automorphism conformation. Existing role similarity metrics cannot handle top k queries on large real world networks due to the high time and space cost. In this paper, we propose a new role similarity metric, namely textsf ForestSim . We prove that textsf ForestSim is an admissible role similarity metric and devise the corresponding top k similarity search algorithm, namely textsf ForestSimSearch , which is able to process a top k query in O(k) time once the precomputation is finished. Moreover, we speed up the precomputation by using a fast approximate algorithm to compute the diagonal entries of the forest matrix, which reduces the time and space complexity of the precomputation to O( epsilon 2 m log 5 n log frac 1 epsilon ) and O(m log 3 n ) , respectively. Finally, we conduct extensive experiments on 26 real world networks. The results show that textsf ForestSim works efficiently on million scale networks and achieves comparable performance to the state of art methods.


Lighting of a monochromatic scatterer with virtual gain. In this work, we discuss the scattering features of a dipolar particle made of large refractive index material by employing the concept of virtual gain and virtual loss. The virtual gain and loss can be achieved in a lossless passive nanostructure by shaping the temporal waveform of incident signals in the complex frequency plane. We show that an appropriate tuning of excitation time of the impinging field allows to capture and release the electromagnetic energy on demand for an arbitrary time scale in a lossless nanosphere. Thus, the nanosphere obliges to emit monochromatic magnetic light which can be tuned throughout the whole visible spectrum by varying the size of the nanosphere. This proposal may find fruitful applications in lab on a chip technologies and the realization of monochromatic sectoral multipole light source with a large quality factor at nanoscale level.


On Generating and Labeling Network Traffic with Realistic, Self Propagating Malware. Research and development of techniques which detect or remediate malicious network activity require access to diverse, realistic, contemporary data sets containing labeled malicious connections. In the absence of such data, said techniques cannot be meaningfully trained, tested, and evaluated. Synthetically produced data containing fabricated or merged network traffic is of limited value as it is easily distinguishable from real traffic by even simple machine learning (ML) algorithms. Real network data is preferable, but while ubiquitous is broadly both sensitive and lacking in ground truth labels, limiting its utility for ML research. This paper presents a multi faceted approach to generating a data set of labeled malicious connections embedded within anonymized network traffic collected from large production networks. Real world malware is defanged and introduced to simulated, secured nodes within those networks to generate realistic traffic while maintaining sufficient isolation to protect real data and infrastructure. Network sensor data, including this embedded malware traffic, is collected at a network edge and anonymized for research use. Network traffic was collected and produced in accordance with the aforementioned methods at two major educational institutions. The result is a highly realistic, long term, multi institution data set with embedded data labels spanning over 1.5 trillion connections and over a petabyte of sensor log data. The usability of this data set is demonstrated by its utility to our artificial intelligence and machine learning (AI ML) research program.


High Spatial Resolution Neutron Transmission Imaging Using a Superconducting Two Dimensional Detector. Neutron imaging is one of the most powerful tools for nondestructive inspection owing to the unique characteristics of neutron beams, such as high permeability for many heavy metals, high sensitivity for certain light elements, and isotope selectivity owing to a specific nuclear reaction between an isotope and neutrons. In this study, we employed a superconducting detector, current biased kinetic inductance detector (CB KID) for neutron imaging using a pulsed neutron source. We employed the delay line method, and high spatial resolution imaging with only four reading channels was achieved. We also performed wavelength resolved neutron imaging by the time of flight method for the pulsed neutron source. We obtained the neutron transmission images of a Gd Al alloy sample, inside which single crystals of GdAl3 were grown, using the delay line CB KID. Single crystals were well imaged, in both shapes and distributions, throughout the Al Gd alloy. We identified Gd nuclei via neutron transmissions that exhibited characteristic suppression above the neutron wavelength of 0.03 nm. In addition, the 155 Gd resonance dip, a dip structure of the transmission caused by the nuclear reaction between an isotope and neutrons, was observed even when the number of events was summed over a limited area of 15 X 12 um 2. Gd selective imaging was performed using the resonance dip of 155 Gd, and it showed clear Gd distribution even with a limited neutron wavelength range of 1 pm.


A Byzantine Fault Tolerant Consensus Library for Hyperledger Fabric. Hyperledger Fabric is an enterprise grade permissioned distributed ledger platform that offers modularity for a broad set of industry use cases. One modular component is a pluggable ordering service that establishes consensus on the order of transactions and batches them into blocks. However, as of the time of this writing, there is no production grade Byzantine Fault Tolerant (BFT) ordering service for Fabric, with the latest version (v2.1) supporting only Crash Fault Tolerance (CFT). In our work, we address crucial aspects of BFT integration into Fabric that were left unsolved in all prior works, making them unfit for production use. In this work we describe the design and implementation of a BFT ordering service for Fabric, employing a new BFT consensus library. The new library, based on the BFT Smart protocol and written in Go, is tailored to the blockchain use case, yet is general enough to cater to a wide variety of other uses. We evaluate the new BFT ordering service by comparing it with the currently supported Raft based CFT ordering service in Hyperledger Fabric.


Comparing spontaneous and pellet triggered ELMs via non linear extended MHD simulations. Injecting frozen deuterium pellets into an ELMy H mode plasma is a well established scheme for triggering edge localized modes (ELMs) before they naturally occur. Based on an ASDEX Upgrade H mode plasma, this article presents a comparison of extended MHD simulations of spontaneous type I ELMs and pellet triggered ELMs allowing to study their non linear dynamics in detail. In particular, pellet triggered ELMs are simulated by injecting deuterium pellets into different time points during the pedestal build up described in A. Cathey et al. Nuclear Fusion 60, 124007 (2020) . Realistic ExB and diamagnetic background plasma flows as well as the time dependent bootstrap current evolution are included during the build up to capture the balance between stabilising and destabilising terms for the edge instabilities accurately. Dependencies on the pellet size and injection times are studied. The spatio temporal structures of the modes and the resulting divertor heat fluxes are compared in detail between spontaneous and triggered ELMs. We observe that the premature excitation of ELMs by means of pellet injection is caused by a helical perturbation described by a toroidal mode number of n 1. In accordance with experimental observations, the pellet triggered ELMs show reduced thermal energy losses and narrower divertor wetted area with respect to spontaneous ELMs. The peak divertor energy fluency is seen to decrease when ELMs are triggered by pellets injected earlier during the pedestal build up.


ManhattanSLAM Robust Planar Tracking and Mapping Leveraging Mixture of Manhattan Frames. In this paper, a robust RGB D SLAM system is proposed to utilize the structural information in indoor scenes, allowing for accurate tracking and efficient dense mapping on a CPU. Prior works have used the Manhattan World (MW) assumption to estimate low drift camera pose, in turn limiting the applications of such systems. This paper, in contrast, proposes a novel approach delivering robust tracking in MW and non MW environments. We check orthogonal relations between planes to directly detect Manhattan Frames, modeling the scene as a Mixture of Manhattan Frames. For MW scenes, we decouple pose estimation and provide a novel drift free rotation estimation based on Manhattan Frame observations. For translation estimation in MW scenes and full camera pose estimation in non MW scenes, we make use of point, line and plane features for robust tracking in challenging scenes. mapping Additionally, by exploiting plane features detected in each frame, we also propose an efficient surfel based dense mapping strategy, which divides each image into planar and non planar regions. Planar surfels are initialized directly from sparse planes in our map while non planar surfels are built by extracting superpixels. We evaluate our method on public benchmarks for pose estimation, drift and reconstruction accuracy, achieving superior performance compared to other state of the art methods. We will open source our code in the future.


Fast Projection Onto Convex Smooth Constraints. The Euclidean projection onto a convex set is an important problem that arises in numerous constrained optimization tasks. Unfortunately, in many cases, computing projections is computationally demanding. In this work, we focus on projection problems where the constraints are smooth and the number of constraints is significantly smaller than the dimension. The runtime of existing approaches to solving such problems is either cubic in the dimension or polynomial in the inverse of the target accuracy. Conversely, we propose a simple and efficient primal dual approach, with a runtime that scales only linearly with the dimension, and only logarithmically in the inverse of the target accuracy. We empirically demonstrate its performance, and compare it with standard baselines.


Multi solitary waves to stochastic nonlinear Schr "odinger equations. In this paper, we present a pathwise construction of multi soliton solutions for focusing stochastic nonlinear Schr "odinger equations with linear multiplicative noise, in both the L 2 critical and subcritical cases. The constructed multi solitons behave asymptotically as a sum of K solitary waves, where K is any given finite number. Moreover, the convergence rate of the remainders can be of either exponential or polynomial type, which reflects the effects of the noise in the system on the asymptotical behavior of the solutions. The major difficulty in our construction of stochastic multi solitons is the absence of pseudo conformal invariance. Unlike in the deterministic case 47,54 , the existence of stochastic multi solitons cannot be obtained from that of stochastic multi bubble blow up solutions in 54,57 . Our proof is mainly based on the rescaling approach in 39 , relying on two types of Doss Sussman transforms, and on the modulation method in 16,44 , in which the crucial ingredient is the monotonicity of the Lyapunov type functional constructed by Martel, Merle and Tsai 45 . In our stochastic case, this functional depends on the Brownian paths in the noise.


SocialNLP EmotionGIF 2020 Challenge Overview Predicting Reaction GIF Categories on Social Media. We present an overview of the EmotionGIF2020 Challenge, held at the 8th International Workshop on Natural Language Processing for Social Media (SocialNLP), in conjunction with ACL 2020. The challenge required predicting affective reactions to online texts, and included the EmotionGIF dataset, with tweets labeled for the reaction categories. The novel dataset included 40K tweets with their reaction GIFs. Due to the special circumstances of year 2020, two rounds of the competition were conducted. A total of 84 teams registered for the task. Of these, 25 teams success fully submitted entries to the evaluation phase in the first round, while 13 teams participated successfully in the second round. Of the top participants, five teams presented a technical report and shared their code. The top score of the winning team using the Recall K metric was 62.47 .


A Critique of Strictly Batch Imitation Learning. Recent work by Jarrett et al. attempts to frame the problem of offline imitation learning (IL) as one of learning a joint energy based model, with the hope of out performing standard behavioral cloning. We suggest that notational issues obscure how the psuedo state visitation distribution the authors propose to optimize might be disconnected from the policy's textit true state visitation distribution. We further construct natural examples where the parameter coupling advocated by Jarrett et al. leads to inconsistent estimates of the expert's policy, unlike behavioral cloning.


Optimal decay of the parabolic semigroup in stochastic homogenization for correlated coefficient fields. We study the large scale behavior of elliptic systems with stationary random coefficient that have only slowly decaying correlations. To this aim we analyze the so called corrector equation, a degenerate elliptic equation posed in the probability space. In this contribution, we use a parabolic approach and optimally quantify the time decay of the semigroup. For the theoretical point of view, we prove an optimal decay estimate of the gradient and flux of the corrector when spatially averaged over a scale R larger than 1. For the numerical point of view, our results provide convenient tools for the analysis of various numerical methods.


Training Vision Transformers for Image Retrieval. Transformers have shown outstanding results for natural language understanding and, more recently, for image classification. We here extend this work and propose a transformer based approach for image retrieval we adopt vision transformers for generating image descriptors and train the resulting model with a metric learning objective, which combines a contrastive loss with a differential entropy regularizer. Our results show consistent and significant improvements of transformers over convolution based approaches. In particular, our method outperforms the state of the art on several public benchmarks for category level retrieval, namely Stanford Online Product, In Shop and CUB 200. Furthermore, our experiments on ROxford and RParis also show that, in comparable settings, transformers are competitive for particular object retrieval, especially in the regime of short vector representations and low resolution images.


MuVAM A Multi View Attention based Model for Medical Visual Question Answering. Medical Visual Question Answering (VQA) is a multi modal challenging task widely considered by research communities of the computer vision and natural language processing. Since most current medical VQA models focus on visual content, ignoring the importance of text, this paper proposes a multi view attention based model(MuVAM) for medical visual question answering which integrates the high level semantics of medical images on the basis of text description. Firstly, different methods are utilized to extract the features of the image and the question for the two modalities of vision and text. Secondly, this paper proposes a multi view attention mechanism that include Image to Question (I2Q) attention and Word to Text (W2T) attention. Multi view attention can correlate the question with image and word in order to better analyze the question and get an accurate answer. Thirdly, a composite loss is presented to predict the answer accurately after multi modal feature fusion and improve the similarity between visual and textual cross modal features. It consists of classification loss and image question complementary (IQC) loss. Finally, for data errors and missing labels in the VQA RAD dataset, we collaborate with medical experts to correct and complete this dataset and then construct an enhanced dataset, VQA RADPh. The experiments on these two datasets show that the effectiveness of MuVAM surpasses the state of the art method.