Mobile Augmented Reality Survey: From Where We Are to Where We Go
---
paper_title: Fast Feature Pyramids for Object Detection

paper_content:

Multi-resolution image features may be approximated via extrapolation from nearby scales, rather than being computed explicitly. This fundamental insight allows us to design object detection algorithms that are as accurate, and considerably faster, than the state-of-the-art. The computational bottleneck of many modern detectors is the computation of features at every scale of a finely-sampled image pyramid. Our key insight is that one may compute finely sampled feature pyramids at a fraction of the cost, without sacrificing performance: for a broad family of features we find that features computed at octave-spaced scale intervals are sufficient to approximate features on a finely-sampled pyramid. Extrapolation is inexpensive as compared to direct feature computation. As a result, our approximation yields considerable speedups with negligible loss in detection accuracy. We modify three diverse visual recognition systems to use fast feature pyramids and show results on both pedestrian detection (measured on the Caltech, INRIA, TUD-Brussels and ETH data sets) and general object detection (measured on the PASCAL VOC). The approach is general and is widely applicable to vision algorithms requiring fine-grained multi-scale analysis. Our approximation is valid for images with broad spectra (most natural images) and fails for images with narrow band-pass spectra (e.g., periodic textures).
---
paper_title: Mobile Collaborative Augmented Reality: the Augmented Stroll

paper_content:

The paper focuses on Augmented Reality systems in which interaction with the real world is augmented by the computer, the task being performed in the real world. We first define what mobile AR systems, collaborative AR systems and finally mobile and collaborative AR systems are. We then present the augmented stroll and its software design as one example of a mobile and collaborative AR system. The augmented stroll is applied to Archaeology in the MAGIC (Mobile Augmented Group Interaction in Context) project.
---
paper_title: Location-based Mobile Augmented Reality Applications - Challenges, Examples, Lessons Learned

paper_content:

The technical capabilities of modern smart mobile devices more and more enable us to run desktop-like applications with demanding resource requirements in mobile environments. Along this trend, numerous concepts, techniques, and prototypes have been introduced, focusing on basic implementation issues of mobile applications. However, only little work exists that deals with the design and implementation (i.e., the engineering) of advanced smart mobile applications and reports on the lessons learned in this context. In this paper, we give profound insights into the design and implementation of such an ::: advanced mobile application, which enables location-based mobile augmented reality on two different mobile operating systems (i.e., iOS and Android). In particular, this kind of mobile application is characterized by high resource ::: demands since various sensors must be queried at run time and numerous virtual objects may have to be drawn in realtime on the screen of the smart mobile device (i.e., a high frame count per second be caused). We focus on the ::: efficient implementation of a robust mobile augmented reality engine, which provides location-based functionality, as well as the implementation of mobile business applications based on this engine. In the latter context, we also discuss the lessons learned when implementing mobile business applications with our mobile augmented reality engine.
---
paper_title: CloudRidAR: a cloud-based architecture for mobile augmented reality

paper_content:

Mobile augmented reality (MAR) has exploded in popularity on mobile devices in various fields. However, building a MAR application from scratch on mobile devices is complicated and time-consuming. In this paper, we propose CloudRidAR, a framework for MAR developers to facilitate the development, deployment, and maintenance of MAR applications with little effort. Despite of advance in mobile devices as a computing platform, their performance for MAR applications is still very limited due to the poor computing capability of mobile devices. In order to alleviate the problem, our CloudRidAR is designed with cloud computing at the core. Computational intensive tasks are offloaded on cloud to accelerate computation in order to guarantee run-time performance. We also present two MAR applications built on CloudRidAR to evaluate our design.
---
paper_title: Telegeoinformatics: Location-based Computing and Services

paper_content:

THEORIES AND TECHNOLOGIES Telegeoinformatics: Current Trends and Future Direction Introduction Architecture Internet-Based GIS Spatial Databases Intelligent Query Analyzer (IQA) Predictive Computing Adaptation Final Remarks References Remote Sensing Introductory Concepts Remote Sensing Systems Imaging Characteristics of Remote Sensing Systems Active Microwave Remote Sensing Extraction of Thematic Information from Remotely Sensed Imagery Extraction of Metric Information from Remotely Sensed Imagery Remote Sensing in Telegeoinformatics References Positioning and Tracking Approaches and Technologies Introduction Global Positioning System Positioning Methods Based on Cellular Networks Other Positioning and Tracking Techniques: An Overview Hybrid Systems Summary References Wireless Communications Introduction Overview of Wireless Systems Radio Propagation and Physical Layer Issues Medium Access in Wireless Networks Network Planning, Design and Deployment Wireless Network Operations Conclusions and the Future References INTEGRATED DATA AND TECHNOLOGIES Chapter Five: Location-Based Computing Introduction LBC Infrastructure Location-Based Interoperability Location-Based Data Management Adaptive Location-Based Computing Location-Based Routing as Adaptive LBC Concluding Remarks References Location-Based Services Introduction Types of Location-Based Services What is Unique About Location-Based Services? Enabling Technologies Market for Location-Based Services Importance of Architecture and Standards Example Location-Based Services: J-Phone J-Navi (Japan) Conclusions References Wearable Tele-Informatic Systems for Personal Imaging Introduction Humanistic Intelligence as a Basis for Intelligent Image Processing Humanistic Intelligence 'WEARCOMP' as a Means of Realizing Humanistic Intelligence Where on the Body Should a Visual Tele-Informatic Device be Placed? Telepointer: Wearable Hands-Free Completely Self Contained Visual Augmented Reality Without Headwear and Without any Infrastructural Reliance Portable Personal Pulse Doppler Radar Vision System When Both the Camera and Display are Headword: Personal Imaging and Mediated Reality Personal Imaging for Location-Based Services Reality Window Manager (RWM) Personal Telegeoinformatics: Blocking Spam with a Photonic Filter Conclusion References Mobile Augmented Reality Introduction MARS: Promises, Applications, and Challenges Components and Requirements MARS UI Concepts Conclusions Acknowledgements References APPLICATIONS Emergency Response Systems Overview of Emergency Response Systems State-of-the-Art ERSs Examples of Developing ERSs for Earthquakes and Other Disasters Future Aspects of Emergency Response Systems Concluding Remarks References Location-Based Computing for Infrastructure Field Tasks Introduction LBC-Infra Concept Technological Components of LBC-Infra General Requirements of LBC-Infra Interaction Patterns and Framework of LBC-Infra Prototype System and Case Study Conclusions References The Role of Telegeoinformatics in ITS Introduction to Intelligent Tranaportation Systems Telegeoinformatics Within ITS The Role of Positioning Systems In ITS Geospatial Data for ITS Communication Systems in ITS ITS-Telegeoinformatics Applications Non-Technical Issues Impacting on ITS Concluding Remarks Remarks The Impact and Penetration of Location-Based Services The Definition of Technologies LBSs: Definitions, Software, and Usage The Market for LBSs: A Model of the Development of LBSs Penetration of Mobile Devices: Predictions of Future Markets Impacts of LBSs on Geographical Locations Conclusions References
---
paper_title: ReadMe: A Real-Time Recommendation System for Mobile Augmented Reality Ecosystems

paper_content:

We introduce ReadMe, a real-time recommendation system (RS) and an online algorithm for Mobile Augmented Reality (MAR) ecosystems. A MAR ecosystem is the one that contains mobile users and virtual objects. The role of ReadMe is to detect and present the most suitable virtual objects on the mobile user's screen. The selection of the proper virtual objects depends on the mobile users' context. We consider the user's context as a set of variables that can be either drawn directly by user's device or can be inferred by it or can be collected in collaboration with other mobile devices.
---
paper_title: A Survey of Augmented Reality

paper_content:

This survey summarizes almost 50 years of research and development in the field of Augmented Reality AR. From early research in the1960's until widespread availability by the 2010's there has been steady progress towards the goal of being able to seamlessly combine real and virtual worlds. We provide an overview of the common definitions of AR, and show how AR fits into taxonomies of other related technologies. A history of important milestones in Augmented Reality is followed by sections on the key enabling technologies of tracking, display and input devices. We also review design guidelines and provide some examples of successful AR applications. Finally, we conclude with a summary of directions for future work and a review of some of the areas that are currently being researched.
---
paper_title: Real-time computer vision with OpenCV

paper_content:

Mobile computer-vision technology will soon become as ubiquitous as touch interfaces.
---
paper_title: OverLay: Practical Mobile Augmented Reality

paper_content:

The idea of augmented reality - the ability to look at a physical object through a camera and view annotations about the object - is certainly not new. Yet, this apparently feasible vision has not yet materialized into a precise, fast, and comprehensively usable system. This paper asks: What does it take to enable augmented reality (AR) on smartphones today? To build a ready-to-use mobile AR system, we adopt a top-down approach cutting across smartphone sensing, computer vision, cloud offloading, and linear optimization. Our core contribution is in a novel location-free geometric representation of the environment - from smartphone sensors - and using this geometry to prune down the visual search space. Metrics of success include both accuracy and latency of object identification, coupled with the ease of use and scalability in uncontrolled environments. Our converged system, OverLay, is currently deployed in the engineering building and open for use to regular public; ongoing work is focussed on campus-wide deployment to serve as a "historical tour guide" of UIUC. Performance results and user responses thus far have been promising, to say the least.
---
paper_title: Offloading Guidelines for Augmented Reality Applications on Wearable Devices

paper_content:

As Augmented Reality (AR) gets popular on wearable devices such as Google Glass, various AR applications have been developed by leveraging synergetic benefits beyond the single technologies. However, the poor computational capability and limited power capacity of current wearable devices degrade runtime performance and sustainability. Computational offloading strategy has been proposed to outsource computation to remote cloud for improving performance. Nevertheless, comparing with mobile devices, the wearable devices have their specific limitations, which induce additional problems and require new thoughts of computational offloading. In this paper, we propose several guidelines of computational offloading for AR applications on wearable devices based on our practical experiences of designing and developing AR applications on Google Glass. The guidelines have been adopted and proved by our application prototypes.
---
paper_title: Real-time visual tracking of less textured three-dimensional objects on mobile platforms

paper_content:

Natural feature-based approaches are still challenging for mobile applications (e.g., mobile augmented reality), because they are feasible only in limited environments such as highly textured and planar scenes/objects, and they need powerful mobile hardware for fast and reliable tracking. In many cases where conventional approaches are not effective, three-dimensional (3-D) knowledge of target scenes would be beneficial. We present a well-established framework for real-time visual tracking of less textured 3-D objects on mobile platforms. Our framework is based on model-based tracking that efficiently exploits partially known 3-D scene knowledge such as object models and a background’s distinctive geometric or photometric knowledge. Moreover, we elaborate on implementation in order to make it suitable for real-time vision processing on mobile hardware. The performance of the framework is tested and evaluated on recent commercially available smartphones, and its feasibility is shown by real-time demonstrations.
---
paper_title: From Virtuality to Reality and Back

paper_content:

There has been a growing research interest in investigating techniques to combine real and virtual spaces. A variety of “reality” concepts such as Virtual Reality and Augmented Reality and their supporting technologies have emerged in the field of design to adopt the task of replacing or merging our physical world with the virtual world. The different realities can be tailored to enhance comprehension for specific design activities along a design life-cycle. This paper presents stateof-the-art applications of these “reality” concepts in design and related areas, and proposes a classification of these realities to address suitability issues for the effective utilization of the concepts and technologies. Their potentials and implications in certain design activities are also discussed.
---
paper_title: Augmented reality navigation systems

paper_content:

The augmented reality (AR) research community has been developing a manifold of ideas and concepts to improve the depiction of virtual objects in a real scene. In contrast, current AR applications require the use of unwieldy equipment which discourages their use. In order to essentially ease the perception of digital information and to naturally interact with the pervasive computing landscape, the required AR equipment has to be seamlessly integrated into the user’s natural environment. Considering this basic principle, this paper proposes the car as an AR apparatus and presents an innovative visualization paradigm for navigation systems that is anticipated to enhance user interaction.
---
paper_title: Recent Advances in Augmented Reality

paper_content:

In 1997, Azuma published a survey on augmented reality (AR). Our goal is to complement, rather than replace, the original survey by presenting representative examples of the new advances. We refer one to the original survey for descriptions of potential applications (such as medical visualization, maintenance and repair of complex equipment, annotation, and path planning); summaries of AR system characteristics (such as the advantages and disadvantages of optical and video approaches to blending virtual and real, problems in display focus and contrast, and system portability); and an introduction to the crucial problem of registration, including sources of registration error and error-reduction strategies.
---
paper_title: General-purpose modular hardware and software framework for mobile outdoor augmented reality applications in engineering

paper_content:

This paper presents a reusable, general-purpose, mobile augmented reality (AR) framework developed to address the critical and repetitive challenges specific to visualization in outdoor AR. In all engineering applications of AR developed thus far, basic functionality that supports accurate user registration, maximizes the range of user motion, and enables data input and output has had to be repeatedly re-implemented. This is primarily due to the fact that designed methods have been traditionally custom created for their respective applications and are not generic enough to be readily shared and reused by others. The objective of this research was to remedy this situation by designing and implementing a reusable and pluggable hardware and software framework that can be used in any AR application without the need to re-implement low-level communication interfaces with selected hardware. The underlying methods of hardware communication as well as the object-oriented design (OOD) of the reusable interface are presented. Details on the validation of framework reusability and pluggability are also described.
---
paper_title: Location-based Mobile Augmented Reality Applications - Challenges, Examples, Lessons Learned

paper_content:

The technical capabilities of modern smart mobile devices more and more enable us to run desktop-like applications with demanding resource requirements in mobile environments. Along this trend, numerous concepts, techniques, and prototypes have been introduced, focusing on basic implementation issues of mobile applications. However, only little work exists that deals with the design and implementation (i.e., the engineering) of advanced smart mobile applications and reports on the lessons learned in this context. In this paper, we give profound insights into the design and implementation of such an ::: advanced mobile application, which enables location-based mobile augmented reality on two different mobile operating systems (i.e., iOS and Android). In particular, this kind of mobile application is characterized by high resource ::: demands since various sensors must be queried at run time and numerous virtual objects may have to be drawn in realtime on the screen of the smart mobile device (i.e., a high frame count per second be caused). We focus on the ::: efficient implementation of a robust mobile augmented reality engine, which provides location-based functionality, as well as the implementation of mobile business applications based on this engine. In the latter context, we also discuss the lessons learned when implementing mobile business applications with our mobile augmented reality engine.
---
paper_title: Managing Complex Augmented Reality Models

paper_content:

Mobile augmented reality requires georeferenced data to present world-registered overlays. To cover a wide area and all artifacts and activities, a database containing this information must be created, stored, maintained, delivered, and finally used by the client application. We present a data model and a family of techniques to address these needs.
---
paper_title: Relevant Aspects for the Integration of Linked Data in Mobile Augmented Reality Applications for Tourism

paper_content:

Mobile augmented reality applications have seen tremendous growth in recent years and tourism is one of the fields in which this set of technologies has been proved to be a natural fit. Augmented reality has the potential of enhancing the surroundings of the tourist in a meaningful way. In order to provide personalized and rich content for the augmented reality application, researchers have explored the use of Semantic Web and especially Linked Data principles and technologies. In this paper we review existing projects at the intersection of these technologies and current aspects, not necessarily specific, but highly relevant to the integration of Linked Open Data in mobile augmented reality applications for tourism. In this respect, we discuss approaches in the area of geodata integration, quality of the open data, provenance information and trust. We conclude with recommendations regarding future research in this area.
---
paper_title: PhoneGuide: museum guidance supported by on-device object recognition on mobile phones

paper_content:

We present PhoneGuide -- an enhanced museum guidance system that uses camera-equipped mobile phones and on-device object recognition.Our main technical achievement is a simple and light-weight object recognition approach that is realized with single-layer perceptron neuronal networks. In contrast to related systems which perform computationally intensive image processing tasks on remote servers, our intention is to carry out all computations directly on the phone. This ensures little or even no network traffic and consequently decreases cost for online times. Our laboratory experiments and field surveys have shown that photographed museum exhibits can be recognized with a probability of over 90%.We have evaluated different feature sets to optimize the recognition rate and performance. Our experiments revealed that normalized color features are most effective for our method. Choosing such a feature set allows recognizing an object below one second on up-to-date phones. The amount of data that is required for differentiating 50 objects from multiple perspectives is less than 6KBytes.
---
paper_title: Archeoguide: system architecture of a mobile outdoor augmented reality system

paper_content:

We present the system architecture of a mobile outdoor augmented reality system for the Archeoguide project. We begin with a short introduction to the project. Then we present the hardware we chose for the mobile system and we describe the system architecture we designed for the software implementation. We conclude this paper with the first results obtained from experiments we made during our trials at ancient Olympia in Greece.
---
paper_title: OverLay: Practical Mobile Augmented Reality

paper_content:

The idea of augmented reality - the ability to look at a physical object through a camera and view annotations about the object - is certainly not new. Yet, this apparently feasible vision has not yet materialized into a precise, fast, and comprehensively usable system. This paper asks: What does it take to enable augmented reality (AR) on smartphones today? To build a ready-to-use mobile AR system, we adopt a top-down approach cutting across smartphone sensing, computer vision, cloud offloading, and linear optimization. Our core contribution is in a novel location-free geometric representation of the environment - from smartphone sensors - and using this geometry to prune down the visual search space. Metrics of success include both accuracy and latency of object identification, coupled with the ease of use and scalability in uncontrolled environments. Our converged system, OverLay, is currently deployed in the engineering building and open for use to regular public; ongoing work is focussed on campus-wide deployment to serve as a "historical tour guide" of UIUC. Performance results and user responses thus far have been promising, to say the least.
---
paper_title: A Novel Individual Location Recommendation System Based on Mobile Augmented Reality

paper_content:

In this paper, a mobile augmented reality (AR) based system for individual location recommendation is proposed. The system consists of the modules of user location, marker detection, 3D display and location guide by combing AR with navigation technology. Experimental results demonstrate that the proposed system is both efficient and effective in helping people location their position, it also enhance users' senses experience by providing users with a more intuitive, three-dimensional, dynamic information display and sharing capabilities.
---
paper_title: 3DVN: A Mixed Reality Platform for Mobile Navigation Assistance

paper_content:

We present 3DVN, a Mixed Reality platform for navigation assistance in indoor environments. Built on top of a PC-based wearable computer running the Windows operating system, the platform provides a multimodal user interface for navigating in existing physical buildings, including wireless networking, data glove gestural input, voice communication, head-mounted display, and a 3DOF head tracker. Local positioning is performed using a software-only positioning engine making use of WLAN features for accurately pinpointing the mobile device in three dimensions. The 3DVN system has been implemented as a proof-of-concept of a navigation guidance support device for visitors, and has been field-tested with human subjects in the student center of our university. Informal evaluation of subject ratings indicate a strong interest for the new system, both for guests as well as people familiar with the campus and the building.
---
paper_title: User Interface Management Techniques for Collaborative Mobile Augmented Reality

paper_content:

Mobile Augmented Reality Systems (MARS) have the potential to revolutionize the way in which information is provided to users. Virtual information can be directly integrated with the real world surrounding the mobile user, who can interact with it to display related information, to pose and resolve queries, and to collaborate with other users. However, we believe that the benefits of MARS will only be achieved if the user interface (UI) is actively managed so as to maximize the relevance and minimize the confusion of the virtual material relative to the real world. This article addresses some of the steps involved in this process, focusing on the design and layout of the mobile user’s overlaid virtual environment. The augmented view of the user’s surroundings presents an interface to context-dependent operations, many of which are related to the objects in view—the augmented world is the user interface. We present three user interface design techniques that are intended to make this interface as obvious and clear to the user as possible: information filtering, UI component design, and view management. Information filtering helps select the most relevant information to present to the user. UI component designdetermines the format in which this information should be conveyed, based on the available display resources and tracking accuracy. For example, the absence of high accuracy position tracking would favor body- or screenstabilized components over world-stabilized ones that would need to be exactly registered with the physical objects to which they refer. View management attempts to ensure that the virtual objects that are displayed visually are arranged appropriately with regard to their projections on the view plane. For example, the relationships among objects should be as unambiguous as possible, and physical or virtual objects should not obstruct the user’s view of more important physical or virtual objects in the scene. We illustrate these interface design techniques using our prototype collaborative, cross-site MARS environment, which is composed of mobile and non-mobile augmented reality and virtual reality systems.
---
paper_title: Face to face collaborative AR on mobile phones

paper_content:

Mobile phones are an ideal platform for augmented reality. In this paper we describe how they also can be used to support face to face collaborative AR applications. We have created a custom port of the ARToolKit library to the Symbian mobile phone operating system and then developed a sample collaborative AR game based on this. We describe the game in detail and user feedback from people who have played it. We also provide general design guidelines that could be useful for others who are developing mobile phone collaborative AR applications.
---
paper_title: Mobile Collaborative Augmented Reality: the Augmented Stroll

paper_content:

The paper focuses on Augmented Reality systems in which interaction with the real world is augmented by the computer, the task being performed in the real world. We first define what mobile AR systems, collaborative AR systems and finally mobile and collaborative AR systems are. We then present the augmented stroll and its software design as one example of a mobile and collaborative AR system. The augmented stroll is applied to Archaeology in the MAGIC (Mobile Augmented Group Interaction in Context) project.
---
paper_title: Implementation of Augmented Reality System for Smartphone Advertisements

paper_content:

The recent development and popularization of personal mobile devices has brought a lot of changes into information search and offering. In particular, the newly emerged augmented reality system made possible an innovative way of information acquisition by providing additional information of the virtual world to the real world. This work used the markerless augmented reality system on smartphones to design and implement the smartphone application service aimed at efficiently conveying information on advertisements to users. The conventional advertising applications simply introduce and explain goods by inducing consumers to have an interest, whereas the markerless application service developed in this work interacts with the augmented reality system and database management system to quickly provide more accurate and diversified information to users.
---
paper_title: Do you remember that building? Exploring old Zakynthos through an augmented reality mobile game

paper_content:

This paper presents a mobile augmented reality application, that was designed with the objective of visitors, to experience the historical center of old Zakynthos, Greece, that was destroyed after an earthquake, and allow the users to re-live the atmosphere and life of the historic place. Special attention is given to the mental model of the landmarks developed by the users after interacting with the application, discussing some of the observed flaws of this model.
---
paper_title: Expected user experience of mobile augmented reality services: a user study in the context of shopping centres

paper_content:

The technical enablers for mobile augmented reality (MAR) are becoming robust enough to allow the development of MAR services that are truly valuable for consumers. Such services would provide a novel interface to the ubiquitous digital information in the physical world, hence serving in great variety of contexts and everyday human activities. To ensure the acceptance and success of future MAR services, their development should be based on knowledge about potential end users' expectations and requirements. We conducted 16 semi-structured interview sessions with 28 participants in shopping centres, which can be considered as a fruitful context for MAR services. We aimed to elicit new knowledge about (1) the characteristics of the expected user experience and (2) central user requirements related to MAR in such a context. From a pragmatic viewpoint, the participants expected MAR services to catalyse their sense of efficiency, empower them with novel context-sensitive and proactive functionalities and raise their awareness of the information related to their surroundings with an intuitive interface. Emotionally, MAR services were expected to offer stimulating and pleasant experiences, such as playfulness, inspiration, liveliness, collectivity and surprise. The user experience categories and user requirements that were identified can serve as targets for the design of user experience of future MAR services.
---
paper_title: Enabling smart retail settings via mobile augmented reality shopping apps

paper_content:

Retail settings are being challenged to become smarter and provide greater value to both consumers and retailers. An increasingly recognised approach having potential for enabling smart retail is mobile augmented reality (MAR) apps. In this research, we seek to describe and discover how, why and to what extent MAR apps contribute to smart retail settings by creating additional value to customers as well as benefiting retailers. In particular, by adopting a retail customer experience perspective on value creation, analysing the content of MAR shopping apps currently available, and conducting large-scale surveys on United States smartphone users representing early technology adopters, we assess level of use, experiential benefits offered, and retail consequences. Our findings suggest that take-up is set to go mainstream as user satisfaction is relatively high and their use provides systematic experiential benefits along with advantages to retailers. Despite some drawbacks, their use is positively associated with multiple retail consequences. MAR apps are seen as changing consumer behaviour and are associated with increasingly high user valuations of retailers offering them. Implications for more effective use to enable smart retail settings are discussed.
---
paper_title: Like bees around the hive: a comparative study of a mobile augmented reality map

paper_content:

We present findings from field trials of MapLens, a mobile augmented reality (AR) map using a magic lens over a paper map. Twenty-six participants used MapLens to play a location-based game in a city centre. Comparisons to a group of 11 users with a standard 2D mobile map uncover phenomena that arise uniquely when interacting with AR features in the wild. The main finding is that AR features facilitate place-making by creating a constant need for referencing to the physical, and in that it allows for ease of bodily configurations for the group, encourages establishment of common ground, and thereby invites discussion, negotiation and public problem-solving. The main potential of AR maps lies in their use as a collaborative tool.
---
paper_title: Human pacman: A mobile entertainment system with ubiquitous computing and tangible interaction over a wide outdoor area

paper_content:

Human Pacman is an interactive role-playing game that envisions to bring the computer gaming experience to a new level of emotional and sensory gratification by setting the real world as a playground. This is a physical fantasy game integrated with human-social and mobile-gaming that emphasizes on collaboration and competition between players. By setting the game in a wide outdoor area, natural human-physical movements have become an integral part of the game. Pacmen and Ghosts are now human players in the real world experiencing mixed reality visualization from the wearable computers on them. Virtual cookies and actual physical objects are incorporated to provide novel experiences of seamless transitions between real and virtual worlds and tangible human computer interface respectively. We believe Human Pacman is pioneering a new form of gaming that anchors on physicality, mobility, social interaction, and ubiquitous computing.
---
paper_title: A Survey of Augmented Reality

paper_content:

This paper surveys the field of augmented reality AR, in which 3D virtual objects are integrated into a 3D real environment in real time. It describes the medical, manufacturing, visualization, path planning, entertainment, and military applications that have been explored. This paper describes the characteristics of augmented reality systems, including a detailed discussion of the tradeoffs between optical and video blending approaches. Registration and sensing errors are two of the biggest problems in building effective augmented reality systems, so this paper summarizes current efforts to overcome these problems. Future directions and areas requiring further research are discussed. This survey provides a starting point for anyone interested in researching or using augmented reality.
---
paper_title: Augmented Reality in the Classroom

paper_content:

Evaluations of AR experiences in an educational setting provide insights into how this technology can enhance traditional learning models and what obstacles stand in the way of its broader use. A related video can be seen here: http://youtu.be/ndUjLwcBIOw. It shows examples of augmented reality experiences in an educational setting.
---
paper_title: The Use of Augmented Reality Games in Education: A Review of the Literature.

paper_content:

This paper provides a review of the literature about the use of augmented reality in education and specifically in the context of formal and informal environments. It examines the research that has been conducted up to date on the use of those games through mobile technology devices such as mobile phones and tablets, both in primary and secondary education. The review of the relative literature was carried out in the period 2000 to early 2014 in ScienceDirect and ERIC. The outcomes of this review illustrated a set of studies that provide evidence of positive outcomes regarding student learning. These studies, which focused mainly on the natural sciences and took place within informal learning environments, used both qualitative and quantitative data collection methods. The earliest study that was conducted about this topic used personal digital assistants, while the more recent one used smart (mobile) phones and tablets. The services of augmented reality focused on markers/quick response codes, virtual it...
---
paper_title: Construct3D: an augmented reality application for mathematics and geometry education

paper_content:

Construct3D is a three dimensional geometry construction tool specifically designed for mathematics and geometry education. It is based on the mobile collaborative augmented reality system "Studierstube". We describe our efforts in developing a system for the improvement of spatial abilities and maximization of transfer of learning. Means of application and integration in mathematics and geometry education at high school as well as university level are being discussed. Anecdotal evidence supports our claim that Construct3D is easy to learn, encourages experimentation with geometric constructions and improves spatial skills.
---
paper_title: Augmented Reality Trends in Education: A Systematic Review of Research and Applications

paper_content:

In recent years, there has been an increasing interest in applying Augmented Reality (AR) to create unique educational settings. So far, however, there is a lack of review studies with focus on investigating factors such as: the uses, advantages, limitations, effectiveness, challenges and features of augmented reality in educational settings. Personalization for promoting an inclusive learning using AR is also a growing area of interest. This paper reports a systematic review of literature on augmented reality in educational settings considering the factors mentioned before. In total, 32 studies published between 2003 and 2013 in 6 indexed journals were analyzed. The main findings from this review provide the current state of the art on research in AR in education. Furthermore, the paper discusses trends and the vision towards the future and opportunities for further research in augmented reality for educational settings.
---
paper_title: Current status, opportunities and challenges of augmented reality in education

paper_content:

Although augmented reality (AR) has gained much research attention in recent years, the term AR was given different meanings by varying researchers. In this article, we first provide an overview of definitions, taxonomies, and technologies of AR. We argue that viewing AR as a concept rather than a type of technology would be more productive for educators, researchers, and designers. Then we identify certain features and affordances of AR systems and applications. Yet, these compelling features may not be unique to AR applications and can be found in other technological systems or learning environments (e.g., ubiquitous and mobile learning environments). The instructional approach adopted by an AR system and the alignment among technology design, instructional approach, and learning experiences may be more important. Thus, we classify three categories of instructional approaches that emphasize the ''roles,'' ''tasks,'' and ''locations,'' and discuss what and how different categories of AR approaches may help students learn. While AR offers new learning opportunities, it also creates new challenges for educators. We outline technological, pedagogical, learning issues related to the implementation of AR in education. For example, students in AR environments may be cognitively overloaded by the large amount of information they encounter, the multiple technological devices they are required to use, and the complex tasks they have to complete. This article provides possible solutions for some of the challenges and suggests topics and issues for future research.
---
paper_title: Augmented Reality Teaching and Learning

paper_content:

This literature review focuses on augmented realities (AR) for learning that utilize mobile, context-aware technologies (e.g., smartphones, tablets), which enable participants to interact with digital information embedded within the physical environment. We summarize research findings about AR in formal and informal learning environments (i.e., schools, universities, museums, parks, zoos, etc.), with an emphasis on the affordances and limitations associated with AR as it relates to teaching, learning, and instructional design. As a cognitive tool and pedagogical approach, AR is primarily aligned with situated and constructivist learning theory, as it positions the learner within a real-world physical and social context while guiding, scaffolding and facilitating participatory and metacognitive learning processes such as authentic inquiry, active observation, peer coaching, reciprocal teaching and legitimate peripheral participation with multiple modes of representation.
---
paper_title: “Studierstube”: An environment for collaboration in augmented reality

paper_content:

We propose an architecture for multi-user augmented reality with applications in visualisation, presentation and education, which we call "Studierstube". Our system presents three-dimensional stereoscopic graphics simultaneously to a group of users wearing light weight see-through head mounted displays. The displays do not affect natural communication and interaction, making working together very effective. Users see the same spatially aligned model, but can independently control their viewpoint and different layers of the data to be displayed. The setup serves computer supported cooperative work and enhances cooperation of visualisation experts. This paper presents the client-server software architecture underlying this system and details that must be addressed to create a high-quality augmented reality setup.
---
paper_title: Mystery at the Museum – A Collaborative Game for Museum Education

paper_content:

Through an iterative design process involving museum educators, learning scientists and technologists, and drawing upon our previous experiences in handheld game design and a growing body of knowledge on learning through gaming, we designed an interactive mystery game called Mystery at the Museum (the High Tech Whodunnit), which was designed for synchronous play of groups of parents and children over a two to three hour period. The primary design goals were to engage visitors more deeply in the museum, engage visitors more broadly across museum exhibits, and encourage collaboration between visitors. The feedback from the participants suggested that the combination of depth and breadth was engaging and effective in encouraging them to think about the museum's exhibits. The roles that were an integral part of the game turned out to be extremely effective in engaging pairs of participants with one another. Feedback from parents was quite positive in terms of how they felt it engaged them and their children. These results suggest that further explorations of technology-based museum experiences of this type are wholly appropriate.
---
paper_title: Augmented Reality and Mobile Learning: the State of the Art

paper_content:

In this paper, the authors examine the state of the art in augmented reality AR for mobile learning. Previous work in the field of mobile learning has included AR as a component of a wider toolkit but little has been done to discuss the phenomenon in detail or to examine in a balanced fashion its potential for learning, identifying both positive and negative aspects. The authors seek to provide a working definition of AR and to examine how it can be embedded within situated learning in outdoor settings. The authors classify it according to key aspects device/technology, mode of interaction/learning design, type of media, personal or shared experiences, whether the experience is portable or static, and the learning activities/outcomes. The authors discuss the technical and pedagogical challenges presented by AR, before looking at ways in which it can be used for learning. Finally, the paper looks ahead to AR technologies that may be employed in the future.
---
paper_title: Augmented reality in education: a meta-review and cross-media analysis

paper_content:

Augmented reality (AR) is an educational medium increasingly accessible to young users such as elementary school and high school students. Although previous research has shown that AR systems have the potential to improve student learning, the educational community remains unclear regarding the educational usefulness of AR and regarding contexts in which this technology is more effective than other educational mediums. This paper addresses these topics by analyzing 26 publications that have previously compared student learning in AR versus non-AR applications. It identifies a list of positive and negative impacts of AR experiences on student learning and highlights factors that are potentially underlying these effects. This set of factors is argued to cause differences in educational effectiveness between AR and other media. Furthermore, based on the analysis, the paper presents a heuristic questionnaire generated for judging the educational potential of AR experiences.
---
paper_title: An Augmented Reality-Based Mobile Learning System to Improve Students' Learning Achievements and Motivations in Natural Science Inquiry Activities

paper_content:

Introduction Recently, the advancement and popularity of handheld devices and sensing technologies has enabled researchers to implement more effective learning methods (Ogata, Li, Hou, Uosaki, El-Bishouty, & Yano, 2011). Several studies have reported the importance of conducting contextual learning and experiential learning in real-world environments, encouraging the use of mobile and sensing technologies in outdoor learning activities (Chu, Hwang, Tsai, & Tseng, 2010; Hung, Hwang, Lin, Wu, & Su, 2013; Yang, 2006). For example, Chu, Hwang, Huang, and Wu (2008) developed a learning system that guided students to learn about the characteristics and life cycle of plants on a school campus using mobile communication and RFID (Radio Frequency Identification). Most of mobile learning studies emphasize the adoption of digital learning aids in real-life scenarios (Sharples, Milrad, Arnedillo-Sanchez, & Vavoula, 2009; Ogata & Yano, 2004; Wong & Looi, 2011). However, regarding supplementary mobile learning aids, the interaction between digital learning aids and the actual environment needs to be emphasized to enable students to effectively manage and incorporate personal knowledge (Wu, Lee, Chang, & Liang, 2013). For example, it is expiated that students can select a virtual learning object from the actual environment using a mobile learning aid, which allows them to obtain a first-hand understanding of the learning environment and, subsequently, increases their learning motivations and experiences. Such a learning support technology is achievable through the use of Augmented Reality (AR), which combines human senses (e.g., sight, sound, and touch) with virtual objects to facilitate real-world environment interactions for users to achieve an authentic perception of the environment (Azuma, 1997). For example, users who employ mobile devices with AR facilities to seek a target building on a street are able to see additional information surrounding individual buildings when they browse the buildings via the camera of their mobile device. Researchers have documented the potential of employing such facilities to assist students in learning in real-world environments in comparisons with traditional instructions (Andujar, Mejias, & Marquez, 2011; Chen, Chi, Hung, & Kang, 2011; Kamarainen, Metcalf, Grotzer, Browne, Mazzuca, Tutwiler, & Dede, 2013; Platonov, Heibel, Meier, & Grollmann, 2006), which showed that AR technology contributed to improve academic achievement compared to traditional teaching methods. On the other hand, numerous educators have contended that computer technology cannot support the learning process entirely; instead, the primary function of computer technology involves a knowledge building tool for students (Jonassen, Carr, & Yueh, 1998). Effective learning strategies remain the most crucial factor for increasing learning motivation. Therefore, effective learning strategies supplemented with appropriate computer technology can greatly enhance learning motivation (Chu, Hwang, & Tsai, 2010; Jonassen, 1999; Hwang, Tsai, Chu, Kinshuk, & Chen, 2012; Jonassen et al., 1998). Previous studies have highlighted that inquiry-based learning strategies supplemented by computer technology in a scenario-based learning environment can effectively increase learning motivation (Shih, Chuang, & Hwang, 2010; Soloway & Wallace, 1997). Inquiry-based learning strategies are student-centric knowledge exploration activities; the teacher serves as a guide, employing structured methods that train and encourage students to learn proactively (Hwang, Wu, Zhuang, & Huang, 2013; Soloway & Wallace, 1997). When students acquire the methods for problem-solving, they use the obtained information to establish a hypothesis or to plan solutions to the problem (Looi, 1998). Consequently, in this study, an innovative learning approach is proposed to support inquiry-based learning activities with mobile AR. …
---
paper_title: Mixed reality training application for an oil refinery: user requirements

paper_content:

Introducing mixed reality (MR) into safety-critical environment like oil refinery is difficult, since the environment and organization lays demanding restrictions for the application. In order to develop usable and safe MR application, we need to study the context of use and derive user requirements from it. This paper describes the user requirements for an MR based oil refinery training tool. The application is aimed to train employees of a specific process unit in the refinery. Training is currently done mainly in a classroom and on-site only when the process is closed down. On-site training is necessary, but expensive and rarely possible. The use of mixed reality offers a way to train employees on-site while the process is running. Users can virtually see "inside" the columns and can modify virtually the process.
---
paper_title: Impact of an augmented reality system on students' motivation for a visual art course

paper_content:

In this paper, the authors show that augmented reality technology has a positive impact on the motivation of middle-school students. The Instructional Materials Motivation Survey (IMMS) (Keller, 2010) based on the ARCS motivation model (Keller, 1987a) was used to gather information; it considers four motivational factors: attention, relevance, confidence, and satisfaction. Motivational factors of attention and satisfaction in an augmented-reality-based learning environment were better rated than those obtained in a slides-based learning environment. When the impact of the augmented reality system was analyzed in isolation, the attention and confidence factors were the best rated. The usability study showed that although this technology is not mature enough to be used massively in education, enthusiasm of middle-school students diminished most of the barriers found.
---
paper_title: Review of Augmented Paper Systems in Education: An Orchestration Perspective

paper_content:

Augmented paper has been proposed as a way to integrate more easily ICTs in settings like formal education, where paper has a strong presence. However, despite the multiplicity of educational applications using paper-based computing, their deployment in authentic settings is still marginal. To better understand this gap between research proposals and everyday classroom application, we surveyed the field of augmented paper systems applied to education, using the notion of "classroom orchestration" as a conceptual tool to understand its potential for integration in everyday educational practice. Our review organizes and classifies the affordances of these systems, and reveals that comparatively few studies provide evidence about the learning effects of system usage, or perform evaluations in authentic setting conditions. The analysis of those proposals that have performed authentic evaluations reveals how paper based-systems can accommodate a variety of contextual constraints and pedagogical approaches, but also highlights the need for further longitudinal, in-the-wild studies, and the existence of design tensions that make the conception, implementation and appropriation of this kind of systems still challenging.
---
paper_title: Development and behavioral pattern analysis of a mobile guide system with augmented reality for painting appreciation instruction in an art museum

paper_content:

A mobile guide system that integrates art appreciation instruction with augmented reality (AR) was designed as an auxiliary tool for painting appreciation, and the learning performance of three groups of visiting participants was explored: AR-guided, audio-guided, and nonguided (i.e., without carrying auxiliary devices). The participants were 135 college students, and a quasi-experimental research design was employed. Several learning performance factors of the museum visitors aided with different guided modes were evaluated, including their learning effectiveness, flow experience, the amount of time spent focusing on the paintings, behavioral patterns, and attitude of using the guide systems. The results showed that compared to the audio- and nonguided participants, the AR guide effectively enhanced visitors' learning effectiveness, promoted their flow experience, and extended the amount of time the visitors spent focusing on the paintings. In addition, the visitors' behavioral patterns were dependent upon the guided mode that they used; the visitors who were the most engaged in the gallery experience were those who were using the AR guide. Most of the visitors using the mobile AR-guide system elicited positive responses and acceptance attitudes.
---
paper_title: BARS: Battlefield Augmented Reality System

paper_content:

Abstract : Situational awareness needs cannot be met using traditional approaches such as radios, maps and handheld displays and more powerful display paradigms are needed. We are researching mobile augmented reality (AR) through the development of the Battlefield Augmented Reality System (BARS) in collaboration with Columbia University. The system consists of a wearable computer a wireless network system and a tracked see-through Head Mounted Display (HMD). The user's perception of the environment is enhanced by superimposing graphics onto the user's field of view. The graphics are registered (aligned) with the actual environment. For example an augmented view of a building could include a wireframe plan of its interior icons to represent reported locations of snipers and the names of adjacent streets.
---
paper_title: Augmented reality in Education — Cases, places, and potentials

paper_content:

Augmented Reality is poised to profoundly transform Education as we know it. The capacity to overlay rich media onto the real world for viewing through web-enabled devices such as phones and tablet devices means that information can be made available to students at the exact time and place of need. This has the potential to reduce cognitive overload by providing students with “perfectly situated scaffolding”, as well as enable learning in a range of other ways. This paper will review uses of Augmented Reality both in mainstream society and in education, and discuss the pedagogical potentials afforded by the technology. Based on the prevalence of information delivery uses of Augmented Reality in Education, we argue the merit of having students design Augmented Reality experiences in order to develop their higher order thinking capabilities. A case study of “learning by design” using Augmented Reality in high school Visual Art is presented, with samples of student work and their feedback indicating that the...
---
paper_title: Augmented reality in medical education?

paper_content:

Learning in the medical domain is to a large extent workplace learning and involves mastery of complex skills that require performance up to professional standards in the work environment. Since training in this real-life context is not always possible for reasons of safety, costs, or didactics, alternative ways are needed to achieve clinical excellence. Educational technology and more specifically augmented reality (AR) has the potential to offer a highly realistic situated learning experience supportive of complex medical learning and transfer. AR is a technology that adds virtual content to the physical real world, thereby augmenting the perception of reality. Three examples of dedicated AR learning environments for the medical domain are described. Five types of research questions are identified that may guide empirical research into the effects of these learning environments. Up to now, empirical research mainly appears to focus on the development, usability and initial implementation of AR for learning. Limited review results reflect the motivational value of AR, its potential for training psychomotor skills and the capacity to visualize the invisible, possibly leading to enhanced conceptual understanding of complex causality.
---
paper_title: Smartphones, Smart Objects, and Augmented Reality

paper_content:

Two major types of augmented reality seem most likely to see academic use in the coming five years, markerless and marked. Markerless augmented reality uses the location determined by a cell phone to serve as a basis for adding local information to the camera view. Marked augmented reality uses a two-dimensional barcode to connect a cell phone or personal computer to information, usually on a web site. Both approaches are already being used in museums and college libraries. Marked augmented reality is especially powerful because it makes physical objects clickable, such as a web page. Augmented reality creates some exciting new opportunities for libraries.
---
paper_title: A Design of Augmented Reality System based on Real-World Illumination Environment for Edutainment

paper_content:

Recently, edutainment system using augmented reality and transparent displays were researched. However, in the case of edutainment system using augmented reality due to the incongruity of environment light conditions, the virtual objects can’t fit to real world environment with natural-looking. In this paper, we designed a tangible edutainment system for synthesizing a render layer which contains real world background illumination information in order to solve virtual object’s low natural-looking problem.
---
paper_title: SMART: a SysteM of Augmented Reality for Teaching 2 nd Grade Students

paper_content:

In this paper, we describe the design and evaluation of SMART, an educational system that uses augmented reality for teaching 2nd grade-level concepts, adequate and integrated with national curriculum guidelines. SMART puts children exploring concepts like means of transportation, types of animals and similar semantic categories through the use of a set of racquets that are used to manipulate a TV-show style game with 3D models which are superimposed to the real time video feed of the whole class. ::: ::: Experiments were performed with several classes of students in three different, local primary schools. Results suggest that SMART is effective in maintaining high levels of motivation among children, and also that SMART has a positive impact on the students' learning experience, especially among the weaker students.
---
paper_title: In-place 3D sketching for authoring and augmenting mechanical systems

paper_content:

We present a framework for authoring three-dimensional virtual scenes for Augmented Reality (AR) which is based on hand sketching. Sketches consisting of multiple components are used to construct a 3D virtual scene augmented on top of the real drawing. Model structure and properties can be modified by editing the sketch itself and printed content can be combined with hand sketches to form a single scene. Authoring by sketching opens up new forms of interaction that have not been previously explored in Augmented Reality. To demonstrate the technology, we implemented an application that constructs 3D AR scenes of mechanical systems from freehand sketches, and animates the scenes using a physics engine. We provide examples of scenes composed from trihedral solid models, forces, and springs. Finally, we describe how sketch interaction can be used to author complicated physics experiments in a natural way.
---
paper_title: Authoring of physical models using mobile computers

paper_content:

Context-aware computers rely on user and physical models to describe the context of a user. In this paper, we focus on the problem of developing and maintaining a physical model of the environment using a mobile computer. We describe a set of tools for automatically creating and modifying three-dimensional contextual information. The tools can be utilized across multiple hardware platforms, with different capabilities, and operating in collaboration with one another. We demonstrate the capabilities of the tools using two mobile platforms. One of them, a mobile augmented reality system is used to construct a geometric model of an indoor environment which is then visualized on the same platform.
---
paper_title: In-Place Sketching for content authoring in Augmented Reality games

paper_content:

Sketching leverages human skills for various purposes. In-Place Augmented Reality Sketching experiences build on the intuitiveness and flexibility of hand sketching for tasks like content creation. In this paper we explore the design space of In-Place Augmented Reality Sketching, with particular attention to content authoring in games. We propose a contextual model that offers a framework for the exploration of this design space by the research community. We describe a sketch-based AR racing game we developed to demonstrate the proposed model. The game is developed on top of our shape recognition and 3D registration library for mobile AR.
---
paper_title: APRIL: a high-level framework for creating augmented reality presentations

paper_content:

While augmented reality (AR) technology is steadily maturing, application development is still lacking advanced authoring tools - even the simple presentation of information, which should not require any programming, is not systematically addressed by development tools. Moreover, there is also a severe lack of agreed techniques or best practices for the structuring of AR content. In this paper we present APRIL, the Augmented Presentation and Interaction Language, an authoring platform for AR presentations which provides concepts and techniques that are independent of specific applications or target hardware platforms, and should be suitable to raise the level of abstraction on which AR content creators can operate.
---
paper_title: Mobile phone based AR scene assembly

paper_content:

In this paper we describe a mobile phone based Augmented Reality application for 3D scene assembly. Augmented Reality on mobile phones extends the interaction capabilities on such handheld devices. It adds a 6 DOF isomorphic interaction technique for manipulating 3D content. We give details of an application that we believe to be the first where 3D content can be manipulated using both the movement of a camera tracked mobile phone and a traditional button interface as input for transformations. By centering the scene in a tangible marker space in front of the phone we provide a mean for bimanual interaction. We describe the implementation, the interaction techniques we have developed and initial user response to trying the application.
---
paper_title: Augmented maintenance of powerplants: a prototyping case study of a mobile AR system

paper_content:

Augmented reality (AR) research has progressed in great strides over the past few years. Most current demonstrations focus on providing robust tracking solutions since this is the most critical issue when demonstrating AR systems. An issue that is typically neglected concerns the online access, analysis and visualization of information. The information required by AR demonstration systems is kept to a minimum, is prepared ahead of time, and is stored locally in the form of three-dimensional geometric descriptions. In complex mobile settings, these simplifying assumptions do not work. The authors report on recent efforts at the TU Munich to analyze the information generation, retrieval, transmission, and visualization process in the context of maintenance procedures that are performed in nuclear power plants. The use of AR to present such information online has significant implications for the way information must be acquired, stored, and transmitted. The paper focuses on pointing out open questions, discussing options for addressing them, and evaluating them in prototypical implementations.
---
paper_title: Augmented reality in the psychomotor phase of a procedural task

paper_content:

Procedural tasks are common to many domains, ranging from maintenance and repair, to medicine, to the arts. We describe and evaluate a prototype augmented reality (AR) user interface designed to assist users in the relatively under-explored psychomotor phase of procedural tasks. In this phase, the user begins physical manipulations, and thus alters aspects of the underlying task environment. Our prototype tracks the user and multiple components in a typical maintenance assembly task, and provides dynamic, prescriptive, overlaid instructions on a see-through head-worn display in response to the user's ongoing activity. A user study shows participants were able to complete psychomotor aspects of the assembly task significantly faster and with significantly greater accuracy than when using 3D-graphics-based assistance presented on a stationary LCD. Qualitative questionnaire results indicate that participants overwhelmingly preferred the AR condition, and ranked it as more intuitive than the LCD condition.
---
paper_title: An augmented reality training platform for assembly and maintenance skills

paper_content:

Training technicians to acquire new maintenance and assembly skills is important for various industries. Because maintenance and assembly tasks can be very complex, training technicians to efficiently perform new skills is challenging. Training of this type can be supported by Augmented Reality, a powerful industrial training technology that directly links instructions on how to perform the service tasks to the machine parts that require processing. Because of the increasing complexity of maintenance tasks, it is not sufficient to train the technicians in task execution. Instead, technicians must be trained in the underlying skills-sensorimotor and cognitive-that are necessary for the efficient acquisition and performance of new maintenance operations. These facts illustrate the need for efficient training systems for maintenance and assembly skills that accelerate the technicians' acquisition of new maintenance procedures. Furthermore, these systems should improve the adjustment of the training process for new training scenarios and enable the reuse of worthwhile existing training material. In this context, we have developed a novel concept and platform for multimodal Augmented Reality-based training of maintenance and assembly skills, which includes sub-skill training and the evaluation of the training system. Because procedural skills are considered as the most important skills for maintenance and assembly operations, we focus on these skills and the appropriate methods for improving them.
---
paper_title: Augmented assembly using a mobile phone

paper_content:

We present a mobile phone based augmented reality (AR) assembly system that enable users to view complex models on their mobile phones. It is based on a client-server architecture, where complex model information is located on a PC, and a mobile phone with the camera is used as a thin client access device to this information. With this system users are able to see an AR view that provides step by step guidance for a real world assembly task. We also present results from a pilot user study evaluating the system, showing that people felt the interface was intuitive and very helpful in supporting the assembly task.
---
paper_title: Comparative effectiveness of augmented reality in object assembly

paper_content:

Although there has been much speculation about the potential of Augmented Reality (AR), there are very few empirical studies about its effectiveness. This paper describes an experiment that tested the relative effectiveness of AR instructions in an assembly task. Task information was displayed in user's field of view and registered with the workspace as 3D objects to explicitly demonstrate the exact execution of a procedure step. Three instructional media were compared with the AR system: a printed manual, computer assisted instruction (CAI) using a monitor-based display, and CAI utilizing a head-mounted display. Results indicate that overlaying 3D instructions on the actual work pieces reduced the error rate for an assembly task by 82%, particularly diminishing cumulative errors - errors due to previous assembly mistakes. Measurement of mental effort indicated decreased mental effort in the AR condition, suggesting some of the mental calculation of the assembly task is offloaded to the system.
---
paper_title: Mobile augmented reality in the data center

paper_content:

Recent advances in compute power, graphics power, and cameras in mobile computing devices have facilitated the development of new augmented reality applications in the mobile device space. Mobile augmented reality provides a straightforward and natural way for users to understand complex data by overlaying visualization on top of a live video feed on their mobile device. In our data center mobile augmented reality project, the user points a mobile device camera at a rack of data center assets, and additional content about these assets is visually overlaid on top of each asset in the video stream from the mobile device camera. This correspondence between digital content and physical things or locations makes mobile augmented reality an intuitive user interface for interacting with the world around us. This paper describes augmented reality techniques for mobile devices and the motivations around using a multimarker computer vision-based technique for visualizing asset data in the data center. Our mobile augmented reality project enables system administrators to easily interact with hardware assets while they are in the data center, providing them with an additional tool to use in managing the data center.
---
paper_title: Virtual Vouchers: Prototyping a Mobile Augmented Reality User Interface for Botanical Species Identification

paper_content:

the tools that botanists require for field-work must evolve and take on new forms. Of particular importance is the ability to identify existing and new species in the field. Mobile augmented reality systems can make it possible to access, view, and inspect a large database of virtual species examples side-by-side with physical specimens. In this paper, we present prototypes of a mobile augmented reality electronic field guide and techniques for displaying and inspecting computer vision-based visual search results in the form of virtual vouchers. Our work addresses head-movement controlled augmented reality for hands-free interaction and tangible augmented reality. We describe results from our design and investigation process and discuss observations and feedback from lab trials by botanists.
---
paper_title: An Augmented Reality System for MR Image – guided Needle Biopsy : Initial Results in a Swine Model 1

paper_content:

Purpose: To evaluate an augmented reality (AR) system in combination with a 1.5-T closed-bore magnetic resonance (MR) imager as a navigation tool for needle biopsies. Materials and Methods: The experimental protocol had institutional animal care and use committee approval. Seventy biopsies were performed in phantoms by using 20 tube targets, each with a diameter of 6 mm, and 50 virtual targets. The position of the needle tip in AR and MR space was compared in multiple imaging planes, and virtual and real needle tip localization errors were calculated. Ten AR-guided biopsies were performed in three pigs, and the duration of each procedure was determined. After successful puncture, the distance to the target was measured on MR images. The confidence limits for the achieved in-plane hit rate and for lateral deviation were calculated. A repeated measures analysis of variance was used to determine whether the placement error in a particular dimension (x, y, or z) differed from the others. Results: For the 50 v...
---
paper_title: When Augmented Reality meets Big Data

paper_content:

With computing and sensing woven into the fabric of everyday life, we live in an era where we are awash in a flood of data from which we can gain rich insights. Augmented reality (AR) is able to collect and help analyze the growing torrent of data about user engagement metrics within our personal mobile and wearable devices. This enables us to blend information from our senses and the digitalized world in a myriad of ways that was not possible before. AR and big data have a logical maturity that inevitably converge them. The tread of harnessing AR and big data to breed new interesting applications is starting to have a tangible presence. In this paper, we explore the potential to capture value from the marriage between AR and big data technologies, following with several challenges that must be addressed to fully realize this potential.
---
paper_title: User Interface Management Techniques for Collaborative Mobile Augmented Reality

paper_content:

Mobile Augmented Reality Systems (MARS) have the potential to revolutionize the way in which information is provided to users. Virtual information can be directly integrated with the real world surrounding the mobile user, who can interact with it to display related information, to pose and resolve queries, and to collaborate with other users. However, we believe that the benefits of MARS will only be achieved if the user interface (UI) is actively managed so as to maximize the relevance and minimize the confusion of the virtual material relative to the real world. This article addresses some of the steps involved in this process, focusing on the design and layout of the mobile user’s overlaid virtual environment. The augmented view of the user’s surroundings presents an interface to context-dependent operations, many of which are related to the objects in view—the augmented world is the user interface. We present three user interface design techniques that are intended to make this interface as obvious and clear to the user as possible: information filtering, UI component design, and view management. Information filtering helps select the most relevant information to present to the user. UI component designdetermines the format in which this information should be conveyed, based on the available display resources and tracking accuracy. For example, the absence of high accuracy position tracking would favor body- or screenstabilized components over world-stabilized ones that would need to be exactly registered with the physical objects to which they refer. View management attempts to ensure that the virtual objects that are displayed visually are arranged appropriately with regard to their projections on the view plane. For example, the relationships among objects should be as unambiguous as possible, and physical or virtual objects should not obstruct the user’s view of more important physical or virtual objects in the scene. We illustrate these interface design techniques using our prototype collaborative, cross-site MARS environment, which is composed of mobile and non-mobile augmented reality and virtual reality systems.
---
paper_title: Recent Advances in Augmented Reality

paper_content:

In 1997, Azuma published a survey on augmented reality (AR). Our goal is to complement, rather than replace, the original survey by presenting representative examples of the new advances. We refer one to the original survey for descriptions of potential applications (such as medical visualization, maintenance and repair of complex equipment, annotation, and path planning); summaries of AR system characteristics (such as the advantages and disadvantages of optical and video approaches to blending virtual and real, problems in display focus and contrast, and system portability); and an introduction to the crucial problem of registration, including sources of registration error and error-reduction strategies.
---
paper_title: BARS: Battlefield Augmented Reality System

paper_content:

Abstract : Situational awareness needs cannot be met using traditional approaches such as radios, maps and handheld displays and more powerful display paradigms are needed. We are researching mobile augmented reality (AR) through the development of the Battlefield Augmented Reality System (BARS) in collaboration with Columbia University. The system consists of a wearable computer a wireless network system and a tracked see-through Head Mounted Display (HMD). The user's perception of the environment is enhanced by superimposing graphics onto the user's field of view. The graphics are registered (aligned) with the actual environment. For example an augmented view of a building could include a wireframe plan of its interior icons to represent reported locations of snipers and the names of adjacent streets.
---
paper_title: Collaborative use of mobile augmented reality with paper maps

paper_content:

The popularity of augmented reality (AR) applications on mobile devices is increasing, but there is as yet little research on their use in real-settings. We review data from two pioneering field trials where MapLens, a magic lens that augments paper-based city maps, was used in small-group collaborative tasks. The first study compared MapLens to a digital version akin to Google Maps, the second looked at using one shared mobile device vs. using multiple devices. The studies find place-making and use of artefacts to communicate and establish common ground as predominant modes of interaction in AR-mediated collaboration with users working on tasks together despite not needing to.
---
paper_title: Like bees around the hive: a comparative study of a mobile augmented reality map

paper_content:

We present findings from field trials of MapLens, a mobile augmented reality (AR) map using a magic lens over a paper map. Twenty-six participants used MapLens to play a location-based game in a city centre. Comparisons to a group of 11 users with a standard 2D mobile map uncover phenomena that arise uniquely when interacting with AR features in the wild. The main finding is that AR features facilitate place-making by creating a constant need for referencing to the physical, and in that it allows for ease of bodily configurations for the group, encourages establishment of common ground, and thereby invites discussion, negotiation and public problem-solving. The main potential of AR maps lies in their use as a collaborative tool.
---
paper_title: Mobile phone based AR scene assembly

paper_content:

In this paper we describe a mobile phone based Augmented Reality application for 3D scene assembly. Augmented Reality on mobile phones extends the interaction capabilities on such handheld devices. It adds a 6 DOF isomorphic interaction technique for manipulating 3D content. We give details of an application that we believe to be the first where 3D content can be manipulated using both the movement of a camera tracked mobile phone and a traditional button interface as input for transformations. By centering the scene in a tangible marker space in front of the phone we provide a mean for bimanual interaction. We describe the implementation, the interaction techniques we have developed and initial user response to trying the application.
---
paper_title: InfoSPOT: A mobile Augmented Reality method for accessing building information through a situation awareness approach

paper_content:

article i nfo Article history: Accepted 9 September 2012 Available online 6 October 2012 The Architecture, Engineering, Construction, and Owner/Operator (AECO) industry is constantly searching for new methods for increasing efficiency and productivity. Facility Managers (FMs), as a part of the owner/ operator role, work in complex and dynamic environments where critical decisions are constantly made. This decision-making process and its consequent performance can be improved by enhancing Situation Awareness (SA) of the FMs through new digital technologies. In this paper, InfoSPOT (Information Surveyed Point for Observation and Tracking), is recommended to FMs as a mobile Augmented Reality (AR) tool for accessing information about the facilities they maintain. AR has been considered as a viable option to reduce inefficiencies of data overload by providing FMs with a SA-based tool for visualizing their "real-world" envi- ronment with added interactive data. A prototype of the AR application was developed and a user participa- tion experiment and analysis conducted to evaluate the features of InfoSPOT. This innovative application of AR has the potential to improve construction practices, and in this case, facility management. Published by Elsevier B.V.
---
paper_title: Gesture-based interaction via finger tracking for mobile augmented reality

paper_content:

The goal of this research is to explore new interaction metaphors for augmented reality on mobile phones, i.e. applications where users look at the live image of the device’s video camera and 3D virtual objects enrich the scene that they see. Common interaction concepts for such applications are often limited to pure 2D pointing and clicking on the device’s touch screen. Such an interaction with virtual objects is not only restrictive but also difficult, for example, due to the small form factor. In this article, we investigate the potential of finger tracking for gesture-based interaction. We present two experiments evaluating canonical operations such as translation, rotation, and scaling of virtual objects with respect to performance (time and accuracy) and engagement (subjective user feedback). Our results indicate a high entertainment value, but low accuracy if objects are manipulated in midair, suggesting great possibilities for leisure applications but limited usage for serious tasks.
---
paper_title: Media studies, mobile augmented reality, and interaction design

paper_content:

You are walking in the Sweetwater Creek State Park near Atlanta and using the Augmented Reality (AR) Trail Guide, a mobile application designed by Isaac Kulka for the Argon Browser (Figure 1). The application offers two views: a now familiar Google-style map, with points of interest marked on its surface, and an AR view, which shows these points located in space. You see the map view when you hold the screen parallel to the ground; when you turn the phone up to look at the world, you get the AR view with the points of interest floating in space in front of you. This simple gesture of raising the phone changes your relationship to the information. You pass from a fully symbolic form of representation to a form of perceiving symbolic information as part of your visual environment. The AR Trail Guide, developed in the Augmented Environments Lab at Georgia Tech [1], illustrates a new realm in AR design that goes beyond current commercial applications. In this article, we discuss some of these new areas, such as designing for experiences in cultural heritage, personal expression, and entertainment. At the same time, we want to address a larger issue. ACM interactions has often been a place for exploring new paradigms and the relevance for interaction design of unusual approaches from other disciplines. In that spirit, we pose the question: Can the humanistic discipline of media studies play a useful role in interaction design? Media studies looks at the history of media and their relationship to culture, and we will focus here on digital media and their relationship to other media, both present and past. Looking at digital media in a historical context is relevant because of the dynamic relationship between "traditional" media (film, television, radio, print) and their digital remediations. How can media studies be made to contribute to the productive work of interaction design? We believe one answer lies in using the historical understanding gained through media studies to develop a kind of media aesthetics that can guide designers as they explore new forms of digital media such as the mobile augmented reality application described above.
---
paper_title: Designing mobile augmented reality

paper_content:

The development of mobile Augmented Reality application became increasingly popular over the last few years. However, many of the existing solutions build on the reuse of available standard metaphors for visualization and interaction without considering the manifold contextual factors of their use. Within this workshop we want to discuss theoretical design approaches and practical tools which should help developers to make more informed choices when exploring the design space of Augmented Reality interfaces in mobile contexts.
---
paper_title: Augmenting Human Cognition with Adaptive Augmented Reality

paper_content:

Wearable Augmented Reality (AR) combines research in AR, mobile/ubiquitous computing, and human ergonomics in which a video or optical see-through head mounted display (HMD) facilitates multi-modal delivery of contextually relevant and computer generated visual and auditory data over a physical, real-world environment. Wearable AR has the capability of delivering on-demand assistance and training across a variety of domains. A primary challenge presented by such advanced HCI technologies is the development of scientifically-grounded methods for identifying appropriate information presentation, user input, and feedback modalities in order to optimize performance and mitigate cognitive overload. A proposed framework and research methodology are described to support instantiation of physiologically-driven, adaptive AR to assess and contextually adapt to an individual’s environmental and cognitive state in real time. Additionally a use case within the medical domain is presented, and future research is discussed.
---
paper_title: User Experience Evaluation of Mobile AR services

paper_content:

Mobile Augmented Reality (MAR) is state-of-the-art technology that has modernized the way of accessing and interacting with information thus invoking new experiences for users all around the world. However User Experience (UX) evaluation for mobile AR is still a widely unexplored area. In order to identify the potential problems faced by the end users while using MAR applications, it is important to evaluate the UX of current Mobile AR service. This paper narrates the findings and analysis from a pilot study. We report a cross-sectional survey for evaluating the user experience of latest mobile AR applications. A framework of qualities related to user experience is applied on the acquired results to understand and articulate different aspects of UX. The outcomes show a positive attitude of users towards mobile AR services. The results help in considering key acceptance issues and potential user expectations in the development of future mobile AR services.
---
paper_title: Preliminary user experience framework for designing mobile augmented reality technologies

paper_content:

User eXperience (UX) is identified as the characteristics of the designed system, the result of user's internal state, and the context within which the interaction between the system and user occurs. UX is becoming increasingly diverse and well established field especially in the context of its usage. Nevertheless, this field of research lack conceptual and practical frameworks to be followed while designing for emerging technologies like Augmented Reality (AR). This paper presents an early framework for designing and evaluating the UX of Mobile Augmented Reality (MAR) applications. The credibility of this work-in-progress UX frame-work is supported by recent validated research on UX and MAR studies.
---
paper_title: When Augmented Reality meets Big Data

paper_content:

With computing and sensing woven into the fabric of everyday life, we live in an era where we are awash in a flood of data from which we can gain rich insights. Augmented reality (AR) is able to collect and help analyze the growing torrent of data about user engagement metrics within our personal mobile and wearable devices. This enables us to blend information from our senses and the digitalized world in a myriad of ways that was not possible before. AR and big data have a logical maturity that inevitably converge them. The tread of harnessing AR and big data to breed new interesting applications is starting to have a tangible presence. In this paper, we explore the potential to capture value from the marriage between AR and big data technologies, following with several challenges that must be addressed to fully realize this potential.
---
paper_title: AR UX design: Applying AEIOU to handheld augmented reality browser

paper_content:

With maturing technologies and availability of the sensor-enriched device, the driving force behind handheld augmented reality (HAR) technology will be leaning towards the experience technology can bring. Though attentions are gathered on usability and conventions for this technology, the user experience cannot be ignored. It will be more commonly available in the hands of the public and become a technology that is not solely used by experts. It will be important to approach the technology with friendlier and more user-centered focus to bring it out of research labs and into people's life. In this paper, we will introduce design exploration constructs inspired from a method that is commonly used in the field of the industrial design to guide designers of HAR application to explore different aspects of user experience. The purpose of proposing AEIOU is to provide a platform to create encompassing user experience that is beyond the usability considerations or the existing design conventions. The advantage of having such a platform is to provide a starting ground for discussion and betterment of HAR application. The use of these constructs is discussed in relation to the design process of AR browser.
---
paper_title: Towards a shared large-area mixed reality system

paper_content:

In this paper we present a large-area interactive mixed reality system where multiple users can experience an event simultaneously. Through the combination of a number of innovative methods, the system can tackle common problems that are inherent in most existing mixed reality solutions, such as robustness against lighting conditions, static occlusion, illumination correction, registration and tracking etc. Most importantly, with our proposed experience server, a shared event among multiple users is seamless. The experience server tracks every user's position and experience state and presents a unique viewpoint of the event to multiple users simultaneously. The effectiveness of the system is demonstrated through an example application at a heritage site, where we perform user testing through multiple focus groups.
---
paper_title: Design Implications for Quality User eXperience in Mobile Augmented Reality Applications

paper_content:

Mobile augmented reality (MAR) technology is a new trend that provide users with the augmented view of digital information in real world. This paper contains design implications for improving the User eXperience (UX) of MAR applications. The scope is limited to MAR and its application in advertisement industry. Results from an experimental study are used to draw the guidelines that can be followed by the designers and practitioners to ensure a quality experience to their users.
---
paper_title: Multi-layered Mobile Augmented Reality Framework for Positive User Experience

paper_content:

Emerging technology like Mobile Augmented Reality (MAR) is becoming increasingly diverse and well established. However, this research field needs conceptual and practical frameworks to be followed while designing for a positive User Experience (UX). This paper presents a multi-layered conceptual MAR framework. It highlight the components needed to design MAR products and demonstrate different types of experiences invoked by the use of such products in a particular timespan. This work presents the important product aspects that need to be considered while designing for MAR to enhance its user experience.
---
paper_title: Demystifying the design of mobile augmented reality applications

paper_content:

This research proposes a set of interaction design principles for the development of mobile augmented reality (MAR) applications. The design recommendations adopt a user-centered perspective and, thus, they focus on the necessary actions to ensure high-quality MAR user experiences. To formulate our propositions we relied on theoretical grounding and an evaluation of eight MAR applications that provide published records of their design properties. The design principles have then been applied to guide the development of a MAR travel application. We performed a field study with 33 tourists in order to elicit whether our design choices effectively lead to enhanced satisfaction and overall user experience. Results suggest that the proposed principles contribute to ensuring high usability and performance of the MAR application as well as evoking positive feelings during user and system interactions. Our prescriptions may be employed either as a guide during the initial stages of the design process (ex-ante usage) or as a benchmark to assess the performance (ex-post usage) of MAR applications.
---
paper_title: Investigating the balance between virtuality and reality in mobile mixed reality UI design: user perception of an augmented city

paper_content:

Examples of mixed reality mobile applications and research combining virtual and real world data in the same view have emerged during recent years. However, currently there is little knowledge of users' perceptions comparing the role of virtual and real world representations in mobile user interfaces (UIs). In this paper, we investigate the initial user perceptions when comparing augmented reality and augmented virtuality UIs in a mobile application. To chart this, we conducted a field study with 35 participants, where they interacted with a simulated mobile mixed reality (MMR) application with two alternative UI designs, and an online survey completed by over a hundred people. Our findings reveal perceived differences e.g. in immersion, recognition, clarity and overall pleasantness, and provide insight to user interface design and methodological challenges of research in the area of mobile mixed reality.
---
paper_title: Integrating Linked Data in Mobile Augmented Reality Applications

paper_content:

Mobile devices are currently the most popular way of delivering ubiquitous augmented reality experiences. Traditionally, content sources for mobile augmented reality applications can be seen as isolated silos of information, being designed specifically for the intended purpose of the application. Recently, due to the raising in popularity and usage of the Semantic Web technologies and the Linked Data, some efforts have been made to overcome current augmented reality content sources limitations by integrating Linked Data principles and taking advantage of the significant increase in size and quality of the Linked Open Data cloud. This paper presents a literature review of the previous efforts in this respect, while highlighting in detail the limitations of current approaches, the advantages of integrating Linked Data principles in mobile augmented reality applications and up-to-date challenges in regarding this still novel approach. The authors conclude by suggesting some future research directions in this area.
---
paper_title: Designing backpacks for high fidelity mobile outdoor augmented reality

paper_content:

This paper presents the design for our latest backpack to support mobile outdoor augmented reality, and how it evolved from lessons learned with our previous designs. We present a number of novel features which help to reduce size and weight, improve reliability and ease of configuration, and reduce CPU usage on laptop computers.
---
paper_title: Structured visual markers for indoor pathfinding

paper_content:

We present a mobile augmented reality (AR) system to guide a user through an unfamiliar building to a destination room. The system presents a world-registered wireframe model of the building labeled with directional information in a see-through heads-up display, and a three-dimensional world-in-miniature (WIM) map on a wrist-worn pad that also acts as an input device. Tracking is done using a combination of wall-mounted ARToolkit markers observed by a head-mounted camera, and an inertial tracker. To allow coverage of arbitrarily large areas with a limited set of markers, a structured marker re-use scheme based on graph coloring has been developed.
---
paper_title: Human pacman: A mobile entertainment system with ubiquitous computing and tangible interaction over a wide outdoor area

paper_content:

Human Pacman is an interactive role-playing game that envisions to bring the computer gaming experience to a new level of emotional and sensory gratification by setting the real world as a playground. This is a physical fantasy game integrated with human-social and mobile-gaming that emphasizes on collaboration and competition between players. By setting the game in a wide outdoor area, natural human-physical movements have become an integral part of the game. Pacmen and Ghosts are now human players in the real world experiencing mixed reality visualization from the wearable computers on them. Virtual cookies and actual physical objects are incorporated to provide novel experiences of seamless transitions between real and virtual worlds and tangible human computer interface respectively. We believe Human Pacman is pioneering a new form of gaming that anchors on physicality, mobility, social interaction, and ubiquitous computing.
---
paper_title: TimeWarp: interactive time travel with a mobile Mixed Reality game

paper_content:

Mobile location-aware applications have become quite popular across a range of new areas such as pervasive games and mobile edutainment applications. However it is only recently, that approaches have been presented which combine gaming and education with mobile Augmented Reality systems. However they typically lack a close crossmedia integration of the surroundings, and often annotate or extend the environment rather than modifying and altering it. In this paper we present a mobile outdoor mixed reality game for exploring the history of a city in the spatial and the temporal dimension. We introduce the design and concept of the game and present a universal mechanism to define and setup multi-modal user interfaces for the game challenges. Finally we discuss the results of the user tests.
---
paper_title: Development of a real time image based object recognition method for mobile AR-devices

paper_content:

In this paper we describe an image based object recognition and tracking method for mobile AR-devices and the correlative process to generate the required data. The object recognition and tracking base on the 3D-geometries of the related objects. Correspondings between live camera images and 3D-models are generated and used to determine the location and orientation of objects in the current scene. The required data for the object recognition is generated from common 3D-CAD-files using a dedicated process model.The object recognition and tracking method as well as the correlative generation process for the needed data are developed within the AR-PDA project. The AR-PDA is a personal digital assistant (e.g. PDA or 3rd generation mobile phone with an integrated camera), which uses AR technology to efficiently support consumers and service forces during their daily tasks.
---
paper_title: ARToolKit on the PocketPC platform

paper_content:

In this paper, we describe the port of ARToolKit onto the PocketPC platform including optimizations that led to a three-fold speedup over the native cross-compiled version. The ported ARToolKit module was successfully used in the Handheld AR project.
---
paper_title: Implementation of an augmented reality system on a PDA

paper_content:

We present a client/server implementation for running demanding mobile AR application on a PDA device. The system incorporates various data compression methods to make it run as fast as possible on a wide range of communication networks, from GSM to WLAN.
---
paper_title: An Augmented Reality Presentation System for Remote Cultural Heritage Sites

paper_content:

Museums often lack the possibility to present archaeological or cultural heritage sites in a realistic and interesting way. Thus are proposing a new way to show augmented reality applications of cultural heritage sites at remote places like museums. In the exhibition space large wall-filling photographs of the real site are superimposed with interactive contextual annotations like 3D reconstructions, images and movies. Therefore we are using two different hardware setups for visualization: Standard UMPCs and a custom made revolving display. The setup has been installed and tested at SIGGRAPH 2008, Allard Pierson Museum in Amsterdam and CeBIT 2009. Museum visitors could experience Forum Romanum and Satricum in an informative and intuitive way by pointing the video see through devices on different areas of the photographs. The result is a more realistic and entertaining way for presenting cultural heritage sites in museums. Furthermore our solution is less expensive than comparable installations regarding content and hardware.
---
paper_title: Generic Interaction Techniques for Mobile Collaborative Mixed Systems

paper_content:

The main characteristic of a mobile collaborative mixed system is that augmentation of the physical environment of one user occurs through available knowledge of where the user is and what the other users are doing. Links between the physical and digital worlds are no longer static but dynamically defined by users to create a collaborative augmented environment. In this article we present generic interaction techniques for smoothly combining the physical and digital worlds of a mobile user in the context of a collaborative situation. We illustrate the generic nature of the techniques with two systems that we developed: MAGIC for archaeological fieldwork and TROC a mobile collaborative game.
---
paper_title: Wide Area Tracking Tools for Augmented Reality

paper_content:

We have developed a hand-held augmented reality platform exploiting a combination of multiple sensors built around an ultra-wideband tracking system. We demonstrate two applications illustrating how an environment exploiting this platform can be set up. Firstly, a technician-support application provides intuitive in-situ instructions on how a wide area tracking system should be configured. The use of 3D registered graphics greatly assists in the debugging of common awkward use cases involving reflections off metal surfaces. Secondly, a navigation application utilises this newly configured and calibrated tracker, as well as other sensors, adapting to whatever is available in a given locale.
---
paper_title: EyeGuardian: a framework of eye tracking and blink detection for mobile device users

paper_content:

Computer Vision Syndrome (CVS) is a common problem in the "Information Age", and it is becoming more serious as mobile devices (e.g. smartphones and tablet PCs) with small, low-resolution screens are outnumbering the home computers. The simplest way to avoid CVS is to blink frequently. However, most people do not realize that they blink less and some do not blink at all in front of the screen. In this paper, we present a mobile application that keeps track of the reader's blink rate and prods the user to blink if an exceptionally low blink rate is detected. The proposed eye detection and tracking algorithm is designed for mobile devices and can keep track of the eyes in spite of camera motion. The main idea is to predict the eye position in the camera frame using the feedback from the built-in accelerometer. The eye tracking system was built on a commercial Tablet PC. The experimental results consistently show that the scheme can withstand very aggressive mobility scenarios.
---
paper_title: Face to face collaborative AR on mobile phones

paper_content:

Mobile phones are an ideal platform for augmented reality. In this paper we describe how they also can be used to support face to face collaborative AR applications. We have created a custom port of the ARToolKit library to the Symbian mobile phone operating system and then developed a sample collaborative AR game based on this. We describe the game in detail and user feedback from people who have played it. We also provide general design guidelines that could be useful for others who are developing mobile phone collaborative AR applications.
---
paper_title: Sketching up the world: in situ authoring for mobile Augmented Reality

paper_content:

We present a novel system allowing in situ content creation for mobile Augmented Reality in unprepared environments. This system targets smartphones and therefore allows a spontaneous authoring while in place. We describe two different scenarios, which are depending on the size of the working environment and consequently use different tracking techniques. A natural feature-based approach for planar targets is used for small working spaces whereas for larger working environments, such as in outdoor scenarios, a panoramic-based orientation tracking is deployed. Both are integrated into one system allowing the user to use the same interaction for creating the content applying a set of simple, yet powerful modeling functions for content creation. The resulting content for Augmented Reality can be shared with other users using a dedicated content server or kept in a private inventory for later use.
---
paper_title: Location-based augmented reality on mobile phones

paper_content:

The computational capability of mobile phones has been rapidly increasing, to the point where augmented reality has become feasible on cell phones. We present an approach to indoor localization and pose estimation in order to support augmented reality applications on a mobile phone platform. Using the embedded camera, the application localizes the device in a familiar environment and determines its orientation. Once the 6 DOF pose is determined, 3D virtual objects from a database can be projected into the image and displayed for the mobile user. Off-line data acquisition consists of acquiring images at different locations in the environment. The online pose estimation is done by a feature-based matching between the cell phone image and an image selected from the precomputed database using the phone's sensors (accelerometer and magnetometer). The application enables the user both to visualize virtual objects in the camera image and to localize the user in a familiar environment. We describe in detail the process of building the database and the pose estimation algorithm used on the mobile phone. We evaluate the algorithm performance as well as its accuracy in terms of reprojection distance of the 3D virtual objects in the cell phone image.
---
paper_title: UMAR: Ubiquitous Mobile Augmented Reality

paper_content:

In this paper we discuss the prospects of using marker based Augmented Reality for context aware applications on mobile phones. We also present the UMAR, a conceptual framework for developing Ubiquitous Mobile Augmented Reality applications which consists of research areas identified as relevant for successfully bridging the physical world and the digital domain using Mobile Augmented Reality. A step towards this we have successfully ported the ARToolkit to consumer mobile phones running on the Symbian platform and present results around this. We also present three sample applications based on UMAR and future case study work planned.
---
paper_title: Collaborative use of mobile augmented reality with paper maps

paper_content:

The popularity of augmented reality (AR) applications on mobile devices is increasing, but there is as yet little research on their use in real-settings. We review data from two pioneering field trials where MapLens, a magic lens that augments paper-based city maps, was used in small-group collaborative tasks. The first study compared MapLens to a digital version akin to Google Maps, the second looked at using one shared mobile device vs. using multiple devices. The studies find place-making and use of artefacts to communicate and establish common ground as predominant modes of interaction in AR-mediated collaboration with users working on tasks together despite not needing to.
---
paper_title: Like bees around the hive: a comparative study of a mobile augmented reality map

paper_content:

We present findings from field trials of MapLens, a mobile augmented reality (AR) map using a magic lens over a paper map. Twenty-six participants used MapLens to play a location-based game in a city centre. Comparisons to a group of 11 users with a standard 2D mobile map uncover phenomena that arise uniquely when interacting with AR features in the wild. The main finding is that AR features facilitate place-making by creating a constant need for referencing to the physical, and in that it allows for ease of bodily configurations for the group, encourages establishment of common ground, and thereby invites discussion, negotiation and public problem-solving. The main potential of AR maps lies in their use as a collaborative tool.
---
paper_title: Video see-through AR on consumer cell-phones

paper_content:

We present a first running video see-through augmented reality system on a consumer cell-phone. It supports the detection and differentiation of different markers, and correct integration of rendered 3D graphics into the live video stream via a weak perspective projection camera model and an OpenGL rendering pipeline.
---
paper_title: Mobile phone based AR scene assembly

paper_content:

In this paper we describe a mobile phone based Augmented Reality application for 3D scene assembly. Augmented Reality on mobile phones extends the interaction capabilities on such handheld devices. It adds a 6 DOF isomorphic interaction technique for manipulating 3D content. We give details of an application that we believe to be the first where 3D content can be manipulated using both the movement of a camera tracked mobile phone and a traditional button interface as input for transformations. By centering the scene in a tangible marker space in front of the phone we provide a mean for bimanual interaction. We describe the implementation, the interaction techniques we have developed and initial user response to trying the application.
---
paper_title: An Augmented Reality Presentation System for Remote Cultural Heritage Sites

paper_content:

Museums often lack the possibility to present archaeological or cultural heritage sites in a realistic and interesting way. Thus are proposing a new way to show augmented reality applications of cultural heritage sites at remote places like museums. In the exhibition space large wall-filling photographs of the real site are superimposed with interactive contextual annotations like 3D reconstructions, images and movies. Therefore we are using two different hardware setups for visualization: Standard UMPCs and a custom made revolving display. The setup has been installed and tested at SIGGRAPH 2008, Allard Pierson Museum in Amsterdam and CeBIT 2009. Museum visitors could experience Forum Romanum and Satricum in an informative and intuitive way by pointing the video see through devices on different areas of the photographs. The result is a more realistic and entertaining way for presenting cultural heritage sites in museums. Furthermore our solution is less expensive than comparable installations regarding content and hardware.
---
paper_title: Telegeoinformatics: Location-based Computing and Services

paper_content:

THEORIES AND TECHNOLOGIES Telegeoinformatics: Current Trends and Future Direction Introduction Architecture Internet-Based GIS Spatial Databases Intelligent Query Analyzer (IQA) Predictive Computing Adaptation Final Remarks References Remote Sensing Introductory Concepts Remote Sensing Systems Imaging Characteristics of Remote Sensing Systems Active Microwave Remote Sensing Extraction of Thematic Information from Remotely Sensed Imagery Extraction of Metric Information from Remotely Sensed Imagery Remote Sensing in Telegeoinformatics References Positioning and Tracking Approaches and Technologies Introduction Global Positioning System Positioning Methods Based on Cellular Networks Other Positioning and Tracking Techniques: An Overview Hybrid Systems Summary References Wireless Communications Introduction Overview of Wireless Systems Radio Propagation and Physical Layer Issues Medium Access in Wireless Networks Network Planning, Design and Deployment Wireless Network Operations Conclusions and the Future References INTEGRATED DATA AND TECHNOLOGIES Chapter Five: Location-Based Computing Introduction LBC Infrastructure Location-Based Interoperability Location-Based Data Management Adaptive Location-Based Computing Location-Based Routing as Adaptive LBC Concluding Remarks References Location-Based Services Introduction Types of Location-Based Services What is Unique About Location-Based Services? Enabling Technologies Market for Location-Based Services Importance of Architecture and Standards Example Location-Based Services: J-Phone J-Navi (Japan) Conclusions References Wearable Tele-Informatic Systems for Personal Imaging Introduction Humanistic Intelligence as a Basis for Intelligent Image Processing Humanistic Intelligence 'WEARCOMP' as a Means of Realizing Humanistic Intelligence Where on the Body Should a Visual Tele-Informatic Device be Placed? Telepointer: Wearable Hands-Free Completely Self Contained Visual Augmented Reality Without Headwear and Without any Infrastructural Reliance Portable Personal Pulse Doppler Radar Vision System When Both the Camera and Display are Headword: Personal Imaging and Mediated Reality Personal Imaging for Location-Based Services Reality Window Manager (RWM) Personal Telegeoinformatics: Blocking Spam with a Photonic Filter Conclusion References Mobile Augmented Reality Introduction MARS: Promises, Applications, and Challenges Components and Requirements MARS UI Concepts Conclusions Acknowledgements References APPLICATIONS Emergency Response Systems Overview of Emergency Response Systems State-of-the-Art ERSs Examples of Developing ERSs for Earthquakes and Other Disasters Future Aspects of Emergency Response Systems Concluding Remarks References Location-Based Computing for Infrastructure Field Tasks Introduction LBC-Infra Concept Technological Components of LBC-Infra General Requirements of LBC-Infra Interaction Patterns and Framework of LBC-Infra Prototype System and Case Study Conclusions References The Role of Telegeoinformatics in ITS Introduction to Intelligent Tranaportation Systems Telegeoinformatics Within ITS The Role of Positioning Systems In ITS Geospatial Data for ITS Communication Systems in ITS ITS-Telegeoinformatics Applications Non-Technical Issues Impacting on ITS Concluding Remarks Remarks The Impact and Penetration of Location-Based Services The Definition of Technologies LBSs: Definitions, Software, and Usage The Market for LBSs: A Model of the Development of LBSs Penetration of Mobile Devices: Predictions of Future Markets Impacts of LBSs on Geographical Locations Conclusions References
---
paper_title: Location based Applications for Mobile Augmented Reality

paper_content:

In this work we investigate building indoor location based applications for a mobile augmented reality system. We believe that augmented reality is a natural interface to visualize spacial information such as position or direction of locations and objects for location based applications that process and present information based on the user's position in the real world. To enable such applications we construct an indoor tracking system that covers a substantial part of a building. It is based on visual tracking of fiducial markers enhanced with an inertial sensor for fast rotational updates. To scale such a system to a whole building we introduce a space partitioning scheme to reuse fiducial markers throughout the environment. Finally we demonstrate two location based applications built upon this facility, an indoor navigation aid and a library search application.
---
paper_title: Sensor fusion and occlusion refinement for tablet-based AR

paper_content:

This paper presents a set of technologies which enable robust, accurate, high resolution augmentation of live video, delivered via a tablet PC to which a video camera has been attached. By combining several technologies, this is achieved without the use of contrived markers in the environment: An outside-in tracker observes the tablet to generate robust, low-accuracy pose estimates. An inside-out tracker running on the tablet observes the video feed from the tablet-mounted camera and provides high-accuracy pose estimates by tracking natural features in the environment. Information from both of these trackers is combined in an extended Kalman filter. Finally, to maximise the quality of the augmented imagery, boundaries where the real world occludes the virtual imagery are identified and another tracker is used to refine the boundaries between real and virtual imagery so that their synthesis is as convincing as possible.
---
paper_title: ARToolKit on the PocketPC platform

paper_content:

In this paper, we describe the port of ARToolKit onto the PocketPC platform including optimizations that led to a three-fold speedup over the native cross-compiled version. The ported ARToolKit module was successfully used in the Handheld AR project.
---
paper_title: Video see-through AR on consumer cell-phones

paper_content:

We present a first running video see-through augmented reality system on a consumer cell-phone. It supports the detection and differentiation of different markers, and correct integration of rendered 3D graphics into the live video stream via a weak perspective projection camera model and an OpenGL rendering pipeline.
---
paper_title: Mobile collaborative augmented reality

paper_content:

The combination of mobile computing and collaborative augmented reality into a single system makes the power of computer enhanced interaction and communication in the real world accessible anytime and everywhere. The paper describes our work to build a mobile collaborative augmented reality system that supports true stereoscopic 3D graphics, a pen and pad interface and direct interaction with virtual objects. The system is assembled from off-the-shelf hardware components and serves as a basic test bed for user interface experiments related to computer supported collaborative work in augmented reality. A mobile platform implementing the described features and collaboration between mobile and stationary users are demonstrated.
---
paper_title: “Studierstube”: An environment for collaboration in augmented reality

paper_content:

We propose an architecture for multi-user augmented reality with applications in visualisation, presentation and education, which we call "Studierstube". Our system presents three-dimensional stereoscopic graphics simultaneously to a group of users wearing light weight see-through head mounted displays. The displays do not affect natural communication and interaction, making working together very effective. Users see the same spatially aligned model, but can independently control their viewpoint and different layers of the data to be displayed. The setup serves computer supported cooperative work and enhances cooperation of visualisation experts. This paper presents the client-server software architecture underlying this system and details that must be addressed to create a high-quality augmented reality setup.
---
paper_title: Sketching up the world: in situ authoring for mobile Augmented Reality

paper_content:

We present a novel system allowing in situ content creation for mobile Augmented Reality in unprepared environments. This system targets smartphones and therefore allows a spontaneous authoring while in place. We describe two different scenarios, which are depending on the size of the working environment and consequently use different tracking techniques. A natural feature-based approach for planar targets is used for small working spaces whereas for larger working environments, such as in outdoor scenarios, a panoramic-based orientation tracking is deployed. Both are integrated into one system allowing the user to use the same interaction for creating the content applying a set of simple, yet powerful modeling functions for content creation. The resulting content for Augmented Reality can be shared with other users using a dedicated content server or kept in a private inventory for later use.
---
paper_title: Ubiquitous animated agents for augmented reality

paper_content:

Most of today's Augmented Reality (AR) systems operate as passive information browsers relying on a finite and deterministic world model and a predefined hardware and software infrastructure. We propose an AR framework that dynamically and proactively exploits hitherto unknown applications and hardware devices, and adapts the appearance of the user interface to persistently stored and accumulated user preferences. Our framework explores proactive computing, multi-user interface adaptation, and user interface migration. We employ mobile and autonomous agents embodied by real and virtual objects as an interface and interaction metaphor, where agent bodies are able to opportunistically migrate between multiple AR applications and computing platforms to best match the needs of the current application context. We present two pilot applications to illustrate design concepts.
---
paper_title: An open software architecture for virtual reality interaction

paper_content:

This article describes OpenTracker, an open software architecture that provides a framework for the different tasks involved in tracking input devices and processing multi-modal input data in virtual environments and augmented reality application. The OpenTracker framework eases the development and maintenance of hardware setups in a more flexible manner than what is typically offered by virtual reality development packages. This goal is achieved by using an object-oriented design based on XML, taking full advantage of this new technology by allowing to use standard XML tools for development, configuration and documentation. The OpenTracker engine is based on a data flow concept for multi-modal events. A multi-threaded execution model takes care of tunable performance. Transparent network access allows easy development of decoupled simulation models. Finally, the application developer's interface features both a time-based and an event based model, that can be used simultaneously, to serve a large range of applications. OpenTracker is a first attempt towards a "'write once, input anywhere"' approach to virtual reality application development. To support these claims, integration into an existing augmented reality system is demonstrated. We also show how a prototype tracking equipment for mobile augmented reality can be assembled from consumer input devices with the aid of OpenTracker. Once development is sufficiently mature, it is planned to make Open-Tracker available to the public under an open source software license.
---
paper_title: APRIL: a high-level framework for creating augmented reality presentations

paper_content:

While augmented reality (AR) technology is steadily maturing, application development is still lacking advanced authoring tools - even the simple presentation of information, which should not require any programming, is not systematically addressed by development tools. Moreover, there is also a severe lack of agreed techniques or best practices for the structuring of AR content. In this paper we present APRIL, the Augmented Presentation and Interaction Language, an authoring platform for AR presentations which provides concepts and techniques that are independent of specific applications or target hardware platforms, and should be suitable to raise the level of abstraction on which AR content creators can operate.
---
paper_title: Muddleware for Prototyping Mixed Reality Multiuser Games

paper_content:

We present Muddleware, a communication platform designed for mixed reality multi-user games for mobile, lightweight clients. An approach inspired by Tuplespaces, which provides decoupling of sender and receiver is used to address the requirements of a potentially large number of mobile clients. A hierarchical database built on XML technology allows convenient prototyping and simple, yet powerful queries. Server side-extensions address persistence and autonomous behaviors through hierarchical state machines. The architecture has been tested with a number of multi-user games and is also used for non-entertainment applications
---
paper_title: OpenTracker-an open software architecture for reconfigurable tracking based on XML

paper_content:

This paper describes OpenTracker, an open software architecture that provides a generic solution to the different tasks involved in tracking input devices and processing tracking data for virtual environments. It combines a highly modular design with a configuration syntax based on XML, thus taking full advantage of this new technology. OpenTracker is a first attempt towards a "write once, track anywhere" approach to virtual reality application development.
---
paper_title: Nexus - an open global infrastructure for spatial-aware applications

paper_content:

Due to the lack of a generic platform for location- and spatial-aware systems, many basic services have to be reimplemented in each application that uses spatial-awareness. A cooperation among different applications is also difficult to achieve without a common platform. In this paper we present a platform that solves these problems. It provides an infrastructure that is based on computer models of regions of the physical world, which are augmented by virtual objects. We show how virtual objects make the integration of existing information systems and services in spatial-aware systems easier. Furthermore, our platform supports interactions between the computer models and the real world and integrates single models in a global 'Augmented World'.
---
paper_title: Marker tracking and HMD calibration for a video-based augmented reality conferencing system

paper_content:

We describe an augmented reality conferencing system which uses the overlay of virtual images on the real world. Remote collaborators are represented on virtual monitors which can be freely positioned about a user in space. Users can collaboratively view and interact with virtual objects using a shared virtual whiteboard. This is possible through precise virtual image registration using fast and accurate computer vision techniques and head mounted display (HMD) calibration. We propose a method for tracking fiducial markers and a calibration method for optical see-through HMD based on the marker tracking.
---
paper_title: UMAR: Ubiquitous Mobile Augmented Reality

paper_content:

In this paper we discuss the prospects of using marker based Augmented Reality for context aware applications on mobile phones. We also present the UMAR, a conceptual framework for developing Ubiquitous Mobile Augmented Reality applications which consists of research areas identified as relevant for successfully bridging the physical world and the digital domain using Mobile Augmented Reality. A step towards this we have successfully ported the ARToolkit to consumer mobile phones running on the Symbian platform and present results around this. We also present three sample applications based on UMAR and future case study work planned.
---
paper_title: Tinmith-Metro: new outdoor techniques for creating city models with an augmented reality wearable computer

paper_content:

This paper presents new techniques for capturing and viewing on site 3D graphical models for large outdoor objects. Using an augmented reality wearable computer, we have developed a software system, known as Tinmith-Metro. Tinmith-Metro allows users to control a 3D constructive solid geometry modeller for building graphical objects of large physical artefacts, for example buildings, in the physical world. The 3D modeller is driven by a new user interface known as Tinmith-Hand, which allows the user to control the modeller using a set of pinch gloves and hand tracking. These techniques allow user to supply their AR renderers with models that would previously have to be captured with manual, time-consuming, and/or expensive methods.
---
paper_title: InfoSPOT: A mobile Augmented Reality method for accessing building information through a situation awareness approach

paper_content:

article i nfo Article history: Accepted 9 September 2012 Available online 6 October 2012 The Architecture, Engineering, Construction, and Owner/Operator (AECO) industry is constantly searching for new methods for increasing efficiency and productivity. Facility Managers (FMs), as a part of the owner/ operator role, work in complex and dynamic environments where critical decisions are constantly made. This decision-making process and its consequent performance can be improved by enhancing Situation Awareness (SA) of the FMs through new digital technologies. In this paper, InfoSPOT (Information Surveyed Point for Observation and Tracking), is recommended to FMs as a mobile Augmented Reality (AR) tool for accessing information about the facilities they maintain. AR has been considered as a viable option to reduce inefficiencies of data overload by providing FMs with a SA-based tool for visualizing their "real-world" envi- ronment with added interactive data. A prototype of the AR application was developed and a user participa- tion experiment and analysis conducted to evaluate the features of InfoSPOT. This innovative application of AR has the potential to improve construction practices, and in this case, facility management. Published by Elsevier B.V.
---
paper_title: KHARMA: An open KML/HTML architecture for mobile augmented reality applications

paper_content:

Widespread future adoption of augmented reality technology will rely on a broadly accessible standard for authoring and distributing content with, at a minimum, the flexibility and interactivity provided by current web authoring technologies. We introduce KHARMA, an open architecture based on KML for geospatial and relative referencing combined with HTML, JavaScript and CSS technologies for content development and delivery. This architecture uses lightweight representations that decouple infrastructure and tracking sources from authoring and content delivery. Our main contribution is a re-conceptualization of KML that turns HTML content formerly confined to balloons into first-class elements in the scene. We introduce the KARML extension that gives authors increase control over the presentation of HTML content and its spatial relationship to other content.
---
paper_title: Developing a Mobile, Service-Based Augmented Reality Tool for Modern Maintenance Work

paper_content:

In the VTT PLAMOS (Plant Model Services for Mobile Process Maintenance Engineer) project new tools were developed for modern maintenance work carried out in industrial plants by either the plant personnel or personnel of an industrial service provider. To formulate the requirements for new tools the work of a maintenance man was studied with a particular method, the Core-Task Analysis which has its roots in study and development of work in complex settings. The aim was to develop and create concepts for novel tools that would support the development of good work practices in a situation where the work is concurrently undergoing several transformations. Hence, the new tools should have potential to enable and affect new ways of working.
---
paper_title: Augmented reality for plant lifecycle management

paper_content:

Augmented reality is a technology expected to have a big impact as a natural and intuitive user interface to industrial applications. A lot of research has been done in industrial augmented reality but real life applications are still rare. One reason for the slow adoption seems to be that augmented reality applications have been developed without taking sufficiently into account the existing industrial processes in information management. This paper describes a prototype software which uses augmented reality as a user interface to plant lifecycle management applications in different lifecycle phases. Several use cases implemented show that augmented reality can be implemented cost efficiently without specific authoring tools if the design information is correctly structured already in the design phase, and if the design, maintenance and real-time information sources are accessed via standard application interfaces.
---
paper_title: Implementation of an augmented reality system on a PDA

paper_content:

We present a client/server implementation for running demanding mobile AR application on a PDA device. The system incorporates various data compression methods to make it run as fast as possible on a wide range of communication networks, from GSM to WLAN.
---
paper_title: CloudRidAR: a cloud-based architecture for mobile augmented reality

paper_content:

Mobile augmented reality (MAR) has exploded in popularity on mobile devices in various fields. However, building a MAR application from scratch on mobile devices is complicated and time-consuming. In this paper, we propose CloudRidAR, a framework for MAR developers to facilitate the development, deployment, and maintenance of MAR applications with little effort. Despite of advance in mobile devices as a computing platform, their performance for MAR applications is still very limited due to the poor computing capability of mobile devices. In order to alleviate the problem, our CloudRidAR is designed with cloud computing at the core. Computational intensive tasks are offloaded on cloud to accelerate computation in order to guarantee run-time performance. We also present two MAR applications built on CloudRidAR to evaluate our design.
---
paper_title: ARTiFICe-Augmented Reality Framework for Distributed Collaboration

paper_content:

This paper introduces a flexible and powerful software framework based on an off the shelf game engine which is used to develop distributed and collaborative virtual and augmented reality applications. We describe ARTiFICe's flexible design and implementation and demonstrate its use in research and teaching where 97 students in two lab courses developed AR applications with it. Applications are presented on mobile, desktop and immersive systems using low cost 6-DOF input devices (Microsoft Kinect, Razer Hydra, SpaceNavigator), that we integrated into our framework.
---
paper_title: Mobile Collaborative Augmented Reality: the Augmented Stroll

paper_content:

The paper focuses on Augmented Reality systems in which interaction with the real world is augmented by the computer, the task being performed in the real world. We first define what mobile AR systems, collaborative AR systems and finally mobile and collaborative AR systems are. We then present the augmented stroll and its software design as one example of a mobile and collaborative AR system. The augmented stroll is applied to Archaeology in the MAGIC (Mobile Augmented Group Interaction in Context) project.
---
paper_title: Data Management Strategies for Mobile Augmented Reality

paper_content:

Any significant real-world application of mobile augmented reality will require a large model of location-bound data. While it may appear that a natural approach is to develop application-specific data formats and management strategies, we have found that such an approach actually prevents reuse of the data and ultimately produces additional complexity in developing the application. In contrast we describe a three-tier architecture to manage a common data model for a set of applications. It is inspired by current Internet application frameworks and consists of a central storage layer using a common data model, a transformation layer responsible for filtering and adapting the data to the requirements of a particular applications on request, and finally of the applications itself. We demonstrate our architecture in a scenario consisting of two multi-user capable mobile AR applications for collaborative navigation and annotation in a city environment.
---
paper_title: Human pacman: A mobile entertainment system with ubiquitous computing and tangible interaction over a wide outdoor area

paper_content:

Human Pacman is an interactive role-playing game that envisions to bring the computer gaming experience to a new level of emotional and sensory gratification by setting the real world as a playground. This is a physical fantasy game integrated with human-social and mobile-gaming that emphasizes on collaboration and competition between players. By setting the game in a wide outdoor area, natural human-physical movements have become an integral part of the game. Pacmen and Ghosts are now human players in the real world experiencing mixed reality visualization from the wearable computers on them. Virtual cookies and actual physical objects are incorporated to provide novel experiences of seamless transitions between real and virtual worlds and tangible human computer interface respectively. We believe Human Pacman is pioneering a new form of gaming that anchors on physicality, mobility, social interaction, and ubiquitous computing.
---
paper_title: Video see-through AR on consumer cell-phones

paper_content:

We present a first running video see-through augmented reality system on a consumer cell-phone. It supports the detection and differentiation of different markers, and correct integration of rendered 3D graphics into the live video stream via a weak perspective projection camera model and an OpenGL rendering pipeline.
---
paper_title: Map torchlight: a mobile augmented reality camera projector unit

paper_content:

The advantages of paper-based maps have been utilized in the field of mobile Augmented Reality (AR) in the last few years. Traditional paper-based maps provide high-resolution, large-scale information with zero power consumption. There are numerous implementations of magic lens interfaces that combine high-resolution paper maps with dynamic handheld displays. From an HCI perspective, the main challenge of magic lens interfaces is that users have to switch their attention between the magic lens and the information in the background. In this paper, we attempt to overcome this problem by using a lightweight mobile camera projector unit to augment the paper map directly with additional information. The "Map Torchlight" is tracked over a paper map and can precisely highlight points of interest, streets, and areas to give directions or other guidance for interacting with the map.
---
paper_title: Mobile augmented reality in the data center

paper_content:

Recent advances in compute power, graphics power, and cameras in mobile computing devices have facilitated the development of new augmented reality applications in the mobile device space. Mobile augmented reality provides a straightforward and natural way for users to understand complex data by overlaying visualization on top of a live video feed on their mobile device. In our data center mobile augmented reality project, the user points a mobile device camera at a rack of data center assets, and additional content about these assets is visually overlaid on top of each asset in the video stream from the mobile device camera. This correspondence between digital content and physical things or locations makes mobile augmented reality an intuitive user interface for interacting with the world around us. This paper describes augmented reality techniques for mobile devices and the motivations around using a multimarker computer vision-based technique for visualizing asset data in the data center. Our mobile augmented reality project enables system administrators to easily interact with hardware assets while they are in the data center, providing them with an additional tool to use in managing the data center.
---
paper_title: Comparison of optical and video see-through, head-mounted displays

paper_content:

One of the most promising and challenging future uses of head-mounted displays (HMDs) is in applications where virtual environments enhance rather than replace real environments. To obtain an enhanced view of the real environment, the user wears a see-through HMD to see 3D computer-generated objects superimposed on his/her real-world view. This see-through capability can be accomplished using either an optical or a video see-through HMD. We discuss the tradeoffs between optical and video see-through HMDs with respect to technological, perceptual, and human factors issues, and discuss our experience designing, building, using, and testing these HMDs.© (1995) COPYRIGHT SPIE--The International Society for Optical Engineering. Downloading of the abstract is permitted for personal use only.
---
paper_title: GroundCam: A Tracking Modality for Mobile Mixed Reality

paper_content:

Anywhere augmentation pursues the goal of lowering the initial investment of time and money necessary to participate in mixed reality work, bridging the gap between researchers in the field and regular computer users. Our paper contributes to this goal by introducing the GroundCam, a cheap tracking modality with no significant setup necessary. By itself, the GroundCam provides high frequency, high resolution relative position information similar to an inertial navigation system, but with significantly less drift. When coupled with a wide area tracking modality via a complementary Kalman filter, the hybrid tracker becomes a powerful base for indoor and outdoor mobile mixed reality work
---
paper_title: BRIDGING THE GAPS : HYBRID TRACKING FOR ADAPTIVE MOBILE AUGMENTED REALITY

paper_content:

Tracking accuracy in a location-aware mobile system can change dynamically as a function of the user's location and other variables specific to the tracking technologies used. This is especially problematic for mobile augmented reality systems, which ideally require extremely precise position tracking for the user's head, but which may not always be able to achieve that level of accuracy. While it is possible to ignore variable positional accuracy in an augmented reality user interface, this can make for a confusing system; for example, when accuracy is low, virtual objects that are nominally registered with real ones may be too far off to be of use. To address this problem, we describe an experimental mobile augmented reality system that: (1) employs multiple position-tracking technologies, including ones that apply heuristics based on environmental knowledge; (2) coordinates these concurrently monitored tracking systems; and (3) automatically adapts the user interface to varying degrees of confidence in...
---
paper_title: Virtual Vouchers: Prototyping a Mobile Augmented Reality User Interface for Botanical Species Identification

paper_content:

the tools that botanists require for field-work must evolve and take on new forms. Of particular importance is the ability to identify existing and new species in the field. Mobile augmented reality systems can make it possible to access, view, and inspect a large database of virtual species examples side-by-side with physical specimens. In this paper, we present prototypes of a mobile augmented reality electronic field guide and techniques for displaying and inspecting computer vision-based visual search results in the form of virtual vouchers. Our work addresses head-movement controlled augmented reality for hands-free interaction and tangible augmented reality. We describe results from our design and investigation process and discuss observations and feedback from lab trials by botanists.
---
paper_title: User Interface Management Techniques for Collaborative Mobile Augmented Reality

paper_content:

Mobile Augmented Reality Systems (MARS) have the potential to revolutionize the way in which information is provided to users. Virtual information can be directly integrated with the real world surrounding the mobile user, who can interact with it to display related information, to pose and resolve queries, and to collaborate with other users. However, we believe that the benefits of MARS will only be achieved if the user interface (UI) is actively managed so as to maximize the relevance and minimize the confusion of the virtual material relative to the real world. This article addresses some of the steps involved in this process, focusing on the design and layout of the mobile user’s overlaid virtual environment. The augmented view of the user’s surroundings presents an interface to context-dependent operations, many of which are related to the objects in view—the augmented world is the user interface. We present three user interface design techniques that are intended to make this interface as obvious and clear to the user as possible: information filtering, UI component design, and view management. Information filtering helps select the most relevant information to present to the user. UI component designdetermines the format in which this information should be conveyed, based on the available display resources and tracking accuracy. For example, the absence of high accuracy position tracking would favor body- or screenstabilized components over world-stabilized ones that would need to be exactly registered with the physical objects to which they refer. View management attempts to ensure that the virtual objects that are displayed visually are arranged appropriately with regard to their projections on the view plane. For example, the relationships among objects should be as unambiguous as possible, and physical or virtual objects should not obstruct the user’s view of more important physical or virtual objects in the scene. We illustrate these interface design techniques using our prototype collaborative, cross-site MARS environment, which is composed of mobile and non-mobile augmented reality and virtual reality systems.
---
paper_title: Mobile Collaborative Augmented Reality: the Augmented Stroll

paper_content:

The paper focuses on Augmented Reality systems in which interaction with the real world is augmented by the computer, the task being performed in the real world. We first define what mobile AR systems, collaborative AR systems and finally mobile and collaborative AR systems are. We then present the augmented stroll and its software design as one example of a mobile and collaborative AR system. The augmented stroll is applied to Archaeology in the MAGIC (Mobile Augmented Group Interaction in Context) project.
---
paper_title: Bringing user-generated content from Internet services to mobile augmented reality clients

paper_content:

In this paper we describe a system for bringing usergenerated content to mobile augmented reality clients, taking in consideration the metadata required for visualizing it, at a sensor based tracking solution. Our proposal assumes that content is stored in multiple external Internet services, simply treated as a “cloud”, thus making the mobile client service agnostic. A prototype implementation was created for the Image Space service and the learnings of integrating with the popular Flickr service are discussed.
---
paper_title: Indoor location sensing using geo-magnetism

paper_content:

We present an indoor positioning system that measures location using disturbances of the Earth's magnetic field caused by structural steel elements in a building. The presence of these large steel members warps the geomagnetic field in a way that is spatially varying but temporally stable. To localize, we measure the magnetic field using an array of e-compasses and compare the measurement with a previously obtained magnetic map. We demonstrate accuracy within 1 meter 88% of the time in experiments in two buildings and across multiple floors within the buildings. We discuss several constraint techniques that can maintain accuracy as the sample space increases.
---
paper_title: Polaris: getting accurate indoor orientations for mobile devices using ubiquitous visual patterns on ceilings

paper_content:

Ubiquitous computing applications commonly use digital compass sensors to obtain orientation of a device relative to the magnetic north of the earth. However, these compass readings are always prone to significant errors in indoor environments due to presence of metallic objects in close proximity. Such errors can adversely affect the performance and quality of user experience of the applications utilizing digital compass sensors. In this paper, we propose Polaris, a novel approach to provide reliable orientation information for mobile devices in indoor environments. Polaris achieves this by aggregating pictures of the ceiling of an indoor environment and applies computer vision based pattern matching techniques to utilize them as orientation references for correcting digital compass readings. To show the feasibility of the Polaris system, we implemented the Polaris system on mobile devices, and field tested the system in multiple office buildings. Our results show that Polaris achieves 4.5° average orientation accuracy, which is about 3.5 times better than what can be achieved through sole use of raw digital compass readings.
---
paper_title: Aided eyes: eye activity sensing for daily life

paper_content:

Our eyes collect a considerable amount of information when we use them to look at objects. In particular, eye movement allows us to gaze at an object and shows our level of interest in the object. In this research, we propose a method that involves real-time measurement of eye movement for human memory enhancement; the method employs gaze-indexed images captured using a video camera that is attached to the user's glasses. We present a prototype system with an infrared-based corneal limbus tracking method. Although the existing eye tracker systems track eye movement with high accuracy, they are not suitable for daily use because the mobility of these systems is incompatible with a high sampling rate. Our prototype has small phototransistors, infrared LEDs, and a video camera, which make it possible to attach the entire system to the glasses. Additionally, the accuracy of this method is compensated by combining image processing methods and contextual information, such as eye direction, for information extraction. We develop an information extraction system with real-time object recognition in the user's visual attention area by using the prototype of an eye tracker and a head-mounted camera. We apply this system to (1) fast object recognition by using a SURF descriptor that is limited to the gaze area and (2) descriptor matching of a past-images database. Face recognition by using haar-like object features and text logging by using OCR technology is also implemented. The combination of a low-resolution camera and a high-resolution, wide-angle camera is studied for high daily usability. The possibility of gaze-guided computer vision is discussed in this paper, as is the topic of communication by the photo transistor in the eye tracker and the development of a sensor system that has a high transparency.
---
paper_title: Data Management Strategies for Mobile Augmented Reality

paper_content:

Any significant real-world application of mobile augmented reality will require a large model of location-bound data. While it may appear that a natural approach is to develop application-specific data formats and management strategies, we have found that such an approach actually prevents reuse of the data and ultimately produces additional complexity in developing the application. In contrast we describe a three-tier architecture to manage a common data model for a set of applications. It is inspired by current Internet application frameworks and consists of a central storage layer using a common data model, a transformation layer responsible for filtering and adapting the data to the requirements of a particular applications on request, and finally of the applications itself. We demonstrate our architecture in a scenario consisting of two multi-user capable mobile AR applications for collaborative navigation and annotation in a city environment.
---
paper_title: Bluetooth and WAP push based location-aware mobile advertising system

paper_content:

Advertising on mobile devices has large potential due to the very personal and intimate nature of the devices and high targeting possibilities. We introduce a novel B-MAD system for delivering permission-based location-aware mobile advertisements to mobile phones using Bluetooth positioning and Wireless Application Protocol (WAP) Push. We present a thorough quantitative evaluation of the system in a laboratory environment and qualitative user evaluation in form of a field trial in the real environment of use. Experimental results show that the system provides a viable solution for realizing permission-based mobile advertising.
---
paper_title: Mobile reality: A pda-based multimodal framework for synchronizing a hybrid tracking solution with 3d-graphics and location-sensitve speech interaction

paper_content:

A maintenance engineer who talks to pumps and pipes may not seem like the ideal person to entrust with keeping a factory running smoothly, but we hope that our Mobile Reality framework will enable such behavior in the future to be anything but suspicious! Described in this paper is how the Mobile Reality framework, running entirely on a Pocket PC, synchronizes a hybrid tracking solution to offer the user a seamless, location-dependent, mobile multimodal interface. The user interface juxtaposes a three-dimensional graphical view with a context-sensitive speech dialog centered upon objects located in the immediate vicinity of the mobile user. In addition, support for collaboration enables shared VRML browsing with annotation and a full-duplex voice channel.
---
paper_title: Indoor wayfinding: developing a functional interface for individuals with cognitive impairments.

paper_content:

PURPOSE ::: Assistive technology for wayfinding will significantly improve the quality of life for many individuals with cognitive impairments. The user interface of such a system is as crucial as the underlying implementation and localisation technology. We studied the user interface of an indoor navigation system for individuals with cognitive impairments. ::: ::: ::: METHOD ::: We built a system using the Wizard-of-Oz technique that let us experiment with many guidance strategies and interface modalities. Through user studies, we evaluated various configurations of the user interface for accuracy of route completion, time to completion, and user preferences. We used a counter-balanced design that included different modalities (images, audio, and text) and different routes. ::: ::: ::: RESULTS ::: We found that although users were able to use all types of modalities to find their way indoors, they varied significantly in their preferred modalities. We also found that timing of directions requires careful attention, as does providing users with confirmation messages at appropriate times. ::: ::: ::: CONCLUSIONS ::: Our findings suggest that the ability to adapt indoor wayfinding devices for specific users' preferences and needs will be particularly important.
---
paper_title: Scan and Tilt – Towards Natural Interaction for Mobile Museum Guides

paper_content:

This paper presents a new interaction technique - scan and tilt - aiming to enable a more natural interaction with mobile museum guides. Our work combines multiple modalities - gestures, physical selection, location, graphical and voice. In particular, physical selection is obtained by scanning RFID tags associated with the artworks, and tilt gestures are used to control and navigate the user interface and multimedia information. We report on how it has been applied to a mobile museum guide in order to enhance the user experience, providing details on a first user test carried out on our prototype.
---
paper_title: Mobile augmented reality in the data center

paper_content:

Recent advances in compute power, graphics power, and cameras in mobile computing devices have facilitated the development of new augmented reality applications in the mobile device space. Mobile augmented reality provides a straightforward and natural way for users to understand complex data by overlaying visualization on top of a live video feed on their mobile device. In our data center mobile augmented reality project, the user points a mobile device camera at a rack of data center assets, and additional content about these assets is visually overlaid on top of each asset in the video stream from the mobile device camera. This correspondence between digital content and physical things or locations makes mobile augmented reality an intuitive user interface for interacting with the world around us. This paper describes augmented reality techniques for mobile devices and the motivations around using a multimarker computer vision-based technique for visualizing asset data in the data center. Our mobile augmented reality project enables system administrators to easily interact with hardware assets while they are in the data center, providing them with an additional tool to use in managing the data center.
---
paper_title: Combination of UWB and GPS for indoor-outdoor vehicle localization

paper_content:

GPS receivers are satellite-based devices widely used for vehicle localization that, given their limitations, are not suitable for performing within indoor or dense urban environments. On the other hand ultra-wide band (UWB), a technology used for efficient wireless communication, has recently being used for vehicle localization in indoor environments with promising results. This paper focuses on the combination of both technologies for accurate positioning of vehicles in a mixed scenario (both indoor and outdoor situations), which is typical in some industrial applications. Our approach is based on combining sensor information in a Monte Carlo localization algorithm (also known as particle Filter), which has revealed its suitability for probabilistically coping with a variety of sensory data. The performance of our approach has been satisfactorily tested on a real robot, endowed with a UWB master antenna and a GPS receiver, within an indoor-outdoor scenario where three UWB slave antennas were placed in the indoor area.
---
paper_title: Archeoguide: system architecture of a mobile outdoor augmented reality system

paper_content:

We present the system architecture of a mobile outdoor augmented reality system for the Archeoguide project. We begin with a short introduction to the project. Then we present the hardware we chose for the mobile system and we describe the system architecture we designed for the software implementation. We conclude this paper with the first results obtained from experiments we made during our trials at ancient Olympia in Greece.
---
paper_title: An accurate ultra wideband (UWB) ranging for precision asset location

paper_content:

This paper investigates a ranging method employing ultra wideband (UWB) pulses under the existence of the line of sight (LOS) path in a multipath environment. Our method is based on the estimation of time of arrival of the first multipath. It averages the received pulses over multiple time frames, performs a correlation operation on the averaged signal, and detects the peak of the correlated signal. Our method reduces the ranging accuracy over conventional methods, and its accuracy is close to the Cramer-Rao lower bound (CRLB) on even for a low SNR.
---
paper_title: Implementation of an augmented reality system on a PDA

paper_content:

We present a client/server implementation for running demanding mobile AR application on a PDA device. The system incorporates various data compression methods to make it run as fast as possible on a wide range of communication networks, from GSM to WLAN.
---
paper_title: Structured visual markers for indoor pathfinding

paper_content:

We present a mobile augmented reality (AR) system to guide a user through an unfamiliar building to a destination room. The system presents a world-registered wireframe model of the building labeled with directional information in a see-through heads-up display, and a three-dimensional world-in-miniature (WIM) map on a wrist-worn pad that also acts as an input device. Tracking is done using a combination of wall-mounted ARToolkit markers observed by a head-mounted camera, and an inertial tracker. To allow coverage of arbitrarily large areas with a limited set of markers, a structured marker re-use scheme based on graph coloring has been developed.
---
paper_title: User Interface Management Techniques for Collaborative Mobile Augmented Reality

paper_content:

Mobile Augmented Reality Systems (MARS) have the potential to revolutionize the way in which information is provided to users. Virtual information can be directly integrated with the real world surrounding the mobile user, who can interact with it to display related information, to pose and resolve queries, and to collaborate with other users. However, we believe that the benefits of MARS will only be achieved if the user interface (UI) is actively managed so as to maximize the relevance and minimize the confusion of the virtual material relative to the real world. This article addresses some of the steps involved in this process, focusing on the design and layout of the mobile user’s overlaid virtual environment. The augmented view of the user’s surroundings presents an interface to context-dependent operations, many of which are related to the objects in view—the augmented world is the user interface. We present three user interface design techniques that are intended to make this interface as obvious and clear to the user as possible: information filtering, UI component design, and view management. Information filtering helps select the most relevant information to present to the user. UI component designdetermines the format in which this information should be conveyed, based on the available display resources and tracking accuracy. For example, the absence of high accuracy position tracking would favor body- or screenstabilized components over world-stabilized ones that would need to be exactly registered with the physical objects to which they refer. View management attempts to ensure that the virtual objects that are displayed visually are arranged appropriately with regard to their projections on the view plane. For example, the relationships among objects should be as unambiguous as possible, and physical or virtual objects should not obstruct the user’s view of more important physical or virtual objects in the scene. We illustrate these interface design techniques using our prototype collaborative, cross-site MARS environment, which is composed of mobile and non-mobile augmented reality and virtual reality systems.
---
paper_title: LANDMARC: indoor location sensing using active RFID

paper_content:

Growing convergence among mobile computing devices and embedded technology sparks the development and deployment of “context-aware” applications, where location is the most essential context. In this paper we present LANDMARC, a location sensing prototype system that uses Radio Frequency Identification (RFID) technology for locating objects inside buildings. The major advantage of LANDMARC is that it improves the overall accuracy of locating objects by utilizing the concept of reference tags. Based on experimental analysis, we demonstrate that active RFID is a viable and cost-effective candidate for indoor location sensing. Although RFID is not designed for indoor location sensing, we point out three major features that should be added to make RFID technologies competitive in this new and growing market.
---
paper_title: Human pacman: A mobile entertainment system with ubiquitous computing and tangible interaction over a wide outdoor area

paper_content:

Human Pacman is an interactive role-playing game that envisions to bring the computer gaming experience to a new level of emotional and sensory gratification by setting the real world as a playground. This is a physical fantasy game integrated with human-social and mobile-gaming that emphasizes on collaboration and competition between players. By setting the game in a wide outdoor area, natural human-physical movements have become an integral part of the game. Pacmen and Ghosts are now human players in the real world experiencing mixed reality visualization from the wearable computers on them. Virtual cookies and actual physical objects are incorporated to provide novel experiences of seamless transitions between real and virtual worlds and tangible human computer interface respectively. We believe Human Pacman is pioneering a new form of gaming that anchors on physicality, mobility, social interaction, and ubiquitous computing.
---
paper_title: Video see-through AR on consumer cell-phones

paper_content:

We present a first running video see-through augmented reality system on a consumer cell-phone. It supports the detection and differentiation of different markers, and correct integration of rendered 3D graphics into the live video stream via a weak perspective projection camera model and an OpenGL rendering pipeline.
---
paper_title: Augmented reality in a wide area sentient environment

paper_content:

Augmented reality (AR) both exposes and supplements the user's view of the real world. Previous AR work has focussed on the close registration of real and virtual objects, which requires very accurate real-time estimates of head position and orientation. Most of these systems have been tethered and restricted to small volumes. In contrast, we have chosen to concentrate on allowing the AR user to roam freely within an entire building. At AT&T Laboratories Cambridge we provide personnel with AR services using data from an ultrasonic tracking system, called the Bat system, which has been deployed building-wide. We have approached the challenge of implementing a wide-area, in-building AR system in two different ways. The first uses a head-mounted display connected to a laptop, which combines sparse position measurements from the Bat system with more frequent rotational information from an inertial tracker to render annotations and virtual objects that relate to or coexist with the real world. The second uses a PDA to provide a convenient portal with which the user can quickly view the augmented world. These systems can be used to annotate the world in a more-or-less seamless way, allowing a richer interaction with both real and virtual objects.
---
paper_title: Mobile reality: A pda-based multimodal framework for synchronizing a hybrid tracking solution with 3d-graphics and location-sensitve speech interaction

paper_content:

A maintenance engineer who talks to pumps and pipes may not seem like the ideal person to entrust with keeping a factory running smoothly, but we hope that our Mobile Reality framework will enable such behavior in the future to be anything but suspicious! Described in this paper is how the Mobile Reality framework, running entirely on a Pocket PC, synchronizes a hybrid tracking solution to offer the user a seamless, location-dependent, mobile multimodal interface. The user interface juxtaposes a three-dimensional graphical view with a context-sensitive speech dialog centered upon objects located in the immediate vicinity of the mobile user. In addition, support for collaboration enables shared VRML browsing with annotation and a full-duplex voice channel.
---
paper_title: Managing Complex Augmented Reality Models

paper_content:

Mobile augmented reality requires georeferenced data to present world-registered overlays. To cover a wide area and all artifacts and activities, a database containing this information must be created, stored, maintained, delivered, and finally used by the client application. We present a data model and a family of techniques to address these needs.
---
paper_title: Scan and Tilt – Towards Natural Interaction for Mobile Museum Guides

paper_content:

This paper presents a new interaction technique - scan and tilt - aiming to enable a more natural interaction with mobile museum guides. Our work combines multiple modalities - gestures, physical selection, location, graphical and voice. In particular, physical selection is obtained by scanning RFID tags associated with the artworks, and tilt gestures are used to control and navigate the user interface and multimedia information. We report on how it has been applied to a mobile museum guide in order to enhance the user experience, providing details on a first user test carried out on our prototype.
---
paper_title: The Cricket location-support system

paper_content:

This paper presents the design, implementation, and evaluation of Cricket , a location-support system for in-building, mobile, location-dependent applications. It allows applications running on mobile and static nodes to learn their physical location by using listeners that hear and analyze information from beacons spread throughout the building. Cricket is the result of several design goals, including user privacy, decentralized administration, network heterogeneity, and low cost. Rather than explicitly tracking user location, Cricket helps devices learn where they are and lets them decide whom to advertise this information to; it does not rely on any centralized management or control and there is no explicit coordination between beacons; it provides information to devices regardless of their type of network connectivity; and each Cricket device is made from off-the-shelf components and costs less than U.S. $10. We describe the randomized algorithm used by beacons to transmit information, the use of concurrent radio and ultrasonic signals to infer distance, the listener inference algorithms to overcome multipath and interference, and practical beacon configuration and positioning techniques that improve accuracy. Our experience with Cricket shows that several location-dependent applications such as in-building active maps and device control can be developed with little effort or manual configuration.
---
paper_title: Structured visual markers for indoor pathfinding

paper_content:

We present a mobile augmented reality (AR) system to guide a user through an unfamiliar building to a destination room. The system presents a world-registered wireframe model of the building labeled with directional information in a see-through heads-up display, and a three-dimensional world-in-miniature (WIM) map on a wrist-worn pad that also acts as an input device. Tracking is done using a combination of wall-mounted ARToolkit markers observed by a head-mounted camera, and an inertial tracker. To allow coverage of arbitrarily large areas with a limited set of markers, a structured marker re-use scheme based on graph coloring has been developed.
---
paper_title: Human pacman: A mobile entertainment system with ubiquitous computing and tangible interaction over a wide outdoor area

paper_content:

Human Pacman is an interactive role-playing game that envisions to bring the computer gaming experience to a new level of emotional and sensory gratification by setting the real world as a playground. This is a physical fantasy game integrated with human-social and mobile-gaming that emphasizes on collaboration and competition between players. By setting the game in a wide outdoor area, natural human-physical movements have become an integral part of the game. Pacmen and Ghosts are now human players in the real world experiencing mixed reality visualization from the wearable computers on them. Virtual cookies and actual physical objects are incorporated to provide novel experiences of seamless transitions between real and virtual worlds and tangible human computer interface respectively. We believe Human Pacman is pioneering a new form of gaming that anchors on physicality, mobility, social interaction, and ubiquitous computing.
---
paper_title: ARToolKitPlus for Pose Tracking on Mobile Devices

paper_content:

In this paper we present ARToolKitPlus, a successor to the popular ARToolKit pose tracking library. ARToolKitPlus has been optimized and extended for the usage on mobile devices such as smartphones, PDAs and Ultra Mobile PCs (UMPCs). We explain the need and specific requirements of pose tracking on mobile devices and how we met those requirements. To prove the applicability we performed an extensive benchmark series on a broad range of off-the-shelf handhelds.
---
paper_title: Mobile collaborative augmented reality

paper_content:

The combination of mobile computing and collaborative augmented reality into a single system makes the power of computer enhanced interaction and communication in the real world accessible anytime and everywhere. The paper describes our work to build a mobile collaborative augmented reality system that supports true stereoscopic 3D graphics, a pen and pad interface and direct interaction with virtual objects. The system is assembled from off-the-shelf hardware components and serves as a basic test bed for user interface experiments related to computer supported collaborative work in augmented reality. A mobile platform implementing the described features and collaboration between mobile and stationary users are demonstrated.
---
paper_title: Marker tracking and HMD calibration for a video-based augmented reality conferencing system

paper_content:

We describe an augmented reality conferencing system which uses the overlay of virtual images on the real world. Remote collaborators are represented on virtual monitors which can be freely positioned about a user in space. Users can collaboratively view and interact with virtual objects using a shared virtual whiteboard. This is possible through precise virtual image registration using fast and accurate computer vision techniques and head mounted display (HMD) calibration. We propose a method for tracking fiducial markers and a calibration method for optical see-through HMD based on the marker tracking.
---
paper_title: Shape recognition and pose estimation for mobile augmented reality

paper_content:

In this paper we present Nestor, a system for real-time recognition and camera pose estimation from planar shapes. The system allows shapes that carry contextual meanings for humans to be used as Augmented Reality (AR) tracking fiducials. The user can teach the system new shapes at runtime by showing them to the camera. The learned shapes are then maintained by the system in a shape library. Nestor performs shape recognition by analyzing contour structures and generating projective invariant signatures from their concavities. The concavities are further used to extract features for pose estimation and tracking. Pose refinement is carried out by minimizing the reprojection error between sample points on each image contour and its library counterpart. Sample points are matched by evolving an active contour in real time. Our experiments show that the system provides stable and accurate registration, and runs at interactive frame rates on a Nokia N95 mobile phone.
---
paper_title: UMAR: Ubiquitous Mobile Augmented Reality

paper_content:

In this paper we discuss the prospects of using marker based Augmented Reality for context aware applications on mobile phones. We also present the UMAR, a conceptual framework for developing Ubiquitous Mobile Augmented Reality applications which consists of research areas identified as relevant for successfully bridging the physical world and the digital domain using Mobile Augmented Reality. A step towards this we have successfully ported the ARToolkit to consumer mobile phones running on the Symbian platform and present results around this. We also present three sample applications based on UMAR and future case study work planned.
---
paper_title: Video see-through AR on consumer cell-phones

paper_content:

We present a first running video see-through augmented reality system on a consumer cell-phone. It supports the detection and differentiation of different markers, and correct integration of rendered 3D graphics into the live video stream via a weak perspective projection camera model and an OpenGL rendering pipeline.
---
paper_title: OpenTracker-an open software architecture for reconfigurable tracking based on XML

paper_content:

This paper describes OpenTracker, an open software architecture that provides a generic solution to the different tasks involved in tracking input devices and processing tracking data for virtual environments. It combines a highly modular design with a configuration syntax based on XML, thus taking full advantage of this new technology. OpenTracker is a first attempt towards a "write once, track anywhere" approach to virtual reality application development.
---
paper_title: Efficient Extraction of Robust Image Features on Mobile Devices

paper_content:

Recent convergence of imaging sensors and general purpose processors on mobile phones creates an opportunity for a new class of augmented reality applications. Robust image feature extraction is a crucial enabler of this type of systems. In this article, we discuss an efficient mobile phone implementation of a state-of-the-art algorithm for computing robust image features called SURF. We implement several improvements to the basic algorithm that significantly improve its performance and reduce its memory footprint making the use of this algorithm on the mobile phone practical. Our prototype implementation has been applied to several practical applications such as image search, object recognition and augmented reality applications.
---
paper_title: Fast Feature Pyramids for Object Detection

paper_content:

Multi-resolution image features may be approximated via extrapolation from nearby scales, rather than being computed explicitly. This fundamental insight allows us to design object detection algorithms that are as accurate, and considerably faster, than the state-of-the-art. The computational bottleneck of many modern detectors is the computation of features at every scale of a finely-sampled image pyramid. Our key insight is that one may compute finely sampled feature pyramids at a fraction of the cost, without sacrificing performance: for a broad family of features we find that features computed at octave-spaced scale intervals are sufficient to approximate features on a finely-sampled pyramid. Extrapolation is inexpensive as compared to direct feature computation. As a result, our approximation yields considerable speedups with negligible loss in detection accuracy. We modify three diverse visual recognition systems to use fast feature pyramids and show results on both pedestrian detection (measured on the Caltech, INRIA, TUD-Brussels and ETH data sets) and general object detection (measured on the PASCAL VOC). The approach is general and is widely applicable to vision algorithms requiring fine-grained multi-scale analysis. Our approximation is valid for images with broad spectra (most natural images) and fails for images with narrow band-pass spectra (e.g., periodic textures).
---
paper_title: BRIEF: Binary robust independent elementary features

paper_content:

We propose to use binary strings as an efficient feature point descriptor, which we call BRIEF. We show that it is highly discriminative even when using relatively few bits and can be computed using simple intensity difference tests. Furthermore, the descriptor similarity can be evaluated using the Hamming distance, which is very efficient to compute, instead of the L2 norm as is usually done. ::: ::: As a result, BRIEF is very fast both to build and to match. We compare it against SURF and U-SURF on standard benchmarks and show that it yields a similar or better recognition performance, while running in a fraction of the time required by either.
---
paper_title: Outdoors augmented reality on mobile phone using loxel-based visual feature organization

paper_content:

We have built an outdoors augmented reality system for mobile phones that matches camera-phone images against a large database of location-tagged images using a robust image retrieval algorithm. We avoid network latency by implementing the algorithm on the phone and deliver excellent performance by adapting a state-of-the-art image retrieval algorithm based on robust local descriptors. Matching is performed against a database of highly relevant features, which is continuously updated to reflect changes in the environment. We achieve fast updates and scalability by pruning of irrelevant features based on proximity to the user. By compressing and incrementally updating the features stored on the phone we make the system amenable to low-bandwidth wireless connections. We demonstrate system robustness on a dataset of location-tagged images and show a smart-phone implementation that achieves a high image matching rate while operating in near real-time.
---
paper_title: Robust Visual Tracking via Structured Multi-Task Sparse Learning

paper_content:

In this paper, we formulate object tracking in a particle filter framework as a structured multi-task sparse learning problem, which we denote as Structured Multi-Task Tracking (S-MTT). Since we model particles as linear combinations of dictionary templates that are updated dynamically, learning the representation of each particle is considered a single task in Multi-Task Tracking (MTT). By employing popular sparsity-inducing $$\ell _{p,q}$$ mixed norms $$(\text{ specifically} p\in \{2,\infty \}$$ and $$q=1),$$ we regularize the representation problem to enforce joint sparsity and learn the particle representations together. As compared to previous methods that handle particles independently, our results demonstrate that mining the interdependencies between particles improves tracking performance and overall computational complexity. Interestingly, we show that the popular $$L_1$$ tracker (Mei and Ling, IEEE Trans Pattern Anal Mach Intel 33(11):2259---2272, 2011) is a special case of our MTT formulation (denoted as the $$L_{11}$$ tracker) when $$p=q=1.$$ Under the MTT framework, some of the tasks (particle representations) are often more closely related and more likely to share common relevant covariates than other tasks. Therefore, we extend the MTT framework to take into account pairwise structural correlations between particles (e.g. spatial smoothness of representation) and denote the novel framework as S-MTT. The problem of learning the regularized sparse representation in MTT and S-MTT can be solved efficiently using an Accelerated Proximal Gradient (APG) method that yields a sequence of closed form updates. As such, S-MTT and MTT are computationally attractive. We test our proposed approach on challenging sequences involving heavy occlusion, drastic illumination changes, and large pose variations. Experimental results show that S-MTT is much better than MTT, and both methods consistently outperform state-of-the-art trackers.
---
paper_title: Pose tracking from natural features on mobile phones

paper_content:

In this paper we present two techniques for natural feature tracking in real-time on mobile phones. We achieve interactive frame rates of up to 20 Hz for natural feature tracking from textured planar targets on current-generation phones. We use an approach based on heavily modified state-of-the-art feature descriptors, namely SIFT and Ferns. While SIFT is known to be a strong, but computationally expensive feature descriptor, Ferns classification is fast, but requires large amounts of memory. This renders both original designs unsuitable for mobile phones. We give detailed descriptions on how we modified both approaches to make them suitable for mobile phones. We present evaluations on robustness and performance on various devices and finally discuss their appropriateness for augmented reality applications.
---
paper_title: Distinctive Image Features from Scale-Invariant Keypoints

paper_content:

The Scale-Invariant Feature Transform (or SIFT) algorithm is a highly robust method to extract and consequently match distinctive invariant features from images. These features can then be used to reliably match objects in diering images. The algorithm was rst proposed by Lowe [12] and further developed to increase performance resulting in the classic paper [13] that served as foundation for SIFT which has played an important role in robotic and machine vision in the past decade.
---
paper_title: Surf: Speeded up robust features

paper_content:

In this paper, we present a novel scale- and rotation-invariant interest point detector and descriptor, coined SURF (Speeded Up Robust Features). It approximates or even outperforms previously proposed schemes with respect to repeatability, distinctiveness, and robustness, yet can be computed and compared much faster. ::: ::: This is achieved by relying on integral images for image convolutions; by building on the strengths of the leading existing detectors and descriptors (in casu, using a Hessian matrix-based measure for the detector, and a distribution-based descriptor); and by simplifying these methods to the essential. This leads to a combination of novel detection, description, and matching steps. The paper presents experimental results on a standard evaluation set, as well as on imagery obtained in the context of a real-life object recognition application. Both show SURF's strong performance.
---
paper_title: Machine learning for high-speed corner detection

paper_content:

Where feature points are used in real-time frame-rate applications, a high-speed feature detector is necessary. Feature detectors such as SIFT (DoG), Harris and SUSAN are good methods which yield high quality features, however they are too computationally intensive for use in real-time applications of any complexity. Here we show that machine learning can be used to derive a feature detector which can fully process live PAL video using less than 7% of tlie available processing time. By comparison neither the Harris detector (120%) nor the detection stage of SIFT (300%) can operate at full frame rate. Clearly a high-speed detector is of limited use if the features produced are unsuitable for downstream processing. In particular, the same scene viewed from two different positions should yield features which correspond to the same real-world 3D locations[1]. Hence the second contribution of this paper is a comparison corner detectors based on this criterion applied to 3D scenes. This comparison supports a number of claims made elsewhere concerning existing corner detectors. Further, contrary to our initial expectations, we show that despite being principally constructed for speed, our detector significantly outperforms existing feature detectors according to this criterion.
---
paper_title: DeCAF: A Deep Convolutional Activation Feature for Generic Visual Recognition

paper_content:

We evaluate whether features extracted from the activation of a deep convolutional network trained in a fully supervised fashion on a large, fixed set of object recognition tasks can be repurposed to novel generic tasks. Our generic tasks may differ significantly from the originally trained tasks and there may be insufficient labeled or unlabeled data to conventionally train or adapt a deep architecture to the new tasks. We investigate and visualize the semantic clustering of deep convolutional features with respect to a variety of such tasks, including scene recognition, domain adaptation, and fine-grained recognition challenges. We compare the efficacy of relying on various network levels to define a fixed feature, and report novel results that significantly outperform the state-of-the-art on several important vision challenges. We are releasing DeCAF, an open-source implementation of these deep convolutional activation features, along with all associated network parameters to enable vision researchers to be able to conduct experimentation with deep representations across a range of visual concept learning paradigms.
---
paper_title: Natural Feature Tracking for Augmented Reality

paper_content:

Natural scene features stabilize and extend the tracking range of augmented reality (AR) pose-tracking systems. We develop robust computer vision methods to detect and track natural features in video images. Point and region features are automatically and adaptively selected for properties that lead to robust tracking. A multistage tracking algorithm produces accurate motion estimates, and the entire system operates in a closed-loop that stabilizes its performance and accuracy. We present demonstrations of the benefits of using tracked natural features for AR applications that illustrate direct scene annotation, pose stabilization, and extendible tracking range. Our system represents a step toward integrating vision with graphics to produce robust wide-area augmented realities.
---
paper_title: A hybrid pose tracking approach for handheld augmented reality

paper_content:

With the rapid advances in mobile computing, handheld Augmented Reality draws increasing attention. Pose tracking of handheld devices is of fundamental importance to register virtual information with the real world and is still a crucial challenge. In this paper, we present a low-cost, accurate and robust approach combining fiducial tracking and inertial sensors for handheld pose tracking. Two LEDs are used as fiducial markers to indicate the position of the handheld device. They are detected by an adaptive thresholding method which is robust to illumination changes, and then tracked by a Kalman filter. By combining inclination information provided by the on-device accelerometer, 6 degree-of-freedom (DoF) pose is estimated. Handheld devices are freed from computer vision processing, leaving most computing power available for applications. When one LED is occluded, the system is still able to recover the 6-DoF pose. Performance evaluation of the proposed tracking approach is carried out by comparing with the ground truth data generated by the state-of-the-art commercial motion tracking system OptiTrack. Experimental results show that the proposed system has achieved an accuracy of 1.77 cm in position estimation and 4.15 degrees in orientation estimation.
---
paper_title: Object Tracking by Oversampling Local Features

paper_content:

In this paper, we present the ALIEN tracking method that exploits oversampling of local invariant representations to build a robust object/context discriminative classifier. To this end, we use multiple instances of scale invariant local features weakly aligned along the object template. This allows taking into account the 3D shape deviations from planarity and their interactions with shadows, occlusions, and sensor quantization for which no invariant representations can be defined. A non-parametric learning algorithm based on the transitive matching property discriminates the object from the context and prevents improper object template updating during occlusion. We show that our learning rule has asymptotic stability under mild conditions and confirms the drift-free capability of the method in long-term tracking. A real-time implementation of the ALIEN tracker has been evaluated in comparison with the state-of-the-art tracking systems on an extensive set of publicly available video sequences that represent most of the critical conditions occurring in real tracking environments. We have reported superior or equal performance in most of the cases and verified tracking with no drift in very long video sequences.
---
paper_title: PCA-SIFT: a more distinctive representation for local image descriptors

paper_content:

Stable local feature detection and representation is a fundamental component of many image registration and object recognition algorithms. Mikolajczyk and Schmid (June 2003) recently evaluated a variety of approaches and identified the SIFT [D. G. Lowe, 1999] algorithm as being the most resistant to common image deformations. This paper examines (and improves upon) the local image descriptor used by SIFT. Like SIFT, our descriptors encode the salient aspects of the image gradient in the feature point's neighborhood; however, instead of using SIFT's smoothed weighted histograms, we apply principal components analysis (PCA) to the normalized gradient patch. Our experiments demonstrate that the PCA-based local descriptors are more distinctive, more robust to image deformations, and more compact than the standard SIFT representation. We also present results showing that using these descriptors in an image retrieval application results in increased accuracy and faster matching.
---
paper_title: Caffe: Convolutional Architecture for Fast Feature Embedding

paper_content:

Caffe provides multimedia scientists and practitioners with a clean and modifiable framework for state-of-the-art deep learning algorithms and a collection of reference models. The framework is a BSD-licensed C++ library with Python and MATLAB bindings for training and deploying general-purpose convolutional neural networks and other deep models efficiently on commodity architectures. Caffe fits industry and internet-scale media needs by CUDA GPU computation, processing over 40 million images a day on a single K40 or Titan GPU ($\approx$ 2.5 ms per image). By separating model representation from actual implementation, Caffe allows experimentation and seamless switching among platforms for ease of development and deployment from prototyping machines to cloud environments. Caffe is maintained and developed by the Berkeley Vision and Learning Center (BVLC) with the help of an active community of contributors on GitHub. It powers ongoing research projects, large-scale industrial applications, and startup prototypes in vision, speech, and multimedia.
---
paper_title: BRISK: Binary Robust invariant scalable keypoints

paper_content:

Effective and efficient generation of keypoints from an image is a well-studied problem in the literature and forms the basis of numerous Computer Vision applications. Established leaders in the field are the SIFT and SURF algorithms which exhibit great performance under a variety of image transformations, with SURF in particular considered as the most computationally efficient amongst the high-performance methods to date. In this paper we propose BRISK1, a novel method for keypoint detection, description and matching. A comprehensive evaluation on benchmark datasets reveals BRISK's adaptive, high quality performance as in state-of-the-art algorithms, albeit at a dramatically lower computational cost (an order of magnitude faster than SURF in cases). The key to speed lies in the application of a novel scale-space FAST-based detector in combination with the assembly of a bit-string descriptor from intensity comparisons retrieved by dedicated sampling of each keypoint neighborhood.
---
paper_title: Streaming mobile augmented reality on mobile phones

paper_content:

Continuous recognition and tracking of objects in live video captured on a mobile device enables real-time user interaction. We demonstrate a streaming mobile augmented reality system with 1 second latency. User interest is automatically inferred from camera movements, so the user never has to press a button. Our system is used to identify and track book and CD covers in real time on a phone's viewfinder. Efficient motion estimation is performed at 30 frames per second on a phone, while fast search through a database of 20,000 images is performed on a server.
---
paper_title: Feature Tracking for Mobile Augmented Reality Using Video Coder Motion Vectors

paper_content:

We propose a novel, low-complexity, tracking scheme that uses motion vectors directly from a video coder. We compare our tracking algorithm against ground truth data, and show that we can achieve a high level of accuracy, even though the motion vectors are rate-distortion optimized and do not represent true motion. We develop a framework for tracking in video sequences with various GOP structures. Such a scheme would find applications in the context of Mobile Augmented Reality. The proposed feature tracking algorithm can significantly reduce the required rate of feature extraction and matching.
---
paper_title: SURFTrac: Efficient tracking and continuous object recognition using local feature descriptors

paper_content:

We present an efficient algorithm for continuous image recognition and feature descriptor tracking in video which operates by reducing the search space of possible interest points inside of the scale space image pyramid. Instead of performing tracking in 2D images, we search and match candidate features in local neighborhoods inside the 3D image pyramid without computing their feature descriptors. The candidates are further validated by fitting to a motion model. The resulting tracked interest points are more repeatable and resilient to noise, and descriptor computation becomes much more efficient because only those areas of the image pyramid that contain features are searched. We demonstrate our method on real-time object recognition and label augmentation running on a mobile device.
---
paper_title: Scene modelling, recognition and tracking with invariant image features

paper_content:

We present a complete system architecture for fully automated markerless augmented reality (AR). The system constructs a sparse metric model of the real-world environment, provides interactive means for specifying the pose of a virtual object, and performs model-based camera tracking with visually pleasing augmentation results. Our approach does not require camera pre-calibration, prior knowledge of scene geometry, manual initialization of the tracker or placement of special markers. Robust tracking in the presence of occlusions and scene changes is achieved by using highly distinctive natural features to establish image correspondences.
---
paper_title: Real-Time Detection and Tracking for Augmented Reality on Mobile Phones

paper_content:

In this paper, we present three techniques for 6DOF natural feature tracking in real time on mobile phones. We achieve interactive frame rates of up to 30 Hz for natural feature tracking from textured planar targets on current generation phones. We use an approach based on heavily modified state-of-the-art feature descriptors, namely SIFT and Ferns plus a template-matching-based tracker. While SIFT is known to be a strong, but computationally expensive feature descriptor, Ferns classification is fast, but requires large amounts of memory. This renders both original designs unsuitable for mobile phones. We give detailed descriptions on how we modified both approaches to make them suitable for mobile phones. The template-based tracker further increases the performance and robustness of the SIFT- and Ferns-based approaches. We present evaluations on robustness and performance and discuss their appropriateness for Augmented Reality applications.
---
paper_title: Multiple target detection and tracking with guaranteed framerates on mobile phones

paper_content:

In this paper we present a novel method for real-time pose estimation and tracking on low-end devices such as mobile phones. The presented system can track multiple known targets in real-time and simultaneously detect new targets for tracking. We present a method to automatically and dynamically balance the quality of detection and tracking to adapt to a variable time budget and ensure a constant frame rate. Results from real data of a mobile phone Augmented Reality system demonstrate the efficiency and robustness of the described approach. The system can track 6 planar targets on a mobile phone simultaneously at framerates of 23fps.
---
paper_title: CNN Features Off-the-Shelf: An Astounding Baseline for Recognition

paper_content:

Recent results indicate that the generic descriptors extracted from the convolutional neural networks are very powerful. This paper adds to the mounting evidence that this is indeed the case. We report on a series of experiments conducted for different recognition tasks using the publicly available code and model of the OverFeat network which was trained to perform object classification on ILSVRC13. We use features extracted from the OverFeat network as a generic image representation to tackle the diverse range of recognition tasks of object image classification, scene recognition, fine grained recognition, attribute detection and image retrieval applied to a diverse set of datasets. We selected these tasks and datasets as they gradually move further away from the original task and data the OverFeat network was trained to solve. Astonishingly, we report consistent superior results compared to the highly tuned state-of-the-art systems in all the visual classification tasks on various datasets. For instance retrieval it consistently outperforms low memory footprint methods except for sculptures dataset. The results are achieved using a linear SVM classifier (or L2 distance in case of retrieval) applied to a feature representation of size 4096 extracted from a layer in the net. The representations are further modified using simple augmentation techniques e.g. jittering. The results strongly suggest that features obtained from deep learning with convolutional nets should be the primary candidate in most visual recognition tasks.
---
paper_title: A Mobile Vision System for Urban Detection with Informative Local Descriptors

paper_content:

We present a computer vision system for the detection and identification of urban objects from mobile phone imagery, e.g., for the application of tourist information services. Recognition is based on MAP decision making over weak object hypotheses from local descriptor responses in the mobile imagery. We present an improvement over the standard SIFT key detector [7] by selecting only informative (i-SIFT) keys for descriptor matching. Selection is applied first to reduce the complexity of the object model and second to accelerate detection by selective filtering. We present results on the MPG-20 mobile phone imagery with severe illumination, scale and viewpoint changes in the images, performing with ≈ 98% accuracy in identification, efficient (100%) background rejection, efficient (0%) false alarm rate, and reliable quality of service under extreme illumination conditions, significantly improving standard SIFT based recognition in every sense, providing - important for mobile vision - runtimes which are ≈ 8 (≈24) times faster for the MPG-20 (ZuBuD) database.
---
paper_title: ORB: An efficient alternative to SIFT or SURF

paper_content:

Feature matching is at the base of many computer vision problems, such as object recognition or structure from motion. Current methods rely on costly descriptors for detection and matching. In this paper, we propose a very fast binary descriptor based on BRIEF, called ORB, which is rotation invariant and resistant to noise. We demonstrate through experiments how ORB is at two orders of magnitude faster than SIFT, while performing as well in many situations. The efficiency is tested on several real-world applications, including object detection and patch-tracking on a smart phone.
---
paper_title: Online tracking and reacquisition using co-trained generative and discriminative trackers

paper_content:

Visual tracking is a challenging problem, as an object may change its appearance due to viewpoint variations, illumination changes, and occlusion. Also, an object may leave the field of view and then reappear. In order to track and reacquire an unknown object with limited labeling data, we propose to learn these changes online and build a model that describes all seen appearance while tracking. To address this semi-supervised learning problem, we propose a co-training based approach to continuously label incoming data and online update a hybrid discriminative generative model. The generative model uses a number of low dimension linear subspaces to describe the appearance of the object. In order to reacquire an object, the generative model encodes all the appearance variations that have been seen. A discriminative classifier is implemented as an online support vector machine, which is trained to focus on recent appearance variations. The online co-training of this hybrid approach accounts for appearance changes and allows reacquisition of an object after total occlusion. We demonstrate that under challenging situations, this method has strong reacquisition ability and robustness to distracters in background.
---
paper_title: InfoSPOT: A mobile Augmented Reality method for accessing building information through a situation awareness approach

paper_content:

article i nfo Article history: Accepted 9 September 2012 Available online 6 October 2012 The Architecture, Engineering, Construction, and Owner/Operator (AECO) industry is constantly searching for new methods for increasing efficiency and productivity. Facility Managers (FMs), as a part of the owner/ operator role, work in complex and dynamic environments where critical decisions are constantly made. This decision-making process and its consequent performance can be improved by enhancing Situation Awareness (SA) of the FMs through new digital technologies. In this paper, InfoSPOT (Information Surveyed Point for Observation and Tracking), is recommended to FMs as a mobile Augmented Reality (AR) tool for accessing information about the facilities they maintain. AR has been considered as a viable option to reduce inefficiencies of data overload by providing FMs with a SA-based tool for visualizing their "real-world" envi- ronment with added interactive data. A prototype of the AR application was developed and a user participa- tion experiment and analysis conducted to evaluate the features of InfoSPOT. This innovative application of AR has the potential to improve construction practices, and in this case, facility management. Published by Elsevier B.V.
---
paper_title: GroundCam: A Tracking Modality for Mobile Mixed Reality

paper_content:

Anywhere augmentation pursues the goal of lowering the initial investment of time and money necessary to participate in mixed reality work, bridging the gap between researchers in the field and regular computer users. Our paper contributes to this goal by introducing the GroundCam, a cheap tracking modality with no significant setup necessary. By itself, the GroundCam provides high frequency, high resolution relative position information similar to an inertial navigation system, but with significantly less drift. When coupled with a wide area tracking modality via a complementary Kalman filter, the hybrid tracker becomes a powerful base for indoor and outdoor mobile mixed reality work
---
paper_title: Model-based visual tracking for outdoor augmented reality applications

paper_content:

Outdoor augmented reality (AR) applications rely on hybrid tracking (GPS, digital compass, visual) for registration. RSC has developed a real-time visual tracking system that uses visual cues of buildings in an urban environment for correcting the results of a conventional tracking system. This approach relies on knowledge of a CAD model of the building. It not only provides motion estimation, but also absolute orientation/position. It is based on the "visual servoing" approach, originally developed for robotics tasks. We have demonstrated this approach in real-time at a building on the NRL campus This poster shows the approach and results. The concept can be generalized to any scenario where a CAD model is available. This system is being prepared for integration into the NRL system BARS (Battlefield Augmented Reality System).
---
paper_title: Sketching up the world: in situ authoring for mobile Augmented Reality

paper_content:

We present a novel system allowing in situ content creation for mobile Augmented Reality in unprepared environments. This system targets smartphones and therefore allows a spontaneous authoring while in place. We describe two different scenarios, which are depending on the size of the working environment and consequently use different tracking techniques. A natural feature-based approach for planar targets is used for small working spaces whereas for larger working environments, such as in outdoor scenarios, a panoramic-based orientation tracking is deployed. Both are integrated into one system allowing the user to use the same interaction for creating the content applying a set of simple, yet powerful modeling functions for content creation. The resulting content for Augmented Reality can be shared with other users using a dedicated content server or kept in a private inventory for later use.
---
paper_title: Going out: robust model-based tracking for outdoor augmented reality

paper_content:

This paper presents a model-based hybrid tracking system for outdoor augmented reality in urban environments enabling accurate, realtime overlays for a handheld device. The system combines several well-known approaches to provide a robust experience that surpasses each of the individual components alone: an edge-based tracker for accurate localisation, gyroscope measurements to deal with fast motions, measurements of gravity and magnetic field to avoid drift, and a back store of reference frames with online frame selection to re-initialize automatically after dynamic occlusions or failures. A novel edge-based tracker dispenses with the conventional edge model, and uses instead a coarse, but textured, 3D model. This yields several advantages: scale-based detail culling is automatic, appearance-based edge signatures can be used to improve matching and the models needed are more commonly available. The accuracy and robustness of the resulting system is demonstrated with comparisons to map-based ground truth data.
---
paper_title: A robust hybrid tracking system for outdoor augmented reality

paper_content:

We present a real-time hybrid tracking system that integrates gyroscopes and line-based vision tracking technology. Gyroscope measurements are used to predict orientation and image line positions. Gyroscope drift is corrected by vision tracking. System robustness is achieved by using a heuristic control system to evaluate measurement quality and select measurements accordingly. Experiments show that the system achieves robust, accurate, and real-time performance for outdoor augmented reality.
---
paper_title: Location-based augmented reality on mobile phones

paper_content:

The computational capability of mobile phones has been rapidly increasing, to the point where augmented reality has become feasible on cell phones. We present an approach to indoor localization and pose estimation in order to support augmented reality applications on a mobile phone platform. Using the embedded camera, the application localizes the device in a familiar environment and determines its orientation. Once the 6 DOF pose is determined, 3D virtual objects from a database can be projected into the image and displayed for the mobile user. Off-line data acquisition consists of acquiring images at different locations in the environment. The online pose estimation is done by a feature-based matching between the cell phone image and an image selected from the precomputed database using the phone's sensors (accelerometer and magnetometer). The application enables the user both to visualize virtual objects in the camera image and to localize the user in a familiar environment. We describe in detail the process of building the database and the pose estimation algorithm used on the mobile phone. We evaluate the algorithm performance as well as its accuracy in terms of reprojection distance of the 3D virtual objects in the cell phone image.
---
paper_title: Fusion of vision, GPS and 3D gyro data in solving camera registration problem for direct visual navigation

paper_content:

This paper presents a precise and robust camera registration solution for the novel vision-based road navigation system - VICNAS, which superimposes virtual 3D navigation indicators and traffic signs upon the real road view in an Augmented Reality (AR) space. Traditional vision based or inertial sensor based solutions of registration problem are mostly designed for well-structured environment, which is however unavailable in a wide-open uncontrolled road environment for navigation purposes. This paper proposed a hybrid system that combines computer vision, GPS and 3D inertial gyroscope technologies to provide precise and robust camera pose estimation. The fusion approach is based on our PMM (parameterized model matching) algorithm, in which the road shape model is derived from the digital map data, and matched with road features extracted from real images. Inertial data estimates the initial possible motion, and also serves as relative tolerance to stable the pose output. The algorithms proposed in this paper are validated with the experimental results of real road tests under different road conditions.
---
paper_title: Towards wearable cognitive assistance

paper_content:

We describe the architecture and prototype implementation of an assistive system based on Google Glass devices for users in cognitive decline. It combines the first-person image capture and sensing capabilities of Glass with remote processing to perform real-time scene interpretation. The system architecture is multi-tiered. It offers tight end-to-end latency bounds on compute-intensive operations, while addressing concerns such as limited battery capacity and limited processing capability of wearable devices. The system gracefully degrades services in the face of network failures and unavailability of distant architectural tiers.
---
paper_title: CloudRidAR: a cloud-based architecture for mobile augmented reality

paper_content:

Mobile augmented reality (MAR) has exploded in popularity on mobile devices in various fields. However, building a MAR application from scratch on mobile devices is complicated and time-consuming. In this paper, we propose CloudRidAR, a framework for MAR developers to facilitate the development, deployment, and maintenance of MAR applications with little effort. Despite of advance in mobile devices as a computing platform, their performance for MAR applications is still very limited due to the poor computing capability of mobile devices. In order to alleviate the problem, our CloudRidAR is designed with cloud computing at the core. Computational intensive tasks are offloaded on cloud to accelerate computation in order to guarantee run-time performance. We also present two MAR applications built on CloudRidAR to evaluate our design.
---
paper_title: Performance evaluation of computation offloading from mobile device to the edge of mobile network

paper_content:

Small Cell Cloud (SCC) consists of Cloud-enabled Small Cells (CeSCs), which serve as radio end-points for mobile user equipments (UEs) and host computation offloaded from mobile UEs. SCC hereby brings advantages of a centralized cloud computation to the users' vicinity. The SCC architecture provides a mechanism for distribution of computation demand across the CeSCs. An effectiveness of the offloading is determined based on quality of radio channel between the UEs and the CeSC and predicted computation complexity. In this paper, we introduce an implementation of an offloading framework to facilitate adaptation of mobile apps for the SCC and to handle low-level communication between the app and the SCC. An evaluation of the offloading framework is conducted using Augmented Reality (AR) app, which requires intensive computations and low latency. The offloading framework and the AR app are a basement for the SCC testbed used to proof the concept of the computation offloading. Various computation and radio parameters are investigated to reveal benefits of the SCC. According to the performed measurements, the computation offloading can decrease latency up to 88 % and energy consumption of the UEs up to 93 %.
---
paper_title: OverLay: Practical Mobile Augmented Reality

paper_content:

The idea of augmented reality - the ability to look at a physical object through a camera and view annotations about the object - is certainly not new. Yet, this apparently feasible vision has not yet materialized into a precise, fast, and comprehensively usable system. This paper asks: What does it take to enable augmented reality (AR) on smartphones today? To build a ready-to-use mobile AR system, we adopt a top-down approach cutting across smartphone sensing, computer vision, cloud offloading, and linear optimization. Our core contribution is in a novel location-free geometric representation of the environment - from smartphone sensors - and using this geometry to prune down the visual search space. Metrics of success include both accuracy and latency of object identification, coupled with the ease of use and scalability in uncontrolled environments. Our converged system, OverLay, is currently deployed in the engineering building and open for use to regular public; ongoing work is focussed on campus-wide deployment to serve as a "historical tour guide" of UIUC. Performance results and user responses thus far have been promising, to say the least.
---
paper_title: Glimpse: Continuous, Real-Time Object Recognition on Mobile Devices

paper_content:

Glimpse is a continuous, real-time object recognition system for camera-equipped mobile devices. Glimpse captures full-motion video, locates objects of interest, recognizes and labels them, and tracks them from frame to frame for the user. Because the algorithms for object recognition entail significant computation, Glimpse runs them on server machines. When the latency between the server and mobile device is higher than a frame-time, this approach lowers object recognition accuracy. To regain accuracy, Glimpse uses an active cache of video frames on the mobile device. A subset of the frames in the active cache are used to track objects on the mobile, using (stale) hints about objects that arrive from the server from time to time. To reduce network bandwidth usage, Glimpse computes trigger frames to send to the server for recognizing and labeling. Experiments with Android smartphones and Google Glass over Verizon, ATT without Glimpse, continuous detection is non-functional (0.2%-1.9% precision).
---
paper_title: Augmenting 3D urban environment using mobile devices

paper_content:

We describe an augmented reality prototype for exploring a 3D urban environment on mobile devices. Our system utilizes the location and orientation sensors on the mobile platform as well as computer vision techniques to register the live view of the device with the 3D urban data. In particular, the system recognizes the buildings in the live video, tracks the camera pose, and augments the video with relevant information about the buildings in the correct perspective. The 3D urban data consist of 3D point clouds and corresponding geo-tagged RGB images of the urban environment. We also discuss the processing steps to make such 3D data scalable and usable by our system.
---
paper_title: Outdoors augmented reality on mobile phone using loxel-based visual feature organization

paper_content:

We have built an outdoors augmented reality system for mobile phones that matches camera-phone images against a large database of location-tagged images using a robust image retrieval algorithm. We avoid network latency by implementing the algorithm on the phone and deliver excellent performance by adapting a state-of-the-art image retrieval algorithm based on robust local descriptors. Matching is performed against a database of highly relevant features, which is continuously updated to reflect changes in the environment. We achieve fast updates and scalability by pruning of irrelevant features based on proximity to the user. By compressing and incrementally updating the features stored on the phone we make the system amenable to low-bandwidth wireless connections. We demonstrate system robustness on a dataset of location-tagged images and show a smart-phone implementation that achieves a high image matching rate while operating in near real-time.
---
paper_title: Managing Complex Augmented Reality Models

paper_content:

Mobile augmented reality requires georeferenced data to present world-registered overlays. To cover a wide area and all artifacts and activities, a database containing this information must be created, stored, maintained, delivered, and finally used by the client application. We present a data model and a family of techniques to address these needs.
---
paper_title: UMAR: Ubiquitous Mobile Augmented Reality

paper_content:

In this paper we discuss the prospects of using marker based Augmented Reality for context aware applications on mobile phones. We also present the UMAR, a conceptual framework for developing Ubiquitous Mobile Augmented Reality applications which consists of research areas identified as relevant for successfully bridging the physical world and the digital domain using Mobile Augmented Reality. A step towards this we have successfully ported the ARToolkit to consumer mobile phones running on the Symbian platform and present results around this. We also present three sample applications based on UMAR and future case study work planned.
---
paper_title: Data Management Strategies for Mobile Augmented Reality

paper_content:

Any significant real-world application of mobile augmented reality will require a large model of location-bound data. While it may appear that a natural approach is to develop application-specific data formats and management strategies, we have found that such an approach actually prevents reuse of the data and ultimately produces additional complexity in developing the application. In contrast we describe a three-tier architecture to manage a common data model for a set of applications. It is inspired by current Internet application frameworks and consists of a central storage layer using a common data model, a transformation layer responsible for filtering and adapting the data to the requirements of a particular applications on request, and finally of the applications itself. We demonstrate our architecture in a scenario consisting of two multi-user capable mobile AR applications for collaborative navigation and annotation in a city environment.
---
paper_title: Streaming mobile augmented reality on mobile phones

paper_content:

Continuous recognition and tracking of objects in live video captured on a mobile device enables real-time user interaction. We demonstrate a streaming mobile augmented reality system with 1 second latency. User interest is automatically inferred from camera movements, so the user never has to press a button. Our system is used to identify and track book and CD covers in real time on a phone's viewfinder. Efficient motion estimation is performed at 30 frames per second on a phone, while fast search through a database of 20,000 images is performed on a server.
---
paper_title: Face to face collaborative AR on mobile phones

paper_content:

Mobile phones are an ideal platform for augmented reality. In this paper we describe how they also can be used to support face to face collaborative AR applications. We have created a custom port of the ARToolKit library to the Symbian mobile phone operating system and then developed a sample collaborative AR game based on this. We describe the game in detail and user feedback from people who have played it. We also provide general design guidelines that could be useful for others who are developing mobile phone collaborative AR applications.
---
paper_title: The need for real time consistency management in P2P mobile gaming environments

paper_content:

The introduction of more powerful, feature rich, portable handsets is enabling more engaging mobile multimedia entertainment. Improvements in wireless technology infrastructure are enabling access to ubiquitous, always-on data networks. We believe that based on the popularity of multiplayer games over IP networks, there will be significant demand for mobile multiplayer games. Based upon our early trials, we also believe that the current trend for using centralised client/server models will not offer the most suitable architecture to enable them. To support our research we have built a novel augmented reality real time game as a requirements gatherer for gaming in a wireless environment. In this paper we indicate why a centralised approach will not be suitable for real-time interactions. As an alternative we propose a decentralised approach which includes support for consistency and interest management over heterogeneous networked wireless environments.
---
paper_title: Map torchlight: a mobile augmented reality camera projector unit

paper_content:

The advantages of paper-based maps have been utilized in the field of mobile Augmented Reality (AR) in the last few years. Traditional paper-based maps provide high-resolution, large-scale information with zero power consumption. There are numerous implementations of magic lens interfaces that combine high-resolution paper maps with dynamic handheld displays. From an HCI perspective, the main challenge of magic lens interfaces is that users have to switch their attention between the magic lens and the information in the background. In this paper, we attempt to overcome this problem by using a lightweight mobile camera projector unit to augment the paper map directly with additional information. The "Map Torchlight" is tracked over a paper map and can precisely highlight points of interest, streets, and areas to give directions or other guidance for interacting with the map.
---
paper_title: Mobile phone based AR scene assembly

paper_content:

In this paper we describe a mobile phone based Augmented Reality application for 3D scene assembly. Augmented Reality on mobile phones extends the interaction capabilities on such handheld devices. It adds a 6 DOF isomorphic interaction technique for manipulating 3D content. We give details of an application that we believe to be the first where 3D content can be manipulated using both the movement of a camera tracked mobile phone and a traditional button interface as input for transformations. By centering the scene in a tangible marker space in front of the phone we provide a mean for bimanual interaction. We describe the implementation, the interaction techniques we have developed and initial user response to trying the application.
---
paper_title: Collecting big datasets of human activity one checkin at a time

paper_content:

A variety of cutting edge applications for mobile phones exploit the availability of phone sensors to accurately infer the user activity and location to offer more effective services. To validate and evaluate these new applications, appropriate and extensive datasets are needed: in particular, large sets of traces of sensor data (accelerometer, GPS, micro- phone, etc.), labelled with corresponding user activities. So far, such traces have only been collected in short-lived, small-scale setups. The primary reason for this is the difficulty in establishing accurate ground truth information outside the laboratory. Here, we present our vision of a system for large-scale sensor data capturing, leveraging all sensors of todays smart phones, with the aim of generating a large dataset that is augmented with appropriate ground-truth information. The primary challenges that we address consider the energy cost on the mobile device and the incentives for users to keep running the system on their device for longer. We argue for leveraging the concept of the checkin - as successfully introduced in online social networks (e.g. Foursquare) - for collecting activity and context related datasets. With a checkin, a user deliberately provides a small piece of data about their behaviour while enabling the system to adjust sensing and data collection around important activities. In this work we present up2, a mobile app letting users check in to their current activity (e.g., "waiting for the bus", "riding a bicycle", "having dinner"). After a checkin, we use the phone's sensors (GPS, accelerometer, microphone, etc.) to gather data about the user's activity and surrounding. This makes up2 a valuable tool for research in sensor based activity detection.
---
paper_title: Data Management Strategies for Mobile Augmented Reality

paper_content:

Any significant real-world application of mobile augmented reality will require a large model of location-bound data. While it may appear that a natural approach is to develop application-specific data formats and management strategies, we have found that such an approach actually prevents reuse of the data and ultimately produces additional complexity in developing the application. In contrast we describe a three-tier architecture to manage a common data model for a set of applications. It is inspired by current Internet application frameworks and consists of a central storage layer using a common data model, a transformation layer responsible for filtering and adapting the data to the requirements of a particular applications on request, and finally of the applications itself. We demonstrate our architecture in a scenario consisting of two multi-user capable mobile AR applications for collaborative navigation and annotation in a city environment.
---
paper_title: Managing Complex Augmented Reality Models

paper_content:

Mobile augmented reality requires georeferenced data to present world-registered overlays. To cover a wide area and all artifacts and activities, a database containing this information must be created, stored, maintained, delivered, and finally used by the client application. We present a data model and a family of techniques to address these needs.
---
paper_title: Data Management Strategies for Mobile Augmented Reality

paper_content:

Any significant real-world application of mobile augmented reality will require a large model of location-bound data. While it may appear that a natural approach is to develop application-specific data formats and management strategies, we have found that such an approach actually prevents reuse of the data and ultimately produces additional complexity in developing the application. In contrast we describe a three-tier architecture to manage a common data model for a set of applications. It is inspired by current Internet application frameworks and consists of a central storage layer using a common data model, a transformation layer responsible for filtering and adapting the data to the requirements of a particular applications on request, and finally of the applications itself. We demonstrate our architecture in a scenario consisting of two multi-user capable mobile AR applications for collaborative navigation and annotation in a city environment.
---
paper_title: Managing Complex Augmented Reality Models

paper_content:

Mobile augmented reality requires georeferenced data to present world-registered overlays. To cover a wide area and all artifacts and activities, a database containing this information must be created, stored, maintained, delivered, and finally used by the client application. We present a data model and a family of techniques to address these needs.
---
paper_title: Data Management Strategies for Mobile Augmented Reality

paper_content:

Any significant real-world application of mobile augmented reality will require a large model of location-bound data. While it may appear that a natural approach is to develop application-specific data formats and management strategies, we have found that such an approach actually prevents reuse of the data and ultimately produces additional complexity in developing the application. In contrast we describe a three-tier architecture to manage a common data model for a set of applications. It is inspired by current Internet application frameworks and consists of a central storage layer using a common data model, a transformation layer responsible for filtering and adapting the data to the requirements of a particular applications on request, and finally of the applications itself. We demonstrate our architecture in a scenario consisting of two multi-user capable mobile AR applications for collaborative navigation and annotation in a city environment.
---
paper_title: Managing Complex Augmented Reality Models

paper_content:

Mobile augmented reality requires georeferenced data to present world-registered overlays. To cover a wide area and all artifacts and activities, a database containing this information must be created, stored, maintained, delivered, and finally used by the client application. We present a data model and a family of techniques to address these needs.
---
paper_title: Location based Applications for Mobile Augmented Reality

paper_content:

In this work we investigate building indoor location based applications for a mobile augmented reality system. We believe that augmented reality is a natural interface to visualize spacial information such as position or direction of locations and objects for location based applications that process and present information based on the user's position in the real world. To enable such applications we construct an indoor tracking system that covers a substantial part of a building. It is based on visual tracking of fiducial markers enhanced with an inertial sensor for fast rotational updates. To scale such a system to a whole building we introduce a space partitioning scheme to reuse fiducial markers throughout the environment. Finally we demonstrate two location based applications built upon this facility, an indoor navigation aid and a library search application.
---
paper_title: User Interface Management Techniques for Collaborative Mobile Augmented Reality

paper_content:

Mobile Augmented Reality Systems (MARS) have the potential to revolutionize the way in which information is provided to users. Virtual information can be directly integrated with the real world surrounding the mobile user, who can interact with it to display related information, to pose and resolve queries, and to collaborate with other users. However, we believe that the benefits of MARS will only be achieved if the user interface (UI) is actively managed so as to maximize the relevance and minimize the confusion of the virtual material relative to the real world. This article addresses some of the steps involved in this process, focusing on the design and layout of the mobile user’s overlaid virtual environment. The augmented view of the user’s surroundings presents an interface to context-dependent operations, many of which are related to the objects in view—the augmented world is the user interface. We present three user interface design techniques that are intended to make this interface as obvious and clear to the user as possible: information filtering, UI component design, and view management. Information filtering helps select the most relevant information to present to the user. UI component designdetermines the format in which this information should be conveyed, based on the available display resources and tracking accuracy. For example, the absence of high accuracy position tracking would favor body- or screenstabilized components over world-stabilized ones that would need to be exactly registered with the physical objects to which they refer. View management attempts to ensure that the virtual objects that are displayed visually are arranged appropriately with regard to their projections on the view plane. For example, the relationships among objects should be as unambiguous as possible, and physical or virtual objects should not obstruct the user’s view of more important physical or virtual objects in the scene. We illustrate these interface design techniques using our prototype collaborative, cross-site MARS environment, which is composed of mobile and non-mobile augmented reality and virtual reality systems.
---
paper_title: Muddleware for Prototyping Mixed Reality Multiuser Games

paper_content:

We present Muddleware, a communication platform designed for mixed reality multi-user games for mobile, lightweight clients. An approach inspired by Tuplespaces, which provides decoupling of sender and receiver is used to address the requirements of a potentially large number of mobile clients. A hierarchical database built on XML technology allows convenient prototyping and simple, yet powerful queries. Server side-extensions address persistence and autonomous behaviors through hierarchical state machines. The architecture has been tested with a number of multi-user games and is also used for non-entertainment applications
---
paper_title: An adaptive training-free feature tracker for mobile phones

paper_content:

While tracking technologies based on fiducial markers have dominated the development of Augmented Reality (AR) applications for almost a decade, various real-time capable approaches to markerless tracking have recently been presented. However, most existing approaches do not yet achieve sufficient frame rates for AR on mobile phones or at least require an extensive training phase in advance. In this paper we will present our approach on feature based tracking applying robust SURF features. The implementation is more than one magnitude faster than previous ones, allowing running even on mobile phones at highly interactive rates. In contrast to other feature based approaches on mobile phones, our implementation may immediately track features captured from a photo without any training. Further, the approach is not restricted to planar surfaces, but may use features of 3D objects.
---
paper_title: 3D mobile augmented reality in urban scenes

paper_content:

In this paper, we present a large-scale mobile augmented reality system that recognizes the buildings in the mobile device's live video and registers this live view with the 3-dimensional models of the buildings. Having the camera pose estimated and tracked, the system adds relevant information about the buildings to the video in the correct perspective. We demonstrate the system on a large database of geo-tagged panoramic images of an urban environment with associated 3-dimensional planar models. The system uses the capabilities of emerging mobile platforms such as location and orientation sensors, and computational power to detect, track, and augment buildings in urban scenes.
---
paper_title: Design and optimization of image processing algorithms on mobile GPU

paper_content:

The advent of GPUs with programmable shaders on mobile phones has motivated developers to utilize GPU to offload computationally intensive tasks and relive the burden of embedded CPU. In this paper, we present a set of metrics to measure characteristics of a mobile phone GPU with the focus on image processing algorithms. These measures assist users in design and implementation stage and in classifying bottlenecks. We propose techniques to achieve increased performance with optimized shader design. To show the effectiveness of the proposed techniques, we employ cartoon-style non-photorealistic rendering (NPR), belief propagation (BP) stereo matching [Yang et al. 2006], and speeded up robust features (SURF) detection [Bay et al. 2008] as our example algorithms.
---
paper_title: OpenCL embedded profile prototype in mobile device

paper_content:

Programmable Graphics Processing Unit (GPU) has over the years become an integral part of today's computing systems. The GPU use-cases have gradually been extended from graphics towards a wide range of applications. Since the programmable GPU is now making its way to mobile devices, it is interesting to study these new use-cases also there. To test this, we created a programming environment based on the embedded profile of the fresh Khronos OpenCL standard and ran it against an image processing workload in a mobile device with CPU and GPU back-ends. The early results on performance and energy consumption with CPU+GPU configuration were promising but also suggest there is room for optimization.
---
paper_title: CPU and GPU parallel processing for mobile Augmented Reality

paper_content:

This paper introduces a parallel processing scheme using CPU and GPU for the mobile Augmented Reality applications. Most of AR applications have to perform intensive image processing algorithms to detect specified objects on which virtual information is overlaid. The object detection generally consists of a feature extraction module and a feature description module. The proposed scheme distributes the feature extraction module and the feature description module to CPU and GPU respectively, and performs the modules in parallel. In experimental results, the proposed scheme outperforms the CPU only scheme and the sequential CPU-GPU execution scheme.
---
paper_title: Implementation and optimization of image processing algorithms on handheld GPU

paper_content:

The advent of GPUs with programmable shaders on handheld devices has motivated embedded application developers to utilize GPU to offload computationally intensive tasks and relieve the burden from embedded CPU. In this work, we propose an image processing toolkit on handheld GPU with programmable shaders using OpenGL ES 2.0 API. By using the image processing toolkit, we show that a range of image processing algorithms map readily to handheld GPU. We employ real-time video scaling, cartoon-style non-photorealistic rendering, and Harris corner detector as our example applications. In addition, we propose techniques to achieve increased performance with optimized shader design and efficient sharing of GPU workload between vertex and fragment shaders. Performance is evaluated in terms of frames per second at varying video stream resolution.
---
paper_title: Implementation of an augmented reality system on a PDA

paper_content:

We present a client/server implementation for running demanding mobile AR application on a PDA device. The system incorporates various data compression methods to make it run as fast as possible on a wide range of communication networks, from GSM to WLAN.
---
paper_title: Augmented assembly using a mobile phone

paper_content:

We present a mobile phone based augmented reality (AR) assembly system that enable users to view complex models on their mobile phones. It is based on a client-server architecture, where complex model information is located on a PC, and a mobile phone with the camera is used as a thin client access device to this information. With this system users are able to see an AR view that provides step by step guidance for a real world assembly task. We also present results from a pilot user study evaluating the system, showing that people felt the interface was intuitive and very helpful in supporting the assembly task.
---
paper_title: Video see-through AR on consumer cell-phones

paper_content:

We present a first running video see-through augmented reality system on a consumer cell-phone. It supports the detection and differentiation of different markers, and correct integration of rendered 3D graphics into the live video stream via a weak perspective projection camera model and an OpenGL rendering pipeline.
---
paper_title: Extraction of Natural Feature Descriptors on Mobile GPUs

paper_content:

In this thesis the feasibility of a GPGPU (general-purpose computing on graphics processing units) approach to natural feature description on mobile phone GPUs is assessed. To this end, the SURF descriptor [4] has been implemented with OpenGL ES 2.0/GLSL ES 1.0 and evaluated across different mobile devices. The implementation is multiple times faster than a comparable CPU variant on the same device. The results proof the feasibility of modern mobile graphics accelerators for GPGPU tasks especially for the detection phase in natural feature tracking used in augmented reality applications. Extensive analysis and benchmarking of this approach in comparison to state of the art methods have been undertaken. Insights into the modifications necessary to adapt and modify the SURF algorithm to the limitations of a mobile GPU are presented. Further, an outlook for a GPGPU-based tracking pipeline on a mobile device is provided.
---
paper_title: Using mobile GPU for general-purpose computing – a case study of face recognition on smartphones

paper_content:

As GPU becomes an integrated component in handheld devices like smartphones, we have been investigating the opportunities and limitations of utilizing the ultra-low-power GPU in a mobile platform as a general-purpose accelerator, similar to its role in desktop and server platforms. The special focus of our investigation has been on mobile GPU's role for energy-optimized real-time applications running on battery-powered handheld devices. In this work, we use face recognition as an application driver for our study. Our implementations on a smartphone reveals that, utilizing the mobile GPU as a co-processor can achieve significant speedup in performance as well as substantial reduction in total energy consumption, in comparison with a mobile-CPU-only implementation on the same platform.
---
paper_title: Boosting mobile GPU performance with a decoupled access/execute fragment processor

paper_content:

Smartphones represent one of the fastest growing markets, providing significant hardware/software improvements every few months. However, supporting these capabilities reduces the operating time per battery charge. The CPU/GPU component is only left with a shrinking fraction of the power budget, since most of the energy is consumed by the screen and the antenna. In this paper, we focus on improving the energy efficiency of the GPU since graphical applications consist an important part of the existing market. Moreover, the trend towards better screens will inevitably lead to a higher demand for improved graphics rendering. We show that the main bottleneck for these applications is the texture cache and that traditional techniques for hiding memory latency (prefetching, multithreading) do not work well or come at a high energy cost. We thus propose the migration of GPU designs towards the decoupled access-execute concept. Furthermore, we significantly reduce bandwidth usage in the decoupled architecture by exploiting inter-core data sharing. Using commercial Android applications, we show that the end design can achieve 93% of the performance of a heavily multithreaded GPU while providing energy savings of 34%.
---
paper_title: Extraction of Natural Feature Descriptors on Mobile GPUs

paper_content:

In this thesis the feasibility of a GPGPU (general-purpose computing on graphics processing units) approach to natural feature description on mobile phone GPUs is assessed. To this end, the SURF descriptor [4] has been implemented with OpenGL ES 2.0/GLSL ES 1.0 and evaluated across different mobile devices. The implementation is multiple times faster than a comparable CPU variant on the same device. The results proof the feasibility of modern mobile graphics accelerators for GPGPU tasks especially for the detection phase in natural feature tracking used in augmented reality applications. Extensive analysis and benchmarking of this approach in comparison to state of the art methods have been undertaken. Insights into the modifications necessary to adapt and modify the SURF algorithm to the limitations of a mobile GPU are presented. Further, an outlook for a GPGPU-based tracking pipeline on a mobile device is provided.
---
paper_title: ARToolKitPlus for Pose Tracking on Mobile Devices

paper_content:

In this paper we present ARToolKitPlus, a successor to the popular ARToolKit pose tracking library. ARToolKitPlus has been optimized and extended for the usage on mobile devices such as smartphones, PDAs and Ultra Mobile PCs (UMPCs). We explain the need and specific requirements of pose tracking on mobile devices and how we met those requirements. To prove the applicability we performed an extensive benchmark series on a broad range of off-the-shelf handhelds.
---
paper_title: Design and optimization of image processing algorithms on mobile GPU

paper_content:

The advent of GPUs with programmable shaders on mobile phones has motivated developers to utilize GPU to offload computationally intensive tasks and relive the burden of embedded CPU. In this paper, we present a set of metrics to measure characteristics of a mobile phone GPU with the focus on image processing algorithms. These measures assist users in design and implementation stage and in classifying bottlenecks. We propose techniques to achieve increased performance with optimized shader design. To show the effectiveness of the proposed techniques, we employ cartoon-style non-photorealistic rendering (NPR), belief propagation (BP) stereo matching [Yang et al. 2006], and speeded up robust features (SURF) detection [Bay et al. 2008] as our example algorithms.
---
paper_title: The State of the Art in Mobile Graphics Research

paper_content:

High-quality computer graphics let mobile-device users access more compelling content. Still, the devices' limitations and requirements differ substantially from those of a PC. This survey of mobile graphics research describes current solutions in terms of specialized hardware (including 3D displays), rendering and transmission, visualization, and user interfaces.
---
paper_title: iPACKMAN: high-quality, low-complexity texture compression for mobile phones

paper_content:

We present a novel texture compression scheme, called iPACKMAN, targeted for hardware implementation. In terms of image quality, it outperforms the previous de facto standard texture compression algorithms in the majority of all cases that we have tested. Our new algorithm is an extension of the PACKMAN texture compression system, and while it is a bit more complex than PACKMAN, it is still very low in terms of hardware complexity.
---
paper_title: Graphics Processing Units for Handhelds

paper_content:

During the past few years, mobile phones and other handheld devices have gone from only handling dull text-based menu systems to, on an increasing number of models, being able to render high-quality three-dimensional graphics at high frame rates. This paper is a survey of the special considerations that must be taken when designing graphics processing units (GPUs) on such devices. Starting off by introducing desktop GPUs as a reference, the paper discusses how mobile GPUs are designed, often with power consumption rather than performance as the primary goal. Lowering the bus traffic between the GPU and the memory is an efficient way of reducing power consumption, and therefore some high-level algorithms for bandwidth reduction are presented. In addition, an overview of the different APIs that are used in the handheld market to handle both two-dimensional and three-dimensional graphics is provided. Finally, we present our outlook for the future and discuss directions of future research on handheld GPUs.
---
paper_title: Graphics for the masses: a hardware rasterization architecture for mobile phones

paper_content:

The mobile phone is one of the most widespread devices with rendering capabilities. Those capabilities have been very limited because the resources on such devices are extremely scarce; small amounts of memory, little bandwidth, little chip area dedicated for special purposes, and limited power consumption. The small display resolutions present a further challenge; the angle subtended by a pixel is relatively large, and therefore reasonably high quality rendering is needed to generate high fidelity images.To increase the mobile rendering capabilities, we propose a new hardware architecture for rasterizing textured triangles. Our architecture focuses on saving memory bandwidth, since an external memory access typically is one of the most energy-consuming operations, and because mobile phones need to use as little power as possible. Therefore, our system includes three new key innovations: I) an inexpensive multisampling scheme that gives relatively high quality at the same cost of previous inexpensive schemes, II) a texture minification system, including texture compression, which gives quality relatively close to trilinear mipmapping at the cost of 1.33 32-bit memory accesses on average, III) a scanline-based culling scheme that avoids a significant amount of z-buffer reads, and that only requires one context. Software simulations show that these three innovations together significantly reduce the memory bandwidth, and thus also the power consumption.
---
paper_title: Pixel-planes 5: a heterogeneous multiprocessor graphics system using processor-enhanced memories

paper_content:

This paper introduces the architecture and initial algorithms for Pixel-Planes 5, a heterogeneous multi-computer designed both for high-speed polygon and sphere rendering (1M Phong-shaded triangles/second) and for supporting algorithm and application research in interactive 3D graphics. Techniques are described for volume rendering at multiple frames per second, font generation directly from conic spline descriptions, and rapid calculation of radiosity form-factors. The hardware consists of up to 32 math-oriented processors, up to 16 rendering units, and a conventional 1280 t 1024-pixel frame buffer, interconnected by a 5 gigabit ring network. Each rendering unit consists of a 128 t 128-pixel array of processors-with-memory with parallel quadratic expression evaluation for every pixel. Implemented on 1.6 micron CMOS chips designed to run at 40MHz, this array has 208 bits/pixel on-chip and is connected to a video RAM memory system that provides 4,096 bits of off-chip memory. Rendering units can be independently reasigned to any part of the screen or to non-screen-oriented computation. As of April 1989, both hardware and software are still under construction, with initial system operation scheduled for fall 1989.
---
paper_title: Flexible point-based rendering on mobile devices

paper_content:

We have seen the growing deployment of ubiquitous computing devices and the proliferation of complex virtual environments. As demand for detailed and high-quality geometric models increases, typical scene size (often including scanned 3D objects) easily reaches millions of geometric primitives. Traditionally, vertices and polygons (faces) represent 3D objects. These representations, coupled with the traditional rendering pipeline, don't adequately support display of complex scenes on different types of platforms with heterogeneous rendering capabilities. To accommodate these constraints, we use a packed hierarchical point-based representation for rendering. Point-based rendering offers a simple-to-use level-of-detail mechanism in which we can adapt the number of points rendered to the underlying object's screen size. Our work strives for flexible rendering - that is, rendering only the interior hierarchy nodes as representatives of the subtree. In particular, we avoid traversal of the entire hierarchy and reconstruction of model attributes (such as normals and color information) for interior nodes because both operations can be prohibitively expensive. Flexible rendering also lets us traverse the hierarchy in a specific order, resulting in a fast, one-pass shadow-mapping algorithm.
---
paper_title: Retargeting vector animation for small displays

paper_content:

We present a method that preserves the recognizability of key object interactions in a vector animation. The method allows an artist to author an animation once, and then output it to any display device. We specifically target mobile devices with small screen sizes. In order to adapt an animation, the author specifies an importance value for objects in the animation. The algorithm then identifies and categorizes the vector graphics objects that comprise the animation, leveraging the implicit relationship between extensible Markup Language (XML) and scalable vector graphics (SVG). Based on importance, the animation can then be automatically retargeted for any display using artistically motivated resizing and grouping algorithms that budget size and spatial detail for each object.
---
paper_title: PCU: the programmable culling unit

paper_content:

Culling techniques have always been a central part of computer graphics, but graphics hardware still lack efficient and flexible support for culling. To improve the situation, we introduce the programmable culling unit, which is as flexible as the fragment program unit and capable of quickly culling entire blocks of fragments. Furthermore, it is very easy for the developer to use the PCU as culling programs can be automatically derived from fragment programs containing a discard instruction. Our PCU can be integrated into an existing fragment program unit with a modest hardware overhead of only about 10%. Using the PCU, we have observed shader speedups between 1.4 and 2.1 for relevant scenes.
---
paper_title: Enhancing 3D Graphics on Mobile Devices by Image-Based Rendering

paper_content:

Compared to a personal computer, mobile devices typically have weaker processing power, less memory capacity, and lower resolution of display. While the former two factors are clearly disadvantages for 3D graphics applications running on mobile devices, the display factor could be turned into an advantage instead. However the traditional 3D graphics pipeline cannot take advantage of the smaller display because its run time depends mostly on the number of polygons to be rendered. In contrast, the run time of image-based rendering methods depends mainly on the display resolution. Therefore it is well suited for mobile devices. Furthermore, we may use the network connection to build a client-server framework, which allows us to integrate with nonimage-based rendering programs. We present our system framework and the experiment results on PocketPC® based devices in this work.
---
paper_title: Streaming mobile augmented reality on mobile phones

paper_content:

Continuous recognition and tracking of objects in live video captured on a mobile device enables real-time user interaction. We demonstrate a streaming mobile augmented reality system with 1 second latency. User interest is automatically inferred from camera movements, so the user never has to press a button. Our system is used to identify and track book and CD covers in real time on a phone's viewfinder. Efficient motion estimation is performed at 30 frames per second on a phone, while fast search through a database of 20,000 images is performed on a server.
---
paper_title: Recent Trends of Mobile Collaborative Augmented Reality Systems

paper_content:

The use of mobile collaborative AR has expended rapidly in recent years, due to the major advances in hardware and networking. The application areas are diverse and multidisciplinary. Recent Trends of Mobile Collaborative Augmented Reality Systems provides a historical overview of previous mobile collaborative AR systems, presents case studies of latest developments in current mobile collaborative AR systems, and latest technologies and system architectures used in this field. Recent Trends of Mobile Collaborative Augmented Reality Systems is designed for a professional audience composed of practitioners and researchers working in the field of augmented reality and human-computer interaction. Advanced-level students in computer science and electrical engineering focused on this topic will also find this book useful as a secondary text or reference.
---
paper_title: A Mobile Vision System for Urban Detection with Informative Local Descriptors

paper_content:

We present a computer vision system for the detection and identification of urban objects from mobile phone imagery, e.g., for the application of tourist information services. Recognition is based on MAP decision making over weak object hypotheses from local descriptor responses in the mobile imagery. We present an improvement over the standard SIFT key detector [7] by selecting only informative (i-SIFT) keys for descriptor matching. Selection is applied first to reduce the complexity of the object model and second to accelerate detection by selective filtering. We present results on the MPG-20 mobile phone imagery with severe illumination, scale and viewpoint changes in the images, performing with ≈ 98% accuracy in identification, efficient (100%) background rejection, efficient (0%) false alarm rate, and reliable quality of service under extreme illumination conditions, significantly improving standard SIFT based recognition in every sense, providing - important for mobile vision - runtimes which are ≈ 8 (≈24) times faster for the MPG-20 (ZuBuD) database.
---
paper_title: Parametric analysis for adaptive computation offloading

paper_content:

Many programs can be invoked under different execution options, input parameters and data files. Such different execution contexts may lead to strikingly different execution instances. The optimal code generation may be sensitive to the execution instances. In this paper, we show how to use parametric program analysis to deal with this issue for the optimization problem of computation offloading.Computation offloading has been shown to be an effective way to improve performance and energy saving on mobile devices. Optimal program partitioning for computation offloading depends on the tradeoff between the computation workload and the communication cost. The computation workload and communication requirement may change with different execution instances. Optimal decisions on program partitioning must be made at run time when sufficient information about workload and communication requirement becomes available.Our cost analysis obtains program computation workload and communication cost expressed as functions of run-time parameters, and our parametric partitioning algorithm finds the optimal program partitioning corresponding to different ranges of run-time parameters. At run time, the transformed program self-schedules its tasks on either the mobile device or the server, based on the optimal program partitioning that corresponds to the current values of run-time parameters. Experimental results on an HP IPAQ handheld device show that different run-time parameters can lead to quite different program partitioning decisions.
---
paper_title: A Streaming-Based Solution for Remote Visualization of 3D Graphics on Mobile Devices

paper_content:

Mobile devices such as personal digital assistants, tablet PCs, and cellular phones have greatly enhanced user capability to connect to remote resources. Although a large set of applications is now available bridging the gap between desktop and mobile devices, visualization of complex 3D models is still a task hard to accomplish without specialized hardware. This paper proposes a system where a cluster of PCs, equipped with accelerated graphics cards managed by the Chromium software, is able to handle remote visualization sessions based on MPEG video streaming involving complex 3D models. The proposed framework allows mobile devices such as smart phones, personal digital assistants (PDAs), and tablet PCs to visualize objects consisting of millions of textured polygons and voxels at a frame rate of 30 fps or more depending on hardware resources at the server side and on multimedia capabilities at the client side. The server is able to concurrently manage multiple clients computing a video stream for each one; resolution and quality of each stream is tailored according to screen resolution and bandwidth of the client. The paper investigates in depth issues related to latency time, bit rate and quality of the generated stream, screen resolutions, as well as frames per second displayed
---
paper_title: Virtualized Screen: A Third Element for Cloud–Mobile Convergence

paper_content:

Mobile and cloud computing have emerged as the new computing platforms and are converging into a powerful cloud-mobile computing platform. This article envisions a virtualized screen as a new dimension in such a platform to further optimize the overall computing experience for users. In a virtualized screen, screen rendering is done in the cloud, and delivered as images to the client for interactive display. This enables thin-client mobile devices to enjoy many computationally intensive and graphically rich services. Technical challenges are discussed and addressed. Two novel cloud-mobile applications, Cloud Browser and Cloud Phone, are presented to demonstrate the advantages of such a virtualized screen.
---
paper_title: Using bandwidth data to make computation offloading decisions

paper_content:

We present a framework for making computation offloading decisions in computational grid settings in which schedulers determine when to move parts of a computation to more capable resources to improve performance. Such schedulers must predict when an offloaded computation will outperform one that is local by forecasting the local cost (execution time for computing locally) and remote cost (execution time for computing remotely and transmission time for the input/output of the computation to/from the remote system). Typically, this decision amounts to predicting the bandwidth between the local and remote systems to estimate these costs. Our framework unifies such decision models by formulating the problem as a statistical decision problem that can either be treated "classically" or using a Bayesian approach. Using an implementation of this framework, we evaluate the efficacy of a number of different decision strategies (several of which have been employed by previous systems). Our results indicate that a Bayesian approach employing automatic change-point detection when estimating the prior distribution is the best-performing approach.
---
paper_title: From Augmented Reality to Augmented Computing: A Look at Cloud-Mobile Convergence

paper_content:

There has been considerable number of virtual and augmented reality applications designed and developed for mobile devices. However the state-of-the-art systems are commonly confined by several limitations. In this position paper the concept ”Cloud-Mobile Convergence for Virtual Reality (CMCVR)” is presented. CMCVR envisions effective and user-friendly integration of the mobile device and cloud-based resources. Through the proposed framework, mobile devices could be augmented to deliver some user experiences comparable to those offered by fixed systems. A preliminary research that follows the CMCVR paradigm is also described.
---
paper_title: Implementation of an augmented reality system on a PDA

paper_content:

We present a client/server implementation for running demanding mobile AR application on a PDA device. The system incorporates various data compression methods to make it run as fast as possible on a wide range of communication networks, from GSM to WLAN.
---
paper_title: An accelerated remote graphics architecture for PDAS

paper_content:

A new category of devices, known as Personal Digital Assistant (PDA), has become increasingly widespread since the end of the nineties. A large number of software applications have been developed for PDAs, but high-quality 3D graphics still remain beyond the computational capability of these devices.This paper tackles this issue by proposing a generic solution for hardware-accelerated remote rendering on cluster. The rendering task is submitted to a PC/workstation cluster (each cluster machine is equipped by a graphics accelerator) by means of the Chromium architecture. Each machine renders a part of the image that is then reassembled and sent to the PDA via a software bridge. On the PDA side, the user can explore the scene using an ad-hoc navigation interface.The proposed solution allows to display extremely realistic and complex models in an interactive way. Moreover, our architecture does not depend on commercial solutions/products and can be easily modified in order to better fulfill requirements of specific applications.Computer GraphicsDistributed/network graphics Distributed SystemsDistributed applications
---
paper_title: Approximate Computing: A Survey

paper_content:

As one of the most promising energy-efficient computing paradigms, approximate computing has gained a lot of research attention in the past few years. This paper presents a survey of state-of-the-art work in all aspects of approximate computing and highlights future research challenges in this field.
---
paper_title: Approximate computing: An emerging paradigm for energy-efficient design

paper_content:

Approximate computing has recently emerged as a promising approach to energy-efficient design of digital systems. Approximate computing relies on the ability of many systems and applications to tolerate some loss of quality or optimality in the computed result. By relaxing the need for fully precise or completely deterministic operations, approximate computing techniques allow substantially improved energy efficiency. This paper reviews recent progress in the area, including design of approximate arithmetic blocks, pertinent error and quality measures, and algorithm-level techniques for approximate computing.
---
paper_title: SNNAP: Approximate computing on programmable SoCs via neural acceleration

paper_content:

Many applications that can take advantage of accelerators are amenable to approximate execution. Past work has shown that neural acceleration is a viable way to accelerate approximate code. In light of the growing availability of on-chip field-programmable gate arrays (FPGAs), this paper explores neural acceleration on off-the-shelf programmable SoCs. We describe the design and implementation of SNNAP, a flexible FPGA-based neural accelerator for approximate programs. SNNAP is designed to work with a compiler workflow that configures the neural network's topology and weights instead of the programmable logic of the FPGA itself. This approach enables effective use of neural acceleration in commercially available devices and accelerates different applications without costly FPGA reconfigurations. No hardware expertise is required to accelerate software with SNNAP, so the effort required can be substantially lower than custom hardware design for an FPGA fabric and possibly even lower than current “C-to-gates” high-level synthesis (HLS) tools. Our measurements on a Xilinx Zynq FPGA show that SNNAP yields a geometric mean of 3.8× speedup (as high as 38.1×) and 2.8× energy savings (as high as 28 x) with less than 10% quality loss across all applications but one. We also compare SNNAP with designs generated by commercial HLS tools and show that SNNAP has similar performance overall, with better resource-normalized throughput on 4 out of 7 benchmarks.
---
paper_title: Analysis and characterization of inherent application resilience for approximate computing

paper_content:

Approximate computing is an emerging design paradigm that enables highly efficient hardware and software implementations by exploiting the inherent resilience of applications to in-exactness in their computations. Previous work in this area has demonstrated the potential for significant energy and performance improvements, but largely consists of ad hoc techniques that have been applied to a small number of applications. Taking approximate computing closer to mainstream adoption requires (i) a deeper understanding of inherent application resilience across a broader range of applications (ii) tools that can quantitatively establish the inherent resilience of an application, and (iii) methods to quickly assess the potential of various approximate computing techniques for a given application. We make two key contributions in this direction. Our primary contribution is the analysis and characterization of inherent application resilience present in a suite of 12 widely used applications from the domains of recognition, data mining, and search. Based on this analysis, we present several new insights into the nature of resilience and its relationship to various key application characteristics. To facilitate our analysis, we propose a systematic framework for Application Resilience Characterization (ARC) that (a) partitions an application into resilient and sensitive parts and (b) characterizes the resilient parts using approximation models that abstract a wide range of approximate computing techniques. We believe that the key insights that we present can help shape further research in the area of approximate computing, while automatic resilience characterization frameworks such as ARC can greatly aid designers in the adoption approximate computing.
---
paper_title: Exploiting Significance of Computations for Energy-Constrained Approximate Computing

paper_content:

Approximate execution is a viable technique for environments with energy constraints, provided that applications are given the mechanisms to produce outputs of the highest possible quality within the available energy budget. This paper introduces a framework for energy-constrained execution with controlled and graceful quality loss. A simple programming model allows developers to structure the computation in different tasks, and to express the relative importance of these tasks for the quality of the end result. For non-significant tasks, the developer can also supply less costly, approximate versions. The target energy consumption for a given execution is specified when the application is launched. A significance-aware runtime system employs an application-specific analytical energy model to decide how many cores to use for the execution, the operating frequency for these cores, as well as the degree of task approximation, so as to maximize the quality of the output while meeting the user-specified energy constraints. Evaluation on a dual-socket 16-core Intel platform using 9 kernels and applications shows that the proposed framework performs very close to an oracle always selecting the optimal configuration, both in terms of energy efficiency and quality of results. Also, a comparison with loop perforation (a well-known compile-time approximation technique), shows that the proposed framework results in significantly higher quality for the same energy budget.
---
paper_title: The State of the Art in Mobile Graphics Research

paper_content:

High-quality computer graphics let mobile-device users access more compelling content. Still, the devices' limitations and requirements differ substantially from those of a PC. This survey of mobile graphics research describes current solutions in terms of specialized hardware (including 3D displays), rendering and transmission, visualization, and user interfaces.
---
paper_title: Performing computation offloading on multiple platforms

paper_content:

An offloading framework designed for supporting multiple platforms is proposed.The solution supports Android and Windows Phone mobile applications.Developers can use static or dynamic offloading decision.The offloading technique improves the performance of mobile applications.The type of serialization used impacts the offloading performance. Mobile devices such as smart phones and tablets are increasingly important tools in daily routine. These devices generally interact with more powerful machines usually hosted on public clouds. In this context, this paper presents MpOS (Multiplatform Offloading System), a framework that supports a method-based offloading technique for applications of different mobile platforms (Android and Windows Phone). In addition, details of MpOS main components and services as well as code examples are presented. To evaluate the proposed solution and to analyse the impact of different serialization types on the offloading performance, we developed two applications and performed several experiments on both Android and Windows Phone platforms using WiFi and 4G/LTE connections to access the remote execution environments. Our results state that offloading to a cloudlet has provided the best performance for both Android and Windows Phone platforms, beyond showing that the type of serialization used by the framework directly impacts on the offloading performance.
---
paper_title: Boosting mobile GPU performance with a decoupled access/execute fragment processor

paper_content:

Smartphones represent one of the fastest growing markets, providing significant hardware/software improvements every few months. However, supporting these capabilities reduces the operating time per battery charge. The CPU/GPU component is only left with a shrinking fraction of the power budget, since most of the energy is consumed by the screen and the antenna. In this paper, we focus on improving the energy efficiency of the GPU since graphical applications consist an important part of the existing market. Moreover, the trend towards better screens will inevitably lead to a higher demand for improved graphics rendering. We show that the main bottleneck for these applications is the texture cache and that traditional techniques for hiding memory latency (prefetching, multithreading) do not work well or come at a high energy cost. We thus propose the migration of GPU designs towards the decoupled access-execute concept. Furthermore, we significantly reduce bandwidth usage in the decoupled architecture by exploiting inter-core data sharing. Using commercial Android applications, we show that the end design can achieve 93% of the performance of a heavily multithreaded GPU while providing energy savings of 34%.
---
paper_title: ThinkAir: Dynamic resource allocation and parallel execution in the cloud for mobile code offloading

paper_content:

Smartphones have exploded in popularity in recent years, becoming ever more sophisticated and capable. As a result, developers worldwide are building increasingly complex applications that require ever increasing amounts of computational power and energy. In this paper we propose ThinkAir, a framework that makes it simple for developers to migrate their smartphone applications to the cloud. ThinkAir exploits the concept of smartphone virtualization in the cloud and provides method-level computation offloading. Advancing on previous work, it focuses on the elasticity and scalability of the cloud and enhances the power of mobile cloud computing by parallelizing method execution using multiple virtual machine (VM) images. We implement ThinkAir and evaluate it with a range of benchmarks starting from simple micro-benchmarks to more complex applications. First, we show that the execution time and energy consumption decrease two orders of magnitude for a N-queens puzzle application and one order of magnitude for a face detection and a virus scan application. We then show that a parallelizable application can invoke multiple VMs to execute in the cloud in a seamless and on-demand manner such as to achieve greater reduction on execution time and energy consumption. We finally use a memory-hungry image combiner tool to demonstrate that applications can dynamically request VMs with more computational power in order to meet their computational requirements.
---
paper_title: Offloading in Mobile Cloudlet Systems with Intermittent Connectivity

paper_content:

The emergence of mobile cloud computing enables mobile users to offload applications to nearby mobile resource-rich devices (i.e., cloudlets) to reduce energy consumption and improve performance. However, due to mobility and cloudlet capacity, the connections between a mobile user and mobile cloudlets can be intermittent. As a result, offloading actions taken by the mobile user may fail (e.g., the user moves out of communication range of cloudlets). In this paper, we develop an optimal offloading algorithm for the mobile user in such an intermittently connected cloudlet system, considering the users’ local load and availability of cloudlets. We examine users’ mobility patterns and cloudlets’ admission control, and derive the probability of successful offloading actions analytically. We formulate and solve a Markov decision process (MDP) model to obtain an optimal policy for the mobile user with the objective to minimize the computation and offloading costs. Furthermore, we prove that the optimal policy of the MDP has a threshold structure. Subsequently, we introduce a fast algorithm for energy-constrained users to make offloading decisions. The numerical results show that the analytical form of the successful offloading probability is a good estimation in various mobility cases. Furthermore, the proposed MDP offloading algorithm for mobile users outperforms conventional baseline schemes.
---
paper_title: Graphics Processing Units for Handhelds

paper_content:

During the past few years, mobile phones and other handheld devices have gone from only handling dull text-based menu systems to, on an increasing number of models, being able to render high-quality three-dimensional graphics at high frame rates. This paper is a survey of the special considerations that must be taken when designing graphics processing units (GPUs) on such devices. Starting off by introducing desktop GPUs as a reference, the paper discusses how mobile GPUs are designed, often with power consumption rather than performance as the primary goal. Lowering the bus traffic between the GPU and the memory is an efficient way of reducing power consumption, and therefore some high-level algorithms for bandwidth reduction are presented. In addition, an overview of the different APIs that are used in the handheld market to handle both two-dimensional and three-dimensional graphics is provided. Finally, we present our outlook for the future and discuss directions of future research on handheld GPUs.
---
paper_title: Signature-based workload estimation for mobile 3D graphics

paper_content:

Until recently, most 3D graphics applications had been regarded as too computationally intensive for devices other than desktop computers and gaming consoles. This notion is rapidly changing due to improving screen resolutions and computing capabilities of mass-market handheld devices such as cellular phones and PDAs. As the mobile 3D gaming industry is poised to expand, significant innovations are required to provide users with high-quality 3D experience under limited processing, memory and energy budgets that are characteristic of the mobile domain. Energy saving schemes such as dynamic voltage and frequency scaling (DVFS), as well as system-level power and performance optimization methods for mobile devices require accurate and fast workload prediction. In this paper, we address the problem of workload prediction for mobile 3D graphics. We propose and describe a signature-based estimation technique for predicting 3D graphics workloads. By analyzing a gaming benchmark, we show that monitoring specific parameters of the 3D pipeline provides better prediction accuracy over conventional approaches. We describe how signatures capture such parameters concisely to make accurate workload predictions. Signature-based prediction is computationally efficient because first, signatures are compact, and second, they do not require elaborate model evaluations. Thus, they are amenable to efficient, real-time prediction. A fundamental difference between signatures and standard history-based predictors is that signatures capture previous outcomes as well as the cause that led to the outcome, and use both to predict future outcomes. We illustrate the utility of signature-based workload estimation technique by using it as a basis for DVFS in 3D graphics pipelines.
---
paper_title: Low-power 3D graphics processors for mobile terminals

paper_content:

A full 3D graphics pipeline is investigated, and optimizations of graphics architecture are assessed for satisfying the performance requirements and overcoming the limited system resources found in mobile terminals. Two mobile 3D graphics processor architectures, RAMP and DigiAcc, are proposed based on the analysis, and a prototype development platform (REMY) is implemented. REMY includes a software graphics library and simulation environment developed for more flexible realization of mobile 3D graphics. The experimental results demonstrate the feasibility of mobile 3D graphics with 3.6 Mpolygons/s at 155 mW power consumption for full 3D operation.
---
paper_title: OPENRP: a reputation middleware for opportunistic crowd computing

paper_content:

The concepts of wisdom of crowd and collective intelligence have been utilized by mobile application developers to achieve large-scale distributed computation, known as crowd computing. The profitability of this method heavily depends on users' social interactions and their willingness to share resources. Thus, different crowd computing applications need to adopt mechanisms that motivate peers to collaborate and defray the costs of participating ones who share their resources. In this article, we propose OPENRP, a novel, lightweight, and scalable system middleware that provides a unified interface to crowd computing and opportunistic networking applications. When an application wants to perform a device-to-device task, it delegates the task to the middleware, which takes care of choosing the best peers with whom to collaborate and sending the task to these peers. OPENRP evaluates and updates the reputation of participating peers based on their mutual opportunistic interactions. To show the benefits of the middleware, we simulated the behavior of two representative crowdsourcing applications: message forwarding and task offloading. Through extensive simulations on real human mobility traces, we show that the traffic generated by the applications is lower compared to two benchmark strategies. As a consequence, we show that when using our middleware, the energy consumed by the nodes is reduced. Finally, we show that when dividing the nodes into selfish and altruistic, the reputation scores of the altruistic peers increase with time, while those of the selfish ones decrease.
---
paper_title: Energy Management Techniques in Modern Mobile Handsets

paper_content:

Managing energy efficiently is paramount in modern smartphones. The diverse range of wireless interfaces and sensors, and the increasing popularity of power-hungry applications that take advantage of these resources can reduce the battery life of mobile handhelds to few hours of operation. The research community, and operating system and hardware vendors found interesting optimisations and techniques to extend the battery life of mobile phones. However, the state of the art of lithium-ion batteries clearly indicates that energy efficiency must be achieved both at the hardware and software level. In this survey, we will cover the software solutions that can be found in the research literature between 1999 and May 2011 at six different levels: energy-aware operating systems, efficient resource management, the impact of users' interaction patterns with mobile devices and applications, wireless interfaces and sensors management, and finally the benefits of integrating mobile devices with cloud computing services.
---
paper_title: Comet: Code offload by migrating execution transparently

paper_content:

In this paper we introduce a runtime system to allow unmodified multi-threaded applications to use multiple machines. The system allows threads to migrate freely between machines depending on the workload. Our prototype, COMET (Code Offload by Migrating Execution Transparently), is a realization of this design built on top of the Dalvik Virtual Machine. COMET leverages the underlying memory model of our runtime to implement distributed shared memory (DSM) with as few interactions between machines as possible. Making use of a new VM-synchronization primitive, COMET imposes little restriction on when migration can occur. Additionally, enough information is maintained so one machine may resume computation after a network failure. ::: ::: We target our efforts towards augmenting smartphones or tablets with machines available in the network. We demonstrate the effectiveness of COMET on several real applications available on Google Play. These applications include image editors, turn-based games, a trip planner, and math tools. Utilizing a server-class machine, COMET can offer significant speed-ups on these real applications when run on a modern smartphone. With WiFi and 3G networks, we observe geometric mean speed-ups of 2.88× and 1.27× relative to the Dalvik interpreter across the set of applications with speed-ups as high as 15× on some applications.
---
paper_title: PCU: the programmable culling unit

paper_content:

Culling techniques have always been a central part of computer graphics, but graphics hardware still lack efficient and flexible support for culling. To improve the situation, we introduce the programmable culling unit, which is as flexible as the fragment program unit and capable of quickly culling entire blocks of fragments. Furthermore, it is very easy for the developer to use the PCU as culling programs can be automatically derived from fragment programs containing a discard instruction. Our PCU can be integrated into an existing fragment program unit with a modest hardware overhead of only about 10%. Using the PCU, we have observed shader speedups between 1.4 and 2.1 for relevant scenes.
---
paper_title: The State of the Art in Mobile Graphics Research

paper_content:

High-quality computer graphics let mobile-device users access more compelling content. Still, the devices' limitations and requirements differ substantially from those of a PC. This survey of mobile graphics research describes current solutions in terms of specialized hardware (including 3D displays), rendering and transmission, visualization, and user interfaces.
---
paper_title: Using mobile GPU for general-purpose computing – a case study of face recognition on smartphones

paper_content:

As GPU becomes an integrated component in handheld devices like smartphones, we have been investigating the opportunities and limitations of utilizing the ultra-low-power GPU in a mobile platform as a general-purpose accelerator, similar to its role in desktop and server platforms. The special focus of our investigation has been on mobile GPU's role for energy-optimized real-time applications running on battery-powered handheld devices. In this work, we use face recognition as an application driver for our study. Our implementations on a smartphone reveals that, utilizing the mobile GPU as a co-processor can achieve significant speedup in performance as well as substantial reduction in total energy consumption, in comparison with a mobile-CPU-only implementation on the same platform.
---
paper_title: An adaptive training-free feature tracker for mobile phones

paper_content:

While tracking technologies based on fiducial markers have dominated the development of Augmented Reality (AR) applications for almost a decade, various real-time capable approaches to markerless tracking have recently been presented. However, most existing approaches do not yet achieve sufficient frame rates for AR on mobile phones or at least require an extensive training phase in advance. In this paper we will present our approach on feature based tracking applying robust SURF features. The implementation is more than one magnitude faster than previous ones, allowing running even on mobile phones at highly interactive rates. In contrast to other feature based approaches on mobile phones, our implementation may immediately track features captured from a photo without any training. Further, the approach is not restricted to planar surfaces, but may use features of 3D objects.
---
paper_title: Graphics Processing Units for Handhelds

paper_content:

During the past few years, mobile phones and other handheld devices have gone from only handling dull text-based menu systems to, on an increasing number of models, being able to render high-quality three-dimensional graphics at high frame rates. This paper is a survey of the special considerations that must be taken when designing graphics processing units (GPUs) on such devices. Starting off by introducing desktop GPUs as a reference, the paper discusses how mobile GPUs are designed, often with power consumption rather than performance as the primary goal. Lowering the bus traffic between the GPU and the memory is an efficient way of reducing power consumption, and therefore some high-level algorithms for bandwidth reduction are presented. In addition, an overview of the different APIs that are used in the handheld market to handle both two-dimensional and three-dimensional graphics is provided. Finally, we present our outlook for the future and discuss directions of future research on handheld GPUs.
---
paper_title: Extraction of Natural Feature Descriptors on Mobile GPUs

paper_content:

In this thesis the feasibility of a GPGPU (general-purpose computing on graphics processing units) approach to natural feature description on mobile phone GPUs is assessed. To this end, the SURF descriptor [4] has been implemented with OpenGL ES 2.0/GLSL ES 1.0 and evaluated across different mobile devices. The implementation is multiple times faster than a comparable CPU variant on the same device. The results proof the feasibility of modern mobile graphics accelerators for GPGPU tasks especially for the detection phase in natural feature tracking used in augmented reality applications. Extensive analysis and benchmarking of this approach in comparison to state of the art methods have been undertaken. Insights into the modifications necessary to adapt and modify the SURF algorithm to the limitations of a mobile GPU are presented. Further, an outlook for a GPGPU-based tracking pipeline on a mobile device is provided.
---
paper_title: Recent Advances in Augmented Reality

paper_content:

In 1997, Azuma published a survey on augmented reality (AR). Our goal is to complement, rather than replace, the original survey by presenting representative examples of the new advances. We refer one to the original survey for descriptions of potential applications (such as medical visualization, maintenance and repair of complex equipment, annotation, and path planning); summaries of AR system characteristics (such as the advantages and disadvantages of optical and video approaches to blending virtual and real, problems in display focus and contrast, and system portability); and an introduction to the crucial problem of registration, including sources of registration error and error-reduction strategies.
---
paper_title: Next Generation 5G Wireless Networks: A Comprehensive Survey

paper_content:

The vision of next generation 5G wireless communications lies in providing very high data rates (typically of Gbps order), extremely low latency, manifold increase in base station capacity, and significant improvement in users’ perceived quality of service (QoS), compared to current 4G LTE networks. Ever increasing proliferation of smart devices, introduction of new emerging multimedia applications, together with an exponential rise in wireless data (multimedia) demand and usage is already creating a significant burden on existing cellular networks. 5G wireless systems, with improved data rates, capacity, latency, and QoS are expected to be the panacea of most of the current cellular networks’ problems. In this survey, we make an exhaustive review of wireless evolution toward 5G networks. We first discuss the new architectural changes associated with the radio access network (RAN) design, including air interfaces, smart antennas, cloud and heterogeneous RAN. Subsequently, we make an in-depth survey of underlying novel mm-wave physical layer technologies, encompassing new channel model estimation, directional antenna design, beamforming algorithms, and massive MIMO technologies. Next, the details of MAC layer protocols and multiplexing schemes needed to efficiently support this new physical layer are discussed. We also look into the killer applications, considered as the major driving force behind 5G. In order to understand the improved user experience, we provide highlights of new QoS, QoE, and SON features associated with the 5G evolution. For alleviating the increased network energy consumption and operating expenditure, we make a detail review on energy awareness and cost efficiency. As understanding the current status of 5G implementation is important for its eventual commercialization, we also discuss relevant field trials, drive tests, and simulation experiments. Finally, we point out major existing research issues and identify possible future research directions.
---
paper_title: Cardea: Context-Aware Visual Privacy Protection from Pervasive Cameras

paper_content:

The growing popularity of mobile and wearable devices with built-in cameras, the bright prospect of camera related applications such as augmented reality and life-logging system, the increased ease of taking and sharing photos, and advances in computer vision techniques have greatly facilitated people's lives in many aspects, but have also inevitably raised people's concerns about visual privacy at the same time. Motivated by recent user studies that people's privacy concerns are dependent on the context, in this paper, we propose Cardea, a context-aware and interactive visual privacy protection framework that enforces privacy protection according to people's privacy preferences. The framework provides people with fine-grained visual privacy protection using: i) personal privacy profiles, with which people can define their context-dependent privacy preferences; and ii) visual indicators: face features, for devices to automatically locate individuals who request privacy protection; and iii) hand gestures, for people to flexibly interact with cameras to temporarily change their privacy preferences. We design and implement the framework consisting of the client app on Android devices and the cloud server. Our evaluation results confirm this framework is practical and effective with 86% overall accuracy, showing promising future for context-aware visual privacy protection from pervasive cameras.
---
paper_title: ReadMe: A Real-Time Recommendation System for Mobile Augmented Reality Ecosystems

paper_content:

We introduce ReadMe, a real-time recommendation system (RS) and an online algorithm for Mobile Augmented Reality (MAR) ecosystems. A MAR ecosystem is the one that contains mobile users and virtual objects. The role of ReadMe is to detect and present the most suitable virtual objects on the mobile user's screen. The selection of the proper virtual objects depends on the mobile users' context. We consider the user's context as a set of variables that can be either drawn directly by user's device or can be inferred by it or can be collected in collaboration with other mobile devices.
---
paper_title: When Augmented Reality meets Big Data

paper_content:

With computing and sensing woven into the fabric of everyday life, we live in an era where we are awash in a flood of data from which we can gain rich insights. Augmented reality (AR) is able to collect and help analyze the growing torrent of data about user engagement metrics within our personal mobile and wearable devices. This enables us to blend information from our senses and the digitalized world in a myriad of ways that was not possible before. AR and big data have a logical maturity that inevitably converge them. The tread of harnessing AR and big data to breed new interesting applications is starting to have a tangible presence. In this paper, we explore the potential to capture value from the marriage between AR and big data technologies, following with several challenges that must be addressed to fully realize this potential.
---
paper_title: Demo: Interactive Visual Privacy Control with Gestures

paper_content:

Built-in cameras of mobile and wearable devices enable a variety of applications such as augmented reality, continuous sensing, and life-logging systems, which bring joy and convenience to human lives. However, being recorded by unauthorized or unnoticed cameras have raised people's concerns about visual privacy. To address this problem, we propose a novel interactive method to control visual privacy. We allow individuals to interact with cameras using static tags and more flexible hand gestures for privacy control. By delivering privacy control messages via visual indicators, devices will automatically perform control operations according to privacy indicators detected and control rules.
---

Title: Mobile Augmented Reality Survey: From Where We Are to Where We Go

Section 1: INTRODUCTION
Description 1: Introduce the concept of Mobile Augmented Reality (MAR), its historical development, and significance in various fields, along with the scope of this survey.

Section 2: APPLICATION FIELDS
Description 2: Discuss the various application domains of MAR, including Tourism and Navigation, Entertainment and Advertisement, Training and Education, Geometry Modeling and Scene Construction, Assembly and Maintenance, and Information Assistant Management.

Section 3: TOURISM AND NAVIGATION
Description 3: Provide examples and discussions of MAR applications in tourism and navigation, detailing specific projects and their implementations.

Section 4: ENTERTAINMENT AND ADVERTISEMENT
Description 4: Explore the use of MAR in entertainment and advertisement, including games and promotional activities.

Section 5: TRAINING AND EDUCATION
Description 5: Assess the impact of MAR on training and education, with specific examples and studies showing how MAR enhances learning experiences.

Section 6: GEOMETRY MODELING AND SCENE CONSTRUCTION
Description 6: Examine the role of MAR in geometry modeling and scene construction, detailing case studies and methodologies employed.

Section 7: ASSEMBLY AND MAINTENANCE
Description 7: Discuss MAR applications in assembly and maintenance, including studies that show its effectiveness in these areas.

Section 8: INFORMATION ASSISTANT MANAGEMENT
Description 8: Show how MAR is used in information assistant management, providing details on system implementations and use cases.

Section 9: BIG DATA DRIVEN MAR
Description 9: Explore how big data integration enhances MAR applications across various fields like Retail, Tourism, Healthcare, and Public Services.

Section 10: REPRESENTATIVE MAR SYSTEMS
Description 10: Present and analyze representative MAR systems, highlighting their components, functionalities, and enabling technologies.

Section 11: UI/UX
Description 11: Examine user interface (UI) and user experience (UX) considerations in MAR, discussing design principles and challenges.

Section 12: SYSTEM COMPONENTS
Description 12: Detail the components of MAR systems, including mobile computing platforms, software frameworks, and display technologies.

Section 13: TRACKING AND REGISTRATION
Description 13: Discuss tracking and registration methods in MAR, dividing them into sensor-based and vision-based methods, and exploring their applications.

Section 14: NETWORK CONNECTIVITY
Description 14: Analyze the role of network connectivity in MAR, detailing the types of networks used and their impact on performance.

Section 15: DATA MANAGEMENT
Description 15: Discuss data management strategies in MAR, focusing on data acquisition, modeling, and storage.

Section 16: SYSTEM PERFORMANCE AND SUSTAINABILITY
Description 16: Address runtime performance and energy efficiency in MAR applications, presenting various strategies and technological advancements.

Section 17: CHALLENGING PROBLEMS
Description 17: Identify and discuss the main challenges in MAR development, including energy efficiency, low-level MAR libraries, killer MAR applications, networking, data management, and UI/UX concerns.

Section 18: SECURITY AND PRIVACY
Description 18: Discuss security and privacy issues related to MAR applications and propose guidelines to address these concerns.

Section 19: SOCIAL ACCEPTANCE
Description 19: Address the social acceptance of MAR technologies, discussing factors such as device intrusion, privacy, and safety considerations.

Section 20: CONCLUSION
Description 20: Summarize the survey's findings, highlight future directions, and discuss the potential of MAR to transform user interactions with the real world.