Haptic Feedback for Microrobotics Applications: A Review
---
paper_title: Methods for haptic feedback in teleoperated robot-assisted surgery

paper_content:

Teleoperated minimally invasive surgical robots can significantly enhance a surgeon's accuracy, dexterity and visualization. However, current commercially available systems do not include significant haptic (force and tactile) feedback to the operator. This paper describes experiments to characterize this problem, as well as several methods to provide haptic feedback in order to improve surgeon's performance. There exist a variety of sensing and control methods that enable haptic feedback, although a number of practical considerations, e.g. cost, complexity and biocompatibility, present significant challenges. The ability of teleoperated robot-assisted surgical systems to measure and display haptic information leads to a number of additional exciting clinical and scientific opportunities, such as active operator assistance through "virtual fixtures" and the automatic acquisition of tissue properties.
---
paper_title: Cutaneous haptic feedback to ensure the stability of robotic teleoperation systems

paper_content:

Cutaneous haptic feedback can be used to enhance the performance of robotic teleoperation systems while guaranteeing their safety. Delivering ungrounded cutaneous cues to the human operator conveys in fact information about the forces exerted at the slave side and does not affect the stability of the control loop. In this work we analyze the feasibility, effectiveness, and implications of providing solely cutaneous feedback in robotic teleoperation. We carried out two peg-in-hole experiments, both in a virtual environment and in a real teleoperated environment. Two novel 3-degree-of-freedom fingertip cutaneous displays deliver a suitable amount of cutaneous feedback at the thumb and index fingers. Results assessed the feasibility and effectiveness of the proposed approach. Cutaneous feedback was outperformed by full haptic feedback provided by grounded haptic interfaces, but it outperformed conditions providing no force feedback at all. Moreover, cutaneous feedback always kept the system stable, even in the presence of destabilizing factors such as communication delays and hard contacts.
---
paper_title: Controlled manipulation of multiple cells using catalytic microbots

paper_content:

Self-propelled microjet engines (microbots) can transport multiple cells into specific locations in a fluid. The motion is externally controlled by a magnetic field which allows to selectively load, transport and deliver the cells.
---
paper_title: Teleoperator Performance with Varying Force and Visual Feedback

paper_content:

An experimental study was conducted to determine the effects of various forms of visual and force feedback on human performance for several "peg-in-hole" -type telemanipulation tasks. Each of six human test subjects used a master/slave manipulator during two experimental sessions. In one session the subjects performed the tasks with direct vision, where sub tended visual angle, force feedback, task difficulty, and the interaction of subtended visual angle and force feedback made significant differences in task completion times. During the other session the tasks were performed using a video monitor for visual feedback, and video frame rate, force feedback, task difficulty, and the interaction of frame rate and force feedback were found to make significant differences in task times. An analysis between the direct and video viewing environments showed that apart from subtended visual angle and reduced frame rate, the video medium itself did not significantly affect task times relative to direct viewing.
---
paper_title: Micromachine-enabled capture and isolation of cancer cells in complex media.

paper_content:

Circulating tumor cells (CTCs) are the primary entities responsible for spawning cancer metastasis. Detection of CTCs provides an indicator for the clinical diagnosis and prognosis of various types of cancers. Several approaches, based primarily on flowing the sample through antibody-coated magnetic-beads[1] or microchip[2,3] surfaces have been described for isolating and counting CTCs. However, these approaches require extensive sample preparation and/or complex surface microstructures to detect the extremely low abundance of CTCs in blood.[3,4] In this study we describe a immunomicromachine-based approach for an in-vitro isolation of cancer cells that holds promise for direct CTC detection without sample pre-processing.
---
paper_title: Magnetic Control of Tubular Catalytic Microbots for the Transport, Assembly, and Delivery of Micro-objects

paper_content:

Recently a significant amount of attention has been paid towards the development of man-made synthetic catalytic micro- and nanomotors that can mimic biological counterparts in terms of propulsion power, motion control, and speed. However, only a few applications of such self propelled vehicles have been described. Here the magnetic control of self-propelled catalytic Ti/ Fe/Pt rolled-up microtubes (microbots) that can be used to perform various tasks such as the selective loading, transportation, and delivery of microscale objects in a fluid is shown; for instance, it is demonstrated for polystyrene particles and thin metallic films ("nanoplates"). Microbots self propel by ejecting microbubbles via a platinum catalytic decomposition of hydrogen peroxide into oxygen and water. The fuel and surfactant concentrations are optimized obtaining a maximum speed of 275 μm s -1 (5.5 body lengths per second) at 15% of peroxide fuel. The microbots exert a force of around 3.77 pN when transporting a single 5 μm diameter particle; evidencing a high propulsion power that allows for the transport of up to 60 microparticles. By the introduction of an Fe thin film into the rolled-up microtubes, their motion can be fully controlled by an external magnetic field.
---
paper_title: Cutaneous Feedback of Fingertip Deformation and Vibration for Palpation in Robotic Surgery

paper_content:

Despite its expected clinical benefits, current teleoperated surgical robots do not provide the surgeon with haptic feedback largely because grounded forces can destabilize the system's closed-loop controller. This paper presents an alternative approach that enables the surgeon to feel fingertip contact deformations and vibrations while guaranteeing the teleoperator's stability. We implemented our cutaneous feedback solution on an Intuitive Surgical da Vinci Standard robot by mounting a SynTouch BioTac tactile sensor to the distal end of a surgical instrument and a custom cutaneous display to the corresponding master controller. As the user probes the remote environment, the contact deformations, dc pressure, and ac pressure (vibrations) sensed by the BioTac are directly mapped to input commands for the cutaneous device's motors using a model-free algorithm based on look-up tables. The cutaneous display continually moves, tilts, and vibrates a flat plate at the operator's fingertip to optimally reproduce the tactile sensations experienced by the BioTac. We tested the proposed approach by having eighteen subjects use the augmented da Vinci robot to palpate a heart model with no haptic feedback, only deformation feedback, and deformation plus vibration feedback. Fingertip deformation feedback significantly improved palpation performance by reducing the task completion time, the pressure exerted on the heart model, and the subject's absolute error in detecting the orientation of the embedded plastic stick. Vibration feedback significantly improved palpation performance only for the seven subjects who dragged the BioTac across the model, rather than pressing straight into it.
---
paper_title: Semi-active guiding systems in surgery. A two-dof prototype of the passive arm with dynamic constraints (PADyC)

paper_content:

Abstract Guiding systems for surgical applications range from passive systems to active ones depending on the level of autonomy left to the surgeon. Passive systems provide the surgeon with suitable information to compare the executed strategy with a planned one whilst active systems autonomously perform a part of the surgical strategy. Depending on the selected technology, guiding systems may be more or less accurate, more or less safe, more or less user-friendly. Actuated robots are generally very accurate but raise safety and ergonomics issues; passive systems (optical localizers, motorless encoded arms, etc.) are well-suited to navigation but their use is difficult for executing complex surgical strategies. A new type of mechanical guiding system has been designed, based on the use of dynamic constraints; this passive arm is actuated by the surgeon who keeps, therefore, fully involved in the execution of the surgical strategy. At each instant, the motions proposed by the operator are “filtered” with respect to the task, before being transmitted to the arm. This system seems a good answer to the guided execution of potentially complex strategies in an accurate and ergonomical way. The principles of the PADyC arm are presented and experiments already done with a two-DOF prototype are described.
---
paper_title: The hands-on orthopaedic robot "acrobot": Early clinical trials of total knee replacement surgery

paper_content:

A "hands-on" robotic system for total knee replacement (TKR) surgery is presented. A computer tomography-based preoperative planning software is used to accurately plan the procedure. Intraoperatively, the surgeon guides a small special-purpose robot, called Acrobot, which is mounted on a gross positioning device. The Acrobot uses active constraint control, which constrains the motion to a predefined region, and thus allows the surgeon to safely cut the knee bones to fit a TKR prosthesis with high precision. A noninvasive anatomical registration method is described. The system has been successfully used in seven clinical trials with encouraging results.
---
paper_title: A Review of Haptic Feedback Teleoperation Systems for Micromanipulation and Microassembly

paper_content:

This paper presents a review of the major haptic feedback teleoperation systems for micromanipulation. During the last decade, the handling of micrometer-sized objects has become a critical issue. Fields of application from material science to electronics demonstrate an urgent need for intuitive and flexible manipulation systems able to deal with small-scale industrial projects and assembly tasks. Two main approaches have been considered: fully automated tasks and manual operation. The first one require fully pre determined tasks, while the later necessitates highly trained operators. To overcome these issues the use of haptic feedback teleoperation where the user manipulates the tool through a joystick whilst feeling a force feedback, appears to be a promising solution as it allows high intuitiveness and flexibility. Major advances have been achieved during this last decade, starting with systems that enable the operator to feel the substrate topology, to the current state-of-the-art where 3D haptic feedback is provided to aid manipulation tasks. This paper details the major achievements and the solutions that have been developed to propose 3D haptic feedback for tools that often lack 3D force measurements. The use of virtual reality to enhance the immersion is also addressed. The strategies developed provide haptic feedback teleoperation systems with a high degree of assistance and for a wide range of micromanipulation tools. Based on this expertise on haptic for micromanipulation and virtual reality assistance it is now possible to propose microassembly systems for objects as small as 1 to 10 micrometers. This is a mature field and will benefit small-scale industrial projects where precision and flexibility in microassembly are required.
---
paper_title: Catalytic Microtubular Jet Engines Self‐Propelled by Accumulated Gas Bubbles

paper_content:

Strain-engineered microtubes with an inner catalytic surface serve as self-propelled microjet engines with speeds of up to ≈2 mm s−1 (approximately 50 body lengths per second). The motion of the microjets is caused by gas bubbles ejecting from one opening of the tube, and the velocity can be well approximated by the product of the bubble radius and the bubble ejection frequency. Trajectories of various different geometries are well visualized by long microbubble tails. If a magnetic layer is integrated into the wall of the microjet engine, we can control and localize the trajectories by applying external rotating magnetic fields. Fluid (i.e., fuel) pumping through the microtubes is revealed and directly clarifies the working principle of the catalytic microjet engines.
---
paper_title: Cutaneous Haptic Feedback in Robotic Teleoperation

paper_content:

This work addresses the challenge of providing effective cutaneous haptic feedback in robotic teleoperation, with the objective of achieving the highest degree of transparency whilst guaranteeing the stability of the considered systems. On the one hand, it evaluates teleoperation systems that provide only cutaneous cues to the operator, thus guaranteeing the highest degree of safety. This cutaneous-only approach shows intermediate performance between no force feedback and full haptic feedback provided by a grounded haptic interface, and it is best suitable for those scenarios where the safety of the system is paramount, e.g., robotic surgery. On the other hand, in order to achieve a higher level of performance, this work also investigates novel robotic teleoperation systems with force reflection able to provide mixed cutaneous and kinesthetic cues to the operator. Cutaneous cues can compensate for the temporary reduction of kinesthetic feedback necessary to satisfy certain stability conditions. This state-of-the-art volume is oriented toward researchers, educators, and students who are interested in force feedback techniques for robotic teleoperation, cutaneous device design, cutaneous rendering methods and perception studies, as well as readers from different disciplines who are interested in applying cutaneous haptic technologies and methods to their field of interest.
---
paper_title: Towards a micropositioning system for targeted drug delivery in wireless capsule endoscopy

paper_content:

This paper describes a novel micropositioning mechanism for achieving 1ml of targeted drug delivery within wireless capsule endoscopes. The mechanism allows a needle to be positioned within a 22.5° segment of a cylindrical capsule and be extendible by up to 4mm. The mechanism achieves both these functions using only a single micromotor and occupying a volume of just 200mm3 (including micromotor), this represents only 6.6% of the total available space. Through a detailed stress analysis it has been shown that the proposed mechanism can be fabricated using FDA approved materials and requires a power budget of under 3.3% of the available capacity. It is envisaged this mechanism would empower a new breed of capsule microrobots for therapy in addition to diagnostics for pathologies such as ulcerative colitis and small intestinal Crohn's disease.
---
paper_title: A haptic tele-manipulation environment for a vibration-driven micromechatronic device

paper_content:

In this paper, a novel haptic tele-manipulation environment is presented. This includes an interface between a master haptic mechanism and a slave mechatronic mechanism for biomedical operations. The novelty stems from the fact that the environment's slave is a micromechatronic device driven by two inexpensive centripetal force vibration micromotors. The unique characteristics and challenges that arise during the haptic micromanipulation of the specific device are described and analyzed. The developed solutions are presented and discussed. The environment employs three input modes and two force control phases, which are described in detail. The haptic tele-manipulation environment is illustrated by several examples. These show that, while the interaction between the haptic mechanism and the vibration driven device is complicated, the micromanipulation of the device can be successful and appear to the operator as simple.
---
paper_title: 2D high speed force feedback teleoperation of optical tweezers

paper_content:

The optical trap is a powerful non-contact approach for manipulating micron sized objects. Teleoperation of optical tweezers can be performed by coupling with a haptic interface, which allows an efficient robotic device to control positions and get force feedback. This provides users direct and intuitive microscopic interactions. The major difficulty in order for haptic devices to generate a reliable tactile sensation lies in its high frequency requirement of more than 1 kHz. This paper presents a fast force feedback teleoperation system for optical tweezers that attains this high frequency. The used force sensor is a novel event-based camera that transmits output as a continuous stream of asynchronous temporal events thus enabling high speed event-based visual processing. This new sensor is compared to a conventional frame based one to show advantages of our setup. A complex task of exploiting three dimensional target surface is performed demonstrating the robustness and efficiency of the presented method. This is the first time microspheres are used to touch targets of arbitrary form and color, which may interest broad-reaching biological and physical applications.
---
paper_title: Dexterous teleoperation for micro parts handling based on haptic/visual interface

paper_content:

Operators suffer much difficulty in manipulating micro/nano-sized objects without the assistance of human interfaces, due to the scaling effects in the micro/nano world. The paper presents a dexterous micromanipulation system based on the teleoperation techniques which cope with flexible and complex microassembly tasks. An experimental workcell consisting of a micromanipulator operated under multiple stereo-microscopic vision with the help of a user interface provides a tool that can be used to visualize and manipulate micro-sized 3D objects in a controlled manner. The key features of a micromanipulation system and teleoperation scheme for handling micro objects based on a haptic/visual interface are described. Experimental results demonstrate the feasibility of this system in precisely controlling trapping and manipulation of micro objects based on teleoperation techniques.
---
paper_title: AFM-based micro force sensor and haptic interface for a nanohandling robot

paper_content:

Nanohandling robots are the result of increasing research activities at the border between microsystem technology and robotics. Today, robots with dimensions of a few cm/sup 3/ can be developed. Like conventional robots, microrobots represent a complex system that usually contains several different types of actuators and sensors. The measurement of gripping forces is the most important sensor application in micro-handling besides visual servoing to protect the parts from too high surface pressures and thereby damage the object during the assembly process. In this paper, suitable force sensing techniques are described, and a concept for the measurement of gripping forces in nanohandling robotics with AFM cantilever-based sensors is introduced. Furthermore, a haptic interface which allows the operator to feel and control the forces in the micro world is presented.
---
paper_title: Toward a tele-nanorobotic manipulation system with atomic scale force feedback and motion resolution

paper_content:

Steps toward the realization of a tele-nanorobotic manipulation system using a magnetically levitated hand controller coupled with a scanning tunneling microscope are discussed. With this system, it is possible to 'manually' probe surfaces at atomic scale, while 'feeling' the atomic-scale topography back in the operator's hand. Results for sputtered gold and graphite are presented. >
---
paper_title: Asynchronous Event-Based Visual Shape Tracking for Stable Haptic Feedback in Microrobotics

paper_content:

Micromanipulation systems have recently been receiving increased attention. Teleoperated or automated micromanipulation is a challenging task due to the need for high-frequency position or force feedback to guarantee stability. In addition, the integration of sensors within micromanipulation platforms is complex. Vision is a commonly used solution for sensing; unfortunately, the update rate of the frame-based acquisition process of current available cameras cannot ensure-at reasonable costs-stable automated or teleoperated control at the microscale level, where low inertia produces highly unreachable dynamic phenomena. This paper presents a novel vision-based microrobotic system combining both an asynchronous address event representation silicon retina and a conventional frame-based camera. Unlike frame-based cameras, recent artificial retinas transmit their outputs as a continuous stream of asynchronous temporal events in a manner similar to the output cells of a biological retina, enabling high update rates. This paper introduces an event-based iterative closest point algorithm to track a microgripper's position at a frequency of 4 kHz. The temporal precision of the asynchronous silicon retina is used to provide a haptic feedback to assist users during manipulation tasks, whereas the frame-based camera is used to retrieve the position of the object that must be manipulated. This paper presents the results of an experiment on teleoperating a sphere of diameter around 50 μm using a piezoelectric gripper in a pick-and-place task.
---
paper_title: Force sensing microinstrument for measuring tissue properties and pulse in microsurgery

paper_content:

Miniaturized and "smart" instruments capable of characterizing the mechanical properties of tiny biological tissues are needed for research in biology, physiology, and biomechanics, and can find very important clinical applications for diagnostics and minimally invasive surgery (MIS). We are developing a set of robotic microinstruments designed to augment the performance of surgeons and clinicians during MIS. These microtools are intended to restore (or even enhance) the finger palpation capabilities that the surgeon exploits to characterize tissue hardness and to measure pulsating vessels in traditional surgery, but which are substantially reduced in MIS. This paper describes the main applications and the performance of a prototype miniature robotic instrument consisting of a microfabricated microgripper, instrumented with semiconductor strain-gauges as force sensors. The experimental set-up used for the in vitro tests reported in this paper consists of the microprobe mounted on a workstation and teleoperated. A haptic interface provides force feedback to the operator. We have demonstrated that the system can discriminate, both qualitatively and quantitatively, tiny skin samples based on their different elastic properties, and "feel" microvessels on the basis of pulsating fluid flowing through them.
---
paper_title: A novel haptic platform for real time bilateral biomanipulation with a MEMS sensor for triaxial force feedback

paper_content:

Abstract A novel triaxial force sensing device developed by the authors with a MEMS sensor as core component is mounted on a subnanometric resolution nanomanipulator having three degrees of freedom (DOF). This sensorized device allows measuring forces on the nanomanipulator tip in the range of 0–3 N for normal and ±50 mN for tangential forces with a resolution of 11 bits. Together with a haptic input device, a setup was created allowing palpation and force feeling. The mathematical model used to drive the master haptic interface force feedback capabilities is based on online force and stiffness measurement. The performance of the novel setup is demonstrated with a cell palpation experiment.
---
paper_title: Delta Haptic Device as a nanomanipulator

paper_content:

At the EPFL, we have developed a force-feedback device and control architecture for high-end research and industrial applications. The Delta Haptic Device (DHD) consists of a 6 degrees-of-freedom (DOF) mecatronic device driven by a PC. Several experiments have been carried out in the fields of manipulation and simulation to assess the dramatic improvement haptic information brings to manipulation. This system is particularly well suited for scaled manipulation such as micro-, nano- and biomanipulation. Not only can it perform geometric and force scaling, but it can also include fairly complex physical models into the control loop to assist manipulation and enhance human understanding of the environment. To demonstrate this ability, we are currently interfacing our DHD with an atomic force microscope (AFM). In a first stage, we will be able to feel in real-time the topology of a given sample while visualizing it in 3D. The aim of the project is to make manipulation of carbon nanotubes possible by including physical models of such nanotubes behavior into the control loop, thus allowing humans to control complex structures. In this paper, we give a brief description of our device and present preliminary results of its interfacing with the AFM.
---
paper_title: A compact nano manipulator based on an atomic force microscope coupling with a scanning electron microscope or an inverted optical microscope

paper_content:

In this paper, we describe a novel nano manipulator based on an atomic force microscope (AFM). The body of the manipulator is enough compact to be operated inside the sample chamber of a scanning electron microscope (SEM). In order to realize the compact body, we employed a self-detection type cantilever for AFM observation. The cantilever includes strain resistance element, which can easily detect a deflection signal of the cantilever without other sensing devices such as optical lever systems. It is possible to observe the manipulation situation in the real time observation by using the SEM. The AFM manipulator is coupled with a haptic device for human interface. Thus, by using this system, the operator can move the AFM probe at any position on the surface with feeling the interaction force detected by the cantilever on the sample surface according to the cantilever deflection. As a performance of the system, biological samples were controllably manipulated under the SEM observation. Furthermore, in order to deal with biological samples in liquid condition, the manipulator can be coupled with an inverted optical microscope. By using the system, we successfully demonstrated manipulation of biological samples in liquid condition. Two AFM manipulators could be used for dissection of biological samples like a knife and fork.
---
paper_title: Bilateral Magnetic Micromanipulation Using Off-Board Force Sensor

paper_content:

This paper introduces a novel haptic-enabled magnetic micromanipulation platform with promising potential for extensive biological and biomedical applications. This platform consists of two separated basic sites: 1) the slave site that uses a controlled magnetic field for manipulating a ferromagnetic microdevice and 2) the master site that uses a haptic-enabled device for the position and the force communication between the human operator and the microdevice. Due to the size restriction of the microdevice, attaching force sensors to the microdevice is impractical. Thus, to preserve a high feeling of a microdomain environment for the human operator, the applied force/torque from the environment to the microdevice is estimated with a novel off-board force sensing mechanism. This force sensing mechanism uses the produced magnetic flux information and the real position of the microdevice to estimate the environmental force applied to the microdevice. A scaled force-position teleoperation scheme is employed for this haptic application to scale down the macrodomain position for microdomain application and scale up the microdomain force for macrodomain sensing of the human operator. Conducting several experiments in different conditions, precise motion tracking with high accurate force transfer to human operator has been reported, RMS of position tracking errors of 0.2 mm with 1.27- $\mu$ N accuracy force sensing for single-axis motion.
---
paper_title: Intuitive control of self-propelled microjets with haptic feedback

paper_content:

Self-propelled microrobots have recently shown promising results in several scenarios at the microscale, such as targeted drug delivery and micromanipulation of cells. However, none of the steering systems available in the literature enable humans to intuitively and effectively control these microrobots in the remote environment, which is a desirable feature. In this paper we present an innovative teleoperation system with force reflection that enables a human operator to intuitively control the positioning of a self-propelled microjet. A particle-filter-based visual tracking algorithm tracks at runtime the position of the microjet in the remote environment. A 6-degrees-of-freedom haptic interface then provides the human operator with compelling haptic feedback about the interaction between the controlled microjet and the environment, as well as enabling the operator to intuitively control the target position of the microjet. Finally, a wireless magnetic control system regulates the orientation of the microjet to reach the target point. The viability of the proposed approach is demonstrated through two experimentsz enrolling twenty-eight subjects. In both experiments providing haptic feedback significantly improved the performance and the perceived realism of the considered tasks.
---
paper_title: A teleoperation system for micro positioning with haptic feedback

paper_content:

This paper presents the research work on a 1 Degree of Freedom (DOF) macro-micro teleoperation system which enables human operator to perform complex task in micro environment such as cell insertion with the capability of haptic feedback. To reach submicron resolution, a nano-motion piezo actuator was used as the slave robot and a servo DC motor was used as the master robot. Force sensors were implemented at both ends for haptic feedback and a microscope equipped with camera was employed for real-time visual feedback. The hysteresis nonlinearity of the piezo motor was modeled using LuGre friction model and compensated for. A Sliding Mode Based Impedance Controller (SMBIC) was designed at the slave side to ensure position tracking while an impedance force controller was designed at the master side to ascertain tracking of the force. Control parameters were chosen based on Llewellyn stability criteria such that the entire system stays stable against parameter uncertainties and constant time delay. The experimental results demonstrated capability of the proposed control frameworks in desirable tracking of the position and force signals while the entire system remained stable. The results of this study can be used for complex tasks in micron environment such as cell insertion.
---
paper_title: Task-based and stable telenanomanipulation in a nanoscale virtual environment

paper_content:

In a haptic interface system with a nanoscale virtual environment (NVE) using an atomic force microscope, not only is stability important, but task-based performance (or fidelity) is crucial. In this paper, we introduce a nanoscale virtual coupling (NSVC) concept and explicitly derive the relationship between performance, stability, and scaling factors of velocity (or position) and force. An available scaling factor region is represented based on Llewellyn's absolute stability criteria and the physical limitation of the haptic device. For the stable haptic interface, the sampled time passivity controller is implemented in the NVE. Experiments have been performed for telenanomanipulation tasks, such as positioning, indenting, and nanolithography with guaranteed stability in the NVE. Note to Practitioners-This paper suggests methods and control schemes for the task-based and stable telenanomanipulation in the nanoscale virtual environment (NVE). The proposed task-based and stable telenanomanipulation in the NVE can be used for an augmented human machine interface for the manipulation of nanoscale objects with the atomic force microscope (AFM). In addition, it is beneficial for learning or performing nanoscale tasks, such as nanolithography, nanoindenting, nanofabrication, and cell manipulation. Also, the interaction with the NVE using haptic device provides a useful tool for researchers in a variety of disciplines, such as biology, chemistry, and physics. Moreover, it may even be applied to educational purposes. In future research, the developed stable haptic interface would be integrated with the AFM system as a slave manipulator for telenanomanipulation experiments, such as pushing a nanoparticle with precise positioning and nanoassembly.
---
paper_title: A new hand-held force-amplifying device for micromanipulation

paper_content:

This paper presents a new hand-held device capable of amplifying delicate micromanipulation forces during minimal invasive surgical tasks. It relays force sensing to the user through a simple sliding feature that is coupled to the surgical tool, which translates relative to the casing of the device held by the operator. This forgoes the need of grounding frames or anchoring mechanisms to the body, allowing the device to be used in general surgical environments without affecting the workflow. The device uses a three-phase linear motor that is compact and capable of generating high forces that allow amplification factors of up to ×15. It features a closed-loop force control scheme to perform the required force amplification in which the force exerted on to the user is measured, forming the feedback in the control loop. The device permits interchangeability of instrumentation through a simple docking feature, and thus can be generalized to a range of surgical instruments for micromanipulation tasks. Detailed bench tests and user trials have been performed to validate the accuracy and practical performance of the device. The results have shown a five times reduction of the minimum force threshold perceived by the subjects and ergonomically sound manipulation advantages.
---
paper_title: A Microassembly System for the Flexible Assembly of Hybrid Robotic Mems Devices

paper_content:

This paper presents a full 6 DOF microassembly system that features a novel remote center of motion configuration paired with advanced vision and illumination modules, as well as innovative user interaction concepts. Even though the semi-automatic design is primarily focused on the assembly of 3D bio-microrobotic devices out of individual 2.5D MEMS components, it can be configured for a large variety of assembly tasks. A gripper exchange mechanism allows reaching for parts with dimensions ranging from 5–800 µm and a micro-fabricated platform featuring a special pattern provides a structured working area. The assembly of a miniature bio-microrobot is presented to demonstrate the dexterity and powerful features of this system. The underlying microassembly station combines multiple concepts for another step towards full manipulation automation in industrial and research applications.
---
paper_title: Force-feedback micromanipulation with unconditionally stable coupling

paper_content:

This paper presents a remote handling force feedback coupling for micromanipulation systems. In the literature, the most generally used coupling mode is 'force position'. This kind of control scheme is not portable and instability is an often occurring problem. The coupling scheme proposed in this paper is based on the passivity considerations on the teleoperated systems. It is independent of the used haptic interface and the manipulator and unconditionally stable regarding scaling ratios. It is experimented using the LRP's (Laboratoire de Robotique de Paris) micromanipulator, which is based on AFM architecture and uses the adhesion forces for pickup and release tasks. A comparison between the force position coupling and proposed coupling is presented. Experimental results show the good performances in terms of stability.
---
paper_title: Human-assisted virtual reality for a magnetic-haptic micromanipulation platform

paper_content:

This paper deals with the development of a virtual reality interface (VRI) for a magnetic-haptic micromanipulation platform (MHMP) (Mehrtash et al. in IEEE/ASME Trans Mechatron 16(3):459---469, 2011). Our previously developed MHMP has shown a great deal of promise in non-contact micromanipulations. This micromanipulation platform concerns the integration of magnetic actuation technology and a bilateral macro---micro teleoperation. The MHMP has two separate stations: one magnetic microrobotic station and one haptic. The magnetic microrobotic station manipulates micro-sized objects based on the commands from the haptic station. The haptic station uses bilateral communication with the magnetic microrobotic station to allow a human operator the feeling of a micro-domain environment. In this paper, we report a VRI that enables human operators to improve their skills in using the MHMP, before carrying out an actual dexterous task. The VRI is made up of three main components: a haptic station, a simulation engine, and a display unit. The haptic station provides the operator with the force/torque information from virtual or remote environments, and is also used to recognize the operator's hand motion command. Dynamical computation and control system modeling have been carried out on the simulation engine. Based on the real-time computation, this engine, as the heart of the system, provides force applied to the operator's hand and the microrobot's position for the haptic station and the display unit, respectively. The display unit employs 3D computer graphics to demonstrate the micromanipulation tasks and environments. The VRI is also developed in such a way that it can be separately used in parallel with the MHMP for the 3D visualization of a real task by providing multiple virtual viewports. This paper introduces the configuration of the proposed VRI, and reports the result of a preliminary experiment using micromanipulation investigation for validation.
---
paper_title: Haptic controlled three degree-of-freedom microgripper system for assembly of detachable surface-micromachined MEMS

paper_content:

Abstract In this work, we describe the implementation of a three degree-of-freedom meso to microscale manipulation system for handling MEMS and micro-objects. A number of commercially available hardware components have been integrated to produce this system. Microgrippers, with an operational range of 0–100 μm, were used as the end-effector for the manipulation system. Three-axis control of the microgripper's position was obtained using micromanipulators coupled to stepper motors. For each of the three linear axes, a total travel of ∼24 mm, with ∼0.5 μm/step size, was achieved. A haptic was used as the primary human interface to the system to control the actuation of the microgripper and control its position. To demonstrate the functionality of the system, the microgripper was used to manipulate micro-objects and to pick and place a surface-micromachined microgripper. The micro-objects included ∼100 μm diameter polysilicon chess pieces that were moved around a microscale polysilicon chessboard (∼ 1 mm × 1 mm). A microgripper was taken off the substrate it was fabricated on and assembled on a printed-circuit board. The assembled surface-micromachined microgripper was demonstrated in handling another microgripper. The entire system is interfaced by LabVIEW software.
---
paper_title: Intuitive haptics interface with accurate force estimation and reflection at nanoscale

paper_content:

Technologies, such as Atomic Force Microscopy (AFM), have proven to be one of the most versatile research equipments in the field of nanotechnology by providing physical access to the materials at nanoscale. Working principles of AFM involve physical interaction with the sample at nanometre scale to estimate the topography of the sample surface. Size of the cantilever tip, within the range of few nanometres diameter, and inherent elasticity of the cantilever allow it to bend in response to the changes in the sample surface leading to accurate estimation of the sample topography. Despite the capabilities of the AFM, there is a lack of intuitive user interfaces that could allow interaction with the materials at nanoscale, analogous to the way we are accustomed to at macro level. To bridge this gap of intuitive interface design and development, a haptics interface is designed in conjunction with Bruker Nanos AFM. Interaction with the materials at nanoscale is characterised by estimating the forces experienced by the cantilever tip employing geometric deformation principles. Estimated forces are reflected to the user, in a controlled manner, through haptics interface. Established mathematical framework for force estimation can be adopted for AFM operations in air as well as in liquid mediums.
---
paper_title: Design and Control of a Force-Re ecting Teleoperation System with Magnetically Levitated Master and Wrist

paper_content:

A new approach to the design of teleoperation systems is presented. It is proposed that the teleoperation slave be a coarse-fine manipulator with a fine-motion wrist identical to the teleoperation master. By using a combination of position and rate control, such a system would require only small operator hand motions but would provide low mechanical impedance, high motion resolution and force feedback over a substantial volume. A new teleoperation system, consisting of a conventional manipulator and two identical magnetically levitated wrists, has been developed using this approach and is described in this paper. Aspects of mechanical, system and computational design are discussed. It is shown that the best way to position the slave is by decoupling position and rate control, with the conventional robot controlled in rate mode and its wrist in position mode. Kinesthetic feedback is achieved through wrist-level coordinated force control. Transparency is improved through feedforward of sensed hand forces to the master and environment forces to the slave. To maintain stability, the amount of damping in the system is controlled by the sensed environment forces. Experimental results demonstrating excellent performance are presented.
---
paper_title: Scaled Teleoperation System for Nano Scale Interaction and Manipulation

paper_content:

In this paper, a visual and haptic human–machine interface is proposed for teleoperated nano-scale object interaction and manipulation. Design specifications for a bilateral scaled tele-operation system with slave and master robots, sensors, actuators and control are discussed. The Phantom™ haptic device is utilized as the master manipulator, and a piezoresistive atomic force microscope probe is selected as the slave manipulator and as topography and force sensors. Using the teleoperation control system, initial experiments are realized for interacting with nano-scale surfaces. It is shown that fine structures can be felt on the operator's finger successfully, and improved nano-scale interaction and manipulation using visual and haptic feedback can be achieved.
---
paper_title: A Review of Haptic Feedback Teleoperation Systems for Micromanipulation and Microassembly

paper_content:

This paper presents a review of the major haptic feedback teleoperation systems for micromanipulation. During the last decade, the handling of micrometer-sized objects has become a critical issue. Fields of application from material science to electronics demonstrate an urgent need for intuitive and flexible manipulation systems able to deal with small-scale industrial projects and assembly tasks. Two main approaches have been considered: fully automated tasks and manual operation. The first one require fully pre determined tasks, while the later necessitates highly trained operators. To overcome these issues the use of haptic feedback teleoperation where the user manipulates the tool through a joystick whilst feeling a force feedback, appears to be a promising solution as it allows high intuitiveness and flexibility. Major advances have been achieved during this last decade, starting with systems that enable the operator to feel the substrate topology, to the current state-of-the-art where 3D haptic feedback is provided to aid manipulation tasks. This paper details the major achievements and the solutions that have been developed to propose 3D haptic feedback for tools that often lack 3D force measurements. The use of virtual reality to enhance the immersion is also addressed. The strategies developed provide haptic feedback teleoperation systems with a high degree of assistance and for a wide range of micromanipulation tools. Based on this expertise on haptic for micromanipulation and virtual reality assistance it is now possible to propose microassembly systems for objects as small as 1 to 10 micrometers. This is a mature field and will benefit small-scale industrial projects where precision and flexibility in microassembly are required.
---
paper_title: Towards Haptic Microrobotic Intracellular Injection

paper_content:

This paper proposes a system providing the operator with an intuitive method for controlling a micromanipulator during intracellular injection. A low-cost haptic device is utilised and 3D position-to-position kinematic mapping allows the operator to control the micropipette using a similar method to handheld needle insertion. The workspaces of the haptic device and micromanipulator are analysed and the importance of appropriate scaling to positioning resolution and tracking performance is investigated. The control issues integral to achieving adequate control of the micromanipulator using the Phantom Omni haptic device are addressed. Aside from offering an intuitive method for controlling the micropipette, this work lays the foundation for real-time haptic assistance in the cell injection task.Copyright © 2009 by ASME
---
paper_title: Tele-touch feedback of surfaces at the micro/nano scale: modeling and experiments

paper_content:

In this paper, a teleoperated micro/nano scale touching system is proposed, and micro/nano contact mechanics models are introduced. Using a 1-DOF haptic device, force-reflecting servo type scaled teleoperation controller, and atomic force microscopy cantilever tip, touching experiments and virtual reality simulator for micro/nano-touch are realized. Scaling issue of the micro/nano forces and positions is discussed and possible solutions are proposed. For the first time upon our knowledge, such issue is discussed with nano scale experimental results. Experimental results show that micro/nano contact force feedback can be held for flat surfaces with the proposed system.
---
paper_title: Augmented reality user interface for an atomic force microscope-based nanorobotic system

paper_content:

A real-time augmented reality (AR) user interface for nanoscale interaction and manipulation applications using an atomic force microscope (AFM) is presented. Nanoscale three-dimensional (3-D) topography and force information sensed by an AFM probe are fed back to a user through a simulated AR system. The sample surface is modeled with a B-spline-based geometry model, upon which a collision detection algorithm determines whether and how the spherical AFM tip penetrates the surface. Based on these results, the induced surface deformations are simulated using continuum micro/nanoforce and Maugis-Dugdale elastic contact mechanics models, and 3-D decoupled force feedback information is obtained in real time. The simulated information is then blended in real time with the force measurements of the AFM in an AR human machine interface, comprising a computer graphics environment and a haptic interface. Accuracy, usability, and reliability of the proposed AR user interface is tested by experiments for three tasks: positioning the AFM probe tip close to a surface, just in contact with a surface, or below a surface by elastically indenting. Results of these tests showed the performance of the proposed user interface. This user interface would be critical for many nanorobotic applications in biotechnology, nanodevice prototyping, and nanotechnology education
---
paper_title: First high speed simultaneous force feedback for multi-trap optical tweezers

paper_content:

Optical tweezers use a tightly focused laser beam to trap dielectric micro-object. Haptic optical tweezers integrate a robotic haptic device into the system to control trap positions and in the meantime feedback forces. This technique opens a broad reaching applications by providing direct and intuitive interactions during micromanipulation. However, 1 kHz sampling rate is required to ensure a reliable tactile sensation. A system satisfying this requirement has been presented in our previous work. This paper goes one big step further to perform real-time high speed haptic feedback on multi-trap optical tweezers. The goal of the system is to enable users to touch the microscale object the same way as using their fingers. The used force sensor is a novel event-based vision sensor that transmits output as a stream of asynchronous timestamped events that eliminates internally the redundant static information. The associated event based algorithm can thus be able to track the positions of two spheres up to 1 kHz. A contact experiment is performed demonstrating the feasibility and the efficiency of the presented method. To our knowledge, this is the first time high speed haptic feedback is performed on multiple microspheres simultaneously in a contact scenario.
---
paper_title: Realistic Visual and Haptic Rendering for Biological-Cell Injection

paper_content:

This paper presents a new bio-micromanipulation system for biological object such as embryo, cell or oocyte. As the cell is very small, kept in the liquid, and observed through a microscope, the two-dimensional visual feedback makes difficult accurate manipulation in the 3-D world. To improve the manipulation work, we proposed an augmented human-machine interface. A 3-D visual information is provided to the operator through a 3-D reconstruction method using vision-based tracking deformations of the cell embryo. In order to stable injection tasks, the operator needs force feedback and haptic assistance during penetrating the cell envelop, the chorion. The proposed human-machine user’s interface allows real-time realistic visual and haptic control strategies for constrained motion in image coordinates. Virtual haptic rendering allows to constrain the path of insertion and removal in the 3-D scene or to avoid cell destruction by controlling adequately position, velocity and force parameters.
---
paper_title: Micro-to-Nano Biomechanical Modeling for Assisted Biological Cell Injection

paper_content:

To facilitate training of biological cell injection operations, we are developing an interactive virtual environment to simulate needle insertion into biological cells. This paper presents methodologies for dynamic modeling, visual/haptic display, and model validation of cell injection. We first investigate the challenging issues in the modeling of the biomechanical properties of living cells. We propose two dynamic models to simulate cell deformation and puncture. The first approach is based on the assumptions that the mechanical response of living cells is mainly determined by the cytoskeleton and that the cytoskeleton is organized as a tensegrity structure including microfilaments, microtubules, and intermediate filaments. Equivalent microtubules struts are represented with a linear mass-tensor finite-element model and equivalent microfilaments and intermediate filaments with viscoelastic Kelvin-Voigt elements. The second modeling method assumes the overall cell as an homogeneous hyperelastic model (St, Venant-Kirchhoff). Both graphic and haptic rendering are provided in real time to the operator through a 3-D virtual environment. Simulated responses are compared to experimental data to show the effectiveness of the proposed physically based model.
---
paper_title: Force feedback interface for cell injection

paper_content:

Manual pronuclei injection and intracytoplasmic sperm injection (ICSI) requires long training and has low success rates primarily due to poor control over the injection force. Consequently, there is a need for quantification of forces during biological cell injection and for an automated cell injection system, which can provide force feedback to the operator improving the success rate of the injection task. We have developed a force feedback interface, which has the capability of measuring forces in the range of /spl mu/N-mN and provide a haptic display of the cell injection forces. The force sensor has been integrated with the biomanipulation system to detect forces in real time. Experiments were performed on two different varieties of egg cells to demonstrate the success in measuring forces in the range in /spl mu/N-mN. Our experimental results indicate the cell puncturing forces were consistent and the operator was able to feel the celt injection forces.
---
paper_title: Haptic interface transparency achieved through viscous coupling

paper_content:

Electromagnetic drives are subjected to an inherent inertia-torque tradeoff that fundamentally limits transparency: the higher the torque, the higher the inertia. We describe a dual-stage design that is not subjected to this tradeoff and that is able to approach perfect transparency for human users. It comprises a large, proximal motor and a small, distal motor to reproduce the transients. The two stages are coupled by a viscous clutch based on eddy currents that, without contact, accurately transforms slip velocity into torque. Such a system can, in general, be controlled to achieve a variety of objectives. Here, we show that an advanced, discrete-time, RST polynomial pole-placement controller can achieve near-perfect transparency. Experimental validation evaluated the human ability to detect small haptic details when using this drive and compared it with when using a conventional, single-motor interface.
---
paper_title: A Stable and Transparent Microscale Force Feedback Teleoperation System

paper_content:

Scaled force feedback teleoperation is a promising approach to assist an operator engaged in a microscale task. Several systems were previously described to achieve such purpose, but much room was left for improvement, especially with regard to the specificities of bilateral coupling with very large scaling coefficients. Here, the objective is to render at human-scale haptic information available at the microscale, and to provide scaled teleoperation that simultaneously achieves stability and transparency. An active force sensor and a novel haptic interface were interconnected to form a complete teleoperation chain through a direct, two-channel scheme. Stability was ensured by enforcing passivity in the slave and in the master subsystems. Several experiments were carried out to demonstrate the capabilities of the system. The first experiment involved noncontact magnetic interaction. A second set of experiments demonstrated the penetration of a thin-glass probe in a water droplet where the operator interactively felt capillary forces.
---
paper_title: 3D particle-based cell modelling for haptic microrobotic cell injection

paper_content:

Introducing haptic interface to conduct microrobotic intracellular injection has many beneficial implications. In particular, the haptic device provides force feedback to the bio-operator's hand. This paper introduces a 3D particle-based model to simulate the deformation of the cell membrane and corresponding cellular forces during microrobotic cell injection. The model is based on the kinematic and dynamic of spring – damper multi particle joints considering visco-elastic fluidic properties. It simulates the indentation force feedback as well as cell visual deformation during the microinjection. The model is verified using experimental data of zebrafish embryo microinjection. The results demonstrate that the developed cell model is capable of estimating zebrafish embryo deformation and force feedback accurately.
---
paper_title: Haptic Microrobotic Cell Injection System

paper_content:

Microrobotic cell injection is the subject of increasing research interest. At present, an operator relies completely on visual information and can be subject to low success rates, poor repeatability, and extended training times. This paper focuses on increasing operator performance during cell injection in two ways. First, our completed haptic cell injection system aims to increase the operator's performance during real-time cell injection. Haptic bilateralism is investigated and a mapping framework provides an intuitive method for manoeuvring the micropipette in a manner similar to handheld needle insertion. Volumetric virtual fixtures are then introduced to haptically assist the operator to penetrate the cell at the desired location. The performance of the volumetric virtual fixtures is also discussed. Second, the haptically enabled cell injection system is replicated as a virtual environment facilitating virtual offline operator training. Virtual operator training utilizes the same mapping framework and haptic virtual fixtures as the physical system allowing the operator to train offline and then directly transfer their skills to real-time cell injection.
---
paper_title: Haptic Virtual Reality Training Environment for Micro-robotic Cell Injection

paper_content:

Micro-robotic cell injection is typically performed manually by a trained bio-operator, and success rates are often low. To enhance bio-operator performance during real-time cell injection, our earlier work introduced a haptically-enabled micro-robotic cell injection system. The system employed haptic virtual fixtures to provide haptic guidance according to particular performance metrics. This paper extends the work by replicating the system within a virtual reality (VR) environment for bio-operator training. Using the virtual environment, the bio-operator is able to control the virtual injection process in the same way they would with the physical haptic micro-robotic cell injection system, while benefiting from the enhanced visualisation capabilities offered by the 3D VR environment. The system is achieved using cost-effective components offering training at much lower cost than using the physical system.
---
paper_title: Multipoint Haptic Guidance for Micrograsping Systems

paper_content:

The ability to perform accurate micromanipulation offers wide-reaching benefits and is of increasing interest to researchers. Recent research into microgripper, microtweezer, and microforcep systems contributes toward accurate micrograsping and manipulation. Despite these efforts, achieving adequate operator control remains a distinct research challenge. Haptic interfaces interact with the human's haptic modality and offer the ability to enhance the operator's controllability of micromanipulation systems. Our previous work introduced single-point haptic guidance to assist the operator during intracellular microinjection. This paper extends the approach to propose multipoint haptic guidance for micrograsping tasks. Accurate micrograsping is valuable in many applications, including microassembly and biomanipulation. A multipoint haptic gripper facilitates haptic interaction, and haptic guidance assists the operator in controlling systems suitable for micrograsping. Force fields are used to guide the operator to suitable grasp points on micrometer-sized objects and consist of attractive and repulsive forces. The ability of the force field to effectively assist the operator in grasping the cell is evaluated using a virtual environment. Evaluation results demonstrate the ability of the approach to significantly reduce participants' average grasping error.
---
paper_title: Haptic guidance for improved task performance in steering microparticles with optical tweezers

paper_content:

We report the manipulation of 4–5 µm diameter polymer microspheres floating in water using optical tweezers (OT) and a haptic device (i.e. force-reflecting robotic arm). Trapped microspheres are steered using the end-effector of a haptic device that is virtually coupled to an XYZ piezo-scanner controlling the movements of the fluid bed. To help with the manipulations, we first calculate a collision-free path for the particle and then display artificial guidance forces to the user through the haptic device to keep him/her on this path during steering. Experiments conducted with 8 subjects show almost two-fold improvements in the average path error and average speed under the guidance of haptic feedback.
---
paper_title: Evaluation of Telerobotic Shared Control Strategy for Efficient Single-Cell Manipulation

paper_content:

Microinjection is a method for the delivery of exogenous materials into cells and is widely used in biomedical research areas such as transgenics and genomics. However, this direct injection is a time-consuming and laborious task, resulting in low throughput and poor reproducibility. Here, we describe a telerobotic shared control framework for microinjection, in which a micromanipulator is controlled by the shared motion commands of both the human operator and the autonomous controller. To determine the weightings between the operator and the controller, we proposed a quantitative evaluation method using a model of speed/accuracy trade-offs in human movement. The results showed that a 40%-60% weighting on the human operator (or the controller) produced the best performance for both speed and accuracy of guiding and targeting task in microinjection suggesting that some level of both automation and human involvement is important for microinjection tasks.
---
paper_title: Haptic guidance for microrobotic intracellular injection

paper_content:

The ability for a bio-operator to utilise a haptic device to manipulate a microrobot for intracellular injection offers immense benefits. One significant benefit is for the bio-operator to receive haptic guidance while performing the injection process. In order to address this, this paper investigates the use of haptic virtual fixtures for cell injection and proposes a novel force field virtual fixture. The guidance force felt by the bio-operator is determined by force field analysis within the virtual fixture. The proposed force field virtual fixture assists the bio-operator when performing intracellular injection by limiting the micropipette tip's motion to a conical volume as well as recommending the desired path for optimal injection. A virtual fixture plane is also introduced to prevent the bio-operator from moving the micropipette tip beyond the deposition target inside the cell. Simulation results demonstrate the operation of the guidance system.
---

Title: Haptic Feedback for Microrobotics Applications: A Review

Section 1: Introduction
Description 1: This section provides an overview of the field of microrobotics and the importance of haptic feedback in enhancing operator control and accuracy.

Section 2: Literature Review
Description 2: This section reviews the literature on haptic feedback systems for microrobotics, categorizing it according to the type of haptic sensing technique employed.

Section 3: Atomic Force Microscopy
Description 3: This section discusses the use of Atomic Force Microscopy (AFM) in haptic-enabled microrobotic systems, including various implementations and developments in AFM-based haptic feedback.

Section 4: Optical Tweezers
Description 4: This section reviews haptic feedback systems that utilize optical tweezers, focusing on their design, implementation, and applications in microrobotics.

Section 5: Electrostatic Active Sensors
Description 5: This section covers the use of electrostatic active sensors in haptic feedback systems, detailing specific applications and benefits in microscale force sensing.

Section 6: Active Constraints
Description 6: This section examines the application of active constraints in haptic feedback systems to assist operators in precision tasks such as cell injection and micromanipulation.

Section 7: Author Contributions
Description 7: This section lists the contributions of the authors to the research and writing of the paper.

Section 8: Funding
Description 8: This section provides details about the funding sources that supported the research, including grants from the European Research Council and the European Union's Horizon 2020 program.