MIT News - Robotics MIT news feed about: Robotics en Thu, 01 Apr 2021 00:00:00 -0400 A robot that senses hidden objects System uses penetrative radio frequency to pinpoint items, even when they’re hidden from view. Thu, 01 Apr 2021 00:00:00 -0400 Daniel Ackerman | MIT News Office <p>In recent years, robots have gained artificial vision, touch, and even smell. “Researchers have been giving robots human-like perception,” says MIT Associate Professor Fadel Adib. In a new paper, Adib’s team is pushing the technology a step further. “We’re trying to give robots superhuman perception,” he says.</p> <p></p> <p>The researchers have developed a robot that uses radio waves, which can pass through walls, to sense occluded objects. The robot, called RF-Grasp, combines this powerful sensing with more traditional computer vision to locate and grasp items that might otherwise be blocked from view. The advance could one day streamline e-commerce fulfillment in warehouses or help a machine pluck a screwdriver from a jumbled toolkit.</p> <p></p> <p>The research will be presented in May at the IEEE International Conference on Robotics and Automation. The paper’s lead author is Tara Boroushaki, a research assistant in the Signal Kinetics Group at the MIT Media Lab. Her MIT co-authors include Adib, who is the director of the Signal Kinetics Group; and Alberto Rodriguez, the Class of 1957 Associate Professor in the Department of Mechanical Engineering. Other co-authors include Junshan Leng, a research engineer at Harvard University, and Ian Clester, a PhD student at Georgia Tech.</p> <p>As e-commerce continues to grow, warehouse work is still usually the domain of humans, not robots, despite sometimes-dangerous working conditions. That’s in part because robots struggle to locate and grasp objects in such a crowded environment. “Perception and picking are two roadblocks in the industry today,” says Rodriguez. Using optical vision alone, robots can’t perceive the presence of an item packed away in a box or hidden behind another object on the shelf — visible light waves, of course, don’t pass through walls.</p> <p></p> <p>But radio waves can.</p> <p></p> <p>For decades, radio frequency (RF) identification has been used to track everything from library books to pets. RF identification systems have two main components: a reader and a tag. The tag is a tiny computer chip that gets attached to — or, in the case of pets, implanted in — the item to be tracked. The reader then emits an RF signal, which gets modulated by the tag and reflected back to the reader.</p> <p></p> <p>The reflected signal provides information about the location and identity of the tagged item. The technology has gained popularity in retail supply chains — Japan aims to use RF tracking for nearly all <a href="">retail purchases</a> in a matter of years. The researchers realized this profusion of RF could be a boon for robots, giving them another mode of perception.</p> <p></p> <p>“RF is such a different sensing modality than vision,” says Rodriguez. “It would be a mistake not to explore what RF can do.”</p> <p></p> <p>RF Grasp uses both a camera and an RF reader to find and grab tagged objects, even when they’re fully blocked from the camera’s view. It consists of a robotic arm attached to a grasping hand. The camera sits on the robot’s wrist. The RF reader stands independent of the robot and relays tracking information to the robot’s control algorithm. So, the robot is constantly collecting both RF tracking data and a visual picture of its surroundings. Integrating these two data streams into the robot’s decision making was one of the biggest challenges the researchers faced.</p> <p></p> <p>“The robot has to decide, at each point in time, which of these streams is more important to think about,” says Boroushaki. “It’s not just eye-hand coordination, it’s RF-eye-hand coordination. So, the problem gets very complicated.”</p> <p></p> <p>The robot initiates the seek-and-pluck process by pinging the target object’s RF tag for a sense of its whereabouts. “It starts by using RF to focus the attention of vision,” says Adib. “Then you use vision to navigate fine maneuvers.” The sequence is akin to hearing a siren from behind, then turning to look and get a clearer picture of the siren’s source.</p> <p></p> <p>With its two complementary senses, RF Grasp zeroes in on the target object. As it gets closer and even starts manipulating the item, vision, which provides much finer detail than RF, dominates the robot’s decision making.</p> <p></p> <p>RF Grasp proved its efficiency in a battery of tests. Compared to a similar robot equipped with only a camera, RF Grasp was able to pinpoint and grab its target object with about half as much total movement. Plus, RF Grasp displayed the unique ability to “declutter” its environment — removing packing materials and other obstacles in its way in order to access the target. Rodriguez says this demonstrates RF Grasp’s “unfair advantage” over robots without penetrative RF sensing. “It has this guidance that other systems simply don’t have.”</p> <p></p> <p>RF Grasp could one day perform fulfilment in packed e-commerce warehouses. Its RF sensing could even instantly verify an item’s identity without the need to manipulate the item, expose its barcode, then scan it. “RF has the potential to improve some of those limitations in industry, especially in perception and localization,” says Rodriguez.</p> <p></p> <p>Adib also envisions potential home applications for the robot, like locating the right Allen wrench to assemble your Ikea chair. “Or you could imagine the robot finding lost items. It’s like a super-Roomba that goes and retrieves my keys, wherever the heck I put them.”</p> <p></p> <p>The research is sponsored by the National Science Foundation, NTT DATA, Toppan, Toppan Forms, and the Abdul Latif Jameel Water and Food Systems Lab (J-WAFS).</p> MIT researchers developed a picking robot that combines vision with radio frequency (RF) sensing to find and grasps objects, even if they’re hidden from view. The technology could aid fulfilment in e-commerce warehouses. Courtesy of the researchers Researchers’ algorithm designs soft robots that sense Deep-learning technique optimizes the arrangement of sensors on a robot’s body to ensure efficient operation. Mon, 22 Mar 2021 00:00:00 -0400 Daniel Ackerman | MIT News Office <p>There are some tasks that traditional robots — the rigid and metallic kind — simply aren’t cut out for. Soft-bodied robots, on the other hand, may be able to interact with people more safely or slip into tight spaces with ease. But for robots to reliably complete their programmed duties, they need to know the whereabouts of all their body parts. That’s a tall task for a soft robot that can deform in a virtually infinite number of ways.</p> <p></p> <p>MIT researchers have developed an algorithm to help engineers design soft robots that collect more useful information about their surroundings. The deep-learning algorithm suggests an optimized placement of sensors within the robot’s body, allowing it to better interact with its environment and complete assigned tasks. The advance is a step toward the automation of robot design. “The system not only learns a given task, but also how to best design the robot to solve that task,” says Alexander Amini. “Sensor placement is a very difficult problem to solve. So, having this solution is extremely exciting.”</p> <p></p> <p>The research will be presented during April’s IEEE International Conference on Soft Robotics and will be published in the journal <em>IEEE Robotics and Automation Letters</em>. Co-lead authors are Amini and Andrew Spielberg, both PhD students in MIT Computer Science and Artificial Intelligence Laboratory (CSAIL). Other co-authors include MIT PhD student Lillian Chin, and professors Wojciech Matusik and Daniela Rus.</p> <p>Creating soft robots that complete real-world tasks has been a long-running challenge in robotics. Their rigid counterparts have a built-in advantage: a limited range of motion. Rigid robots’ finite array of joints and limbs usually makes for manageable calculations by the algorithms that control mapping and motion planning. Soft robots are not so tractable.</p> <p></p> <p>Soft-bodied robots are flexible and pliant — they generally feel more like a bouncy ball than a bowling ball. “The main problem with soft robots is that they are infinitely dimensional,” says Spielberg. “Any point on a soft-bodied robot can, in theory, deform in any way possible.” That makes it tough to design a soft robot that can map the location of its body parts. Past efforts have used an external camera to chart the robot’s position and feed that information back into the robot’s control program. But the researchers wanted to create a soft robot untethered from external aid.</p> <p></p> <p>“You can’t put an infinite number of sensors on the robot itself,” says Spielberg. “So, the question is: How many sensors do you have, and where do you put those sensors in order to get the most bang for your buck?” The team turned to deep learning for an answer.</p> <p></p> <p>The researchers developed a novel neural network architecture that both optimizes sensor placement and learns to efficiently complete tasks. First, the researchers divided the robot’s body into regions called “particles.” Each particle’s rate of strain was provided as an input to the neural network. Through a process of trial and error, the network “learns” the most efficient sequence of movements to complete tasks, like gripping objects of different sizes. At the same time, the network keeps track of which particles are used most often, and it culls the lesser-used particles from the set of inputs for the networks’ subsequent trials.</p> <p></p> <p>By optimizing the most important particles, the network also suggests where sensors should be placed on the robot to ensure efficient performance. For example, in a simulated robot with a grasping hand, the algorithm might suggest that sensors be concentrated in and around the fingers, where precisely controlled interactions with the environment are vital to the robot’s ability to manipulate objects. While that may seem obvious, it turns out the algorithm vastly outperformed humans’ intuition on where to site the sensors.</p> <p></p> <p>The researchers pitted their algorithm against a series of expert predictions. For three different soft robot layouts, the team asked roboticists to manually select where sensors should be placed to enable the efficient completion of tasks like grasping various objects. Then they ran simulations comparing the human-sensorized robots to the algorithm-sensorized robots. And the results weren’t close. “Our model vastly outperformed humans for each task, even though I looked at some of the robot bodies and felt very confident on where the sensors should go,” says Amini. “It turns out there are a lot more subtleties in this problem than we initially expected.”</p> <p></p> <p>Spielberg says their work could help to automate the process of robot design. In addition to developing algorithms to control a robot’s movements, “we also need to think about how we’re going to sensorize these robots, and how that will interplay with other components of that system,” he says. And better sensor placement could have industrial applications, especially where robots are used for fine tasks like gripping. “That’s something where you need a very robust, well-optimized sense of touch,” says Spielberg. “So, there’s potential for immediate impact.”</p> <p></p> <p>“Automating the design of sensorized soft robots is an important step toward rapidly creating intelligent tools that help people with physical tasks,” says Rus. “The sensors are an important aspect of the process, as they enable the soft robot to “see” and understand the world and its relationship with the world.”</p> <p></p> <p>This research was funded, in part, by the National Science Foundation and the Fannie and John Hertz Foundation.</p> MIT researchers have developed a deep learning neural network to aid the design of soft-bodied robots, such as these iterations of a robotic elephant. Courtesy of the researchers Tactile textiles sense movement via touch By measuring a person’s movements and poses, smart clothes developed at MIT CSAIL could be used for athletic training, rehabilitation, or health-monitoring for elder-care facilities. Wed, 31 Mar 2021 15:50:00 -0400 Adam Conner-Simons | MIT CSAIL <p>In recent years there have been exciting breakthroughs in wearable technologies, like smartwatches that can monitor your breathing and blood oxygen levels.&nbsp;</p> <p>But what about a wearable that can detect how you move as you do a physical activity or play a sport, and could potentially even offer feedback on how to improve your technique?&nbsp;</p> <p>And, as a major bonus, what if the wearable were something you’d actually already be wearing, like a shirt of a pair of socks?</p> <p>That’s the idea behind a new set of MIT-designed clothing that use special fibers to sense a person’s movement via touch. Among other things, the researchers showed that their clothes can actually <a href=";preview=SupplementaryVideo5.mp4" target="_blank">determine things like if someone is sitting, walking, or doing particular poses</a>.</p> <p>The group from MIT’s Computer Science and Artificial Intelligence Lab (CSAIL) says that their clothes could be used for athletic training and rehabilitation. With patients’ permission, they could even help passively monitor the health of residents in assisted-care facilities and determine if, for example, someone has fallen or is unconscious. <em>&nbsp;</em></p> <p>The researchers have developed a range of prototypes, from socks and gloves to a full vest. The team’s “tactile electronics” use a mix of more typical textile fibers alongside a small amount of custom-made functional fibers that sense pressure from the person wearing the garment.</p> <p>According to CSAIL graduate student Yiyue Luo, a key advantage of the team’s design is that, unlike many existing wearable electronics, theirs can be incorporated into traditional large-scale clothing production. The machine-knitted tactile textiles are soft, stretchable, breathable, and can take a wide range of forms.&nbsp;</p> <p>“Traditionally it’s been hard to develop a mass-production wearable that provides high-accuracy data across a large number of sensors,” says Luo, lead author on a new paper about the project that is appearing in this month’s edition of <em>Nature Electronics</em>. “When you manufacture lots of sensor arrays, some of them will not work and some of them will work worse than others, so we developed a self-correcting mechanism that uses a self-supervised machine learning algorithm to recognize and adjust when certain sensors in the design are off-base.”</p> <p>The team’s clothes have a range of capabilities. Their socks predict motion by looking at how different sequences of tactile footprints correlate to different poses as the user transitions from one pose to another. The full-sized vest can also detect the wearers' pose, activity, and the texture of the contacted surfaces.</p> <p>The authors imagine a coach using the sensor to analyze people’s postures and give suggestions on improvement. It could also be used by an experienced athlete to record their posture so that beginners can learn from them. In the long term, they even imagine that robots could be trained to learn how to do different activities using data from the wearables.&nbsp;</p> <p>“Imagine robots that are no longer tactilely blind, and that have ‘skins’ that can provide tactile sensing just like we have as humans,” says corresponding author Wan Shou, a postdoc at CSAIL. “Clothing with high-resolution tactile sensing opens up a lot of exciting new application areas for researchers to explore in the years to come.”</p> <p>The paper was co-written by MIT professors Antonio Torralba, Wojciech Matusik, and Tomás Palacios, alongside PhD students Yunzhu Li, Pratyusha Sharma, and Beichen Li; postdoc Kui Wu; and research engineer Michael Foshey.&nbsp;</p> <p>The work was partially funded by Toyota Research Institute.</p> Tactile electronics developed at MIT use a mix of more typical textile fibers alongside a small amount of custom-made functional fibers that sense pressure from the person wearing the garment. Image courtesy of Nature Electronics. More transparency and understanding into machine behaviors A new tool helps humans better understand and develop artificial intelligence models by searching and highlighting representative scenarios. Mon, 22 Mar 2021 15:45:00 -0400 Rachel Gordon | MIT CSAIL <p>Explaining, interpreting, and understanding the human mind presents a unique set of challenges.&nbsp;</p> <p>Doing the same for the behaviors of machines, meanwhile, is a whole other story.&nbsp;</p> <p>As artificial intelligence (AI) models are increasingly used in complex situations — approving or denying loans, helping doctors with medical diagnoses, assisting drivers on the road, or even taking complete control — humans still lack a holistic understanding of their capabilities and behaviors.&nbsp;</p> <p>Existing research focuses mainly on the basics: How accurate is this model? Oftentimes, centering on the notion of simple accuracy can lead to dangerous oversights. What if the model makes mistakes with very high confidence? How would the model behave if it encountered something previously unseen, such as a self-driving car seeing a new type of traffic sign?</p> <p>In the quest for better human-AI interaction, a team of researchers from MIT’s Computer&nbsp;Science and Artificial Intelligence Laboratory (CSAIL) have created a new tool called <a href="" target="_blank">Bayes-TrEx</a> that allows developers and users to gain transparency into their AI model.<strong> </strong>Specifically, it does so by finding concrete examples that lead to a particular behavior. The method makes use of&nbsp; “Bayesian posterior inference,” a widely-used mathematical framework to reason about model uncertainty.</p> <p>In experiments, the researchers applied Bayes-TrEx to several image-based datasets, and found new insights that were previously overlooked by standard evaluations focusing solely on prediction accuracy.&nbsp;</p> <p>“Such analyses are important to verify that the model is indeed functioning correctly in all cases,” says MIT CSAIL PhD student Yilun Zhou, co-lead researcher on Bayes-TrEx. “An especially alarming situation is when the model is making mistakes, but with very high confidence. Due to high user trust over the high reported confidence, these mistakes might fly under the radar for a long time and only get discovered after causing extensive damage.”</p> <p>For example, after a medical diagnosis system finishes learning on a set of X-ray images, a doctor can use Bayes-TrEx to find images that the model misclassified with very high confidence, to ensure that it doesn't miss any particular variant of a disease.&nbsp;</p> <p>Bayes-TrEx can also help with understanding model behaviors in novel situations. Take autonomous driving systems, which often rely on camera images to take in traffic lights, bike lanes, and obstacles. These common occurrences can be easily recognized with high accuracy by the camera, but more complicated situations can provide literal and metaphorical roadblocks. A zippy Segway could potentially be interpreted as something as big as a car or as small as a bump on the road, leading to a tricky turn or costly collision. Bayes-TrEx could help address these novel situations ahead<em> </em>of time, and enable developers to correct any undesirable outcomes before potential tragedies occur.&nbsp;</p> <p>In addition to images, the researchers are also tackling a less-static domain: robots. Their tool, called “RoCUS”, inspired by Bayes-TrEx, uses additional adaptations to analyze robot-specific behaviors.&nbsp;</p> <p>While still in a testing phase, experiments with RoCUS point to new discoveries that could be easily missed if the evaluation was focused solely on task completion. For example, a 2D navigation robot that used a deep learning approach preferred to navigate tightly around obstacles, due to how the training data was collected. Such a preference, however, could be risky if the robot’s obstacle sensors are not fully accurate. For a robot arm reaching a target on a table, the asymmetry in the robot’s kinematic structure showed larger implications on its ability to reach targets on the left versus the right.</p> <p>“We want to make human-AI interaction safer by giving humans more insight into their AI collaborators,” says MIT CSAIL PhD student Serena Booth, co-lead author with Zhou. “Humans should be able to understand how these agents make decisions, to predict how they will act in the world, and — most critically — to anticipate and circumvent failures.”&nbsp;&nbsp;</p> <p>Booth and Zhou are coauthors on the Bayes-TrEx work alongside MIT CSAIL PhD student Ankit Shah and MIT Professor Julie Shah. They presented the paper virtually at the AAAI conference on Artificial Intelligence. Along with Booth, Zhou, and Shah, MIT CSAIL postdoc Nadia Figueroa Fernandez has contributed work on the RoCUS tool.<br /> &nbsp;</p> Left to right: MIT CSAIL PhD student Ankit Shah, postdoc Nadia Figueroa Fernandez, PhD student Serena Booth, and PhD student Yilun Zhou. Photo courtesy of the researchers. Algorithm helps artificial intelligence systems dodge “adversarial” inputs Method builds on gaming techniques to help autonomous vehicles navigate in the real world, where signals may be imperfect. Mon, 08 Mar 2021 09:00:00 -0500 Jennifer Chu | MIT News Office <p>In a perfect world, what you see is what you get. If this were the case, the job of artificial intelligence systems would be refreshingly straightforward.</p> <p>Take collision avoidance systems in self-driving cars. If visual input to on-board cameras could be trusted entirely, an AI system could directly map that input to an appropriate action — steer right, steer left, or continue straight — to avoid hitting a pedestrian that its cameras see in the road.</p> <p>But what if there’s a glitch in the cameras that slightly shifts an image by a few pixels? If the car blindly trusted so-called “adversarial inputs,” it might take unnecessary and potentially dangerous action.</p> <p>A new deep-learning algorithm developed by MIT researchers is designed to help machines navigate in the real, imperfect world, by building a healthy “skepticism” of the measurements and inputs they receive.</p> <p>The team combined a reinforcement-learning algorithm with a deep neural network, both used separately to train computers in playing video games like Go and chess, to build an approach they call CARRL, for Certified Adversarial Robustness for Deep Reinforcement Learning.</p> <p>The researchers tested the approach in several scenarios, including a simulated collision-avoidance test and the video game Pong, and found that CARRL performed better — avoiding collisions and winning more Pong games — over standard machine-learning techniques, even in the face of uncertain, adversarial inputs.</p> <p>“You often think of an adversary being someone who’s hacking your computer, but it could also just be that your sensors are not great, or your measurements aren’t perfect, which is often the case,” says Michael Everett, a postdoc in MIT’s Department of Aeronautics and Astronautics (AeroAstro). “Our approach helps to account for that imperfection and make a safe decision. In any safety-critical domain, this is an important approach to be thinking about.”</p> <p>Everett is the lead author of a study outlining the new approach, which appears in <em>IEEE’s Transactions on Neural Networks and Learning Systems</em>. The study originated from MIT PhD student Björn Lütjens’ master’s thesis and was advised by MIT AeroAstro Professor Jonathan How.</p> <p><strong>Possible realities</strong></p> <p>To make AI systems robust against adversarial inputs, researchers have tried implementing defenses for supervised learning. Traditionally, a neural network is trained to associate specific labels or actions with given inputs. For instance, a neural network that is fed thousands of images labeled as cats, along with images labeled as houses and hot dogs, should correctly label a new image as a cat.</p> <p>In robust AI systems, the same supervised-learning techniques could be tested with many slightly altered versions of the image. If the network lands on the same label — cat — for every image, there’s a good chance that, altered or not, the image is indeed of a cat, and the network is robust to any adversarial influence.</p> <p>But running through every possible image alteration is computationally exhaustive and difficult to apply successfully to time-sensitive tasks such as collision avoidance. Furthermore, existing methods also don’t identify what label to use, or what action to take, if the network is less robust and labels some altered cat images as a house or a hotdog.</p> <p>“In order to use neural networks in safety-critical scenarios, we had to find out how to take real-time decisions based on worst-case assumptions on these possible realities,” Lütjens says.</p> <p><strong>The best reward</strong></p> <p>The team instead looked to build on reinforcement learning, another form of machine learning that does not require associating labeled inputs with outputs, but rather aims to reinforce certain actions in response to certain inputs, based on a resulting reward. This approach is typically used to train computers to play and win games such as chess and Go.</p> <p>Reinforcement learning has mostly been applied to situations where inputs are assumed to be true. Everett and his colleagues say they are the first to bring “certifiable robustness” to uncertain, adversarial inputs in reinforcement learning.</p> <p>Their approach, CARRL, uses an existing deep-reinforcement-learning algorithm to train a deep Q-network, or DQN — a neural network with multiple layers that ultimately associates an input with a Q value, or level of reward.</p> <p>The approach takes an input, such as an image with a single dot, and considers an adversarial influence, or a region around the dot where it actually might be instead. Every possible position of the dot within this region is fed through a DQN to find an associated action that would result in the most optimal worst-case reward, based on <a href="">a technique</a> developed by recent MIT graduate student Tsui-Wei “Lily” Weng PhD ’20.</p> <p><strong>An adversarial world</strong></p> <p>In tests with the video game Pong, in which two players operate paddles on either side of a screen to pass a ball back and forth, the researchers introduced an “adversary” that pulled the ball slightly further down than it actually was. They found that CARRL won more games than standard techniques, as the adversary’s influence grew.</p> <p>“If we know that a measurement shouldn’t be trusted exactly, and the ball could be anywhere within a certain region, then our approach tells the computer that it should put the paddle in the middle of that region, to make sure we hit the ball even in the worst-case deviation,” Everett says.</p> <img alt="pong" data-caption="In a game of Pong, MIT researchers show that, with perfect measurements, a standard deep learning algorithm is able to win most games (left). But in a scenario where the measurements are influenced by an “adversary” that shifts the ball’s position by a few pixels (middle), the computer easily beats the standard algorithm. The team’s new algorithm, CARRL, handles such adversarial attacks, or manipulations to measurements, winning against the computer, even though it doesn't know exactly where the ball is.&amp;nbsp;Courtesy of the researchers" data-entity-type="file" data-entity-uuid="2dc386f1-cc47-4149-bd37-ee40d4ada916" src="/sites/default/files/images/inline/adversarial-learning.gif" /> <p>The method was similarly robust in tests of collision avoidance, where the team simulated a blue and an orange agent attempting to switch positions without colliding. As the team perturbed the orange agent’s observation of the blue agent’s position, CARRL steered the orange agent around the other agent, taking a wider berth as the adversary grew stronger, and the blue agent’s position became more uncertain.</p> <p>There did come a point when CARRL became too conservative, causing the orange agent to assume the other agent could be anywhere in its vicinity, and in response completely avoid its destination. This extreme conservatism is useful, Everett says, because researchers can then use it as a limit to tune the algorithm’s robustness. For instance, the algorithm might consider a smaller deviation, or region of uncertainty, that would still allow an agent to achieve a high reward and reach its destination.</p> <p>In addition to overcoming imperfect sensors, Everett says CARRL may be a start to helping robots safely handle unpredictable interactions in the real world.</p> <p>“People can be adversarial, like getting in front of a robot to block its sensors, or interacting with them, not necessarily with the best intentions,” Everett says. “How can a robot think of all the things people might try to do, and try to avoid them? What sort of adversarial models do we want to defend against? That’s something we’re thinking about how to do.”</p> <p>This research was supported, in part, by Ford Motor Company as part of the Ford-MIT Alliance.</p> A deep-learning algorithm developed by MIT researchers is designed to help machines navigate in the real world. In tests with the video game Pong, the researchers introduced an “adversary” that pulled the ball slightly further down than it actually was. Image: Jose-Luis Olivares, MIT The (robotic) doctor will see you now Study finds patients are receptive to interacting with robots designed to evaluate symptoms in a contact-free way. Thu, 04 Mar 2021 11:00:00 -0500 Anne Trafton | MIT News Office <p>In the era of social distancing, using robots for some health care interactions is a promising way to reduce in-person contact between health care workers and sick patients. However, a key question that needs to be answered is how patients will react to a robot entering the exam room.</p> <p></p> <p>Researchers from MIT and Brigham and Women’s Hospital recently set out to answer that question. In a study performed in the emergency department at Brigham and Women’s, the team found that a large majority of patients reported that interacting with a health care provider via a video screen mounted on a robot was similar to an in-person interaction with a health care worker.</p> <p></p> <p>“We’re actively working on robots that can help provide care to maximize the safety of both the patient and the health care workforce. The results of this study give us some confidence that people are ready and willing to engage with us on those fronts,” says Giovanni Traverso, an MIT assistant professor of mechanical engineering, a gastroenterologist at Brigham and Women’s Hospital, and the senior author of the study.</p> <p></p> <p>In a larger online survey conducted nationwide, the researchers also found that a majority of respondents were open to having robots not only assist with patient triage but also perform minor procedures such as taking a nose swab.</p> <p></p> <p>Peter Chai, an assistant professor of emergency medicine at Brigham and Women’s Hospital and a research affiliate in Traverso’s lab, is the lead author of the study, which appears today in <em>JAMA Network Open</em>.</p> <p></p> <p><strong>Triage by robot</strong></p> <p></p> <p>After the Covid-19 pandemic began early last year, Traverso and his colleagues turned their attention toward new strategies to minimize interactions between potentially sick patients and health care workers. To that end, they worked with Boston Dynamics to create a <a href="">mobile robot</a> that could interact with patients as they waited in the emergency department. The robots were equipped with sensors that allow them to measure vital signs, including skin temperature, breathing rate, pulse rate, and blood oxygen saturation. The robots also carried an iPad that allowed for remote video communication with a health care provider.</p> <p>This kind of robot could reduce health care workers’ risk of exposure to Covid-19 and help to conserve the personal protective equipment that is needed for each interaction. However, the question still remained whether patients would be receptive to this type of interaction.</p> <p></p> <p>“Often as engineers, we think about different solutions, but sometimes they may not be adopted because people are not fully accepting of them,” Traverso says. “So, in this study we were trying to tease that out and understand if the population is receptive to a solution like this one.”</p> <p></p> <p>The researchers first conducted a nationwide survey of about 1,000 people, working with a market research company called YouGov. They asked questions regarding the acceptability of robots in health care, including whether people would be comfortable with robots performing not only triage but also other tasks such as performing nasal swabs, inserting a catheter, or turning a patient over in bed. On average, the respondents stated that they were open to these types of interactions.</p> <p></p> <p>The researchers then tested one of their robots in the emergency department at Brigham and Women’s Hospital last spring, when Covid-19 cases were surging in Massachusetts. Fifty-one patients were approached in the waiting room or a triage tent and asked if they would be willing to participate in the study, and 41 agreed. These patients were interviewed about their symptoms via video connection, using an iPad carried by a quadruped, dog-like robot developed by Boston Dynamics. More than 90 percent of the participants reported that they were satisfied with the robotic system.</p> <p></p> <p>“For the purposes of gathering quick triage information, the patients found the experience to be similar to what they would have experienced talking to a person,” Chai says.</p> <p></p> <p><strong>Robotic assistants</strong></p> <p></p> <p>The numbers from the study suggest that it could be worthwhile to try to develop robots that can perform procedures that currently require a lot of human effort, such as turning a patient over in bed, the researchers say. Turning Covid-19 patients onto their stomachs, also known as “proning,” has been shown to boost their blood oxygen levels and make breathing easier. Currently the process requires several people to perform. Administering Covid-19 tests is another task that requires a lot of time and effort from health care workers, who could be deployed for other tasks if robots could help perform swabs.</p> <p></p> <p>“Surprisingly, people were pretty accepting of the idea of having a robot do a nasal swab, which suggests that potential engineering efforts could go into thinking about building some of these systems,” Chai says.</p> <p></p> <p>The MIT team is continuing to develop sensors that can obtain vital sign data from patients remotely, and they are working on integrating these systems into smaller robots that could operate in a variety of environments, such as field hospitals or ambulances.</p> <p></p> <p>Other authors of the paper include Farah Dadabhoy, Hen-wei Huang, Jacqueline Chu, Annie Feng, Hien Le, Joy Collins, Marco da Silva, Marc Raibert, Chin Hur, and Edward Boyer. The research was funded by the National Institutes of Health, the Hans and Mavis Lopater Psychosocial Foundation, e-ink corporation, the Karl Van Tassel (1925) Career Development Professorship, MIT’s Department of Mechanical Engineering, and the Brigham and Women’s Hospital Division of Gastroenterology.</p> A majority of patients interacting with a health care provider via a video screen mounted on a robot said it was similar to an in-person interaction with a health care worker. Image: MIT News, with images from iStockphoto Helping soft robots turn rigid on demand New type of control system may broaden robots’ range of tasks and allow safer interactions with people. Wed, 03 Mar 2021 00:00:00 -0500 Daniel Ackerman | MIT News Office <p>Imagine a robot.</p> <p></p> <p>Perhaps you’ve just conjured a machine with a rigid, metallic exterior. While robots armored with hard exoskeletons are common, they’re not always ideal. Soft-bodied robots, inspired by fish or other squishy creatures, might better adapt to changing environments and work more safely with people.</p> <p></p> <p>Roboticists generally have to decide whether to design a hard- or soft-bodied robot for a particular task. But that tradeoff may no longer be necessary.</p> <p></p> <p>Working with computer simulations, MIT researchers have developed a concept for a soft-bodied robot that can turn rigid on demand. The approach could enable a new generation of robots that combine the strength and precision of rigid robots with the fluidity and safety of soft ones.</p> <p></p> <p>“This is the first step in trying to see if we can get the best of both worlds,” says James Bern, the paper’s lead author and a postdoc in MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL).</p> <p></p> <p>Bern will present the research at the IEEE International Conference on Soft Robotics next month. Bern’s advisor, Daniela Rus, who is the CSAIL director and the Andrew and Erna Viterbi Professor of Electrical Engineering and Computer Science, is the paper’s other author.</p> <p></p> <p>Roboticists have experimented with myriad mechanisms to operate soft robots, including inflating balloon-like chambers in a robot’s arm or <a href="" target="_blank">grabbing objects with vacuum-sealed coffee grounds</a>. However, a key unsolved challenge for soft robotics is control — how to drive the robot's actuators in order to achieve a given goal.</p> <p></p> <p>Until recently, most soft robots were controlled manually, but in 2017 Bern and his colleagues proposed that an algorithm could take the reigns. Using a <a href="" target="_blank">simulation</a> to help control a cable-driven soft robot, they picked a target position for the robot and had a computer figure out how much to pull on each of the cables in order to get there. A similar sequence happens in our bodies each time we reach for something: A target position for our hand is translated into contractions of the muscles in our arm.</p> <p></p> <p>Now, Bern and his colleagues are using similar techniques to ask a question that goes beyond the robot’s movement: “If I pull the cables in just the right way, can I get the robot to act stiff?” Bern says he can — at least in a computer simulation — thanks to inspiration from the human arm. While contracting the biceps alone can bend your elbow to a certain degree, contracting the biceps and triceps simultaneously can lock your arm rigidly in that position. Put simply, “you can get stiffness by pulling on both sides of something,” says Bern. So, he applied the same principle to his robots.</p> <p></p> <p>The researchers’ paper lays out a way to simultaneously control the position and stiffness of a cable-driven soft robot. The method takes advantage of the robots’ multiple cables — using some to twist and turn the body, while using others to counterbalance each other to tweak the robot’s rigidity. Bern emphasizes that the advance isn’t a revolution in mechanical engineering, but rather a new twist on controlling cable-driven soft robots.</p> <p></p> <p>“This is an intuitive way of expanding how you can control a soft robot,” he says. “It’s just encoding that idea [of on-demand rigidity] into something a computer can work with.” Bern hopes his roadmap will one day allow users to control a robot’s rigidity as easily as its motion.</p> <p></p> <p>On the computer, Bern used his roadmap to simulate movement and rigidity adjustment in robots of various shapes. He tested how well the robots, when stiffened, could resist displacement when pushed. Generally, the robots remained rigid as intended, though they were not equally resistant from all angles.</p> <p></p> <p>“Dual-mode materials that can change stiffness are always fascinating,” says Muhammad Hussain, an electrical engineer at the University of California at Berkeley, who was not involved with the research. He suggested potential applications in health care, where soft robots could one day travel through the blood stream then stiffen to perform microsurgery at a particular site in the body. Hussain say Bern’s demonstration “shows a viable path toward that future.”</p> <p></p> <p>Bern is building a prototype robot to test out his rigidity-on-demand control system. But he hopes to one day take the technology out of the lab. “Interacting with humans is definitely a vision for soft robotics,” he says. Bern points to potential applications in caring for human patients, where a robot’s softness could enhance safety, while its ability to become rigid could allow for lifting when necessary.</p> <p></p> <p>“The core message is to make it easy to control robots’ stiffness,” says Bern. “Let’s start making soft robots that are safe but can also act rigid on demand, and expand the spectrum of tasks robots can perform.”</p> A simulated soft robot controlled to reach the same target (red dot) while acting either soft (left) or stiff (right). Image: courtesy of the researchers New surgery may enable better control of prosthetic limbs Reconnecting muscle pairs during amputation gives patients more sensory feedback from the limb. Mon, 15 Feb 2021 15:00:00 -0500 Anne Trafton | MIT News Office <p>MIT researchers have invented a new type of amputation surgery that can help amputees to better control their residual muscles and sense where their “phantom limb” is in space. This restored sense of proprioception should translate to better control of prosthetic limbs, as well as a reduction of limb pain, the researchers say.</p> <p></p> <p>In most amputations, muscle pairs that control the affected joints, such as elbows or ankles, are severed. However, the MIT team has found that reconnecting these muscle pairs, allowing them to retain their normal push-pull relationship, offers people much better sensory feedback.</p> <p></p> <p>“Both our study and previous studies show that the better patients can dynamically move their muscles, the more control they’re going to have. The better a person can actuate muscles that move their phantom ankle, for example, the better they’re actually able to use their prostheses,” says Shriya Srinivasan, an MIT postdoc and lead author of the study.</p> <p></p> <p>In a study that will appear this week in the <em>Proceedings of the National Academy of Sciences</em>, 15 patients who received this new type of surgery, known as agonist-antagonist myoneural interface (AMI), could control their muscles more precisely than patients with traditional amputations. The AMI patients also reported feeling more freedom of movement and less pain in their affected limb.</p> <p></p> <p>“Through surgical and regenerative techniques that restore natural agonist-antagonist muscle movements, our study shows that persons with an AMI amputation experience a greater phantom joint range of motion, a reduced level of pain, and an increased fidelity of prosthetic limb controllability,” says Hugh Herr, a professor of media arts and sciences, head of the Biomechatronics group in the Media Lab, and the senior author of the paper.</p> <p></p> <p>Other authors of the paper include Samantha Gutierrez-Arango and Erica Israel, senior research support associates at the Media Lab; Ashley Chia-En Teng, an MIT undergraduate; Hyungeun Song, a graduate student in the Harvard-MIT Program in Health Sciences and Technology; Zachary Bailey, a former visiting researcher at the Media Lab; Matthew Carty, a visiting scientist at the Media Lab; and Lisa Freed, a Media Lab research scientist.</p> <p></p> <p><strong>Restoring sensation</strong></p> <p></p> <p>Most muscles that control limb movement occur in pairs that alternately stretch and contract. One example of these agonist-antagonist pairs is the biceps and triceps. When you bend your elbow, the biceps muscle contracts, causing the triceps to stretch, and that stretch sends sensory information back to the brain.</p> <p>During a conventional limb amputation, these muscle movements are restricted, cutting off this sensory feedback and making it much harder for amputees to feel where their prosthetic limbs are in space or to sense forces applied to those limbs.</p> <p>“When one muscle contracts, the other one doesn’t have its antagonist activity, so the brain gets confusing signals,” says Srinivasan, a former member of the Biomechatronics group now working at MIT’s Koch Institute for Integrative Cancer Research. “Even with state-of-the-art prostheses, people are constantly visually following the prosthesis to try to calibrate their brains to where the device is moving.”</p> <p></p> <p>A few years ago, the MIT Biomechatronics group invented and scientifically developed in <a href="">preclinical studies</a> a new amputation technique that maintains the relationships between those muscle pairs. Instead of severing each muscle, they connect the two ends of the muscles so that they still dynamically communicate with each other within the residual limb. In a 2017 <a href="">study of rats</a>, they showed that when the animals contracted one muscle of the pair, the other muscle would stretch and send sensory information back to the brain.</p> <p></p> <p>Since these preclinical studies, about 25 people have undergone the AMI procedure at Brigham and Women’s Hospital, performed by Carty, a surgeon in the Division of Plastic and Reconstructive Surgery at Brigham and Women’s Hospital. In the new <em>PNAS</em> study, the researchers measured the precision of muscle movements in the ankle and subtalar joints of 15 patients who had AMI amputations performed below the knee. These patients had two sets of muscles reconnected during their amputation: the muscles that control the ankle, and those that control the subtalar joint, which allows the sole of the foot to tilt inward or outward. The study compared these patients to seven people who had traditional amputations below the knee.</p> <p></p> <p>Each patient was evaluated while lying down with their legs propped on a foam pillow, allowing their feet to extend into the air. Patients did not wear prosthetic limbs during the study. The researchers asked them to flex their ankle joints — both the intact one and the “phantom” one — by 25, 50, 75, or 100 percent of their full range of motion. Electrodes attached to each leg allowed the researchers to measure the activity of specific muscles as each movement was performed repeatedly.</p> <p></p> <p>The researchers compared the electrical signals coming from the muscles in the amputated limb with those from the intact limb and found that for AMI patients, they were very similar. They also found that patients with the AMI amputation were able to control the muscles of their amputated limb much more precisely than the patients with traditional amputations. Patients with traditional amputations were more likely to perform the same movement over and over in their amputated limb, regardless of how far they were asked to flex their ankle.</p> <p>“The AMI patients’ ability to control these muscles was a lot more intuitive than those with typical amputations, which largely had to do with the way their brain was processing how the phantom limb was moving,” Srinivasan says.</p> <p>In a <a href="">paper</a> that recently appeared in <em>Science Translational Medicine</em>, the researchers reported that brain scans of the AMI amputees showed that they were getting more sensory feedback from their residual muscles than patients with traditional amputations. In work that is now ongoing, the researchers are measuring whether this ability translates to better control of a prosthetic leg while walking.</p> <p><strong>Freedom of movement</strong></p> <p>The researchers also discovered an effect they did not anticipate: AMI patients reported much less pain and a greater sensation of freedom of movement in their amputated limbs.</p> <p>“Our study wasn’t specifically designed to achieve this, but it was a sentiment our subjects expressed over and over again. They had a much greater sensation of what their foot actually felt like and how it was moving in space,” Srinivasan says. “It became increasingly apparent that restoring the muscles to their normal physiology had benefits not only for prosthetic control, but also for their day-to-day mental well-being.”</p> <p>The research team has also developed a modified version of the surgery that can be performed on people who have already had a traditional amputation. This process, which they call “regenerative AMI,” involves grafting small muscle segments to serve as the agonist and antagonist muscles for an amputated joint. They are also working on developing the AMI procedure for other types of amputations, including <a href="">above the knee</a> and above and below the elbow.</p> <p>“We’re learning that this technique of rewiring the limb, and using spare parts to reconstruct that limb, is working, and it’s applicable to various parts of the body,” Herr says.</p> <p>The research was funded by the MIT Media Lab Consortia; the National Institutes of Health's National Institute of Child Health and Human Development and National Center for Medical Rehabilitation Research; and the Congressionally Directed Medical Research Programs of the U.S. Department of Defense.</p> MIT researchers in collaboration with surgeons at Harvard Medical School have devised a new type of amputation surgery that can help amputees better control their residual muscles and receive sensory feedback. Image courtesy of the researchers Driving on the cutting edge of autonomous vehicle tech Leveraging research done on campus, student-run MIT Driverless partners with industry collaborators to develop and test autonomous technologies in real-world racing scenarios. Thu, 25 Feb 2021 14:40:00 -0500 Daniel de Wolff | MIT Industrial Liaison Program <p>In October, a modified Dallara-15 Indy Lights race car programmed by MIT Driverless will hit the famed Indianapolis Motor Speedway at speeds of up to 120 miles per hour. The Indy Autonomous Challenge (IAC) is the world’s first head-to-head, high-speed autonomous race. It offers MIT Driverless a chance to grab a piece of the $1.5 million purse while outmaneuvering fellow university innovators on what is arguably the most iconic racecourse.</p> <p>But the IAC has implications beyond the track. Stakeholders for the event include Sebastian Thrun, a former winner of the DARPA Grand Challenge for autonomous vehicles, and Reilly Brennan, a lecturer at Stanford University’s Center for Automotive Research and a partner at Trucks Venture Capital. The hosts are well aware that, much like the DARPA Grand Challenge, the IAC has the potential to catalyze a new wave of innovation in the private sector.</p> <p>Formed in 2018 and hosted by the Edgerton Center at MIT, <a href="" target="_blank">MIT Driverless</a> comprises 50 highly motivated engineers with diverse skill sets. The team is intent on learning by doing, pushing the boundaries of the autonomous driving field. “There is so much strategy involved in multiagent autonomous racing, from reinforcement learning to AI and game theory,” says systems architecture lead and chief engineer Nick Stathas, a graduate student in electrical engineering and computer science (EECS). “What excites us the most is coming up with our own approaches to problems in autonomous driving — we're looking to define state-of the-art solutions.”</p> <p>In the lead up to the big day, the team has been testing their algorithms at hackathons and competing in a championship series called RoboRace. The series features 12 races hosted over six events covered by livestream. In this format, MIT Driverless and their competitors program and race a sleek electric vehicle dubbed the DEVBot 2.0. Reminiscent of a Tesla Roadster, the DEVBot was designed specifically to explore the relationship between human and machine.</p> <p>The twist is that RoboRace blends the physical world with a virtual world dubbed the Metaverse. Teams must traverse the track while interacting with an augmented reality replete with virtual obstacles that raise lap times and collectibles that lower them. “Think of it as real-life racing meets Mario Kart,” says Yueyang “Kylie” Ying ’19, a graduate student in EECS who works in the Path Planning division at MIT Driverless.</p> <p>For this challenge, Ying and her teammates have developed a unique planning algorithm they call Spline Racer, which determines if and when their vehicle needs to deviate from the most expedient course around the track to avoid obstacles or collect rewards. “Spline Racer essentially computes potential paths and then chooses the best one to take based on total time to negotiate the path and total cost or reward from bumping into obstacles or collectibles along that path,” explains Ying.</p> <p>MIT is home to cutting-edge research that benefits MIT Driverless whenever the checkered flag is waved. Roboticist and Professor Daniela Rus is just one of their trusted advisors. Rus is director of MIT's Computer Science and Artificial Intelligence Laboratory (CSAIL), the associate director of MIT's Quest for Intelligence Core, and director of the Toyota-CSAIL Joint Research Center, which focuses on the advancement of AI research and its applications to intelligent vehicles.</p> <p>Sertac Karaman of the MIT Department of Aeronautics and Astronautics also serves as an advisor to the team. In addition to pioneering research in controls and robotics theory, Karaman is a co-founder of Optimus Ride, the leading self-driving vehicle technology company developing systems for geo-fenced environments.</p> <p>“One of the competitive advantages of our team is that by virtue of being at MIT, we have firsthand access to a rich concentration of research expertise that we can apply to our own development,” says team captain Jorge Castillo, a graduate student in the MIT Sloan School of Management.</p> <p>Consider the connection between the Han Lab at MIT and MIT Driverless. Assistant professor of electrical engineering and computer science Song Han’s work on efficient computing, particularly his innovative algorithms and hardware systems based on his own deep compression technique for machine learning, is a boon for an autonomous racing team looking to make their algorithms run faster.</p> <p>“Dr. Han is a big fan of MIT Driverless, and he's been extremely helpful,” says Castillo. "We can only put a limited amount of computing in our car,” he explains, "so the faster we can make our algorithms run, the better we will be able to make them and the faster the car will be able to go safely.”</p> <p>Think of MIT Driverless as an essential pit stop in the autonomous knowledge pipeline that flows between the Institute and industry. Their mission is to become the hub of applied autonomy at MIT, leveraging the research done on campus to help their engineers develop a broad skill set that is applicable beyond just the specific use case of autonomous driving.</p> <p>“There are labs at MIT working to solve some of the most complex problems in the world,” says Castillo. “At MIT Driverless, we believe it’s vital to have a place that functions as a proving ground for this research while training the engineers that will help re-imagine the future of the tech industry when it comes to autonomous systems and robotics.”</p> <p>And the MIT Driverless approach to autonomous vehicle racing, particularly as it pertains to architecture and data processing, is similar to the way industry addresses the self-driving problem for streets and highways — which is just one reason why the team has no shortage of industry sponsors who want to get involved. “We have a tight integration between the components that make the car run,” says Stathas. “From a systems perspective, we have well-defined sub-systems that our industry partners appreciate because it aligns with real-world autonomous vehicle development.”</p> <p>In addition to gaining access to some of the most brilliant young talent in the world, industry partners can boost brand awareness while participating in the emerging sport of autonomous racing. “We've formed tight bonds with industry-leading companies," says Castillo. “Very often, our sponsors are our biggest fans. They also place their trust in us and want to recruit from us because our engineers are well equipped to perform in the real world.”</p> Kylie Ying (left) works on path planning and controls for MIT Driverless, while Jorge Castillo serves as team captain. Photo: David Sella An origami-inspired medical patch for sealing internal injuries The patch, which can be folded around surgical tools, may someday be used in robotic surgery to repair tissues and organs. Tue, 02 Feb 2021 00:00:00 -0500 Jennifer Chu | MIT News Office <p>Many surgeries today are performed via minimally invasive procedures, in which a small incision is made and miniature cameras and surgical tools are threaded through the body to remove tumors and repair damaged tissues and organs. The process results in less pain and shorter recovery times compared to open surgery.</p> <p>While many procedures can be performed in this way, surgeons can face challenges at an important step in the process: the sealing of internal wounds and tears.</p> <p>Taking inspiration from origami, MIT engineers have now designed a medical patch that can be folded around minimally invasive surgical tools and delivered through airways, intestines, and other narrow spaces, to patch up internal injuries. The patch resembles a foldable, paper-like film when dry. Once it makes contact with wet tissues or organs, it transforms into a stretchy gel, similar to a contact lens, and can stick to an injured site.</p> <p>In contrast to existing surgical adhesives, the team’s new tape is designed to resist contamination when exposed to bacteria and bodily fluids. Over time, the patch can safely biodegrade away. The team has published its results in the journal <em>Advanced Materials</em>.</p> <div></div> <p>The researchers are working with clinicians and surgeons to optimize the design for surgical use, and they envision that the new bioadhesive could be delivered via minimally invasive surgical tools, operated by a surgeon either directly or remotely via a medical robot.</p> <p>“Minimally invasive surgery and robotic surgery are being increasingly adopted, as they decrease trauma and hasten recovery related to open surgery. However, the sealing of internal wounds is challenging in these surgeries,” says Xuanhe Zhao, professor of mechanical engineering and of civil and environmental engineering at MIT.</p> <p>“This patch technology spans many fields,” adds co-author Christoph Nabzdyk, a cardiac anesthesiologist and critical care physician at the Mayo Clinic in Rochester, Minnesota. “This could be used to repair a perforation from a coloscopy, or seal solid organs or blood vessels after a trauma or elective surgical intervention. Instead of having to carry out a full open surgical approach, one could go from the inside to deliver a patch to seal a wound at least temporarily and maybe even long-term.”</p> <p>The study’s co-authors include lead authors Sarah Wu and Hyunwoo Yuk, and Jingjing Wu at MIT.</p> <p><strong>Layered protection</strong></p> <p>The bioadhesives currently used in minimally invasive surgeries are available mostly as biodegradable liquids and glues that can be spread over damaged tissues. When these glues solidify, however, they can stiffen over the softer underlying surface, creating an imperfect seal. Blood and other biological fluids can also contaminate glues, preventing successful adhesion to the injured site. Glues can also wash away before an injury has fully healed, and, after application, they can also cause inflammation and scar tissue formation.</p> <p>Given the limitations of current designs, the team aimed to engineer an alternative that would meet three functional requirements. It should be able to stick to the wet surface of an injured site, avoid binding to anything before reaching its destination, and once applied to an injured site resist bacterial contamination and excessive inflammation.</p> <p>The team’s design meets all three requirements, in the form of a three-layered patch. The middle layer is the main bioadhesive, made from a hydrogel material that is embedded with compounds called NHS esters. When in contact with a wet surface, the adhesive absorbs any surrounding water and becomes pliable and stretchy, molding to a tissue’s contours. Simultaneously, the esters in the adhesive form strong covalent bonds with compounds on the tissue surface, creating a tight seal between the two materials. The design of this middle layer is based on previous work in Zhao’s group.</p> <p>The team then sandwiched the adhesive with two layers, each with a different protective effect. The bottom layer is made from a material coated with silicone oil, which acts to temporarily lubricate the adhesive, preventing it from sticking to other surfaces as it travels through the body. When the adhesive reaches its destination and is pressed lightly against an injured tissue, the silicone oil is squeezed out, allowing the adhesive to bind to the tissue.&nbsp;</p> <p>The adhesive’s top layer consists of an elastomer film embedded with zwitterionic polymers, or molecular chains made from both positive and negative ions that act to attract any surrounding water molecules to the elastomer’s surface. In this way, the adhesive’s outward-facing layer forms a water-based skin, or barrier against bacteria and other contaminants.</p> <p>“In minimally invasive surgery, you don’t have the luxury of easily accessing a site to apply an adhesive,” Yuk says. “You really are battling a lot of random contaminants and body fluids on your way to your destination.”</p> <p><strong>Fit for robots</strong></p> <p>In a series of demonstrations, the researchers showed that the new bioadhesive strongly adheres to animal tissue samples, even after being submerged in beakers of fluid, including blood, for long periods of time.</p> <p>They also used origami-inspired techniques to fold the adhesive around instruments commonly used in minimally invasive surgeries, such as a balloon catheter and a surgical stapler. They threaded these tools through animal models of major airways and vessels, including the trachea, esophagus, aorta, and intestines. By inflating the balloon catheter or applying light pressure to the stapler, they were able to stick the patch onto torn tissues and organs, and found no signs of contamination on or near the patched-up site up to one month after its application.</p> <p>The researchers envision that the new bioadhesive could be manufactured in prefolded configurations that surgeons can easily fit around minimally invasive instruments as well as on tools that are currently being used in robotic surgery. They are seeking to collaborate with designers to integrate the bioadhesive into robotic surgery platforms.</p> <p>“We believe that the conceptual novelty in the form and function of this patch represents an exciting step toward overcoming translational barriers in robotic surgery and facilitating the wider clinical adoption of bioadhesive materials,” Wu says.</p> <p>This research was supported, in part, by the National Science Foundation.</p> MIT engineers have design paper-like medical tapes (shown here) that can fold around surgical tools and transform into soft, strong adhesives when pressed against tissues. Credit: Felice Frankel Healing with hydrogels Inspired by personal tragedy, graduate student Hyunwoo Yuk used his background in soft materials to develop a bioadhesive tape for repairing damaged tissue. Thu, 25 Feb 2021 00:00:00 -0500 Mary Beth Gallagher | Department of Mechanical Engineering <p>In November, mechanical engineering PhD candidate Hyunwoo Yuk earned the top prize at the Collegiate Inventors Competition hosted by the National Inventor’s Hall of Fame. Yuk was named the graduate winner for his invention <a href="">SanaHeal</a>, a bioadhesive tape that can easily bind to tissues or organs. The tape could one day be used in place of sutures to promote healing and minimize complications after surgery.</p> <p>Yuk accepted the prize a few weeks prior to successfully defending his doctoral thesis last December. These accomplishments marked the culmination of a personal journey that has its root in family tragedy.</p> <p>As he was completing his bachelor’s degree, Yuk received a call that his brother was involved in a horrific accident. He suffered multiple traumatic injuries and required intensive care. Yuk spent the next two years by his brother’s side as he was ushered in and out of operating rooms and intensive care units.</p> <p>“I’m 10 years older than my little brother, so I view him much like a son. Seeing what he went through was pure pain,” says Yuk.</p> <p>The years spent with his brother in the hospital gave Yuk insight into the problems and limitations of medical technologies. He started approaching these problems from the perspective of an engineer.</p> <p>“I believe that personal problems are the best problems for engineers to solve,” adds Yuk. “If there is a chance that I can do something meaningful as an inventor to solve the problems my brother faced, there is no better motivation.”</p> <p><strong>Working with soft materials</strong></p> <p>Before his brother’s accident, Yuk was focused primarily on bioinspired robotics during his undergraduate studies at the Korea Advanced Institute of Science and Technology (KAIST). While serving in the Korean military, Yuk was stationed at a local school to help children with special needs. Working with children with limited motor skills inspired Yuk to apply to MIT for graduate school to build robots for use in rehabilitation.</p> <p>Shortly after being accepted into MIT’s mechanical engineering graduate program, Yuk received an email from a new faculty member, Xuanhe Zhao, now professor of mechanical engineering and George N. Hatsopoulos (1949) Faculty Fellow. An expert in the field of soft materials, Zhao was looking for a graduate student to take the lead on understanding and design of soft materials, especially hydrogels.</p> <p>Despite being a complete newcomer to the world of soft materials, Yuk quickly found excitement from the interesting properties of soft materials and their unique potential to bridge artificial machines and devices to the human body. Through their research on how humans and machines interface, Zhao and Yuk began exploring the adhesive properties of hydrogels.</p> <p>“Adhesion of soft materials, especially hydrogels, greatly attracted our research interests, because the state-of-the-art adhesion of hydrogels was fairly weak and often led to failures,” Zhao says. “In 2015, we proposed the first <a href="">general strategy</a> to achieve tough adhesion for diverse hydrogels and other materials.”</p> <p>Like humans, hydrogels are made out of polymers and water. The similarities in mechanical and electrical properties make biological tissue and hydrogels compatible, opening up a world of biomedical applications. Now, Zhao and Yuk could further integrate hydrogels within various materials, devices, and even the human body with unprecedented robustness.</p> <p>“We gradually started realizing that one major impact of hydrogels and hydrogel adhesion lies in biomedicine, but we needed someone who could help us better understand unmet clinical demands,” explains Yuk.</p> <p>That role was filled by Christoph Nabzdyk, a critical care physician, cardiothoracic anesthesiologist, and assistant professor at the Mayo Clinic.</p> <p><strong>Understanding clinical needs</strong></p> <p>In 2017, the research team was approached by Nabzdyk, who at the time was a fellow in critical care medicine and cardiothoracic anesthesiology at Massachusetts General Hospital and Harvard University. Nabzdyk had research experience in wound healing and novel biomaterials. After seeing Zhao present at a conference, Nabzdyk saw an opportunity to apply Zhao and Yuk’s work on bioadhesive hydrogels to tissue and organ healing.</p> <p>“Despite decades of research, the most common modes of sealing tissue defects are sutures and staples — both rather antiquated and traumatic approaches to tissue approximation,” says Nabzdyk. “The bioadhesive work that Hyunwoo and Xuanhe have been developing has broad implications for various scenarios of organ defect repairs, hemostasis, and implantable adhesive electronics.”</p> <p>Using Nabzdyk’s connections, Yuk conducted over 30 interviews with surgeons. These interviews with end users reminded Yuk of his two years in hospitals with his brother.</p> <p>“I realized that so many of the problems my brother experienced, like uncontrollable bleeding and dozens of scars, could possibly have been solved with these hydrogel materials,” adds Yuk.</p> <p>The research team started exploring the mechanics of sealing wet tissues. Together with Ellen Roche, assistant professor of mechanical engineering, and graduate student Claudia Varela, they created a bioadhesive patch that can seal organs in a matter of seconds. This double-sided tape could potentially replace sutures, preventing the leakage of blood and reducing the risk of infection, pain, and scarring.</p> <p>“Working on this research is personally very fulfilling. I feel like I have found my purpose as an engineer,” says Yuk.</p> <p>He admits that to have true impact, the bioadhesive tape needs to move from the laboratory to the operating room.</p> <p><strong>Focus on social impact</strong></p> <p>To navigate the uncharted territory of regulations, standards, and investors, Yuk, Zhao, and Nabzdyk formed a team, named SanaHeal, that sought guidance and seed funding from organizations including the MIT Deshpande Center for Technological Innovation and MIT Venture Mentoring Service. These programs helped Yuk polish his ability to communicate about SanaHeal, a skill set that helped him win the graduate award at the Collegiate Inventors Competition.</p> <p>In addition to winning at the Collegiate Inventors Competition, Yuk has received a number of accolades including being named one of Forbes 30 Under 30, receiving the Materials Research Society Gold Graduate Student Award, and getting first place at MIT’s de Florez Awards. According to his collaborators, these honors are well-deserved.</p> <p>“Driven by the genuine need to improve his younger brother’s health, Hyunwoo has been working long hours over the last six-and-a-half years — co-filing over 10 patent applications, leading or co-leading over 15 papers including two in <em>Nature</em>, and motivating and caring for other team members.” Zhao says. “I cannot wait to see how Hyunwoo and SanaHeal will impact his family, other patients, and the world.”</p> <p>“Hyunwoo stands out even amongst his already brilliant peers. I have not worked with anyone who at such an early stage in their career has had such a long list of academic accomplishments and advanced level of scientific thinking,” Nabzdyk adds. “Despite all his hard work and drive, Hyunwoo himself is a sweet and caring person, a gentle and humble soul.”</p> <p>With his doctoral studies coming to a close, Yuk is looking to expand SanaHeal so it can have real impact in saving lives or helping those who, like his brother, have suffered traumatic injury.</p> <p>“It feels like we are ready to step forward to the real battlefield,” he says. “I’ve grown from the mindset of an academic research to someone who is more translationally minded and focused on the social impact our work can have.”</p> Mechanical engineering graduate student Hyunwoo Yuk has used his expertise in soft materials to develop a hydrogel bandage that promotes healing in people who, like his brother, have suffered a traumatic injury or undergone surgery. Photo: John Freidah Learning with — and about — AI technology At Open Learning Talks, Cynthia Breazeal and Eric Klopfer discuss artificial intelligence education. Mon, 25 Jan 2021 14:30:00 -0500 MIT Open Learning <p>Between remote learning, more time spent at home, and working parents trying to keep their kids occupied, children across the United States have clocked in record-breaking hours of screen time during the pandemic. Much of it is supervised and curated by teachers or parents — but increasingly, kids of all ages are watching videos, playing games, and interacting with devices powered by artificial intelligence. As head of the Personal Robots group and AI Education at MIT, Media Lab Professor Cynthia Breazeal is on a mission to help this generation of young people to grow up understanding the AI they use.</p> <p>At “AI Education: Research and Practice,” an <a href="">Open Learning Talks</a> event in December, Breazeal shared her vision for educating students not only about how AI works, but how to design and use it themselves — an initiative she calls AI Literacy for All. The <a href="">AI Education</a> project Breazeal is leading at MIT is a collaboration between MIT Open Learning and the Abdul Latif Jameel World Education Lab, the Media Lab, and the MIT Schwarzman College of Computing. Through research projects, hands-on activities, and scalable learning modules, Breazeal and her AI Education affiliates across MIT are creating a robust resource hub for educators, parents, and learners of all ages to understand how AI functions in different day-to-day roles, and how to approach both using and creating artificial intelligence with a basis in ethics, inclusion, and empathy.</p> <p>“It’s at this intersection of human psychology, engagement, and AI and technology, and we’re learning a lot,” Breazeal said as she explained her group’s research to the audience. “We’re not trying to build technologies to replace teachers or compete with parents. These are fluffy, pet-like robots, but they can engage children in this interaction where there are aspects like a motivating ally, like a friend ... there are aspects like this companion animal, and this nonjudgmental companion animal gives the nature of this relationship this very different flavor, where even if they’re embarrassed to make mistakes in front of their teacher or their friends, they seem not to be in front of the robot — and you can’t learn if you’re not willing to take learning risks.”&nbsp;</p> <p>Breazeal shared examples from her Personal Robots group’s efforts, including recent studies on personalized learning companions for early childhood education, developing comprehensive K-12 AI literacy programs, and creating tools to help kids get creative using AI technologies.</p> <p>“So how do you empower kids to create things with AI? You’re not going to put a middle-schooler on Tensorflow and say ‘Good luck,’ right?” Breazeal said. “MIT is the home base for things like Scratch and App Inventor, so the team is taking these more advanced AI methods and curricula and concepts, and augmenting these platforms to empower kids to use these AI technologies, to learn about them and then design projects of their own, and port them to different kinds of platforms.”&nbsp;&nbsp;</p> <p>Host Professor Eric Klopfer, director of the Scheller Teacher Education Program and the Education Arcade at MIT and head of MIT Comparative Media Studies and Writing, engaged Breazeal in a dialogue about all aspects of AI education and fielded questions from the live audience, ranging from emotional connection with robots to screen time, data collection, and representation in research and design.&nbsp;</p> <p>“How does AI in education narrow the gap that we see between socioeconomic groups? How do we see AI bridging that gap rather than widening the gap?” asked Klopfer, as he and Breazeal shared insights on training teachers, providing hands-on activities and paper prototyping to expand access and inclusion on technology education. “The technology itself is not the impetus for the divide anymore; it’s the way the technologies are being used, and the way people are trained to be able to use them,” Klopfer said. “It’s so key that we don’t repeat our mistakes from past technological innovations, where we just distribute devices to schools without thinking about the training and expertise that needs to go with that.”</p> <p>And in an increasingly tech-driven society, access and education are key to creating equity for, and encouraging thoughtful participation from, all users. “We want a much more diverse, inclusive group of people being able to participate in shaping this future [with AI],” said Breazeal.</p> <p>Launched last fall, Open Learning Talks is a public, online event series that features conversations between leaders from MIT and around the world, sharing their research and insights on education, teaching, and the science of learning. Upcoming events include William Bonvillian and Sanjay Sarma discussing their new book, “Workforce Education,” on Feb. 23; and Professor D. Fox Harrell and Rocky Bucano, executive director of the Universal Hip Hop Museum, in mid-March.&nbsp;</p> During an Open Learning Talks event on artificial intelligence education, Cynthia Breazeal shared a video of a research study on personalized learning companions for early childhood. Photo courtesy of the Personal Robots group, MIT Media Lab. Designing customized “brains” for robots A new system devises hardware architectures to hasten robots’ response time. Thu, 21 Jan 2021 00:00:00 -0500 Daniel Ackerman | MIT News Office <p>Contemporary robots <em>can</em> move quickly. “The motors are fast, and they’re powerful,” says Sabrina Neuman.</p> <p></p> <p>Yet in complex situations, like interactions with people, robots often <em>don’t</em> move quickly. “The hang up is what’s going on in the robot’s head,” she adds.</p> <p></p> <p>Perceiving stimuli and calculating a response takes a “boatload of computation,” which limits reaction time, says Neuman, who recently graduated with a PhD from the MIT Computer Science and Artificial Intelligence Laboratory (CSAIL). Neuman has found a way to fight this mismatch between a robot’s “mind” and body. The method, called robomorphic computing, uses a robot’s physical layout and intended applications to generate a customized computer chip that minimizes the robot’s response time.</p> <p></p> <p>The advance could fuel a variety of robotics applications, including, potentially, frontline medical care of contagious patients. “It would be fantastic if we could have robots that could help reduce risk for patients and hospital workers,” says Neuman.</p> <p></p> <p>Neuman will present the research at this April’s International Conference on Architectural Support for Programming Languages and Operating Systems. MIT co-authors include graduate student Thomas Bourgeat and Srini Devadas, the Edwin Sibley Webster Professor of Electrical Engineering and Neuman’s PhD advisor. Other co-authors include Brian Plancher, Thierry Tambe, and Vijay Janapa Reddi, all of Harvard University. Neuman is now a postdoctoral NSF Computing Innovation Fellow at Harvard’s School of Engineering and Applied Sciences.</p> <p></p> <p>There are three main steps in a robot’s operation, according to Neuman. The first is perception, which includes gathering data using sensors or cameras. The second is mapping and localization: “Based on what they’ve seen, they have to construct a map of the world around them and then localize themselves within that map,” says Neuman. The third step is motion planning and control — in other words, plotting a course of action.</p> <p></p> <p>These steps can take time and an awful lot of computing power. “For robots to be deployed into the field and safely operate in dynamic environments around humans, they need to be able to think and react very quickly,” says Plancher. “Current algorithms cannot be run on current CPU hardware fast enough.”</p> <p></p> <p>Neuman adds that researchers have been investigating better algorithms, but she thinks software improvements alone aren’t the answer. “What’s relatively new is the idea that you might also explore better hardware.” That means moving beyond a standard-issue CPU processing chip that comprises a robot’s brain — with the help of hardware acceleration.</p> <p></p> <p>Hardware acceleration refers to the use of a specialized hardware unit to perform certain computing tasks more efficiently. A commonly used hardware accelerator is the graphics processing unit (GPU), a chip specialized for parallel processing. These devices are handy for graphics because their parallel structure allows them to simultaneously process thousands of pixels. “A GPU is not the best at everything, but it’s the best at what it’s built for,” says Neuman. “You get higher performance for a particular application.” Most robots are designed with an intended set of applications and could therefore benefit from hardware acceleration. That’s why Neuman’s team developed robomorphic computing.</p> <p></p> <p>The system creates a customized hardware design to best serve a particular robot’s computing needs. The user inputs the parameters of a robot, like its limb layout and how its various joints can move. Neuman’s system translates these physical properties into mathematical matrices. These matrices are “sparse,” meaning they contain many zero values that roughly correspond to movements that are impossible given a robot’s particular anatomy. (Similarly, your arm’s movements are limited because it can only bend at certain joints — it’s not an infinitely pliable spaghetti noodle.)</p> <p></p> <p>The system then designs a hardware architecture specialized to run calculations only on the non-zero values in the matrices. The resulting chip design is therefore tailored to maximize efficiency for the robot’s computing needs. And that customization paid off in testing.</p> <p></p> <p>Hardware architecture designed using this method for a particular application outperformed off-the-shelf CPU and GPU units. While Neuman’s team didn’t fabricate a specialized chip from scratch, they programmed a customizable field-programmable gate array (FPGA) chip according to their system’s suggestions. Despite operating at a slower clock rate, that chip performed eight times faster than the CPU and 86 times faster than the GPU.</p> <p></p> <p>“I was thrilled with those results,” says Neuman. “Even though we were hamstrung by the lower clock speed, we made up for it by just being more efficient.”</p> <p></p> <p>Plancher sees widespread potential for robomorphic computing. “Ideally we can eventually fabricate a custom motion-planning chip for every robot, allowing them to quickly compute safe and efficient motions,” he says. “I wouldn't be surprised if 20 years from now every robot had a handful of custom computer chips powering it, and this could be one of them.” Neuman adds that robomorphic computing might allow robots to relieve humans of risk in a range of settings, such as caring for covid-19 patients or manipulating heavy objects.</p> <p></p> <p>“This work is exciting because it shows how specialized circuit designs can be used to accelerate a core component of robot control,” says Robin Deits, a robotics engineer at Boston Dynamics who was not involved in the research. “Software performance is crucial for robotics because the real world never waits around for the robot to finish thinking.” He adds that Neuman’s advance could enable robots to think faster, “unlocking exciting behaviors that previously would be too computationally difficult.”</p> <p></p> <p>Neuman next plans to automate the entire system of robomorphic computing. Users will simply drag and drop their robot’s parameters, and “out the other end comes the hardware description. I think that’s the thing that’ll push it over the edge and make it really useful.”</p> <p></p> <p>This research was funded by the National Science Foundation, the Computing Research Agency, the CIFellows Project, and the Defense Advanced Research Projects Agency.</p> MIT researchers have developed an automated way to design customized hardware, or “brains,” that speeds up a robot’s operation. Image: Jose-Luis Olivares, MIT MIT to share in $3.2 million grant to create a statewide technician-training program in advanced manufacturing MassBridge will create curriculum for the advanced manufacturing needs of today’s economy. Wed, 16 Dec 2020 16:50:00 -0500 Julie Diop | George Westerman | Department of Materials Science and Engineering | Office of Open Learning <p>At the end of October, the Commonwealth of Massachusetts announced it won a $3.2 million, two-year grant, in collaboration with MIT, community colleges, and state agencies, to prepare workers for stable high-paying jobs in advanced manufacturing. The program, called MassBridge, will create a curriculum that bridges between the Commonwealth’s excellent traditional manufacturing education and the advanced manufacturing needs of today’s economy. Massachusetts will serve as a foundry and pilot for this curriculum, which can later be used by other states to meet the nation’s growing need for these skills.</p> <p>Manufacturing is sometimes seen as an unattractive industry for work, where jobs are dirty, dull, and dingy. But the new wave of advanced manufacturing — in areas like robotics, photonics, and 3D printing — are a completely different experience. These are well-paid “middle-skill” jobs that can provide stable employment, good working conditions, and opportunities for advancement. Some estimates say that more than 2 million of these jobs will go unfilled over the next decade. Potential workers need to know about the opportunities, and educational institutions need to build programs that prepare people for these careers.&nbsp;</p> <p>When the U.S. Department of Defense’s Manufacturing Technology office first considered working with a state on middle-skill worker training, the Massachusetts Technology Collaborative, or MassTech, quickly put together a plan. The commonwealth had many elements that made it a strong candidate. Massachusetts had created four regions — Northeast, Southeast, Central, and Western Massachusetts — which worked together to standardize training across the commonwealth. The governor’s skills cabinet coordinated training-related activities across the many branches of state government. And finally, the commonwealth had already committed $100 million in capital equipment grants to support the defense department’s Manufacturing-USA institutes.</p> <p>Workforce training activities for one of these institutes, AIM Photonics, is led by MIT Professor Lionel Kimerling. Kimerling’s group IKIM (Initiative for Knowledge and Innovation in Manufacturing) had already won an Office of Naval Research grant to develop technician-training programs in robotics and photonics, and IKIM was also developing online classes. MassTech enlisted IKIM’s help.</p> <p>Since the program will rely extensively on digital learning modules, MassTech also turned to the MIT Office of Open Learning (OL). MIT OL has extensive expertise in digital education and learning science. It also has an ongoing commitment to workforce learning through its Abdul Latif Jameel World Education Lab and several educational programs.</p> <p>“This award is an important national acknowledgment of Massachusetts’ successes in manufacturing and the competitive training programs that support our industry,” says Secretary Mike Kennealy of the Executive Office of Housing and Economic Development. “The grant from the Department of Defense presents a major opportunity to supercharge our manufacturing sector by engaging new students and adult learners, and helping them develop their skills to better succeed in these emerging industries.”</p> <p>MIT’s IKIM and OL will be involved in almost all aspects of the MassBridge project, and they will lead the work on skills roadmapping and curriculum benchmarking. “MIT is well-known for its deep expertise in engineering. We are also known for the pioneering work we’ve done in digital and blended learning. We see our mission as educating not only our students, but also outside MIT,” says Professor Sanjay Sarma, vice president of open learning. “Combining the expertise of MIT, the Commonwealth, and our community college collaborators can make a real difference in the lives of people who do not yet have the skills for well-paying and fulfilling careers.”&nbsp;&nbsp;</p> <p>“The Commonwealth of Massachusetts has a clear vision for recovery of its manufacturing sector by leveraging its science and engineering leadership,” says Kimerling. “MassTech has been a valued partner for MIT, identifying a strong technician workforce as the missing ingredient to success. Our team is privileged and delighted to share our advanced manufacturing knowledge and to engage with outstanding students and faculty in building the vital technician workforce.”</p> <p>Other collaborators in the program team include Cape Cod Community College, Quinsigamond Community College, Massachusetts Manufacturing Extension Program, Massachusetts hiring boards, and representatives from state government and the Department of Defense. After initially building and testing the curriculum in Massachusetts, the team plans to make the curriculum available nationwide to any states that wish to adopt it.</p> Students and faculty from Springfield Technical Community College visit an MIT lab before the Covid-19 pandemic. Photo: Anu Agarwal Computer-aided creativity in robot design MIT researchers’ new system optimizes the shape of robots for traversing various terrain types. Mon, 30 Nov 2020 00:00:00 -0500 Daniel Ackerman | MIT News Office <p>So, you need a robot that climbs stairs. What shape should that robot be? Should it have two legs, like a person? Or six, like an ant?</p> <p>Choosing the right shape will be vital for your robot’s ability to traverse a particular terrain. And it’s impossible to build and test every potential form. But now an MIT-developed system makes it possible to simulate them and determine which design works best.</p> <p>You start by telling the system, called RoboGrammar, which robot parts are lying around your shop — wheels, joints, etc. You also tell it what terrain your robot will need to navigate. And RoboGrammar does the rest, generating an optimized structure and control program for your robot.</p> <p>The advance could inject a dose of computer-aided creativity into the field. “Robot design is still a very manual process,” says Allan Zhao, the paper’s lead author and a PhD student in the MIT Computer Science and Artificial Intelligence Laboratory (CSAIL). He describes RoboGrammar as “a way to come up with new, more inventive robot designs that could potentially be more effective.”</p> <p>Zhao is the lead author of the paper, which he will present at this month’s SIGGRAPH Asia conference. Co-authors include PhD student Jie Xu, postdoc Mina Konaković-Luković, postdoc Josephine Hughes, PhD student Andrew Spielberg, and professors Daniela Rus and Wojciech Matusik, all of MIT.</p> <p><strong>Ground rules</strong></p> <p>Robots are built for a near-endless variety of tasks, yet “they all tend to be very similar in their overall shape and design,” says Zhao. For example, “when you think of building a robot that needs to cross various terrains, you immediately jump to a quadruped,” he adds, referring to a four-legged animal like a dog. “We were wondering if that’s really the optimal design.”</p> <p>Zhao’s team speculated that more innovative design could improve functionality. So they built a computer model for the task — a system that wasn’t unduly influenced by prior convention. And while inventiveness was the goal, Zhao did have to set some ground rules.</p> <p>The universe of possible robot forms is “primarily composed of nonsensical designs,” Zhao writes in the paper. “If you can just connect the parts in arbitrary ways, you end up with a jumble,” he says. To avoid that, his team developed a “graph grammar” — a set of constraints on the arrangement of a robot’s components. For example, adjoining leg segments should be connected with a joint, not with another leg segment. Such rules ensure each computer-generated design works, at least at a rudimentary level.</p> <p>Zhao says the rules of his graph grammar were inspired not by other robots but by animals — arthropods in particular. These invertebrates include insects, spiders, and lobsters. As a group, arthropods are an evolutionary success story, accounting for more than 80 percent of known animal species. “They’re characterized by having a central body with a variable number of segments. Some segments may have legs attached,” says Zhao. “And we noticed that that’s enough to describe not only arthropods but more familiar forms as well,” including quadrupeds. Zhao adopted the arthropod-inspired rules thanks in part to this flexibility, though he did add some mechanical flourishes. For example, he allowed the computer to conjure wheels instead of legs.</p> <p><strong>A phalanx of robots</strong></p> <p>Using Zhao’s graph grammar, RoboGrammar operates in three sequential steps: defining the problem, drawing up possible robotic solutions, then selecting the optimal ones. Problem definition largely falls to the human user, who inputs the set of available robotic components, like motors, legs, and connecting segments. “That’s key to making sure the final robots can actually be built in the real world,” says Zhao. The user also specifies the variety of terrain to be traversed, which can include combinations of elements like steps, flat areas, or slippery surfaces.</p> <p>With these inputs, RoboGrammar then uses the rules of the graph grammar to design hundreds of thousands of potential robot structures. Some look vaguely like a racecar. Others look like a spider, or a person doing a push-up. “It was pretty inspiring for us to see the variety of designs,” says Zhao. “It definitely shows the expressiveness of the grammar.” But while the grammar can crank out quantity, its designs aren’t always of optimal quality.</p> <p>Choosing the best robot design requires controlling each robot’s movements and evaluating its function. “Up until now, these robots are just structures,” says Zhao. The controller is the set of instructions that brings those structures to life, governing the movement sequence of the robot’s various motors. The team developed a controller for each robot with an algorithm called Model Predictive Control, which prioritizes rapid forward movement.</p> <p>“The shape and the controller of the robot are deeply intertwined,” says Zhao, “which is why we have to optimize a controller for every given robot individually.” Once each simulated robot is free to move about, the researchers seek high-performing robots with a “graph heuristic search.” This neural network algorithm iteratively samples and evaluates sets of robots, and it learns which designs tend to work better for a given task. “The heuristic function improves over time,” says Zhao, “and the search converges to the optimal robot.”</p> <p>This all happens before the human designer ever picks up a screw.</p> <p>“This work is a crowning achievement in the a 25-year quest to automatically design the morphology and control of robots,” says Hod Lipson, a mechanical engineer and computer scientist at Columbia University, who was not involved in the project. “The idea of using shape-grammars has been around for a while, but nowhere has this idea been executed as beautifully as in this work. Once we can get machines to design, make and program robots automatically, all bets are off.”</p> <p>Zhao intends the system as a spark for human creativity. He describes RoboGrammar as a “tool for robot designers to expand the space of robot structures they draw upon.” To show its feasibility, his team plans to build and test some of RoboGrammar’s optimal robots in the real world. Zhao adds that the system could be adapted to pursue robotic goals beyond terrain traversing. And he says RoboGrammar could help populate virtual worlds. “Let’s say in a video game you wanted to generate lots of kinds of robots, without an artist having to create each one,” says Zhao. “RoboGrammar would work for that almost immediately.”</p> <p>One surprising outcome of the project? “Most designs did end up being four-legged in the end,” says Zhao. Perhaps manual robot designers were right to gravitate toward quadrupeds all along. “Maybe there really is something to it.”</p> MIT researchers have automated and optimized robot design with a system called RoboGrammar. The system creates arthropod-inspired robots for traversing a variety of terrains. Pictured are several robot designs generated with RoboGrammar. Courtesy of the researchers Why we shouldn’t fear the future of work MIT task force wraps up with a final conference, sounds note of optimism that new ideas, better policies can help sustain good careers. Fri, 20 Nov 2020 14:00:00 -0500 Peter Dizikes | MIT News Office <p>The American workforce is at a crossroads. Digitization and automation have replaced millions of middle-class jobs, while wages have stagnated for many who remain employed. A lot of labor has become insecure, low-income freelance work.</p> <p></p> <p>Yet there is reason for optimism on behalf of workers, as scholars and business leaders outlined in an MIT conference on Wednesday. Automation and artificial intelligence do not just replace jobs; they also create them. And many labor, education, and safety-net policies could help workers greatly as well.</p> <p></p> <p>That was the outlook of many participants at the <a href="">conference</a>, the “AI and the Work of the Future Congress,” marking the release of the final <a href="">report</a> of MIT’s Task Force on the Work of the Future. The report concludes that there is no technology-driven jobs wipeout on the horizon, but new policies are needed to match the steady march of innovation; technology has mostly helped white-collar workers, but not the rest of the work force in the U.S.</p> <p></p> <p>“We’re not going to run out of work,” Elisabeth Beck Reynolds, executive director of the task force, and executive director of the MIT Industrial Performance Center, said Wednesday.</p> <p></p> <p>She added: “Clearly the distributional effects of technological change are uneven. We’ve seen the reduction of middle-skill jobs [due] to automation, [along with] jobs in manufacturing, administration, in clerical work, while we’ve seen an increase in jobs for those with higher education and higher skill sets. … Our challenge is to try to train [workers] and make sure we have workers in good positions for those jobs.”</p> <p></p> <p>Indeed, the notion of social responsibility was a leading motif of the conference, which drew an audience of about 1,500 online viewers.&nbsp;</p> <p></p> <p>“I believe that those of us who are technologists, and who educate tomorrow’s technologists, have a special role to play,” said MIT President L. Rafael Reif, in his introductory remarks at the conference. “It means that, while we are teaching students, in every field, to be fluent in the use of AI strategies and tools, we must be sure that we equip tomorrow’s technologists with equal fluency in the cultural values and ethical principles that should ground and govern how those tools are designed and how they’re used.”</p> <p></p> <p>The daylong event was organized by MIT’s Task Force on the Work of the Future, along with the Initiative on the Digital Economy and the Computer Science and Artificial Intelligence Laboratory.</p> <p><strong>Conditions on the ground</strong></p> <p>The report notes that over the last four decades, innovation has driven increases in productivity, but that earnings have not followed in step. Since 1978, overall U.S. productivity has risen by 66 percent; yet over the same time, compensation for production and nonsupervisory workers has only risen by 10 percent.</p> <p></p> <p>“Work has become a lot more fragile,” said James Manyika, a senior partner at the consulting firm McKinsey and Company, chair of the McKinsey Global Institute, and a member of McKinsey's board of directors. “This has affected both middle-wage and lower-wage workers.”</p> <p></p> <p>To be sure, information technology in particular has helped people in engineering, design, medicine, marketing, and many other white-collar fields; and while middle-income jobs have become more scarce, service-sector jobs have expanded but tend to be lower-income.</p> <p></p> <p>“Certainly the United States is a good place for high-wage workers to be, but not for lower-wage [workers] and those in the middle,” said Susan Houseman, vice president and director of research at the W.E. Upjohn Institute for Employment Research. “We should be concerned about the growth of nontraditional work arangements.”</p> <p></p> <p>Moreover, “The U.S. doesn’t seem to be getting a very positive return on its inequality,” said David Autor, the Ford Professor of Economics at MIT, associate head of MIT’s Department of Economics, and a co-chair of the task force. “That is, we have a lot of inequality, but we do not have faster growth.”</p> <p></p> <p>In general, most workers are “not seeming to share in the prosperity that improved technology has got us,” said Robert M. Solow, Institute Professor Emeritus and 1987 Nobel laureate in economics, in recorded remarks shown during the conference.</p> <p></p> <p>That said, Solow observed, “There’s room for a lot of ingenuity here, because since the nature of employment has changed, as we become a service economy rather than a goods-producing economy, there’s room for innovation in how to organize union work. … More active enforcement of antitrust laws, to try to increase the degree of competition in the production of goods and services, would also have the effect of improving the prospects for wages and salaries.”</p> <p></p> <p>He added: “The main factor in the disturbance in the distribution of incomes is probably not technological change.”</p> <p></p> <p><strong>What are the next steps?</strong></p> <p></p> <p>But if there is room for policy interventions to ease the social jolts resulting from technology, which ones make the most sense? In general terms, some conference participants advocated for an openness to market-driven technological change, paired with a substantial safety net to help people handle those disruptive waves of innovation.</p> <p></p> <p>“The real fundamental shift is, we have to think of service jobs the way 100 years ago we thought about manufacturing jobs. In other words, we have to start putting in place … protections and benefits,” said Fareed Zakaria, author and host of the CNN show, “Fareed Zakaria GPS.” He added, “Ultimately, that is the only way you are going to really address this problem. We are not going to bring back tens of millions of manufacturing jobs to the United States. We are going to take these service jobs and make them better jobs. And companies can do that.”</p> <p></p> <p>One conference panel focused on the support of education, particularly public universities and community colleges, where traditionally overlooked pools of workplace talent reside.</p> <p></p> <p>“One of the most important skills or approaches that we need to talk about is how to make sure that people know how to think, how to learn, how to adapt,” said Freeman Hrabowski, president of the University of Maryland at Baltimore County. That said, he noted, people receiving a broad college education can also receive specialist certificates and credentials in particular technical areas and add layers to their skills that are more closely linked to evolving job opportunities. “Both are very important,” he noted.</p> <p></p> <p>Juan Salgado, chancellor of the City Colleges of Chicago, a group of community colleges, pointed out that there are 11.8 million community college students in America — many of whom already hold jobs and have workplace skills in addition to the academic skills they are acquiring.</p> <p></p> <p>“It’s about the assets that are in our institutions, our students, and the fact that we’re not paying enough attention to them,” said Salgado.</p> <p></p> <p>“We know what works,” said Paul Osterman, a professor of human resources and management at the MIT Sloan School of Management, pointing out that many training programs, internships, and other work-directed educational programs have been rigorously assessed and proven to be effective. “It’s taking what we know works and making it work at scale.”</p> <p></p> <p>Saru Jayaraman, president of the advocacy group One Fair Wage and director of the Food Labor Research Center at the University of California at Berkeley, noted that simply raising the minimum wage, especially for food service workers, would have multiple benefits that only start with the increased earnings for roughly 10 percent of the workforce.</p> <p></p> <p>“Increased wages reduce turnover in an industry that has some of the highest turnover rates in any industry in the United States,” said Jayaraman, adding that better wages have “increased employee morale, [and] increased employee productivity and consumer service.”</p> <p></p> <p>Karen Mills, a senior fellow at the Harvard Business School and a former administrator of the Small Business Administration, suggested that good policies are especially important for small businesses, which may not be able to capitalize on technology as much as bigger firms.</p> <p></p> <p>“In the jobs of the future, not all robots are going to be serving you coffee,” said Mills. “There’s still going to be Main Street.” She emphasized the continued need for supportive policies for small businesses, including access to health care for employees and access to capital for firm founders, which would also help small businesses owned by women and people of color.</p> <p></p> <p>Rep. Lisa Blunt Rochester of Delaware, who will start her third term as a congresswoman in January, helped found the Congressional Future of Work Caucus, and suggested there is more bipartisan support for federal action than observers may suspect.&nbsp;</p> <p></p> <p>“We launched the caucus right before Covid-19 struck,” she said. “We literally had standing room only. Democrats, Republicans, we had the council on Black mayors, we had the unions, AFL-CIO, just this diversity, academics — I held up your [interim] report — there was this common agreement that we need to have the conversation.”</p> <p></p> <p><strong>“Something we shape and create”</strong></p> <p></p> <p>The conference also included extended discussion about the state of technology itself, especially artificial intelligence, examining its paths of progress and forms of deployment.</p> <p></p> <p>“Technology is not something that happens to us,” said David Mindell, task force co-chair, professor of aeronautics and astronautics, and the Dibner Professor of the History of Engineering and Manufacturing at MIT. “It’s something we shape and create.”</p> <p></p> <p>“You can’t say, ‘AI did it,’” said Microsoft CEO Satya Nadella, in a taped conversation with Autor.&nbsp; “We, as creators of AI, first and foremost have a set of design principles. … We have to go from ethics to actual engineering and design and [a] process that allows us to be more accountable.”</p> <p></p> <p>Related:&nbsp;<a href="">MIT forum examines the rise of automation in the workplace</a></p> <p></p> <p>A number of conference participants suggested that we should be careful to construct policies that don’t rein in technological advances, but can ameliorate their effects.</p> <p></p> <p>“I don’t think we should constrain technological progress, because it is a competitive advantage of nations, and we have to let innovation thrive. We have to let technology proceed,” said Indra Nooyi, the former chairman and CEO of Pepsico. “At best, what we can do is anticipate the negative consequences of technology ... and put in some checks and balances.”</p> <p></p> <p>As a few conference panelists noted throughout the event, the overlapping issues of work, technology, and inequality have become even more complicated and relevant during the Covid-19 pandemic, with roughly one-third of the work force able to work more securely from home, while many service workers and others have to perform their jobs in person.</p> <p></p> <p>Surveying the employment landscape of 2020, Nooyi noted, “In many ways Covid has exacerbated all the societal divides.” Indeed, Reynolds said, “We believe this work is more important, not less important, in the time of Covid.”</p> <p></p> <p>Overall, the task force members noted, making the work of the future better is a task that starts today.</p> <p></p> <p>“I really come away from this concerned about the direction [of work], but optimistic about our ability to change it,” Autor said.</p> The virtual conference, AI and the Work of the Future Congress, was held on November 18, 2020. “We’re not going to run out of work,” said Elisabeth Beck Reynolds, executive director of the task force, and executive director of the MIT Industrial Performance Center, pictured on the right. Also pictured: John Van Reenen, MIT Sloan Professor of Applied Economics and Department of Economics; Jeff Wilke, CEO, Amazon Worldwide Consumer; U.S. Representative Lisa Blunt Rochester, Co-Chair, Congressional Future of Work Caucus; and Karen Mills, Senior Fellow, Harvard Business School and Administrator of the U.S. Small Business Administration. Image: MIT News MIT forum examines the rise of automation in the workplace Technologies like robots and artificial intelligence could partner with humans, not oust them from work, research and business leaders say. Fri, 20 Nov 2020 14:00:00 -0500 Daniel Ackerman | MIT News Office <p>“Pop culture does a great job of scaring us that AI will take over the world,” said Professor Daniela Rus, speaking at a virtual MIT event on Wednesday. But realistically, said Rus, who directs the MIT Computer Science and Artificial Intelligence Laboratory (CSAIL), robots aren’t going to steal everyone’s jobs overnight — they’re not yet good enough at tasks requiring high dexterity or generalized processing of different kinds of information.</p> <p></p> <p>Still, automation has crept into some workplaces in recent years, a trend that’s likely to continue. Throughout the daylong conference, the “AI and the Work of the Future Congress,” which convened speakers from academia, industry, and government, one key theme consistently emerged: Task automation shouldn’t be viewed as a replacement for human work, but a partner for it. With the exception of some middle-skilled manufacturing jobs, automation has generally improved human productivity, not eliminated the need for it. If people thoughtfully guide the development and deployment of new workplace technologies, the speakers agreed, we could see improvements in both productivity and well-being.</p> <p></p> <p>The daylong event was organized by MIT’s Task Force on the Work of the Future, which released its <a href="">final report</a> this week, along with the Initiative on the Digital Economy and CSAIL. During the forum, task force participants and other science and industry leaders discussed both the [social] and technological dimensions of these changes.</p> <p></p> <p><strong>Narrow AI</strong></p> <p></p> <p>Rus emphasized that current industrial applications of artificial intelligence are relatively narrow. “What today’s AI systems can do is specialized intelligence, or the ability to solve a very fixed, limited number of problems,” she said. In select industries like insurance and health care, artificial intelligence has been used to boost efficiency for individual tasks, but it hasn’t generally displaced human workers. Fully automated systems, like driverless cars, remain decades in the future.&nbsp;</p> <p></p> <p>While the rise of artificial intelligence in industry remains gradual, multiple speakers noted how other technologies have rocketed to widespread adoption due to the Covid-19 pandemic. Microsoft CEO Satya Nadella described how videoconferencing and related technologies have enabled the transmission of potentially lifesaving information. “The expert can be remote, but can perhaps more seamlessly transfer their knowledge to the person on the front line,” he said.</p> <p></p> <p>Nadella added that, since so many companies have grown used to videoconferencing, they may never return to 100 percent face-to-face interactions. “There’s going to be real, structural change,” he said. “People are going to question what really requires presence that is physical, versus telepresence. And I think the workflow will adjust.” He noted that workplaces would have to be more intentional about fostering social cohesion among workers in lieu of casual in-person conversations.</p> <p></p> <p>Pandemic aside, some speakers pointed out that automation’s impact on work, though generally positive, has been unequal. Some middle-skill manufacturing jobs have been lost due to automation. But those losses aren’t inevitable — they can be avoided through careful deployment of automation, said Bosch CEO Volkmar Denner. “You could go a very aggressive path and say ‘the robot finally could replace human workers,’” said Denner. “The path we chose was completely different.” Robots on Bosch’s manufacturing line are designed not to oust humans, but to make them even more valuable by assisting with particular tasks to make them more efficient overall.</p> <p></p> <p>“We can find a balance between the economic aspects — introducing automation — and also the social aspects — keeping workers in work,” he said. “Technology always should serve human beings and not vice versa.”</p> <p></p> <p>Other industry leaders agreed. Jeanne Magoulick, engineering manager for Ford Motor Company, said her team is developing artificial intelligence for predictive maintenance of machinery. “It’s going to notify us when a machine seems to be trending out of control, and then we can schedule that for maintenance during the next available window,” she said. “It’s going to make us more efficient.”</p> <p></p> <p><strong>“It’s a choice”</strong></p> <p></p> <p>Rus also discussed the use of machines as guardian systems — safeguards that help ensure human workers are performing at their best. She cited a study where radiologists and an artificial intelligence algorithm were separately shown images of lymph node cells and tasked with determining whether they were cancerous or not. The humans’ error rate was 7.5 percent, and the computer’s was 3.5 percent. However, when an image was scanned by both a human and a computer, the resulting error rate was just 0.5 percent, “which is extraordinary,” said Rus.</p> <p></p> <p>Julie Shah, MIT associate professor in the Department of Aeronautics and Astronautics, added that this sort of “guardian” relationship between humans and automation could extend to many domains, including self-driving cars and manufacturing systems.</p> <p></p> <p>Nadella envisioned that one day the very tools of automation — the ability to design and program computers and robots — will become accessible to those without specialized training. He pointed to examples, like word processing and spreadsheet programs like Excel, where automation turbocharged productivity without requiring users to learn computer code.</p> <p></p> <p>“Knowledge work got fundamentally transformed,” said Nadella. In the future, “this notion of a citizen-app developer, a citizen-data scientist — I think it’s real.”</p> <p></p> <p>Denner also cautioned, however, that certain tasks — like valuing human lives in an automated driving scenario — are best left to ethicists and society as a whole, not to industrial programmers.</p> <p></p> <p>In an afternoon panel about shaping workplace technologies in the future, MIT professor of economics Daron Acemoglu reiterated the refrain that technology isn’t an inevitable force — it’s shaped by humans. Ultimately, he said policymakers and managers will decide how automation fits into the workplace. “There isn’t an ironclad rule of what it is that humans can do and technologies cannot do. They are both fluid. It depends on what we value and how we use technology,” Acemoglu said. “It’s a choice.”</p> “What today’s AI systems can do is specialized intelligence, or the ability to solve a very fixed, limited number of problems,” said Daniela Rus, top middle, speaking at the virtual conference, AI and the Work of the Future Congress. Also pictured: Daron Acemoglu, MIT Professor of Economics; David Mindell, Co-Chair, MIT Task Force on the Work of the Future; MIT Professor of Aeronautics and Astronautics; and CEO and Founder, Humatics Corporation; Jeanne Magoulick, Engineering Manager, Ford Motor Company; and Julie Shah, MIT Associate Professor, Department of Aeronautics and Astronautics. Image: MIT News Report outlines route toward better jobs, wider prosperity MIT Task Force on the Work of the Future identifies ways to align new technologies with durable careers. Tue, 17 Nov 2020 07:00:00 -0500 Peter Dizikes | MIT News Office <p>Decades of technological change have polarized the earnings of the American workforce, helping highly educated white-collar workers thrive, while hollowing out the middle class. Yet present-day advances like robots and artificial intelligence do not spell doom for middle-tier or lower-wage workers, since innovations create jobs as well. With better policies in place, more people could enjoy good careers even as new technology transforms workplaces.</p> <p>That’s the conclusion of the <a href="" target="_blank">final report</a> from MIT’s Task Force on the Work of the Future, which summarizes over two years of research on technology and jobs. The report, “The Work of the Future: Building Better Jobs in an Age of Intelligent Machines,” was released today, and the task force is hosting an online conference on Wednesday, the “<a href="">AI &amp; the Future of Work Congress</a>.”</p> <p>At the core of the task force’s findings: A robot-driven jobs apocalypse is not on the immediate horizon. As technology takes jobs away, it provides new opportunities; about 63 percent of jobs performed in 2018 did not exist in 1940. Rather than a robot revolution in the workplace, we are witnessing a gradual tech evolution. At issue is how to improve the quality of jobs, particularly for middle- and lower-wage workers, and ensure there is greater shared prosperity than the U.S. has seen in recent decades.</p> <p>“The sky is not falling, but it is slowly lowering,” says David Autor, the Ford Professor of Economics at MIT, associate head of MIT’s Department of Economics, and a co-chair of the task force. “We need to respond. The world is gradually changing in very important ways, and if we just keep going in the direction we’re going, it is going to produce bad outcomes.”</p> <p>That starts with a realistic understanding of technological change, say the task force leaders.</p> <p>The task force aimed “to move past the hype about what [technologies] might be here, and now we’re looking at what can we feasibly do to move things forward for workers,” says Elisabeth Beck Reynolds, executive director of the task force as well as executive director of the MIT Industrial Performance Center. “We looked across a range of industries and examined the numerous factors — social, cognitive, organizational, economic — that shape how firms adopt technology.”</p> <p>“We want to inject into the public discourse a more nuanced way of talking about technology and work,” adds David Mindell, task force co-chair, professor of aeronautics and astronautics, and the Dibner Professor of the History of Engineering and Manufacturing at MIT. “It’s not that the robots are coming tomorrow and there’s nothing we can do about it. Technology is an aggregate of human choices.”</p> <p>The report also addresses why Americans may be concerned about work and the future. It states: “Where innovation fails to drive opportunity, it generates a palpable fear of the future: the suspicion that technological progress will make the country wealthier while threatening the people’s livelihoods. This fear exacts a high price: political and regional divisions, distrust of institutions, and mistrust of innovation itself. The last four decades of economic history give credence to that fear.”</p> <p>"Automation is transforming our work, our lives, our society," says MIT President L. Rafael Reif, who initiated the formation of the task force in 2017. "Fortunately, the harsh societal consequences that concern us all are not inevitable. How we design tomorrow’s technologies, and the policies and practices we build around them, will profoundly shape their impact."</p> <p>Reif adds: "Getting this right is among the most important and inspiring challenges of our time — and it should be a priority for everyone who hopes to enjoy the benefits of a society that’s healthy and stable, because it offers opportunity for all."</p> <p><strong>Six big conclusions</strong></p> <p>The task force, an Institute-wide group of scholars and researchers, spent over two years studying work and technology in depth. The final report presents six overarching conclusions and a set of policy recommendations. The conclusions:</p> <p><em>1) Technological change is simultaneously replacing existing work and creating new work. It is not eliminating work altogether.</em><strong> </strong></p> <p>Over the last several decades, technology has significantly changed many workplaces, especially through digitization and automation, which have replaced clerical, administrative, and assembly-line workers across the country. But the overall percentage of adults in paid employment has largely risen for over a century. In theory, the report states, there is “no instrinsic conflict between technological change, full employment, and rising earnings.”</p> <p>In practice, however, technology has polarized the economy. White-collar workers — in medicine, marketing, design, research, and more — have become more productive and richer, while middle-tier workers have lost out. Meanwhile, there has been growth in lower-paying service-industry jobs where digitization has little impact — such as food services, janitors, and drivers. Since 1978, aggregate U.S. productivity has risen by 66 percent, while compensation for production and nonsupervisory workers has risen by only 10 percent. Wage gaps also exist by race and gender, and cities do not provide the “<a href="">escalator</a>” to the middle class they once did.</p> <p>While innovations have replaced many receptionists, clerks, and assembly-line workers, they have simultaneously created new occupations. Since the middle of the 20th century, the U.S. has seen major growth in the computer industry, renewable energy, medical specialties, and many areas of design, engineering, marketing, and health care. These industries can support many middle-income jobs as well — while the services sector keeps growing.</p> <p>As the task force leaders state in the report, “The dynamic interplay among task automation, innovation, and new work creation, while always disruptive, is a primary wellspring of rising productivity. Innovation improves the quantity, quality, and variety of work that a worker can accomplish in a given time. This rising productivity, in turn, enables improving living standards and the flourishing of human endeavors.”</p> <p>However, a bit ruefully, the authors also note that “in what <em>should be</em> a virtuous cycle, rising productivity provides society with the resources to invest in those whose livelihoods are disrupted by the changing structure of work.”</p> <p>But this has not come to pass, as the distribution of value from these jobs has been lopsided. In the U.S., lower-skill jobs only pay 79 percent as much when compared to Canada, 74 percent compared to the U.K., and 57 percent compared to Germany.</p> <p>“People understand that automation can make the country richer and make them poorer, and that they’re not sharing in those gains,” Autor says. “We think that can be fixed.”</p> <p><em>2) Momentous impacts of technological change are unfolding gradually.</em></p> <p>Time and again, media coverage about technology and jobs focuses on dramatic scenarios in which robots usurp people, and we face a future without work.</p> <p>But this picture elides a basic point: Technologies mimicking human actions are difficult to build, and expensive. It is generally cheaper to simply hire people for those tasks. On the other hand, technologies that augment human abilities — like tools that let doctors make diagnoses — help those workers become more productive. Apart from clerical and assembly-line jobs, many technologies exist in concert with workers, not as a substitute for them.</p> <p>Thus workplace technology usually involves “augmentation tasks more than replacement tasks,” Mindell says. The task force report surveys technology adoption in industries including insurance, health care, manufacturing, and autonomous vehicles, finding growth in “narrow” AI systems that complement workers. Meanwhile, technologists are working on difficult problems like better robotic dexterity, which could lead to more direct replacement of workers, but such advances at a high level are further off in the future.</p> <p>“That’s what technological adoption looks like,” Mindell says. “It’s uneven, it’s lumpy, it goes in fits and starts.” The key question is how innovators at MIT and elsewhere can shape new technology to broad social benefit.</p> <p><em>3) Rising labor productivity has not translated into broad increases in incomes because societal institutions and labor market policies have fallen into disrepair. </em></p> <p>While the U.S. has witnessed a lot of technological innovation in recent decades, it has not seen as much policy innovation, particularly on behalf of workers. The polarizing effects of technology on jobs would be lessened if middle- and lower-income workers had relatively better support in other ways. Instead, in terms of pay, working environment, termination notice time, paid vacation time, sick time, and family leave, “less-educated and low-paid U.S. workers fare worse than comparable workers in other wealthy industrialized nations,” the report notes. The adjusted gross hourly earnings of lower-skill workers in the U.S. in 2015 averaged $10.33, compared to $24.28 in Denmark, $18.18 in Germany, and $17.61 in Australia.</p> <p>“It’s untenable that the labor market has this growing gulf without shared prosperity,” Autor says. “We need to restore the synergy between rising productivity and improvements in labor market opportunity.” He adds: “We’ve had real institutional failure, and it’s within our hands to change it. … That includes worker voice, minimum wages, portable benefits, and incentives that cause companies to invest in workers.”</p> <p>Looking ahead, the report cautions, “If those technologies deploy into the labor institutions of today, which were designed for the last century, we will see similar effects to recent decades: downward pressure on wages, skills, and benefits, and an increasingly bifurcated labor market.” The task force argues instead for institutional innovations that complement technological change.</p> <p><em>4) Improving the quality of jobs requires innovation in labor market institutions.&nbsp;</em></p> <p>The task force contends the U.S. needs to modernize labor policies on several fronts, including restoring the federal minimum wage to a reasonable percentage of the national median wage and, crucially, indexing it to inflation.&nbsp;</p> <p>The report also suggests upgrading unemployment insurance in several ways, including: using very recent earnings to determine eligibility or linking eligibility to hours worked, not earnings; making it easier to receive partial benefits in case of events like loss of a second job; and dropping the requirement that people need to seek full-time work to receive benefits, since so many people hold part-time positions.&nbsp;</p> <p>The report also observes that U.S. collective bargaining law and processes are antiquated. The authors argue that workers need better protection of their current collective bargaining rights; new forms of workplace representation beyond traditional unions; and legal protections allowing groups to organize that include home-care workers, farmworkers, and independent contractors.</p> <p><em>5) Fostering opportunity and economic mobility necessitates cultivating and refreshing worker skills.</em></p> <p>Technological advancement may often be incremental, but changes happen often enough that workers’ skills and career paths can become obsolete. The report emphasizes that U.S. workers need more opportunities to add new skills — whether through the community college system, online education, company-based retraining, or other means. &nbsp;</p> <p>The report calls for making ongoing skills development accessible, engaging, and cost-effective. This requires buttressing what already works, while advancing new tools: blended online and in-person offerings, machine-supervised learning, and augmented and virtual reality learning environments.</p> <p>The greatest needs are among workers without four-year college degrees. “We need to focus on those who are between high school and the four-year degree,” Reynolds says. “There should be pathways for those people to increase their skill set and make it meaningful to the labor market. We really need a shift that makes this a high priority.”</p> <p><em>6) Investing in innovation will drive new job creation, speed growth, and meet rising competitive challenges.</em></p> <p>The rate of new-job creation over the last century is heavily driven by technological innovation, the report notes, with a considerable portion of that stemming from federal investment in R&amp;D, which has helped produce many forms of computing and medical advances, among other things. As of 2015, the U.S. invested 2.7 percent of its GDP in R&amp;D, compared to 2.9 percent in Germany and 2.1 percent in China. But the public share of that R&amp;D investment has fallen from 40 percent in 1985 to 25 percent in 2015. The task force calls for a recommitment to this federal support.</p> <p>“Innovation has a key role in job creation and growth,” Autor says.</p> <p>Given the significance of innovation to job and wealth creation, the report calls for increased overall federal research funding; targeted assistance that helps small- and medium-sized businesses adopt technology; policies creating a wider geographical spread of innovation in the U.S.; and policies that enhance investment in workers, not just capital, including the elimination of accelerated capital depreciation claims, and an employer training tax credit that functions like the R&amp;D tax credit.</p> <p><strong>Global issues, U.S. suggestions </strong></p> <p>In addition to Reynolds, Autor, and Mindell, MIT’s Task Force on the Work of the Future consisted of a group of 18 MIT professors representing all five Institute schools and the MIT Schwarzman College of Computing; a 22-person advisory board drawn from the ranks of industry leaders, former government officials, and academia; a 14-person research board of scholars; and over 20 graduate students. The task force also consulted with business executives, labor leaders, and community college leaders, among others. The final document includes case studies from specific firms and sectors as well, and the Task Force is publishing nearly two dozen <a href="">research briefs </a>that go into the primary research in more detail.&nbsp;</p> <p>The task force observed global patterns at play in the way technology is adopted and diffused through the workplace, although its recommendations are focused on U.S. policy issues.</p> <p>“While our report is very geared toward the U.S. in policy terms, it clearly is speaking to a lot of trends and issues that exist globally,” Reynolds said. “The message is not just for the U.S. Many of the challenges we outline are found in other countries too, albeit to lesser degrees.&nbsp;As we wrote in the report, ‘the central challenge ahead, indeed the work of the future, is to advance labor market opportunity to meet, complement, and shape technological innovations.’”</p> <p>The task force intends to circulate ideas from the report among policymakers and politicians, corporate leaders and other business managers, and researchers, as well as anyone with an interest in the condition of work in the 21st century.</p> <p>“I hope people are receptive,” Reynolds adds. “We have made forceful recommendations that tie together different policy areas — skills, job quality, and innovation. These issues are critical, particularly as we think about recovery and rebuilding in the age of Covid-19. I hope our message will be picked up by both the public sector and private sector leaders, because both of those are essential to forge the path forward.”</p> The MIT Task Force on the Work of the Future released its final report on Nov. 17. An underwater navigation system powered by sound New approach could spark an era of battery-free ocean exploration, with applications ranging from marine conservation to aquaculture. Mon, 02 Nov 2020 09:00:00 -0500 Daniel Ackerman | MIT News Office <p>GPS isn’t waterproof. The navigation system depends on radio waves, which break down rapidly in liquids, including seawater. To track undersea objects like drones or whales, researchers rely on acoustic signaling. But devices that generate and send sound usually require batteries — bulky, short-lived batteries that need regular changing. Could we do without them?</p> <p>MIT researchers think so. They’ve built a battery-free pinpointing system dubbed Underwater Backscatter Localization (UBL). Rather than emitting its own acoustic signals, UBL reflects modulated signals from its environment. That provides researchers with positioning information, at net-zero energy. Though the technology is still developing, UBL could someday become a key tool for marine conservationists, climate scientists, and the U.S. Navy.</p> <p>These advances are described in a paper being presented this week at the Association for Computing Machinery’s Hot Topics in Networks workshop, by members of the Media Lab’s Signal Kinetics group. Research Scientist Reza Ghaffarivardavagh led the paper, along with co-authors Sayed Saad Afzal, Osvy Rodriguez, and Fadel Adib, who leads the group and is the Doherty Chair of Ocean Utilization as well as an associate professor in the MIT Media Lab and the MIT Department of Electrical Engineering and Computer Science.</p> <p><strong>“Power-hungry”</strong></p> <p>It’s nearly impossible to escape GPS’ grasp on modern life. The technology, which relies on satellite-transmitted radio signals, is used in shipping, navigation, targeted advertising, and more. Since its introduction in the 1970s and ’80s, GPS has changed the world. But it hasn’t changed the ocean. If you had to hide from GPS, your best bet would be underwater.</p> <p>Because radio waves quickly deteriorate as they move through water, subsea communications often depend on acoustic signals instead. Sound waves travel faster and further underwater than through air, making them an efficient way to send data. But there’s a drawback.</p> <p>“Sound is power-hungry,” says Adib. For tracking devices that produce acoustic signals, “their batteries can drain very quickly.” That makes it hard to precisely track objects or animals for a long time-span — changing a battery is no simple task when it’s attached to a migrating whale. So, the team sought a battery-free way to use sound.</p> <p><strong>Good vibrations</strong></p> <p>Adib’s group turned to a unique resource they’d <a href="">previously used</a> for low-power acoustic signaling: piezoelectric materials. These materials generate their own electric charge in response to mechanical stress, like getting pinged by vibrating soundwaves. Piezoelectric sensors can then use that charge to selectively reflect some soundwaves back into their environment. A receiver translates that sequence of reflections, called backscatter, into a pattern of 1s (for soundwaves reflected) and 0s (for soundwaves not reflected). The resulting binary code can carry information about ocean temperature or salinity.</p> <p>In principle, the same technology could provide location information. An observation unit could emit a soundwave, then clock how long it takes that soundwave to reflect off the piezoelectric sensor and return to the observation unit. The elapsed time could be used to calculate the distance between the observer and the piezoelectric sensor. But in practice, timing such backscatter is complicated, because the ocean can be an echo chamber.</p> <p>The sound waves don’t just travel directly between the observation unit and sensor. They also careen between the surface and seabed, returning to the unit at different times. “You start running into all of these reflections,” says Adib. “That makes it complicated to compute the location.” Accounting for reflections is an even greater challenge in shallow water — the short distance between seabed and surface means the confounding rebound signals are stronger.</p> <p>The researchers overcame the reflection issue with “frequency hopping.” Rather than sending acoustic signals at a single frequency, the observation unit sends a sequence of signals across a range of frequencies. Each frequency has a different wavelength, so the reflected sound waves return to the observation unit at different phases. By combining information about timing and phase, the observer can pinpoint the distance to the tracking device. Frequency hopping was successful in the researchers’ deep-water simulations, but they needed an additional safeguard to cut through the reverberating noise of shallow water.</p> <p>Where echoes run rampant between the surface and seabed, the researchers had to slow the flow of information. They reduced the bitrate, essentially waiting longer between each signal sent out by the observation unit. That allowed the echoes of each bit to die down before potentially interfering with the next bit. Whereas a bitrate of 2,000 bits/second sufficed in simulations of deep water, the researchers had to dial it down to 100 bits/second in shallow water to obtain a clear signal reflection from the tracker. But a slow bitrate didn’t solve everything.</p> <p>To track moving objects, the researchers actually had to boost the bitrate. One thousand bits/second was too slow to pinpoint a simulated object moving through deep water at 30 centimeters/second. “By the time you get enough information to localize the object, it has already moved from its position,” explains Afzal. At a speedy 10,000 bits/second, they were able to track the object through deep water.</p> <p><strong>Efficient exploration</strong></p> <p>Adib’s team is working to improve the UBL technology, in part by solving challenges like the conflict between low bitrate required in shallow water and the high bitrate needed to track movement. They’re working out the kinks through tests in the Charles River. “We did most of the experiments last winter,” says Rodriguez. That included some days with ice on the river. “It was not very pleasant.”</p> <p>Conditions aside, the tests provided a proof-of-concept in a challenging shallow-water environment. UBL estimated the distance between a transmitter and backscatter node at various distances up to nearly half a meter. The team is working to increase UBL’s range in the field, and they hope to test the system with their collaborators at the Woods Hole Oceanographic Institution on Cape Cod.</p> <p>They hope UBL can help fuel a boom in ocean exploration. Ghaffarivardavagh notes that scientists have better maps of the moon’s surface than of the ocean floor. “Why can’t we send out unmanned underwater vehicles on a mission to explore the ocean? The answer is: We will lose them,” he says.</p> <p>UBL could one day help autonomous vehicles stay found underwater, without spending precious battery power. The technology could also help subsea robots work more precisely, and provide information about climate change impacts in the ocean. “There are so many applications,” says Adib. “We’re hoping to understand the ocean at scale. It’s a long-term vision, but that’s what we’re working toward and what we’re excited about.”</p> <p>This work was supported, in part, by the Office of Naval Research.</p> MIT researchers have built a battery-free pinpointing system dubbed Underwater Backscatter Localization (UBL). This photo shows the battery-free sensor encapsulated in a polymer before it is dipped into the Charles river. Image: Reza Ghaffarivardavagh “What to Expect When You’re Expecting Robots” Book co-authored by Associate Professor Julie Shah and Laura Major SM ’05 explores a future populated with robot helpers. Thu, 22 Oct 2020 00:00:00 -0400 Jennifer Chu | MIT News Office <p>As Covid-19 has made it necessary for people to keep their distance from each other, robots are stepping in to fill essential roles, such as sanitizing warehouses and hospitals, ferrying test samples to laboratories, and serving as telemedicine avatars.</p> <p>There are signs that people may be increasingly receptive to robotic help, preferring, at least hypothetically, to be picked up by a self-driving taxi or have their food delivered via robot, to reduce their risk of catching the virus.</p> <p>As more intelligent, independent machines make their way into the public sphere, engineers Julie Shah and Laura Major are urging designers to rethink not just how robots fit in with society, but also how society can change to accommodate these new, “working” robots.</p> <p>Shah is an associate professor of aeronautics and astronautics at MIT and the associate dean of social and ethical responsibilities of computing in the MIT Schwarzman College of Computing. Major SM ’05 is CTO of Motional, a self-driving car venture supported by automotive companies Hyundai and Aptiv. Together, they have written a new book, “What to Expect When You’re Expecting Robots: The Future of Human-Robot Collaboration,” published this month by Basic Books.</p> <p>What we can expect, they write, is that robots of the future will no longer work for us, but with<em> </em>us. They will be less like tools, programmed to carry out specific tasks in controlled environments, as factory automatons and domestic Roombas have been, and more like partners, interacting with and working among people in the more complex and chaotic real world. As such, Shah and Major say that robots and humans will have to establish a mutual understanding.</p> <p>“Part of the book is about designing robotic systems that think more like people, and that can understand the very subtle social signals that we provide to each other, that make our world work,” Shah says. “But equal emphasis in the book is on how we have to structure the way we live our lives, from our crosswalks to our social norms, so that robots can more effectively live in our world.”</p> <p><strong>Getting to know you</strong></p> <p>As robots increasingly enter public spaces, they may do so safely if they have a better understanding of human and social behavior.</p> <p>Consider a package delivery robot on a busy sidewalk: The robot may be programmed to give a standard berth to obstacles in its path, such as traffic cones and lampposts. But what if the robot is coming upon a person wheeling a stroller while balancing a cup of coffee? A human passerby would read the social cues and perhaps step to the side to let the stroller by. Could a robot pick up the same subtle signals to change course accordingly?</p> <p>Shah believes the answer is yes. As head of the Interactive Robotics Group at MIT, she is developing tools to help robots understand and predict human behavior, such as where people move, what they do, and who they interact with in physical spaces. She’s implemented these tools in robots that can recognize and collaborate with humans in environments such as the factory floor and the hospital ward. She is hoping that robots trained to read social cues can more safely be deployed in more unstructured public spaces.</p> <p>Major, meanwhile, has been helping to make robots, and specifically self-driving cars, work safely and reliably in the real world, beyond the controlled, gated environments where most driverless cars operate today. About a year ago, she and Shah met for the first time, at a robotics conference.</p> <p>“We were working in parallel universes, me in industry, and Julie in academia, each trying to galvanize understanding for the need to accommodate machines and robots,” Major recalls.</p> <p>From that first meeting, the seeds for their new book began quickly to sprout.</p> <p><strong>A cyborg city</strong></p> <p>In their book, the engineers describe ways that robots and automated systems can perceive and work with humans — but also ways in which our environment and infrastructure can change to accommodate robots.</p> <p>A cyborg-friendly city, engineered to manage and direct robots, could avoid scenarios such as the one that played out in San Francisco in 2017. Residents there were seeing an uptick in delivery robots deployed by local technology startups. The robots were causing congestion on city sidewalks and were an unexpected hazard to seniors with disabilities. Lawmakers ultimately enforced strict regulations on the number of delivery robots allowed in the city — a move that improved safety, but potentially at the expense of innovation.</p> <p>If in the near future there are to be multiple robots sharing a sidewalk with humans at any given time, Shah and Major propose that cities might consider installing dedicated robot lanes, similar to bike lanes, to avoid accidents between robots and humans. The engineers also envision a system to organize robots in public spaces, similar to the way airplanes keep track of each other in flight.</p> <p>In 1965, the Federal Aviation Agency was created, partly in response to a catastrophic crash between two planes flying through a cloud over the Grand Canyon. Prior to that crash, airplanes were virtually free to fly where they pleased. The FAA began organizing airplanes in the sky through innovations like the traffic collision avoidance system, or TCAS — a system onboard most planes today, that detects other planes outfitted with a universal transponder. TCAS alerts the pilot of nearby planes, and automatically charts a path, independent of ground control, for the plane to take in order to avoid a collision.</p> <p>Similarly, Shah and Major say that robots in public spaces could be designed with a sort of universal sensor that enables them to see and communicate with each other, regardless of their software platform or manufacturer. This way, they might stay clear of certain areas, avoiding potential accidents and congestion, if they sense robots nearby.</p> <p>“There could also be transponders for people that broadcast to robots,” Shah says. “For instance, crossing guards could use batons that can signal any robot in the vicinity to pause so that it’s safe for children to cross the street.”</p> <p>Whether we are ready for them or not, the trend is clear: The robots are coming, to our sidewalks, our grocery stores, and our homes. And as the book’s title suggests, preparing for these new additions to society will take some major changes, in our perception of technology, and in our infrastructure.</p> <p>“It takes a village to raise a child to be a well-adjusted member of society, capable of realizing his or her full potential,” write Shah and Major. “So, too, a robot.”</p> A new book co-authored by MIT engineers Julie Shah and Laura Major SM ’05, “What to Expect When You’re Expecting Robots,” explores a future populated with robot helpers. “It takes a village to raise a child to be a well-adjusted member of society, capable of realizing his or her full potential,” they write. “So, too, a robot.” Image: Jose-Luis Olivares, MIT Autonomous boats could be your next ride Five years in the making, MIT’s autonomous floating vessels get a size upgrade and learn a new way to communicate aboard the waters.  Mon, 26 Oct 2020 09:00:00 -0400 Rachel Gordon | MIT CSAIL <p>The feverish race to produce the shiniest, safest, speediest self-driving car has spilled over into our wheelchairs, scooters, and even golf carts. Recently, there’s been movement from land to sea, as marine autonomy stands to change the canals of our cities, with the potential to deliver goods and services and collect waste across our waterways.&nbsp;</p> <p>In an update to a five-year project from MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL) and the Senseable City Lab, researchers have been developing the world's first fleet of autonomous boats for the City of Amsterdam, the Netherlands, and have recently added a new, larger vessel to the group: “Roboat II.” Now sitting at 2 meters long, which is roughly a “Covid-friendly” 6 feet, the new robotic boat is capable of carrying passengers.</p> <p>This work was supported by grant from the Amsterdam Institute for Advanced Metropolitan Solutions (AMS) in Netherlands.</p> <p>Alongside <a href="">the Amsterdam Institute for Advanced Metropolitan Solutions</a>, the team also created navigation and control algorithms to update the communication and collaboration among the boats.&nbsp;</p> <p>“Roboat II navigates autonomously using algorithms similar to those used by self-driving cars, but now adapted for water,” says MIT Professor Daniela Rus, a senior author on a new paper about Roboat and the director of CSAIL. “We’re developing fleets of Roboats that can deliver people and goods, and connect with other Roboats to form a range of autonomous platforms to enable water activities.”&nbsp;</p> <p>Self-driving boats have been able to transport small items for years, but adding human passengers has felt somewhat intangible due to the current size of the vessels. Roboat II is the “half-scale” boat in the growing body of work, and joins the previously developed quarter-scale Roboat, which is 1 meter long. The third installment, which is under construction in Amsterdam and is considered to be “full scale,” is 4 meters long and aims to carry anywhere from four to six passengers.&nbsp;</p> <p>Aided by powerful algorithms, Roboat II autonomously navigated the canals of Amsterdam for three hours collecting data, and returned back to its start location with an error margin of only 0.17 meters, or fewer&nbsp;than 7 inches.&nbsp;</p> <p>“The development of an autonomous boat system capable of accurate mapping, robust control, and human transport is a crucial step towards having the system implemented in the full-scale Roboat,” says senior postdoc Wei Wang, lead author on a new paper about Roboat II. “We also hope it will eventually be implemented in other boats in order to make them autonomous.”</p> <p>Wang wrote the paper alongside<em> </em>MIT Senseable City Lab postdoc Tixiao Shan, research fellow Pietro Leoni, postdoc David Fernandez-Gutierrez, research fellow Drew Meyers, and MIT professors Carlo Ratti and Daniela Rus. The work was supported by a grant from the Amsterdam Institute for Advanced Metropolitan Solutions in the Netherlands. A paper on Roboat II will be virtually presented at the International Conference on Intelligent Robots and Systems.&nbsp;</p> <p>To coordinate communication among the boats, another team from MIT CSAIL and Senseable City Lab, also led by Wang, came up with a new control strategy for robot coordination.&nbsp;</p> <p>With the intent of self-assembling into connected, multi-unit trains — with distant homage to children’s train sets&nbsp;— “collective transport” takes a different path to complete various tasks. The system uses a distributed controller, which is a collection of <a href="">sensors, controllers, and associated computers</a> distributed throughout a system), and a strategy inspired by how a colony of ants can transport food without communication. Specifically, there’s no direct communication among the connected robots — only one leader knows the destination. The leader initiates movement to the destination, and then the other robots can estimate the intention of the leader, and align their movements accordingly.&nbsp;</p> <p>“Current cooperative algorithms have rarely considered dynamic systems on the water,” says Ratti, the Senseable City Lab director. “Cooperative transport, using a team of water vehicles, poses unique challenges not encountered in aerial or ground vehicles. For example, inertia and load of the vehicles become more significant factors that make the system harder to control. Our study investigates the cooperative control of the surface vehicles and validates the algorithm on that.”&nbsp;</p> <p>The team tested their control method on two scenarios: one where three robots are connected in a series, and another where three robots are connected in parallel. The results showed that the coordinated group was able to track various trajectories and orientations in both configurations, and that the magnitudes of the followers’ forces positively contributed to the group — indicating that the follower robots helped the leader.&nbsp;</p> <p>Wang wrote a paper about collective transport alongside Stanford University PhD student Zijian Wang, MIT postdoc Luis Mateos, MIT researcher Kuan Wei Huang, Stanford Assistant Professor Mac Schwager, Ratti, and Rus.&nbsp;</p> <p><strong>Roboat II</strong></p> <p>In 2016, MIT researchers <a href="">tested</a> a prototype that could move “forward, backward, and laterally along a pre-programmed path in the canals.” Three years later, the team’s robots were updated to “shapeshift” by <a href="">autonomously disconnecting and reassembling</a> into a variety of configurations.&nbsp;</p> <p>Now, Roboat II has scaled up to explore transportation tasks, aided by updated research. These include a new algorithm for Simultaneous Localization and Mapping (SLAM), a model-based optimal controller called nonlinear model predictive controller, and an optimization-based state estimator, called moving horizon estimation.&nbsp;</p> <p>Here’s how it works: When a passenger pickup task is required from a user at a specific position, the system coordinator will assign the task to an unoccupied boat that’s closest to the passenger. As Roboat II picks up the passenger, it will create a feasible path to the desired destination, based on the current traffic conditions.&nbsp;</p> <p>Then, Roboat II, which weighs more than 50 kilograms, will start to localize itself by running the SLAM algorithm and utilizing <a href="">lidar</a> and GPS sensors, as well as an inertial measurement unit for localization, pose, and velocity. The controller then tracks the reference trajectories from the planner, which updates the path to avoid obstacles that are detected to avoid potential collisions.&nbsp;&nbsp;</p> <p>The team notes that the improvements in their control algorithms have made the obstacles feel like less of a giant iceberg since their last update; the SLAM algorithm provides a higher localization accuracy for Roboat, and allows for online mapping during navigation, which they didn’t have in previous iterations.&nbsp;</p> <p>Increasing the size of Roboat also required a larger area to conduct the experiments, which began in the MIT pools and subsequently moved to the Charles River, which cuts through Boston and Cambridge, Massachusetts.</p> <p>While navigating the congested roads of cities alike can lead drivers to feel trapped in a maze, canals largely avoid this. Nevertheless, tricky scenarios in the waterways can still emerge. Given that, the team is working on developing more efficient planning algorithms to let the vessel handle more complicated scenarios, by applying active object detection and identification to improve Roboat’s understanding of its environment. The team plans to estimate disturbances such as currents and waves, to further improve the tracking performance in more noisy waters.&nbsp;</p> <p>“All of these expected developments will be incorporated into the first prototype of the full-scale Roboat and tested in the canals of the City of Amsterdam,” says Rus.&nbsp;</p> <p><strong>Collective transport </strong></p> <p>Making our intuitive abilities a reality for machines has been the persistent intention since the birth of the field, from straightforward commands for picking up items to the nuances of organizing in a group.&nbsp;</p> <p>One of the main goals of the project is enabling self-assembly to complete the aforementioned tasks of collecting waste, delivering items, and transporting people in the canals — but controlling this movement on the water has been a challenging obstacle. Communication in robotics can often be unstable or have delays, which may worsen the robot coordination.&nbsp;</p> <p>Many control algorithms for this collective transport require direct communication, the relative positions in the group, and the destination of the task — but the team’s new algorithm simply needs one robot to know<em> </em>the desired trajectory and orientation.&nbsp;</p> <p>Normally, the distributed controller running on each robot requires the velocity information of the connected structure (represented by the velocity of the center of the structure), but this requires that each robot knows the relative position to the center of the structure. In the team’s algorithm, they don’t need the relative position, and each robot simply uses its local velocity instead of the velocity of the center of the structure.</p> <p>When the leader initiates the movement to the destination, the other robots can therefore estimate the intention of the leader and align their movements. The leader can also steer the rest of the robots by adjusting its input, without any communication between any two robots.&nbsp;</p> <p>In the future, the team plans to use machine learning to estimate (online) the key parameters of the robots. They’re also aiming to explore adaptive controllers that allow for dynamic change to the structure when objects are placed on the boat. Eventually, the boats will also be extended&nbsp;to outdoor water environments, where large disturbances such as currents and waves exist.</p> The latest version of MIT's autonomous boat is now 2 meters long and capable of carrying passengers. Photo courtesy of the researchers. Helping robots avoid collisions Realtime Robotics has created a controller that helps robots safely move around on the fly. Thu, 17 Sep 2020 15:00:00 -0400 Zach Winn | MIT News Office <p>George Konidaris still remembers his disheartening introduction to robotics.</p> <p>“When you’re a young student and you want to program a robot, the first thing that hits you is this immense disappointment at how much you can’t do with that robot,” he says.</p> <p>Most new roboticists want to program their robots to solve interesting, complex tasks — but it turns out that just moving them through space without colliding with objects is more difficult than it sounds.</p> <p>Fortunately, Konidaris is hopeful that future roboticists will have a more exciting start in the field. That’s because roughly four years ago, he co-founded Realtime Robotics, a startup that’s solving the “motion planning problem” for robots.</p> <p>The company has invented a solution that gives robots the ability to quickly adjust their path to avoid objects as they move to a target. The Realtime controller is a box that can be connected to a variety of robots and deployed in dynamic environments.</p> <p>“Our box simply runs the robot according to the customer’s program,” explains Konidaris, who currently serves as Realtime’s chief roboticist. “It takes care of the movement, the speed of the robot, detecting obstacles, collision detection. All [our customers] need to say is, ‘I want this robot to move here.’”</p> <p>Realtime’s key enabling technology is a unique circuit design that, when combined with proprietary software, has the effect of a plug-in motor cortex for robots. In addition to helping to fulfill the expectations of starry-eyed roboticists, the technology also represents a fundamental advance toward robots that can work effectively in changing environments.</p> <p><strong>Helping robots get around</strong></p> <p>Konidaris was not the first person to get discouraged about the motion planning problem in robotics. Researchers in the field have been working on it for 40 years. During a four-year postdoc at MIT, Konidaris worked with School of Engineering Professor in Teaching Excellence Tomas Lozano-Perez, a pioneer in the field who was publishing papers on motion planning before Konidaris was born.</p> <p>Humans take collision avoidance for granted. Konidaris points out that the simple act of grabbing a beer from the fridge actually requires a series of tasks such as opening the fridge, positioning your body to reach in, avoiding other objects in the fridge, and deciding where to grab the beer can.</p> <p>“You actually need to compute more than one plan,” Konidaris says. “You might need to compute hundreds of plans to get the action you want. … It’s weird how the simplest things humans do hundreds of times a day actually require immense computation.”</p> <p>In robotics, the motion planning problem revolves around the computational power required to carry out frequent tests as robots move through space. At each stage of a planned path, the tests help determine if various tiny movements will make the robot collide with objects around it. Such tests have inspired researchers to think up ever more complicated algorithms in recent years, but Konidaris believes that’s the wrong approach.</p> <p>“People were trying to make algorithms smarter and more complex, but usually that’s a sign that you’re going down the wrong path,” Konidaris says. “It’s actually not that common that super technically sophisticated techniques solve problems like that.”</p> <p>Konidaris left MIT in 2014 to join the faculty at Duke University, but he continued to collaborate with researchers at MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL). Duke is also where Konidaris met Realtime co-founders Sean Murray, Dan Sorin, and Will Floyd-Jones. In 2015, the co-founders collaborated to make a new type of computer chip with circuits specifically designed to perform the frequent collision tests required to move a robot safely through space. The custom circuits could perform operations in parallel to more efficiently test short motion collisions.</p> <p>“When I left MIT for Duke, one thing bugging me was this motion planning thing should really be solved by now,” Konidaris says. “It really did come directly out of a lot of experiences at MIT. I wouldn’t have been able to write a single paper on motion planning before I got to MIT.”</p> <p>The researchers founded Realtime in 2016 and quickly brought on robotics industry veteran Peter Howard SM&nbsp;’87, who currently serves as Realtime’s CEO and is also considered a co-founder.</p> <p>“I wanted to start the company in Boston because I knew MIT and lot of robotics work was happening there,” says Konidaris, who moved to Brown University in 2016. “Boston is a hub for robotics. There’s a ton of local talent, and I think a lot of that is because MIT is here — PhDs from MIT became faculty at local schools, and those people started robotics programs. That network effect is very strong.”</p> <p><strong>Removing robot restraints</strong></p> <p>Today the majority of Realtime’s customers are in the automotive, manufacturing, and logistics industries. The robots using Realtime’s solution are doing everything from spot welding to making inspections to picking items from bins.</p> <p>After customers purchase Realtime’s control box, they load in a file describing the configuration of the robot’s work cell, information about the robot such as its end-of-arm tool, and the task the robot is completing. Realtime can also help optimally place the robot and its accompanying sensors around a work area. Konidaris says Realtime can shorten the process of deploying robots from an average of 15 weeks to one week.</p> <p>Once the robot is up and running, Realtime’s box controls its movement, giving it instant collision-avoidance capabilities.</p> <p>“You can use it for any robot,” Konidaris says. “You tell it where it needs to go and we’ll handle the rest.”</p> <p>Realtime is part of MIT’s Industrial Liaison Program (ILP), which helps companies make connections with larger industrial partners, and it recently joined ILP’s STEX25 startup accelerator.</p> <p>With a few large rollouts planned for the coming months, the Realtime team’s excitement is driven by the belief that solving a problem as fundamental as motion planning unlocks a slew of new applications for the robotics field.</p> <p>“What I find most exciting about Realtime is that we are a true technology company,” says Konidaris. “The vast majority of startups are aimed at finding a new application for existing technology; often, there’s no real pushing of the technical boundaries with a new app or website, or even a new robotics ‘vertical.’ But we really did invent something new, and that edge and that energy is what drives us. All of that feels very MIT to me.”</p> The startup Realtime Robotics is helping robots solve the motion planning problem by giving them collision avoidance capabilities. Here, a robot avoids a researcher’s waving hand. Courtesy of Realtime Robotics Robot takes contact-free measurements of patients’ vital signs Mobile system could reduce health care workers’ exposure to Covid-19 virus. Mon, 31 Aug 2020 00:00:00 -0400 Anne Trafton | MIT News Office <p><em>The research described in this article has been published on a preprint server but has not yet been peer-reviewed by scientific or medical experts.</em></p> <p>During the current coronavirus pandemic, one of the riskiest parts of a health care worker’s job is assessing people who have symptoms of Covid-19. Researchers from MIT, Boston Dynamics, and Brigham and Women’s Hospital hope to reduce that risk by using robots to remotely measure patients’ vital signs.</p> <p>The robots, which are controlled by a handheld device, can also carry a tablet that allows doctors to ask patients about their symptoms without being in the same room.</p> <p>“In robotics, one of our goals is to use automation and robotic technology to remove people from dangerous jobs,” says Henwei Huang, an MIT postdoc. “We thought it should be possible for us to use a robot to remove the health care worker from the risk of directly exposing themselves to the patient.”</p> <p>Using four cameras mounted on a dog-like robot developed by Boston Dynamics, the researchers have shown that they can measure skin temperature, breathing rate, pulse rate, and blood oxygen saturation in healthy patients, from a distance of 2 meters. They are now making plans to test it in patients with Covid-19 symptoms.</p> <p>“We are thrilled to have forged this industry-academia partnership in which scientists with engineering and robotics expertise worked with clinical teams at the hospital to bring sophisticated technologies to the bedside,” says Giovanni Traverso, an MIT assistant professor of mechanical engineering, a gastroenterologist at Brigham and Women’s Hospital, and the senior author of the study.</p> <p>The researchers have posted a paper on their system on the preprint server techRxiv, and have submitted it to a peer-reviewed journal. Huang is one of the lead authors of the study, along with Peter Chai, an assistant professor of emergency medicine at Brigham and Women’s Hospital, and Claas Ehmke, a visiting scholar from ETH Zurich.</p> <p><strong>Measuring vital signs</strong></p> <p>When Covid-19 cases began surging in Boston in March, many hospitals, including Brigham and Women’s, set up triage tents outside their emergency departments to evaluate people with Covid-19 symptoms. One major component of this initial evaluation is measuring vital signs, including body temperature.</p> <p>The MIT and BWH researchers came up with the idea to use robotics to enable contactless monitoring of vital signs, to allow health care workers to minimize their exposure to potentially infectious patients. They decided to use existing computer vision technologies that can measure temperature, breathing rate, pulse, and blood oxygen saturation, and worked to make them mobile.</p> <p>To achieve that, they used a robot known as Spot, which can walk on four legs, similarly to a dog. Health care workers can maneuver the robot to wherever patients are sitting, using a handheld controller. The researchers mounted four different cameras onto the robot — an infrared camera plus three monochrome cameras that filter different wavelengths of light.</p> <p>The researchers developed algorithms that allow them to use the infrared camera to measure both elevated skin temperature and breathing rate. For body temperature, the camera measures skin temperature on the face, and the algorithm correlates that temperature with core body temperature. The algorithm also takes into account the ambient temperature and the distance between the camera and the patient, so that measurements can be taken from different distances, under different weather conditions, and still be accurate.</p> <p>Measurements from the infrared camera can also be used to calculate the patient’s breathing rate. As the patient breathes in and out, wearing a mask, their breath changes the temperature of the mask. Measuring this temperature change allows the researchers to calculate how rapidly the patient is breathing.</p> <p>The three monochrome cameras each filter a different wavelength of light — 670, 810, and 880 nanometers. These wavelengths allow the researchers to measure the slight color changes that result when hemoglobin in blood cells binds to oxygen and flows through blood vessels. The researchers’ algorithm uses these measurements to calculate both pulse rate and blood oxygen saturation.</p> <p>“We didn’t really develop new technology to do the measurements,” Huang says. “What we did is integrate them together very specifically for the Covid application, to analyze different vital signs at the same time.”</p> <p><strong>Continuous monitoring</strong></p> <p>In this study, the researchers performed the measurements on healthy volunteers, and they are now making plans to test their robotic approach in people who are showing symptoms of Covid-19, in a hospital emergency department.</p> <p>While in the near term, the researchers plan to focus on triage applications, in the longer term, they envision that the robots could be deployed in patients’ hospital rooms. This would allow the robots to continuously monitor patients and also allow doctors to check on them, via tablet, without having to enter the room. Both applications would require approval from the U.S. Food and Drug Administration.</p> <p>The research was funded by the MIT Department of Mechanical Engineering and the Karl van Tassel (1925) Career Development Professorship, and Boston Dynamics.</p> Researchers from MIT and Brigham and Women’s Hospital hope to reduce the risk to healthcare workers posed by Covid-19 by using robots to remotely measure patients’ vital signs. Credit: Spot image courtesy of the researchers, edited by MIT News New US postage stamp highlights MIT research For the robotics category in a new series celebrating innovation, the USPS chose the bionic prosthesis designed and built by the Media Lab's Biomechatronics group. Sun, 02 Aug 2020 00:00:00 -0400 Alexandra Kahn | MIT Media Lab <p>Letter writers across the country will soon have a fun and beautiful new Forever stamp to choose from, featuring novel research from the Media Lab's Biomechatronics research group.&nbsp;</p> <p>The stamp is part of a new U.S. Postal Service (USPS) series on innovation, representing computing, biomedicine, genome sequencing, robotics, and solar technology.&nbsp;For the robotics category, the USPS chose the bionic prosthesis designed and built by Matt Carney PhD ’20 and members of the Biomechatronics group, led by Professor Hugh Herr.</p> <p>The image used in the stamp was taken by photographer Andy Ryan, whose portfolio spans&nbsp;images from around the world, and who for many years has been capturing the MIT experience — from stunning architectural shots to the research work of labs across campus. Ryan suggested the bionic work of the biomechatronics group to USPS to represent the future of robotics. Ryan also created the images that became the computing and solar technology stamps in the series.&nbsp;</p> <p>“I was aware that Hugh Herr and his research team were incorporating robotic elements into the prosthetic legs they were developing and testing,” Ryan notes.&nbsp;“This vision of robotics was, in my mind, a true depiction of how robots and robotics would manifest and impact society in the future."&nbsp;</p> <p>With encouragement from Herr, Ryan submitted high-definition, stylized, and close-up images of Matt Carney working on the group's latest designs.&nbsp;</p> <p>Carney, who recently completed his PhD in media arts and sciences at the Media Lab, views bionic limbs as the ultimate humanoid robot, and an ideal innovation to represent and portray robotics in 2020. He was all-in for sharing that work with the world.</p> <p>"Robotic prostheses integrate biomechanics, mechanical, electrical, and software engineering, and no piece is off-the-shelf,” Carney says. “To attempt to fit within the confines of the human form, and to match the bandwidth and power density of the human body, we must push the bounds of every discipline: computation, strength of materials, magnetic energy densities, sensors, biological interfaces, and so much more."</p> <p>In his childhood, Carney himself collected stamps from different corners of the globe, and so the selection of his research for a U.S. postal stamp has been especially meaningful.&nbsp;</p> <p>"It's a freakin' honor to have my PhD work featured as a USPS stamp," Carney says, breaking into a big smile. "I hope this feat is an inspiration to young students everywhere to crush their homework, and to build the skills to make a positive impact on the world. And while I worked insane hours to build this thing — and really tried to inspire with its design as much as its engineering — it's truly the culmination of powered prosthesis work pioneered by Dr. Hugh Herr and our entire team at the Media Lab's Biomechatronics group, and it expands on work from a global community over more than a decade of development."</p> <p>The new MIT stamp joins a venerable list of other stamps associated with the Institute.&nbsp;Formerly issued stamps have featured Apollo 11 astronaut and moonwalker Buzz Aldrin ScD ’63, Nobel Prize winner Richard Feynman ’39, and architect Robert Robinson Taylor, who graduated from MIT in 1892 and is considered the nation’s first academically trained African American architect, followed by Pritzker Prize-winning architect I.M. Pei ’40, whose work includes the Louvre Glass Pyramid and the East Building on the National Gallery in Washington, as well as numerous buildings on the MIT campus.&nbsp;</p> <p>The new robotics stamp, however, is the first to feature MIT research, as well as members of the MIT community.</p> <p>"I'm deeply honored that a USPS Forever stamp has been created to celebrate technologically-advanced robotic prostheses, and along with that, the determination to alleviate human impairment," Herr says. "Through the marriage of human physiology and robotics, persons with leg amputation can now walk with powered prostheses that closely emulate the biological leg. By integrating synthetic sensors, artificial computation, and muscle-like actuation, these technologies are already improving people's lives in profound ways, and may one day soon bring about the end of disability."</p> <p>The Innovation Stamp series will be <a href="">available for purchase</a> through the U.S. Postal Service later this month.</p> <p>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;&nbsp;</p> Matt Carney, together with a team of researchers from the Media Lab's Biomechatronics group, designed and built the robotic prothesis featured in the new U.S. postage stamp. Photo: Andy Ryan Letting robots manipulate cables Robotic gripper with soft sensitive fingers developed at MIT can handle cables with unprecedented dexterity. Mon, 13 Jul 2020 07:00:00 -0400 Rachel Gordon | MIT CSAIL <p>For humans, it can be challenging to manipulate thin flexible objects like ropes, wires, or cables. But if these problems are hard for humans, they are nearly impossible for robots. As a cable slides between the fingers, its shape is constantly changing, and the robot’s fingers must be constantly sensing and adjusting the cable’s position and motion.</p> <p>Standard approaches have used a series of slow and incremental deformations, as well as mechanical fixtures, to get the job done. Recently, a group of researchers from MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL) and from the MIT Department of Mechanical Engineering pursued the task from a different angle, in a manner that more closely mimics us humans. The team’s <a href="" target="_blank">new system</a> uses a pair of soft robotic grippers with high-resolution tactile sensors (and no added mechanical constraints) to successfully manipulate freely moving cables.</p> <p>One could imagine using a system like this for both industrial and household tasks, to one day enable robots to help us with things like tying knots, wire shaping, or even surgical suturing.&nbsp;</p> <p>The team’s first step was to build a novel two-fingered gripper. The opposing fingers are lightweight and quick moving, allowing nimble, real-time adjustments of force and position. On the tips of the fingers are vision-based <a href="" target="_self">“GelSight” sensors</a>, built from soft rubber with embedded cameras. The gripper is mounted on a robot arm, which can move as part of the control system.</p> <p>The team’s second step was to create a perception-and-control framework to allow cable manipulation. For perception, they used the GelSight sensors to estimate the pose of the cable between the fingers, and to measure the frictional forces as the cable slides. Two controllers run in parallel: one modulates grip strength, while the other adjusts the gripper pose to keep the cable within the gripper.</p> <p>When mounted on the arm, the gripper could reliably follow a USB cable starting from a random grasp position. Then, in combination with a second gripper, the robot can move the cable “hand over hand” (as a human would) in order to find the end of the cable. It could also adapt to cables of different materials and thicknesses.</p> <p>As a further demo of its prowess, the robot performed an action that humans routinely do when plugging earbuds into a cell phone. Starting with a free-floating earbud cable, the robot was able to slide the cable between its fingers, stop when it felt the plug touch its fingers, adjust the plug’s pose, and finally insert the plug into the jack.&nbsp;</p> <p>“Manipulating soft objects is so common in our daily lives, like cable manipulation, cloth folding, and string knotting,” says Yu She, MIT postdoc and lead author on a new paper about the system. “In many cases, we would like to have robots help humans do this kind of work, especially when the tasks are repetitive, dull, or unsafe.”&nbsp;</p><p><strong>String me along</strong>&nbsp;</p><p>Cable following is challenging for two reasons. First, it requires controlling the “grasp force” (to enable smooth sliding), and the “grasp pose” (to prevent the cable from falling from the gripper’s fingers).&nbsp;&nbsp;</p><p>This information is hard to capture from conventional vision systems during continuous manipulation, because it’s usually occluded, expensive to interpret, and sometimes inaccurate.&nbsp;</p> <p>What’s more, this information can’t be directly observed with just vision sensors, hence the team’s use of tactile<em> </em>sensors. The gripper’s joints are also flexible — protecting them from potential impact.&nbsp;</p> <p>The algorithms can also be generalized to different cables with various physical properties like material, stiffness, and diameter, and also to those at different speeds.&nbsp;</p> <p>When comparing different controllers applied to the team’s gripper, their control policy could retain the cable in hand for longer distances than three others. For example, the “open-loop” controller only followed 36 percent of the total length, the gripper easily lost the cable when it curved, and it needed many regrasps to finish the task.&nbsp;</p> <p><strong>Looking ahead&nbsp;</strong></p> <p>The team observed that it was difﬁcult to pull the cable back when it reached the edge of the ﬁnger, because of the convex surface of the GelSight sensor. Therefore, they hope to improve the ﬁnger-sensor shape to enhance the overall performance.&nbsp;</p> <p>In the future, they plan to study more complex cable manipulation tasks such as cable routing and cable inserting through obstacles, and they want to eventually explore autonomous cable manipulation tasks in the auto industry.</p> <p>Yu She wrote the paper alongside MIT PhD students Shaoxiong Wang, Siyuan Dong, and Neha Sunil; Alberto Rodriguez,&nbsp;MIT associate professor of mechanical engineering; and Edward Adelson, the <span class="person__info__def">John and Dorothy Wilson Professor in the MIT Department of Brain and Cognitive Sciences</span>.&nbsp;</p> <p>This work was supported by the Amazon Research Awards, the Toyota Research Institute, and the Office of Naval Research.</p> The system uses a pair of soft robotic grippers with high-resolution tactile sensors to successfully manipulate freely moving cables. Photo courtesy of MIT CSAIL. CSAIL robot disinfects Greater Boston Food Bank Using UV-C light, the system can disinfect a warehouse floor in half an hour — and could one day be employed in grocery stores, schools, and other spaces. Sun, 28 Jun 2020 23:59:59 -0400 Rachel Gordon | MIT CSAIL <p>With every droplet that we can’t see, touch, or feel dispersed into the air, the threat of spreading Covid-19 persists. It’s become increasingly critical to keep these heavy droplets from lingering — especially on surfaces, which are welcoming and generous hosts.&nbsp;</p> <p>Thankfully, our chemical cleaning products are effective, but using them to disinfect larger settings can be expensive, dangerous, and time-consuming. Across the globe there are thousands of warehouses, grocery stores, schools, and other spaces where cleaning workers are at risk.</p> <p>With that in mind, a team from MIT’s <a href="">Computer Science and Artificial Intelligence Laboratory</a> (CSAIL), in collaboration with <a href="">Ava Robotics</a> and the <a href="">Greater Boston Food Bank</a> (GBFB), designed a new robotic system that powerfully disinfects surfaces and neutralizes aerosolized forms of the coronavirus.</p> <p>The approach uses a custom UV-C light fixture designed at CSAIL that is integrated with Ava Robotics’ mobile robot base. The results were encouraging enough that researchers say that the approach could be useful for autonomous UV disinfection in other environments, such as factories, restaurants, and supermarkets.&nbsp;</p> <p>UV-C light has proven to be effective at killing viruses and bacteria on surfaces and aerosols, but it’s unsafe for humans to be exposed. Fortunately, Ava’s telepresence robot doesn’t require any human supervision. Instead of the telepresence top, the team subbed in a UV-C array for disinfecting surfaces. Specifically, the array uses short-wavelength ultraviolet light to kill microorganisms and disrupt their DNA in a process called ultraviolet germicidal irradiation.</p> <p>The complete robot system is capable of mapping the space — in this case, GBFB’s warehouse — and navigating between waypoints and other specified areas. In testing the system, the team used a UV-C dosimeter, which confirmed that the robot was delivering the expected dosage of UV-C light predicted by the model.</p> <p>“Food banks provide an essential service to our communities, so it is critical to help keep these operations running,” says Alyssa Pierson, CSAIL research scientist and technical lead of the UV-C lamp assembly. “Here, there was a unique opportunity to provide additional disinfecting power to their current workflow, and help reduce the risks of Covid-19 exposure.”&nbsp;</p> <p>Food banks are also facing a particular demand due to the stress of Covid-19. The United Nations projected that, because of the virus, the number of people facing severe food insecurity worldwide <a href="">could double to 265 million</a>. In the United States alone, the five-week total of job losses has risen to 26 million, potentially pushing millions more into food insecurity.&nbsp;</p> <p>During tests at GBFB, the robot was able to drive by the pallets and storage aisles at a speed of roughly 0.22 miles per hour. At this speed, the robot could cover a 4,000-square-foot space in GBFB’s warehouse in just half an hour. The UV-C dosage delivered during this time can neutralize approximately 90 percent of coronaviruses on surfaces. For many surfaces, this dose will be higher, resulting in more of the virus neutralized.</p> <p>Typically, this method of ultraviolet germicidal irradiation is used largely in hospitals and medical settings, to sterilize patient rooms and stop the spread of microorganisms like <span>methicillin-resistant <em>staphylococcus aureus</em></span> and <span class="st"><em>Clostridium difficile</em></span>, and the UV-C light also works against airborne pathogens. While it’s most effective in the direct “line of sight,” it can get to nooks and crannies as the light bounces off surfaces and onto other surfaces.&nbsp;</p> <p>"Our 10-year-old warehouse is a relatively new food distribution facility with AIB-certified, state-of-the-art cleanliness and food safety standards,” says Catherine D’Amato, president and CEO of the Greater Boston Food Bank. “Covid-19 is a new pathogen that GBFB, and the rest of the world, was not designed to handle. We are pleased to have this opportunity to work with MIT CSAIL and Ava Robotics to innovate and advance our sanitation techniques to defeat this menace."&nbsp;</p> <p>As a first step, the team teleoperated the robot to teach it the path around the warehouse — meaning it’s equipped with autonomy to move around, without the team needing to navigate it remotely.&nbsp;</p> <p>It can go to defined waypoints on its map, such as going to the loading dock, then the warehouse shipping floor, then returning to base. They define those waypoints from the expert human user in teleop mode, and then can add new waypoints to the map as needed.&nbsp;</p> <p>Within GBFB, the team identified the warehouse shipping floor as a “high-importance area” for the robot to disinfect. Each day, workers stage aisles of products and arrange them for up to 50 pickups by partners and distribution trucks the next day. By focusing on the shipping area, it prioritizes disinfecting items leaving the warehouse to reduce Covid-19 spread out into the community.</p> <p>Currently, the team is exploring how to use its onboard sensors to adapt to changes in the environment, such that in new territory, the robot would adjust its speed to ensure the recommended dosage is applied to new objects and surfaces.&nbsp;</p> <p>A unique challenge is that the shipping area is constantly changing, so each night, the robot encounters a slightly new environment. When the robot is deployed, it doesn’t necessarily know which of the staging aisles will be occupied, or how full each aisle might be. Therefore, the team notes that they need to teach the robot to differentiate between the occupied and unoccupied aisles, so it can change its planned path accordingly.</p> <p>As far as production went, “in-house manufacturing” took on a whole new meaning for this prototype and the team. The UV-C lamps were assembled in Pierson's basement, and CSAIL PhD student Jonathan Romanishin crafted a makeshift shop in his apartment for the electronics board assembly.&nbsp;</p> <p>“As we drive the robot around the food bank, we are also researching new control policies that will allow the robot to adapt to changes in the environment and ensure all areas receive the proper estimated dosage,” says Pierson. “We are focused on remote operation to minimize&nbsp; human supervision, and, therefore, the additional risk of spreading Covid-19, while running our system.”&nbsp;</p> <p>For immediate next steps, the team is focused on increasing the capabilities of the robot at GBFB, as well as eventually implementing design upgrades. Their broader intention focuses on how to make these systems more capable at adapting to our world: how a robot can dynamically change its plan based on estimated UV-C dosages, how it can work in new environments, and how to coordinate teams of UV-C robots to work together.</p> <p>“We are excited to see the UV-C disinfecting robot support our community in this time of need,” says CSAIL director and project lead Daniela Rus. “The insights we received from the work at GBFB has highlighted several algorithmic challenges. We plan to tackle these in order to extend the scope of autonomous UV disinfection in complex spaces, including dorms, schools, airplanes, and grocery stores.”&nbsp;</p> <p>Currently, the team’s focus is on GBFB, although the algorithms and systems they are developing could be transferred to other use cases in the future, like warehouses, grocery stores, and schools.&nbsp;</p> <p>"MIT has been a great partner, and when they came to us, the team was eager to start the integration, which took just four weeks to get up and running,” says Ava Robotics CEO Youssef Saleh. “The opportunity for robots to solve workplace challenges is bigger than ever, and collaborating with MIT to make an impact at the food bank has been a great experience."&nbsp;</p> <p>Pierson and Romanishin worked alongside Hunter Hansen (software capabilities), Bryan Teague of MIT Lincoln Laboratory (who assisted with the UV-C lamp assembly), Igor Gilitschenski and Xiao Li (assisting with future autonomy research), MIT professors Daniela Rus and Saman Amarasinghe, and Ava leads Marcio Macedo and Youssef Saleh.&nbsp;</p> <p>This project was supported in part by Ava Robotics, who provided their platform and team support.</p> In tests, the CSAIL team's robot could disinfect a 4,000-square-foot space in the food bank's warehouse in just half an hour. Photo: Alyssa Pierson/CSAIL Photorealistic simulator made MIT robot racing competition a live online experience Teaching assistants in Robotics: Science and Systems pulled out all the stops to help engineering students race across the finish line this spring. Tue, 09 Jun 2020 15:20:01 -0400 Ashley Belanger | School of Engineering <p>Every spring, the basement of the Ray and Maria Stata Center becomes a racetrack for tiny self-driving cars that tear through the halls one by one. Sprinting behind each car on foot is a team of three to six students, sometimes carrying wireless routers or open laptops extended out like Olympic torches. Lining the basement walls, their classmates cheer them on, knowing the effort it took to program the algorithms steering the cars around the course during this annual MIT autonomous racing competition.</p> <p>The competition is the final project for Course 6.141/16.405 (Robotics: Science and Systems). It’s an end-of-semester event that gets pulses speeding, and prizes are awarded for finishing different race courses with the fastest times out of 20 teams.</p> <p>With campus evacuated this spring due to the Covid-19 pandemic, however, not a single robotic car burned rubber in the Stata Center basement. Instead, a new race was on as Luca Carlone, the Charles Stark Draper Assistant Professor of Aeronautics and Astronautics and member of the Institute for Data, Systems, and Society; Nicholas Roy, professor of aeronautics and astronautics; and teaching assistants (TAs) including Marcus Abate, Lukas Lao Beyer, and Caris Mariah Moses had only four weeks to figure out how to bring the excitement of this highly-anticipated race online.</p> <p>Because the lab sometimes uses a simple simulator for other research, Carlone says they considered taking the race in that direction. With this simple simulator, students could watch as their self-driving cars snaked around a flat map, like a car depicted by a dot moving along a GPS navigation system. Ultimately, they decided that wasn’t the right route. The racing competition needed to be noisy. Realistic. Exciting. The dynamics of the car needed to be nearly as complex as the robotic cars the students had planned to use. Building on his prior research in collaboration with MIT Lincoln Laboratory, Abate worked with Lao Beyer and engineering graduate student Sabina Chen to develop a new photorealistic simulator at the last minute.</p> <p>The race was back on, and Carlone was impressed by how everything from the cityscape to the sleek car designs looked “as realistic as possible.”</p> <p>“The modifications involved introducing an outdoor environment based on open-source assets, building in realistic car dynamics for the agent, and adding lidar sensors,” Abate says. “I also had to revamp the interfacing with Python and Robot Operating System (ROS) to make it all plug-and-play for the students.”</p> <p>What that means is that the race ran a lot like a racing game, such as Gran Turismo or Forza. Only instead of sitting on your couch thumbing the joystick to direct the car, students developed algorithms to anticipate every roadblock and bend ahead. For students, programming for this new environment was perhaps the biggest adjustment. “The simulator used an outdoor scene and a full-sized car with a very different dynamics model than the real-life race car in the Stata basement,” Abate says.</p> <p>The TAs also had to adjust to complications behind the scenes of the race’s new setting. “A huge amount of effort was put into the new simulator, as well as into the logistics of obtaining and evaluating students' software,” Lao Beyer says. “Usually, teams are able to configure the software on their race car however they want, but it is very difficult to accommodate for such a diversity of software setups in the virtual race.”</p><p>Once the simulator was ready, there was no time to troubleshoot, so TAs made themselves available to debug on the fly any issues that arose. “I think that saved the day for the final project and the final race,” Carlone says.</p> <p>Programming their autonomous racing code wasn’t the only way that students customized their race experience, though. Co-instructor Jane Abbott brought Writing, Rhetoric, and Professional Communication (WRAP) into the course. As coordinator of the communication-intensive team that focused on helping teams work effectively, she says she was worried the silence that often looms on Zoom would suck out all the energy of the race. She suggested the TAs add a soundtrack.</p> <p>In the end, the remote race ran for nearly four hours, bringing together more than 100 people in one Zoom call with commentators and Mario Kart music playing. “We got to watch every student’s solution with some cool visualization code running that showed the trajectory and any obstacles hit,” says Samuel Ubellacker, an electrical engineering and computer science student who raced this year. “We got to see how each team's solution ran much clearer in the simulator because the camera was always following the race car.”</p> <p>For Yorai Shaoul, another electrical engineering and computer science student in the race, getting out of the basement helped him become more engaged with other teams’ projects. “Before leaving campus, we found ourselves working long hours in the Stata basement,” Shaoul says. “So focused on our robot, we failed to notice that other teams were right there next to us the whole time.”</p> <p>During the race, other programming solutions his team had overlooked became clear. “The TAs showcased and narrated each team’s run, finally allowing us to see the diverse approaches other teams were developing,” Shaoul says.</p> <p>“One thing that was nice: When we've done it live in the tunnels, you can only see a part of it,” Abbott says. “You sort of stand at a fixed point and you see the car go by. It’s like watching the marathon: you see the runners for 100 yards and then they're gone.”</p> <p>Over Zoom, participants could watch every impressive cruise and spectacular crash as it happened, plus replays. Many stayed to watch, and Lao Beyer says, “We managed to retain as much excitement and suspense about the final challenge as possible.” Ubellacker agrees: “It was certainly an unforgettable experience!”</p> <p>For those students who don’t bro down with Mario, they could also choose the music they wanted to accompany their races. “Near, far, wherever you are,” these lyrics from one team’s choice to use the “Titanic” movie theme “My Heart Will Go On” are a wink to the extra challenge of collaborating as teams at a distance.</p> <p>One of the masters of ceremonies for the 2020 race, Marwa Abdulhai '20, was a TA last year and says one obvious benefit of the online race is that it’s a lot easier to figure out why your car crashed. “Pros of this virtual approach have been allowing students to race through the track multiple times and knowing that the car’s performance was primarily due to the algorithm and not any physical constraints,” Abdulhai says.</p> <p>For Ubellacker that was actually a con, though: “The biggest element that I missed without having a physical car was not being able to experience the differences between simulation and real life.” He says, “Part of the fun to me is designing a system that works perfectly in the simulator, and then getting to figure out all the crazy ways it will fail in the real world!”</p> <p>Shaoul says instead of working on one car, sometimes it felt like they were working on five individual cars that lived on each team member’s computer. “With one car, it was easy to see how well it did and what required fixing, whereas virtually it was more ambiguous,” Shaoul says. “We faced challenges with keeping track of up-to-date code versions and also simple communication.”</p> <p>Carlone was concerned students wouldn’t be as invested in their algorithms without the experience of seeing the car’s performance play out in real life to motivate them to push harder. “Every year, the record time on that Stata Center track was getting better and better,” he says. “This year, we were a bit concerned about the performance.”</p> <p>Fortunately, many students were very much still in the race, with some teams beating the most optimistic predictions, despite having to adjust to new racing conditions and greater challenges collaborating as a team fully online. The winning students completed the race courses sometimes three times faster than other teams, without any collisions. “It was just beyond expectation,” Carlone says.</p><p>Although this shift in the final project somewhat changed the takeaways from the course, Carlone says the experience will still advance algorithmic skills for students working on robotics, as well as introducing them to the intensity of communication required to work effectively as remote teams. “Many robotics groups are doing research using photorealistic simulation, because you can test conditions that you cannot test on the real robot,” he says. Co-instructor Roy says it worked so well, the new simulator might become a permanent feature of the course — not to replace the physical race, but as an extra element. “The robotics experience was good,” Carlone says of the 2020 race, but still: “The human experience is, of course, different.”</p> More than 100 people participated in a four-hour online robot race, which served as the final project for MIT Course 6.141/16.405 (Robotics: Science and Systems). Giving soft robots feeling In a pair of papers from MIT CSAIL, two teams enable better sense and perception for soft robotic grippers. Mon, 01 Jun 2020 09:00:00 -0400 Rachel Gordon | MIT CSAIL <p>One of the hottest topics in robotics is the field of soft robots, which utilizes squishy and flexible materials rather than traditional rigid materials. But soft robots have been limited due to their lack of good sensing. A good robotic gripper needs to feel what it is touching (tactile sensing), and it needs to sense the positions of its fingers (proprioception). Such sensing has been missing from most soft robots.</p> <p>In a new pair of papers, researchers from MIT’s <a href="">Computer Science and Artificial Intelligence Laboratory</a> (CSAIL) came up with new tools to let robots better perceive what they’re interacting with: the ability to see and classify items, and a softer, delicate touch.&nbsp;</p> <p>“We wish to enable seeing the world by feeling the world. Soft robot hands have sensorized skins that allow them to pick up a range of objects, from delicate, such as potato chips, to heavy, such as milk bottles,” says CSAIL Director Daniela Rus, the Andrew and Erna Viterbi Professor of Electrical Engineering and Computer Science and the deputy dean of research for the MIT Stephen A. Schwarzman College of Computing.&nbsp;</p> <p>This work was supported by the Toyota Research Institute.</p> <p><a href="">One paper</a> builds off last year’s <a href="">research</a> from MIT and Harvard University, where a team developed a soft and strong robotic gripper in the form of a cone-shaped origami structure. It collapses in on objects much like a Venus' flytrap, to pick up items that are as much as 100 times its weight.&nbsp;</p> <p>To get that newfound versatility and adaptability even closer to that of a human hand, a new team came up with a sensible addition: tactile sensors, made from latex “bladders” (balloons) connected to pressure transducers. The new sensors let the gripper not only pick up objects as delicate as potato chips, but it also classifies them — letting the robot better understand what it’s picking up, while also exhibiting that light touch.&nbsp;</p> <p>When classifying objects, the sensors correctly identified 10 objects with over 90 percent accuracy, even when an object slipped out of grip.</p> <p>“Unlike many other soft tactile sensors, ours can be rapidly fabricated, retrofitted into grippers, and show sensitivity and reliability,” says MIT postdoc Josie Hughes, the lead author on a new paper about the sensors. “We hope they provide a new method of soft sensing that can be applied to a wide range of different applications in manufacturing settings, like packing and lifting.”&nbsp;</p> <p>In <a href="">a second paper</a>, a group of researchers created a soft robotic finger called “GelFlex” that uses embedded cameras and deep learning to enable high-resolution tactile sensing and “proprioception” (awareness of positions and movements of the body).&nbsp;</p> <p>The gripper, which looks much like a two-finger cup gripper you might see at a soda station, uses a tendon-driven mechanism to actuate the fingers. When tested on metal objects of various shapes, the system had over 96 percent recognition accuracy.&nbsp;</p> <p>“Our soft finger can provide high accuracy on proprioception and accurately predict grasped objects, and also withstand considerable impact without harming the interacted environment and itself,” says Yu She, lead author on a new paper on GelFlex. “By constraining soft fingers with a flexible exoskeleton, and performing high-resolution sensing with embedded cameras, we open up a large range of capabilities for soft manipulators.”&nbsp;</p> <p><strong>Magic ball senses&nbsp;</strong></p> <p>The magic ball gripper is made from a soft origami structure, encased by a soft balloon. When a vacuum is applied to the balloon, the origami structure closes around the object, and the gripper deforms to its structure.&nbsp;</p> <p>While this motion lets the gripper grasp a much wider range of objects than ever before, such as soup cans, hammers, wine glasses, drones, and even a single broccoli floret, the greater intricacies of delicacy and understanding were still out of reach — until they added the sensors.&nbsp;&nbsp;</p> <p>When the sensors experience force or strain, the internal pressure changes, and the team can measure this change in pressure to identify when it will feel that again.&nbsp;</p> <p>In addition to the latex sensor, the team also developed an algorithm which uses feedback to let the gripper possess a human-like duality of being both strong and precise — and 80 percent of the tested objects were successfully grasped without damage.&nbsp;</p> <p>The team tested the gripper-sensors on a variety of household items, ranging from heavy bottles to small, delicate objects, including cans, apples, a toothbrush, a water bottle, and a bag of cookies.&nbsp;</p> <p>Going forward, the team hopes to make the methodology scalable, using computational design and reconstruction methods to improve the resolution and coverage using this new sensor technology. Eventually, they imagine using the new sensors to create a fluidic sensing skin that shows scalability and sensitivity.&nbsp;</p> <p>Hughes co-wrote the new paper with Rus, which they will present virtually at the 2020 International Conference on Robotics and Automation.&nbsp;</p> <p><strong>GelFlex</strong></p> <p>In the second paper, a CSAIL team looked at giving a soft robotic gripper more nuanced, human-like senses. Soft fingers allow a wide range of deformations, but to be used in a controlled way there must be rich tactile and proprioceptive sensing. The team used embedded cameras with wide-angle “fisheye” lenses that capture the finger’s deformations in great detail.</p> <p>To create GelFlex, the team used silicone material to fabricate the soft and transparent finger, and put one camera near the fingertip and the other in the middle of the finger. Then, they painted reflective ink on the front and side surface of the finger, and added LED lights on the back. This allows the internal fish-eye camera to observe the status of the front and side surface of the finger.&nbsp;</p> <p>The team trained neural networks to extract key information from the internal cameras for feedback. One neural net was trained to predict the bending angle of GelFlex, and the other was trained to estimate the shape and size of the objects being grabbed. The gripper could then pick up a variety of items such as a Rubik’s cube, a DVD case, or a block of aluminum.&nbsp;</p> <p>During testing, the average positional error while gripping was less than 0.77 millimeter, which is better than that of a human finger. In a second set of tests, the gripper was challenged with grasping and recognizing cylinders and boxes of various sizes. Out of 80 trials, only three were classified incorrectly.&nbsp;</p> <p>In the future, the team hopes to improve the proprioception and tactile sensing algorithms, and utilize vision-based sensors to estimate more complex finger configurations, such as twisting or lateral bending, which are challenging for common sensors, but should be attainable with embedded cameras.</p> <p>Yu She co-wrote the GelFlex paper with MIT graduate student Sandra Q. Liu, Peiyu Yu of Tsinghua University, and MIT Professor Edward Adelson. They will present the paper virtually at the 2020 International Conference on Robotics and Automation.</p> <p></p> Professor Ted Adelson's team created a soft robotic finger that uses embedded cameras and deep learning to enable high-resolution tactile sensing and “proprioception” (awareness of positions and movements of the body). Photo courtesy of the researchers. Muscle signals can pilot a robot CSAIL's Conduct-A-Bot system uses muscle signals to cue a drone’s movement, enabling more natural human-robot communication. Mon, 27 Apr 2020 13:30:01 -0400 Rachel Gordon | CSAIL <p>Albert Einstein famously postulated that “the only real valuable thing is intuition,” arguably one of the most important keys to understanding intention and communication.&nbsp;</p> <p>But intuitiveness is hard to teach — especially to a machine. Looking to improve this, a team from MIT’s <a href="">Computer Science and Artificial Intelligence Laboratory</a> (CSAIL) came up with a method that dials us closer to more seamless human-robot collaboration. The system, called “Conduct-A-Bot,” uses human muscle signals from wearable sensors to pilot a robot’s movement.&nbsp;</p> <p>“We envision a world in which machines help people with cognitive and physical work, and to do so, they adapt to people rather than the other way around,” says Professor Daniela Rus, director of CSAIL, deputy dean of research for the MIT Stephen A. Schwarzman College of Computing, and co-author on a paper about the system.&nbsp;</p> <p>To enable seamless teamwork between people and machines, electromyography and motion sensors are worn on the biceps, triceps, and forearms to measure muscle signals and movement. Algorithms then process the signals to detect gestures in real time, without any offline calibration or per-user training data. The system uses just two or three wearable sensors, and nothing in the environment — largely reducing the barrier to casual users interacting with robots.</p> <p>This work was funded, in part, by the Boeing Company.&nbsp;</p> <p>While Conduct-A-Bot could potentially be used for various scenarios, including navigating menus on electronic devices or supervising autonomous robots, for this research the team used a Parrot Bebop 2 drone, although any commercial drone could be used.</p> <p>By detecting actions like rotational gestures, clenched fists, tensed arms, and activated forearms, Conduct-A-Bot can move the drone left, right, up, down, and forward, as well as allow it to rotate and stop.&nbsp;</p> <p>If you gestured toward the right to your friend, they could likely interpret that they should move in that direction. Similarly, if you waved your hand to the left, for example, the drone would follow suit and make a left turn.&nbsp;</p> <p>In tests, the drone correctly responded to 82 percent of over 1,500 human gestures when it was remotely controlled to fly through hoops. The system also correctly identified approximately 94 percent of cued gestures when the drone was not being controlled.</p> <p>“Understanding our gestures could help robots interpret more of the nonverbal cues that we naturally use in everyday life,” says Joseph DelPreto, lead author on the new paper. “This type of system could help make interacting with a robot more similar to interacting with another person, and make it easier for someone to start using robots without prior experience or external sensors.”&nbsp;</p> <p>This type of system could eventually target a range of applications for human-robot collaboration, including remote exploration, assistive personal robots, or manufacturing tasks like delivering objects or lifting materials.&nbsp;</p> <p>These intelligent tools are also consistent with social distancing — and could potentially open up a realm of future contactless work. For example, you can imagine machines being controlled by humans to safely clean a hospital room, or drop off medications, while letting us humans stay a safe distance.</p> <p>Muscle signals can often provide information about states that are hard to observe from vision, such as joint stiffness or fatigue.&nbsp;&nbsp;&nbsp;&nbsp;</p> <p>For example, if you watch a video of someone holding a large box, you might have difficulty guessing how much effort or force was needed — and a machine would also have difficulty gauging that from vision alone. Using muscle sensors opens up possibilities to estimate not only motion, but also the force and torque required to execute that physical trajectory.</p> <p>For the gesture vocabulary currently used to control the robot, the movements were detected as follows:&nbsp;</p> <ul> <li> <p>stiffening the upper arm to stop the robot (similar to briefly cringing when seeing something going wrong): biceps and triceps muscle signals;</p> </li> <li> <p>waving the hand left/right and up/down to move the robot sideways or vertically: forearm muscle signals (with the forearm accelerometer indicating hand orientation);</p> </li> <li> <p>fist clenching to move the robot forward: forearm muscle signals; and</p> </li> <li> <p>rotating clockwise/counterclockwise to turn the robot: forearm gyroscope.</p> </li> </ul> <p>Machine learning classifiers detected the gestures using the wearable sensors. Unsupervised classifiers processed the muscle and motion data and clustered it in real time to learn how to separate gestures from other motions. A neural network also predicted wrist flexion or extension from forearm muscle signals.&nbsp;&nbsp;</p> <p>The system essentially calibrates itself to each person's signals while they're making gestures that control the robot, making it faster and easier for casual users to start interacting with robots.</p> <p>In the future, the team hopes to expand the tests to include more subjects. And while the movements for Conduct-A-Bot cover common gestures for robot motion, the researchers want to extend the vocabulary to include more continuous or user-defined gestures. Eventually, the hope is to have the robots learn from these interactions to better understand the tasks and provide more predictive assistance or increase their autonomy.&nbsp;</p> <p>“This system moves one step closer to letting us work seamlessly with robots so they can become more effective and intelligent tools for everyday tasks,” says DelPreto. “As such collaborations continue to become more accessible and pervasive, the possibilities for synergistic benefit continue to deepen.”&nbsp;</p> <p>DelPreto and Rus presented the paper virtually earlier this month at the ACM/IEEE International Conference on Human Robot Interaction.</p> Lead author Joseph DelPreto controls a "Conduct-A-Bot" drone with his arm muscles. Photo courtesy of the researchers. Showing robots how to do your chores By observing humans, robots learn to perform complex tasks, such as setting a table. Thu, 05 Mar 2020 23:59:59 -0500 Rob Matheson | MIT News Office <p>Training interactive robots may one day be an easy job for everyone, even those without programming expertise. Roboticists are developing automated robots that can learn new tasks solely by observing humans. At home, you might someday show a domestic robot how to do routine chores. In the workplace, you could train robots like new employees, showing them how to perform many duties.</p><p>Making progress on that vision, MIT researchers have designed a system that lets these types of robots learn complicated tasks that would otherwise stymie them with too many confusing rules. One such task is setting a dinner table under certain conditions. &nbsp;</p><p>At its core, the researchers’ “Planning with Uncertain Specifications” (PUnS) system gives robots the humanlike planning ability to simultaneously weigh many ambiguous —&nbsp;and potentially contradictory —&nbsp;requirements to reach an end goal. In doing so, the system always chooses the most likely action to take, based on a “belief” about some probable specifications for the task it is supposed to perform.</p><p>In their work, the researchers compiled a dataset with information about how eight objects — a mug, glass, spoon, fork, knife, dinner plate, small plate, and bowl — could be placed on a table in various configurations. A robotic arm first observed randomly selected human demonstrations of setting the table with the objects. Then, the researchers tasked the arm with automatically setting a table in a specific configuration, in real-world experiments and in simulation, based on what it had seen.</p><p>To succeed, the robot had to weigh many possible placement orderings, even when items were purposely removed, stacked, or hidden. Normally, all of that would confuse robots too much. But the researchers’ robot made no mistakes over several real-world experiments, and only a handful of mistakes over tens of thousands of simulated test runs. &nbsp;</p><p>“The vision is to put programming in the hands of domain experts, who can program robots through intuitive ways, rather than describing orders to an engineer to add to their code,” says first author Ankit Shah, a graduate student in the Department of Aeronautics and Astronautics (AeroAstro) and the Interactive Robotics Group, who emphasizes that their work is just one step in fulfilling that vision. “That way, robots won’t have to perform preprogrammed tasks anymore. Factory workers can teach a robot to do multiple complex assembly tasks. Domestic robots can learn how to stack cabinets, load the dishwasher, or set the table from people at home.”</p><p>Joining Shah on the paper are AeroAstro and Interactive Robotics Group graduate student Shen Li and Interactive Robotics Group leader Julie Shah, an associate professor in AeroAstro and the Computer Science and Artificial Intelligence Laboratory.</p><p></p> <p><strong>Bots hedging bets</strong></p><p>Robots are fine planners in tasks with clear “specifications,” which help describe the task the robot needs to fulfill, considering its actions, environment, and end goal. Learning to set a table by observing demonstrations, is full of uncertain specifications. Items must be placed in certain spots, depending on the menu and where guests are seated, and in certain orders, depending on an item’s immediate availability or social conventions. Present approaches to planning are not capable of dealing with such uncertain specifications.</p><p>A popular approach to planning is “reinforcement learning,” a trial-and-error machine-learning technique that rewards and penalizes them for actions as they work to complete a task. But for tasks with uncertain specifications, it’s difficult to define clear rewards and penalties. In short, robots never fully learn right from wrong.</p><p>The researchers’ system, called PUnS (for Planning with Uncertain Specifications), enables a robot to hold a “belief” over a range of possible specifications. The belief itself can then be used to dish out rewards and penalties. “The robot is essentially hedging its bets in terms of what’s intended in a task, and takes actions that satisfy its belief, instead of us giving it a clear specification,” Ankit Shah says.</p><p>The system is built on “linear temporal logic” (LTL), an expressive language that enables robotic reasoning about current and future outcomes. The researchers defined templates in LTL that model various time-based conditions, such as what must happen now, must eventually happen, and must happen until something else occurs. The robot’s observations of 30 human demonstrations for setting the table yielded a probability distribution over 25 different LTL formulas. Each formula encoded a slightly different preference — or specification — for setting the table. That probability distribution becomes its belief.</p><p>“Each formula encodes something different, but when the robot considers various combinations of all the templates, and tries to satisfy everything together, it ends up doing the right thing eventually,” Ankit Shah says.</p><p><strong>Following criteria</strong></p><p>The researchers also developed several criteria that guide the robot toward satisfying the entire belief over those candidate formulas. One, for instance, satisfies the most likely formula, which discards everything else apart from the template with the highest probability. Others satisfy the largest number of unique formulas, without considering their overall probability, or they satisfy several formulas that represent highest total probability. Another simply minimizes error, so the system ignores formulas with high probability of failure.</p><p>Designers can choose any one of the four criteria to preset before training and testing. Each has its own tradeoff between flexibility and risk aversion. The choice of criteria depends entirely on the task. In safety critical situations, for instance, a designer may choose to limit possibility of failure. But where consequences of failure are not as severe, designers can choose to give robots greater flexibility to try different approaches.</p><p>With the criteria in place, the researchers developed an algorithm to convert the robot’s belief — the probability distribution pointing to the desired formula — into an equivalent reinforcement learning problem. This model will ping the robot with a reward or penalty for an action it takes, based on the specification it’s decided to follow.</p><p>In simulations asking the robot to set the table in different configurations, it only made six mistakes out of 20,000 tries. In real-world demonstrations, it showed behavior similar to how a human would perform the task. If an item wasn’t initially visible, for instance, the robot would finish setting the rest of the table without the item. Then, when the fork was revealed, it would set the fork in the proper place. “That’s where flexibility is very important,” Ankit Shah says. “Otherwise it would get stuck when it expects to place a fork and not finish the rest of table setup.”</p><p>Next, the researchers hope to modify the system to help robots change their behavior based on verbal instructions, corrections, or a user’s assessment of the robot’s performance. “Say a person demonstrates to a robot how to set a table at only one spot. The person may say, ‘do the same thing for all other spots,’ or, ‘place the knife before the fork here instead,’” Ankit Shah says. “We want to develop methods for the system to naturally adapt to handle those verbal commands, without needing additional demonstrations.”&nbsp;&nbsp;</p> Roboticists are developing automated robots that can learn new tasks solely by observing humans. At home, you might someday show a domestic robot how to do routine chores. Image: Christine Daniloff, MIT To self-drive in the snow, look under the road Weather’s a problem for autonomous cars. MIT’s new system shows promise by using “ground-penetrating radar” instead of cameras or lasers. Wed, 26 Feb 2020 14:50:00 -0500 Adam Conner-Simons | MIT CSAIL <p>Car companies have been feverishly working to improve the technologies behind self-driving cars. But so far even the most high-tech vehicles still fail when it comes to safely navigating in rain and snow.&nbsp;</p> <p>This is because these weather conditions wreak havoc on the most common approaches for sensing, which usually involve either lidar sensors or cameras. In the snow, for example, cameras can no longer recognize lane markings and traffic signs, while the lasers of lidar sensors malfunction when there’s, say, stuff flying down from the sky.</p> <p>MIT researchers have recently been wondering whether an entirely different approach might work. Specifically, what if we instead looked under the road?&nbsp;</p> <p>A team from MIT’s <a href="">Computer Science and Artificial Intelligence Laboratory</a> (CSAIL) has developed a new system that uses an existing technology called ground-penetrating radar (GPR) to send electromagnetic pulses underground that measure the area’s specific combination of soil, rocks, and roots. Specifically, the CSAIL team used a particular form of GPR instrumentation developed at <a href="" target="_blank">MIT Lincoln Laboratory</a> called <a href="" target="_self">localizing ground-penetrating radar</a>, or LGPR. The mapping process creates a unique fingerprint of sorts that the car can later use to localize itself when it returns to that particular plot of land.</p> <p>“If you or I grabbed a shovel and dug it into the ground, all we’re going to see is a bunch of dirt,” says CSAIL PhD student Teddy Ort, lead author on a new paper about the project that will be published in the <em>IEEE Robotics and Automation Letters</em> journal later this month. “But LGPR can quantify the specific elements there and compare that to the map it’s already created, so that it knows exactly where it is, without needing cameras or lasers.”</p> <p>In tests, the team found that in snowy conditions the navigation system’s average margin of error was on the order of only about an inch compared to clear weather. The researchers were surprised to find that it had a bit more trouble with rainy conditions, but was still only off by an average of 5.5 inches. (This is because rain leads to more water soaking into the ground, leading to a larger disparity between the original mapped LGPR reading and the current condition of the soil.)</p> <p>The researchers said the system’s robustness was further validated by the fact that, over a period of six months of testing, they never had to unexpectedly step in to take the wheel.&nbsp;</p> <p>“Our work demonstrates that this approach is actually a practical way to help self-driving cars navigate poor weather without actually having to be able to ‘see’ in the traditional sense using laser scanners or cameras,” says MIT Professor Daniela Rus, director of CSAIL and senior author on the new paper, which will also be presented in May at the International Conference on Robotics and Automation in Paris.</p> <p>While the team has only tested the system at low speeds on a closed country road, Ort said that existing work from Lincoln Laboratory suggests that the system could easily be extended to highways and other high-speed areas.&nbsp;</p> <p>This is the first time that developers of self-driving systems have employed ground-penetrating radar, which has previously been used in fields like construction planning, landmine detection, and even <a href="" target="_blank">lunar exploration</a>. The approach wouldn’t be able to work completely on its own, since it can’t detect things above ground. But its ability to localize in bad weather means that it would couple nicely with lidar and vision approaches.</p> <p>“Before releasing autonomous vehicles on public streets, localization and navigation have to be totally reliable at all times,” says Roland Siegwart, a professor of autonomous systems at ETH Zurich who was not involved in the project. “The CSAIL team’s innovative and novel concept has the potential to push autonomous vehicles much closer to real-world deployment.”&nbsp;</p> <p>One major benefit of mapping out an area with LGPR is that underground maps tend to hold up better over time than maps created using vision or lidar, since features of an above-ground map are much more likely to change. LGPR maps also take up only about 80 percent of the space used by traditional 2D sensor maps that many companies use for their cars.&nbsp;</p> <p>While the system represents an important advance, Ort notes that it’s far from road-ready. Future work will need to focus on designing mapping techniques that allow LGPR datasets to be stitched together to be able to deal with multi-lane roads and intersections. In addition, the current hardware is bulky and 6 feet wide, so major design advances need to be made before it’s small and light enough to fit into commercial vehicles.</p> <p>Ort and Rus co-wrote the paper with CSAIL postdoc Igor Gilitschenski. The project was supported, in part, by MIT Lincoln Laboratory.</p> MIT's new system allows a self-driving car to situate itself in snowy conditions. Photo courtesy of the researchers “Sensorized” skin helps soft robots find their bearings Flexible sensors and an artificial intelligence model tell deformable robots how their bodies are positioned in a 3D environment. Wed, 12 Feb 2020 23:59:59 -0500 Rob Matheson | MIT News Office <p>For the first time, MIT researchers have enabled a soft robotic arm to understand its configuration in 3D space, by leveraging only motion and position data from its own “sensorized” skin.</p><p>Soft robots constructed from highly compliant materials, similar to those found in living organisms, are being championed as safer, and more adaptable, resilient, and bioinspired alternatives to traditional rigid robots. But giving autonomous control to these deformable robots is a monumental task because they can move in a virtually infinite number of directions at any given moment. That makes it difficult to train planning and control models that drive automation.</p><p>Traditional methods to achieve autonomous control use large systems of multiple motion-capture cameras that provide the robots feedback about 3D movement and positions. But those are impractical for soft robots in real-world applications.</p><p>In a paper being published in the journal <em>IEEE Robotics and Automation Letters</em>, the researchers describe a system of soft sensors that cover a robot’s body to provide “proprioception” — meaning awareness of motion and position of its body. That feedback runs into a novel deep-learning model that sifts through the noise and captures clear signals to estimate the robot’s 3D configuration. The researchers validated their system on a soft robotic arm resembling an elephant trunk, that can predict its own position as it autonomously swings around and extends.</p><p>The sensors can be fabricated using off-the-shelf materials, meaning any lab can develop their own systems, says Ryan Truby, a postdoc in the MIT Computer Science and Artificial Laboratory (CSAIL) who is co-first author on the paper along with CSAIL postdoc Cosimo Della Santina.</p><p>“We’re sensorizing soft robots to get feedback for control from sensors, not vision systems, using a very easy, rapid method for fabrication,” he says. “We want to use these soft robotic trunks, for instance, to orient and control themselves automatically, to pick things up and interact with the world. This is a first step toward that type of more sophisticated automated control.”</p><p>One future aim is to help make artificial limbs that can more dexterously handle and manipulate objects in the environment. “Think of your own body: You can close your eyes and reconstruct the world based on feedback from your skin,” says co-author Daniela Rus, director of CSAIL and the Andrew and Erna Viterbi Professor of Electrical Engineering and Computer Science. “We want to design those same capabilities for soft robots.”</p><p><strong>Shaping soft sensors</strong></p><p>A longtime goal in soft robotics has been fully integrated body sensors. Traditional rigid sensors detract from a soft robot body’s natural compliance, complicate its design and fabrication, and can cause various mechanical failures. Soft-material-based sensors are a more suitable alternative, but require specialized materials and methods for their design, making them difficult for many robotics labs to fabricate and integrate in soft robots.</p><p>While working in his CSAIL lab one day looking for inspiration for sensor materials, Truby made an interesting connection. “I found these sheets of conductive materials used for electromagnetic interference shielding, that you can buy anywhere in rolls,” he says. These materials have “piezoresistive” properties, meaning they change in electrical resistance when strained. Truby realized they could make effective soft sensors if they were placed on certain spots on the trunk. As the sensor deforms in response to the trunk’s stretching and compressing, its electrical resistance is converted to a specific output voltage. The voltage is then used as a signal correlating to that movement.</p><p>But the material didn’t stretch much, which would limit its use for soft robotics. Inspired by kirigami —&nbsp;a variation of origami that includes making cuts in a material — Truby designed and laser-cut rectangular strips of conductive silicone sheets into various patterns, such as rows of tiny holes or crisscrossing slices like a chain link fence. That made them far more flexible, stretchable, “and beautiful to look at,” Truby says.</p> <p><img alt="" src="/sites/default/files/images/inline/images/sensorized-skin-1_0.gif" style="width: 500px; height: 281px;" /></p> <p><img alt="" src="/sites/default/files/images/inline/images/sensorized-skin-2.gif" style="width: 500px; height: 281px;" /></p> <p><em style="font-size: 10px;">Credit: Ryan L. Truby, MIT&nbsp;CSAIL</em></p><p>The researchers’ robotic trunk comprises three segments, each with four fluidic actuators (12 total) used to move the arm. They fused one sensor over each segment, with each sensor covering and gathering data from one embedded actuator in the soft robot. They used “plasma bonding,” a technique that energizes a surface of a material to make it bond to another material. It takes roughly a couple hours to shape dozens of sensors that can be bonded to the soft robots using a handheld plasma-bonding device.</p> <p><img alt="" src="/sites/default/files/images/inline/images/sensorized-skin-3.gif" style="width: 500px; height: 281px;" /></p> <p><span style="font-size:10px;"><em>Credit: Ryan L. Truby, MIT&nbsp;CSAIL</em></span></p><p><strong>“Learning” configurations</strong></p><p>As hypothesized, the sensors did capture the trunk’s general movement. But they were really noisy. “Essentially, they’re nonideal sensors in many ways,” Truby says. “But that’s just a common fact of making sensors from soft conductive materials. Higher-performing and more reliable sensors require specialized tools that most robotics labs do not have.”</p> <p>To estimate the soft robot’s configuration using only the sensors, the researchers built a deep neural network to do most of the heavy lifting, by sifting through the noise to capture meaningful feedback signals. The researchers developed a new model to kinematically describe the soft robot’s shape that vastly reduces the number of variables needed for their model to process.</p><p>In experiments, the researchers had the trunk swing around and extend itself in random configurations over approximately an hour and a half. They used the traditional motion-capture system for ground truth data. In training, the model analyzed data from its sensors to predict a configuration, and compared its predictions to that ground truth data which was being collected simultaneously. In doing so, the model “learns” to map signal patterns from its sensors to real-world configurations. Results indicated, that for certain and steadier configurations, the robot’s estimated shape matched the ground truth.</p><p>Next, the researchers aim to explore new sensor designs for improved sensitivity and to develop new models and deep-learning methods to reduce the required training for every new soft robot. They also hope to refine the system to better capture the robot’s full dynamic motions.</p><p>Currently, the neural network and sensor skin are not sensitive to capture subtle motions or dynamic movements. But, for now, this is an important first step for learning-based approaches to soft robotic control, Truby says: “Like our soft robots, living systems don’t have to be totally precise. Humans are not precise machines, compared to our rigid robotic counterparts, and we do just fine.”</p> MIT researchers have created a “sensorized” skin, made with kirigami-inspired sensors, that gives soft robots greater awareness of the motion and position of their bodies. Ryan L. Truby, MIT CSAIL Engineers design bionic “heart” for testing prosthetic valves, other cardiac devices Device made of heart tissue and a robotic pumping system beats like the real thing. Wed, 29 Jan 2020 14:00:00 -0500 Jennifer Chu | MIT News Office <p>As the geriatric population is expected to balloon in the coming decade, so too will rates of heart disease in the United States. The demand for prosthetic heart valves and other cardiac devices — a market that is valued at more than $5 billion dollars today — is predicted to rise by almost 13 percent in the next six years.</p><p>Prosthetic valves are designed to mimic a real, healthy heart valve in helping to circulate blood through the body. However, many of them have issues such as leakage around the valve, and engineers working to improve these designs must test them repeatedly, first in simple benchtop simulators, then in animal subjects, before reaching human trials — an arduous and expensive process.</p><p>Now engineers at MIT and elsewhere have developed a bionic “heart” that offers a more realistic model for testing out artificial valves and other cardiac devices.</p><p>The device is a real biological heart whose tough muscle tissue has been replaced with a soft robotic matrix of artificial heart muscles, resembling bubble wrap. The orientation of the artificial muscles mimics the pattern of the heart’s natural muscle fibers, in such a way that when the researchers remotely inflate the bubbles, they act together to squeeze and twist the inner heart, similar to the way a real, whole heart beats and pumps blood.</p><p>With this new design, which they call a “biorobotic hybrid heart,” the researchers envision that device designers and engineers could iterate and fine-tune designs more quickly by testing on the biohybrid heart, significantly reducing the cost of cardiac device development.</p><p>“Regulatory testing of cardiac devices requires many fatigue tests and animal tests,” says Ellen Roche, assistant professor of mechanical engineering at MIT. “[The new device] could realistically represent what happens in a real heart, to reduce the amount of animal testing or iterate the design more quickly.”</p><p>Roche and her colleagues have published their results today in the journal <em>Science Robotics.</em> Her co-authors are lead author and MIT graduate student Clara Park, along with Yiling Fan, Gregor Hager, Hyunwoo Yuk, Manisha Singh, Allison Rojas, and Xuanhe Zhao at MIT, along with collaborators from Nanyang Technology University, the Royal College of Surgeons in Dublin, Boston’s Children’s Hospital, Harvard Medical School, and Massachusetts General Hospital.</p> <p><img alt="" src="/sites/default/files/images/inline/images/biorobotic-heart-1.gif" style="width: 500px; height: 615px;" /></p> <p><em><span style="font-size:10px;">The structure of the biorobotic hybrid heart under magnetic resonance imaging. Credit: Christopher T. Nguyen</span></em></p><p><strong>“Mechanics of the heart”</strong></p><p>Before coming to MIT, Roche worked briefly in the biomedical industry, helping to test cardiac devices on artificial heart models in the lab.</p><p>“At the time I didn’t feel any of these benchtop setups were representative of both the anatomy and the physiological biomechanics of the heart,” Roche recalls. “There was an unmet need in terms of device testing.”</p><p>In separate research as part of her doctoral work at Harvard University, she developed a soft, robotic, implantable sleeve, designed to wrap around a whole, live heart, to help it pump blood in patients suffering from heart failure.</p><p>At MIT, she and Park wondered if they could combine the two research avenues, to develop a hybrid heart: a heart that is made partly of chemically preserved, explanted heart tissue and partly of soft artificial actuators that help the heart pump blood. Such a model, they proposed, should be a more realistic and durable environment in which to test cardiac devices, compared with models that are either entirely artificial but do not capture the heart’s complex anatomy, or are made from a real explanted heart, requiring highly controlled conditions to keep the tissue alive.</p><p>The team briefly considered wrapping a whole, explanted heart in a soft robotic sleeve, similar to Roche’s previous work, but realized the heart’s outer muscle tissue, the myocardium, quickly stiffened when removed from the body. Any robotic contraction by the sleeve would fail to translate sufficiently to the heart within.</p><p>Instead, the team looked for ways to design a soft robotic matrix to replace the heart’s natural muscle tissue, in both material and function. They decided to try out their idea first on the heart’s left ventricle, one of four chambers in the heart, which pumps blood to the rest of the body, while the right ventricle uses less force to pump blood to the lungs.</p><p>“The left ventricle is the harder one to recreate given its higher operating pressures, and we like to start with the hard challenges,” Roche says.</p><p><strong>The heart, unfurled</strong></p><p>The heart normally pumps blood by squeezing and twisting, a complex combination of motions that is a result of the alignment of muscle fibers along the outer myocardium that covers each of the heart’s ventricles. The team planned to fabricate a matrix of artificial muscles resembling inflatable bubbles, aligned in the orientations of the natural cardiac muscle. But copying these patterns by studying a ventricle’s three-dimensional geometry proved extremely challenging.</p><p>They eventually came across the helical ventricular myocardial band theory, the idea that cardiac muscle is essentially a large helical band that wraps around each of the heart’s ventricles. This theory is still a subject of debate by some researchers, but Roche and her colleagues took it as inspiration for their design. Instead of trying to copy the left ventricle’s muscle fiber orientation from a 3D perspective, the team decided to remove the ventricle’s outer muscle tissue and unwrap it to form a long, flat band — a geometry that should be far easier to recreate. In this case, they used the cardiac tissue from an explanted pig heart.</p><p>In collaboration with co-lead author Chris Nguyen at MGH, the researchers used diffusion tensor imaging, an advanced technique that typically tracks how water flows through white matter in the brain, to map the microscopic fiber orientations of a left ventricle’s unfurled, two-dimensional muscle band. They then fabricated a matrix of artificial muscle fibers made from thin air tubes, each connected to a series of inflatable pockets, or bubbles, the orientation of which they patterned after the imaged muscle fibers.</p> <p><img alt="" src="/sites/default/files/images/inline/images/biorobotic-heart-2.gif" style="width: 500px; height: 380px;" /></p> <p><em><span style="font-size:10px;">Motion of the biorobotic hybrid heart mimics the pumping motion of the heart under echocardiography. Credit: Mossab Saeed</span></em></p><p>The soft matrix consists of two layers of silicone, with a water-soluble layer between them to prevent the layers from sticking, as well as two layers of laser-cut paper, which ensures that the bubbles inflate in a specific orientation.</p><p>The researchers also developed a new type of bioadhesive to glue the bubble wrap to the ventricle’s real, intracardiac tissue. While adhesives exist for bonding biological tissues to each other, and and for materials like silicone to each other, the team &nbsp;realized few soft adhesives do an adequate job of gluing together biological tissue with synthetic materials, silicone in particular.</p><p>So Roche collaborated with Zhao, associate professor of mechanical engineering at MIT, who specializes in developing hydrogel-based adhesives. The new adhesive, named TissueSil, was made by functionalizing silicone in a chemical cross-linking process, to bond with components in heart tissue. The result was a viscous liquid that the researchers brushed onto the soft robotic matrix. They also brushed the glue onto a new explanted pig heart that had its left ventricle removed but its endocardial structures preserved. When they wrapped the artificial muscle matrix around this tissue, the two bonded tightly.</p><p>Finally, the researchers placed the entire hybrid heart in a mold that they had previously cast of the original, whole heart, and filled the mold with silicone to encase the hybrid heart in a uniform covering — a step that produced a form similar to a real heart and ensured that the robotic bubble wrap fit snugly around the real ventricle.</p><p>“That way, you don’t lose transmission of motion from the synthetic muscle to the biological tissue,” Roche says.</p><p>When the researchers pumped air into the bubble wrap at frequencies resembling a naturally beating heart, and imaged the bionic heart’s response, it contracted in a manner similar to the way a real heart moves to pump blood through the body.</p><p>Ultimately, the researchers hope to use the bionic heart as a realistic environment to help designers test cardiac devices such as prosthetic heart valves.</p><p>“Imagine that a patient before cardiac device implantation could have their heart scanned, and then clinicians could tune the device to perform optimally in the patient well before the surgery,” says Nyugen. “Also, with further tissue engineering, we could potentially see the biorobotic hybrid heart be used as an artificial heart — a very needed potential solution given the global heart failure epidemic where millions of people are at the mercy of a competitive heart transplant list.”</p><p>This research was supported in part by the National Science Foundation.</p> A preserved heart muscle (1) is removed and replaced with a soft synthetic matrix (2). The two structures (inner cardiac tissue and synthetic matrix) (3) are bonded using a newly developed adhesive, TissueSil (4). The resulting piece is the biohybrid heart containing the preserved intracardiac structures and synthetic heart muscle (5). Image: Clara Park Bringing artificial intelligence and MIT to middle school classrooms MIT researchers and collaborators have developed an open-source curriculum to teach young students about ethics and artificial intelligence. Mon, 30 Dec 2019 10:45:01 -0500 Stefanie Koperniak | MIT Open Learning <p>In the age of Alexa, YouTube recommendations, and Spotify playlists, artificial intelligence has become a way of life, improving marketing and advertising, e-commerce, and more. But what are the ethical implications of technology that collects and learns personal information? How should society navigate these issues and shape the future?</p> <p>A new curriculum designed for middle school students aims to help them understand just that at an early age, as they grow up surrounded by the technology. The open-source educational material, designed by an MIT team and piloted at this year’s Massachusetts STEM Week this past fall, teaches students how AI systems are designed, how they can be used to influence the public — and also how to use them to be successful in jobs of the future.</p> <p>During Mass STEM Week in October, middle schools across the commonwealth replaced their regular curriculum with an immersive week of hands-on learning led by a team including Cynthia Breazeal, associate professor of media arts and sciences at MIT; Randi Williams ’18, graduate research assistant in the <a href="">Personal Robots Group</a> at the MIT Media Lab; and the nonprofit organization <a href="">i2 Learning</a>.</p> <p>“Preparing students for the future means having them engage in technology through hands-on activities. We provide students with tools and conceptual frameworks where we want them to engage with our materials as conscientious designers of AI-enabled technologies,” Breazeal says. “As they think through designing a solution to address a problem in their community, we get them to think critically about the ethical implications of the technology.”</p> <p>Three years ago, the Personal Robots Group began a program around teaching AI concepts to preschoolers. This effort then broadened into planning learning experiences for more children, and the group developed a curriculum geared toward middle school students. Last spring, an AI curriculum was shared with teachers and piloted in Somerville, Massachusetts, to determine which activities resonated the most in the classrooms.</p> <p>“We want to make a curriculum in which middle-schoolers can build and use AI — and, more importantly, we want them to take into account the societal impact of any technology,” says Williams.</p> <p>This curriculum, <a href="">How to Train Your Robot</a>, was first piloted at an i2 summer camp in Boston before being presented to teachers from local schools during Mass STEM Week. The teachers, many of whom had little familiarity with STEM subjects, also participated in two days of professional development training to prepare them to deliver more than 20 class hours of AI content to their students. The curriculum ran in three schools across six classrooms.</p> <p>The AI curriculum incorporates the work of Blakeley Hoffman Payne, a graduate research assistant in the Personal Robots Group, whose research focuses on the ethics of artificial intelligence and how to teach children to design, use, and think about AI. Students participated in discussions and creative activities, designing robot companions and using machine learning to solve real-world problems they have observed. At the end of the week, students share their inventions with their communities.</p> <p>“AI is an area that is becoming increasingly important in people’s lives,” says Ethan Berman, founder of i2 Learning and MIT parent. “This curriculum is very relevant to both students and teachers. Beyond just being a class on technology, it focuses on what it means to be a global citizen.”</p> <p>The creative projects provided opportunities for students to consider problems from a variety of angles, including thinking about issues of bias ahead of time, before a system is designed. For example, for one project that focused on sign language, the student trained her algorithm for understanding sign language around students of a wide range of skin tones, and incorporated adults, too — considering potential algorithmic bias to inform the design of the system.</p> <p>Another group of students built a “library robot,” designed to help find and retrieve a book for people with mobility challenges. Students had to think critically about why and how this might be helpful, and also to consider the job of a librarian and how this would impact a librarian’s work. They considered how a robot that finds and retrieves books might be able to free up more of a librarian’s time to actually help people and find information for them.</p> <p>Some of the current opportunities include scaling for more classrooms and schools, and also incorporating some other disciplines. There is interest in incorporating social studies, math, science, art, and music by finding ways to weave these other subjects into the AI projects. The main focus is on experiential learning that impacts how students think about AI.</p> <p>“We hope students walk away with a different understanding of AI and how it works in the world,” says Williams, “and that they feel empowered to play an important role in shaping the technology.”</p> MIT researchers piloted a new curriculum to teach middle school students about ethics and artificial intelligence at the second annual Mass STEM Week, a statewide event to encourage more young people to explore science, technology, engineering, and math studies and careers. Photo courtesy of i2 Learning. Boston-area girls discover a passion for coding Weekend robotics workshops help middle and high school girls dispel “computing phobia.” Fri, 13 Dec 2019 12:20:01 -0500 Dorothy Ryan | Lincoln Laboratory <p>“My goal is to make computing 'normal' for girls,” says Sabina Chen from Lincoln Laboratory's Advanced Capabilities and Systems Group, who led a workshop that taught middle school girls how to program a robotic car to autonomously follow colored cones. The girls attended this enrichment class for eight consecutive Saturdays from September to November. “The class is about exposure [to computing] and interest-building,” she explains.</p> <p>While Chen was introducing the 20 middle school girls to coding and computer visualization software at the Beaver Works facility in MIT's Building 31, Eyassu Shimelis of the laboratory's Advanced Concepts and Technologies Group was conducting a similar series of classes for 21 high school girls in MIT's Building 33.</p> <p>The motivation behind holding the girls-only workshops is to foster a curiosity and familiarity in computing that may lead to a future increase in the number of women engaged in computer science. According to, in 2018 only 18 percent of bachelor's degrees in computer science were awarded to women; in electrical engineering, a major that often leads to professions involving computing, the percentage is even lower, at 13.7. The Bureau of Labor Statistics reports that women make up only about 21 percent of computer programmers, 19 percent of software developers, and 32 percent of website developers.</p> <p>While multiple theories exist as to why women are underrepresented in computer-related majors and jobs, one consensus is that young women do not have confidence in their ability to master computers. “The girls came in thinking they can't do it,” Chen says, adding that she finds the course worthwhile when “their eyes sort of sparkle when they realize they can do it.”</p> <p>Both workshops are based on a rigorous four-week course offered to high school seniors through the <a href="">Beaver Works Summer Institute</a> (BWSI). The summer course lets students explore the various technologies that can be used to enable an MIT-designed RACECAR (Rapid Autonomous Complex Environment Competing Ackermann steeRing) robotic vehicle to compete in fast, autonomous navigation around a mini “Grand Prix” racetrack. Although the Saturday sessions could not delve as deeply as the summer course into the software and various systems needed for the cars to tackle the obstacle-ridden Grand Prix, these weekend “crash courses” did cover the coding and computer-vison technology that allowed the girls to race their cars around a circular course set up in Building 31's high-bay space.</p> <p>Chen developed the curriculum for the middle school program that was offered to both boys and girls this past summer in conjunction with the BWSI. “It is designed so students learn a specific concept, apply it, and see an immediate result,” she says. The gratification of witnessing a hands-on application of a lesson is what keeps the students interested. Her curriculum will soon be online so that schools, robotics clubs, or even interested individuals can adapt it for themselves.&nbsp;</p> <p>Shimelis taught a similar version of a <a href="">RACECAR preliminary course</a> for Boston-area high schoolers. That course was developed in 2018 by Andrew Fishberg, who passed along his program when he moved on to tackle graduate studies at MIT. Shimelis is tweaking the course to address feedback from BWSI RACECAR students and teaching assistants, and to adapt it to his teaching style.</p> <p>Both Chen and Shimelis say they did not modify their courses for the girls-only sessions that were new this fall. They agree that the girls were eager to learn and capable of handling the classwork. “Many of the girls were faster at grasping the concepts than students in my summer course,” Shimelis notes. This is high praise, because to be accepted for the BWSI program, students must complete a prerequisite RACECAR online tutorial and submit teacher recommendations and stellar school transcripts.</p> <p>Chen says she was pleased by the change she saw in the girls from the beginning to the end of her workshop. “At the end, they were a lot more sure of themselves and more willing to explore their own ideas without fear.”</p> <p>According to Chen and Shimelis, the success of the two workshops can, in large part, be attributed to the dedicated help of a number of people. Sertac Karaman, an MIT engineering professor who developed the original RACECAR course for undergraduate students, provided guidance to both the instructors and students. A cadre of volunteers served as teaching assistants: Kourosh Arasteh, Olivia Brown, Juliana Furgala, Saivamsi Hanumanthu, Elisa Kircheim, Tzofi Klinghoffer, Marko Kocic, Aryk Ledet, Harrison Packer, Joyce Tam, and Jing Wang from Lincoln Laboratory's staff; and Andrew Schoer, a Boston University grad student who is a participant in the Laboratory's Lincoln Scholars tuition assistance program.</p> <p>The success of the workshops is captured in one student's answer to a course-evaluation question about what she gained: “I see myself coding in the future!”</p> Middle school and high school girls meet in MIT's Building 31 to showcase the capabilities of the autonomous cars their teams programmed to navigate a circuitous course. Photo: Joel Grimm Intelligent Towing Tank propels human-robot-computer research A novel experimental facility integrates automation and active learning, illuminating a path to accelerated scientific discovery. Mon, 09 Dec 2019 16:05:01 -0500 MIT Sea Grant <p>In its first year of operation, the Intelligent Towing Tank (ITT) conducted about 100,000 total experiments, essentially completing the equivalent of a PhD student’s five years’ worth of experiments in a matter of weeks.</p> <p>The automated experimental facility, developed in the MIT Sea Grant Hydrodynamics Laboratory, automatically and adaptively performs, analyzes, and designs experiments exploring vortex-induced vibrations (VIVs). Important for engineering offshore ocean structures like marine drilling risers that connect underwater oil wells to the surface, VIVs remain somewhat of a phenomenon to researchers due to the high number of parameters involved.</p> <p>Guided by active learning, the ITT conducts series of experiments wherein the parameters of each next experiment are selected by a computer. Using an “explore-and-exploit” methodology, the system dramatically reduces the number of experiments required to explore and map the complex forces governing VIVs.</p> <p>What began as then-PhD candidate Dixia Fan’s quest to cut back on conducting a thousand or so laborious experiments — by hand — led to the design of the innovative system and a <a href="">paper</a> recently published in the journal <em>Science Robotics</em>.</p> <p>Fan, now a postdoc, and a team of researchers from the MIT Sea Grant College Program and MIT’s Department of Mechanical Engineering, École Normale Supérieure de Rennes, and Brown University, reveal a potential paradigm shift in experimental research, where humans, computers, and robots can collaborate more effectively to accelerate scientific discovery.</p> <p>The 33-foot whale of a tank comes alive, working without interruption or supervision on the venture at hand — in this case, exploring a canonical problem in the field of fluid-structure interactions. But the researchers envision applications of the active learning and automation approach to experimental research across disciplines, potentially leading to new insights and models in multi-input/multi-output nonlinear systems.</p> <p>VIVs are inherently-nonlinear motions induced on a structure&nbsp;in an oncoming irregular cross-stream, which prove vexing to study. The researchers report that the number of experiments completed by the ITT is already comparable to the total number of experiments done to date worldwide on the subject of VIVs.</p> <p>The reason for this is the large number of independent parameters, from flow velocity to pressure, involved in studying the complex forces at play. According to Fan, a systematic brute-force approach — blindly conducting 10 measurements per parameter in an eight-dimensional parametric space — would require 100 million experiments.</p> <p>With the ITT, Fan and his collaborators have taken the problem into a wider parametric space than previously practicable to explore. “If we performed traditional techniques on the problem we studied,” he explains, “it would take 950 years to finish the experiment.” Clearly infeasible, so Fan and the team integrated a Gaussian process regression learning algorithm into the ITT. In doing so, the researchers reduced the experimental burden by several orders of magnitude, requiring only a few thousand experiments.</p> <p>The robotic system automatically conducts an initial sequence of experiments, periodically towing a submerged structure along the length of the tank at a constant velocity. Then, the ITT takes partial control over the parameters of each next experiment by minimizing suitable acquisition functions of quantified uncertainties and adapting to achieve a range of objectives, like reduced drag.</p> <p>Earlier this year, Fan was awarded an&nbsp;MIT Mechanical Engineering de Florez Award for "Outstanding Ingenuity and Creative Judgment"&nbsp;in the development of the ITT. “Dixia’s design of the Intelligent Towing Tank is an outstanding example of using novel methods to reinvigorate mature fields,” says Michael Triantafyllou, Henry L. and Grace Doherty Professor in Ocean Science and Engineering, who acted as Fan’s doctoral advisor.</p> <p>Triantafyllou, a co-author on this paper and the director of the MIT Sea Grant College Program, says, “MIT Sea Grant has committed resources and funded projects using deep-learning methods in ocean-related problems for several years that are already paying off.” Funded by the National Oceanic and Atmospheric Administration and administered by the National Sea Grant Program, MIT Sea Grant is a federal-Institute partnership that brings the research and engineering core of MIT&nbsp;to bear on ocean-related challenges.</p> <p>Fan’s research points to a number of others utilizing automation and artificial intelligence in science: At Caltech, a robot scientist named “Adam” generates and tests hypotheses; at the Defense Advanced Research Projects Agency, the Big Mechanism program reads tens of thousands of research papers to generate new models.</p> <p>Similarly, the ITT applies human-computer-robot collaboration to accelerate experimental efforts. The system demonstrates a potential paradigm shift in conducting research, where automation and uncertainty quantification can considerably accelerate scientific discovery. The researchers assert that the machine learning methodology described in this paper can be adapted and applied in and beyond fluid mechanics, to other experimental fields.</p> <p>Other contributors to the paper include George Karniadakis from Brown University, who is also affiliated with MIT Sea Grant; Gurvan Jodin from ENS Rennes; MIT PhD candidate in mechanical engineering Yu Ma; and Thomas Consi, Luca Bonfiglio, and Lily Keyes from MIT Sea Grant.</p> <p>This work was supported by DARPA, Fariba Fahroo, and Jan Vandenbrande through an EQUiPS (Enabling Quantification of Uncertainty in Physical Systems) grant, as well as Shell, Subsea 7, and the MIT Sea Grant College Program.</p> Dixia Fan stands in front of the ITT holding some of the structures towed in the tank to study VIVs, which are important for engineering offshore ocean structures like marine drilling risers that connect underwater oil wells to the surface. Photo: Lily Keyes/MIT Sea Grant Paul McEuen delivers inaugural Dresselhaus Lecture on cell-sized robots Cornell University professor and physicist uses nanoscale parts to create smart, active microbots. Wed, 04 Dec 2019 15:30:01 -0500 Amanda Stoll | MIT.nano <p>Functional, intelligent robots the size of a single cell are within reach, said Cornell University Professor Paul McEuen at the inaugural Mildred S. Dresselhaus Lecture at MIT on Nov. 13.</p> <p>“To build a robot that is on the scale of 100 microns in size, and have it work, that’s a big dream,” said McEuen, the John A. Newman Professor of Physical Science at Cornell University and director of Kavli Institute at Cornell for Nanoscale Science. “One hundred microns is a very special size. It is the border between the visible and the invisible, or microscopic, world.”</p> <p>In a talk entitled “Cell-sized Sensors and Robots” in front of a large audience in MIT’s 10-250 lecture hall, McEuen introduced his concept for a new generation of machines that work at the microscale by combining microelectronics, solar cells, and light.&nbsp;The microbots, as he calls them, operate&nbsp;using optical wireless integrated circuits and&nbsp;surface electrochemical actuators.</p> <p><strong>Kicking off the Dresselhaus Lectures</strong></p> <p>Inaugurated this year to honor MIT professor and physicist Mildred "Millie" Dresselhaus, the Dresselhaus Lecture recognizes a significant figure in science and engineering whose&nbsp;leadership and impact echo the late Institute Professor's life, accomplishments, and values. The lecture will be presented annually in November, the month of her birth.</p> <p>Dresselhaus spent over 50 years at MIT, where she was a professor in the Department of Electrical Engineering and Computer Science (originally the Department of Electrical Engineering) as well as in the Department of Physics. She was MIT’s first female Institute Professor, co-organizer of the first MIT Women’s Forum, the first solo recipient of a Kavli Prize, and the first woman to win the National Medal of Science in the engineering category.</p> <p>Her research into the fundamental properties of carbon earned her the nickname the “Queen of Carbon Science.” She was also nationally known for her work to develop wider opportunities for women in science and engineering.</p> <p>“Millie was a physicist, a materials scientist, and an electrical engineer; an MIT professor, researcher, and doctoral supervisor; a prolific author; and a longtime leader in the scientific community,” said&nbsp;Asu Ozdaglar, current EECS department head, in her opening remarks. “Even in her final years, she was active in her field at MIT and in the department, attending EECS faculty meetings and playing an important role in developing the MIT.nano facility.”</p> <p><strong>Pushing the boundaries of physics</strong></p> <p>McEuen,&nbsp;who first met Dresselhaus when he attended graduate school at Yale University with her son, expressed what a privilege it was to celebrate Millie as the inaugural speaker. “When I think of my scientific heroes, it’s a very, very short list. And I think at the top of it would be Millie Dresselhaus.&nbsp;To be able to give this lecture in her honor means the world to me.”</p> <p>After earning his bachelor’s degree in engineering physics from the University of Oklahoma, McEuen continued his research at Yale University, where he completed his PhD in 1990 in applied physics. McEuen spent two years at MIT as a postdoc studying condensed matter physics, and then became a principal investigator at the Lawrence Berkeley National Laboratory. He spent eight years teaching at the University of California at Berkeley before joining the faculty at Cornell as a professor in the physics department in 2001.</p> <p>“Paul is a pioneer for our generation, exploring the domain of atoms and molecules to push the frontier even further. It is no exaggeration to say that his discoveries and innovations will help define the Nano Age,” said Vladimir Bulović, the founding faculty director of MIT.nano and the&nbsp;Fariborz Maseeh (1990) Professor in Emerging Technology.</p> <p><strong>“</strong><strong>The world is our oyster”</strong></p> <p>McEuen joked at the beginning of his talk that speaking of technology measured in microns sounds “so 1950s” in today’s world, in which researchers can manipulate at the scale of nanometers. One micron — an abbreviation for micrometer — is one millionth of a meter; a nanometer is one billionth of a meter.</p> <p>“[But] if you want a micro robot, you need nanoscale parts. Just as the birth of the transistor gave rise to all the computational systems we have now,” he said, “the birth of simple, nanoscale mechanical and electronic elements is going to give birth to a robotics technology at the microscopic scale of less than 100 microns.”</p> <p>The motto of McEuen and his research group at Cornell is “anything, as long as it’s small.” This focus includes fundamentals of nanostructures, atomically-thin origami for metamaterials and micromachines, and microscale smart phones and optobots. McEuen emphasized the importance of borrowing from other fields, such as microelectronics technology, to build something new. Cornell researchers have used this technology to build an optical wireless integrated circuit (OWIC) — essentially a microscopic cellphone made of solar cells that power it and receive external information, a simple transistor circuit to serve as its brain, and a light-emitting diode to blink out data.</p> <p>Why make something so small? The first reason is cost; the second is its wide array of applications. Such tiny devices could measure voltage or temperature, making them useful for microfluidic experiments. In the future, they could be deployed&nbsp;as&nbsp;smart, secure tags for counterfeiting, invisible sensors for the internet of things, or used for neural interfacing to measure electrical activity in the brain.</p> <p>Adding a&nbsp;surface electrochemical actuator to these OWICs brings mechanical movement to McEuen’s microbots. By capping a very thin piece of platinum on one side and applying a voltage to the other, “we could make all kinds of cool things.”</p> <p>At the end of his talk, McEuen answered audience questions moderated by&nbsp;Bulović, such as how do the microbots communicate with one another and what is their functional lifespan. He closed&nbsp;with a final quote from Millie Dresselhaus: “Follow your interests, get the best available education and training, set your sights high, be persistent, be flexible, keep your options open, accept help when offered, and be prepared to help others.”</p> <p>Nominations for the 2020 Dresselhaus lecture can be submitted <a href="" target="_blank">on MIT.nano’s website</a>. Any significant figure in science and engineering from anywhere in the world may be considered.</p> Cornell University’s Paul McEuen gives the inaugural Mildred S. Dresselhaus Lecture on cell-sized sensors and robots. Photo: Justin Knight How to design and control robots with stretchy, flexible bodies Optimizing soft robots to perform specific tasks is a huge computational problem, but a new model can help. Fri, 22 Nov 2019 00:00:00 -0500 Rob Matheson | MIT News Office <p>MIT researchers have invented a way to efficiently optimize the control and design of soft robots for target tasks, which has traditionally been a monumental undertaking in computation.</p><p>Soft robots have springy, flexible, stretchy bodies that can essentially move an infinite number of ways at any given moment. Computationally, this represents a highly complex “state representation,” which describes how each part of the robot is moving. State representations for soft robots can have potentially millions of dimensions, making it difficult to calculate the optimal way to make a robot complete complex tasks.</p><p>At the Conference on Neural Information Processing Systems next month, the MIT researchers will present a model that learns a compact, or “low-dimensional,” yet detailed state representation, based on the underlying physics of the robot and its environment, among other factors. This helps the model iteratively co-optimize movement control and material design parameters catered to specific tasks.</p><p>“Soft robots are infinite-dimensional creatures that bend in a billion different ways at any given moment,” says first author Andrew Spielberg, a graduate student in the Computer Science and Artificial Intelligence Laboratory (CSAIL). “But, in truth, there are natural ways soft objects are likely to bend. We find the natural states of soft robots can be described very compactly in a low-dimensional description. We optimize control and design of soft robots by learning a good description of the likely states.”</p><p>In simulations, the model enabled 2D and 3D soft robots to complete tasks — such as moving certain distances or reaching a target spot —more quickly and accurately than current state-of-the-art methods. The researchers next plan to implement the model in real soft robots.</p><p>Joining Spielberg on the paper are CSAIL graduate students Allan Zhao, Tao Du, and Yuanming Hu; Daniela Rus, director of CSAIL and the Andrew and Erna Viterbi Professor of Electrical Engineering and Computer Science; and Wojciech Matusik, an MIT associate professor in electrical engineering and computer science and head of the Computational Fabrication Group.</p><p><strong>“Learning-in-the-loop”</strong></p><p>Soft robotics is a relatively new field of research, but it holds promise for advanced robotics. For instance, flexible bodies could offer safer interaction with humans, better object manipulation, and more maneuverability, among other benefits.</p><p>Control of robots in simulations relies on an “observer,” a program that computes variables that see how the soft robot is moving to complete a task. In previous work, the researchers decomposed the soft robot into hand-designed clusters of simulated particles. Particles contain important information that help narrow down the robot’s possible movements. If a robot attempts to bend a certain way, for instance, actuators may resist that movement enough that it can be ignored. But, for such complex robots, manually choosing which clusters to track during simulations can be tricky.</p><p>Building off that work, the researchers designed a “learning-in-the-loop optimization” method, where all optimized parameters are learned during a single feedback loop over many simulations. And, at the same time as learning optimization —&nbsp;or “in the loop” — the method also learns the state representation.</p><p>The model employs a technique called a material point method (MPM), which simulates the behavior of particles of continuum materials, such as foams and liquids, surrounded by a background grid. In doing so, it captures the particles of the robot and its observable environment into pixels or 3D pixels, known as voxels, without the need of any additional computation. &nbsp;&nbsp;&nbsp;&nbsp;</p><p>In a learning phase, this raw particle grid information is fed into a machine-learning component that learns to input an image, compress it to a low-dimensional representation, and decompress the representation back into the input image. If this “autoencoder” retains enough detail while compressing the input image, it can accurately recreate the input image from the compression.</p><p>In the researchers’ work, the autoencoder’s learned compressed representations serve as the robot’s low-dimensional state representation. In an optimization phase, that compressed representation loops back into the controller, which outputs a calculated actuation for how each particle of the robot should move in the next MPM-simulated step.</p><p>Simultaneously, the controller uses that information to adjust the optimal stiffness for each particle to achieve its desired movement. In the future, that material information can be useful for 3D-printing soft robots, where each particle spot may be printed with slightly different stiffness. “This allows for creating robot designs catered to the robot motions that will be relevant to specific tasks,” Spielberg says. “By learning these parameters together, you keep everything as synchronized as much as possible to make that design process easier.”</p><p><strong>Faster optimization</strong></p><p>All optimization information is, in turn, fed back into the start of the loop to train the autoencoder. Over many simulations, the controller learns the optimal movement and material design, while the autoencoder learns the increasingly more detailed state representation. “The key is we want that low-dimensional state to be very descriptive,” Spielberg says.</p><p>After the robot gets to its simulated final state over a set period of time —&nbsp;say, as close as possible to the target destination —&nbsp;it updates a “loss function.” That’s a critical component of machine learning, which tries to minimize some error. In this case, it minimizes, say, how far away the robot stopped from the target. That loss function flows back to the controller, which uses the error signal to tune all the optimized parameters to best complete the task.</p><p>If the researchers tried to directly feed all the raw particles of the simulation into the controller, without the compression step, “running and optimization time would explode,” Spielberg says. Using the compressed representation, the researchers were able to decrease the running time for each optimization iteration from several minutes down to about 10 seconds.</p><p>The researchers validated their model on simulations of various 2D and 3D biped and quadruped robots. They researchers also found that, while robots using traditional methods can take up to 30,000 simulations to optimize these parameters, robots trained on their model took only about 400 simulations.</p> <p>"Our goal is to enable quantum leaps in the way engineers go from specification to design, prototyping, and programming of soft robots. In this paper, we explore the potential of co-optimizing the body and control system of a soft robot can lead the rapid creation of soft bodied robots customized to the tasks they have to do," Rus says.</p><p>Deploying the model into real soft robots means tackling issues with real-world noise and uncertainty that may decrease the model’s efficiency and accuracy. But, in the future, the researchers hope to design a full pipeline, from simulation to fabrication, for soft robots.</p> An MIT-invented model efficiently and simultaneously optimizes control and design of soft robots for target tasks, which has traditionally been a monumental undertaking in computation. The model, for instance, was significantly faster and more accurate than state-of-the-art methods at simulating how quadrupedal robots (pictured) should move to reach target destinations. Image courtesy of the researchers Predicting people's driving personalities System from MIT CSAIL sizes up drivers as selfish or selfless. Could this help self-driving cars navigate in traffic?<br /> Mon, 18 Nov 2019 16:10:01 -0500 Adam Conner-Simons | Rachel Gordon | MIT CSAIL <p>Self-driving cars are coming. But for all their fancy sensors and intricate data-crunching abilities, even the most cutting-edge cars lack something that (almost) every 16-year-old with a learner’s permit has: social awareness.<br /> <br /> While autonomous technologies have improved substantially, they still ultimately view the drivers around them as obstacles made up of ones and zeros, rather than human beings with specific intentions, motivations, and personalities.<br /> <br /> But recently a team led by researchers at MIT’s <a href="">Computer Science and Artificial Intelligence Laboratory</a> (CSAIL) has been exploring whether self-driving cars can be programmed to classify the social personalities of other drivers, so that they can better predict what different cars will do — and, therefore, be able to drive more safely among them.<br /> <br /> In a new paper, the scientists integrated tools from social psychology to classify driving behavior with respect to how selfish or selfless a particular driver is.<br /> <br /> Specifically, they used something called social value orientation (SVO), which represents the degree to which someone is selfish (“egoistic”) versus altruistic or cooperative (“prosocial”). The system then estimates drivers’ SVOs to create real-time driving trajectories for self-driving cars.</p> <p>Testing their algorithm on the tasks of merging lanes and making unprotected left turns, the team showed that they could better predict the behavior of other cars by a factor of 25 percent. For example, in the left-turn simulations their car knew to wait when the approaching car had a more egoistic driver, and to then make the turn when the other car was more prosocial.</p> <p>While not yet robust enough to be implemented on real roads, the system could have some intriguing use cases, and not just for the cars that drive themselves. Say you’re a human driving along and a car suddenly enters your blind spot — the system could give you a warning in the rear-view mirror that the car has an aggressive driver, allowing you to adjust accordingly. It could also allow self-driving cars to actually learn to exhibit more human-like behavior that will be easier for human drivers to understand.<br /> <br /> “Working with and around humans means figuring out their intentions to better understand their behavior,” says graduate student Wilko Schwarting, who was lead author on the new paper that will be published this week in the latest issue of the <em>Proceedings of the National Academy of Sciences</em>. “People’s tendencies to be collaborative or competitive often spills over into how they behave as drivers. In this paper, we sought to understand if this was something we could actually quantify.”<br /> <br /> Schwarting’s co-authors include MIT professors Sertac Karaman and Daniela Rus, as well as research scientist Alyssa Pierson and former CSAIL postdoc Javier Alonso-Mora.<br /> <br /> A central issue with today’s self-driving cars is that they’re programmed to assume that all humans act the same way. This means that, among other things, they’re quite conservative in their decision-making at four-way stops and other intersections.<br /> <br /> While this caution reduces the chance of fatal accidents, it also <a href="">creates bottlenecks</a> that can be frustrating for other drivers, not to mention hard for them to understand. (This may be why the majority of traffic incidents have involved <a href="">getting rear-ended by impatient drivers</a>.)<br /> <br /> “Creating more human-like behavior in autonomous vehicles (AVs) is fundamental for the safety of passengers and surrounding vehicles, since behaving in a predictable manner enables humans to understand and appropriately respond to the AV’s actions,” says Schwarting.<br /> <br /> To try to expand the car’s social awareness, the CSAIL team combined methods from social psychology with game theory, a theoretical framework for conceiving social situations among competing players.<br /> <br /> The team modeled road scenarios where each driver tried to maximize their own utility and analyzed their “best responses” given the decisions of all other agents. Based on that small snippet of motion from other cars, the team’s algorithm could then predict the surrounding cars’ behavior as cooperative, altruistic, or egoistic — grouping the first two as “prosocial.” People’s scores for these qualities rest on a continuum with respect to how much a person demonstrates care for themselves versus care for others.<br /> <br /> In the merging and left-turn scenarios, the two outcome options were to either let somebody merge into your lane (“prosocial”) or not (“egoistic”). The team’s results showed that, not surprisingly, merging cars are deemed more competitive than non-merging cars.<br /> <br /> The system was trained to try to better understand when it’s appropriate to exhibit different behaviors. For example, even the most deferential of human drivers knows that certain types of actions — like making a lane change in heavy traffic — require a moment of being more assertive and decisive.<br /> <br /> For the next phase of the research, the team plans to work to apply their model to pedestrians, bicycles, and other agents in driving environments. In addition, they will be investigating other robotic systems acting among humans, such as household robots, and integrating SVO into their prediction and decision-making algorithms. Pierson says that the ability to estimate SVO distributions directly from observed motion, instead of in laboratory conditions, will be important for fields far beyond autonomous driving.<br /> <br /> “By modeling driving personalities and incorporating the models mathematically using the SVO in the decision-making module of a robot car, this work opens the door to safer and more seamless road-sharing between human-driven and robot-driven cars,” says Rus.<br /> <br /> The research was supported by the Toyota Research Institute for the MIT team. The Netherlands Organization for Scientific Research provided support for the specific participation of Mora.</p> <p></p> In lane-merging scenarios, a system developed at MIT could distinguish between altruistic and egoistic driving behavior. Image courtesy of the researchers. Flexible yet sturdy robot is designed to “grow” like a plant Its extendable appendage can meander through tight spaces and then lift heavy loads. Thu, 07 Nov 2019 10:28:28 -0500 Jennifer Chu | MIT News Office <p>In today’s factories and warehouses, it’s not uncommon to see robots whizzing about, shuttling items or tools from one station to another. For the most part, robots navigate pretty easily across open layouts. But they have a much harder time winding through narrow spaces to carry out tasks such as reaching for a product at the back of a cluttered shelf, or snaking around a car’s engine parts to unscrew an oil cap.</p><p>Now MIT engineers have developed a robot designed to extend a chain-like appendage flexible enough to twist and turn in any necessary configuration, yet rigid enough to support heavy loads or apply torque to assemble parts in tight spaces. When the task is complete, the robot can retract the appendage and extend it again, at a different length and shape, to suit the next task.</p><p>The appendage design is inspired by the way plants grow, which involves the transport of nutrients, in a fluidized form, up to the plant’s tip. There, they are converted into solid material to produce, bit by bit, a supportive stem.</p><p>Likewise, the robot consists of a “growing point,” or gearbox, that pulls a loose chain of interlocking blocks into the box. Gears in the box then lock the chain units together and feed the chain out, unit by unit, as a rigid appendage.</p><p>The researchers presented the plant-inspired “growing robot” this week at the IEEE International Conference on Intelligent Robots and Systems (IROS) in Macau. They envision that grippers, cameras, and other sensors could be mounted onto the robot’s gearbox, enabling it to meander through an aircraft’s propulsion system and tighten a loose screw, or to reach into a shelf and grab a product without disturbing the organization of surrounding inventory, among other tasks.</p><p>“Think about changing the oil in your car,” says Harry Asada, professor of mechanical engineering at MIT. “After you open the engine roof, you have to be flexible enough to make sharp turns, left and right, to get to the oil filter, and then you have to be strong enough to twist the oil filter cap to remove it.”</p><p>“Now we have a robot that can potentially accomplish such tasks,” says Tongxi Yan, a former graduate student in Asada’s lab, who led the work. “It can grow, retract, and grow again to a different shape, to adapt to its environment.”</p><p>The team also includes MIT graduate student Emily Kamienski and visiting scholar Seiichi Teshigawara, who presented the results at the conference.</p> <p></p> <p></p> <p><strong>The last foot</strong></p><p>The design of the new robot is an offshoot of Asada’s work in addressing the “last one-foot problem” — an engineering term referring to the last step, or foot, of a robot’s task or exploratory mission. While a robot may spend most of its time traversing open space, the last foot of its mission may involve more nimble navigation through tighter, more complex spaces to complete a task.</p><p>Engineers have devised various concepts and prototypes to address the last one-foot problem, including robots made from soft, balloon-like materials that grow like vines to squeeze through narrow crevices. But Asada says such soft extendable robots aren’t sturdy enough to support “end effectors,” or add-ons such as grippers, cameras, and other sensors that would be necessary in carrying out a task, once the robot has wormed its way to its destination.</p><p>“Our solution is not actually soft, but a clever use of rigid materials,” says Asada, who is the Ford Foundation Professor of Engineering.</p><p><strong>Chain links</strong></p><p>Once the team defined the general functional elements of plant growth, they looked to mimic this in a general sense, in an extendable robot.</p><p>“The realization of the robot is totally different from a real plant, but it exhibits the same kind of functionality, at a certain abstract level,” Asada says.</p><p>The researchers designed a gearbox to represent the robot’s “growing tip,” akin to the bud of a plant, where, as more nutrients flow up to the site, the tip feeds out more rigid stem. Within the box, they fit a system of gears and motors, which works to pull up a fluidized material — in this case, a bendy sequence of 3-D-printed plastic units interlocked with each other, similar to a bicycle chain.</p><p>As the chain is fed into the box, it turns around a winch, which feeds it through a second set of motors programmed to lock certain units in the chain to their neighboring units, creating a rigid appendage as it is fed out of the box.</p><p>The researchers can program the robot to lock certain units together while leaving others unlocked, to form specific shapes, or to “grow” in certain directions. In experiments, they were able to program the robot to turn around an obstacle as it extended or grew out from its base.</p><p>“It can be locked in different places to be curved in different ways, and have a wide range of motions,” Yan says.</p><p>When the chain is locked and rigid, it is strong enough to support a heavy, one-pound weight. If a gripper were attached to the robot’s growing tip, or gearbox, the researchers say the robot could potentially grow long enough to meander through a narrow space, then apply enough torque to loosen a bolt or unscrew a cap.</p><p>Auto maintenance is a good example of tasks the robot could assist with, according to Kamienski. “The space under the hood is relatively open, but it’s that last bit where you have to navigate around an engine block or something to get to the oil filter, that a fixed arm wouldn’t be able to navigate around. This robot could do something like that.”</p><p>This research was funded, in part, by NSK Ltd.</p> MIT engineers identified “functional elements” in plant growth that they realized in the design of a physical robot. In a plant (right), “fluidized material,” in the form of nutrients, flow up to the tip, where they convert into solid material, in the form of the plant’s stem. The design basis (left) for the new robot works similar, with fluidized material, in the form of a flexible chain (in green), feeding into a tip, or gearbox (in grey) that locks and feeds the chain out as a rigid appendage (red), in effect, “growing” the entire structure. Image courtesy of researchers, edited by MIT News Autonomous system improves environmental sampling at sea Robotic boats could more rapidly locate the most valuable sampling spots in uncharted waters. Mon, 04 Nov 2019 14:54:51 -0500 Rob Matheson | MIT News Office <p>An autonomous robotic system invented by researchers at MIT and the Woods Hole Oceanographic Institution (WHOI) efficiently sniffs out the most scientifically interesting — but hard-to-find —&nbsp;sampling spots in vast, unexplored waters.</p><p>Environmental scientists are often interested in gathering samples at the most interesting locations, or “maxima,” in an environment. One example could be a source of leaking chemicals, where the concentration is the highest and mostly unspoiled by external factors. But a maximum can be any quantifiable value that researchers want to measure, such as water depth or parts of coral reef most exposed to air.</p><p>Efforts to deploy maximum-seeking robots suffer from efficiency and accuracy issues. Commonly, robots will move back and forth like lawnmowers to cover an area, which is time-consuming and collects many uninteresting samples. Some robots sense and follow high-concentration trails to their leak source. But they can be misled. For example, chemicals can get trapped and accumulate in crevices far from a source. Robots may identify those high-concentration spots as the source yet be nowhere close.</p><p>In a paper being presented at the International Conference on Intelligent Robots and Systems (IROS), the researchers describe “PLUMES,” a system that enables autonomous mobile robots to zero in on a maximum far faster and more efficiently. PLUMES leverages probabilistic techniques to predict which paths are likely to lead to the maximum, while navigating obstacles, shifting currents, and other variables. As it collects samples, it weighs what it’s learned to determine whether to continue down a promising path or search the unknown — which may harbor more valuable samples.</p><p>Importantly, PLUMES reaches its destination without ever getting trapped in those tricky high-concentration spots. “That’s important, because it’s easy to think you’ve found gold, but really you’ve found fool’s gold,” says co-first author Victoria Preston, a PhD student in the Computer Science and Artificial Intelligence Laboratory (CSAIL) and in the MIT-WHOI Joint Program.</p><p>The researchers built a PLUMES-powered robotic boat that successfully detected the most exposed coral head in the Bellairs Fringing Reef in Barbados —&nbsp;meaning, it was located in the shallowest spot —&nbsp;which is useful for studying how sun exposure impacts coral organisms. In 100 simulated trials in diverse underwater environments, a virtual PLUMES robot also consistently collected seven to eight times more samples of maxima than traditional coverage methods in allotted time frames.</p><p>“PLUMES does the minimal amount of exploration necessary to find the maximum and then concentrates quickly on collecting valuable samples there,” says co-first author Genevieve Flaspohler, a PhD student and in CSAIL and the MIT-WHOI Joint Program.</p><p>Joining Preston and Flaspohler on the paper are: Anna P.M. Michel and Yogesh Girdhar, both scientists in the Department of Applied Ocean Physics and Engineering at the WHOI; and Nicholas Roy, a professor in CSAIL and in the Department of Aeronautics and Astronautics. &nbsp;</p><p><strong>Navigating an exploit-explore tradeoff</strong></p><p>A key insight of PLUMES was using techniques from probability to reason about navigating the notoriously complex tradeoff between exploiting what’s learned about the environment and exploring unknown areas that may be more valuable.</p><p>“The major challenge in maximum-seeking is allowing the robot to balance exploiting information from places it already knows to have high concentrations and exploring places it doesn’t know much about,” Flaspohler says. “If the robot explores too much, it won’t collect enough valuable samples at the maximum. If it doesn’t explore enough, it may miss the maximum entirely.”</p><p>Dropped into a new environment, a PLUMES-powered robot uses a probabilistic statistical model called a Gaussian process to make predictions about environmental variables, such as chemical concentrations, and estimate sensing uncertainties. PLUMES then generates a distribution of possible paths the robot can take, and uses the estimated values and uncertainties to rank each path by how well it allows the robot to explore and exploit.</p><p>At first, PLUMES will choose paths that randomly explore the environment. Each sample, however, provides new information about the targeted values in the surrounding environment — such as spots with highest concentrations of chemicals or shallowest depths. The Gaussian process model exploits that data to narrow down possible paths the robot can follow from its given position to sample from locations with even higher value. PLUMES uses a novel objective function —&nbsp;commonly used in machine-learning to maximize a reward — to make the call of whether the robot should exploit past knowledge or explore the new area.</p><p><strong>“Hallucinating” paths</strong></p><p>The decision where to collect the next sample relies on the system’s ability to “hallucinate” all possible future action from its current location. To do so, it leverages a modified version of Monte Carlo Tree Search (MCTS), a path-planning technique popularized for powering artificial-intelligence systems that master complex games, such as Go and Chess.</p><p>MCTS uses a decision tree — a map of connected nodes and lines — to simulate a path, or sequence of moves, needed to reach a final winning action. But in games, the space for possible paths is finite. In unknown environments, with real-time changing dynamics, the space is effectively infinite, making planning extremely difficult. The researchers designed “continuous-observation MCTS,” which leverages the Gaussian process and the novel objective function to search over this unwieldy space of possible real paths.</p><p>The root of this MCTS decision tree starts with a “belief” node, which is the next immediate step the robot can take. This node contains the entire history of the robot’s actions and observations up until that point. Then, the system expands the tree from the root into new lines and nodes, looking over several steps of future actions that lead to explored and unexplored areas.</p><p>Then, the system simulates what would happen if it took a sample from each of those newly generated nodes, based on some patterns it has learned from previous observations. Depending on the value of the final simulated node, the entire path receives a reward score, with higher values equaling more promising actions. Reward scores from all paths are rolled back to the root node. The robot selects the highest-scoring path, takes a step, and collects a real sample. Then, it uses the real data to update its Gaussian process model and repeats the “hallucination” process.</p><p>“As long as the system continues to hallucinate that there may be a higher value in unseen parts of the world, it must keep exploring,” Flaspohler says. “When it finally converges on a spot it estimates to be the maximum, because it can’t hallucinate a higher value along the path, it then stops exploring.”</p><p>Now, the researchers are collaborating with scientists at WHOI to use PLUMES-powered robots to localize chemical plumes at volcanic sites and study methane releases in melting coastal estuaries in the Arctic. Scientists are interested in the source of chemical gases released into the atmosphere, but these test sites can span hundreds of square miles.</p><p>“They can [use PLUMES to] spend less time exploring that huge area and really concentrate on collecting scientifically valuable samples,” Preston says.</p> Even in unexplored waters, an MIT-developed robotic system can efficiently sniff out valuable, hard-to-find spots to collect samples from. When implemented in autonomous boats deployed off the coast of Barbados (pictured), the system quickly found the most exposed coral head —meaning it was located in the shallowest spot — which is useful for studying how sun exposure impacts coral organisms. Image courtesy of the researchers Better autonomous “reasoning” at tricky intersections Model alerts driverless cars when it’s safest to merge into traffic at intersections with obstructed views. Mon, 04 Nov 2019 12:44:49 -0500 Rob Matheson | MIT News Office <p>MIT and Toyota researchers have designed a new model to help autonomous vehicles determine when it’s safe to merge into traffic at intersections with obstructed views.</p><p>Navigating intersections can be dangerous for driverless cars and humans alike. In 2016, roughly 23 percent of fatal and 32 percent of nonfatal U.S. traffic accidents occurred at intersections, according to a 2018 Department of Transportation study. Automated systems that help driverless cars and human drivers steer through intersections can require direct visibility of the objects they must avoid. When their line of sight is blocked by nearby buildings or other obstructions, these systems can fail.</p><p>The researchers developed a model that instead uses its own uncertainty to estimate the risk of potential collisions or other traffic disruptions at such intersections. It weighs several critical factors, including all nearby visual obstructions, sensor noise and errors, the speed of other cars, and even the attentiveness of other drivers. Based on the measured risk, the system may advise the car to stop, pull into traffic, or nudge forward to gather more data.</p><p>“When you approach an intersection there is potential danger for collision. Cameras and other sensors require line of sight. If there are occlusions, they don’t have enough visibility to assess whether it’s likely that something is coming,” says Daniela Rus, director of the Computer Science and Artificial Intelligence Laboratory (CSAIL) and the Andrew and Erna Viterbi Professor of Electrical Engineering and Computer Science. “In this work, we use a predictive-control model that’s more robust to uncertainty, to help vehicles safely navigate these challenging road situations.”</p><p>The researchers tested the system in more than 100 trials of remote-controlled cars turning left at a busy, obstructed intersection in a mock city, with other cars constantly driving through the cross street. Experiments involved fully autonomous cars and cars driven by humans but assisted by the system. In all cases, the system successfully helped the cars avoid collision from 70 to 100 percent of the time, depending on various factors. Other similar models implemented in the same remote-control cars sometimes couldn’t complete a single trial run without a collision.</p><p>Joining Rus on the paper are: first author Stephen G. McGill, Guy Rosman, and Luke Fletcher of the Toyota Research Institute (TRI); graduate students Teddy Ort and Brandon Araki, researcher Alyssa Pierson, and postdoc Igor Gilitschenski, all of CSAIL; Sertac Karaman, an MIT associate professor of aeronautics and astronautics; and John J. Leonard, the Samuel C. Collins Professor of Mechanical and Ocean Engineering of MIT and a TRI technical advisor.</p><p><strong>Modeling road segments</strong></p><p>The model is specifically designed for road junctions in which there is no stoplight and a car must yield before maneuvering into traffic at the cross street, such as taking a left turn through multiple lanes or roundabouts. In their work, the researchers split a road into small segments. This helps the model determine if any given segment is occupied to estimate a conditional risk of collision.</p><p>Autonomous cars are equipped with sensors that measure the speed of other cars on the road. When a sensor clocks a passing car traveling into a visible segment, the model uses that speed to predict the car’s progression through all other segments. A probabilistic “Bayesian network” also considers uncertainties — such as noisy sensors or unpredictable speed changes — to determine the likelihood that each segment is occupied by a passing car.</p><p>Because of nearby occlusions, however, this single measurement may not suffice. Basically, if a sensor can’t ever see a designated road segment, then the model assigns it a high likelihood of being occluded. From where the car is positioned, there’s increased risk of collision if the car just pulls out fast into traffic. This encourages the car to nudge forward to get a better view of all occluded segments. As the car does so, the model lowers its uncertainty and, in turn, risk.</p><p>But even if the model does everything correctly, there’s still human error, so the model also estimates the awareness of other drivers. “These days, drivers may be texting or otherwise distracted, so the amount of time it takes to react may be a lot longer,” McGill says. “We model that conditional risk, as well.”</p><p>That depends on computing the probability that a driver saw or didn’t see the autonomous car pulling into the intersection. To do so, the model looks at the number of segments a traveling car has passed through before the intersection. The more segments it had occupied before reaching the intersection, the higher the likelihood it has spotted the autonomous car and the lower the risk of collision.</p><p>The model sums all risk estimates from traffic speed, occlusions, noisy sensors, and driver awareness. It also considers how long it will take the autonomous car to steer a preplanned path through the intersection, as well as all safe stopping spots for crossing traffic. This produces a total risk estimate.</p><p>That risk estimate gets updated continuously for wherever the car is located at the intersection. In the presence of multiple occlusions, for instance, it’ll nudge forward, little by little, to reduce uncertainty. When the risk estimate is low enough, the model tells the car to drive through the intersection without stopping. Lingering in the middle of the intersection for too long, the researchers found, also increases risk of a collision.</p><p><strong>Assistance and intervention</strong></p><p>Running the model on remote-control cars in real-time indicates that it’s efficient and fast enough to deploy into full-scale autonomous test cars in the near future, the researchers say. (Many other models are too computationally heavy to run on those cars.) The model still needs far more rigorous testing before being used for real-world implementation in production vehicles.</p><p>The model would serve as a supplemental risk metric that an autonomous vehicle system can use to better reason about driving through intersections safely. The model could also potentially be implemented in certain “advanced driver-assistive systems” (ADAS), where humans maintain shared control of the vehicle.</p><p>Next, the researchers aim to include other challenging risk factors in the model, such as the presence of pedestrians in and around the road junction.</p> MIT and Toyota researchers have designed a new model that weighs various uncertainties and risks to help autonomous vehicles determine when it’s safe to merge into traffic at intersections with objects obstructing views, such as buildings blocking the line of sight. Image courtesy of the researchers Technique helps robots find the front door Navigation method may speed up autonomous last-mile delivery. Mon, 04 Nov 2019 00:00:00 -0500 Jennifer Chu | MIT News Office <p>In the not too distant future, robots may be dispatched as last-mile delivery vehicles to drop your takeout order, package, or meal-kit subscription at your doorstep — if they can find the door.</p><p>Standard approaches for robotic navigation involve mapping an area ahead of time, then using algorithms to guide a robot toward a specific goal or GPS coordinate on the map. While this approach might make sense for exploring specific environments, such as the layout of a particular building or planned obstacle course, it can become unwieldy in the context of last-mile delivery.</p><p>Imagine, for instance, having to map in advance every single neighborhood within a robot’s delivery zone, including the configuration of each house within that neighborhood along with the specific coordinates of each house’s front door. Such a task can be difficult to scale to an entire city, particularly as the exteriors of houses often change with the seasons. Mapping every single house could also run into issues of security and privacy.</p><p>Now MIT engineers have developed a navigation method that doesn’t require mapping an area in advance. Instead, their approach enables a robot to use clues in its environment to plan out a route to its destination, which can be described in general semantic terms, such as “front door” or “garage,” rather than as coordinates on a map. For example, if a robot is instructed to deliver a package to someone's front door, it might start on the road and see a driveway, which it has been trained to recognize as likely to lead toward a sidewalk, which in turn is likely to lead to the front door.</p> <p></p> <p></p> <p>The new technique can greatly reduce the time a robot spends exploring a property before identifying its target, and it doesn’t rely on maps of specific residences.&nbsp;</p><p>“We wouldn’t want to have to make a map of every building that we’d need to visit,” says Michael Everett, a graduate student in MIT’s Department of Mechanical Engineering. “With this technique, we hope to drop a robot at the end of any driveway and have it find a door.”</p><p>Everett will present the group’s results this week at the International Conference on Intelligent Robots and Systems. The paper, which is co-authored by Jonathan How, professor of aeronautics and astronautics at MIT, and Justin Miller of the Ford Motor Company, is a finalist for “Best Paper for Cognitive Robots.”</p><p><strong>“A sense of what things are”</strong></p><p>In recent years, researchers have worked on introducing natural, semantic language to robotic systems, training robots to recognize objects by their semantic labels, so they can visually process a door as a door, for example, and not simply as a solid, rectangular obstacle.</p><p>“Now we have an ability to give robots a sense of what things are, in real-time,” Everett says.</p><p>Everett, How, and Miller are using similar semantic techniques as a springboard for their new navigation approach, which leverages pre-existing algorithms that extract features from visual data to generate a new map of the same scene, represented as semantic clues, or context.</p><p>In their case, the researchers used an algorithm to build up a map of the environment as the robot moved around, using the semantic labels of each object and a depth image. This algorithm is called semantic SLAM (Simultaneous Localization and Mapping).</p><p>While other semantic algorithms have enabled robots to recognize and map objects in their environment for what they are, they haven’t allowed a robot to make decisions in the moment while navigating a new environment, on the most efficient path to take to a semantic destination such as a “front door.”</p><p>“Before, exploring was just, plop a robot down and say ‘go,’ and it will move around and eventually get there, but it will be slow,” How says.</p><p><strong>The cost to go</strong></p><p>The researchers looked to speed up a robot’s path-planning through a semantic, context-colored world. They developed a new “cost-to-go estimator,” an algorithm that converts a semantic map created by preexisting SLAM algorithms into a second map, representing the likelihood of any given location being close to the goal.</p><p>“This was inspired by image-to-image translation, where you take a picture of a cat and make it look like a dog,” Everett says. “The same type of idea happens here where you take one image that looks like a map of the world, and turn it into this other image that looks like the map of the world but now is colored based on how close different points of the map are to the end goal.”</p><p>This cost-to-go map is colorized, in gray-scale, to represent darker regions as locations far from a goal, and lighter regions as areas that are close to the goal. For instance, the sidewalk, coded in yellow in a semantic map, might be translated by the cost-to-go algorithm as a darker region in the new map, compared with a driveway, which is progressively lighter as it approaches the front door — the lightest region in the new map.</p><p>The researchers trained this new algorithm on satellite images from Bing Maps containing 77 houses from one urban and three suburban neighborhoods. The system converted a semantic map into a cost-to-go map, and mapped out the most efficient path, following lighter regions in the map, to the end goal. For each satellite image, Everett assigned semantic labels and colors to context features in a typical front yard, such as grey for a front door, blue for a driveway, and green for a hedge.</p><p>During this training process, the team also applied masks to each image to mimic the partial view that a robot’s camera would likely have as it traverses a yard.</p><p>“Part of the trick to our approach was [giving the system] lots of partial images,” How explains. “So it really had to figure out how all this stuff was interrelated. That’s part of what makes this work robustly.”</p><p>The researchers then tested their approach in a simulation of an image of an entirely new house, outside of the training dataset, first using the preexisting SLAM algorithm to generate a semantic map, then applying their new cost-to-go estimator to generate a second map, and path to a goal, in this case, the front door.</p><p>The group’s new cost-to-go technique found the front door 189 percent faster than classical navigation algorithms, which do not take context or semantics into account, and instead spend excessive steps exploring areas that are unlikely to be near their goal.</p><p>Everett says the results illustrate how robots can use context to efficiently locate a goal, even in unfamiliar, unmapped environments.</p><p>“Even if a robot is delivering a package to an environment it’s never been to, there might be clues that will be the same as other places it’s seen,” Everett says. “So the world may be laid out a little differently, but there’s probably some things in common.”</p><p>This research is supported, in part, by the Ford Motor Company.</p> For last-mile delivery, robots of the future may use a new MIT algorithm to find the front door, using clues in their environment. Image: MIT News Two-legged robot mimics human balance while running and jumping New control system may enable humanoid robots to do heavy lifting and other physically demanding tasks. Wed, 30 Oct 2019 14:00:00 -0400 Jennifer Chu | MIT News Office <p>Rescuing victims from a burning building, a chemical spill, or any disaster that is inaccessible to human responders could one day be a mission for resilient, adaptable robots. Imagine, for instance, rescue-bots that can bound through rubble on all fours, then rise up on two legs to push aside a heavy obstacle or break through a locked door.</p><p>Engineers are making strides on the design of four-legged robots and their ability to <a href="">run</a>, <a href="">jump</a> and even do <a href="">backflips</a>. But getting two-legged, humanoid robots to exert force or push against something without falling has been a significant stumbling block.</p><p>Now engineers at MIT and the University of Illinois at Urbana-Champaign have developed a method to control balance in a two-legged, teleoperated robot — an essential step toward enabling a humanoid to carry out high-impact tasks in challenging environments.</p><p>The team’s robot, physically resembling a machined torso and two legs, is controlled remotely by a human operator wearing a vest that transmits information about the human’s motion and ground reaction forces to the robot.</p><p>Through the vest, the human operator can both direct the robot’s locomotion and feel the robot’s motions. If the robot is starting to tip over, the human feels a corresponding pull on the vest and can adjust in a way to rebalance both herself and, synchronously, the robot.</p><p>In experiments with the robot to test this new “balance feedback” approach, the researchers were able to remotely maintain the robot’s balance as it jumped and walked in place in sync with its human operator.</p><p>“It’s like running with a heavy backpack — you can feel how the dynamics of the backpack move around you, and you can compensate properly,” says Joao Ramos, who developed the approach as an MIT postdoc. “Now if you want to open a heavy door, the human can command the robot to throw its body at the door and push it open, without losing balance.”</p><p>Ramos, who is now an assistant professor at the University of Illinois at Urbana-Champaign, has detailed the approach in a study appearing today in <em>Science Robotics</em>. His co-author on the study is Sangbae Kim, associate professor of mechanical engineering at MIT.</p><p><strong>More than motion</strong></p><p>Previously, Kim and Ramos built the two-legged robot <a href="">HERMES</a> (for Highly Efficient Robotic Mechanisms and Electromechanical System) and developed methods for it to mimic the motions of an operator via teleoperation, an approach that the researchers say comes with certain humanistic advantages.</p><p>“Because you have a person who can learn and adapt on the fly, a robot can perform motions that it’s never practiced before [via teleoperation],” Ramos says.</p><p>In demonstrations, HERMES has poured coffee into a cup, wielded an ax to chop wood, and handled an extinguisher to put out a fire.</p><p>All these tasks have involved the robot’s upper body and algorithms to match the robot’s limb positioning with that of its operator’s. HERMES was able to carry out high-impact motions because the robot &nbsp;was rooted in place. Balance, in these cases, was much simpler to maintain. If the robot were required to take any steps, however, it would have likely tipped over in attempting to mimic the operator’s motions.</p><p>“We realized in order to generate high forces or move heavy objects, just copying motions wouldn’t be enough, because the robot would fall easily,” Kim says. “We needed to copy the operator’s dynamic balance.”</p><p>Enter Little HERMES, a miniature version of HERMES that is about a third the size of an average human adult. The team engineered the robot as simply a torso and two legs, and designed the system specifically to test lower-body tasks, such as locomotion and balance. As with its full-body counterpart, Little HERMES is designed for teleoperation, with an operator suited up in a vest to control the robot’s actions.</p><p>For the robot to copy the operator’s balance rather than just their motions, the team had to first find a simple way to represent balance. Ramos eventually realized that balance could be stripped down to two main ingredients: a person’s center of mass and their center of pressure — basically, a point on the ground where a force equivalent to all supporting forces is exerted.</p><p>The location of the center of mass in relation to the center of pressure, Ramos found, relates directly to how balanced a person is at any given time. He also found that the position of these two ingredients could be physically represented as an inverted &nbsp;pendulum. Imagine swaying from side to side while staying rooted to the same spot. The effect is similar to the swaying of an upside-down pendulum, the top end representing a human’s center of mass (usually in the torso) and the bottom representing their center of pressure on the ground.&nbsp;</p><p><strong>Heavy lifting</strong></p><p>To define how center of mass relates to center of pressure, Ramos gathered human motion data, including measurements in the lab, where he swayed back and forth, walked in place, and jumped on a force plate that measured the forces he exerted on the ground, as the position of his feet and torso were recorded. He then condensed this data into measurements of the center of mass and the center of pressure, and developed a model to represent each in relation to the other, as an inverted pendulum.</p><p>He then developed a second model, similar to the model for human balance but scaled to the dimensions of the smaller, lighter robot, and he developed a control algorithm to link and enable feedback between the two models.</p><p>The researchers tested this balance feedback model, first on a simple inverted pendulum that they built in the lab, in the form of a beam about the same height as Little HERMES. They connected the beam to their teleoperation system, and it swayed back and forth along a track in response to an operator’s movements. As the operator swayed to one side, the beam did likewise — a movement that the operator could also feel through the vest. If the beam swayed too far, the operator, feeling the pull, could lean the other way to compensate, and keep the beam balanced.</p><p>The experiments showed that the new feedback model could work to maintain balance on the beam, so the researchers then tried the model on Little HERMES. They also developed an algorithm for the robot to automatically translate the simple model of balance to the forces that each of its feet would have to generate, to copy the operator’s feet.</p><p>In the lab, Ramos found that as he wore the vest, he could not only control the robot’s motions and balance, but he also could feel the robot’s movements. When the robot was struck with a hammer from various directions, Ramos felt the vest jerk in the direction the robot moved. Ramos instinctively resisted the tug, which the robot registered as a subtle shift in the center of mass in relation to center of pressure, which it in turn mimicked. The result was that the robot was able to keep from tipping over, even amidst repeated blows to its body.</p><p>Little HERMES also mimicked Ramos in other exercises, including running and jumping in place, and walking on uneven ground, all while maintaining its balance without the aid of tethers or supports.</p><p>“Balance feedback is a difficult thing to define because it’s something we do without thinking,” Kim says. “This is the first time balance feedback is properly defined for the dynamic actions. This will change how we control a teleoperated humanoid.”</p><p>Kim and Ramos will continue to work on developing a full-body humanoid with similar balance control, to one day be able to gallop through a disaster zone and rise up to push away barriers as part of rescue or salvage missions.</p><p>“Now we can do heavy door opening or lifting or throwing heavy objects, with proper balance communication,” Kim says.</p><p>This research was supported, in part, by Hon Hai Precision Industry Co. Ltd. and Naver Labs Corporation.</p> Joao Ramos (center), co-inventor of HERMES (left), and Little HERMES (right) Photo: Tony Pulsone Self-transforming robot blocks jump, spin, flip, and identify each other Developed at MIT’s Computer Science and Artificial Intelligence Laboratory, robots can self-assemble to form various structures with applications including inspection. Wed, 30 Oct 2019 00:00:00 -0400 Rachel Gordon | MIT CSAIL <p>Swarms of simple, interacting robots have the potential to unlock stealthy abilities for accomplishing complex tasks. Getting these robots to achieve a true hive-like mind of coordination, though, has proved to be a hurdle.</p> <p>In an effort to change this, a team from MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL) came up with a surprisingly simple scheme: self-assembling robotic cubes that can climb over and around one another, leap through the air, and roll across the ground.</p> <p>Six years after the project’s first iteration, the robots can now “communicate” with each other using a barcode-like system on each face of the block that allows the modules to identify each other. The autonomous fleet of 16 blocks can now accomplish simple tasks or behaviors, such as forming a line, following arrows, or tracking light.</p> <p>Inside each modular “M-Block” is a flywheel that moves at 20,000 revolutions per minute, using angular momentum when the flywheel is braked. On each edge and every face are permanent magnets that let any two cubes attach to each other.</p> <p>While the cubes can’t be manipulated quite as easily as, say, those from the video game "Minecraft," the team envisions strong applications in inspection, and eventually disaster response. Imagine a burning building where a staircase has disappeared. In the future, you can envision simply throwing M-Blocks on the ground, and watching them build out a temporary staircase for climbing up to the roof, or down to the basement to rescue victims.</p> <p>“M stands for motion, magnet, and magic,” says MIT Professor and CSAIL Director Daniela Rus. “'Motion,' because the cubes can move by jumping. 'Magnet,' because the cubes can connect to other cubes using magnets, and once connected they can move together and connect to assemble structures. 'Magic,' because we don’t see any moving parts, and the cube appears to be driven by magic.”</p> <p>While the mechanism is quite intricate on the inside, the exterior is just the opposite, which enables more robust connections. Beyond inspection and rescue, the researchers also imagine using the blocks for things like gaming, manufacturing, and health care.</p><p>“The unique thing about our approach is that it’s inexpensive, robust, and potentially easier to scale to a million modules,'' says CSAIL PhD student John Romanishin, lead author on a new paper about the system. “M-Blocks can move in a general way. Other robotic systems have much more complicated movement mechanisms that require many steps, but our system is more scalable.”</p> <p>Romanishin wrote the paper alongside Rus and undergraduate student John Mamish of the University of Michigan. They will present the paper on M-blocks at IEEE’s International Conference on Intelligent Robots and Systems in November in Macau.</p> <p>Previous modular robot systems typically tackle movement using unit modules with small robotic arms known as external actuators. These systems require a lot of coordination for even the simplest movements, with multiple commands for one jump or hop.</p> <p>On the communication side, other attempts have involved the use of infrared light or radio waves, which can quickly get clunky: If you have lots of robots in a small area and they're all trying to send each other signals, it opens up a messy channel of conflict and confusion.</p> <p>When a system uses radio signals to communicate, the signals can interfere with each other when there are many radios in a small volume.</p> <p>Back in 2013, the team built out their mechanism for M-Blocks. They created six-faced cubes that move about using something called “inertial forces.” This means that, instead of using moving arms that help connect the structures, the blocks have a mass inside of them which they “throw” against the side of the module, which causes the block to rotate and move.</p> <p>Each module can move in four cardinal directions when placed on any one of the six faces, which results in 24 different movement directions. Without little arms and appendages sticking out of the blocks, it’s a lot easier for them to stay free of damage and avoid collisions.</p> <p>Knowing that the team had tackled the physical hurdles, the critical challenge still persisted: How to make these cubes communicate and reliably identify the configuration of neighboring modules?</p> <p>Romanishin came up with algorithms designed to help the robots accomplish simple tasks, or "behaviors,” which led them to the idea of a barcode-like system where the robots can sense the identity and face of what other blocks they’re connected to.</p> <p>In one experiment, the team had the modules turn into a line from a random structure, and they watched if the modules could determine the specific way that they were connected to each other. If they weren’t, they’d have to pick a direction and roll that way until they ended up on the end of the line.</p> <p>Essentially, the blocks used the configuration of how they're connected to each other in order to guide the motion that they choose to move — and 90 percent of the M-Blocks succeeded in getting into a line.</p> <p>The team notes that building out the electronics was very challenging, especially when trying to fit intricate hardware inside such a small package. To make the M-Block swarms a larger reality, the team wants just that — more and more robots to make bigger swarms with stronger capabilities for various structures.</p> <p>The project was supported, in part, by the National Science Foundation and Amazon Robotics.</p> One modular robotic cube snaps into place with rest of the M-blocks. Image: Jason Dorfman/MIT CSAIL Helping autonomous vehicles see around corners By sensing tiny changes in shadows, a new system identifies approaching objects that may cause a collision. Sun, 27 Oct 2019 23:59:59 -0400 Rob Matheson | MIT News Office <p>To improve the safety of autonomous systems, MIT engineers have developed a system that can sense tiny changes in shadows on the ground to determine if there’s a moving object coming around the corner. &nbsp;</p><p>Autonomous cars could one day use the system to quickly avoid a potential collision with another car or pedestrian emerging from around a building’s corner or from in between parked cars. In the future, robots that may navigate hospital hallways to make medication or supply deliveries could use the system to avoid hitting people.</p><p>In a paper being presented at next week’s International Conference on Intelligent Robots and Systems (IROS), the researchers describe successful experiments with an autonomous car driving around a parking garage and an autonomous wheelchair navigating hallways. When sensing and stopping for an approaching vehicle, the car-based system beats traditional LiDAR — which can only detect visible objects — by more than half a second.</p><p>That may not seem like much, but fractions of a second matter when it comes to fast-moving autonomous vehicles, the researchers say.</p><p>“For applications where robots are moving around environments with other moving objects or people, our method can give the robot an early warning that somebody is coming around the corner, so the vehicle can slow down, adapt its path, and prepare in advance to avoid a collision,” adds co-author Daniela Rus, director of the Computer Science and Artificial Intelligence Laboratory (CSAIL) and the Andrew and Erna Viterbi Professor of Electrical Engineering and Computer Science. “The big dream is to provide ‘X-ray vision’ of sorts to vehicles moving fast on the streets.”</p><p>Currently, the system has only been tested in indoor settings. Robotic speeds are much lower indoors, and lighting conditions are more consistent, making it easier for the system to sense and analyze shadows.</p><p>Joining Rus on the paper are: first author Felix Naser SM ’19, a former CSAIL researcher; Alexander Amini, a CSAIL graduate student; Igor Gilitschenski, a CSAIL postdoc; recent graduate Christina Liao ’19; Guy Rosman of the Toyota Research Institute; and Sertac Karaman, an associate professor of aeronautics and astronautics at MIT.</p><p><strong>Extending ShadowCam</strong></p><p>For their work, the researchers built on their system, called “ShadowCam,” that uses computer-vision techniques to detect and classify changes to shadows on the ground. MIT professors William Freeman and Antonio Torralba, who are not co-authors on the IROS paper, collaborated on the earlier versions of the system, which were presented at conferences in 2017 and 2018.</p><p>For input, ShadowCam uses sequences of video frames from a camera targeting a specific area, such as the floor in front of a corner. It detects changes in light intensity over time, from image to image, that may indicate something moving away or coming closer. Some of those changes may be difficult to detect or invisible to the naked eye, and can be determined by various properties of the object and environment. ShadowCam computes that information and classifies each image as containing a stationary object or a dynamic, moving one. If it gets to a dynamic image, it reacts accordingly.</p><p>Adapting ShadowCam for autonomous vehicles required a few advances. The early version, for instance, relied on lining an area with augmented reality labels called “AprilTags,” which resemble simplified QR codes. Robots scan AprilTags to detect and compute their precise 3D position and orientation relative to the tag. ShadowCam used the tags as features of the environment to zero in on specific patches of pixels that may contain shadows. But modifying real-world environments with AprilTags is not practical.</p><p>The researchers developed a novel process that combines image registration and a new visual-odometry technique. Often used in computer vision, image registration essentially overlays multiple images to reveal variations in the images. Medical image registration, for instance, overlaps medical scans to compare and analyze anatomical differences.</p><p>Visual odometry, used for Mars Rovers, estimates the motion of a camera in real-time by analyzing pose and geometry in sequences of images. The researchers specifically employ “Direct Sparse Odometry” (DSO), which can compute feature points in environments similar to those captured by AprilTags. Essentially, DSO plots features of an environment on a 3D point cloud, and then a computer-vision pipeline selects only the features located in a region of interest, such as the floor near a corner. (Regions of interest were annotated manually beforehand.)</p><p>As ShadowCam takes input image sequences of a region of interest, it uses the DSO-image-registration method to overlay all the images from same viewpoint of the robot. Even as a robot is moving, it’s able to zero in on the exact same patch of pixels where a shadow is located to help it detect any subtle deviations between images.</p><p>Next is signal amplification, a technique introduced in the first paper. Pixels that may contain shadows get a boost in color that reduces the signal-to-noise ratio. This makes extremely weak signals from shadow changes far more detectable. If the boosted signal reaches a certain threshold — based partly on how much it deviates from other nearby shadows —&nbsp;ShadowCam classifies the image as “dynamic.” Depending on the strength of that signal, the system may tell the robot to slow down or stop.</p><p>“By detecting that signal, you can then be careful. It may be a shadow of some person running from behind the corner or a parked car, so the autonomous car can slow down or stop completely,” Naser says.</p><p><strong>Tag-free testing</strong></p><p>In one test, the researchers evaluated the system’s performance in classifying moving or stationary objects using AprilTags and the new DSO-based method. An autonomous wheelchair steered toward various hallway corners while humans turned the corner into the wheelchair’s path. Both methods achieved the same 70-percent classification accuracy, indicating AprilTags are no longer needed.</p><p>In a separate test, the researchers implemented ShadowCam in an autonomous car in a parking garage, where the headlights were turned off, mimicking nighttime driving conditions. They compared car-detection times versus LiDAR. In an example scenario, ShadowCam detected the car turning around pillars about 0.72 seconds faster than LiDAR. Moreover, because the researchers had tuned ShadowCam specifically to the garage’s lighting conditions, the system achieved a classification accuracy of around 86 percent.</p><p>Next, the researchers are developing the system further to work in different indoor and outdoor lighting conditions. In the future, there could also be ways to speed up the system’s shadow detection and automate the process of annotating targeted areas for shadow sensing.</p><p>This work was funded by the Toyota Research Institute.</p> MIT engineers have developed a system for autonomous vehicles that senses tiny changes in shadows on the ground to determine if there’s a moving object coming around the corner, such as when another car is approaching from behind a pillar in a parking garage. System prevents speedy drones from crashing in unfamiliar areas Drones can fly at high speeds to a destination while keeping safe “backup” plans if things go awry. Fri, 25 Oct 2019 09:21:43 -0400 Rob Matheson | MIT News Office <p>Autonomous drones are cautious when navigating the unknown. They creep forward, frequently mapping unfamiliar areas before proceeding lest they crash into undetected objects. But this slowdown isn’t ideal for drones carrying out time-sensitive tasks, such as flying search-and-rescue missions through dense forests. &nbsp;</p><p>Now MIT researchers have developed a trajectory-planning model that helps drones fly at high speeds through previously unexplored areas, while staying safe.</p><p>The model — aptly named “FASTER” — estimates the quickest possible path from a starting point to a destination point across all areas the drone can and can’t see, with no regard for safety. But, as the drone flies, the model continuously logs collision-free “back-up” paths that slightly deviate from that fast flight path. When the drone is unsure about a particular area, it detours down the back-up path and replans its path. The drone can thus cruise at high speeds along the quickest trajectory while occasionally slowing down slightly to ensure safety.</p><p>“We always want to execute the fastest path, but we don’t always know it’s safe. If, as we move along this fastest path, we discover there’s a problem, we need to have a backup plan,” says Jesus Tordesillas, a graduate student in the Department of Aeronautics and Astronautics (AeroAstro) and first author on a paper describing the model being presented at next month’s International Conference on Intelligent Robots and Systems. “We obtain a higher velocity trajectory that may not be safe and a slow-velocity trajectory that’s completely safe. The two paths are stitched together at first, but then one deviates for performance and the other for safety.”</p><p>In forest simulations, where a virtual drone navigates around cylinders representing trees, FASTER-powered drones safely completed flight paths about two times quicker than traditional models. In real-life tests, FASTER-powered drones maneuvering around cardboard boxes in a large room achieved speeds of 7.8 meters per second. That’s pushing limits for how fast the drones can fly, based on weight and reaction times, the researchers say.</p><p>“That’s about as fast as you can go,” says co-author Jonathan How, the Richard Cockburn Maclaurin Professor of Aeronautics and Astronautics. “If you were standing in a room with a drone flying 7 to 8 meters per second in it, you’d probably take a step back."</p><p>The paper’s other co-author is Brett T. Lopez, a former PhD student in AeroAstro and now a postdoc at NASA’s Jet Propulsion Laboratory.</p><p><strong>Splitting paths </strong></p><p>Drones use cameras to capture environment as voxels, 3D cubes generated from depth information. As the drone flies, each detected voxel gets labeled as “free-known space,” unoccupied by objects, and “occupied-known space,” which contains objects. The rest of the environment is “unknown space.”&nbsp;</p><p>FASTER utilizes all of those areas to plan three types of trajectories — “whole,” “safe,” and “committed.” The whole trajectory is the entire path from starting point A to goal location B, through known and unknown areas. To do so, “convex decomposition,” a technique that breaks down complex models into discrete components, generates overlapping polyhedrons that model those three areas in an environment. Using some geometric techniques and mathematical constraints, the model uses these polyhedrons to compute an optimal whole trajectory.</p><p>Simultaneously, the model plans a safe trajectory. Somewhere along the whole trajectory, it plots a “rescue” point that indicates the last moment a drone can detour to unobstructed free-known space, based on its speed and other factors. To find a safe destination, it computes new polyhedrons that cover the free-known space. Then, it locates a spot inside these new polyhedrons. Basically, the drone stops in a spot that’s safe but as close as possible to unknown space, enabling a very quick and efficient detour.</p><p><strong>Committed trajectory</strong></p><p>The committed trajectory consists of the first interval of the whole trajectory, as well as the entire safe trajectory. But this first interval is independent of the safe trajectory, and therefore it is not affected by the braking needed for the safe trajectory.</p><p>The drone computes one whole trajectory at a time, while always keeping track of the safe trajectory. But it’s given a time limit: When it reaches the rescue point, it must have successfully computed the next whole trajectory through known or unknown space. If it does, it will continue following the whole trajectory. Otherwise, it diverts to the safe trajectory. This approach enables the drone to maintain high velocities along the committed trajectories, which is key to achieving high overall speeds.</p><p>For this to all work, the researchers designed ways for the drones to process all the planning data very quickly, which was challenging. Because the maps are so varied, for instance, the time limit given to each committed trajectory initially varied dramatically. That was computationally expensive and slowed down the drone’s planning, so the researchers developed a method to quickly compute fixed times for all the intervals along the trajectories, which simplified computations. The researchers also designed methods to reduce how many polyhedrons the drone must process to map its surroundings. Both of those methods dramatically increased planning times.</p> <p>"How to increase the flight speed and maintain safety is one of the hardest problems for drone’s motion planning,” says Sikang Liu, a software engineer at Waymo, formerly Google’s self-driving car project, and an expert in trajectory-planning algorithms. “This work showed a great solution to this problem by enhancing the existing trajectory generation framework. In the trajectory optimization pipeline, the time allocation is always a tricky problem that could lead to convergence issue and undesired behavior. This paper addressed this problem through a novel approach … which could be an insightful contribution to this field."</p><p>The researchers are currently building larger FASTER-powered drones with propellers designed to enable steady horizontal flight. Traditionally, drones will need to roll and pitch as they’re flying. But this custom drone would stay completely flat for various applications.</p><p>A potential application for FASTER, which has been developed with support by U.S. Department of Defense,&nbsp;could be improving search-and-rescue missions in forest environments, which present many planning and navigational challenges for autonomous drones. “But the unknown area doesn’t have to be forest,” How says. “It could be any area where you don’t know what’s coming, and it matters how quickly you acquire that knowledge. The main motivation is building more agile drones.”</p> MIT researchers have developed a trajectory-planning model that helps drones fly more safely at high speeds through previously unexplored areas, which could aid search-and-rescue missions through dense forests. Pushy robots learn the fundamentals of object manipulation Systems “learn” from novel dataset that captures how pushed objects move, to improve their physical interactions with new objects. Mon, 21 Oct 2019 23:59:59 -0400 Rob Matheson | MIT News Office <p>MIT researchers have compiled a dataset that captures the detailed behavior of a robotic system physically pushing hundreds of different objects. Using the dataset — the largest and most diverse of its kind — researchers can train robots to “learn” pushing dynamics that are fundamental to many complex object-manipulation tasks, including reorienting and inspecting objects, and uncluttering scenes.</p><p>To capture the data, the researchers designed an automated system consisting of an industrial robotic arm with precise control, a 3D motion-tracking system, depth and traditional cameras, and software that stitches everything together. The arm pushes around modular objects that can be adjusted for weight, shape, and mass distribution. For each push, the system captures how those characteristics affect the robot’s push.</p><p>The dataset, called “Omnipush,” contains 250 different pushes of 250 objects, totaling roughly 62,500 unique pushes. It’s already being used by researchers to, for instance, build models that help robots predict where objects will land when they’re pushed.</p><p>“We need a lot of rich data to make sure our robots can learn,” says Maria Bauza, a graduate student in the Department of Mechanical Engineering (MechE) and first author of a paper describing Omnipush that’s being presented at the upcoming International Conference on Intelligent Robots and Systems. “Here, we’re collecting data from a real robotic system, [and] the objects are varied enough to capture the richness of the pushing phenomena. This is important to help robots understand how pushing works, and to translate that information to other similar objects in the real world.”</p><p>Joining Bauza on the paper are: Ferran Alet and Yen-Chen Lin, graduate students in the Computer Science and Artificial Intelligence Laboratory and the Department of Electrical Engineering and Computer Science (EECS); Tomas Lozano-Perez, the School of Engineering Professor of Teaching Excellence; Leslie P. Kaelbling, the Panasonic Professor of Computer Science and Engineering; Phillip Isola, an assistant professor in EECS; and Alberto Rodriguez, an associate professor in MechE.</p><p><strong>Diversifying data</strong></p><p>Why focus on pushing behavior? Modeling pushing dynamics that involve friction between objects and surfaces, Rodriguez explains, is critical in higher-level robotic tasks. Consider the visually and technically impressive robot that can play Jenga, which Rodriguez recently co-designed. “The robot is performing a complex task, but the core of the mechanics driving that task is still that of pushing an object affected by, for instance, the friction between blocks,” Rodriguez says.</p><p>Omnipush builds on a similar dataset built in the Manipulation and Mechanisms Laboratory (MCube) by Rodriguez, Bauza, and other researchers that captured pushing data on only 10 objects. After making the dataset public in 2016, they gathered feedback from researchers. One complaint was lack of object diversity: Robots trained on the dataset struggled to generalize information to new objects. There was also no video, which is important for computer vision, video prediction, and other tasks.</p><p>For their new dataset, the researchers leverage an industrial robotic arm with precision control of the velocity and position of a pusher, basically a vertical steel rod. As the arm pushes the objects, a “Vicon”&nbsp;motion-tracking system —&nbsp;which has been used in films, virtual reality, and for research —&nbsp;follows the objects. There’s also an RGB-D camera, which adds depth information to captured video.</p><p>The key was building modular objects. The uniform central pieces, made from aluminum, look like four-pointed stars and weigh about 100 grams. Each central piece contains markers on its center and points, so the Vicon system can detect its pose within a millimeter.</p><p>Smaller pieces in four shapes — concave, triangular, rectangular, and circular — can be magnetically attached to any side of the central piece. Each piece weighs between 31 to 94 grams, but extra weights, ranging from 60 to 150 grams, can be dropped into little holes in the pieces. All pieces of the puzzle-like objects align both horizontally and vertically, which helps emulate the friction a single object with the same shape and mass distribution would have. All combinations of different sides, weights, and mass distributions added up to 250 unique objects.</p><p>For each push, the arm automatically moves to a random position several centimeters from the object. Then, it selects a random direction and pushes the object for one second. Starting from where it stopped, it then chooses another random direction and repeats the process 250 times. Each push records &nbsp;the pose of the object and RGB-D video, which can be used for various video-prediction purposes. Collecting the data took 12 hours a day, for two weeks, totaling more than 150 hours. Humans intervention was only needed when manually reconfiguring the objects.</p><p>The objects don’t specifically mimic any real-life items. Instead, they’re designed to capture the diversity of “kinematics” and “mass asymetries” expected of real-world objects, which model the physics of the motion of real-world objects. Robots can then extrapolate, say, the physics model of an Omnipush object with uneven mass distribution to any real-world object with similar uneven weight distributions.</p><p>“Imagine pushing a table with four legs, where most weight is over one of the legs. When you push the table, you see that it rotates on the heavy leg and have to readjust. Understanding that mass distribution, and its effect on the outcome of a push, is something robots can learn with this set of objects,” Rodriguez says.</p><p><strong>Powering new research</strong></p><p>In one experiment, the researchers used Omnipush to train a model to predict the final pose of pushed objects, given only the initial pose and description of the push. They trained the model on 150 Omnipush objects, and tested it on a held-out portion of objects. Results showed that the Omnipush-trained model was twice as accurate as models trained on a few similar datasets. In their paper, the researchers also recorded benchmarks in accuracy that other researchers can use for comparison.&nbsp;</p><p>Because Omnipush captures video of the pushes, one potential application is video prediction. A collaborator, for instance, is now using the dataset to train a robot to essentially “imagine” pushing objects between two points. After training on Omnipush, the robot is given as input two video frames, showing an object in its starting position and ending position. Using the starting position, the robot predicts all future video frames that ensure the object reaches its ending position. Then, it pushes the object in a way that matches each predicted video frame, until it gets to the frame with the ending position.</p><p>“The robot is asking, ‘If I do this action, where will the object be in this frame?’ Then, it selects the action that maximizes the likelihood of getting the object in the position it wants,” Bauza says. “It decides how to move objects by first imagining how the pixels in the image will change after a push.”</p><p>“Omnipush includes precise measurements of object motion, as well as visual data, for an important class of interactions between robot and objects in the world,” says Matthew T. Mason, a professor of computer science and robotics at Carnegie Melon University. “Robotics researchers can use this data to develop and test new robot learning approaches … that will fuel continuing advances in robotic manipulation.”</p> A key to compiling the novel Omnipush dataset was building modular objects (pictured) that enabled the robotic system to capture a vast diversity of pushing behavior. The central pieces contain markers on their centers and points so a motion-detection system can detect their position within a millimeter. Image courtesy of the researchers Deep learning with point clouds Research aims to make it easier for self-driving cars, robotics, and other applications to understand the 3D world. Mon, 21 Oct 2019 12:10:01 -0400 Adam Conner-Simons | CSAIL <p>If you’ve ever seen a self-driving car in the wild, you might wonder about that spinning cylinder on top of it.&nbsp;</p> <p>It’s a “lidar sensor,” and it’s what allows the car to navigate the world. By sending out pulses of infrared light and measuring the time it takes for them to bounce off objects, the sensor creates a “point cloud” that builds a 3D snapshot of the car’s surroundings.&nbsp;</p> <p>Making sense of raw point-cloud data is difficult, and before the age of machine learning it traditionally required highly trained engineers to tediously specify which qualities they wanted to capture by hand. But in a new series of papers out of MIT’s <a href="">Computer Science and Artificial Intelligence Laboratory</a> (CSAIL), researchers show that they can use deep learning to automatically process point clouds for a wide range of 3D-imaging applications.</p> <p>“In computer vision and machine learning today, 90 percent of the advances deal only with two-dimensional images,” says MIT Professor Justin Solomon, who was senior author of the new series of papers spearheaded by PhD student Yue Wang. “Our work aims to address a fundamental need to better represent the 3D world, with application not just in autonomous driving, but any field that requires understanding 3D shapes.”&nbsp;</p> <p>Most previous approaches haven’t been especially successful at capturing the patterns from data that are needed to get meaningful information out of a bunch of 3D points in space. But in one of the team’s papers, they showed that their “EdgeConv” method of analyzing point clouds using a type of neural network called a dynamic graph convolutional neural network allowed them to classify and segment individual objects.&nbsp;</p> <p>“By building ‘graphs’ of neighboring points, the algorithm can capture hierarchical patterns and therefore infer multiple types of generic information that can be used by a myriad of downstream tasks,” says Wadim Kehl, a machine learning scientist at Toyota Research Institute who was not involved in the work.&nbsp;</p> <p>In addition to developing EdgeConv, the team also explored other specific aspects of point-cloud processing. For example, one challenge is that most sensors change perspectives as they move around the 3D world; every time we take a new scan of the same object, its position may be different than the last time we saw it. To merge multiple point clouds together into a single detailed view of the world, you need to align multiple 3D points in a process called “registration.”&nbsp;</p> <p>Registration is vital for many forms of imaging, from satellite data to medical procedures. For example, when a doctor has to take multiple magnetic resonance imaging scans of a patient over time, registration is what makes it possible to align the scans to see what’s changed.&nbsp;</p> <p>“Registration is what allows us to integrate 3D data from different sources into a common coordinate system,” says Wang. “Without it, we wouldn’t actually be able to get as meaningful information from all these methods that have been developed.”</p> <p>Solomon and Wang’s second paper demonstrates a new registration algorithm called “Deep Closest Point” (DCP) that was shown to better find a point cloud’s distinguishing patterns, points, and edges (known as “local features”) in order to align it with other point clouds. This is especially important for such tasks as enabling self-driving cars to situate themselves in a scene (“localization”), as well as for robotic hands to locate and grasp individual objects.</p> <p>One limitation of DCP is that it assumes we can see an entire shape instead of just one side. This means it can’t handle the more difficult task of aligning partial views of shapes (known as “partial-to-partial registration”). As a result, in a third paper the researchers presented an improved algorithm for this task that they call the Partial Registration Network (PRNet).&nbsp;</p> <p>Solomon says that existing 3D data tends to be “quite messy and unstructured compared to 2D images and photographs.” His team sought to figure out how to get meaningful information out of all that disorganized 3D data without the controlled environment that a lot of machine learning technologies now require.</p> <p>A key observation behind the success of DCP and PRNet is the idea that a critical aspect of point-cloud processing is context. The geometric features on point cloud A that suggest the best ways to align it to point cloud B may be different from the features needed to align it to point cloud C. For example, in partial registration, an interesting part of a shape in one point cloud may not be visible in the other — making it useless for registration.</p> <p>Wang says that the team’s tools have already been deployed by many researchers in the computer vision community and beyond. Even physicists are using them for an application the CSAIL team had never considered: <a href="">particle</a> <a href="">physics</a>.&nbsp;</p> <p>Moving forward, the researchers hope to use the algorithms on real-world data, including data gathered from self-driving cars. Wang says they also plan to explore the potential of training their systems using self-supervised learning, to minimize the amount of human annotation needed.</p> <p>Solomon and Wang were the two sole authors of the DCP and PRNet papers. Their co-authors on the EdgeConv paper were research assistant Yongbin Sun and Professor Sanjay Sarma of MIT, alongside postdoc Ziwei Liu of University of California at Berkeley and Professor Michael M. Bronstein of Imperial College London.&nbsp;</p> <p>The projects were supported, in part, by the U.S. Air Force, the U.S. Army Research Office, Amazon, Google Research, IBM, the National Science Foundation, the Skoltech-MIT Next Generation Program, and the Toyota Research Institute.</p> At left, EdgeConv, a method developed at MIT, successfully finds meaningful parts of 3D shapes, like the surface of a table, wings of an airplane, and wheels of a skateboard. At right is the ground-truth comparison. Image: Courtesy of the researchers Giving robots a faster grasp An algorithm speeds up the planning process robots use to adjust their grip on objects, for picking and sorting, or tool use. Thu, 17 Oct 2019 09:39:41 -0400 Jennifer Chu | MIT News Office <p>If you’re at a desk with a pen or pencil handy, try this move: Grab the pen by one end with your thumb and index finger, and push the other end against the desk. Slide your fingers down the pen, then flip it upside down, without letting it drop. Not too hard, right?</p><p>But for a robot — say, one that’s sorting through a bin of objects and attempting to get a good grasp on one of them — this is a computationally taxing maneuver. Before even attempting the move it must calculate a litany of properties and probabilities, such as the friction and geometry of the table, the pen, and its two fingers, and how various combinations of these properties interact mechanically, based on fundamental laws of physics.</p><p>Now MIT engineers have found a way to significantly speed up the planning process required for a robot to adjust its grasp on an object by pushing that object against a stationary surface. Whereas traditional algorithms would require tens of minutes for planning out a sequence of motions, the new team’s approach shaves this preplanning process down to less than a second.</p><p>Alberto Rodriguez, associate professor of mechanical engineering at MIT, says the speedier planning process will enable robots, particularly in industrial settings, to quickly figure out how to push against, slide along, or otherwise use features in their environments to reposition objects in their grasp. Such nimble manipulation is useful for any tasks that involve picking and sorting, and even intricate tool use.</p><p>“This is a way to extend the dexterity of even simple robotic grippers, because at the end of the day, the environment is something every robot has around it,” Rodriguez says.</p><p>The team’s results are published today in <em>The International Journal of Robotics Research</em>. Rodriguez’ co-authors are lead author Nikhil Chavan-Dafle, a graduate student in mechanical engineering, and Rachel Holladay, a graduate student in electrical engineering and computer science.</p><p><strong>Physics in a cone</strong></p><p>Rodriguez’ group works on enabling robots to leverage their environment to help them accomplish physical tasks, such as picking and sorting objects in a bin. &nbsp;</p><p>Existing algorithms typically take hours to preplan a sequence of motions for a robotic gripper, mainly because, for every motion that it considers, the algorithm must first calculate whether that motion would satisfy a number of physical laws, such as Newton’s laws of motion and Coulomb’s law describing frictional forces between objects.</p><p>“It’s a tedious computational process to integrate all those laws, to consider all possible motions the robot can do, and to choose a useful one among those,” Rodriguez says.</p><p>He and his colleagues found a compact way to solve the physics of these manipulations, in advance of deciding how the robot’s hand should move. They did so by using “motion cones,” which are essentially visual, cone-shaped maps of friction.</p><p>The inside of the cone depicts all the pushing motions that could be applied to an object in a specific location, while satisfying the fundamental laws of physics and enabling the robot to keep hold of the object. The space outside of the cone represents all the pushes that would in some way cause an object to slip out of the robot’s grasp.</p><p>“Seemingly simple variations, such as how hard robot grasps the object, can significantly change how the object moves in the grasp when pushed,” Holladay explains. “Based on how hard you’re grasping, there will be a different motion. And that’s part of the physical reasoning that the algorithm handles.”</p><p>The team’s algorithm calculates a motion cone for different possible configurations between a robotic gripper, an object that it is holding, and the environment against which it is pushing, in order to select and sequence different feasible pushes to reposition the object.</p> <p><img alt="" src="/sites/default/files/images/inline/images/MIT-Robot-Gripper.gif" style="width: 500px; height: 281px;" /></p> <p><em><span style="font-size:10px;">A new algorithm speeds up the planning process for robotic grippers.&nbsp;A robot in the lab is shown picking up a block letter, T, and pushing it against a nearby wall to re-angle it, before setting it back down in an upright position.</span></em></p><p>“It’s a complicated process but still much faster than the traditional method — fast enough that planning an entire series of pushes takes half a second,” Holladay says.</p><p><strong>Big plans</strong></p><p>The researchers tested the new algorithm on a physical setup with a three-way interaction, in which a simple robotic gripper was holding a T-shaped block and pushing against a vertical bar. They used multiple starting configurations, with the robot gripping the block at a particular position and pushing it against the bar from a certain angle. For each starting configuration, the algorithm instantly generated the map of all the possible forces that the robot could apply and the position of the block that would result.</p><p>“We did several thousand pushes to verify our model correctly predicts what happens in the real world,” Holladay says. “If we apply a push that’s inside the cone, the grasped object should remain under control. If it’s outside, the object should slip from the grasp.”</p><p>The researchers found that the algorithm’s predictions reliably matched the physical outcome in the lab, planning out sequences of motions — such as reorienting the block against the bar before setting it down on a table in an upright position — in less than a second, compared with traditional algorithms that take over 500 seconds to plan out.</p><p>“Because we have this compact representation of the mechanics of this three-way-interaction between robot, object, and their environment, we can now attack bigger planning problems,” Rodriguez says.</p><p>The group is hoping to apply and extend its approach to enable a robotic gripper to handle different types of tools, for instance in a manufacturing setting.</p><p>“Most factory robots that use tools have a specially designed hand, so instead of having the abiity to grasp a screwdriver and use it in a lot of different ways, they just make the hand a screwdriver,” Holladay says. “You can imagine that requires less dexterous planning, but it’s much more limiting. We’d like a robot to be able to use and pick lots of different things up.”</p><p>This research was supported, in part, by Mathworks, the MIT-HKUST Alliance, and the National Science Foundation.</p> A new algorithm speeds up the planning process for robotic grippers to manipulate objects using the surrounding environment. Image courtesy of the researchers