Tag Archives: boston
#437639 Boston Dynamics’ Spot Is Helping ...
In terms of places where you absolutely want a robot to go instead of you, what remains of the utterly destroyed Chernobyl Reactor 4 should be very near the top of your list. The reactor, which suffered a catastrophic meltdown in 1986, has been covered up in almost every way possible in an effort to keep its nuclear core contained. But eventually, that nuclear material is going to have to be dealt with somehow, and in order to do that, it’s important to understand which bits of it are just really bad, and which bits are the actual worst. And this is where Spot is stepping in to help.
The big open space that Spot is walking through is right next to what’s left of Reactor 4. Within six months of the disaster, Reactor 4 was covered in a sarcophagus made of concrete and steel to try and keep all the nasty nuclear fuel from leaking out more than it already had, and it still contains “30 tons of highly contaminated dust, 16 tons of uranium and plutonium, and 200 tons of radioactive lava.” Oof. Over the next 10 years, the sarcophagus slowly deteriorated, and despite the addition of that gigantic network of steel support beams that you can see in the video, in the late 1990s it was decided to erect an enormous building over the entire mess to try and stabilize it for as long as possible.
Reactor 4 is now snugly inside the massive New Safe Confinement (NSC) structure, and the idea is that eventually, the structure will allow for the safe disassembly of what’s left of the reactor, although nobody is quite sure how to do that. This is all just to say that the area inside of the containment structure offers a lot of good opportunities for robots to take over from humans.
This particular Spot is owned by the U.K. Atomic Energy Authority, and was packed off to Russia with the assistance of the Robotics and Artificial Intelligence in Nuclear (RAIN) initiative and the National Centre for Nuclear Robotics. Dr. Dave Megson-Smith, who is a researcher at the University of Bristol, in the U.K., and part of the Hot Robotics Facility at the National Nuclear User Facility, was one of the scientists lucky enough to accompany Spot on its adventure. Megson-Smith specializes in sensor development, and he equipped Spot with a collimated radiation sensor in addition to its mapping payload. “We actually built a map of the radiation coming out of the front wall of Chernobyl power plant as we were in there with it,” Megson-Smith told us, and was able to share this picture, which shows a map of gamma photon count rate:
Image: University of Bristol
Researchers equipped Spot with a collimated radiation sensor and use one of the data readings (gamma photon count rate) to create a map of the radiation coming out of the front wall of the Chernobyl power plant.
So what’s the reason you’d want to use a very expensive legged robot to wander around what looks like a very flat and robot friendly floor? As it turns out, the floor is very dusty in there, and a priority inside the NSC is to keep dust down as much as possible, since the dust is radioactive and gets on everything and is consequently the easiest way for radioactivity to escape the NSC. “You want to minimize picking up material, so we consider the total contact surface area,” says Megson-Smith. “If you use a legged system rather than a wheeled or tracked system, you have a much smaller footprint and you disturb the environment a lot less.” While it’s nice that Spot is nimble and can climb stairs and stuff, tracked vehicles can do that as well, so in this case, the primary driving factor of choosing a robot to work inside Chernobyl is minimizing those contact points.
Right now, routine weekly measurements in contaminated spaces at Chernobyl are done by humans, which puts those humans at risk. Spot, or a robot like it, could potentially take over from those humans, as a sort of “automated safety checker”
Right now, routine weekly measurements in contaminated spaces at Chernobyl are done by humans, which puts those humans at risk. Spot, or a robot like it, could potentially take over from those humans, as a sort of “automated safety checker” able to work in medium level contaminated environments.” As far as more dangerous areas go, there’s a lot of uncertainty about what Spot is actually capable of, according to Megson-Smith. “What you think the problems are, and what the industry thinks the problems are, are subtly different things.
We were thinking that we’d have to make robots incredibly radiation proof to go into these contaminated environments, but they said, “can you just give us a system that we can send into places where humans already can go, but where we just don’t want to send humans.” Making robots incredibly radiation proof is challenging, and without extensive testing and ruggedizing, failures can be frequent, as many robots discovered at Fukushima. Indeed, Megson-Smith that in Fukushima there’s a particular section that’s known as a “robot graveyard” where robots just go to die, and they’ve had to up their standards again and again to keep the robots from failing. “So the thing they’re worried about with Spot is, what is its tolerance? What components will fail, and what can we do to harden it?” he says. “We’re approaching Boston Dynamics at the moment to see if they’ll work with us to address some of those questions.
There’s been a small amount of testing of how robots fair under harsh radiation, Megson-Smith told us, including (relatively recently) a KUKA LBR800 arm, which “stopped operating after a large radiation dose of 164.55(±1.09) Gy to its end effector, and the component causing the failure was an optical encoder.” And in case you’re wondering how much radiation that is, a 1 to 2 Gy dose to the entire body gets you acute radiation sickness and possibly death, while 8 Gy is usually just straight-up death. The goal here is not to kill robots (I mean, it sort of is), but as Megson-Smith says, “if we can work out what the weak points are in a robotic system, can we address those, can we redesign those, or at least understand when they might start to fail?” Now all he has to do is convince Boston Dynamics to send them a Spot that they can zap until it keels over.
The goal for Spot in the short term is fully autonomous radiation mapping, which seems very possible. It’ll also get tested with a wider range of sensor packages, and (happily for the robot) this will all take place safely back at home in the U.K. As far as Chernobyl is concerned, robots will likely have a substantial role to play in the near future. “Ultimately, Chernobyl has to be taken apart and decommissioned. That’s the long-term plan for the facility. To do that, you first need to understand everything, which is where we come in with our sensor systems and robotic platforms,” Megson-Smith tells us. “Since there are entire swathes of the Chernobyl nuclear plant where people can’t go in, we’d need robots like Spot to do those environmental characterizations.” Continue reading →
Posted in Human Robots
#437614 Video Friday: Poimo Is a Portable ...
Video Friday is your weekly selection of awesome robotics videos, collected by your Automaton bloggers. We’ll also be posting a weekly calendar of upcoming robotics events for the next few months; here's what we have so far (send us your events!):
IROS 2020 – October 25-29, 2020 – [Online]
ROS World 2020 – November 12, 2020 – [Online]
CYBATHLON 2020 – November 13-14, 2020 – [Online]
ICSR 2020 – November 14-16, 2020 – Golden, Colo., USA
Let us know if you have suggestions for next week, and enjoy today's videos.
Engineers at the University of California San Diego have built a squid-like robot that can swim untethered, propelling itself by generating jets of water. The robot carries its own power source inside its body. It can also carry a sensor, such as a camera, for underwater exploration.
[ UCSD ]
Thanks Ioana!
Shark Robotics, French and European leader in Unmanned Ground Vehicles, is announcing today a disinfection add-on for Boston Dynamics Spot robot, designed to fight the COVID-19 pandemic. The Spot robot with Shark’s purpose-built disinfection payload can decontaminate up to 2,000 m2 in 15 minutes, in any space that needs to be sanitized – such as hospitals, metro stations, offices, warehouses or facilities.
[ Shark Robotics ]
Here’s an update on the Poimo portable inflatable mobility project we wrote about a little while ago; while not strictly robotics, it seems like it holds some promise for rapidly developing different soft structures that robotics might find useful.
[ University of Tokyo ]
Thanks Ryuma!
Pretty cool that you can do useful force feedback teleop while video chatting through a “regular broadband Internet connection.” Although, what “regular” means to you is a bit subjective, right?
[ HEBI Robotics ]
Thanks Dave!
While NASA's Mars rover Perseverance travels through space toward the Red Planet, its nearly identical rover twin is hard at work on Earth. The vehicle system test bed (VSTB) rover named OPTIMISM is a full-scale engineering version of the Mars-bound rover. It is used to test hardware and software before the commands are sent up to the Perseverance rover.
[ NASA ]
Jacquard takes ordinary, familiar objects and enhances them with new digital abilities and experiences, while remaining true to their original purpose — like being your favorite jacket, backpack or a pair of shoes that you love to wear.
Our ambition is simple: to make life easier. By staying connected to your digital world, your things can do so much more. Skip a song by brushing your sleeve. Take a picture by tapping on a shoulder strap. Get reminded about the phone you left behind with a blink of light or a haptic buzz on your cuff.
[ Google ATAP ]
Should you attend the IROS 2020 workshop on “Planetary Exploration Robots: Challenges and Opportunities”? Of course you should!
[ Workshop ]
Kuka makes a lot of these videos where I can’t help but think that if they put as much effort into programming the robot as they did into producing the video, the result would be much more impressive.
[ Kuka ]
The Colorado School of Mines is one of the first customers to buy a Spot robot from Boston Dynamics to help with robotics research. Watch as scientists take Spot into the school's mine for the first time.
[ HCR ] via [ CNET ]
A very interesting soft(ish) actuator from Ayato Kanada at Kyushu University's Control Engineering Lab.
A flexible ultrasonic motor (FUSM), which generates linear motion as a novel soft actuator. This motor consists of a single metal cube stator with a hole and an elastic elongated coil spring inserted into the hole. When voltages are applied to piezoelectric plates on the stator, the coil spring moves back and forward as a linear slider. In the FUSM that uses the friction drive as the principle, the most important parameter for optimizing its output is the preload between the stator and slider. The coil spring has a slightly larger diameter than the stator hole and generates the preload by expanding in a radial direction. The coil springs act not only as a flexible slider but also as a resistive positional sensor. Changes in the resistance between the stator and the coil spring end are converted to a voltage and used for position detection.
[ Control Engineering Lab ]
Thanks Ayato!
We show how to use the limbs of a quadruped robot to identify fine-grained soil, representative for Martian regolith.
[ Paper ] via [ ANYmal Research ]
PR2 is serving breakfast and cleaning up afterwards. It’s slow, but all you have to do is eat and leave.
That poor PR2 is a little more naked than it's probably comfortable with.
[ EASE ]
NVIDIA researchers present a hierarchical framework that combines model-based control and reinforcement learning (RL) to synthesize robust controllers for a quadruped robot (the Unitree Laikago).
[ NVIDIA ]
What's interesting about this assembly task is that the robot is using its arm only for positioning, and doing the actual assembly with just fingers.
[ RC2L ]
In this electronics assembly application, Kawasaki's cobot duAro2 uses a tool changing station to tackle a multitude of tasks and assemble different CPU models.
Okay but can it apply thermal paste to a CPU in the right way? Personally, I find that impossible.
[ Kawasaki ]
You only need to watch this video long enough to appreciate the concept of putting a robot on a robot.
[ Impress ]
In this lecture, we’ll hear from the man behind one of the biggest robotics companies in the world, Boston Dynamics, whose robotic dog, Spot, has been used to encourage social distancing in Singapore and is now getting ready for FDA approval to be able to measure patients’ vital signs in hospitals.
[ Alan Turing Institute ]
Greg Kahn from UC Berkeley wrote in to share his recent dissertation talk on “Mobile Robot Learning.”
In order to create mobile robots that can autonomously navigate real-world environments, we need generalizable perception and control systems that can reason about the outcomes of navigational decisions. Learning-based methods, in which the robot learns to navigate by observing the outcomes of navigational decisions in the real world, offer considerable promise for obtaining these intelligent navigation systems. However, there are many challenges impeding mobile robots from autonomously learning to act in the real-world, in particular (1) sample-efficiency–how to learn using a limited amount of data? (2) supervision–how to tell the robot what to do? and (3) safety–how to ensure the robot and environment are not damaged or destroyed during learning? In this talk, I will present deep reinforcement learning methods for addressing these real world mobile robot learning challenges and show results which enable ground and aerial robots to navigate in complex indoor and outdoor environments.
[ UC Berkeley ]
Thanks Greg!
Leila Takayama from UC Santa Cruz (and previously Google X and Willow Garage) gives a talk entitled “Toward a more human-centered future of robotics.”
Robots are no longer only in outer space, in factory cages, or in our imaginations. We interact with robotic agents when withdrawing cash from bank ATMs, driving cars with adaptive cruise control, and tuning our smart home thermostats. In the moment of those interactions with robotic agents, we behave in ways that do not necessarily align with the rational belief that robots are just plain machines. Through a combination of controlled experiments and field studies, we use theories and concepts from the social sciences to explore ways that human and robotic agents come together, including how people interact with personal robots and how people interact through telepresence robots. Together, we will explore topics and raise questions about the psychology of human-robot interaction and how we could invent a future of a more human-centered robotics that we actually want to live in.
[ Leila Takayama ]
Roboticist and stand-up comedian Naomi Fitter from Oregon State University gives a talk on “Everything I Know about Telepresence.”
Telepresence robots hold promise to connect people by providing videoconferencing and navigation abilities in far-away environments. At the same time, the impacts of current commercial telepresence robots are not well understood, and circumstances of robot use including internet connection stability, odd personalizations, and interpersonal relationship between a robot operator and people co-located with the robot can overshadow the benefit of the robot itself. And although the idea of telepresence robots has been around for over two decades, available nonverbal expressive abilities through telepresence robots are limited, and suitable operator user interfaces for the robot (for example, controls that allow for the operator to hold a conversation and move the robot simultaneously) remain elusive. So where should we be using telepresence robots? Are there any pitfalls to watch out for? What do we know about potential robot expressivity and user interfaces? This talk will cover my attempts to address these questions and ways in which the robotics research community can build off of this work
[ Talking Robotics ] Continue reading →
Posted in Human Robots
#437590 Why We Need a Robot Registry
I have a confession to make: A robot haunts my nightmares. For me, Boston Dynamics’ Spot robot is 32.5 kilograms (71.1 pounds) of pure terror. It can climb stairs. It can open doors. Seeing it in a video cannot prepare you for the moment you cross paths on a trade-show floor. Now that companies can buy a Spot robot for US $74,500, you might encounter Spot anywhere.
Spot robots now patrol public parks in Singapore to enforce social distancing during the pandemic. They meet with COVID-19 patients at Boston’s Brigham and Women’s Hospital so that doctors can conduct remote consultations. Imagine coming across Spot while walking in the park or returning to your car in a parking garage. Wouldn’t you want to know why this hunk of metal is there and who’s operating it? Or at least whom to call to report a malfunction?
Robots are becoming more prominent in daily life, which is why I think governments need to create national registries of robots. Such a registry would let citizens and law enforcement look up the owner of any roaming robot, as well as learn that robot’s purpose. It’s not a far-fetched idea: The U.S. Federal Aviation Administration already has a registry for drones.
Governments could create national databases that require any companies operating robots in public spaces to report the robot make and model, its purpose, and whom to contact if the robot breaks down or causes problems. To allow anyone to use the database, all public robots would have an easily identifiable marker or model number on their bodies. Think of it as a license plate or pet microchip, but for bots.
There are some smaller-scale registries today. San Jose’s Department of Transportation (SJDOT), for example, is working with Kiwibot, a delivery robot manufacturer, to get real-time data from the robots as they roam the city’s streets. The Kiwibots report their location to SJDOT using the open-source Mobility Data Specification, which was originally developed by Los Angeles to track Bird scooters.
Real-time location reporting makes sense for Kiwibots and Spots wandering the streets, but it’s probably overkill for bots confined to cleaning floors or patrolling parking lots. That said, any robots that come in contact with the general public should clearly provide basic credentials and a way to hold their operators accountable. Given that many robots use cameras, people may also be interested in looking up who’s collecting and using that data.
I starting thinking about robot registries after Spot became available in June for anyone to purchase. The idea gained specificity after listening to Andra Keay, founder and managing director at Silicon Valley Robotics, discuss her five rules of ethical robotics at an Arm event in October. I had already been thinking that we needed some way to track robots, but her suggestion to tie robot license plates to a formal registry made me realize that people also need a way to clearly identify individual robots.
Keay pointed out that in addition to sating public curiosity and keeping an eye on robots that could cause harm, a registry could also track robots that have been hacked. For example, robots at risk of being hacked and running amok could be required to report their movements to a database, even if they’re typically restricted to a grocery store or warehouse. While we’re at it, Spot robots should be required to have sirens, because there’s no way I want one of those sneaking up on me.
This article appears in the December 2020 print issue as “Who’s Behind That Robot?” Continue reading →
Posted in Human Robots
#437562 Video Friday: Aquanaut Robot Takes to ...
Video Friday is your weekly selection of awesome robotics videos, collected by your Automaton bloggers. We’ll also be posting a weekly calendar of upcoming robotics events for the next few months; here's what we have so far (send us your events!):
IROS 2020 – October 25-25, 2020 – [Online]
ICSR 2020 – November 14-16, 2020 – Golden, Colo., USA
Bay Area Robotics Symposium – November 20, 2020 – [Online]
ACRA 2020 – December 8-10, 2020 – [Online]
Let us know if you have suggestions for next week, and enjoy today's videos.
To prepare the Perseverance rover for its date with Mars, NASA’s Mars 2020 mission team conducted a wide array of tests to help ensure a successful entry, descent and landing at the Red Planet. From parachute verification in the world’s largest wind tunnel, to hazard avoidance practice in Death Valley, California, to wheel drop testing at NASA’s Jet Propulsion Laboratory and much more, every system was put through its paces to get ready for the big day. The Perseverance rover is scheduled to land on Mars on February 18, 2021.
[ JPL ]
Awesome to see Aquanaut—the “underwater transformer” we wrote about last year—take to the ocean!
Also their new website has SHARKS on it.
[ HMI ]
Nature has inspired engineers at UNSW Sydney to develop a soft fabric robotic gripper which behaves like an elephant's trunk to grasp, pick up and release objects without breaking them.
[ UNSW ]
Collaborative robots offer increased interaction capabilities at relatively low cost but, in contrast to their industrial counterparts, they inevitably lack precision. We address this problem by relying on a dual-arm system with laser-based sensing to measure relative poses between objects of interest and compensate for pose errors coming from robot proprioception.
[ Paper ]
Developed by NAVER LABS, with Korea University of Technology & Education (Koreatech), the robot arm now features an added waist, extending the available workspace, as well as a sensor head that can perceive objects. It has also been equipped with a robot hand “BLT Gripper” that can change to various grasping methods.
[ NAVER Labs ]
In case you were still wondering why SoftBank acquired Aldebaran and Boston Dynamics:
[ RobotStart ]
DJI's new Mini 2 drone is here with a commercial so hip it makes my teeth scream.
[ DJI ]
Using simple materials, such as plastic struts and cardboard rolls, the first prototype of the RBO Hand 3 is already capable of grasping a large range of different objects thanks to its opposable thumb.
The RBO Hand 3 performs an edge grasp before handing-over the object to a person. The hand actively exploits constraints in the environment (the tabletop) for grasping the object. Thanks to its compliance, this interaction is safe and robust.
[ TU Berlin ]
Flyability's Elios 2 helped researchers inspect Reactor Five at the Chernobyl nuclear disaster site in order to determine whether any uranium was present. Prior to this mission, Reactor Five had not been investigated since the disaster in April of 1986.
[ Flyability ]
Thanks Zacc!
SOTO 2 is here! Together with our development partners from the industry, we have greatly enhanced the SOTO prototype over the last two years. With the new version of the robot, Industry 4.0 will become a great deal more real: SOTO brings materials to the assembly line, just-in-time and completely autonomously.
[ Magazino ]
A drone that can fly sustainably for long distances over land and water, and can land almost anywhere, will be able to serve a wide range of applications. There are already drones that fly using ‘green’ hydrogen, but they either fly very slowly or cannot land vertically. That’s why researchers at TU Delft, together with the Royal Netherlands Navy and the Netherlands Coastguard, developed a hydrogen-powered drone that is capable of vertical take-off and landing whilst also being able to fly horizontally efficiently for several hours, much like regular aircraft. The drone uses a combination of hydrogen and batteries as its power source.
[ MAVLab ]
The National Nuclear User Facility for Hot Robotics (NNUF-HR) is an EPSRC funded facility to support UK academia and industry to deliver ground-breaking, impactful research in robotics and artificial intelligence for application in extreme and challenging nuclear environments.
[ NNUF ]
At the Karolinska University Laboratory in Sweden, an innovation project based around an ABB collaborative robot has increased efficiency and created a better working environment for lab staff.
[ ABB ]
What I find interesting about DJI's enormous new agricultural drone is that it's got a spinning obstacle detecting sensor that's a radar, not a lidar.
Also worth noting is that it seems to detect the telephone pole, but not the support wire that you can see in the video feed, although the visualization does make it seem like it can spot the power lines above.
[ DJI ]
Josh Pieper has spend the last year building his own quadruped, and you can see what he's been up to in just 12 minutes.
[ mjbots ]
Thanks Josh!
Dr. Ryan Eustice, TRI Senior Vice President of Automated Driving, delivers a keynote speech — “The Road to Vehicle Automation, a Toyota Guardian Approach” — to SPIE's Future Sensing Technologies 2020. During the presentation, Eustice provides his perspective on the current state of automated driving, summarizes TRI's Guardian approach — which amplifies human drivers, rather than replacing them — and summarizes TRI's recent developments in core AD capabilities.
[ TRI ]
Two excellent talks this week from UPenn GRASP Lab, from Ruzena Bajcsy and Vijay Kumar.
A panel discussion on the future of robotics and societal challenges with Dr. Ruzena Bajcsy as a Roboticist and Founder of the GRASP Lab.
In this talk I will describe the role of the White House Office of Science and Technology Policy in supporting science and technology research and education, and the lessons I learned while serving in the office. I will also identify a few opportunities at the intersection of technology and policy and broad societal challenges.
[ UPenn ]
The IROS 2020 “Perception, Learning, and Control for Autonomous Agile Vehicles” workshop is all online—here's the intro, but you can click through for a playlist that includes videos of the entire program, and slides are available as well.
[ NYU ] Continue reading →
Posted in Human Robots
#437543 This Is How We’ll Engineer Artificial ...
Take a Jeopardy! guess: this body part was once referred to as the “consummation of all perfection as an instrument.”
Answer: “What is the human hand?”
Our hands are insanely complex feats of evolutionary engineering. Densely-packed sensors provide intricate and ultra-sensitive feelings of touch. Dozens of joints synergize to give us remarkable dexterity. A “sixth sense” awareness of where our hands are in space connects them to the mind, making it possible to open a door, pick up a mug, and pour coffee in total darkness based solely on what they feel.
So why can’t robots do the same?
In a new article in Science, Dr. Subramanian Sundaram at Boston and Harvard University argues that it’s high time to rethink robotic touch. Scientists have long dreamed of artificially engineering robotic hands with the same dexterity and feedback that we have. Now, after decades, we’re at the precipice of a breakthrough thanks to two major advances. One, we better understand how touch works in humans. Two, we have the mega computational powerhouse called machine learning to recapitulate biology in silicon.
Robotic hands with a sense of touch—and the AI brain to match it—could overhaul our idea of robots. Rather than charming, if somewhat clumsy, novelties, robots equipped with human-like hands are far more capable of routine tasks—making food, folding laundry—and specialized missions like surgery or rescue. But machines aren’t the only ones to gain. For humans, robotic prosthetic hands equipped with accurate, sensitive, and high-resolution artificial touch is the next giant breakthrough to seamlessly link a biological brain to a mechanical hand.
Here’s what Sundaram laid out to get us to that future.
How Does Touch Work, Anyway?
Let me start with some bad news: reverse engineering the human hand is really hard. It’s jam-packed with over 17,000 sensors tuned to mechanical forces alone, not to mention sensors for temperature and pain. These force “receptors” rely on physical distortions—bending, stretching, curling—to signal to the brain.
The good news? We now have a far clearer picture of how biological touch works. Imagine a coin pressed into your palm. The sensors embedded in the skin, called mechanoreceptors, capture that pressure, and “translate” it into electrical signals. These signals pulse through the nerves on your hand to the spine, and eventually make their way to the brain, where they gets interpreted as “touch.”
At least, that’s the simple version, but one too vague and not particularly useful for recapitulating touch. To get there, we need to zoom in.
The cells on your hand that collect touch signals, called tactile “first order” neurons (enter Star Wars joke) are like upside-down trees. Intricate branches extend from their bodies, buried deep in the skin, to a vast area of the hand. Each neuron has its own little domain called “receptor fields,” although some overlap. Like governors, these neurons manage a semi-dedicated region, so that any signal they transfer to the higher-ups—spinal cord and brain—is actually integrated from multiple sensors across a large distance.
It gets more intricate. The skin itself is a living entity that can regulate its own mechanical senses through hydration. Sweat, for example, softens the skin, which changes how it interacts with surrounding objects. Ever tried putting a glove onto a sweaty hand? It’s far more of a struggle than a dry one, and feels different.
In a way, the hand’s tactile neurons play a game of Morse Code. Through different frequencies of electrical beeps, they’re able to transfer information about an object’s size, texture, weight, and other properties, while also asking the brain for feedback to better control the object.
Biology to Machine
Reworking all of our hands’ greatest features into machines is absolutely daunting. But robots have a leg up—they’re not restricted to biological hardware. Earlier this year, for example, a team from Columbia engineered a “feeling” robotic finger using overlapping light emitters and sensors in a way loosely similar to receptor fields. Distortions in light were then analyzed with deep learning to translate into contact location and force.
Although a radical departure from our own electrical-based system, the Columbia team’s attempt was clearly based on human biology. They’re not alone. “Substantial progress is being made in the creation of soft, stretchable electronic skins,” said Sundaram, many of which can sense forces or pressure, although they’re currently still limited.
What’s promising, however, is the “exciting progress in using visual data,” said Sundaram. Computer vision has gained enormously from ubiquitous cameras and large datasets, making it possible to train powerful but data-hungry algorithms such as deep convolutional neural networks (CNNs).
By piggybacking on their success, we can essentially add “eyes” to robotic hands, a superpower us humans can’t imagine. Even better, CNNs and other classes of algorithms can be readily adopted for processing tactile data. Together, a robotic hand could use its eyes to scan an object, plan its movements for grasp, and use touch for feedback to adjust its grip. Maybe we’ll finally have a robot that easily rescues the phone sadly dropped into a composting toilet. Or something much grander to benefit humanity.
That said, relying too heavily on vision could also be a downfall. Take a robot that scans a wide area of rubble for signs of life during a disaster response. If touch relies on sight, then it would have to keep a continuous line-of-sight in a complex and dynamic setting—something computer vision doesn’t do well in, at least for now.
A Neuromorphic Way Forward
Too Debbie Downer? I got your back! It’s hard to overstate the challenges, but what’s clear is that emerging machine learning tools can tackle data processing challenges. For vision, it’s distilling complex images into “actionable control policies,” said Sundaram. For touch, it’s easy to imagine the same. Couple the two together, and that’s a robotic super-hand in the making.
Going forward, argues Sundaram, we need to closely adhere to how the hand and brain process touch. Hijacking our biological “touch machinery” has already proved useful. In 2019, one team used a nerve-machine interface for amputees to control a robotic arm—the DEKA LUKE arm—and sense what the limb and attached hand were feeling. Pressure on the LUKE arm and hand activated an implanted neural interface, which zapped remaining nerves in a way that the brain processes as touch. When the AI analyzed pressure data similar to biological tactile neurons, the person was able to better identify different objects with their eyes closed.
“Neuromorphic tactile hardware (and software) advances will strongly influence the future of bionic prostheses—a compelling application of robotic hands,” said Sundaram, adding that the next step is to increase the density of sensors.
Two additional themes made the list of progressing towards a cyborg future. One is longevity, in that sensors on a robot need to be able to reliably produce large quantities of high-quality data—something that’s seemingly mundane, but is a practical limitation.
The other is going all-in-one. Rather than just a pressure sensor, we need something that captures the myriad of touch sensations. From feather-light to a heavy punch, from vibrations to temperatures, a tree-like architecture similar to our hands would help organize, integrate, and otherwise process data collected from those sensors.
Just a decade ago, mind-controlled robotics were considered a blue sky, stretch-goal neurotechnological fantasy. We now have a chance to “close the loop,” from thought to movement to touch and back to thought, and make some badass robots along the way.
Image Credit: PublicDomainPictures from Pixabay Continue reading →
Posted in Human Robots