Tag Archives: drones
#439826 Autonomous Racing Drones Dodge Through ...
It seems inevitable that sooner or later, the performance of autonomous drones will
surpass the performance of even the best human pilots. Usually things in robotics that seem inevitable happen later as opposed to sooner, but drone technology seems to be the exception to this. We've seen an astonishing amount of progress over the past few years, even to the extent of sophisticated autonomy making it into the hands of consumers at an affordable price.
The cutting edge of drone research right now is putting drones with relatively simple onboard sensing and computing in situations that require fast and highly aggressive maneuvers. In a paper
published yesterday in Science Robotics, roboticists from Davide Scaramuzza's Robotics and Perception Group at the University of Zurich along with partners at Intel demonstrate a small, self-contained, fully autonomous drone that can aggressively fly through complex environments at speeds of up to 40kph.
The trick here, to the extent that there's a trick, is that the drone performs a direct mapping of sensor input (from an Intel RealSense 435 stereo depth camera) to collision-free trajectories. Conventional obstacle avoidance involves first collecting sensor data; making a map based on that sensor data; and finally making a plan based on that map. This approach works perfectly fine as long as you're not concerned with getting all of that done quickly, but for a drone with limited onboard resources moving at high speed, it just takes too long. UZH's approach is instead to go straight from sensor input to trajectory output, which is much faster and allows the speed of the drone to increase substantially.
The convolutional network that performs this sensor-to-trajectory mapping was trained entirely in simulation, which is cheaper and easier but (I would have to guess) less fun than letting actual drones hammer themselves against obstacles over and over until they figure things out. A simulated “expert” drone pilot that has access to a 3D point cloud, perfect state estimation, and computation that's not constrained by real-time requirements trains its own end-to-end policy, which is of course not achievable in real life. But then, the simulated system that will be operating under real-life constraints just learns in simulation to match the expert as closely as possible, which is how you get that expert-level performance in a way that can be taken out of simulation and transferred to a real drone without any adaptation or fine-tuning.
The other big part of this is making that sim-to-real transition, which can be problematic because simulation doesn't always do a great job of simulating everything that happens in the world that can screw with a robot. But this method turns out to be very robust against motion blur, sensor noise, and other perception artifacts. The drone has successfully navigated through real world environments including snowy terrains, derailed trains, ruins, thick vegetation, and collapsed buildings.
“While humans require years to train, the AI, leveraging high-performance simulators, can reach comparable navigation abilities much faster, basically overnight.” -Antonio Loquercio, UZH
This is not to say that the performance here is flawless—the system still has trouble with very low illumination conditions (because the cameras simply can't see), as well as similar vision challenges like dust, fog, glare, and transparent or reflective surfaces. The training also didn't include dynamic obstacles, although the researchers tell us that moving things shouldn't be a problem even now as long as their speed relative to the drone is negligible. Many of these problems could potentially be mitigated by using
event cameras rather than traditional cameras, since faster sensors, especially ones tuned to detect motion, would be ideal for high speed drones.
The researchers tell us that their system does not (yet) surpass the performance of expert humans in these challenging environments:
Analyzing their performance indicates that humans have a very rich and detailed understanding of their surroundings and are capable of planning and executing plans that span far in the future (our approach plans only one second into the future). Both are capabilities that today's autonomous systems still lack. We see our work as a stepping stone towards faster autonomous flight that is enabled by directly predicting collision-free trajectories from high-dimensional (noisy) sensory input.
This is one of the things that is likely coming next, though—giving the drone the ability to learn and improve from real-world experience. Coupled with more capable sensors and always increasing computer power, pushing that flight envelope past 40 kph in complex environments seems like it's not just possible, but inevitable. Continue reading
Posted in Human Robots
#439568 Corvus Robotics’ Autonomous Drones ...
Warehouses offer all kinds of opportunities for robots. Semi-structured controlled environments, lots of repetitive tasks, and humans that would almost universally rather be somewhere else. Robots have been doing great at taking over jobs that involve moving stuff from one place to another, but there are all kinds of other things that have to happen to keep warehouses operating efficiently.
Corvus Robotics, a YC-backed startup that's just coming out of stealth, has decided that they want to go after warehouse inventory tracking. That is, making sure that a warehouse knows exactly what's inside of it and where. This is a more complicated task than it seems like it should be, and not just any robot is able to do it. Corvus' solution involves autonomous drones that can fly unattended for weeks on end, collecting inventory data without any human intervention at all.
Many warehouses have a dedicated team of humans whose job is to wander around the warehouse scanning stuff to maintain an up to date list of where everything is, a task which is both very important and very boring. As it turns out, autonomous drones can scan up to ten times faster than humans—Corvus Robotics' drones are able to inventory an entire warehouse on a rolling basis in just a couple days, while it would take a human team weeks to do the same task.
Inventory is a significant opportunity for robotics, and we've seen a bunch of different attempts at doing inventory in places like supermarkets, but warehouses are different. Warehouses can be huge, in every dimension, meaning that the kinds of robots that can make supermarket inventory work just won't cut it in a warehouse environment for the simple reason that they can't see inventory stacked on shelves all the way to the ceiling, which can be over 20m high. And this is why the drone form factor, while novel, actually offers a uniquely useful solution.
It's probably fair to think of a warehouse as a semi-structured environment, with emphasis on the “semi.” At the beginning of a deployment, Corvus will generate one map of the operating area that includes both geometric and semantic information. After that, the drones will autonomously update that map with each flight throughout their entire lifetimes. There are walls and ceilings that don't move, along with large shelving units that are mostly stationary, but those things aren't going to do your localization system any favors since they all look the same. And the stuff that does offer some uniqueness, like the items on those shelves, is changing all the time. “That's a huge problem for us,” says Mohammed Kabir, Corvus Robotics' CTO. “Being able to do place recognition at the granularity that we need while everything is changing is really hard.” If you were looking closely at the video, you may have spotted some fiducials (optical patterns placed in the environment that vision systems find easy to spot), but we're told that the video was shot in Corvus Robotics' development warehouse where those markers are used for ground truth testing.
In real deployments, fiducials (or anything else) isn't necessary. The drone has its charging dock, and the initial map, but otherwise it's doing onboard visual-inertial SLAM (simultaneous localization and mapping), dense volumetric mapping, and motion planning with its 10 camera array and an autonomy stack running on ROS and PX4 for real time flight control. Corvus isn't willing to let us in on all of their secrets, but they did tell us that they incorporate some of the structured components of the environment into their SLAM solution, as well as some things are semi-static—that is, things that are unlikely to change over the duration of a single flight, helping the drone with loop closure.
One of the big parts of being able to do this is the ability to localize in very large, unstructured environments where things are constantly changing without having to rely on external infrastructure. For example, a WiFi connection back to our base station is not guaranteed, so everything needs to run on-board the drone, which is a non-trivial task. It's essentially all of the compute of a self-driving car, compressed into the drone. -Mohammed KabirCorvus is able to scan between 200 and 400 pallet positions per hour per drone, inclusive of recharge time. At ground level, this is probably about equivalent in speed to a human (although more sustainable). But as you start looking at inventory higher off the ground, the drone maintains a constant scan rate, while for a human, it gets exponentially harder, involving things like strapping yourself to a forklift. And of course the majority of the items in a high warehouse are not at ground level, because ground level only covers a tier or two of a space that may soar to 20 meters. Overall, Corvus says that they can do inventory up to 10x faster than a human.
With a few exceptions, it's unlikely that most warehouses are going to be able to go human-free in the foreseeable future, meaning that any time you talk about robot autonomy, you also have to talk about safety. “We can operate when no one's around, so our customers often schedule the drones during the third shift when the warehouse is dark,” says Mohammed Kabir. “There are also customers who want us to operate around people, which initially terrified us, because interacting with humans can be quite tricky. But over the last couple years, we've built safety systems to be able to deal with that.” In addition to the collision avoidance that comes with the 360 degree vision system that the drone uses to navigate, it has a variety of safety-first behaviors all the way up to searching for clear flat spots to land in the event of an emergency. But it sounds like the primary way that Corvus tries to maintain safety is by keeping drones and humans as separate as possible, which may involve process changes for the warehouse, explains Corvus Robotics CEO Jackie Wu. “If you see a drone in an aisle, just don't go in until it's done.”
We also asked Wu about what exactly he means when he calls the Corvus Robotics' drone “fully autonomous,” because depending on who you ask (and what kind of robot and task you're talking about), full autonomy can mean a lot of different things.
For us, full autonomy means continuous end to end operation with no human in the loop within a certain scenario or environment. Obviously, it's not level five autonomy, because nobody is doing level five, which would take some kind of generalized intelligence that can fly anywhere. But, for level four, for the warehouse interior, the drones fly on scheduled missions, intelligently find objects of interest while avoiding collisions, come back to land, recharge and share that data, all without anybody touching them. And we're able to do this repeatedly, without external localization infrastructure. -Jackie WuAs tempting as it is, we're not going to get into the weeds here about what exactly constitutes “full autonomy” in the context of drones. Well, okay, maybe we'll get into the weeds a little bit, just to say that being able to repeatedly do a useful task end-to-end without a human in the loop seems close enough to whatever your definition of full autonomy is that it's probably a fair term to apply here. Are there other drones that are arguably more autonomous, in the sense that they require even less structure in the environment? Sure. Are those same drones arguably less autonomous because they don't autonomously recharge? Probably. Corvus Robotics' perspective that the ability to run a drone autonomously for weeks at a time is a more important component of autonomy is perfectly valid considering their use case, but I think we're at the point where “full autonomy” at this level is becoming domain-specific enough to make direct comparisons difficult and maybe not all that useful.
Corvus has just recently come out of stealth, and they're currently working on pilot projects with a handful of Global 2000 companies. Continue reading
Posted in Human Robots
#439247 Drones and Sensors Could Spot Fires ...
The speed at which a wildfire can rip through an area and wreak havoc is nothing short of awe-inspiring and terrifying. Early detection of these events is critical for fire management efforts, whether that involves calling in firefighters or evacuating nearby communities.
Currently, early fire detection in remote areas is typically done by satellite—but this approach can be hindered by cloud cover. What’s more, even the most advanced satellite systems detect fires once the burning area reaches an average seize of 18.4 km2 (7.1 square miles).
To detect wildfires earlier on, some researchers are proposing a novel solution that harnesses a network of Internet of Things (IoT) sensors and a fleet of drones, or unmanned aerial vehicles (UAVs). The researchers tested their approach through simulations, described in a study published May 5 in IEEE Internet of Things Journal, finding that it can detect fires that are just 2.5 km2 (just under one square mile) in size with near perfect accuracy.
Their idea is timely, as climate change is driving an increase in wildfires around many regions of the world, as seen recently in California and Australia.
“In the last few years, the number, frequency, and severity of wildfires have increased dramatically worldwide, significantly impacting countries’ economies, ecosystems, and communities. Wildfire management presents a significant challenge in which early fire detection is key,” emphasizes Osama Bushnaq, a senior researcher at the Autonomous Robotics Research Center of the Technology Innovation Institute in Abu Dhabi, who was involved in the study.
The approach that Bushnaq and his colleagues are proposing involves a network of IoT sensors scattered throughout regions of concern, such as a national park or forests situated near communities. If a fire ignites, IoT devices deployed in the area will detect it and wait until a patrolling UAV is within transmission range to report their measurements. If a UAV receives multiple positive detections by the IoT devices, it will notify the nearby firefighting department that a wildfire has been verified.
The researchers evaluated a number of different UAVs and IoT sensors based on cost and features to determine the optimal combinations. Next, they tested their UAV-IoT approach through simulations, whereby 420 IoT sensors were deployed and 18 UAVs patrolled per square kilometer of simulated forest. The system could detect fires covering 2.5 km2 with greater than 99 percent accuracy. For smaller fires covering 0.5 km2 the approach yielded 69 percent accuracy.
These results suggest that, if an optimal number of UAVs and IoT devices are present, wildfires can be detected in much shorter time than with the satellite imaging. But Bushnaq acknowledges that this approach has its limitations. “UAV-IoT networks can only cover relatively smaller areas,” he explains. “Therefore, the UAV-IoT network would be particularly suitable for wildfire detection at high-risk regions.”
For these reasons, the researchers are proposing that UAV-IoT approach be used alongside satellite imaging, which can cover vast areas but with less wildfire detection speed and reliability.
Moving forward, the team plans to explore ways of further improving upon this approach, for example by optimizing the trajectory of the UAVs or addressing issues related to the battery life of UAVs.
Bushnaq envisions such UAV-IoT systems having much broader applications, too. “Although the system is designed for wildfire detection, it can be used for monitoring different forest parameters, such as wind speed, moisture content, or temperature estimation,” he says, noting that such a system could also be extended beyond the forest setting, for example by monitoring oil spills in bodies of water. Continue reading
Posted in Human Robots