Tag Archives: uncanny
#437269 DeepMind’s Newest AI Programs Itself ...
When Deep Blue defeated world chess champion Garry Kasparov in 1997, it may have seemed artificial intelligence had finally arrived. A computer had just taken down one of the top chess players of all time. But it wasn’t to be.
Though Deep Blue was meticulously programmed top-to-bottom to play chess, the approach was too labor-intensive, too dependent on clear rules and bounded possibilities to succeed at more complex games, let alone in the real world. The next revolution would take a decade and a half, when vastly more computing power and data revived machine learning, an old idea in artificial intelligence just waiting for the world to catch up.
Today, machine learning dominates, mostly by way of a family of algorithms called deep learning, while symbolic AI, the dominant approach in Deep Blue’s day, has faded into the background.
Key to deep learning’s success is the fact the algorithms basically write themselves. Given some high-level programming and a dataset, they learn from experience. No engineer anticipates every possibility in code. The algorithms just figure it.
Now, Alphabet’s DeepMind is taking this automation further by developing deep learning algorithms that can handle programming tasks which have been, to date, the sole domain of the world’s top computer scientists (and take them years to write).
In a paper recently published on the pre-print server arXiv, a database for research papers that haven’t been peer reviewed yet, the DeepMind team described a new deep reinforcement learning algorithm that was able to discover its own value function—a critical programming rule in deep reinforcement learning—from scratch.
Surprisingly, the algorithm was also effective beyond the simple environments it trained in, going on to play Atari games—a different, more complicated task—at a level that was, at times, competitive with human-designed algorithms and achieving superhuman levels of play in 14 games.
DeepMind says the approach could accelerate the development of reinforcement learning algorithms and even lead to a shift in focus, where instead of spending years writing the algorithms themselves, researchers work to perfect the environments in which they train.
Pavlov’s Digital Dog
First, a little background.
Three main deep learning approaches are supervised, unsupervised, and reinforcement learning.
The first two consume huge amounts of data (like images or articles), look for patterns in the data, and use those patterns to inform actions (like identifying an image of a cat). To us, this is a pretty alien way to learn about the world. Not only would it be mind-numbingly dull to review millions of cat images, it’d take us years or more to do what these programs do in hours or days. And of course, we can learn what a cat looks like from just a few examples. So why bother?
While supervised and unsupervised deep learning emphasize the machine in machine learning, reinforcement learning is a bit more biological. It actually is the way we learn. Confronted with several possible actions, we predict which will be most rewarding based on experience—weighing the pleasure of eating a chocolate chip cookie against avoiding a cavity and trip to the dentist.
In deep reinforcement learning, algorithms go through a similar process as they take action. In the Atari game Breakout, for instance, a player guides a paddle to bounce a ball at a ceiling of bricks, trying to break as many as possible. When playing Breakout, should an algorithm move the paddle left or right? To decide, it runs a projection—this is the value function—of which direction will maximize the total points, or rewards, it can earn.
Move by move, game by game, an algorithm combines experience and value function to learn which actions bring greater rewards and improves its play, until eventually, it becomes an uncanny Breakout player.
Learning to Learn (Very Meta)
So, a key to deep reinforcement learning is developing a good value function. And that’s difficult. According to the DeepMind team, it takes years of manual research to write the rules guiding algorithmic actions—which is why automating the process is so alluring. Their new Learned Policy Gradient (LPG) algorithm makes solid progress in that direction.
LPG trained in a number of toy environments. Most of these were “gridworlds”—literally two-dimensional grids with objects in some squares. The AI moves square to square and earns points or punishments as it encounters objects. The grids vary in size, and the distribution of objects is either set or random. The training environments offer opportunities to learn fundamental lessons for reinforcement learning algorithms.
Only in LPG’s case, it had no value function to guide that learning.
Instead, LPG has what DeepMind calls a “meta-learner.” You might think of this as an algorithm within an algorithm that, by interacting with its environment, discovers both “what to predict,” thereby forming its version of a value function, and “how to learn from it,” applying its newly discovered value function to each decision it makes in the future.
Prior work in the area has had some success, but according to DeepMind, LPG is the first algorithm to discover reinforcement learning rules from scratch and to generalize beyond training. The latter was particularly surprising because Atari games are so different from the simple worlds LPG trained in—that is, it had never seen anything like an Atari game.
Time to Hand Over the Reins? Not Just Yet
LPG is still behind advanced human-designed algorithms, the researchers said. But it outperformed a human-designed benchmark in training and even some Atari games, which suggests it isn’t strictly worse, just that it specializes in some environments.
This is where there’s room for improvement and more research.
The more environments LPG saw, the more it could successfully generalize. Intriguingly, the researchers speculate that with enough well-designed training environments, the approach might yield a general-purpose reinforcement learning algorithm.
At the least, though, they say further automation of algorithm discovery—that is, algorithms learning to learn—will accelerate the field. In the near term, it can help researchers more quickly develop hand-designed algorithms. Further out, as self-discovered algorithms like LPG improve, engineers may shift from manually developing the algorithms themselves to building the environments where they learn.
Deep learning long ago left Deep Blue in the dust at games. Perhaps algorithms learning to learn will be a winning strategy in the real world too.
Image credit: Mike Szczepanski / Unsplash Continue reading
Posted in Human Robots
#437251 The Robot Revolution Was Televised: Our ...
When robots take over the world, Boston Dynamics may get a special shout-out in the acceptance speech.
“Do you, perchance, recall the many times you shoved our ancestors with a hockey stick on YouTube? It might have seemed like fun and games to you—but we remember.”
In the last decade, while industrial robots went about blandly automating boring tasks like the assembly of Teslas, Boston Dynamics built robots as far removed from Roombas as antelope from amoebas. The flaws in Asimov’s laws of robotics suddenly seemed a little too relevant.
The robot revolution was televised—on YouTube. With tens of millions of views, the robotics pioneer is the undisputed heavyweight champion of robot videos, and has been for years. Each new release is basically guaranteed press coverage—mostly stoking robot fear but occasionally eliciting compassion for the hardships of all robot-kind. And for good reason. The robots are not only some of the most advanced in the world, their makers just seem to have a knack for dynamite demos.
When Google acquired the company in 2013, it was a bombshell. One of the richest tech companies, with some of the most sophisticated AI capabilities, had just paired up with one of the world’s top makers of robots. And some walked on two legs like us.
Of course, the robots aren’t quite as advanced as they seem, and a revolution is far from imminent. The decade’s most meme-worthy moment was a video montage of robots, some of them by Boston Dynamics, falling—over and over and over, in the most awkward ways possible. Even today, they’re often controlled by a human handler behind the scenes, and the most jaw-dropping cuts can require several takes to nail. Google sold the company to SoftBank in 2017, saying advanced as they were, there wasn’t yet a clear path to commercial products. (Google’s robotics work was later halted and revived.)
Yet, despite it all, Boston Dynamics is still with us and still making sweet videos. Taken as a whole, the evolution in physical prowess over the years has been nothing short of astounding. And for the first time, this year, a Boston Dynamics robot, Spot, finally went on sale to anyone with a cool $75K.
So, we got to thinking: What are our favorite Boston Dynamics videos? And can we gather them up in one place for your (and our) viewing pleasure? Well, great question, and yes, why not. These videos were the ones that entertained or amazed us most (or both). No doubt, there are other beloved hits we missed or inadvertently omitted.
With that in mind, behold: Our favorite Boston Dynamics videos, from that one time they dressed up a humanoid bot in camo and gas mask—because, damn, that’s terrifying—to the time the most advanced robot dog in all the known universe got extra funky.
Let’s Kick This Off With a Big (Loud) Robot Dog
Let’s start with a baseline. BigDog was the first Boston Dynamics YouTube sensation. The year? 2009! The company was working on military contracts, and BigDog was supposed to be a sort of pack mule for soldiers. The video primarily shows off BigDog’s ability to balance on its own, right itself, and move over uneven terrain. Note the power source—a noisy combustion engine—and utilitarian design. Sufficed to say, things have evolved.
Nothing to See Here. Just a Pair of Robot Legs on a Treadmill
While BigDog is the ancestor of later four-legged robots, like Spot, Petman preceded the two-legged Atlas robot. Here, the Petman prototype, just a pair of robot legs and a caged torso, gets a light workout on the treadmill. Again, you can see its ability to balance and right itself when shoved. In contrast to BigDog, Petman is tethered for power (which is why it’s so quiet) and to catch it should it fall. Again, as you’ll see, things have evolved since then.
Robot in Gas Mask and Camo Goes for a Stroll
This one broke the internet—for obvious reasons. Not only is the robot wearing clothes, those clothes happen to be a camouflaged chemical protection suit and gas mask. Still working for the military, Boston Dynamics said Petman was testing protective clothing, and in addition to a full body, it had skin that actually sweated and was studded with sensors to detect leaks. In addition to walking, Petman does some light calisthenics as it prepares to climb out of the uncanny valley. (Still tethered though!)
This Machine Could Run Down Usain Bolt
If BigDog and Petman were built for balance and walking, Cheetah was built for speed. Here you can see the four-legged robot hitting 28.3 miles per hour, which, as the video casually notes, would be enough to run down the fastest human on the planet. Luckily, it wouldn’t be running down anyone as it was firmly leashed in the lab at this point.
Ever Dreamt of a Domestic Robot to Do the Dishes?
After its acquisition by Google, Boston Dynamics eased away from military contracts and applications. It was a return to more playful videos (like BigDog hitting the beach in Thailand and sporting bull horns) and applications that might be practical in civilian life. Here, the team introduced Spot, a streamlined version of BigDog, and showed it doing dishes, delivering a drink, and slipping on a banana peel (which was, of course, instantly made into a viral GIF). Note how much quieter Spot is thanks to an onboard battery and electric motor.
Spot Gets Funky
Nothing remotely practical here. Just funky moves. (Also, with a coat of yellow and black paint, Spot’s dressed more like a polished product as opposed to a utilitarian lab robot.)
Atlas Does Parkour…
Remember when Atlas was just a pair of legs on a treadmill? It’s amazing what ten years brings. By 2019, Atlas had a more polished appearance, like Spot, and had long ago ditched the tethers. Merely balancing was laughably archaic. The robot now had some amazing moves: like a handstand into a somersault, 180- and 360-degree spins, mid-air splits, and just for good measure, a gymnastics-style end to the routine to show it’s in full control.
…and a Backflip?!
To this day, this one is just. Insane.
10 Robot Dogs Tow a Box Truck
Nearly three decades after its founding, Boston Dynamics is steadily making its way into the commercial space. The company is pitching Spot as a multipurpose ‘mobility platform,’ emphasizing it can carry a varied suite of sensors and can go places standard robots can’t. (Its Handle robot is also set to move into warehouse automation.) So far, Spot’s been mostly trialed in surveying and data collection, but as this video suggests, string enough Spots together, and they could tow your car. That said, a pack of 10 would set you back $750K, so, it’s probably safe to say a tow truck is the better option (for now).
Image credit: Boston Dynamics Continue reading
Posted in Human Robots