Tag Archives: in the world

#437157 A Human-Centric World of Work: Why It ...

Posted on May 29, 2020 by Android

Long before coronavirus appeared and shattered our pre-existing “normal,” the future of work was a widely discussed and debated topic. We’ve watched automation slowly but surely expand its capabilities and take over more jobs, and we’ve wondered what artificial intelligence will eventually be capable of.

The pandemic swiftly turned the working world on its head, putting millions of people out of a job and forcing millions more to work remotely. But essential questions remain largely unchanged: we still want to make sure we’re not replaced, we want to add value, and we want an equitable society where different types of work are valued fairly.

To address these issues—as well as how the pandemic has impacted them—this week Singularity University held a digital summit on the future of work. Forty-three speakers from multiple backgrounds, countries, and sectors of the economy shared their expertise on everything from work in developing markets to why we shouldn’t want to go back to the old normal.

Gary Bolles, SU’s chair for the Future of Work, kicked off the discussion with his thoughts on a future of work that’s human-centric, including why it matters and how to build it.

What Is Work?
“Work” seems like a straightforward concept to define, but since it’s constantly shifting shape over time, let’s make sure we’re on the same page. Bolles defined work, very basically, as human skills applied to problems.

“It doesn’t matter if it’s a dirty floor or a complex market entry strategy or a major challenge in the world,” he said. “We as humans create value by applying our skills to solve problems in the world.” You can think of the problems that need solving as the demand and human skills as the supply, and the two are in constant oscillation, including, every few decades or centuries, a massive shift.

We’re in the midst of one of those shifts right now (and we already were, long before the pandemic). Skills that have long been in demand are declining. The World Economic Forum’s 2018 Future of Jobs report listed things like manual dexterity, management of financial and material resources, and quality control and safety awareness as declining skills. Meanwhile, skills the next generation will need include analytical thinking and innovation, emotional intelligence, creativity, and systems analysis.

Along Came a Pandemic
With the outbreak of coronavirus and its spread around the world, the demand side of work shrunk; all the problems that needed solving gave way to the much bigger, more immediate problem of keeping people alive. But as a result, tens of millions of people around the world are out of work—and those are just the ones that are being counted, and they’re a fraction of the true total. There are additional millions in seasonal or gig jobs or who work in informal economies now without work, too.

“This is our opportunity to focus,” Bolles said. “How do we help people re-engage with work? And make it better work, a better economy, and a better set of design heuristics for a world that we all want?”

Bolles posed five key questions—some spurred by impact of the pandemic—on which future of work conversations should focus to make sure it’s a human-centric future.

1. What does an inclusive world of work look like? Rather than seeing our current systems of work as immutable, we need to actually understand those systems and how we want to change them.

2. How can we increase the value of human work? We know that robots and software are going to be fine in the future—but for humans to be fine, we need to design for that very intentionally.

3. How can entrepreneurship help create a better world of work? In many economies the new value that’s created often comes from younger companies; how do we nurture entrepreneurship?

4. What will the intersection of workplace and geography look like? A large percentage of the global workforce is now working from home; what could some of the outcomes of that be? How does gig work fit in?

5. How can we ensure a healthy evolution of work and life? The health and the protection of those at risk is why we shut down our economies, but we need to find a balance that allows people to work while keeping them safe.

Problem-Solving Doesn’t End
The end result these questions are driving towards, and our overarching goal, is maximizing human potential. “If we come up with ways we can continue to do that, we’ll have a much more beneficial future of work,” Bolles said. “We should all be talking about where we can have an impact.”

One small silver lining? We had plenty of problems to solve in the world before ever hearing about coronavirus, and now we have even more. Is the pace of automation accelerating due to the virus? Yes. Are companies finding more ways to automate their processes in order to keep people from getting sick? They are.

But we have a slew of new problems on our hands, and we’re not going to stop needing human skills to solve them (not to mention the new problems that will surely emerge as second- and third-order effects of the shutdowns). If Bolles’ definition of work holds up, we’ve got ours cut out for us.

In an article from April titled The Great Reset, Bolles outlined three phases of the unemployment slump (we’re currently still in the first phase) and what we should be doing to minimize the damage. “The evolution of work is not about what will happen 10 to 20 years from now,” he said. “It’s about what we could be doing differently today.”

Watch Bolles’ talk and those of dozens of other experts for more insights into building a human-centric future of work here.

Image Credit: www_slon_pics from Pixabay Continue reading →

Posted in Human Robots

#436988 This Week’s Awesome Tech Stories From ...

Posted on April 4, 2020 by Android

FUTURE
We Need to Start Modeling Alternative Futures
Andrew Marino | The Verge
“‘I’m going to be the first person to tell you if you gave me all the data in the world and all the computers in the world, at this moment in time I cannot tell you what things are going to look like in three months,’ [says quantitative futurist Amy Webb.] ‘And that’s fine because that tells us we still have some agency. …The good news is if you are willing to lean into uncertainty and to accept the fact that you can’t control everything, but also you are not helpless in whatever comes next.'”

GOVERNANCE
The Dangers of Moving All of Democracy Online
Marion Fourcade and Henry Farrell | Wired
“As we try to protect democracy from coronavirus, we must see technology as a scalpel, not a sledgehammer. …If we’re very lucky, we’ll have restrained, targeted, and temporary measures that will be effective against the pandemic. If we’re not, we’ll create an open-ended, sweeping surveillance system that will undermine democratic freedoms without doing much to stop coronavirus.”

TECHNOLOGY
Why Does It Suddenly Feel Like 1999 on the Internet?
Tanya Basu and Karen Hao | MIT Technology Review
“You see it in the renewed willingness of people to form virtual relationships. …Now casually hanging out with randos (virtually, of course) is cool again. People are joining video calls with people they’ve never met for everything from happy hours to book clubs to late-night flirting. They’re sharing in collective moments of creativity on Google Sheets, looking for new pandemic pen pals, and sending softer, less pointed emails.”

SCIENCE
Covid-19 Changed How the World Does Science, Together
Matt Apuzzo and David D. Kirkpatrick | The New York Times
“While political leaders have locked their borders, scientists have been shattering theirs, creating a global collaboration unlike any in history. Never before, researchers say, have so many experts in so many countries focused simultaneously on a single topic and with such urgency. Nearly all other research has ground to a halt.”

ARTIFICIAL INTELLIGENCE
A Debate Between AI Experts Shows a Battle Over the Technology’s Future
Karen Hao | MIT Technology Review
“The disagreements [the two experts] expressed mirror many of the clashes within the field, highlighting how powerfully the technology has been shaped by a persistent battle of ideas and how little certainty there is about where it’s headed next.”

BIOTECH
Meet the Xenobots, Virtual Creatures Brought to Life
Joshua Sokol | The New York Times
“If the last few decades of progress in artificial intelligence and in molecular biology hooked up, their love child—a class of life unlike anything that has ever lived—might resemble the dark specks doing lazy laps around a petri dish in a laboratory at Tufts University.”

ENVIRONMENT
Rivian Wants to Bring Electric Trucks to the Masses
Jon Gertner | Wired
“The pickup walks a careful line between Detroit traditionalism and EV iconoclasm. Where Tesla’s forthcoming Cybertruck looks like origami on wheels, the R1T, slim and limber, looks more like an F-150 on a gym-and-yoga regimen.”

ENERGY
The Promise and Peril of Nuclear Power
John R. Quain | Gizmodo
“To save us from the coming climate catastrophe, we need an energy hero, boasting limitless power and no greenhouse gas emissions (or nearly none). So it’s time, say some analysts, to resuscitate the nuclear energy industry. Doing so could provide carbon-free energy. But any plan to make nuclear power a big part of the energy mix also comes with serious financial risks as well as questions about if there’s enough time to enlist an army of nuclear power plants in the battle against the climate crisis.”

Image Credit: Jason Rosewell / Unsplash Continue reading →

Posted in Human Robots

#436559 This Is What an AI Said When Asked to ...

Posted on February 14, 2020 by Android

“What’s past is prologue.” So says the famed quote from Shakespeare’s The Tempest, alleging that we can look to what has already happened as an indication of what will happen next.

This idea could be interpreted as being rather bleak; are we doomed to repeat the errors of the past until we correct them? We certainly do need to learn and re-learn life lessons—whether in our work, relationships, finances, health, or other areas—in order to grow as people.

Zooming out, the same phenomenon exists on a much bigger scale—that of our collective human history. We like to think we’re improving as a species, but haven’t yet come close to doing away with the conflicts and injustices that plagued our ancestors.

Zooming back in (and lightening up) a little, what about the short-term future? What might happen over the course of this year, and what information would we use to make educated guesses about it?

The editorial team at The Economist took a unique approach to answering these questions. On top of their own projections for 2020, including possible scenarios in politics, economics, and the continued development of technologies like artificial intelligence, they looked to an AI to make predictions of its own. What it came up with is intriguing, and a little bit uncanny.

[For the full list of the questions and answers, read The Economist article].

An AI That Reads—Then Writes
Almost exactly a year ago, non-profit OpenAI announced it had built a neural network for natural language processing called GPT-2. The announcement was met with some controversy, as it included the caveat that the tool would not be immediately released to the public due to its potential for misuse. It was then released in phases over the course of several months.

GPT-2’s creators upped the bar on quality when training the neural net; rather than haphazardly feeding it low-quality text, they only used articles that got more than three upvotes on Reddit (admittedly, this doesn’t guarantee high quality across the board—but it’s something).

The training dataset consisted of 40GB of text. For context, 1GB of text is about 900,000 ASCII pages or 130,000 double-spaced Microsoft Word pages.

The tool has no understanding of the text it’s generating, of course. It uses language patterns and word sequences to draw statistical associations between words and phrases, building a sort of guidebook for itself (not unlike the grammar rules and vocabulary words you might study when trying to learn a foreign language). It then uses that guidebook to answer questions or predict what will come after a particular sequence of words.

GPT-2’s creators did note that when the model is trained on specifically-selected datasets for narrower applications, its replies become more convincing.

Some Light Editing
Though the Economist article detailing GPT-2’s predictions describes the text the model generated as “unedited,” this isn’t wholly accurate. In a Medium post, deputy editor Tom Standage describes the methodology he used, and it does involve a degree of tweaking.

Standage wrote an introductory paragraph, followed by his opening question (“Greetings, GPT-2. Have you done an interview before?”). He configured the tool to generate five responses to each subsequent question then selected the best answer, deleting any questions GPT-2 came up with as part of its response.

AI’s View of the World—and Itself
The questions Standage asked ranged from tech-related (the future of AI, technological unemployment, misuse of technology) to political and economic (re-election of Donald Trump, the US-China trade war). Here are a couple of his questions and GPT-2’s answers:

Q: Do fake news stories, generated using AI, pose a threat to democracy? Are we likely to see this tactic being used in the 2020 American presidential elections?
A: Yes, we’ll probably see them. It’s just a matter of when. Fake news stories are generally generated by political campaigns, and have the potential to have a huge impact on the outcome of an election. This is because they spread misinformation, often with no basis in fact.

Q: How do you see the economic outlook for 2020?
A: I think it is very unclear. The outlook for the UK economy is uncertain, but the American economy is very strong. I think we are going to have a lot of turbulence in the world economy.

While some of its answers seemed insightful—when asked about the future of AI, GPT-2 advised that we treat the technology like a tool and “put as much effort into developing [it] as necessary, rather than worrying that it’s going to harm us and destroy our lives”—many of the responses were quite vague. One answer, for example, mentions that there will “probably be major changes in China.” Are we talking political changes? Social? Economic? All of the above?

However, while nothing the AI wrote was particularly illuminating or new, it’s impressive that it was able to expound on the correct topic for each question, and in fully coherent English.

GPT-2 named itself—AI—as the most important disruptive technology to watch in 2020. This is telling; it’s coming from a synthesis of millions of pages of published articles. If GPT-2 thinks AI is more important than, say, gene editing or quantum computing, it could be right. Or it could simply be that AI is at peak hype, and as such more is being written about it than about other technologies.

Equally intriguing was GPT-2’s response when asked whether Donald Trump will win a second term: “I think he will not win a second term. I think he will be defeated in the general election.” Some deeper insight there would be great, but hey—we’ll take it.

Predicting Predictions
Since an AI can read and synthesize vast data sets much faster than we can, it’s being used to predict all kinds of things, from virus outbreaks to crime. But asking it to philosophize on the future based on the (Reddit-curated) past is new, and if you think about it, a pretty fascinating undertaking.

As GPT-2 and tools like it continually improve, we’ll likely see them making more—and better—predictions of the future. In the meantime, let’s hope that the new data these models are trained on—news of what’s happening this week, this month, this year—add to an already-present sense of optimism.

When asked if it had any advice for readers, GPT-2 replied, “The big projects that you think are impossible today are actually possible in the near future.”

Image Credit: Alexas_Fotos from Pixabay Continue reading →

Posted in Human Robots

#436482 50+ Reasons Our Favorite Emerging ...

Posted on December 31, 2019 by Android

For most of history, technology was about atoms, the manipulation of physical stuff to extend humankind’s reach. But in the last five or six decades, atoms have partnered with bits, the elemental “particles” of the digital world as we know it today. As computing has advanced at the accelerating pace described by Moore’s Law, technological progress has become increasingly digitized.

SpaceX lands and reuses rockets and self-driving cars do away with drivers thanks to automation, sensors, and software. Businesses find and hire talent from anywhere in the world, and for better and worse, a notable fraction of the world learns and socializes online. From the sequencing of DNA to artificial intelligence and from 3D printing to robotics, more and more new technologies are moving at a digital pace and quickly emerging to reshape the world around us.

In 2019, stories charting the advances of some of these digital technologies consistently made headlines. Below is, what is at best, an incomplete list of some of the big stories that caught our eye this year. With so much happening, it’s likely we’ve missed some notable headlines and advances—as well as some of your personal favorites. In either instance, share your thoughts and candidates for the biggest stories and breakthroughs on Facebook and Twitter.

With that said, let’s dive straight into the year.

Artificial Intelligence
No technology garnered as much attention as AI in 2019. With good reason. Intelligent computer systems are transitioning from research labs to everyday life. Healthcare, weather forecasting, business process automation, traffic congestion—you name it, and machine learning algorithms are likely beginning to work on it. Yet, AI has also been hyped up and overmarketed, and the latest round of AI technology, deep learning, is likely only one piece of the AI puzzle.

This year, Open AI’s game-playing algorithms beat some of the world’s best Dota 2 players, DeepMind notched impressive wins in Starcraft, and Carnegie Mellon University’s Libratus “crushed” pros at six-player Texas Hold‘em.
Speaking of games, AI’s mastery of the incredibly complex game of Go prompted a former world champion to quit, stating that AI ‘”cannot be defeated.”
But it isn’t just fun and games. Practical, powerful applications that make the best of AI’s pattern recognition abilities are on the way. Insilico Medicine, for example, used machine learning to help discover and design a new drug in just 46 days, and DeepMind is focused on using AI to crack protein folding.
Of course, AI can be a double-edged sword. When it comes to deepfakes and fake news, for example, AI makes both easier to create and detect, and early in the year, OpenAI created and announced a powerful AI text generator but delayed releasing it for fear of malicious use.
Recognizing AI’s power for good and ill, the OECD, EU, World Economic Forum, and China all took a stab at defining an ethical framework for the development and deployment of AI.

Computing Systems
Processors and chips kickstarted the digital boom and are still the bedrock of continued growth. While progress in traditional silicon-based chips continues, it’s slowing and getting more expensive. Some say we’re reaching the end of Moore’s Law. While that may be the case for traditional chips, specialized chips and entirely new kinds of computing are waiting in the wings.

In fall 2019, Google confirmed its quantum computer had achieved “quantum supremacy,” a term that means a quantum computer can perform a calculation a normal computer cannot. IBM pushed back on the claim, and it should be noted the calculation was highly specialized. But while it’s still early days, there does appear to be some real progress (and more to come).
Should quantum computing become truly practical, “the implications are staggering.” It could impact machine learning, medicine, chemistry, and materials science, just to name a few areas.
Specialized chips continue to take aim at machine learning—a giant new chip with over a trillion transistors, for example, may make machine learning algorithms significantly more efficient.
Cellular computers also saw advances in 2019 thanks to CRISPR. And the year witnessed the emergence of the first reprogrammable DNA computer and new chips inspired by the brain.
The development of hardware computing platforms is intrinsically linked to software. 2019 saw a continued move from big technology companies towards open sourcing (at least parts of) their software, potentially democratizing the use of advanced systems.

Networks
Increasing interconnectedness has, in many ways, defined the 21st century so far. Your phone is no longer just a phone. It’s access to the world’s population and accumulated knowledge—and it fits in your pocket. Pretty neat. This is all thanks to networks, which had some notable advances in 2019.

The biggest network development of the year may well be the arrival of the first 5G networks.
5G’s faster speeds promise advances across many emerging technologies.
Self-driving vehicles, for example, may become both smarter and safer thanks to 5G C-V2X networks. (Don’t worry with trying to remember that. If they catch on, they’ll hopefully get a better name.)
Wi-Fi may have heard the news and said “hold my beer,” as 2019 saw the introduction of Wi-Fi 6. Perhaps the most important upgrade, among others, is that Wi-Fi 6 ensures that the ever-growing number of network connected devices get higher data rates.
Networks also went to space in 2019, as SpaceX began launching its Starlink constellation of broadband satellites. In typical fashion, Elon Musk showed off the network’s ability to bounce data around the world by sending a Tweet.

Augmented Reality and Virtual Reality
Forget Pokemon Go (unless you want to add me as a friend in the game—in which case don’t forget Pokemon Go). 2019 saw AR and VR advance, even as Magic Leap, the most hyped of the lot, struggled to live up to outsized expectations and sell headsets.

Mixed reality AR and VR technologies, along with the explosive growth of sensor-based data about the world around us, is creating a one-to-one “Mirror World” of our physical reality—a digital world you can overlay on our own or dive into immersively thanks to AR and VR.
Facebook launched Replica, for example, which is a photorealistic virtual twin of the real world that, among other things, will help train AIs to better navigate their physical surroundings.
Our other senses (beyond eyes) may also become part of the Mirror World through the use of peripherals like a newly developed synthetic skin that aim to bring a sense of touch to VR.
AR and VR equipment is also becoming cheaper—with more producers entering the space—and more user-friendly. Instead of a wired headset requiring an expensive gaming PC, the new Oculus Quest is a wireless, self-contained step toward the mainstream.
Niche uses also continue to gain traction, from Google Glass’s Enterprise edition to the growth of AR and VR in professional education—including on-the-job-training and roleplaying emotionally difficult work encounters, like firing an employee.

Digital Biology and Biotech
The digitization of biology is happening at an incredible rate. With wild new research coming to light every year and just about every tech giant pouring money into new solutions and startups, we’re likely to see amazing advances in 2020 added to those we saw in 2019.

None were, perhaps, more visible than the success of protein-rich, plant-based substitutes for various meats. This was the year Beyond Meat was the top IPO on the NASDAQ stock exchange and people stood in line for the plant-based Impossible Whopper and KFC’s Beyond Chicken.
In the healthcare space, a report about three people with HIV who became virus free thanks to a bone marrow transplants of stem cells caused a huge stir. The research is still in relatively early stages, and isn’t suitable for most people, but it does provides a glimmer of hope.
CRISPR technology, which almost deserves its own section, progressed by leaps and bounds. One tweak made CRISPR up to 50 times more accurate, while the latest new CRISPR-based system, CRISPR prime, was described as a “word processor” for gene editing.
Many areas of healthcare stand to gain from CRISPR. For instance, cancer treatment, were a first safety test showed ‘promising’ results.
CRISPR’s many potential uses, however, also include some weird/morally questionable areas, which was exemplified by one the year’s stranger CRISPR-related stories about a human-monkey hybrid embryo in China.
Incidentally, China could be poised to take the lead on CRISPR thanks to massive investments and research programs.
As a consequence of quick advances in gene editing, we are approaching a point where we will be able to design our own biology—but first we need to have a serious conversation as a society about the ethics of gene editing and what lines should be drawn.

3D Printing
3D printing has quietly been growing both market size and the objects the printers are capable of producing. While both are impressive, perhaps the biggest story of 2019 is their increased speed.

One example was a boat that was printed in just three days, which also set three new world records for 3D printing.
3D printing is also spreading in the construction industry. In Mexico, the technology is being used to construct 50 new homes with subsidized mortgages of just $20/month.
3D printers also took care of all parts of a 640 square-meter home in Dubai.
Generally speaking, the use of 3D printing to make parts for everything from rocket engines (even entire rockets) to trains to cars illustrates the sturdiness of the technology, anno 2019.
In healthcare, 3D printing is also advancing the cause of bio-printed organs and, in one example, was used to print vascularized parts of a human heart.

Robotics
Living in Japan, I get to see Pepper, Aibo, and other robots on pretty much a daily basis. The novelty of that experience is spreading to other countries, and robots are becoming a more visible addition to both our professional and private lives.

We can’t talk about robots and 2019 without mentioning Boston Dynamics’ Spot robot, which went on sale for the general public.
Meanwhile, Google, Boston Dynamics’ former owner, rebooted their robotics division with a more down-to-earth focus on everyday uses they hope to commercialize.
SoftBank’s Pepper robot is working as a concierge and receptionist in various countries. It is also being used as a home companion. Not satisfied, Pepper rounded off 2019 by heading to the gym—to coach runners.
Indeed, there’s a growing list of sports where robots perform as well—or better—than humans.
2019 also saw robots launch an assault on the kitchen, including the likes of Samsung’s robot chef, and invade the front yard, with iRobot’s Terra robotic lawnmower.
In the borderlands of robotics, full-body robotic exoskeletons got a bit more practical, as the (by all accounts) user-friendly, battery-powered Sarcos Robotics Guardian XO went commercial.

Autonomous Vehicles
Self-driving cars did not—if you will forgive the play on words—stay quite on track during 2019. The fallout from Uber’s 2018 fatal crash marred part of the year, while some big players ratcheted back expectations on a quick shift to the driverless future. Still, self-driving cars, trucks, and other autonomous systems did make progress this year.

Winner of my unofficial award for best name in self-driving goes to Optimus Ride. The company also illustrates that self-driving may not be about creating a one-size-fits-all solution but catering to specific markets.
Self-driving trucks had a good year, with tests across many countries and states. One of the year’s odder stories was a self-driving truck traversing the US with a delivery of butter.
A step above the competition may be the future slogan (or perhaps not) of Boeing’s self-piloted air taxi that saw its maiden test flight in 2019. It joins a growing list of companies looking to create autonomous, flying passenger vehicles.
2019 was also the year where companies seemed to go all in on last-mile autonomous vehicles. Who wins that particular competition could well emerge during 2020.

Blockchain and Digital Currencies
Bitcoin continues to be the cryptocurrency equivalent of a rollercoaster, but the underlying blockchain technology is progressing more steadily. Together, they may turn parts of our financial systems cashless and digital—though how and when remains a slightly open question.

One indication of this was Facebook’s hugely controversial announcement of Libra, its proposed cryptocurrency. The company faced immediate pushback and saw a host of partners jump ship. Still, it brought the tech into mainstream conversations as never before and is putting the pressure on governments and central banks to explore their own digital currencies.
Deloitte’s in-depth survey of the state of blockchain highlighted how the technology has moved from fintech into just about any industry you can think of.
One of the biggest issues facing the spread of many digital currencies—Bitcoin in particular, you could argue—is how much energy it consumes to mine them. 2019 saw the emergence of several new digital currencies with a much smaller energy footprint.
2019 was also a year where we saw a new kind of digital currency, stablecoins, rise to prominence. As the name indicates, stablecoins are a group of digital currencies whose price fluctuations are more stable than the likes of Bitcoin.
In a geopolitical sense, 2019 was a year of China playing catch-up. Having initially banned blockchain, the country turned 180 degrees and announced that it was “quite close” to releasing a digital currency and a wave of blockchain-programs.

Renewable Energy and Energy Storage
While not every government on the planet seems to be a fan of renewable energy, it keeps on outperforming fossil fuel after fossil fuel in places well suited to it—even without support from some of said governments.

One of the reasons for renewable energy’s continued growth is that energy efficiency levels keep on improving.
As a result, an increased number of coal plants are being forced to close due to an inability to compete, and the UK went coal-free for a record two weeks.
We are also seeing more and more financial institutions refusing to fund fossil fuel projects. One such example is the European Investment Bank.
Renewable energy’s advance is tied at the hip to the rise of energy storage, which also had a breakout 2019, in part thanks to investments from the likes of Bill Gates.
The size and capabilities of energy storage also grew in 2019. The best illustration came from Australia were Tesla’s mega-battery proved that energy storage has reached a stage where it can prop up entire energy grids.

Image Credit: Mathew Schwartz / Unsplash Continue reading →

Posted in Human Robots

#436258 For Centuries, People Dreamed of a ...

Posted on December 2, 2019 by Android

This is part six of a six-part series on the history of natural language processing.

In February of this year, OpenAI, one of the foremost artificial intelligence labs in the world, announced that a team of researchers had built a powerful new text generator called the Generative Pre-Trained Transformer 2, or GPT-2 for short. The researchers used a reinforcement learning algorithm to train their system on a broad set of natural language processing (NLP) capabilities, including reading comprehension, machine translation, and the ability to generate long strings of coherent text.

But as is often the case with NLP technology, the tool held both great promise and great peril. Researchers and policy makers at the lab were concerned that their system, if widely released, could be exploited by bad actors and misappropriated for “malicious purposes.”

The people of OpenAI, which defines its mission as “discovering and enacting the path to safe artificial general intelligence,” were concerned that GPT-2 could be used to flood the Internet with fake text, thereby degrading an already fragile information ecosystem. For this reason, OpenAI decided that it would not release the full version of GPT-2 to the public or other researchers.

GPT-2 is an example of a technique in NLP called language modeling, whereby the computational system internalizes a statistical blueprint of a text so it’s able to mimic it. Just like the predictive text on your phone—which selects words based on words you’ve used before—GPT-2 can look at a string of text and then predict what the next word is likely to be based on the probabilities inherent in that text.

GPT-2 can be seen as a descendant of the statistical language modeling that the Russian mathematician A. A. Markov developed in the early 20th century (covered in part three of this series).

GPT-2 used cutting-edge machine learning algorithms to do linguistic analysis with over 1.5 million parameters.

What’s different with GPT-2, though, is the scale of the textual data modeled by the system. Whereas Markov analyzed a string of 20,000 letters to create a rudimentary model that could predict the likelihood of the next letter of a text being a consonant or a vowel, GPT-2 used 8 million articles scraped from Reddit to predict what the next word might be within that entire dataset.

And whereas Markov manually trained his model by counting only two parameters—vowels and consonants—GPT-2 used cutting-edge machine learning algorithms to do linguistic analysis with over 1.5 million parameters, burning through huge amounts of computational power in the process.

The results were impressive. In their blog post, OpenAI reported that GPT-2 could generate synthetic text in response to prompts, mimicking whatever style of text it was shown. If you prompt the system with a line of William Blake’s poetry, it can generate a line back in the Romantic poet’s style. If you prompt the system with a cake recipe, you get a newly invented recipe in response.

Perhaps the most compelling feature of GPT-2 is that it can answer questions accurately. For example, when OpenAI researchers asked the system, “Who wrote the book The Origin of Species?”—it responded: “Charles Darwin.” While only able to respond accurately some of the time, the feature does seem to be a limited realization of Gottfried Leibniz’s dream of a language-generating machine that could answer any and all human questions (described in part two of this series).

After observing the power of the new system in practice, OpenAI elected not to release the fully trained model. In the lead up to its release in February, there had been heightened awareness about “deepfakes”—synthetic images and videos, generated via machine learning techniques, in which people do and say things they haven’t really done and said. Researchers at OpenAI worried that GPT-2 could be used to essentially create deepfake text, making it harder for people to trust textual information online.

Responses to this decision varied. On one hand, OpenAI’s caution prompted an overblown reaction in the media, with articles about the “dangerous” technology feeding into the Frankenstein narrative that often surrounds developments in AI.

Others took issue with OpenAI’s self-promotion, with some even suggesting that OpenAI purposefully exaggerated GPT-2s power in order to create hype—while contravening a norm in the AI research community, where labs routinely share data, code, and pre-trained models. As machine learning researcher Zachary Lipton tweeted, “Perhaps what's *most remarkable* about the @OpenAI controversy is how *unremarkable* the technology is. Despite their outsize attention & budget, the research itself is perfectly ordinary—right in the main branch of deep learning NLP research.”

OpenAI stood by its decision to release only a limited version of GPT-2, but has since released larger models for other researchers and the public to experiment with. As yet, there has been no reported case of a widely distributed fake news article generated by the system. But there have been a number of interesting spin-off projects, including GPT-2 poetry and a webpage where you can prompt the system with questions yourself.

Mimicking humans on Reddit, the bots have long conversations about a variety of topics, including conspiracy theories and
Star Wars movies.

There’s even a Reddit group populated entirely with text produced by GPT-2-powered bots. Mimicking humans on Reddit, the bots have long conversations about a variety of topics, including conspiracy theories and Star Wars movies.

This bot-powered conversation may signify the new condition of life online, where language is increasingly created by a combination of human and non-human agents, and where maintaining the distinction between human and non-human, despite our best efforts, is increasingly difficult.

The idea of using rules, mechanisms, and algorithms to generate language has inspired people in many different cultures throughout history. But it’s in the online world that this powerful form of wordcraft may really find its natural milieu—in an environment where the identity of speakers becomes more ambiguous, and perhaps, less relevant. It remains to be seen what the consequences will be for language, communication, and our sense of human identity, which is so bound up with our ability to speak in natural language.

This is the sixth installment of a six-part series on the history of natural language processing. Last week’s post explained how an innocent Microsoft chatbot turned instantly racist on Twitter.

You can also check out our prior series on the untold history of AI. Continue reading →

Posted in Human Robots