Tag Archives: based

#437103 How to Make Sense of Uncertainty in a ...

Posted on May 1, 2020 by Android

As the internet churns with information about Covid-19, about the virus that causes the disease, and about what we’re supposed to do to fight it, it can be difficult to see the forest for the trees. What can we realistically expect for the rest of 2020? And how do we even know what’s realistic?

Today, humanity’s primary, ideal goal is to eliminate the virus, SARS-CoV-2, and Covid-19. Our second-choice goal is to control virus transmission. Either way, we have three big aims: to save lives, to return to public life, and to keep the economy functioning.

To hit our second-choice goal—and maybe even our primary goal—countries are pursuing five major public health strategies. Note that many of these advances cross-fertilize: for example, advances in virus testing and antibody testing will drive data-based prevention efforts.

Five major public health strategies are underway to bring Covid-19 under control and to contain the spread of SARS-CoV-2.
These strategies arise from things we can control based on the things that we know at any given moment. But what about the things we can’t control and don’t yet know?

The biology of the virus and how it interacts with our bodies is what it is, so we should seek to understand it as thoroughly as possible. How long any immunity gained from prior infection lasts—and indeed whether people develop meaningful immunity at all after infection—are open questions urgently in need of greater clarity. Similarly, right now it’s important to focus on understanding rather than making assumptions about environmental factors like seasonality.

But the biggest question on everyone’s lips is, “When?” When will we see therapeutic progress against Covid-19? And when will life get “back to normal”? There are lots of models out there on the internet; which of those models are right? The simple answer is “none of them.” That’s right—it’s almost certain that every model you’ve seen is wrong in at least one detail, if not all of them. But modeling is meant to be a tool for deeper thinking, a way to run mental (and computational) experiments before—and while—taking action. As George E. P. Box famously wrote in 1976, “All models are wrong, but some are useful.”

Here, we’re seeking useful insights, as opposed to exact predictions, which is why we’re pulling back from quantitative details to get at the mindsets that will support agency and hope. To that end, I’ve been putting together timelines that I believe will yield useful expectations for the next year or two—and asking how optimistic I need to be in order to believe a particular timeline.

For a moderately optimistic scenario to be relevant, breakthroughs in science and technology come at paces expected based on previous efforts and assumptions that turn out to be basically correct; accessibility of those breakthroughs increases at a reasonable pace; regulation achieves its desired effects, without major surprises; and compliance with regulations is reasonably high.

In contrast, if I’m being highly optimistic, breakthroughs in science and technology and their accessibility come more quickly than they ever have before; regulation is evidence-based and successful in the first try or two; and compliance with those regulations is high and uniform. If I’m feeling not-so-optimistic, then I anticipate serious setbacks to breakthroughs and accessibility (with the overturning of many important assumptions), repeated failure of regulations to achieve their desired outcomes, and low compliance with those regulations.

The following scenarios outline the things that need to happen in the fight against Covid-19, when I expect to see them, and how confident I feel in those expectations. They focus on North America and Europe because there are data missing about China’s 2019 outbreak and other regions are still early in their outbreaks. Perhaps the most important thing to keep in mind throughout: We know more today than we did yesterday, but we still have much to learn. New knowledge derived from greater study and debate will almost certainly inspire ongoing course corrections.

As you dive into the scenarios below, practice these three mindset shifts. First, defeating Covid-19 will be a marathon, not a sprint. We shouldn’t expect life to look like 2019 for the next year or two—if ever. As Ed Yong wrote recently in The Atlantic, “There won’t be an obvious moment when everything is under control and regular life can safely resume.” Second, remember that you have important things to do for at least a year. And third, we are all in this together. There is no “us” and “them.” We must all be alert, responsive, generous, and strong throughout 2020 and 2021—and willing to throw away our assumptions when scientific evidence invalidates them.

The Middle Way: Moderate Optimism
Let’s start with the case in which I have the most confidence: moderate optimism.

This timeline considers milestones through late 2021, the earliest that I believe vaccines will become available. The “normal” timeline for developing a vaccine for diseases like seasonal flu is 18 months, which leads to my projection that we could potentially have vaccines as soon as 18 months from the first quarter of 2020. While Melinda Gates agrees with that projection, others (including AI) believe that 3 to 5 years is far more realistic, based on past vaccine development and the need to test safety and efficacy in humans. However, repurposing existing vaccines against other diseases—or piggybacking off clever synthetic platforms—could lead to vaccines being available sooner. I tried to balance these considerations for this moderately optimistic scenario. Either way, deploying vaccines at the end of 2021 is probably much later than you may have been led to believe by the hype engine. Again, if you take away only one message from this article, remember that the fight against Covid-19 is a marathon, not a sprint.

Here, I’ve visualized a moderately optimistic scenario as a baseline. Think of these timelines as living guides, as opposed to exact predictions. There are still many unknowns. More or less optimistic views (see below) and new information could shift these timelines forward or back and change the details of the strategies.
Based on current data, I expect that the first wave of Covid-19 cases (where we are now) will continue to subside in many areas, leading governments to ease restrictions in an effort to get people back to work. We’re already seeing movement in that direction, with a variety of benchmarks and changes at state and country levels around the world. But depending on the details of the changes, easing restrictions will probably cause a second wave of sickness (see Germany and Singapore), which should lead governments to reimpose at least some restrictions.

In tandem, therapeutic efforts will be transitioning from emergency treatments to treatments that have been approved based on safety and efficacy data in clinical trials. In a moderately optimistic scenario, assuming clinical trials currently underway yield at least a few positive results, this shift to mostly approved therapies could happen as early as the third or fourth quarter of this year and continue from there. One approval that should come rather quickly is for plasma therapies, in which the blood from people who have recovered from Covid-19 is used as a source of antibodies for people who are currently sick.

Companies around the world are working on both viral and antibody testing, focusing on speed, accuracy, reliability, and wide accessibility. While these tests are currently being run in hospitals and research laboratories, at-home testing is a critical component of the mass testing we’ll need to keep viral spread in check. These are needed to minimize the impact of asymptomatic cases, test the assumption that infection yields resistance to subsequent infection (and whether it lasts), and construct potential immunity passports if this assumption holds. Testing is also needed for contact tracing efforts to prevent further spread and get people back to public life. Finally, it’s crucial to our fundamental understanding of the biology of SARS-CoV-2 and Covid-19.

We need tests that are very reliable, both in the clinic and at home. So, don’t go buying any at-home test kits just yet, even if you find them online. Wait for reliable test kits and deeper understanding of how a test result translates to everyday realities. If we’re moderately optimistic, in-clinic testing will rapidly expand this quarter and/or next, with the possibility of broadly available, high-quality at-home sampling (and perhaps even analysis) thereafter.

Note that testing is not likely to be a “one-and-done” endeavor, as a person’s infection and immunity status change over time. Expect to be testing yourself—and your family—often as we move later into 2020.

Testing data are also going to inform distancing requirements at the country and local levels. In this scenario, restrictions—at some level of stringency—could persist at least through the end of 2020, as most countries are way behind the curve on testing (Iceland is an informative exception). Governments will likely continue to ask citizens to work from home if at all possible; to wear masks or face coverings in public; to employ heightened hygiene and social distancing in workplaces; and to restrict travel and social gatherings. So while it’s likely we’ll be eating in local restaurants again in 2020 in this scenario, at least for a little while, it’s not likely we’ll be heading to big concerts any time soon.

The Extremes: High and Low Optimism
How would high and low levels of optimism change our moderately optimistic timeline? The milestones are the same, but the time required to achieve them is shorter or longer, respectively. Quantifying these shifts is less important than acknowledging and incorporating a range of possibilities into our view. It pays to pay attention to our bias. Here are a few examples of reasonable possibilities that could shift the moderately optimistic timeline.

When vaccines become available
Vaccine repurposing could shorten the time for vaccines to become available; today, many vaccine candidates are in various stages of testing. On the other hand, difficulties in manufacture and distribution, or faster-than-expected mutation of SARS-CoV-2, could slow vaccine development. Given what we know now, I am not strongly concerned about either of these possibilities—drug companies are rapidly expanding their capabilities, and viral mutation isn’t an urgent concern at this time based on sequencing data—but they could happen.

At first, governments will likely supply vaccines to essential workers such as healthcare workers, but it is essential that vaccines become widely available around the world as quickly and as safely as possible. Overall, I suggest a dose of skepticism when reading highly optimistic claims about a vaccine (or multiple vaccines) being available in 2020. Remember, a vaccine is a knockout punch, not a first line of defense for an outbreak.

When testing hits its stride
While I am confident that testing is a critical component of our response to Covid-19, reliability is incredibly important to testing for SARS-CoV-2 and for immunity to the disease, particularly at home. For an individual, a false negative (being told you don’t have antibodies when you really do) could be just as bad as a false positive (being told you do have antibodies when you really don’t). Those errors are compounded when governments are trying to make evidence-based policies for social and physical distancing.

If you’re highly optimistic, high-quality testing will ramp up quickly as companies and scientists innovate rapidly by cleverly combining multiple test modalities, digital signals, and cutting-edge tech like CRISPR. Pop-up testing labs could also take some pressure off hospitals and clinics.

If things don’t go well, reliability issues could hinder testing, manufacturing bottlenecks could limit availability, and both could hamstring efforts to control spread and ease restrictions. And if it turns out that immunity to Covid-19 isn’t working the way we assumed, then we must revisit our assumptions about our path(s) back to public life, as well as our vaccine-development strategies.

How quickly safe and effective treatments appear
Drug development is known to be long, costly, and fraught with failure. It’s not uncommon to see hope in a drug spike early only to be dashed later on down the road. With that in mind, the number of treatments currently under investigation is astonishing, as is the speed through which they’re proceeding through testing. Breakthroughs in a therapeutic area—for example in treating the seriously ill or in reducing viral spread after an infection takes hold—could motivate changes in the focus of distancing regulations.

While speed will save lives, we cannot overlook the importance of knowing a treatment’s efficacy (does it work against Covid-19?) and safety (does it make you sick in a different, or worse, way?). Repurposing drugs that have already been tested for other diseases is speeding innovation here, as is artificial intelligence.

Remarkable collaborations among governments and companies, large and small, are driving innovation in therapeutics and devices such as ventilators for treating the sick.

Whether government policies are effective and responsive
Those of us who have experienced lockdown are eager for it to be over. Businesses, economists, and governments are also eager to relieve the terrible pressure that is being exerted on the global economy. However, lifting restrictions will almost certainly lead to a resurgence in sickness.

Here, the future is hard to model because there are many, many factors at play, and at play differently in different places—including the extent to which individuals actually comply with regulations.

Reliable testing—both in the clinic and at home—is crucial to designing and implementing restrictions, monitoring their effectiveness, and updating them; delays in reliable testing could seriously hamper this design cycle. Lack of trust in governments and/or companies could also suppress uptake. That said, systems are already in place for contact tracing in East Asia. Other governments could learn important lessons, but must also earn—and keep—their citizens’ trust.

Expect to see restrictions descend and then lift in response to changes in the number of Covid-19 cases and in the effectiveness of our prevention strategies. Also expect country-specific and perhaps even area-specific responses that differ from each other. The benefit of this approach? Governments around the world are running perhaps hundreds of real-time experiments and design cycles in balancing health and the economy, and we can learn from the results.

A Way Out
As Jeremy Farrar, head of the Wellcome Trust, told Science magazine, “Science is the exit strategy.” Some of our greatest technological assistance is coming from artificial intelligence, digital tools for collaboration, and advances in biotechnology.

Our exit strategy also needs to include empathy and future visioning—because in the midst of this crisis, we are breaking ground for a new, post-Covid future.

What do we want that future to look like? How will the hard choices we make now about data ethics impact the future of surveillance? Will we continue to embrace inclusiveness and mass collaboration? Perhaps most importantly, will we lay the foundation for successfully confronting future challenges? Whether we’re thinking about the next pandemic (and there will be others) or the cascade of catastrophes that climate change is bringing ever closer—it’s important to remember that we all have the power to become agents of that change.

Special thanks to Ola Kowalewski and Jason Dorrier for significant conversations.

Image Credit: Drew Beamer / Unsplash Continue reading →

Posted in Human Robots

#436977 The Top 100 AI Startups Out There Now, ...

Posted on March 30, 2020 by Android

New drug therapies for a range of chronic diseases. Defenses against various cyber attacks. Technologies to make cities work smarter. Weather and wildfire forecasts that boost safety and reduce risk. And commercial efforts to monetize so-called deepfakes.

What do all these disparate efforts have in common? They’re some of the solutions that the world’s most promising artificial intelligence startups are pursuing.

Data research firm CB Insights released its much-anticipated fourth annual list of the top 100 AI startups earlier this month. The New York-based company has become one of the go-to sources for emerging technology trends, especially in the startup scene.

About 10 years ago, it developed its own algorithm to assess the health of private companies using publicly-available information and non-traditional signals (think social media sentiment, for example) thanks to more than $1 million in grants from the National Science Foundation.

It uses that algorithm-generated data from what it calls a company’s Mosaic score—pulling together information on market trends, money, and momentum—along with other details ranging from patent activity to the latest news analysis to identify the best of the best.

“Our final list of companies is a mix of startups at various stages of R&D and product commercialization,” said Deepashri Varadharajanis, a lead analyst at CB Insights, during a recent presentation on the most prominent trends among the 2020 AI 100 startups.

About 10 companies on the list are among the world’s most valuable AI startups. For instance, there’s San Francisco-based Faire, which has raised at least $266 million since it was founded just three years ago. The company offers a wholesale marketplace that uses machine learning to match local retailers with goods that are predicted to sell well in their specific location.

Image courtesy of CB Insights
Funding for AI in Healthcare
Another startup valued at more than $1 billion, referred to as a unicorn in venture capital speak, is Butterfly Network, a company on the East Coast that has figured out a way to turn a smartphone phone into an ultrasound machine. Backed by $350 million in private investments, Butterfly Network uses AI to power the platform’s diagnostics. A more modestly funded San Francisco startup called Eko is doing something similar for stethoscopes.

In fact, there are more than a dozen AI healthcare startups on this year’s AI 100 list, representing the most companies of any industry on the list. In total, investors poured about $4 billion into AI healthcare startups last year, according to CB Insights, out of a record $26.6 billion raised by all private AI companies in 2019. Since 2014, more than 4,300 AI startups in 80 countries have raised about $83 billion.

One of the most intensive areas remains drug discovery, where companies unleash algorithms to screen potential drug candidates at an unprecedented speed and breadth that was impossible just a few years ago. It has led to the discovery of a new antibiotic to fight superbugs. There’s even a chance AI could help fight the coronavirus pandemic.

There are several AI drug discovery startups among the AI 100: San Francisco-based Atomwise claims its deep convolutional neural network, AtomNet, screens more than 100 million compounds each day. Cyclica is an AI drug discovery company in Toronto that just announced it would apply its platform to identify and develop novel cannabinoid-inspired drugs for neuropsychiatric conditions such as bipolar disorder and anxiety.

And then there’s OWKIN out of New York City, a startup that uses a type of machine learning called federated learning. Backed by Google, the company’s AI platform helps train algorithms without sharing the necessary patient data required to provide the sort of valuable insights researchers need for designing new drugs or even selecting the right populations for clinical trials.

Keeping Cyber Networks Healthy
Privacy and data security are the focus of a number of AI cybersecurity startups, as hackers attempt to leverage artificial intelligence to launch sophisticated attacks while also trying to fool the AI-powered systems rapidly coming online.

“I think this is an interesting field because it’s a bit of a cat and mouse game,” noted Varadharajanis. “As your cyber defenses get smarter, your cyber attacks get even smarter, and so it’s a constant game of who’s going to match the other in terms of tech capabilities.”

Few AI cybersecurity startups match Silicon Valley-based SentinelOne in terms of private capital. The company has raised more than $400 million, with a valuation of $1.1 billion following a $200 million Series E earlier this year. The company’s platform automates what’s called endpoint security, referring to laptops, phones, and other devices at the “end” of a centralized network.

Fellow AI 100 cybersecurity companies include Blue Hexagon, which protects the “edge” of the network against malware, and Abnormal Security, which stops targeted email attacks, both out of San Francisco. Just down the coast in Los Angeles is Obsidian Security, a startup offering cybersecurity for cloud services.

Deepfakes Get a Friendly Makeover
Deepfakes of videos and other types of AI-manipulated media where faces or voices are synthesized in order to fool viewers or listeners has been a different type of ongoing cybersecurity risk. However, some firms are swapping malicious intent for benign marketing and entertainment purposes.

Now anyone can be a supermodel thanks to Superpersonal, a London-based AI startup that has figured out a way to seamlessly swap a user’s face onto a fashionista modeling the latest threads on the catwalk. The most obvious use case is for shoppers to see how they will look in a particular outfit before taking the plunge on a plunging neckline.

Another British company called Synthesia helps users create videos where a talking head will deliver a customized speech or even talk in a different language. The startup’s claim to fame was releasing a campaign video for the NGO Malaria Must Die showing soccer star David Becham speak in nine different languages.

There’s also a Seattle-based company, Wellsaid Labs, which uses AI to produce voice-over narration where users can choose from a library of digital voices with human pitch, emphasis, and intonation. Because every narrator sounds just a little bit smarter with a British accent.

AI Helps Make Smart Cities Smarter
Speaking of smarter: A handful of AI 100 startups are helping create the smart city of the future, where a digital web of sensors, devices, and cloud-based analytics ensure that nobody is ever stuck in traffic again or without an umbrella at the wrong time. At least that’s the dream.

A couple of them are directly connected to Google subsidiary Sidewalk Labs, which focuses on tech solutions to improve urban design. A company called Replica was spun out just last year. It’s sort of SimCity for urban planning. The San Francisco startup uses location data from mobile phones to understand how people behave and travel throughout a typical day in the city. Those insights can then help city governments, for example, make better decisions about infrastructure development.

Denver-area startup AMP Robotics gets into the nitty gritty details of recycling by training robots on how to recycle trash, since humans have largely failed to do the job. The U.S. Environmental Protection Agency estimates that only about 30 percent of waste is recycled.

Some people might complain that weather forecasters don’t even do that well when trying to predict the weather. An Israeli AI startup, ClimaCell, claims it can forecast rain block by block. While the company taps the usual satellite and ground-based sources to create weather models, it has developed algorithms to analyze how precipitation and other conditions affect signals in cellular networks. By analyzing changes in microwave signals between cellular towers, the platform can predict the type and intensity of the precipitation down to street level.

And those are just some of the highlights of what some of the world’s most promising AI startups are doing.

“You have companies optimizing mining operations, warehouse logistics, insurance, workflows, and even working on bringing AI solutions to designing printed circuit boards,” Varadharajanis said. “So a lot of creative ways in which companies are applying AI to solve different issues in different industries.”

Image Credit: Butterfly Network Continue reading →

Posted in Human Robots

#436911 Scientists Linked Artificial and ...

Posted on March 11, 2020 by Android

Scientists have linked up two silicon-based artificial neurons with a biological one across multiple countries into a fully-functional network. Using standard internet protocols, they established a chain of communication whereby an artificial neuron controls a living, biological one, and passes on the info to another artificial one.

Whoa.

We’ve talked plenty about brain-computer interfaces and novel computer chips that resemble the brain. We’ve covered how those “neuromorphic” chips could link up into tremendously powerful computing entities, using engineered communication nodes called artificial synapses.

As Moore’s law is dying, we even said that neuromorphic computing is one path towards the future of extremely powerful, low energy consumption artificial neural network-based computing—in hardware—that could in theory better link up with the brain. Because the chips “speak” the brain’s language, in theory they could become neuroprosthesis hubs far more advanced and “natural” than anything currently possible.

This month, an international team put all of those ingredients together, turning theory into reality.

The three labs, scattered across Padova, Italy, Zurich, Switzerland, and Southampton, England, collaborated to create a fully self-controlled, hybrid artificial-biological neural network that communicated using biological principles, but over the internet.

The three-neuron network, linked through artificial synapses that emulate the real thing, was able to reproduce a classic neuroscience experiment that’s considered the basis of learning and memory in the brain. In other words, artificial neuron and synapse “chips” have progressed to the point where they can actually use a biological neuron intermediary to form a circuit that, at least partially, behaves like the real thing.

That’s not to say cyborg brains are coming soon. The simulation only recreated a small network that supports excitatory transmission in the hippocampus—a critical region that supports memory—and most brain functions require enormous cross-talk between numerous neurons and circuits. Nevertheless, the study is a jaw-dropping demonstration of how far we’ve come in recreating biological neurons and synapses in artificial hardware.

And perhaps one day, the currently “experimental” neuromorphic hardware will be integrated into broken biological neural circuits as bridges to restore movement, memory, personality, and even a sense of self.

The Artificial Brain Boom
One important thing: this study relies heavily on a decade of research into neuromorphic computing, or the implementation of brain functions inside computer chips.

The best-known example is perhaps IBM’s TrueNorth, which leveraged the brain’s computational principles to build a completely different computer than what we have today. Today’s computers run on a von Neumann architecture, in which memory and processing modules are physically separate. In contrast, the brain’s computing and memory are simultaneously achieved at synapses, small “hubs” on individual neurons that talk to adjacent ones.

Because memory and processing occur on the same site, biological neurons don’t have to shuttle data back and forth between processing and storage compartments, massively reducing processing time and energy use. What’s more, a neuron’s history will also influence how it behaves in the future, increasing flexibility and adaptability compared to computers. With the rise of deep learning, which loosely mimics neural processing as the prima donna of AI, the need to reduce power while boosting speed and flexible learning is becoming ever more tantamount in the AI community.

Neuromorphic computing was partially born out of this need. Most chips utilize special ingredients that change their resistance (or other physical characteristics) to mimic how a neuron might adapt to stimulation. Some chips emulate a whole neuron, that is, how it responds to a history of stimulation—does it get easier or harder to fire? Others imitate synapses themselves, that is, how easily they will pass on the information to another neuron.

Although single neuromorphic chips have proven to be far more efficient and powerful than current computer chips running machine learning algorithms in toy problems, so far few people have tried putting the artificial components together with biological ones in the ultimate test.

That’s what this study did.

A Hybrid Network
Still with me? Let’s talk network.

It’s gonna sound complicated, but remember: learning is the formation of neural networks, and neurons that fire together wire together. To rephrase: when learning, neurons will spontaneously organize into networks so that future instances will re-trigger the entire network. To “wire” together, downstream neurons will become more responsive to their upstream neural partners, so that even a whisper will cause them to activate. In contrast, some types of stimulation will cause the downstream neuron to “chill out” so that only an upstream “shout” will trigger downstream activation.

Both these properties—easier or harder to activate downstream neurons—are essentially how the brain forms connections. The “amping up,” in neuroscience jargon, is long-term potentiation (LTP), whereas the down-tuning is LTD (long-term depression). These two phenomena were first discovered in the rodent hippocampus more than half a century ago, and ever since have been considered as the biological basis of how the brain learns and remembers, and implicated in neurological problems such as addition (seriously, you can’t pass Neuro 101 without learning about LTP and LTD!).

So it’s perhaps especially salient that one of the first artificial-brain hybrid networks recapitulated this classic result.

To visualize: the three-neuron network began in Switzerland, with an artificial neuron with the badass name of “silicon spiking neuron.” That neuron is linked to an artificial synapse, a “memristor” located in the UK, which is then linked to a biological rat neuron cultured in Italy. The rat neuron has a “smart” microelectrode, controlled by the artificial synapse, to stimulate it. This is the artificial-to-biological pathway.

Meanwhile, the rat neuron in Italy also has electrodes that listen in on its electrical signaling. This signaling is passed back to another artificial synapse in the UK, which is then used to control a second artificial neuron back in Switzerland. This is the biological-to-artificial pathway back. As a testimony in how far we’ve come in digitizing neural signaling, all of the biological neural responses are digitized and sent over the internet to control its far-out artificial partner.

Here’s the crux: to demonstrate a functional neural network, just having the biological neuron passively “pass on” electrical stimulation isn’t enough. It has to show the capacity to learn, that is, to be able to mimic the amping up and down-tuning that are LTP and LTD, respectively.

You’ve probably guessed the results: certain stimulation patterns to the first artificial neuron in Switzerland changed how the artificial synapse in the UK operated. This, in turn, changed the stimulation to the biological neuron, so that it either amped up or toned down depending on the input.

Similarly, the response of the biological neuron altered the second artificial synapse, which then controlled the output of the second artificial neuron. Altogether, the biological and artificial components seamlessly linked up, over thousands of miles, into a functional neural circuit.

Cyborg Mind-Meld
So…I’m still picking my jaw up off the floor.

It’s utterly insane seeing a classic neuroscience learning experiment repeated with an integrated network with artificial components. That said, a three-neuron network is far from the thousands of synapses (if not more) needed to truly re-establish a broken neural circuit in the hippocampus, which DARPA has been aiming to do. And LTP/LTD has come under fire recently as the de facto brain mechanism for learning, though so far they remain cemented as neuroscience dogma.

However, this is one of the few studies where you see fields coming together. As Richard Feynman famously said, “What I cannot recreate, I cannot understand.” Even though neuromorphic chips were built on a high-level rather than molecular-level understanding of how neurons work, the study shows that artificial versions can still synapse with their biological counterparts. We’re not just on the right path towards understanding the brain, we’re recreating it, in hardware—if just a little.

While the study doesn’t have immediate use cases, practically it does boost both the neuromorphic computing and neuroprosthetic fields.

“We are very excited with this new development,” said study author Dr. Themis Prodromakis at the University of Southampton. “On one side it sets the basis for a novel scenario that was never encountered during natural evolution, where biological and artificial neurons are linked together and communicate across global networks; laying the foundations for the Internet of Neuro-electronics. On the other hand, it brings new prospects to neuroprosthetic technologies, paving the way towards research into replacing dysfunctional parts of the brain with AI chips.”

Image Credit: Gerd Altmann from Pixabay Continue reading →

Posted in Human Robots

#436774 AI Is an Energy-Guzzler. We Need to ...

Posted on February 29, 2020 by Android

There is a saying that has emerged among the tech set in recent years: AI is the new electricity. The platitude refers to the disruptive power of artificial intelligence for driving advances in everything from transportation to predicting the weather.

Of course, the computers and data centers that support AI’s complex algorithms are very much dependent on electricity. While that may seem pretty obvious, it may be surprising to learn that AI can be extremely power-hungry, especially when it comes to training the models that enable machines to recognize your face in a photo or for Alexa to understand a voice command.

The scale of the problem is difficult to measure, but there have been some attempts to put hard numbers on the environmental cost.

For instance, one paper published on the open-access repository arXiv claimed that the carbon emissions for training a basic natural language processing (NLP) model—algorithms that process and understand language-based data—are equal to the CO2 produced by the average American lifestyle over two years. A more robust model required the equivalent of about 17 years’ worth of emissions.

The authors noted that about a decade ago, NLP models could do the job on a regular commercial laptop. Today, much more sophisticated AI models use specialized hardware like graphics processing units, or GPUs, a chip technology popularized by Nvidia for gaming that also proved capable of supporting computing tasks for AI.

OpenAI, a nonprofit research organization co-founded by tech prophet and profiteer Elon Musk, said that the computing power “used in the largest AI training runs has been increasing exponentially with a 3.4-month doubling time” since 2012. That’s about the time that GPUs started making their way into AI computing systems.

Getting Smarter About AI Chip Design
While GPUs from Nvidia remain the gold standard in AI hardware today, a number of startups have emerged to challenge the company’s industry dominance. Many are building chipsets designed to work more like the human brain, an area that’s been dubbed neuromorphic computing.

One of the leading companies in this arena is Graphcore, a UK startup that has raised more than $450 million and boasts a valuation of $1.95 billion. The company’s version of the GPU is an IPU, which stands for intelligence processing unit.

To build a computer brain more akin to a human one, the big brains at Graphcore are bypassing the precise but time-consuming number-crunching typical of a conventional microprocessor with one that’s content to get by on less precise arithmetic.

The results are essentially the same, but IPUs get the job done much quicker. Graphcore claimed it was able to train the popular BERT NLP model in just 56 hours, while tripling throughput and reducing latency by 20 percent.

An article in Bloomberg compared the approach to the “human brain shifting from calculating the exact GPS coordinates of a restaurant to just remembering its name and neighborhood.”

Graphcore’s hardware architecture also features more built-in memory processing, boosting efficiency because there’s less need to send as much data back and forth between chips. That’s similar to an approach adopted by a team of researchers in Italy that recently published a paper about a new computing circuit.

The novel circuit uses a device called a memristor that can execute a mathematical function known as a regression in just one operation. The approach attempts to mimic the human brain by processing data directly within the memory.

Daniele Ielmini at Politecnico di Milano, co-author of the Science Advances paper, told Singularity Hub that the main advantage of in-memory computing is the lack of any data movement, which is the main bottleneck of conventional digital computers, as well as the parallel processing of data that enables the intimate interactions among various currents and voltages within the memory array.

Ielmini explained that in-memory computing can have a “tremendous impact on energy efficiency of AI, as it can accelerate very advanced tasks by physical computation within the memory circuit.” He added that such “radical ideas” in hardware design will be needed in order to make a quantum leap in energy efficiency and time.

It’s Not Just a Hardware Problem
The emphasis on designing more efficient chip architecture might suggest that AI’s power hunger is essentially a hardware problem. That’s not the case, Ielmini noted.

“We believe that significant progress could be made by similar breakthroughs at the algorithm and dataset levels,” he said.

He’s not the only one.

One of the key research areas at Qualcomm’s AI research lab is energy efficiency. Max Welling, vice president of Qualcomm Technology R&D division, has written about the need for more power-efficient algorithms. He has gone so far as to suggest that AI algorithms will be measured by the amount of intelligence they provide per joule.

One emerging area being studied, Welling wrote, is the use of Bayesian deep learning for deep neural networks.

It’s all pretty heady stuff and easily the subject of a PhD thesis. The main thing to understand in this context is that Bayesian deep learning is another attempt to mimic how the brain processes information by introducing random values into the neural network. A benefit of Bayesian deep learning is that it compresses and quantifies data in order to reduce the complexity of a neural network. In turn, that reduces the number of “steps” required to recognize a dog as a dog—and the energy required to get the right result.

A team at Oak Ridge National Laboratory has previously demonstrated another way to improve AI energy efficiency by converting deep learning neural networks into what’s called a spiking neural network. The researchers spiked their deep spiking neural network (DSNN) by introducing a stochastic process that adds random values like Bayesian deep learning.

The DSNN actually imitates the way neurons interact with synapses, which send signals between brain cells. Individual “spikes” in the network indicate where to perform computations, lowering energy consumption because it disregards unnecessary computations.

The system is being used by cancer researchers to scan millions of clinical reports to unearth insights on causes and treatments of the disease.

Helping battle cancer is only one of many rewards we may reap from artificial intelligence in the future, as long as the benefits of those algorithms outweigh the costs of using them.

“Making AI more energy-efficient is an overarching objective that spans the fields of algorithms, systems, architecture, circuits, and devices,” Ielmini said.

Image Credit: analogicus from Pixabay Continue reading →

Posted in Human Robots

#436559 This Is What an AI Said When Asked to ...

Posted on February 14, 2020 by Android

“What’s past is prologue.” So says the famed quote from Shakespeare’s The Tempest, alleging that we can look to what has already happened as an indication of what will happen next.

This idea could be interpreted as being rather bleak; are we doomed to repeat the errors of the past until we correct them? We certainly do need to learn and re-learn life lessons—whether in our work, relationships, finances, health, or other areas—in order to grow as people.

Zooming out, the same phenomenon exists on a much bigger scale—that of our collective human history. We like to think we’re improving as a species, but haven’t yet come close to doing away with the conflicts and injustices that plagued our ancestors.

Zooming back in (and lightening up) a little, what about the short-term future? What might happen over the course of this year, and what information would we use to make educated guesses about it?

The editorial team at The Economist took a unique approach to answering these questions. On top of their own projections for 2020, including possible scenarios in politics, economics, and the continued development of technologies like artificial intelligence, they looked to an AI to make predictions of its own. What it came up with is intriguing, and a little bit uncanny.

[For the full list of the questions and answers, read The Economist article].

An AI That Reads—Then Writes
Almost exactly a year ago, non-profit OpenAI announced it had built a neural network for natural language processing called GPT-2. The announcement was met with some controversy, as it included the caveat that the tool would not be immediately released to the public due to its potential for misuse. It was then released in phases over the course of several months.

GPT-2’s creators upped the bar on quality when training the neural net; rather than haphazardly feeding it low-quality text, they only used articles that got more than three upvotes on Reddit (admittedly, this doesn’t guarantee high quality across the board—but it’s something).

The training dataset consisted of 40GB of text. For context, 1GB of text is about 900,000 ASCII pages or 130,000 double-spaced Microsoft Word pages.

The tool has no understanding of the text it’s generating, of course. It uses language patterns and word sequences to draw statistical associations between words and phrases, building a sort of guidebook for itself (not unlike the grammar rules and vocabulary words you might study when trying to learn a foreign language). It then uses that guidebook to answer questions or predict what will come after a particular sequence of words.

GPT-2’s creators did note that when the model is trained on specifically-selected datasets for narrower applications, its replies become more convincing.

Some Light Editing
Though the Economist article detailing GPT-2’s predictions describes the text the model generated as “unedited,” this isn’t wholly accurate. In a Medium post, deputy editor Tom Standage describes the methodology he used, and it does involve a degree of tweaking.

Standage wrote an introductory paragraph, followed by his opening question (“Greetings, GPT-2. Have you done an interview before?”). He configured the tool to generate five responses to each subsequent question then selected the best answer, deleting any questions GPT-2 came up with as part of its response.

AI’s View of the World—and Itself
The questions Standage asked ranged from tech-related (the future of AI, technological unemployment, misuse of technology) to political and economic (re-election of Donald Trump, the US-China trade war). Here are a couple of his questions and GPT-2’s answers:

Q: Do fake news stories, generated using AI, pose a threat to democracy? Are we likely to see this tactic being used in the 2020 American presidential elections?
A: Yes, we’ll probably see them. It’s just a matter of when. Fake news stories are generally generated by political campaigns, and have the potential to have a huge impact on the outcome of an election. This is because they spread misinformation, often with no basis in fact.

Q: How do you see the economic outlook for 2020?
A: I think it is very unclear. The outlook for the UK economy is uncertain, but the American economy is very strong. I think we are going to have a lot of turbulence in the world economy.

While some of its answers seemed insightful—when asked about the future of AI, GPT-2 advised that we treat the technology like a tool and “put as much effort into developing [it] as necessary, rather than worrying that it’s going to harm us and destroy our lives”—many of the responses were quite vague. One answer, for example, mentions that there will “probably be major changes in China.” Are we talking political changes? Social? Economic? All of the above?

However, while nothing the AI wrote was particularly illuminating or new, it’s impressive that it was able to expound on the correct topic for each question, and in fully coherent English.

GPT-2 named itself—AI—as the most important disruptive technology to watch in 2020. This is telling; it’s coming from a synthesis of millions of pages of published articles. If GPT-2 thinks AI is more important than, say, gene editing or quantum computing, it could be right. Or it could simply be that AI is at peak hype, and as such more is being written about it than about other technologies.

Equally intriguing was GPT-2’s response when asked whether Donald Trump will win a second term: “I think he will not win a second term. I think he will be defeated in the general election.” Some deeper insight there would be great, but hey—we’ll take it.

Predicting Predictions
Since an AI can read and synthesize vast data sets much faster than we can, it’s being used to predict all kinds of things, from virus outbreaks to crime. But asking it to philosophize on the future based on the (Reddit-curated) past is new, and if you think about it, a pretty fascinating undertaking.

As GPT-2 and tools like it continually improve, we’ll likely see them making more—and better—predictions of the future. In the meantime, let’s hope that the new data these models are trained on—news of what’s happening this week, this month, this year—add to an already-present sense of optimism.

When asked if it had any advice for readers, GPT-2 replied, “The big projects that you think are impossible today are actually possible in the near future.”

Image Credit: Alexas_Fotos from Pixabay Continue reading →

Posted in Human Robots