Tag Archives: big

#434324 Big Brother Nation: The Case for ...

Posted on January 13, 2019 by Android

Powerful surveillance cameras have crept into public spaces. We are filmed and photographed hundreds of times a day. To further raise the stakes, the resulting video footage is fed to new forms of artificial intelligence software that can recognize faces in real time, read license plates, even instantly detect when a particular pre-defined action or activity takes place in front of a camera.

As most modern cities have quietly become surveillance cities, the law has been slow to catch up. While we wait for robust legal frameworks to emerge, the best way to protect our civil liberties right now is to fight technology with technology. All cities should place local surveillance video into a public cloud-based data trust. Here’s how it would work.

In Public Data We Trust
To democratize surveillance, every city should implement three simple rules. First, anyone who aims a camera at public space must upload that day’s haul of raw video file (and associated camera meta-data) into a cloud-based repository. Second, this cloud-based repository must have open APIs and a publicly-accessible log file that records search histories and tracks who has accessed which video files. And third, everyone in the city should be given the same level of access rights to the stored video data—no exceptions.

This kind of public data repository is called a “data trust.” Public data trusts are not just wishful thinking. Different types of trusts are already in successful use in Estonia and Barcelona, and have been proposed as the best way to store and manage the urban data that will be generated by Alphabet’s planned Sidewalk Labs project in Toronto.

It’s true that few people relish the thought of public video footage of themselves being looked at by strangers and friends, by ex-spouses, potential employers, divorce attorneys, and future romantic prospects. In fact, when I propose this notion when I give talks about smart cities, most people recoil in horror. Some turn red in the face and jeer at my naiveté. Others merely blink quietly in consternation.

The reason we should take this giant step towards extreme transparency is to combat the secrecy that surrounds surveillance. Openness is a powerful antidote to oppression. Edward Snowden summed it up well when he said, “Surveillance is not about public safety, it’s about power. It’s about control.”

Let Us Watch Those Watching Us
If public surveillance video were put back into the hands of the people, citizens could watch their government as it watches them. Right now, government cameras are controlled by the state. Camera locations are kept secret, and only the agencies that control the cameras get to see the footage they generate.

Because of these information asymmetries, civilians have no insight into the size and shape of the modern urban surveillance infrastructure that surrounds us, nor the uses (or abuses) of the video footage it spawns. For example, there is no swift and efficient mechanism to request a copy of video footage from the cameras that dot our downtown. Nor can we ask our city’s police force to show us a map that documents local traffic camera locations.

By exposing all public surveillance videos to the public gaze, cities could give regular people tools to assess the size, shape, and density of their local surveillance infrastructure and neighborhood “digital dragnet.” Using the meta-data that’s wrapped around video footage, citizens could geo-locate individual cameras onto a digital map to generate surveillance “heat maps.” This way people could assess whether their city’s camera density was higher in certain zip codes, or in neighborhoods populated by a dominant ethnic group.

Surveillance heat maps could be used to document which government agencies were refusing to upload their video files, or which neighborhoods were not under surveillance. Given what we already know today about the correlation between camera density, income, and social status, these “dark” camera-free regions would likely be those located near government agencies and in more affluent parts of a city.

Extreme transparency would democratize surveillance. Every city’s data trust would keep a publicly-accessible log of who’s searching for what, and whom. People could use their local data trust’s search history to check whether anyone was searching for their name, face, or license plate. As a result, clandestine spying on—and stalking of—particular individuals would become difficult to hide and simpler to prove.

Protect the Vulnerable and Exonerate the Falsely Accused
Not all surveillance video automatically works against the underdog. As the bungled (and consequently no longer secret) assassination of journalist Jamal Khashoggi demonstrated, one of the unexpected upsides of surveillance cameras has been the fact that even kings become accountable for their crimes. If opened up to the public, surveillance cameras could serve as witnesses to justice.

Video evidence has the power to protect vulnerable individuals and social groups by shedding light onto messy, unreliable (and frequently conflicting) human narratives of who did what to whom, and why. With access to a data trust, a person falsely accused of a crime could prove their innocence. By searching for their own face in video footage or downloading time/date stamped footage from a particular camera, a potential suspect could document their physical absence from the scene of a crime—no lengthy police investigation or high-priced attorney needed.

Given Enough Eyeballs, All Crimes Are Shallow
Placing public surveillance video into a public trust could make cities safer and would streamline routine police work. Linus Torvalds, the developer of open-source operating system Linux, famously observed that “given enough eyeballs, all bugs are shallow.” In the case of public cameras and a common data repository, Torvald’s Law could be restated as “given enough eyeballs, all crimes are shallow.”

If thousands of citizen eyeballs were given access to a city’s public surveillance videos, local police forces could crowdsource the work of solving crimes and searching for missing persons. Unfortunately, at the present time, cities are unable to wring any social benefit from video footage of public spaces. The most formidable barrier is not government-imposed secrecy, but the fact that as cameras and computers have grown cheaper, a large and fast-growing “mom and pop” surveillance state has taken over most of the filming of public spaces.

While we fear spooky government surveillance, the reality is that we’re much more likely to be filmed by security cameras owned by shopkeepers, landlords, medical offices, hotels, homeowners, and schools. These businesses, organizations, and individuals install cameras in public areas for practical reasons—to reduce their insurance costs, to prevent lawsuits, or to combat shoplifting. In the absence of regulations governing their use, private camera owners store video footage in a wide variety of locations, for varying retention periods.

The unfortunate (and unintended) result of this informal and decentralized network of public surveillance is that video files are not easy to access, even for police officers on official business. After a crime or terrorist attack occurs, local police (or attorneys armed with a subpoena) go from door to door to manually collect video evidence. Once they have the videos in hand, their next challenge is searching for the right “codex” to crack the dozens of different file formats they encounter so they can watch and analyze the footage.

The result of these practical barriers is that as it stands today, only people with considerable legal or political clout are able to successfully gain access into a city’s privately-owned, ad-hoc collections of public surveillance videos. Not only are cities missing the opportunity to streamline routine evidence-gathering police work, they’re missing a radically transformative benefit that would become possible once video footage from thousands of different security cameras were pooled into a single repository: the ability to apply the power of citizen eyeballs to the work of improving public safety.

Why We Need Extreme Transparency
When regular people can’t access their own surveillance videos, there can be no data justice. While we wait for the law to catch up with the reality of modern urban life, citizens and city governments should use technology to address the problem that lies at the heart of surveillance: a power imbalance between those who control the cameras and those who don’t.

Cities should permit individuals and organizations to install and deploy as many public-facing cameras as they wish, but with the mandate that camera owners must place all resulting video footage into the mercilessly bright sunshine of an open data trust. This way, cloud computing, open APIs, and artificial intelligence software can help combat abuses of surveillance and give citizens insight into who’s filming us, where, and why.

Image Credit: VladFotoMag / Shutterstock.com Continue reading →

Posted in Human Robots

#434297 How Can Leaders Ensure Humanity in a ...

Posted on January 9, 2019 by Android

It’s hard to avoid the prominence of AI in our lives, and there is a plethora of predictions about how it will influence our future. In their new book Solomon’s Code: Humanity in a World of Thinking Machines, co-authors Olaf Groth, Professor of Strategy, Innovation and Economics at HULT International Business School and CEO of advisory network Cambrian.ai, and Mark Nitzberg, Executive Director of UC Berkeley’s Center for Human-Compatible AI, believe that the shift in balance of power between intelligent machines and humans is already here.

I caught up with the authors about how the continued integration between technology and humans, and their call for a “Digital Magna Carta,” a broadly-accepted charter developed by a multi-stakeholder congress that would help guide the development of advanced technologies to harness their power for the benefit of all humanity.

Lisa Kay Solomon: Your new book, Solomon’s Code, explores artificial intelligence and its broader human, ethical, and societal implications that all leaders need to consider. AI is a technology that’s been in development for decades. Why is it so urgent to focus on these topics now?

Olaf Groth and Mark Nitzberg: Popular perception always thinks of AI in terms of game-changing narratives—for instance, Deep Blue beating Gary Kasparov at chess. But it’s the way these AI applications are “getting into our heads” and making decisions for us that really influences our lives. That’s not to say the big, headline-grabbing breakthroughs aren’t important; they are.

But it’s the proliferation of prosaic apps and bots that changes our lives the most, by either empowering or counteracting who we are and what we do. Today, we turn a rapidly growing number of our decisions over to these machines, often without knowing it—and even more often without understanding the second- and third-order effects of both the technologies and our decisions to rely on them.

There is genuine power in what we call a “symbio-intelligent” partnership between human, machine, and natural intelligences. These relationships can optimize not just economic interests, but help improve human well-being, create a more purposeful workplace, and bring more fulfillment to our lives.

However, mitigating the risks while taking advantage of the opportunities will require a serious, multidisciplinary consideration of how AI influences human values, trust, and power relationships. Whether or not we acknowledge their existence in our everyday life, these questions are no longer just thought exercises or fodder for science fiction.

In many ways, these technologies can challenge what it means to be human, and their ramifications already affect us in real and often subtle ways. We need to understand how

LKS: There is a lot of hype and misconceptions about AI. In your book, you provide a useful distinction between the cognitive capability that we often associate with AI processes, and the more human elements of consciousness and conscience. Why are these distinctions so important to understand?

OG & MN: Could machines take over consciousness some day as they become more powerful and complex? It’s hard to say. But there’s little doubt that, as machines become more capable, humans will start to think of them as something conscious—if for no other reason than our natural inclination to anthropomorphize.

Machines are already learning to recognize our emotional states and our physical health. Once they start talking that back to us and adjusting their behavior accordingly, we will be tempted to develop a certain rapport with them, potentially more trusting or more intimate because the machine recognizes us in our various states.

Consciousness is hard to define and may well be an emergent property, rather than something you can easily create or—in turn—deduce to its parts. So, could it happen as we put more and more elements together, from the realms of AI, quantum computing, or brain-computer interfaces? We can’t exclude that possibility.

Either way, we need to make sure we’re charting out a clear path and guardrails for this development through the Three Cs in machines: cognition (where AI is today); consciousness (where AI could go); and conscience (what we need to instill in AI before we get there). The real concern is that we reach machine consciousness—or what humans decide to grant as consciousness—without a conscience. If that happens, we will have created an artificial sociopath.

LKS: We have been seeing major developments in how AI is influencing product development and industry shifts. How is the rise of AI changing power at the global level?

OG & MN: Both in the public and private sectors, the data holder has the power. We’ve already seen the ascendance of about 10 “digital barons” in the US and China who sit on huge troves of data, massive computing power, and the resources and money to attract the world’s top AI talent. With these gaps already open between the haves and the have-nots on the technological and corporate side, we’re becoming increasingly aware that similar inequalities are forming at a societal level as well.

Economic power flows with data, leaving few options for socio-economically underprivileged populations and their corrupt, biased, or sparse digital footprints. By concentrating power and overlooking values, we fracture trust.

We can already see this tension emerging between the two dominant geopolitical models of AI. China and the US have emerged as the most powerful in both technological and economic terms, and both remain eager to drive that influence around the world. The EU countries are more contained on these economic and geopolitical measures, but they’ve leaped ahead on privacy and social concerns.

The problem is, no one has yet combined leadership on all three critical elements of values, trust, and power. The nations and organizations that foster all three of these elements in their AI systems and strategies will lead the future. Some are starting to recognize the need for the combination, but we found just 13 countries that have created significant AI strategies. Countries that wait too long to join them risk subjecting themselves to a new “data colonialism” that could change their economies and societies from the outside.

LKS: Solomon’s Code looks at AI from a variety of perspectives, considering both positive and potentially dangerous effects. You caution against the rising global threat and weaponization of AI and data, suggesting that “biased or dirty data is more threatening than nuclear arms or a pandemic.” For global leaders, entrepreneurs, technologists, policy makers and social change agents reading this, what specific strategies do you recommend to ensure ethical development and application of AI?

OG & MN: We’ve surrendered many of our most critical decisions to the Cult of Data. In most cases, that’s a great thing, as we rely more on scientific evidence to understand our world and our way through it. But we swing too far in other instances, assuming that datasets and algorithms produce a complete story that’s unsullied by human biases or intellectual shortcomings. We might choose to ignore it, but no one is blind to the dangers of nuclear war or pandemic disease. Yet, we willfully blind ourselves to the threat of dirty data, instead believing it to be pristine.

So, what do we do about it? On an individual level, it’s a matter of awareness, knowing who controls your data and how outsourcing of decisions to thinking machines can present opportunities and threats alike.

For business, government, and political leaders, we need to see a much broader expansion of ethics committees with transparent criteria with which to evaluate new products and services. We might consider something akin to clinical trials for pharmaceuticals—a sort of testing scheme that can transparently and independently measure the effects on humans of algorithms, bots, and the like. All of this needs to be multidisciplinary, bringing in expertise from across technology, social systems, ethics, anthropology, psychology, and so on.

Finally, on a global level, we need a new charter of rights—a Digital Magna Carta—that formalizes these protections and guides the development of new AI technologies toward all of humanity’s benefit. We’ve suggested the creation of a multi-stakeholder Cambrian Congress (harkening back to the explosion of life during the Cambrian period) that can not only begin to frame benefits for humanity, but build the global consensus around principles for a basic code-of-conduct, and ideas for evaluation and enforcement mechanisms, so we can get there without any large-scale failures or backlash in society. So, it’s not one or the other—it’s both.

Image Credit: whiteMocca / Shutterstock.com Continue reading →

Posted in Human Robots

#434256 Singularity Hub’s Top Articles of the ...

Posted on December 29, 2018 by Android

2018 was a big year for science and technology. The first gene-edited babies were born, as were the first cloned monkeys. SpaceX successfully launched the Falcon Heavy, and NASA’s InSight lander placed a seismometer on Mars. Bitcoin’s value plummeted, as did the cost of renewable energy. The world’s biggest neuromorphic supercomputer was switched on, and quantum communication made significant progress.

As 2018 draws to a close and we start anticipating the developments that will happen in 2019, here’s a look back at our ten most-read articles of the year.

This 3D Printed House Goes Up in a Day for Under $10,000
Vanessa Bates Ramirez | 3/18/18
“ICON and New Story’s vision is one of 3D printed houses acting as a safe, affordable housing alternative for people in need. New Story has already built over 800 homes in Haiti, El Salvador, Bolivia, and Mexico, partnering with the communities they serve to hire local labor and purchase local materials rather than shipping everything in from abroad.”

Machines Teaching Each Other Could Be the Biggest Exponential Trend in AI
Aaron Frank | 1/21/18
“Data is the fuel of machine learning, but even for machines, some data is hard to get—it may be risky, slow, rare, or expensive. In those cases, machines can share experiences or create synthetic experiences for each other to augment or replace data. It turns out that this is not a minor effect, it actually is self-amplifying, and therefore exponential.”

Low-Cost Soft Robot Muscles Can Lift 200 Times Their Weight and Self-Heal
Edd Gent | 1/11/18
“Now researchers at the University of Colorado Boulder have built a series of low-cost artificial muscles—as little as 10 cents per device—using soft plastic pouches filled with electrically insulating liquids that contract with the force and speed of mammalian skeletal muscles when a voltage is applied to them.”

These Are the Most Exciting Industries and Jobs of the Future
Raya Bidshahri | 1/29/18
“Technological trends are giving rise to what many thought leaders refer to as the “imagination economy.” This is defined as “an economy where intuitive and creative thinking create economic value, after logical and rational thinking have been outsourced to other economies.” Unsurprisingly, humans continue to outdo machines when it comes to innovating and pushing intellectual, imaginative, and creative boundaries, making jobs involving these skills the hardest to automate.”

Inside a $1 Billion Real Estate Company Operating Entirely in VR
Aaron Frank | 4/8/18
“Incredibly, this growth is largely the result of eXp Realty’s use of an online virtual world similar to Second Life. That means every employee, contractor, and the thousands of agents who work at the company show up to work—team meetings, training seminars, onboarding sessions—all inside a virtual reality campus.To be clear, this is a traditional real estate brokerage helping people buy and sell physical homes—but they use a virtual world as their corporate offices.”

How Fast Is AI Progressing? Stanford’s New Report Card for Artificial Intelligence
Thomas Hornigold | 1/18/18
“Progress in AI over the next few years is far more likely to resemble a gradual rising tide—as more and more tasks can be turned into algorithms and accomplished by software—rather than the tsunami of a sudden intelligence explosion or general intelligence breakthrough. Perhaps measuring the ability of an AI system to learn and adapt to the work routines of humans in office-based tasks could be possible.”

When Will We Finally Achieve True Artificial Intelligence?
Thomas Hornigold | 1/1/18
“The issue with trying to predict the exact date of human-level AI is that we don’t know how far is left to go. This is unlike Moore’s Law. Moore’s Law, the doubling of processing power roughly every couple of years, makes a very concrete prediction about a very specific phenomenon. We understand roughly how to get there—improved engineering of silicon wafers—and we know we’re not at the fundamental limits of our current approach. You cannot say the same about artificial intelligence.”

IBM’s New Computer Is the Size of a Grain of Salt and Costs Less Than 10 Cents
Edd Gent | 3/26/18
“Costing less than 10 cents to manufacture, the company envisions the device being embedded into products as they move around the supply chain. The computer’s sensing, processing, and communicating capabilities mean it could effectively turn every item in the supply chain into an Internet of Things device, producing highly granular supply chain data that could streamline business operations.”

Why the Rise of Self-Driving Vehicles Will Actually Increase Car Ownership
Melba Kurman and Hod Lipson / 2/14/18
“When people predict the demise of car ownership, they are overlooking the reality that the new autonomous automotive industry is not going to be just a re-hash of today’s car industry with driverless vehicles. Instead, the automotive industry of the future will be selling what could be considered an entirely new product: a wide variety of intelligent, self-guiding transportation robots. When cars become a widely used type of transportation robot, they will be cheap, ubiquitous, and versatile.”

A Model for the Future of Education
Peter Diamandis | 9/12/18
“I imagine a relatively near-term future in which robotics and artificial intelligence will allow any of us, from ages 8 to 108, to easily and quickly find answers, create products, or accomplish tasks, all simply by expressing our desires. From ‘mind to manufactured in moments.’ In short, we’ll be able to do and create almost whatever we want. In this future, what attributes will be most critical for our children to learn to become successful in their adult lives? What’s most important for educating our children today?”

Image Credit: Yurchanka Siarhei / Shutterstock.com Continue reading →

Posted in Human Robots

#434194 Educating the Wise Cyborgs of the Future

Posted on December 21, 2018 by Android

When we think of wisdom, we often think of ancient philosophers, mystics, or spiritual leaders. Wisdom is associated with the past. Yet some intellectual leaders are challenging us to reconsider wisdom in the context of the technological evolution of the future.

With the rise of exponential technologies like virtual reality, big data, artificial intelligence, and robotics, people are gaining access to increasingly powerful tools. These tools are neither malevolent nor benevolent on their own; human values and decision-making influence how they are used.

In future-themed discussions we often focus on technological progress far more than on intellectual and moral advancements. In reality, the virtuous insights that future humans possess will be even more powerful than their technological tools.

Tom Lombardo and Ray Todd Blackwood are advocating for exactly this. In their interdisciplinary paper “Educating the Wise Cyborg of the Future,” they propose a new definition of wisdom—one that is relevant in the context of the future of humanity.

We Are Already Cyborgs
The core purpose of Lombardo and Blackwood’s paper is to explore revolutionary educational models that will prepare humans, soon-to-be-cyborgs, for the future. The idea of educating such “cyborgs” may sound like science fiction, but if you pay attention to yourself and the world around you, cyborgs came into being a long time ago.

Techno-philosophers like Jason Silva point out that our tech devices are an abstract form of brain-machine interfaces. We use smartphones to store and retrieve information, perform calculations, and communicate with each other. Our devices are an extension of our minds.

According to philosophers Andy Clark and David Chalmers’ theory of the extended mind, we use this technology to expand the boundaries of our minds. We use tools like machine learning to enhance our cognitive skills or powerful telescopes to enhance our visual reach. Such is how technology has become a part of our exoskeletons, allowing us to push beyond our biological limitations.

In other words, you are already a cyborg. You have been all along.

Such an abstract definition of cyborgs is both relevant and thought-provoking. But it won’t stay abstract for much longer. The past few years have seen remarkable developments in both the hardware and software of brain-machine interfaces. Experts are designing more intricate electrodes while programming better algorithms to interpret the neural signals. Scientists have already succeeded in enabling paralyzed patients to type with their minds, and are even allowing people to communicate purely through brainwaves. Technologists like Ray Kurzweil believe that by 2030 we will connect the neocortex of our brains to the cloud via nanobots.

Given these trends, humans will continue to be increasingly cyborg-like. Our future schools may not necessarily educate people as we are today, but rather will be educating a new species of human-machine hybrid.

Wisdom-Based Education
Whether you take an abstract or literal definition of a cyborg, we need to completely revamp our educational models. Even if you don’t buy into the scenario where humans integrate powerful brain-machine interfaces into our minds, there is still a desperate need for wisdom-based education to equip current generations to tackle 21st-century issues.

With an emphasis on isolated subjects, standardized assessments, and content knowledge, our current educational models were designed for the industrial era, with the intended goal of creating masses of efficient factory workers—not to empower critical thinkers, innovators, or wise cyborgs.

Currently, the goal of higher education is to provide students with the degree that society tells them they need, and ostensibly to prepare them for the workforce. In contrast, Lombardo and Blackwood argue that wisdom should be the central goal of higher education, and they elaborate on how we can practically make this happen. Lombardo has developed a comprehensive two-year foundational education program for incoming university students aimed at the development of wisdom.

What does such an educational model look like? Lombardo and Blackwood break wisdom down into individual traits and capacities, each of which can be developed and measured independently or in combination with others. The authors lay out an expansive list of traits that can influence our decision-making as we strive to tackle global challenges and pave a more exciting future. These include big-picture thinking, curiosity, wonder, compassion, self-transcendence, love of learning, optimism, and courage.

As the authors point out, “given the complex and transforming nature of the world we live in, the development of wisdom provides a holistic, perspicacious, and ethically informed foundation for understanding the world, identifying its critical problems and positive opportunities, and constructively addressing its challenges.”

After all, many of the challenges we see in our world today boil down to out-dated ways of thinking, be they regressive mindsets, superficial value systems, or egocentric mindsets. The development of wisdom would immunize future societies against such debilitating values; imagine what our world would be like if wisdom was ingrained in all leaders and participating members of society.

The Wise Cyborg
Lombardo and Blackwood invite us to imagine how the wise cyborgs of the future would live their lives. What would happen if the powerful human-machine hybrids of tomorrow were also purpose-driven, compassionate, and ethical?

They would perceive the evolving digital world through a lens of wonder, awe, and curiosity. They would use digital information as a tool for problem-solving and a source of infinite knowledge. They would leverage immersive mediums like virtual reality to enhance creative expression and experimentation. They would continue to adapt and thrive in an unpredictable world of accelerating change.

Our media often depict a dystopian future for our species. It is worth considering a radically positive yet plausible scenario where instead of the machines taking over, we converge with them into wise cyborgs. This is just a glimpse of what is possible if we combine transcendent wisdom with powerful exponential technologies.

Image Credit: Peshkova / Shutterstock.com Continue reading →

Posted in Human Robots

#434151 Life-or-Death Algorithms: The Black Box ...

Posted on December 18, 2018 by Android

When it comes to applications for machine learning, few can be more widely hyped than medicine. This is hardly surprising: it’s a huge industry that generates a phenomenal amount of data and revenue, where technological advances can improve or save the lives of millions of people. Hardly a week passes without a study that suggests algorithms will soon be better than experts at detecting pneumonia, or Alzheimer’s—diseases in complex organs ranging from the eye to the heart.

The problems of overcrowded hospitals and overworked medical staff plague public healthcare systems like Britain’s NHS and lead to rising costs for private healthcare systems. Here, again, algorithms offer a tantalizing solution. How many of those doctor’s visits really need to happen? How many could be replaced by an interaction with an intelligent chatbot—especially if it can be combined with portable diagnostic tests, utilizing the latest in biotechnology? That way, unnecessary visits could be reduced, and patients could be diagnosed and referred to specialists more quickly without waiting for an initial consultation.

As ever with artificial intelligence algorithms, the aim is not to replace doctors, but to give them tools to reduce the mundane or repetitive parts of the job. With an AI that can examine thousands of scans in a minute, the “dull drudgery” is left to machines, and the doctors are freed to concentrate on the parts of the job that require more complex, subtle, experience-based judgement of the best treatments and the needs of the patient.

High Stakes
But, as ever with AI algorithms, there are risks involved with relying on them—even for tasks that are considered mundane. The problems of black-box algorithms that make inexplicable decisions are bad enough when you’re trying to understand why that automated hiring chatbot was unimpressed by your job interview performance. In a healthcare context, where the decisions made could mean life or death, the consequences of algorithmic failure could be grave.

A new paper in Science Translational Medicine, by Nicholson Price, explores some of the promises and pitfalls of using these algorithms in the data-rich medical environment.

Neural networks excel at churning through vast quantities of training data and making connections, absorbing the underlying patterns or logic for the system in hidden layers of linear algebra; whether it’s detecting skin cancer from photographs or learning to write in pseudo-Shakespearean script. They are terrible, however, at explaining the underlying logic behind the relationships that they’ve found: there is often little more than a string of numbers, the statistical “weights” between the layers. They struggle to distinguish between correlation and causation.

This raises interesting dilemmas for healthcare providers. The dream of big data in medicine is to feed a neural network on “huge troves of health data, finding complex, implicit relationships and making individualized assessments for patients.” What if, inevitably, such an algorithm proves to be unreasonably effective at diagnosing a medical condition or prescribing a treatment, but you have no scientific understanding of how this link actually works?

Too Many Threads to Unravel?
The statistical models that underlie such neural networks often assume that variables are independent of each other, but in a complex, interacting system like the human body, this is not always the case.

In some ways, this is a familiar concept in medical science—there are many phenomena and links which have been observed for decades but are still poorly understood on a biological level. Paracetamol is one of the most commonly-prescribed painkillers, but there’s still robust debate about how it actually works. Medical practitioners may be keen to deploy whatever tool is most effective, regardless of whether it’s based on a deeper scientific understanding. Fans of the Copenhagen interpretation of quantum mechanics might spin this as “Shut up and medicate!”

But as in that field, there’s a debate to be had about whether this approach risks losing sight of a deeper understanding that will ultimately prove more fruitful—for example, for drug discovery.

Away from the philosophical weeds, there are more practical problems: if you don’t understand how a black-box medical algorithm is operating, how should you approach the issues of clinical trials and regulation?

Price points out that, in the US, the “21st-Century Cures Act” allows the FDA to regulate any algorithm that analyzes images, or doesn’t allow a provider to review the basis for its conclusions: this could completely exclude “black-box” algorithms of the kind described above from use.

Transparency about how the algorithm functions—the data it looks at, and the thresholds for drawing conclusions or providing medical advice—may be required, but could also conflict with the profit motive and the desire for secrecy in healthcare startups.

One solution might be to screen algorithms that can’t explain themselves, or don’t rely on well-understood medical science, from use before they enter the healthcare market. But this could prevent people from reaping the benefits that they can provide.

Evaluating Algorithms
New healthcare algorithms will be unable to do what physicists did with quantum mechanics, and point to a track record of success, because they will not have been deployed in the field. And, as Price notes, many algorithms will improve as they’re deployed in the field for a greater amount of time, and can harvest and learn from the performance data that’s actually used. So how can we choose between the most promising approaches?

Creating a standardized clinical trial and validation system that’s equally valid across algorithms that function in different ways, or use different input or training data, will be a difficult task. Clinical trials that rely on small sample sizes, such as for algorithms that attempt to personalize treatment to individuals, will also prove difficult. With a small sample size and little scientific understanding, it’s hard to tell whether the algorithm succeeded or failed because it’s bad at its job or by chance.

Add learning into the mix and the picture gets more complex. “Perhaps more importantly, to the extent that an ideal black-box algorithm is plastic and frequently updated, the clinical trial validation model breaks down further, because the model depends on a static product subject to stable validation.” As Price describes, the current system for testing and validation of medical products needs some adaptation to deal with this new software before it can successfully test and validate the new algorithms.

Striking a Balance
The story in healthcare reflects the AI story in so many other fields, and the complexities involved perhaps illustrate why even an illustrious company like IBM appears to be struggling to turn its famed Watson AI into a viable product in the healthcare space.

A balance must be struck, both in our rush to exploit big data and the eerie power of neural networks, and to automate thinking. We must be aware of the biases and flaws of this approach to problem-solving: to realize that it is not a foolproof panacea.

But we also need to embrace these technologies where they can be a useful complement to the skills, insights, and deeper understanding that humans can provide. Much like a neural network, our industries need to train themselves to enhance this cooperation in the future.

Image Credit: Connect world / Shutterstock.com Continue reading →

Posted in Human Robots