Tag Archives: piece
The best storytellers react to their audience. They look for smiles, signs of awe, or boredom; they simultaneously and skillfully read both the story and their sitters. Kevin Brooks, a seasoned storyteller working for Motorola’s Human Interface Labs, explains, “As the storyteller begins, they must tune in to… the audience’s energy. Based on this energy, the storyteller will adjust their timing, their posture, their characterizations, and sometimes even the events of the story. There is a dialog between audience and storyteller.”
Shortly after I read the script to Melita, the latest virtual reality experience from Madrid-based immersive storytelling company Future Lighthouse, CEO Nicolas Alcalá explained to me that the piece is an example of “reactive content,” a concept he’s been working on since his days at Singularity University.
For the first time in history, we have access to technology that can merge the reactive and affective elements of oral storytelling with the affordances of digital media, weaving stunning visuals, rich soundtracks, and complex meta-narratives in a story arena that has the capability to know you more intimately than any conventional storyteller could.
It’s no understatement to say that the storytelling potential here is phenomenal.
In short, we can refer to content as reactive if it reads and reacts to users based on their body rhythms, emotions, preferences, and data points. Artificial intelligence is used to analyze users’ behavior or preferences to sculpt unique storylines and narratives, essentially allowing for a story that changes in real time based on who you are and how you feel.
The development of reactive content will allow those working in the industry to go one step further than simply translating the essence of oral storytelling into VR. Rather than having a narrative experience with a digital storyteller who can read you, reactive content has the potential to create an experience with a storyteller who knows you.
This means being able to subtly insert minor personal details that have a specific meaning to the viewer. When we talk to our friends we often use experiences we’ve shared in the past or knowledge of our audience to give our story as much resonance as possible. Targeting personal memories and aspects of our lives is a highly effective way to elicit emotions and aid in visualizing narratives. When you can do this with the addition of visuals, music, and characters—all lifted from someone’s past—you have the potential for overwhelmingly engaging and emotionally-charged content.
Future Lighthouse inform me that for now, reactive content will rely primarily on biometric feedback technology such as breathing, heartbeat, and eye tracking sensors. A simple example would be a story in which parts of the environment or soundscape change in sync with the user’s heartbeat and breathing, or characters who call you out for not paying attention.
The next step would be characters and situations that react to the user’s emotions, wherein algorithms analyze biometric information to make inferences about states of emotional arousal (“why are you so nervous?” etc.). Another example would be implementing the use of “arousal parameters,” where the audience can choose what level of “fear” they want from a VR horror story before algorithms modulate the experience using information from biometric feedback devices.
The company’s long-term goal is to gather research on storytelling conventions and produce a catalogue of story “wireframes.” This entails distilling the basic formula to different genres so they can then be fleshed out with visuals, character traits, and soundtracks that are tailored for individual users based on their deep data, preferences, and biometric information.
The development of reactive content will go hand in hand with a renewed exploration of diverging, dynamic storylines, and multi-narratives, a concept that hasn’t had much impact in the movie world thus far. In theory, the idea of having a story that changes and mutates is captivating largely because of our love affair with serendipity and unpredictability, a cultural condition theorist Arthur Kroker refers to as the “hypertextual imagination.” This feeling of stepping into the unknown with the possibility of deviation from the habitual translates as a comforting reminder that our own lives can take exciting and unexpected turns at any moment.
The inception of the concept into mainstream culture dates to the classic Choose Your Own Adventure book series that launched in the late 70s, which in its literary form had great success. However, filmic takes on the theme have made somewhat less of an impression. DVDs like I’m Your Man (1998) and Switching (2003) both use scene selection tools to determine the direction of the storyline.
A more recent example comes from Kino Industries, who claim to have developed the technology to allow filmmakers to produce interactive films in which viewers can use smartphones to quickly vote on which direction the narrative takes at numerous decision points throughout the film.
The main problem with diverging narrative films has been the stop-start nature of the interactive element: when I’m immersed in a story I don’t want to have to pick up a controller or remote to select what’s going to happen next. Every time the audience is given the option to take a new path (“press this button”, “vote on X, Y, Z”) the narrative— and immersion within that narrative—is temporarily halted, and it takes the mind a while to get back into this state of immersion.
Reactive content has the potential to resolve these issues by enabling passive interactivity—that is, input and output without having to pause and actively make decisions or engage with the hardware. This will result in diverging, dynamic narratives that will unfold seamlessly while being dependent on and unique to the specific user and their emotions. Passive interactivity will also remove the game feel that can often be a symptom of interactive experiences and put a viewer somewhere in the middle: still firmly ensconced in an interactive dynamic narrative, but in a much subtler way.
While reading the Melita script I was particularly struck by a scene in which the characters start to engage with the user and there’s a synchronicity between the user’s heartbeat and objects in the virtual world. As the narrative unwinds and the words of Melita’s character get more profound, parts of the landscape, which seemed to be flashing and pulsating at random, come together and start to mimic the user’s heartbeat.
In 2013, Jane Aspell of Anglia Ruskin University (UK) and Lukas Heydrich of the Swiss Federal Institute of Technology proved that a user’s sense of presence and identification with a virtual avatar could be dramatically increased by syncing the on-screen character with the heartbeat of the user. The relationship between bio-digital synchronicity, immersion, and emotional engagement is something that will surely have revolutionary narrative and storytelling potential.
Image Credit: Tithi Luadthong / Shutterstock.com Continue reading
When it comes to biomolecules, RNA doesn’t get a lot of love.
Maybe you haven’t even heard of the silent workhorse. RNA is the cell’s de facto translator: like a game of telephone, RNA takes DNA’s genetic code to a cellular factory called ribosomes. There, the cell makes proteins based on RNA’s message.
But RNA isn’t just a middleman. It controls what proteins are formed. Because proteins wiz around the cell completing all sorts of important processes, you can say that RNA is the gatekeeper: no RNA message, no proteins, no life.
In a new study published in Nature, RNA finally took center stage. By adding bits of genetic material to the E. Coli bacteria, a team of biohackers at the Wyss Institute hijacked the organism’s RNA messengers so that they only spring into action following certain inputs.
The result? A bacterial biocomputer capable of performing 12-input logic operations—AND, OR, and NOT—following specific inputs. Rather than outputting 0s and 1s, these biocircuits produce results based on the presence or absence of proteins and other molecules.
“It’s the greatest number of inputs in a circuit that a cell has been able to process,” says study author Dr. Alexander Green at Arizona State University. “To be able to analyze those signals and make a decision is the big advance here.”
When given a specific set of inputs, the bacteria spit out a protein that made them glow neon green under fluorescent light.
But synthetic biology promises far more than just a party trick—by tinkering with a cell’s RNA repertoire, scientists may one day coax them to photosynthesize, produce expensive drugs on the fly, or diagnose and hunt down rogue tumor cells.
Illustration of an RNA-based ‘ribocomputing’ device that makes logic-based decisions in living cells. The long gate RNA (blue) detects the binding of an input RNA (red). The ribosome (purple/mauve) reads the gate RNA to produce an output protein. Image Credit: Alexander Green / Arizona State University
The software of life
This isn’t the first time that scientists hijacked life’s algorithms to reprogram cells into nanocomputing systems. Previous work has already introduced to the world yeast cells that can make anti-malaria drugs from sugar or mammalian cells that can perform Boolean logic.
Yet circuits with multiple inputs and outputs remain hard to program. The reason is this: synthetic biologists have traditionally focused on snipping, fusing, or otherwise arranging a cell’s DNA to produce the outcomes they want.
But DNA is two steps removed from proteins, and tinkering with life’s code often leads to unexpected consequences. For one, the cell may not even accept and produce the extra bits of DNA code. For another, the added code, when transformed into proteins, may not act accordingly in the crowded and ever-changing environment of the cell.
What’s more, tinkering with one gene is often not enough to program an entirely new circuit. Scientists often need to amp up or shut down the activity of multiple genes, or multiple biological “modules” each made up of tens or hundreds of genes.
It’s like trying to fit new Lego pieces in a specific order into a room full of Lego constructions. Each new piece has the potential to wander off track and click onto something it’s not supposed to touch.
Getting every moving component to work in sync—as you might have guessed—is a giant headache.
The RNA way
With “ribocomputing,” Green and colleagues set off to tackle a main problem in synthetic biology: predictability.
Named after the “R (ribo)” in “RNA,” the method grew out of an idea that first struck Green back in 2012.
“The synthetic biological circuits to date have relied heavily on protein-based regulators that are difficult to scale up,” Green wrote at the time. We only have a limited handful of “designable parts” that work well, and these circuits require significant resources to encode and operate, he explains.
RNA, in comparison, is a lot more predictable. Like its more famous sibling DNA, RNA is composed of units that come in four different flavors: A, G, C, and U. Although RNA is only single-stranded, rather than the double helix for which DNA is known for, it can bind short DNA-like sequences in a very predictable manner: Gs always match up with Cs and As always with Us.
Because of this predictability, it’s possible to design RNA components that bind together perfectly. In other words, it reduces the chance that added RNA bits might go rogue in an unsuspecting cell.
Normally, once RNA is produced it immediately rushes to the ribosome—the cell’s protein-building factory. Think of it as a constantly “on” system.
However, Green and his team found a clever mechanism to slow them down. Dubbed the “toehold switch,” it works like this: the artificial RNA component is first incorporated into a chain of A, G, C, and U folded into a paperclip-like structure.
This blocks the RNA from accessing the ribosome. Because one RNA strand generally maps to one protein, the switch prevents that protein from ever getting made.
In this way, the switch is set to “off” by default—a “NOT” gate, in Boolean logic.
To activate the switch, the cell needs another component: a “trigger RNA,” which binds to the RNA toehold switch. This flips it on: the RNA grabs onto the ribosome, and bam—proteins.
String a few RNA switches together, with the activity of each one relying on the one before, and it forms an “AND” gate. Alternatively, if the activity of each switch is independent, that’s an “OR” gate.
“Basically, the toehold switches performed so well that we wanted to find a way to best exploit them for cellular applications,” says Green. They’re “kind of the equivalent of your first transistors,” he adds.
Once the team optimized the designs for different logic gates, they carefully condensed the switches into “gate RNA” molecules. These gate RNAs contain both codes for proteins and the logic operations needed to kickstart the process—a molecular logic circuit, so to speak.
If you’ve ever played around with an Arduino-controlled electrical circuit, you probably know the easiest way to test its function is with a light bulb.
That’s what the team did here, though with a biological bulb: green fluorescent protein, a light-sensing protein not normally present in bacteria that—when turned on—makes the microbugs glow neon green.
In a series of experiments, Green and his team genetically inserted gate RNAs into bacteria. Then, depending on the type of logical function, they added different combinations of trigger RNAs—the inputs.
When the input RNA matched up with its corresponding gate RNA, it flipped on the switch, causing the cell to light up.
Their most complex circuit contained five AND gates, five OR gates, and two NOTs—a 12-input ribocomputer that functioned exactly as designed.
That’s quite the achievement. “Everything is interacting with everything else and there are a million ways those interactions could flip the switch on accident,” says RNA researcher Dr. Julies Lucks at Northwestern University.
The specificity is thanks to RNA, the authors explain. Because RNAs bind to others so predictably, we can now design massive libraries of gate and trigger units to mix-and-match into all types of nano-biocomputers.
Although the technology doesn’t have any immediate applications, the team has high hopes.
For the first time, it’s now possible to massively scale up the process of programming new circuits into living cells. We’ve expanded the library of available biocomponents that can be used to reprogram life’s basic code, the authors say.
What’s more, when freeze-dried onto a piece of tissue paper, RNA keeps very well. We could potentially print RNA toehold switches onto paper that respond to viruses or to tumor cells, the authors say, essentially transforming the technology into highly accurate diagnostic platforms.
But Green’s hopes are even wilder for his RNA-based circuits.
“Because we’re using RNA, a universal molecule of life, we know these interactions can also work in other cells, so our method provides a general strategy that could be ported to other organisms,” he says.
Ultimately, the hope is to program neural network-like capabilities into the body’s other cells.
Imagine cells endowed with circuits capable of performing the kinds of computation the brain does, the authors say.
Perhaps one day, synthetic biology will transform our own cells into fully programmable entities, turning us all into biological cyborgs from the inside. How wild would that be?
Image Credit: Wyss Institute at Harvard University Continue reading