Deepfakes, Disinformation, & Digital Dystopia
Envisioning a (very) dark digital future
Last week, there was a fascinating piece on “deepfake” commercial advertising featuring (non-consensual) digital renderings of prominent celebrities. From the WSJ:
Celebrity deepfakes are coming to advertising.
Among the recent entries: Last year, Russian telecommunications company MegaFon released a commercial in which a simulacrum of Hollywood legend Bruce Willis helps defuse a bomb.
Just last week, Elon Musk seemed to star in a marketing video from real-estate investment startup reAlpha Tech Corp.
And last month a promotional video for machine-learning firm Paperspace Co. showed talking semblances of the actors Tom Cruise and Leonardo DiCaprio.
None of these celebrities ever spent a moment filming these campaigns. In the cases of Messrs. Musk, Cruise and DiCaprio, they never even agreed to endorse the companies in question.
All the videos of digital simulations were created with so-called deepfake technology, which uses computer-generated renditions to make the Hollywood and business notables say and do things they never actually said or did.
Some of the ads are broad parodies, and the meshing of the digital to the analog in the best of cases might not fool an alert viewer. Even so, the growing adoption of deepfake software could eventually shape the industry in profound ways while creating new legal and ethical questions, experts said.
Human attention is an incredibly valuable resource. It’s also fundamentally zero-sum. At any given moment in time, there’s a virtually limitless number of things each one of us can shift our focus upon…to the determinant of everything else. This feels 100x the case now versus a mere two decades ago given the ubiquity of the smartphone, social media, and the internet. We’re living in a time defined by a vast wealth (overload) of information, and, yet, a poverty of attention. The commodification of attention/data forms the bedrock of the 21st-century high-tech economy.
The always astute Tim Wu, author of The Attention Merchants, breaks down the science behind this:
Every instant of every day we are bombarded with information. In fact, all complex organisms, especially those with brains, suffer from information overload. Our eyes and ears receive lights and sounds (respectively) across the spectrum of visible and audible wave lengths; our skin and the rest of our innervated parts send their own messages of sore muscles or cold feet. All told, every second, our senses transmit an estimated 11 million bits of information to our poor brains as if a giant fiber-optic cable were plugged directly into them, firing information at full bore.
Philosopher Vincent Hendricks, meanwhile, puts it into contemporary perspective:
In the 21st century, it’s not microchips, windmills or solar panels, but the attention of mankind which is of value…Attention is a scarce and valuable cognitive resource—but also a potentially highly profitable capital asset.
This is where fame comes into play. There’s a vast (yet ultimately finite) pool of collective human attention—and celebrities capture a disproportionate share of it. They are uniquely suited to breaking through the omnipresent background noise in an age of mass distraction and “stolen focus.” For better or worse (I would argue worse), celebrities, whether from the world of sports, music, fashion, cinema, politics, royalty, comedy, social media, or business, command vast swaths of public attention; a superpower that can be effectively parlayed into influence over mass-level political and commercial behavior. The value of the rich and famous—or their convincing simulacrum—as advertisers, spokespeople, and brand ambassadors is simply immense.
To my mind, all of the dynamics at play here—the cult of the celebrity, their value as agents for commercial/political actors, the hyper-competitive nature of the attention economy, the extreme deficit of public focus, the utter pervasiveness of social media—will only accelerate in the years ahead. Layered on top of that is a combination of rapidly improving technological sophistication and ever-lower barriers to entry (witness the popularity of programs like Midjourney and OpenAI)—all underpinned by radical advances in artificial intelligence and machine learning—that has made the creation and instantaneous dissemination of seemingly authentic deepfakes, manipulated audio/video, and AI-rendered art and design easier than ever. We’re only at the beginning of the story here. As the WSJ notes: “experts and practitioners say deepfake technology will become increasingly popular in advertising, because it can help brands and agencies produce more content faster while eliminating many of the expenses involved in production.”
Source: OpenAI
As it stands, there’s the thicket of issues around outdated regulations, intellectual property, licensing, and brand risk (for businesses and celebrities alike) raised by the impending mass-scale proliferation of artificial celebrity likenesses across the digital space.
One example worth honing in on concerns the ongoing fiasco around hip-hip mogul Ye’s (very authentic) track record of anti-Semitic comments—a debacle that underscores how quickly the relationship between corporate power and celebrity influence can turn toxic. Now imagine a not-so-distant future where prominent, consumer-facing brands—like Gap or Adidas—must respond, in real-time, to hyper-viral deepfake content of dubious origin and questionable authenticity showcasing a prominent celebrity brand rep doing or saying something outrageous, violent, or provocative.
The Epistemological Crisis & Democracy
It goes far beyond the deepfakes, however. A society that is perpetually distracted, willfully ignores trustworthy, high-quality information in favor of polemic, spectacle, and partisan combat, and lacks a shared semblance of reality and baseline of accepted facts and truths is, almost assuredly, a society that suffers from a toxic information supply chain. It is likely a society prone to supporting conspiratorial, conflict-minded demagogues, who, through a combination of anger, charisma, and force of personality, can monopolize public attention by harnessing raw emotions of fear, confusion, anger, distrust, and vengeance while acting as catalysts for further social polarization. Is it a society that, simply put, cannot operate as a healthy and inclusive liberal democracy.
Unfortunately, and against the backdrop of the ongoing digital communications revolution that’s empowered a broader range of media actors and influencers to compete for our collective attention than ever before, that state of affairs accurately describes large swaths of the democratic world today—not least of all the United States.
For some broader context here, I will block quote a few of the more illuminating bits of writing I’ve read recently on the topic.
From Zac Gershberg & Sean Illing (The Paradox of Democracy):
The paradox of democracy has reached its zenith with a flood of new digital communication platforms inundating free societies with instantaneous information and nonstop networked dialogue. The public sphere of the twenty-first century is more democratic and open than ever before. Political leaders communicate directly with their publics en masse; citizens provide immediate feedback and can publish or broadcast to mass audiences of their own. Yet the democratic openness of communication in the twenty-first century has destabilized democratic politics. There are no longer any controls on the flow of information. The public is now angry, distrustful of whether their representatives can even make sound decisions. That may be healthy from a democratic perspective, but with so much noise on social media and so many news outlets disseminating contradictory information, citizens are justifiably confused and cynical. Many find it easier to retreat into echo chambers and share misinformation than discern what’s real amid the chaos of the public sphere.
From The Commission on Information Disorder (Aspen Institute):
Information disorder is a crisis that exacerbates all other crises. When bad information becomes as prevalent, persuasive, and persistent as good information, it creates a chain reaction of harm.
Information disorder makes any health crisis more deadly. It slows down our response time on climate change. It undermines democracy. It creates a culture in which racist, ethnic, and gender attacks are seen as solutions, not problems. Today, mis- and disinformation have become a force multiplier for exacerbating our worst problems as a society. Hundreds of millions of people pay the price, every single day, for a world disordered by lies…
…We are in a crisis of trust and truth.
The problem extends well beyond state-sponsored disinformation, or health scams promoting miracle cures; it is rooted in broader challenges facing the nation—from increasing income inequality, to decreasing levels of public trust in institutions, to the constant churning cycle of news and information, to the splintering of media writ large, to the explosion of social media. Combined, these enormous changes are fertile ground for the seeds of information disorder.
From David Roberts (Vox):
The US is experiencing a deep epistemic breach, a split not just in what we value or want, but in who we trust, how we come to know things, and what we believe we know — what we believe exists, is true, has happened and is happening.
America’s hyper-polarized zero-sum political and cultural climate is both a cause and effect of what Roberts terms “tribal epistemology”—conditions wherein “information is evaluated based not on conformity to common standards of evidence or correspondence to a common understanding of the world, but on whether it supports the tribe’s values and goals and is vouchsafed by tribal leaders. “Good for our side” and “true” begin to blur into one.”
Sadly, I think it’s safe to say that epistemic breach, information disorder, hyper-polarization, and democratic decay—all underpinned by radical changes in digital communications technologies—provide accurate, albeit highly depressing, descriptions of the world we now inhabit.
We’re seeing the consequences of this severely degraded information environment play out all around us, all the time. It feels perhaps the most acute here in the US, where the two sides of the political divide have effectively hived off into separate (and largely incompatible) information ecosystems. On virtually any hot-button issue of the day it often feels as if we’re occupying totally separate versions of reality; reflected in massive partisan deltas on vaccines, COVID, impeachment, January 6, election fraud, and climate change.
Nor is this a strictly US phenomenon. Further afield, in places like Brazil, India, and the Philippines (all of which can be accurately deemed faltering democracies), the digital media space is awash in outright lies, misinformation, hate speech, conspiracies, and distorted renditions of national history. In all three countries, reactionary political movements have proven remarkably adept at harnessing the power of platforms like TikTok, YouTube, and Facebook—often with the support and amplification of well-known celebrities and social media influencers—to push out their message and build up loyal support bases.
Deepfakes & Dystopia
This all rounds back to the deepfakes. Setting aside the concerns over brand reputation, celebrity licensing, and obvious political lampooning (see below), their growing prominence raises alarming—possibly existential—questions over truth, trust, authenticity, and security. In the big picture, we’re moving full-steam ahead into a golden age of digital manipulation—defined by widespread access to quality technological tools for creating/altering audio, video, and imagery—that will massively challenge the human psyche’s capacity to accurately separate the fake from the authentic and fiction from reality.
Her’’s Robert Chesney and Danielle Citron, writing for Foreign Affairs, assessing what that may all come to look like:
Thanks to the rise of “deepfakes”—highly realistic and difficult-to-detect digital manipulations of audio or video—it is becoming easier than ever to portray someone saying or doing something he or she never said or did. Worse, the means to create deepfakes are likely to proliferate quickly, producing an ever-widening circle of actors capable of deploying them for political purposes. Disinformation is an ancient art, of course, and one with a renewed relevance today. But as deepfake technology develops and spreads, the current disinformation wars may soon look like the propaganda equivalent of the era of swords and shields.
Again and again and again, we’re seeing early indications of exactly how challenging all of this is going to be to our baseline of shared reality, to say nothing of the torrent of racist hate speech already coursing through “mainstream” social media platforms:
So where does that leave us?
Takeaway one: the tools of digital connectivity—so effective at capturing our collective attention—have already helped splinter our common conception of reality. Given how poorly Western societies have handled the global information disorder to this point, I see minimal grounds for optimism that this golden age of digital manipulation will herald anything less than dire ramifications going forward.
We already know that shocking, scandalous, and misleading content—especially when it reinforces partisan priors—is extremely potent when it comes to breaking through in the attention marketplace; a phenomenon that spikes in the run-up to contentious elections. From Buzzfeed’s 2016 election analysis:
In the final three months of the US presidential campaign, the top-performing fake election news stories on Facebook generated more engagement than the top stories from major news outlets such as the New York Times, Washington Post, Huffington Post, NBC News, and others, a BuzzFeed News analysis has found.
During these critical months of the campaign, 20 top-performing false election stories from hoax sites and hyperpartisan blogs generated 8,711,000 shares, reactions, and comments on Facebook.
Within the same time period, the 20 best-performing election stories from 19 major news websites generated a total of 7,367,000 shares, reactions, and comments on Facebook.
Up until those last three months of the campaign, the top election content from major outlets had easily outpaced that of fake election news on Facebook. Then, as the election drew closer, engagement for fake content on Facebook skyrocketed and surpassed that of the content from major news outlets.
We know erroneous beliefs—once they’ve firmly taken root—are extremely hard to shake. As the saying (often misattributed to Mark Twain) famously goes: “a lie travels halfway around the world before the truth puts on its shoes. By the time the truth catches up, it’s too late.”
Given the combination of minimal barriers to entry, improving quality, minimal deterrence, unclear safeguards mechanisms, and the capacity for near-instantaneous dissemination, I can see a potential near future in which trolls and rabidly partisan political actors will have a maximal incentive to “flood the zone with shit,” in the form of outright deepfakes and other forms of audio/visual distortion, in hopes of grabbing attention, capturing mindshare, and converting it into social and political influence.
Takeaway two: all of this raises extremely probing questions around enforcement, liability, and brand risk for the dominant platforms of our digital age. The Facebooks, Twitters, YouTubes, and TikToks of the world are not only the primary mediums through which misinformation spreads and partisan conflict plays out but also drivers of massive political controversy themselves given their outsized role in disseminating and regulating the flow of information and shaping public opinion. Many of the most contentious debates in US politics today stem from differing interpretations of their motives, obligations, and broader social responsibilities. Unsurprisingly, social media’s credibility in the eyes of the general public now lies in tatters and they are a repository of anger over everything that ails the democratic world. Per the Pew Research Center:
About two-thirds of Americans (64%) say social media have a mostly negative effect on the way things are going in the country today, according to a Pew Research Center survey of U.S. adults conducted July 13-19, 2020. Just one-in-ten Americans say social media sites have a mostly positive effect on the way things are going, and one-quarter say these platforms have a neither positive nor negative effect.
Those who have a negative view of the impact of social media mention, in particular, misinformation and the hate and harassment they see on social media. They also have concerns about users believing everything they see or read – or not being sure about what to believe. Additionally, they bemoan social media’s role in fomenting partisanship and polarization, the creation of echo chambers, and the perception that these platforms oppose President Donald Trump and conservatives.
I think there’s an obvious Goldilocks problem here. While there is a wellspring of cross-partisan anger directed at the major social media platforms, past that the consensus shatters. As far as I can see, there are no easy or straightforward solutions because—at the risk of sounding reductive—liberals and conservatives are having fundamentally different and largely incompatible conversations about the underlying problems and necessary fixes. Whatever these platforms do, a ton of people—from across the political spectrum—will inevitably come away furious and distrustful. Will Oremus succinctly captures this toxic social faultline:
What people can and can’t say online — and the role of Big Tech in making those calls — has emerged as a critical fault line in American politics. The left cries for content moderation to tamp down disinformation, racism and misogyny. The right cries censorship and demands the right to free speech.
As does this recent Pew polling data:
Source: Pew
It’s worth recalling the case of Facebook, which came under heavy fire over both its failure to remove a digitally doctored video of House Speaker Nancy Pelosi apparently slurring her words and appearing incoherent (laxity) and its decision to restrict sharing of a controversial NY Post article (censoriousness) detailing the contents of Hunter Biden’s laptop in the lead-up to the 2020 election. In both instances, CEO Mark Zuckerberg admitted the company was too slow/got it wrong and apologized. I imagine his assurances fell mostly on deaf ears. While Facebook notably banned AI-altered deepfake videos in the aftermath of the Pelosi incident, the platform remains highly susceptible to less technologically sophisticated forms of audio and visual manipulation. FB’s handling of the NY Post controversy remains a major source of outrage on the American right.
It’s not just about governments either. From a commercial standpoint, the big risk here—and it’s something at the forefront of the public conversation following Elon Musk’s Twitter acquisition—is the relationship between platforms and the advertisers who serve as their major revenue drivers.
Here’s the The Hollywood Reporter:
Ads made up 89 percent of Twitter’s 2021 revenue, according to its annual report.
But major advertisers, who have traditionally been small-c conservative when it comes to where they spend their dollars, are beginning to raise concerns.
General Motors, one of the largest ad buyers in the country, said that it is “temporarily pausing” its paid advertising on Twitter, adding that “we are engaging with Twitter to understand the direction of the platform under their new ownership.”
A source close to a major national ad buyer tells The Hollywood Reporter that for companies like GM, brand safety is paramount, with TV networks and streaming services like Hulu (not to mention social platforms like Snapchat) able to guarantee brand-safe environments around their ads.
Musk has promised to blow up Twitter, promising both unfettered free speech but also a “warm and welcoming” environment.
I can think of a plethora of reasons why newer “free speech” platforms like Gettr, Parler, and Truth Social—all of whom have built their brand identities around a laissez-faire approach to content moderation—have failed to gain traction, but a primary reason is this: from the standpoint of risk-averse brands and non-rightwing users, they are deeply unpleasant hellscapes awash in misinformation, hate speech, and conspiracy. Unless you fall on the right/far-right of the ideological spectrum, they are not enjoyable places to focus your limited bank of attention.
If we are indeed heading for a Cambrian Explosion of digital manipulation, then all the major platforms will find themselves at great risk of becoming overwhelmed and further devolving into the kinds of toxic hellscapes commonly associated with the smaller fringe players. A saturation of attention-grabbing but inauthentic content will turn away swaths of users and pile pressure on brand-protective advertisers to shift their marketing dollars elsewhere while avoiding negative associations with the social networks deemed responsible for our deepening reality crisis.
Takeaway three: all of the risks at play here are exacerbated in the broader international context. Social networks are now ubiquitous across the globe, including in many areas lacking the state capacity, social cohesion, digital media literacy, and democratic resiliency commonly found in the developed world. We’ve already seen the major social media platforms flail in the face of the global information disorder in far-flung places like Myanmar, Ethiopia, India, and the Philippines—with deadly results. Given their track record to this point, it’s hard to envision they’ll have sufficient control mechanisms in place to stem the coming tsunami of easily-made content that is provocative, popular, attention-grabbing, and fake; especially in instances where host governments may have a vested interest in weaponizing inflammatory fake content to suit their political ends.
The major problem here is that addressing these challenges will, by necessity, require a much more aggressive approach to content moderation and oversight. That will, in turn, require massive expenditures of resources and invariably upset a large swath of the general public already alarmed over excessive censoriousness and the extensive power of big tech. While the various social networks will draw bright redlines around banning and policing deepfakes and outright fabrications, there will be loopholes, grey areas, and reputation-draining scandals that will turn off advertisers and users alike. As the quantity and quality of inauthentic digital content explode, one can easily envision a future wherein more and more of what comes across our screens (and drains more and more out of our finite pool of individual and collective attention) will be of dubious authenticity; rendering the public information space increasingly confused, angry, divided, distracted, apathetic, and pessimistic.
While the goofy celebrity videos may be a humorous distraction, they may well foretell something much darker in the days ahead.