Shoshana Zuboff: Secrets of Surveillance Capitalism

Google surpassed Apple as the world’s most highly valued company in January for the first time since 2010. (Back then each company was worth less than 200 billion. Now each is valued at well over 500 billion.) While Google’s new lead lasted only a few days, the company’s success has implications for everyone who lives within the reach of the Internet. Why? Because Google is ground zero for a wholly new subspecies of capitalism in which profits derive from the unilateral surveillance and modification of human behavior. This is a new surveillance capitalism that is unimaginable outside the inscrutable high velocity circuits of Google’s digital universe, whose signature feature is the Internet and its successors. While the world is riveted by the showdown between Apple and the FBI, the real truth is that the surveillance capabilities being developed by surveillance capitalists are the envy of every state security agency. What are the secrets of this new capitalism, how do they produce such staggering wealth, and how can we protect ourselves from its invasive power?

(German Version: „Wie wir Sklaven von Google wurden“ von Shoshana Zuboff)

“Most Americans realize that there are two groups of people who are monitored regularly as they move about the country. The first group is monitored involuntarily by a court order requiring that a tracking device be attached to their ankle. The second group includes everyone else…”

Some will think that this statement is certainly true. Others will worry that it could become true. Perhaps some think it’s ridiculous. It’s not a quote from a dystopian novel, a Silicon Valley executive, or even an NSA official. These are the words of an auto insurance industry consultant intended as a defense of “automotive telematics” and the astonishingly intrusive surveillance capabilities of the allegedly benign systems that are already in use or under development. It’s an industry that has been notoriously exploitative toward customers and has had obvious cause to be anxious about the implications of self-driving cars for its business model. Now, data about where we are, where we’re going, how we’re feeling, what we’re saying, the details of our driving, and the conditions of our vehicle are turning into beacons of revenue that illuminate a new commercial prospect. According to the industry literature, these data can be used for dynamic real-time driver behavior modification triggering punishments (real-time rate hikes, financial penalties, curfews, engine lock-downs) or rewards (rate discounts, coupons, gold stars to redeem for future benefits).

The goal: to change people’s actual behavior at scale

This is just one peephole, in one corner, of one industry, and the peepholes are multiplying like cockroaches. Among the many interviews I’ve conducted over the past three years, the Chief Data Scientist of a much-admired Silicon Valley company that develops applications to improve students’ learning told me, “The goal of everything we do is to change people’s actual behavior at scale. When people use our app, we can capture their behaviors, identify good and bad behaviors, and develop ways to reward the good and punish the bad. We can test how actionable our cues are for them and how profitable for us”.

The very idea of a functional, effective, affordable product as a sufficient basis for economic exchange is dying. The sports apparel company Under Armour is reinventing its products as wearable technologies. The CEO wants to be like Google. He says, "If it all sounds eerily like those ads that, because of your browsing history, follow you around the Internet, that's exactly the point--except Under Armour is tracking real behavior and the data is more specific… making people better athletes makes them need more of our gear.” The examples of this new logic are endless, from smart vodka bottles to Internet-enabled rectal thermometers and quite literally everything in between. A Goldman Sachs report calls it a “gold rush,” a race to “vast amounts of data.”

The assault on behavioral data

We’ve entered virgin territory here. The assault on behavioral data is so sweeping that it can no longer be circumscribed by the concept of privacy and its contests. This is a different kind of challenge now, one that threatens the existential and political canon of the modern liberal order defined by principles of self-determination that have been centuries, even millennia, in the making. I am thinking of matters that include, but are not limited to, the sanctity of the individual and the ideals of social equality; the development of identity, autonomy, and moral reasoning; the integrity of contract, the freedom that accrues to the making and fulfilling of promises; norms and rules of collective agreement; the functions of market democracy; the political integrity of societies; and the future of democratic sovereignty. In the fullness of time, we will look back on the establishment in Europe of the “Right to be Forgotten” and the EU’s more recent invalidation of the Safe Harbor doctrine as early milestones in a gradual reckoning with the true dimensions of this challenge.

There was a time when we laid responsibility for the assault on behavioral data at the door of the state and its security agencies. Later, we also blamed the cunning practices of a handful of banks, data brokers, and Internet companies. Some attribute the assault to an inevitable “age of big data,” as if it were possible to conceive of data born pure and blameless, data suspended in some celestial place where facts sublimate into truth.

Capitalism has been hijacked by surveillance

I’ve come to a different conclusion: The assault we face is driven in large measure by the exceptional appetites of a wholly new genus of capitalism, a systemic coherent new logic of accumulation that I call surveillance capitalism. Capitalism has been hijacked by a lucrative surveillance project that subverts the “normal” evolutionary mechanisms associated with its historical success and corrupts the unity of supply and demand that has for centuries, however imperfectly, tethered capitalism to the genuine needs of its populations and societies, thus enabling the fruitful expansion of market democracy.

Surveillance capitalism is a novel economic mutation bred from the clandestine coupling of the vast powers of the digital with the radical indifference and intrinsic narcissism of the financial capitalism and its neoliberal vision that have dominated commerce for at least three decades, especially in the Anglo economies. It is an unprecedented market form that roots and flourishes in lawless space. It was first discovered and consolidated at Google, then adopted by Facebook, and quickly diffused across the Internet. Cyberspace was its birthplace because, as Google/Alphabet Chairperson Eric Schmidt and his coauthor, Jared Cohen, celebrate on the very first page of their book about the digital age, “the online world is not truly bound by terrestrial laws…it’s the world’s largest ungoverned space.”

While surveillance capitalism taps the invasive powers of the Internet as the source of capital formation and wealth creation, it is now, as I have suggested, poised to transform commercial practice across the real world too. An analogy is the rapid spread of mass production and administration throughout the industrialized world in the early twentieth century, but with one major caveat. Mass production was interdependent with its populations who were its consumers and employees. In contrast, surveillance capitalism preys on dependent populations who are neither its consumers nor its employees and are largely ignorant of its procedures.

Internet access as a fundamental human right

We once fled to the Internet as solace and solution, our needs for effective life thwarted by the distant and increasingly ruthless operations of late twentieth century capitalism. In less than two decades after the Mosaic web browser was released to the public enabling easy access to the World Wide Web, a 2010 BBC poll found that 79% of people in 26 countries considered Internet access to be a fundamental human right. This is the Scylla and Charybdis of our plight. It is nearly impossible to imagine effective social participation ––from employment, to education, to healthcare–– without Internet access and know-how, even as these once flourishing networked spaces fall to a new and even more exploitative capitalist regime. It’s happened quickly and without our understanding or agreement. This is because the regime’s most poignant harms, now and later, have been difficult to grasp or theorize, blurred by extreme velocity and camouflaged by expensive and illegible machine operations, secretive corporate practices, masterful rhetorical misdirection, and purposeful cultural misappropriation.

Taming this new force depends upon careful naming. This symbiosis of naming and taming is vividly illustrated in the recent history of HIV research, and I offer it as analogy. For three decades scientists aimed to create a vaccine that followed the logic of earlier cures, training the immune system to produce neutralizing antibodies, but mounting data revealed unanticipated behaviors of the HIV virus that defy the patterns of other infectious diseases.

HIV research as analogy

The tide began to turn at the International AIDS Conference in 2012, when new strategies were presented that rely on a close understanding of the biology of rare HIV carriers whose blood produces natural antibodies. Research began to shift toward methods that reproduce this self-vaccinating response. A leading researcher announced, “We know the face of the enemy now, and so we have some real clues about how to approach the problem.”

The point for us is that every successful vaccine begins with a close understanding of the enemy disease. We tend to rely on mental models, vocabularies, and tools distilled from past catastrophes. I am thinking of the twentieth century’s totalitarian nightmares or the monopolistic predations of Gilded Age capitalism. But the vaccines we’ve developed to fight those earlier threats are not sufficient or even appropriate for the novel challenges we face. It’s like we’re hurling snowballs at a smooth marble wall only to watch them slide down its façade, leaving nothing but a wet smear: a fine paid here, an operational detour there.

An evolutionary dead-end

I want to say plainly that surveillance capitalism is not the only current modality of information capitalism, nor is it the only possible model for the future. Its fast track to capital accumulation and rapid institutionalization, however, has made it the default model of information capitalism. The questions I pose are these: Will surveillance capitalism become the dominant logic of accumulation in our time or, will it be an evolutionary dead-end –– a toothed bird in capitalism’s longer journey? What will an effective vaccine entail?

A cure depends upon many individual, social, and legal adaptations, but I am convinced that fighting the “enemy disease” cannot begin without a fresh grasp of the novel mechanisms that account for surveillance capitalism’s successful transformation of investment into capital. This has been one focus of my work in a new book, Master or Slave: The Fight for the Soul of Our Information Civilization, which will be published early next year. In the short space of this essay, I’d like to share some of my thoughts on this problem.

Fortune telling and selling

New economic logics and their commercial models are discovered by people in a time and place and then perfected through trial and error. Ford discovered and systematized mass production. General Motors institutionalized mass production as a new phase of capitalist development with the discovery and perfection of large-scale administration and professional management. In our time, Google is to surveillance capitalism what Ford and General Motors were to mass-production and managerial capitalism a century ago: discoverer, inventor, pioneer, role model, lead practitioner, and diffusion hub.

Specifically, Google is the mothership and ideal type of a new economic logic based on fortune telling and selling, an ancient and eternally lucrative craft that has exploited the human confrontation with uncertainty from the beginning of the human story. Paradoxically, the certainty of uncertainty is both an enduring source of anxiety and one of our most fruitful facts. It produced the universal need for social trust and cohesion, systems of social organization, familial bonding, and legitimate authority, the contract as formal recognition of reciprocal rights and obligations, and the theory and practice of what we call “free will.” When we eliminate uncertainty, we forfeit the human replenishment that attaches to the challenge of asserting predictability in the face of an always-unknown future in favor of the blankness of perpetual compliance with someone else’s plan.

Only incidentally related to advertising

Most people credit Google’s success to its advertising model. But the discoveries that led to Google’s rapid rise in revenue and market capitalization are only incidentally related to advertising. Google’s success derives from its ability to predict the future – specifically the future of behavior. Here is what I mean:

From the start, Google had collected data on users’ search-related behavior as a byproduct of query activity. Back then, these data logs were treated as waste, not even safely or methodically stored. Eventually, the young company came to understand that these logs could be used to teach and continuously improve its search engine.

The problem was this: Serving users with amazing search results “used up” all the value that users created when they inadvertently provided behavioral data. It’s a complete and self-contained process in which users are ends-in-themselves. All the value that users create is reinvested in the user experience in the form of improved search. In this cycle, there was nothing left over for Google to turn into capital. As long as the effectiveness of the search engine needed users’ behavioral data about as much as users needed search, charging a fee for service was too risky. Google was cool, but it wasn’t yet capitalism –– just one of many Internet startups that boasted “eyeballs” but no revenue.

Shift in the use of behavioral data

The year 2001 brought the dot.com bust and mounting investor pressures at Google. Back then advertisers selected the search term pages for their displays. Google decided to try and boost ad revenue by applying its already substantial analytical capabilities to the challenge of increasing an ad’s relevance to users –– and thus its value to advertisers. Operationally this meant that Google would finally repurpose its growing cache of behavioral data. Now the data would also be used to match ads with keywords, exploiting subtleties that only its access to behavioral data, combined with its analytical capabilities, could reveal.

It’s now clear that this shift in the use of behavioral data was an historic turning point. Behavioral data that were once discarded or ignored were rediscovered as what I call behavioral surplus. Google’s dramatic success in “matching” ads to pages revealed the transformational value of this behavioral surplus as a means of generating revenue and ultimately turning investment into capital. Behavioral surplus was the game-changing zero-cost asset that could be diverted from service improvement toward a genuine market exchange. Key to this formula, however, is the fact that this new market exchange was not an exchange with users but rather with other companies who understood how to make money from bets on users’ future behavior. In this new context, users were no longer an end-in-themselves. Instead they became a means to profits in a new kind of marketplace in which users are neither buyers nor sellers nor products. Users are the source of free raw material that feeds a new kind of manufacturing process.

While these facts are known, their significance has not been fully appreciated or adequately theorized. What just happened was the discovery of a surprisingly profitable commercial equation –– a series of lawful relationships that were gradually institutionalized in the sui generis economic logic of surveillance capitalism. It’s like a newly sighted planet with its own physics of time and space, its sixty-seven hour days, emerald sky, inverted mountain ranges, and dry water.

A parasitic form of profit

The equation: First, the push for more users and more channels, services, devices, places, and spaces is imperative for access to an ever-expanding range of behavioral surplus. Users are the human nature-al resource that provides this free raw material. Second, the application of machine learning, artificial intelligence, and data science for continuous algorithmic improvement constitutes an immensely expensive, sophisticated, and exclusive twenty-first century “means of production.” Third, the new manufacturing process converts behavioral surplus into prediction products designed to predict behavior now and soon. Fourth, these prediction products are sold into a new kind of meta-market that trades exclusively in future behavior. The better (more predictive) the product, the lower the risks for buyers, and the greater the volume of sales. Surveillance capitalism’s profits derive primarily, if not entirely, from such markets for future behavior.

While advertisers have been the dominant buyers in the early history of this new kind of marketplace, there is no substantive reason why such markets should be limited to this group. The already visible trend is that any actor with an interest in monetizing probabilistic information about our behavior and/or influencing future behavior can pay to play in a marketplace where the behavioral fortunes of individuals, groups, bodies, and things are told and sold. This is how in our own lifetimes we observe capitalism shifting under our gaze: once profits from products and services, then profits from speculation, and now profits from surveillance. This latest mutation may help explain why the explosion of the digital has failed, so far, to decisively impact economic growth, as so many of its capabilities are diverted into a fundamentally parasitic form of profit.

Unoriginal Sin

The significance of behavioral surplus was quickly camouflaged, both at Google and eventually throughout the Internet industry, with labels like “digital exhaust,” “digital breadcrumbs,” and so on. These euphemisms for behavioral surplus operate as ideological filters, in exactly the same way that the earliest maps of the North American continent labeled whole regions with terms like “heathens,” “infidels,” “idolaters,” “primitives,” “vassals,” or “rebels.” On the strength of those labels, native peoples, their places and claims, were erased from the invaders’ moral and legal equations, legitimating their acts of taking and breaking in the name of Church and Monarchy.

We are the native peoples now whose tacit claims to self-determination have vanished from the maps of our own behavior. They are erased in an astonishing and audacious act of dispossession by surveillance that claims its right to ignore every boundary in its thirst for knowledge of and influence over the most detailed nuances of our behavior. For those who wondered about the logical completion of the global processes of commodification, the answer is that they complete themselves in the dispossession of our intimate quotidian reality, now reborn as behavior to be monitored and modified, bought and sold.

The process that began in cyberspace mirrors the nineteenth century capitalist expansions that preceded the age of imperialism. Back then, as Hannah Arendt described it in The Origins of Totalitarianism, “the so-called laws of capitalism were actually allowed to create realities” as they traveled to less developed regions where law did not follow. “The secret of the new happy fulfillment,” she wrote, “was precisely that economic laws no longer stood in the way of the greed of the owning classes.” There, “money could finally beget money,” without having to go “the long way of investment in production…”

“The original sin of simple robbery”

For Arendt, these foreign adventures of capital clarified an essential mechanism of capitalism. Marx had developed the idea of “primitive accumulation” as a big-bang theory –– Arendt called it “the original sin of simple robbery” –– in which the taking of lands and natural resources was the foundational event that enabled capital accumulation and the rise of the market system. The capitalist expansions of the 1860s and 1870s demonstrated, Arendt wrote, that this sort of original sin had to be repeated over and over, “lest the motor of capital accumulation suddenly die down.”

In his book The New Imperialism, geographer and social theorist David Harvey built on this insight with his notion of “accumulation by dispossession.” “What accumulation by dispossession does,” he writes, “is to release a set of assets…at very low (and in some instances zero) cost. Overaccumulated capital can seize hold of such assets and immediately turn them to profitable use…It can also reflect attempts by determined entrepreneurs…to ‘join the system’ and seek the benefits of capital accumulation.”

Breakthrough into “the system”

The process by which behavioral surplus led to the discovery of surveillance capitalism exemplifies this pattern. It is the foundational act of dispossession for a new logic of capitalism built on profits from surveillance that paved the way for Google to become a capitalist enterprise. Indeed, in 2002, Google’s first profitable year, founder Sergey Brin relished his breakthrough into “the system”, as he told Levy,

Honestly, when we were still in the dot-com boom days, I felt like a schmuck. I had an Internet start- up — so did everybody else. It was unprofitable, like everybody else’s, and how hard is that? But when we became profitable, I felt like we had built a real business.”

Brin was a capitalist all right, but it was a mutation of capitalism unlike anything the world had seen.

Once we understand this equation, it becomes clear that demanding privacy from surveillance capitalists or lobbying for an end to commercial surveillance on the Internet is like asking Henry Ford to make each Model T by hand. It’s like asking a giraffe to shorten its neck or a cow to give up chewing. Such demands are existential threats that violate the basic mechanisms of the entity’s survival. How can we expect companies whose economic existence depends upon behavioral surplus to cease capturing behavioral data voluntarily? It’s like asking for suicide.

More behavioral surplus for Google

The imperatives of surveillance capitalism mean that there must always be more behavioral surplus for Google and others to turn into surveillance assets, master as prediction, sell into exclusive markets for future behavior, and transform into capital. At Google and its new holding company called Alphabet, for example, every operation and investment aims to increasing the harvest of behavioral surplus from people, bodies, things, processes, and places in both the virtual and the real world. This is how a sixty-seven hour day dawns and darkens in an emerald sky. Nothing short of a social revolt that revokes collective agreement to the practices associated with the dispossession of behavior will alter surveillance capitalism’s claim to manifest data destiny.

What is the new vaccine? We need to reimagine how to intervene in the specific mechanisms that produce surveillance profits and in so doing reassert the primacy of the liberal order in the twenty-first century capitalist project. In undertaking this challenge we must be mindful that contesting Google, or any other surveillance capitalist, on the grounds of monopoly is a 20th century solution to a 20th century problem that, while still vitally important, does not necessarily disrupt surveillance capitalism’s commercial equation. We need new interventions that interrupt, outlaw, or regulate 1) the initial capture of behavioral surplus, 2) the use of behavioral surplus as free raw material, 3) excessive and exclusive concentrations of the new means of production, 4) the manufacture of prediction products, 5) the sale of prediction products, 6) the use of prediction products for third-order operations of modification, influence, and control, and 5) the monetization of the results of these operations. This is necessary for society, for people, for the future, and it is also necessary to restore the healthy evolution of capitalism itself.

A coup from above

In the conventional narrative of the privacy threat, institutional secrecy has grown, and individual privacy rights have been eroded. But that framing is misleading, because privacy and secrecy are not opposites but rather moments in a sequence. Secrecy is an effect; privacy is the cause. Exercising one’s right to privacy produces choice, and one can choose to keep something secret or to share it. Privacy rights thus confer decision rights, but these decision rights are merely the lid on the Pandora’s Box of the liberal order. Inside the box, political and economic sovereignty meet and mingle with even deeper and subtler causes: the idea of the individual, the emergence of the self, the felt experience of free will.

Surveillance capitalism does not erode these decision rights –– along with their causes and their effects –– but rather it redistributes them. Instead of many people having some rights, these rights have been concentrated within the surveillance regime, opening up an entirely new dimension of social inequality. The full implications of this development have preoccupied me for many years now, and with each day my sense of danger intensifies. The space of this essay does not allow me to follow these facts to their conclusions, but I offer this thought in summary.

Surveillance capitalism reaches beyond the conventional institutional terrain of the private firm. It accumulates not only surveillance assets and capital, but also rights. This unilateral redistribution of rights sustains a privately administered compliance regime of rewards and punishments that is largely free from detection or sanction. It operates without meaningful mechanisms of consent either in the traditional form of “exit, voice, or loyalty” associated with markets or in the form of democratic oversight expressed in law and regulation.

Profoundly anti-democratic power

In result, surveillance capitalism conjures a profoundly anti-democratic power that qualifies as a coup from above: not a coup d’état, but rather a coup des gens, an overthrow of the people’s sovereignty. It challenges principles and practices of self-determination ––in psychic life and social relations, politics and governance –– for which humanity has suffered long and sacrificed much. For this reason alone, such principles should not be forfeit to the unilateral pursuit of a disfigured capitalism. Worse still would be their forfeit to our own ignorance, learned helplessness, inattention, inconvenience, habituation, or drift. This, I believe, is the ground on which our contests for the future will be fought.

Hannah Arendt once observed that indignation is the natural human response to that which degrades human dignity. Referring to her work on the origins of totalitarianism she wrote, “If I describe these conditions without permitting my indignation to interfere, then I have lifted this particular phenomenon out of its context in human society and have thereby robbed it of part of its nature, deprived it of one of its important inherent qualities.”

So it is for me and perhaps for you: The bare facts of surveillance capitalism necessarily arouse my indignation because they demean human dignity. The future of this narrative will depend upon the indignant scholars and journalists drawn to this frontier project, indignant elected officials and policy makers who understand that their authority originates in the foundational values of democratic communities, and indignant citizens who act in the knowledge that effectiveness without autonomy is not effective, dependency-induced compliance is no social contract, and freedom from uncertainty is no freedom.