Avoiding a Soylent Green Data Revolution

“There are, first of all, people, ordinary individuals with their heuristics, their intuitions, and their rules of thumb, with their laziness, their impulses, and their myopia. They have choices to make for themselves and their loved ones, and they make some of them well and many of them badly.

Then there are those whom Sunstein refers to as “we.” We know this, we know that, and we know better about the way ordinary people make their choices. We are the law professors and the behavioral economists [and data scientists!] who (a) understand human choosing and its foibles much better than members of the first group and (b) are in a position to design and manipulate the architecture of the choices that face ordinary folk. In other words, the members of this second group are endowed with a happy combination of power and expertise.

Of course regulators are people too. And like the rest of us, they are fallible. In the original Nudge, Sunstein engagingly confessed to many of the decisional foibles that Thaler exposed. Worse, though, is the fact that regulators are apt to make mistakes in their regulatory behavior: “For every bias identified for individuals, there is an accompanying bias in the public sphere.” Sometimes governments blunder because they feel compelled to defer to the irrationalities of ordinary people. But we all know they are perfectly capable of screwing things up on their own, whether it’s the invasion of Iraq or the rollout of Obamacare.”

“There are deeper questions, too, than these issues of trust and competence. As befits someone who was “regulation czar” in the Obama White House, Sunstein’s point of view is a rather lofty one and at times it has an uncomfortable affinity with what Bernard Williams once called “Government House utilitarianism.” Government House utilitarianism was a moral philosophy that envisaged an elite who knew the moral truth [powered by data!] and could put out simple rules for the natives (or ordinary people) to use, even though in the commissioner’s bungalow it was known that the use of these rules would not always be justified. We (the governors) know that lying, for example, is sometimes justified, but we don’t want to let on to the natives, who may not have the wit to figure out when this is so; we don’t trust them to make the calculations that we make about when the ordinary rules should not be followed. Williams saw the element of insult in this sort of approach to morality, and I think it is discernable in Sunstein’s nudging as well…Nudging takes advantage of my deficiencies in the way one indulges a child. The people doing this (up in Government House) are not exactly using me as a mere means in violation of some Kantian imperative. They are supposed to be doing it for my own good. Still, my choosing is being made a mere means to my ends by somebody else—and I think this is what the concern about dignity is all about.”

“Yes, the commercial interests of technology companies and the policy interests of government agencies have converged: both are interested in the collection and rapid analysis of user data. Google and Facebook are compelled to collect ever more data to boost the effectiveness of the ads they sell. Government agencies need the same data—they can collect it either on their own or in coöperation with technology companies—to pursue their own programs.

…

Many of those programs deal with national security. But such data can be used in many other ways that also undermine privacy. The Italian government, for example, is using a tool called the redditometro, or income meter, which analyzes receipts and spending patterns to flag people who spend more than they claim in income as potential tax cheaters. Once mobile payments replace a large percentage of cash transactions—with Google and Facebook as intermediaries—the data collected by these companies will be indispensable to tax collectors. Likewise, legal academics are busy exploring how data mining can be used to craft contracts or wills tailored to the personalities, characteristics, and past behavior of individual citizens, boosting efficiency and reducing malpractice.

…

Even if we tie the hands of the NSA—by some combination of better oversight, stricter rules on data access, or stronger and friendlier encryption technologies—the data hunger of other state institutions would remain. They will justify it. On issues like obesity or climate change—where the policy makers are quick to add that we are facing a ticking-bomb scenario—they will say a little deficit of democracy can go a long way.

Here’s what that deficit would look like: the new digital infrastructure, thriving as it does on real-time data contributed by citizens, allows the technocrats to take politics, with all its noise, friction, and discontent, out of the political process. It replaces the messy stuff of coalition-building, bargaining, and deliberation with the cleanliness and efficiency of data-powered administration.

This phenomenon has a meme-friendly name: “algorithmic regulation,” as Silicon Valley publisher Tim O’Reilly calls it. In essence, information-rich democracies have reached a point where they want to try to solve public problems without having to explain or justify themselves to citizens. Instead, they can simply appeal to our own self-interest—and they know enough about us to engineer a perfect, highly personalized, irresistible nudge.”