The Problem of Value Alignment
There is a famous sequence in 2001: A Space Odyssey where HAL 9000, the spaceship’s resident supercomputer, turns against the human occupants of the vessel and begins murdering them one by one, all because it cannot reconcile a simple contradiction in its programming: it must conceal the voyage’s true mission from the astronauts, but it also must not lie to them. Ergo, HAL’s only way out is to kill them all.
The portrayal of HAL in 2001 reflects our longstanding anxieties about robots, computers, and artificial intelligence. Reason free from passion is an alluring but frightening prospect. How much should we entrust to algorithms?
Artificial intelligence has already insinuated itself into the daily lives of millions, for better and for worse. We are far from the mass automation of the economy, but AI is advancing faster all the time. Robots are beginning to threaten the livelihoods of many blue-collar and even some white-collar workers.
More outlandishly, there is a case to be made that AI may also threaten the viability of human government in the long run. Smart technology has already been introduced to various aspects of the political process in many places, with productive results. So far, it is generally only effective as a complement to human efforts (like self-driving cars), but this may not always be the case. If AI could be programmed to produce viable legislation more efficiently and more effectively than human legislators, would we be better off entrusting our government to AI?
Smart Cities
Cities across the world are already experimenting with IoT (Internet of Things) technology, a growing phenomenon known as the “smart city.” In smart cities, AI is improving the efficiency of everything from utility meters and transportation to electric grids, waste management, and air quality monitors. Examples include Paris’s plan to build high-rises with positive energy outputs; London’s SmartPark project; Copenhagen’s efforts to become a carbon-neutral smart city by 2025; New York City’s connected vehicle project to increase traffic safety; and San Francisco’s Smart Traffic Signals Pilot project.
Nor do AI applications end at the city level. Singapore implemented a chatbots service to function as digital representatives to the public; an American chatbot lawyer app can help refugees apply for asylum; the Mexican government designed an algorithm to route citizen petitions; Japan piloted an AI system that would help Parliament members respond to citizen questions automatically. These services have proven useful for freeing up time for government employees and representatives, and their demand will only keep growing, given the workloads of modern legislators.
Automating specific processes is not, however, the same thing as automating government itself. Legislating involves a complex process of drafting legislation, revising it, collaborating on it, voting on it, signing it into law, enacting it, and enforcing it. This is a process for which AI is obviously not yet remotely suited. We are a long way from replacing our democratically elected officials with robots. But the prospect is not as outlandish as it sounds.
We already rely on computers to do most of the informational analysis on which we base most of our public policy. Computers can synthesize and evaluate vast quantities of data far more quickly and efficiently than human beings, and the results sometimes surprise us. Yet our ideological divisions remain because achieving analytical results is not the same as reaching a consensus on those results.
The problem with all research is that the researcher needs to ask the right questions and survey the right data. Even the best research is never perfect, and it is always possible to rationalize one’s way around even convincing results, when they are inconvenient, by postulating that a different set of questions should have been asked.
An AI system equipped with the capacity to track patterns in the economy, public policy, demographics, and other relevant data could theoretically devise the best possible legislation at any given moment (because it would essentially be equipped to ask all possible questions with all the available data at the same time), and could do so more efficiently than human legislators. Unfortunately, this would raise many new problems. For instance, who would decide what data was relevant and irrelevant? How would we program the system to account for biases in the data? Perhaps most importantly, what values would be programmed into the system to determine the ends it should seek to achieve?
The Problem of Value Alignment
According to the Moral Foundations Theory posited by Jonathan Haidt and a group of other political psychologists, our political orientations are broadly defined by our attitudes toward roughly five key metrics: care/harm, fairness/reciprocity, loyalty, authority/respect, and purity/sanctity. Some Moral Foundations theorists also posit a sixth value category, liberty/oppression, given that personal freedom is a value which the large political group known as libertarians tend to value above the other five categories. Traditional liberals and conservatives are defined more by how they value the original five categories. Liberals tend to value care and fairness particularly highly, while valuing the other three as relatively unimportant; Conservatives, by contrast, tend to rank all five categories as about equally important.
The issue of conflicting moral values emblematizes how an AI-based government would differ from one comprised of human legislators. An ordinary legislature represents an aggregation of complex individual orientations. However, the most complex AI system, even if it could be programmed to share these political values, could presumably represent only one political orientation of its own.
How would we decide what that orientation should be? Some sort of democratic process would be necessary in order for it to represent an aggregate of the values of the citizenry. The alternative would be to place all political power in the hands of technocrats. Yet it is difficult to imagine the democratic process by which the masses could specifically convey their will to the machine without feeling even less directly represented than they are today.
More to the point, value differences represent only the ideological component of the process by which policy preferences are articulated. The question of which policies we should be pursuing at all is even more important, and it is even more difficult to imagine how an AI system could be programmed to determine uncontroversial policies to pursue.
Consider the concept of economic growth. The question of what policies promote and hinder economic growth is an ideological one, but most people probably assume economic growth is a good thing. It is not that simple, however, because there is much debate as to whether economic growth is sustainable, whether it can be separated from physical growth at the expense of the environment, and what viable alternatives may exist. If an AI system were programmed to pursue economic growth at the expense of other considerations, it could easily lead us into disaster more quickly than any human government.
The system could presumably be programmed to balance the issue against other considerations, but these are likely to be equally problematic. Suppose the system was also programmed to extend human life expectancy. A worthy goal, surely, but to what extent? The bioethicist Ezekiel J. Emanuel, a former member of the Obama administration, argues that we should not seek to prolong life after the age of 75, citing the disproportionate deterioration in quality of life and the burden placed on the economy and the health care system.
Obviously, few of us would be comfortable with a system that would potentially euthanize us at age 75. The point is that computer programs ultimately reduce themselves to a binary code, meaning that everything is ultimately either a “yes” or a “no.” This is what should make us nervous about entrusting public policy to a computer, for whom nuance can never be more than a sequence of counterintuitive settings. At best, this would limit the system to a much smaller range of concerns than is normally in play when a legislature formulates public policy. At worst…well, we have the example of HAL 9000 to bear in mind.
Empathy & Artificial Intelligence
Artificial intelligence represents a valuable way for governments to make many processes more efficient and effective, as it has done in many other sectors. It might even prove helpful in researching and preparing legislation. The role of AI should probably remain limited, however, to a complementary role in existing political processes, rather than being used to replace them.
For all its technical aspects, the political process remains a fundamentally intuitive art in which human decisions are determined by human values. The biggest problem with AI as a replacement for any human component is that it cannot empathize. A moral AI system would be, by definition, a righteous system, with no alternative to which it could default in the absence of reprogramming. “Empathy is an antidote to righteousness,” observes Haidt, “although it’s very difficult to empathize across a moral divide”. Until AI can serve our interests without threatening them, it should remain a tool in human hands, not the reverse.
