Algorithmic Inequality

Disruptive. One of the attributes that most use to describe in minimalistic terms the potential impact of new and emerging information and communication technologies (ICTs) in society. While its actual meaning can vary from one person to another, disruption is usually linked to dramatic short-term change where old and obsolete technologies, processes and institutions -not to mention people – will be either replaced or purge altogether, all for the best.

Disruption is thus implicitly connected to the concept of progress, especially to its linear version. Here, progress is seen almost like time is in physics: it always goes forward, and it is impossible to go back and say edit the past. Recent research has challenged the linear conception of progress¹ See for example Amy Allen’s book, The End of Progress, for an overview of the issues surrounding progress from a critical theory perspective. but this has yet to have to permeate the predominant perception.

No one will certainly challenge the idea that ICTs has “progressed” by leaps and bounds in the last 25 to 30 years. But what about its social impact? After all, during that same period of time, we have seen inequality grow at a fast pace in most if not all countries. Surely, correlation does not mean causation. But we probably do need to revisit the idea of progress. Perhaps more relevant for us practitioners on ICTs and human development is if we can use technology to tackle inequality in compelling fashion. Technologies such as Artificial Intelligence, Machine Learning, and Blockchains could indeed be leveraged to address the challenge. Is this happening on the ground?

The recent book by Virginia Eubanks, Automating Inequality, suggests a positive response. However, the impact has so far been mostly negative. That is, new smart technologies are being used by local governments to monitor and control those eligible for state assistance – mostly the poor –  and eventually disqualify them from receiving such benefits based on “objective” data and “impartial” algorithms.

Eubanks traces the historical evolution of poverty assistance in the US to the first poorhouses in the 17th century. Poorhouses became the central poverty assistance mechanism in the early 19th century. However, this started to change after the economic crisis of 1873 and the general strike of 1877. While Europe back then launched the basic structure of the welfare state, the US opted instead for “scientific charity” with the goal to ensure that only those who really deserve assistance get it while alleged “free riders” could be filtered out of the system and punished accordingly. This approach required both better methods and data and ended up giving birth to the function of casework and its human incarnation, caseworkers.

The 1929 Great Depression brought additional changes. In a nutshell, the New Deal introduced the distinction between social insurance and public assistance. Unlike the former, public aid was directly controlled by states and municipalities, free to add the controls and limitations they deemed necessary for disbursing public resources. This ended up reinforcing the “scientific charity” approach targeting, in particular, the so-called “working mothers.”

Computers were introduced in the 1970s when the demand for public assistance was at its peak in several key states. Lousiana, the first state to push the concept of the “employable mother” – which excluded almost by default most African American women – was also the first to deploy new technologies. But the rate of technology deployment to “disrupt” public assistance was very uneven. In the end, technologies ended up supporting, not disrupting, the long-term policies and ideas that triggered public assistance in its current form and the overall stigmatization of the poor.

Eubanks closely examines three cases to support this point: 1. Indiana’s successful efforts to restrict and reduce public assistance via automation and privatization. 2.  Los Angeles county management information system to record and manage the homeless. And 3. Pennsylvania’s Allegheny county predictive analytics platform which tries to guess the future behavior of the poor based on past recurrences including actions by all those who happen to surround them. This systems, in fact, have created the new digital poorhouses.

However, the digital poorhouses are more ominous than their analog predecessors. The former are scaleable, obscure, difficult to change, and seemingly eternal, all thanks to the very nature of the technology being deployed. Moreover, these systems also build on the idea of “rational discrimination” which in essence assumes that bias does not exist.  We could link this idea to the popular notion that technology is neutral and thus unbiased. If the algorithm gives one specific output or recommendation,  it is clear that this is not coming from some disgruntled and biased bureaucrat. Instead, it is the result of a supposedly scientific procedure which we average humans cannot really argue against. The computer is never wrong.

Eubanks ends the book by proposing an “Oath of No-Harm for an Age of Big Data.” She also revisits Martin Luther King’s Poor People’s Campaign that failed to deliver the goods after his assassination and suggests a restart to dismantle the digital powerhouse for good. Some groups are already working along this lines, but more grassroots support will be needed to ensure success in the medium term.

When it comes to public assistance in the US, progress is indeed not its most outstanding characteristic. Eubanks does not really deal with additional technologies such as artificial intelligence and machine learning which by now have become controversial when it comes to scalability, obscurity, persistence and being unbiased.

Technological progress is undoubtedly linked to the Enlightenment and its stern endorsement of rationality as a way to free human beings from the dictatorship of nature and of other humans.  This, however, does not seem to be happening here at the moment.

Cheers, Raúl