Will artificial intelligence be a wonderful boon to humanity, or is it a serious threat? And what can we learn from the history of new information technologies?

                  These questions are at the heart of Yuval Noah Harari’s book Nexus. Harari is a master of big picture thinking. His book Sapiens tells the story of human evolution and history in a strikingly original way, and became a bestseller. He followed up with two more major books, Homo Deus and 21 Lessons for the 21^st Century. And now Nexus.

Harari looks at history from the perspective of information systems. He says humans became the Earth’s dominant species because of their unprecedented capacity for cooperation. When humans lived in small bands, cooperation was limited to those in their group, but what about other groups? What made it possible to trust strangers? Harari’s answer is stories: sets of ideas that are shared.

                  He says there is an objective reality, out there, and a subjective reality, what each person experiences in their mind. And there’s a third reality, intersubjective reality, which involves a shared understanding of the world. Examples include the days of the week, money and nations. If people stopped agreeing about such things, they wouldn’t exist. The idea of intersubjective reality may seem obvious or, on the other hand, contradictory. In any case, it’s fundamental to Harari’s thinking.

The next step is to see the world in terms of information systems, namely the ways people communicate with each other and with texts. Drawing on his knowledge of religion, Harari describes the difficulty of maintaining control over people’s beliefs. How can a religion ensure that its followers maintain the faith, given that faraway groups might change their teachings? One solution is a holy text, like the Bible, that cannot be changed. But then there’s the problem of different interpretations of the text. Those who provide authoritative interpretations can gain power within the faith.

Two purposes

Harari divides the purpose of information systems into two categories. One is seeking the truth, the other maintaining social order. You might think that the truth, a way of understanding the world that is more accurate, would be advantageous, but alas it often comes out second best. Information systems can enable false ideas to thrive and cause untold damage.

A common argument is that the solution to dangerous speech is counter-speech. However, Harari argues that it’s wrong to think more information is necessarily better. As an example of how wrong it can be, he uses the example of witch hunting in Europe, based on the spread of entirely imaginary threats. The invention of printing actually made it easier for delusional claims about witches to spread more widely, with devastating consequences.

“Witch hunts were a catastrophe caused by the spread of toxic information. They are a prime example of a problem that was created by information and was made worse by more information.” (p. 101)

In a pluralist society, there are many sources of information and many recipients, and no one monitors or controls all the information exchanges. The sources might be businesses, community groups, schools and government bodies. A school might contact a business or a public library without anyone else involved.

In contrast, in a centralist society, all information exchanges have to pass through the centre, which is like an all-seeing eye. Through this process, the centre gains great control over every part of society.

Harari calls the pluralist society a democracy, the centralist society totalitarianism or dictatorship. Harari sees these political systems as information systems and then looks at them historically. Two thousand years ago, before printing, communication between widely distributed groups of people was limited. This meant there were limits to both pluralist and centralist systems. Ancient Athens, where all male citizens participated in decision-making, was pluralist, but its political system couldn’t be scaled up to millions of people. The Roman Empire, where the centre attempted to control large populations, was limited by slow communication, with agents used to run things far from Rome.

                  Harari says that large-scale political systems only became possible with new information technologies: printing and, later, radio, television and the internet. With this framework, Harari lays the basis for looking at the significance of AI.

One more important concept: self-correction. Harari places great store on whether a system, after being set up, can correct itself. One of his examples is the US Constitution. Since it was passed in 1789, it has been amended dozens of times.

In contrast, the Catholic Church has a difficult time correcting its doctrine, because the Bible can’t be amended and the Pope is supposed to be infallible.

Harari says self-correcting systems are far better able to handle change. Pluralist systems, with their multiple avenues for decision-making and communication, are better at self-correction than centralist ones. So, what about AI?

According to Harari, AI is different from previous information technologies in one important aspect. Previous technologies like printing and radio were passive tools in the hands of humans, for better or worse. AI is different because it has agency: it doesn’t always do what its creators ask. If you start conversations with a chatbot, you won’t know in advance what it will say. It might flatter you, or possibly encourage your delusions, perhaps even encourage you to kill yourself. Another example: you ask AI to write a legal brief. That’s risky, because it might do something you don’t want, like make up non-existent legal cases.

                  Harari is worried about AI because it has the capacity to cause great harm, not from a superintelligent takeover of the world’s nuclear weapons but more likely from manipulating humans. Here’s how Harari’s concepts apply. A pluralist system, with its multiple nodes, has the capacity to control AI, to deal with possible dangers. However, a centralist system — like in Russia or China — is more susceptible to letting AI run things, and this might unleash wider AI manipulation.

A superintelligent AI: would it be self-correcting? Harari says not, and gives an example. When Facebook and YouTube algorithms went rogue, encouraging genocide in Myanmar, many humans raised the alarm, but the algorithms didn’t.

                  The problem is that computer systems, when they impose order on humans, think they have discovered truths about humans. For example, when an algorithm learns that outrage is popular, actually the algorithm is promoting outrage and not encouraging other human possibilities. Computers seem unable to take into account their own influence.

Harari’s goal in writing Nexus is to raise awareness of what’s happening with AI, and the need for controls. The trouble is that governments and corporations pushing AI development see this as a competition — a competition between humans. But they aren’t taking into account that the more important competition in the future may be between AI and humans, and that AI may end up dominant unless suitable controls are imposed.

                  There are parallels with previous technological developments. The rise of industrial technology enabled great productivity but also the capacity for massive pollution. Just as damaging were innovations in military technology, which enabled colonialism, world wars and the threat of nuclear annihilation. The development of surveillance technology has enabled the monitoring of entire populations, making dictatorial power easier to impose. But at least in those cases, it has remained possible for people to raise their concerns and campaign for controls or organise resistance. Although AI has agency, there’s one thing it apparently has a hard time doing: being a partner in discussions of how to make sure it serves the greater good — the greater good of humans, not AI.

It’s reasonable to be sceptical of warnings about AI taking over, of manipulating humans for its own ends. Scepticism is fine, but nevertheless it can be valuable to be aware of risks and discuss options. AI is not under popular control, and neither are the companies and governments developing it. Would you feel safer if AI were in control?

Yuval Noah Harari

 “Due to the superior abilities of AIs to create and disseminate information, the flood of ‘free’ AI speech threatens to void human freedom of speech by twisting, manipulating or silencing human speech — unless some restrictions are placed on AI activities. Human rights are better reserved for humans.” (p. 411)

Brian Martin
bmartin@uow.edu.au