How AI is Giving Governments Terrifying Surveillance Powers
And How Your Smartphone Is Betraying You
Note : After the 14-part series in which we looked at how nation-states are being disrupted by the Internet and globalization, and a 4-part series on why now is the worst time for them, I have concluded that there are two main ways for governments to respond to these disruptions:
Increasingly control what their residents do, via permanent digital surveillance and the use of AI and machine learning (China's way)
Accept that their role will diminish and treat their residents more like customers
In this new series, corresponding to chapter 4 of my forthcoming book, we analyze the (very) slippery slope towards generalized surveillance to which many countries, including the most democratic, are committing themselves.
Here are the 11 articles of the series :
The state is, by its very nature, an entity that wants to make the world as transparent as possible, by organizing it in the most rational and "gridded" way possible, and if it can't, by monitoring with the greatest possible precision what happens on its territory and elsewhere1 .
Until recently, states were very limited in their ability to collect and process large amounts of data, but this is changing:
At least since the late 1990s, the digitization of society has enabled states to collect massive amounts of data on their citizens and the rest of the world, as we have seen in this chapter.
And the development of machine learning and artificial intelligence now enables them to identify the most interesting data in this mass of data, in order to spot trends or particular individuals.
Here are some possible uses of machine-learning and AI for mass surveillance of the population:
Facial recognition makes it possible to monitor the actions of all individuals, catalog them and raise alarms for "suspicious" behavior. China has launched a large-scale program of this type, called "one person, one file2 ".
Use cell phone geolocation to build up a portrait of each individual, knowing the places they visit, the people they hang out with, who they travel with, and so on.
Massive monitoring and analysis of public online communications on social networks. By exploiting this data, a government could identify opinion trends, protest movements or even dissident ideas, thereby undermining freedom of expression.
Predicting people's future behavior based on their browsing history, online purchases and social interactions. This predictive capability can lead to a form of pre-emptive discrimination, where individuals are targeted or monitored not for what they have done, but for what an algorithm thinks they might do.
Monitoring employees at work, assessing everything from their productivity to their emotional state, is likely to create an oppressive working environment. Employers can use this information to tightly control employee behavior, even predicting and punishing "undesirable" behavior before it occurs.
Inspired by systems such as "social credit" in China, the application of scores based on people's behavior in various aspects of daily life could become a reality. These scores could affect access to services, employability and even social relationships, creating a society where every action is evaluated and rated, leading to constant pressure to conform to established standards.
Analyzing facial expressions and voice modulations to determine a person's emotional state could lead to invasive emotional surveillance. This technology could be used to identify people considered a potential threat, based on subjective emotional criteria, calling into question the right to emotional privacy.
China is at the forefront of the use of these technologies to control not only its minorities, such as the Tibetans and Uyghurs, but also its entire population.
Thus, by early 2020, the country had nearly 350 million cameras, most of which are capable of facial recognition, and the Chinese owned more than 840 million smartphones sending a constant stream of location data to telecom operators3 .
Almost all Chinese use WeChat, an app combining numerous functions, similar to what a fusion of WhatsApp, Google Maps, Facebook, Twitter, Tinder, PayPal, your medical records, and Apple Pay would allow (!), with the essential difference that the Chinese state has total access to all conversations, uses and transactions its citizens make on this app4 .
And the app has so many uses in so many aspects of daily life that it's impossible to live a normal life in China without having it on your phone5 . In particular, you need it to pay for just about everything, as the number of businesses accepting cash is shrinking by the day6 .
The almost compulsory use of this app allows the Chinese state to know almost everything about the lives of all its citizens: everything they buy, everything they say online, who they chat with, where they go, who they hang out with, etc.
And it was able to do this in record time, as WeChat was only launched in 2011.
All this data can now be fed into artificial intelligences looking for anything that might identify dissidents7 - the Party's dream being to be able to identify dissidents before they become dissidents8 .
China thus represents exactly the model of what not to do, and a terrifying reminder of where we're heading if we let democratic countries follow this slippery slope towards generalized surveillance.
Even if it's unlikely that democratic countries will go that far, to come even remotely close to what China is doing would be terrifying.
The Chinese government in its current form is therefore the number one enemy of all lovers of freedom and privacy, and our counter-model.
Coming soon
In the next article, we’ll take a look at why even democratic nation-states are strongly tempted to use the new powers of mass surveillance that digital tools give them, even if this goes against the spirit of their laws and even their Constitutions.
Stay tuned ! In the meantime, feel free to follow Disruptive Horizons on Twitter, and join the tribe of Intelligent Rebels by subscribing to the newsletter :
And here are the first nine articles of this series :
Other articles in the series
Seeing Like A State, James C. Scott, 1999
Eduardo Baptista, "Insight: China uses AI software to improve its surveillance capabilities", Reuters, 2022.
Josh Chin, Liza Lin, Surveillance State: Inside China's Quest to Launch a New Era of Social Control, 2022.
Eva Dou, "Jailed for a Text: China's Censors Are Spying on Mobile Chat Groups", The Washington Post, 2017. The police have access to this data after a simple authorization from a superior - no need for a judge's warrant! Surveillance State, op. cit.
Stephen McDonell, "China social media: WeChat and the Surveillance State", BBC, 2019.
"Overseas visitors can now go cashless in China", TechHQ, 2023.
Miles Kenyon, "WeChat Surveillance Explained", Citizen Lab, 2020
Surveillance State, op. cit. It's not certain that the Chinese state can do it, but it's certainly trying!
How true and absolutely terrifying... Kudos Olivier !!
Greetings from Nicole (Brussels)