Inevitably, the AI revolution of the last few years has seen machine learning technologies be deployed for increasingly punitive means, creeping not just into on-the-street policing but the judicial system too. Most of the warranted criticism of this change has been looking at AI tools used in the justice system as a strictly US phenomenon, but make no mistake, automated policing is well underway here in the UK too.

Just last week the Metropolitan Police came under fire once again for using automated facial recognition (AFR) software outside of Romford train station—the final deployment in a controversial trial that’s been underway since 2016 and seen the tech tested in various spots around London.

AFR works by scanning the faces of passers-by in real-time and cross-referencing them with the police’s own database of wanted suspects. If someone is flagged as a match—or, as was the case in Romford, simply tries to cover their face—then the officers manning the cameras are alerted and quickly apprehend the ‘suspect’ to search and question. Described by the anti-surveillance campaign group Big Brother Watch as “lawless and authoritarian”, and criticised for its lack of transparency and legal ambiguity, it’s not hard to see how the collection of biometric data without consent poses serious questions about privacy; a further example of technology’s increasingly-sinister encroachment on public space.

Even outside of all the predictable and clichéd references to Orwell’s 1984 surveillance-dystopia, there are countless other reasons we should be wary of this too. Firstly, because it’s both incredibly expensive and incredibly useless. Instead of using the money to put more officers on the street or invest in youth services – which would surely go a long way in curbing the wave of knife crime which is their rationale behind this new approach – millions have been spent on acquiring and testing this technology. And, unsurprisingly, it’s totally ineffective. An FOI request filed by Big Brother Watch found that when the Met was doing tests last year, the system incorrectly matched people 98 per cent of the time. Bearing that in mind and harking back to privacy rights for a second, all of a sudden the Met’s defence that “only images which alert against a watch list subject will be retained for a limited period” is hardly a reassuring one.  

Another societal problem posed by this automated policing is a common product of machine-learning itself: the entrenchment of racial biases. Speaking to HuffPost UK, who were reporting on the use of AFR in Romford, Hannah Couchman of the human rights advocacy group Liberty made the case that AFR is disproportionately used to target black people. To get even a rudimentary understanding of how these technologies could be deployed along the lines of racial prejudice, all you need do is look at where the Met has been testing it: not just in places such as Romford and Stratford, both being some of the most racially-diverse areas of London, but perhaps more tellingly the already extremely heavily-policed Notting Hill carnival for two consecutive years—with 95 people falsely flagged as criminals in 2017.  

It’s not just with the use of facial recognition that we’re seeing the rise of automated policing either. A report called ‘Policing By Machine’ published yesterday by Liberty revealed that twelve police forces across the UK have been using ‘predictive mapping programs’ that collate data on previous crimes to identify ‘high-risk’ areas on a map, allowing them to figure out where to increase police presence. As well as that, they also found that three forces were using a machine learning programme called Harm Assessment Risk Tool (HART) that supposedly enabled them to predict the likeliness of individuals committing, or being victim to, certain crimes—from stalking and burglary to domestic violence and sexual assault. HART comes to these conclusions by using 34 pieces of data, ranging from your criminal record to gender, income, name and postcode.  

Though many argue that algorithms can’t be biased; that they’re somehow magically neutral entities free from human influence, what certainly is bias is the masses of raw data that they rely on. In its dependence on pre-existing crime data, automated policing only serves to reinforce the racial biases that already consume the criminal justice system, discriminating in an overtly racist fashion sometimes, or via trivial things like postcode or socio-economic class, which the Liberty report refers to as “proxies for race.” With machine-learning simply duplicating the deep-seated racist practices of the police, a dangerous form of automation bias can occur: with officers becoming over-dependent on these algorithmic decisions, abandoning reason and simply refusing to question them—even when faced with contradicting evidence.

Whether it’s using this already-biased data pool for facial recognition, creating supposedly ‘high-risk’ areas that are likely already disproportionately policed or conducting psychological assessments to determine potential criminality, the automation of our police force not only concerns our privacy but could also easily serve to perpetuate disciplinary biases across not just race, but also class, gender, sexuality or any other data-set that could potentially replicate discriminations—at a human or institutional level.