Overview
Governments already use AI for life-or-death decisions, often without saying so: screening visa applications in the UK, analysing drone footage on the battlefield in Ukraine. In the US, Palantir connects data across agencies into what it calls an Ontology, on the conviction that “the West must win, and technology is our deciding factor.”
Automated government has a record. Risk-assessment tools in US criminal justice carried racial bias into sentencing, and between 2013 and 2015 Michigan’s ATLAS system wrongly accused more than 40,000 low-income people of unemployment fraud, ending in a $20 million settlement.
Context and ontology
For an AI, context is the data it can reach, and data is never a complete or objective picture, so context is biased. The essay walks from context windows (at the time, 1 million tokens for Gemini 1.5 against GPT-4’s 128K) to an ontology, using a dinner party planned by an AI that knows your fridge, your bank balance, your guests’ diets and schedules, and the stock of every nearby store. Palantir’s Ontology does that across government departments instead of a kitchen.
The stakes
If we do not act now, we risk sleepwalking into a future where a machine will be the agent deciding if you should live to see another day, and you won’t even know it.Treyden Chiaravalloti, 2024
What it argues for
Forbid systems from automating the taking of a human life, or from setting what a human life is worth: those calculations require human oversight. And keep asking the questions only moral imagination raises. Who are we cutting costs for? Whose lives will be impacted? How can we minimise suffering?
