I use ChatGPT and Claude every day. Claude is better than ChatGPT – both are fascinating and powerful tools – but they both have serious, structural limitations.
In its current incarnation, AI is like a very fast but very dumb research assistant with a peculiar set of cognitive biases (of which it’s perfectly aware when pressed but to which it’s very blind in practice), highly questionable research habits, and an irrepressible inclination to make unsubstantiated (or hallucinated) claims without qualification.
I use AI every day and I’ve never had a conversation where I didn’t have to correct it, in some way, shape, or form.
I would never rely on AI for an irreversible decision. Certainly not before thoroughly interrogating its claims.
It’s therefore mind-boggling to me that our defense forces (not to mention the US armed forces) reportedly rely on AI – the same models available to the public, with augmented data and additional integrations – for creating target lists.
You don’t get more irreversible than that.
And they know AI is fallible – but as long as it speeds up the killing process and allows them to conduct rapid strikes at scale, they’re willing to accept the deaths of innocent people wrongly targeted by an algorithm.
Not to mention anyone caught in the crossfire.
In case you’re wondering, the number of wrongful deaths the strategists overseeing our defense forces were explicitly willing to accept was 3,700.
Out of a list of 37,000 targets (each reviewed for exactly 20 seconds by an actual human being) they knew that at least 3,700 of them were mistakenly identified as targets (the actual number was probably much higher) and were not actually connected to the enemy and they killed them anyway.
“Fog of war” doesn’t apply to that kind of cold calculus.
Imagine if 3,700 Israelis had died on October 7.
(Basically, take the pain and suffering left in the wake of that horrible day and double it.)
I’m still trying to wrap my head around this brutal and hypocritical approach to modern warfare. I’ll be honest, I’m not worried about actually super-intelligent AI – I’m worried about the politicians and generals who insist on blindly using the inherently limited and very fallible LLMs we have today in order to make organized mass killing more “efficient.”