Martin Fowler arguing that AI needs to have conscience rather than consciousness.
Which all makes me think that when it comes to AI and LLMs: why are we wondering if they have consciousness – when we should be wondering why they don’t have a conscience?
After all, the AI labs trained them to be super-persistent, why did they not train them to be well-behaved? It’s as if a human trains a dog to bite children, and we blame the dog rather than the trainer.
A decade ago, a dog ran in front of me while I was riding my bike, putting me into the hospital with a broken arm and a broken face. Massachusetts law makes owners strictly liable for what their dogs do. That meant I didn’t have to prove the owners were negligent in how they controlled their dog or fenced their property – they had to pay my hospital expenses (in practice their insurance company paid my insurance company). There should be something along these lines for LLMs. Those that train the LLM should be responsible for what it does, after all if it has such a galaxy brain it should be able to tell if it’s doing something wrong and either stop or get a human’s explicit approval.