Category: Artificial Intelligence

Benedict Evans talks about how AI is getting better at giving better answers, but still lags behind when it comes to giving the right answer.

Here’s a practical example of the kind of thing that I do quite often, that I’d like to be able to automate. I asked ChatGPT 4o how many people were employed as elevator operators in the USA in 1980. The US Census collected this data and published it: the answer is 21,982

First, I try the answer cold, and I get an answer that’s specific, unsourced, and wrong. Then I try helping it with the primary source, and I get a different wrong answer with a list of sources, that are indeed the US Census, and the first link goes to the correct PDF… but the number is still wrong. Hmm. Let’s try giving it the actual PDF? Nope. Explaining exactly where in the PDF to look? Nope. Asking it to browse the web? Nope, nope, nope…

I faced this issue when I asked AI “what is the composition of nifty 500 between large caps mid caps and small caps“. ChatGPT came close by the getting the right document to refer, but ended up picking the wrong value. I needed the right answer. As Benedict Evans calls it, a deterministic task and not a probablistic task.

The useful critique of my ‘elevator operator’ problem is not that I’m prompting it wrong or using the wrong version of the wrong model, but that I am in principle trying to use a non-deterministic system for a a deterministic task. I’m trying to use a LLM as though it was SQL: it isn’t, and it’s bad at that.

But don’t write off AI so soon. Benedict Evans goes on to talk about how disruption happens.

Part of the concept of ‘Disruption’ is that important new technologies tend to be bad at the things that matter to the previous generation of technology, but they do something else important instead. Asking if an LLM can do very specific and precise information retrieval might be like asking if an Apple II can match the uptime of a mainframe, or asking if you can build Photoshop inside Netscape. No, they can’t really do that, but that’s not the point and doesn’t mean they’re useless. They do something else, and that ‘something else’ matters more and pulls in all of the investment, innovation and company creation. Maybe, 20 years later, they can do the old thing too – maybe you can run a bank on PCs and build graphics software in a browser, eventually – but that’s not what matters at the beginning. They unlock something else.

Dustin Ewers arguing AI will create more software jobs rather than taking away.

AI tools create a significant productivity boost for developers. Different folks report different gains, but most people who try AI code generation recognize its ability to increase velocity. Many people think that means we’re going to need fewer developers, and our industry is going to slowly circle the drain.

This view is based on a misunderstanding of why people pay for software. A business creates software because they think that it will give them some sort of economic advantage. The investment needs to pay for itself with interest. There are many software projects that would help a business, but businesses aren’t going to do them because the return on investment doesn’t make sense.

When software development becomes more efficient, the ROI of any given software project increases, which unlocks more projects. That legacy modernization project that no one wants to tackle because it’s super costly. Now you can make AI do most of the work. That project now makes sense. That cool new software product idea that might be awesome but might also crash and burn. AI can make it cheaper for a business to roll the dice. Cheaper software means people are going to want more of it. More software means more jobs for increasingly efficient software developers.

Economists call this Jevons Paradox.

This gives me hope.

Bonus: I first learnt about Jevons Paradox while reading Kim Stanley Robinson’s The Ministry For The Future.

An insightful article by Bruce Schneier on how humans have built guardrails to manage mistakes made by humans. But we are not equipped to manage the weird mistakes made by AI.

Humanity is now rapidly integrating a wholly different kind of mistake-maker into society: AI. Technologies like large language models (LLMs) can perform many cognitive tasks traditionally fulfilled by humans, but they make plenty of mistakes. It seems ridiculous when chatbots tell you to eat rocks or add glue to pizza. But it’s not the frequency or severity of AI systems’ mistakes that differentiates them from human mistakes. It’s their weirdness. AI systems do not make mistakes in the same ways that humans do.

Much of the friction—and risk—associated with our use of AI arise from that difference. We need to invent new security systems that adapt to these differences and prevent harm from AI mistakes.

Gary Marcus on AI Agents

I do genuinely think we will all have our own AI agents, and companies will have armies of them. And they will be worth trillions, since eventually (no time soon) they will do a huge fraction of all human knowledge work, and maybe physical labor too. 

But not this year (or next, or the one after that, and probably not this decade, except in narrow use cases). All that we will have this year are demos.

Funny.

And I am hoping it plays out the way Gary is describing it. I get to keep my job a little longer. And build a retirement corpus.