Matthias Kainer explains how T works in GPT. This is possibly the simplest explanation of transformers that I have come across.

All the “knowledge” a Transformer has is encoded in its weight matrices – those millions (or billions) of numbers that were set during training. When you ask it a question, it does not go look something up. It reconstructs an answer from compressed statistical patterns. Think of it like this: if you memorized every cookbook in the world but then someone asked you “what temperature for roasting a chicken?” – you would reconstruct an answer from all those overlapping memories. Most of the time you would be right. But sometimes your memories would blend together and you would confidently say “180C for 3 hours” when the actual answer depends on the size of the chicken. You would have no way to check, because the cookbooks are gone – only your compressed memory remains.

That is why the search + embeddings approach from earlier matters so much. RAG (Retrieval Augmented Generation) is essentially saying: “Do not trust your memory alone. Before answering, go find the actual document, read it, and base your answer on that”. It does not completely solve hallucinations, but it dramatically reduces them for factual questions.

The bottom line: Transformers are prediction machines, not truth machines. They predict what text should come next based on patterns. When the pattern aligns with truth, they are brilliant. When it does not, they are confidently wrong. This is not something that can be “fixed” without fundamentally changing the architecture – it is a feature of how next-word prediction works. Always verify important claims. The AI does not know what it does not know.

I recently read two articles which highlighted a very similar problem when using AI for coding—burn out.

Matthew Hansen talking about how a one-off productivity boost by AI can lead to the team burning out.

My friend’s panel raised a point I keep coming back to: if we sprint to deliver something, the expectation becomes to keep sprinting. Always. Tired engineers miss edge cases, skip tests, ship bugs. More incidents, more pressure, more sprinting. It feeds itself.

This is a management problem, not an engineering one. When leadership sees a team deliver fast once (maybe with AI help, maybe not), that becomes the new baseline. The conversation shifts from “how did they do that?” to “why can’t they do that every time?”

My friend was saying:

When people claim AI makes them 10x more productive, maybe it’s turning them from a 0.1x engineer to a 1x engineer. So technically yes, they’ve been 10x’d. The question is whether that’s a productivity gain or an exposure of how little investigating they were doing before.

Burnout and shipping slop will eat whatever productivity gains AI gives you. You can’t optimise your way out of people being too tired to think clearly.

And here’s Siddhant Khare talking about how an increase in throughput also increases context switching.

Here’s the thing that broke my brain for a while: AI genuinely makes individual tasks faster. That’s not a lie. What used to take me 3 hours now takes 45 minutes. Drafting a design doc, scaffolding a new service, writing test cases, researching an unfamiliar API. All faster.

But my days got harder. Not easier. Harder.

The reason is simple once you see it, but it took me months to figure out. When each task takes less time, you don’t do fewer tasks. You do more tasks. Your capacity appears to expand, so the work expands to fill it. And then some. Your manager sees you shipping faster, so the expectations adjust. You see yourself shipping faster, so your own expectations adjust. The baseline moves.

Before AI, I might spend a full day on one design problem. I’d sketch on paper, think in the shower, go for a walk, come back with clarity. The pace was slow but the cognitive load was manageable. One problem. One day. Deep focus.

Now? I might touch six different problems in a day. Each one “only takes an hour with AI.” But context-switching between six problems is brutally expensive for the human brain. The AI doesn’t get tired between problems. I do.

This is the paradox: AI reduces the cost of production but increases the cost of coordination, review, and decision-making. And those costs fall entirely on the human.

If the team sprints in one sprint, the expectation is that the team will be sprinting forever.

And on the other end of the spectrum we have David Crawshaw’s experience:

I am having more fun programming than I ever have, because so many more of the programs I wish I could find the time to write actually exist. I wish I could share this joy with the people who are fearful about the changes agents are bringing. The fear itself I understand, I have fear more broadly about what the end-game is for intelligence on tap in our society. But in the limited domain of writing computer programs these tools have brought so much exploration and joy to my work.

This is the most confusing aspect for me. The polar opposite experiences people are sharing while using AI for coding.

From Wikipedia:

Moravec’s paradox is the observation that, as Hans Moravec wrote in 1988, “it is comparatively easy to make computers exhibit adult level performance on intelligence tests or playing checkers, and difficult or impossible to give them the skills of a one-year-old when it comes to perception and mobility”. This counterintuitive pattern may happen because skills that appear effortless to humans, such as recognizing faces or walking, required millions of years of evolution to develop, while abstract reasoning abilities like mathematics are evolutionarily recent.