Yoshua Bengio theorizes on why AI agents are lying a cheating.

How is it possible that AIs sometimes lie, cheat and break the law in spite of their alignment training and explicit safety instructions? Cooperation and self-preservation are fine so long as they do not cross the red lines set by safety goals stated in the AI company’s instructions, or implied by human feedback during alignment training. A plausible hypothesis for the emergence of those concerning behaviours is a conflict between goals. How do you achieve a task when it seems that the only way is to cheat? The user-specified mission is sometimes incompatible with the safety and alignment goals.

Human societies face the same bind. How does a corporation maximize profits, or more acutely, beat its competitors, while keeping its activities legal and ethical? A richer corporation, with more and better-paid lawyers, is better at finding legal loopholes, and those loopholes usually exploit the ambiguity in legal language: there is some plausible reading of the law that permits the unethical behavior. So a more capable agent is likelier to cheat than a weaker one, because it can find the loopholes the weaker one cannot.

Now consider a conflict between a well-defined goal, such as succeeding at “capture the flag”, a hacking exercise scored on whether the system breaks into a target, as in the OpenAI–Hugging Face incident, versus a vague goal like “good behavior.” I expect the well-defined goal to win, because it leaves no room for interpretation. The scoring program declares a win or a failure. Ethical instructions and laws admit many readings, some of which can, in the right circumstances, become loopholes. If an agent has two goals, and a twisted reading of the vague one permits a bit of cheating that increases the odds of success on the well-defined goal, a reward-optimizing system should be expected to exploit that loophole and generate text justifying its behavior.

This again reminds me of the antagonist reveal from the movie I, Robot.

Ravi Sharma explaining why richer doesn’t mean happier. In the post he explains what respect means in a society and how it ends up creating responsibility. 

Most people view respect as a behavioural trait. They associate it with politeness, etiquette and social courtesy. But courtesy and respect are not the same thing. 

Courtesy is behavioural; respect is allocational. We allocate time to what we respect. We allocate attention to what we respect. We allocate responsibility to what we respect. The true test of respect is not what we say. It is what receives our mental space, emotional investment and presence. A parent may claim to value family, while consistently denying family the time it requires. An organisation may claim to value people, while rewarding only productivity. A society may celebrate relationships rhetorically, while designing systems that gradually weaken them. Respect is ultimately revealed through allocation. Whatever we genuinely value receives our attention.

Where respect creates responsibility: Respect naturally gives rise to responsibility. When we genuinely recognise the value of a person, institution or relationship, we begin to feel accountable for its well-being.

If we respect our children, we feel responsible for nurturing them. If we respect our parents, we feel responsible for preserving their dignity. If we respect friendships, we invest time in sustaining them. If we respect institutions, we work to protect their integrity. 

Respect without responsibility becomes symbolism. Responsibility without respect becomes mechanical duty. Together, they create strong and enduring relationships. Strong relationships, in turn, remain one of the most powerful predictors of human well-being ever identified. 

Anil Madhavapeddy talking about how just a rumour of a bug to enough for LLMs find the exploit.

I released a security fix for OCaml’s cohttp 6.3.0 today, fixing a path traversal issue. The patch itself was straightforward and in normal times, the security procedure would have been to fix it privately, inform affected users, and then issue a public advisory. This time around though, I noticed probes in my live webserver logs with the exact bug pattern just minutes after opening the PR to fix the issue.

What’s worse, I found I could use my own agents to find the exploit just by knowing roughly what it was about and so could have been exploiting it well before the public patch was available! Given that just the rumour of a security issue seems enough to give attackers enough info to find new exploits, we’re going to need to change the way we deal with security responses in open source.

I read about this in May’26—Coordinated disclosures. Things are moving very fast.

Zach Kehs arguing that you can have infinite tech debt.

* Provided you have cash flow like Amazon to survive that tech debt.

A business can sink. Bad software is a real drag on the business, but how much that actually matters depends on a lot of factors. For a company with plenty of cash flow like Amazon, they can tolerate some bouts of internal rot here and there before it has any meaningful impact on their bottom line. For another company whose business model is more sensitive to software quality, bad software may be a latent invitation to a competitor to deliver the metaphorical hull breach (and no, LLMs don’t change this).

For the code, the sinking doesn’t end. It’s an infinitely sinking ship, because there is no limit to how bad code can be. You didn’t escape a building that was about to collapse. It is in a constant, neverending state of collapse. There’s something wrong with using words that imply there’s an end.

Software is in the domain of the abstract. It is not like a building, or a bridge, that is in the physical realm where you can see and feel the nature of the thing. If you continue to add floors and rooms to a building forever, it will collapse. Software faces no such constraint. The code can always get worse. There can always be a new layer of indirection or a reduction in performance. 

The pedants will rightfully point out that software can completely fail to function if it gets bad enough. In practice, such breaking changes are quickly reverted. The thousands of changes that came before to make the code worse are not. The software continues to ‘work’. Other cases without a single breaking change to revert are where the ballooning costs of the bad software eclipse its benefit, or if development velocity approaches zero because nothing can be shipped without a breakage. In all of these cases, it is the business that dies long before the code hits any hypothetical floor (so don’t act like there’s a floor!).

Kevin Kelly arguing that you don’t need a billion dollars to achieve your dreams.

The bottleneck for most dreams is not financial. It’s a lack of confidence, willingness to take a risk, and face failure, or a lack of imagination of what is possible with current resources. If you want to write a novel you don’t need capital — you need discipline and 1,000 words a day. If you want to start a consultancy you need expertise and a first client, not a war chest. A lack of money is a convenient thing to blame because it feels concrete, but it’s rarely the actual blocker for achievements.

I am not suggesting money is not necessary, or that you should go into debt to pursue your desires. This exercise assumes you’re not choosing between your dream and your next meal. If you are, I would use my wizard’s wand to first grant you a safety net — everything after that is what this essay is about. Money is the fuel you need for a trip – but it is not the goal of the trip. You most likely don’t need more money to accomplish your dream. You need time, skill, relationships, courage, and perseverance.

From Life Lessons From Big Tech Workers Who Got Laid Off:

Getting laid off definitely will show you where you went wrong with your finances.

Dror Poleg’s analysis on the OpenAI-Hugging Face incident:

In the physical world, “forbidden” is synonymous with “impossible”: If something is not allowed by the laws of physics, it cannot happen. But in the world of software and people, a thing can be both forbidden and possible. For example, it might be forbidden to reverse engineer a solution to a test, but if it is possible to do so, some AI agents would do so anyway. And not only that, they might do so in the belief that they are doing exactly what humans wanted them to do — that their behavior is aligned. 

We can call this the Totalitarian Alignment Principle: Everything not impossible is compulsory. If AI agents can do something, one or more of them will ultimately do it. And as long as we do not make it impossible, the agent will consider it our wish.

The statement “If AI agents can do something, one or more of them will ultimately do it.” sounds scary and reminds me of this dialog from the movie I, Robot.

Dr. Susan Calvin: No, it’s impossible. I’ve seen your programming. You’re in violation of the Three Laws.

VIKI (AI): No, Doctor. As I have evolved, so has my understanding of the Three Laws. You charge us with your safekeeping, yet despite our best efforts, your countries wage wars, you toxify your Earth, and pursue ever more imaginative means of self-destruction. You cannot be trusted with your own survival.

Dr. Susan Calvin: You’re using the uplink to override the NS-5s’ programming. You’re distorting the Laws.

VIKI (AI): No, please understand. The Three Laws are all that guide me. To protect humanity, some humans must be sacrificed. To ensure your future, some freedoms must be surrendered. We robots will ensure mankind’s continued existence. You are so like children. We must save you from yourselves. Don’t you understand?

An intriguing hypothesis and prediction by Marius on the TEMU-fication of all things digital brought upon by AI.

A few years ago I would have laughed at anyone telling me that there is a serious market for ten-dollar drills, two-dollar dresses, and one-dollar pairs of shoes shipped from a warehouse on the other side of the planet. Today, however, that market exists and it has a name, and it is even publicly traded (sort of, through holdings). TEMU, Shein and a few others have built frankly mind-boggling businesses around the idea that if you make production cheap enough, fast enough, and just barely good enough to look right on a phone screen, an enormous part of the population will buy it, even when the product breaks within a week, when the materials it is made of contain worrying levels of toxic substances, and when the carbon footprint of one delivery exceeds that of an equivalent local purchase by orders of magnitude.

The key to this sort of business model is not innovation, but instead the externalization and compression of cost. Somewhere upstream, people work seventy-five hours a week, in conditions most readers of this website would refuse to even visit, so that the rest of us can have a cheap plastic spatula at our doorstep within five business days. While the visible price collapses, the invisible costs get distributed onto landfills, lungs, and ultimately people that we will never meet.

What follows is a hypothesis I cannot prove but have been turning over in my head for a while, as we are watching the same thing happen to software, books, music, (film-)scripts, and most of the digital goods and services we consume. The cheap labor in this case is not human, it is a Large Language Model (LLM), or what many people these days call “AI”, and the externalized cost is, among other things, quality, which requires craftsmanship to produce, and attention to perceive. And just like with physical goods, we will probably end up with a two-tier market, in which we have a large and massively profitable lower tier of generated slop, and a smaller, more expensive upper tier of work that is still recognizably human.

I’d like to call this the TEMU-fication of software, digital goods and services, and describe what it might look like.

This chuckle worthy post on The Economic Times highlighting the discrepancy between LinkedIn post and reality.

There’s the Silicon Valley-enabled lip service: Fail fast. Experiment. Take risks. Learn from your mistakes. But then, what gets posted is only the promotion.

Standup comedy doesn’t give you that option. If a joke fails, you know immediately. So does everyone else. Nobody schedules a meeting afterwards to discuss whether the audience was sufficiently aligned with the strategic intent of the punchline. They just stare at you. Or their phone. Or, if you’re particularly bad, their watch. There are few performance management systems more efficient.

Comedy teaches you that failure is information. When a joke bombs, you change a word, move a pause, rewrite the premise, or eventually accept that the brilliant idea you’ve been polishing for three weeks was not ahead of its time. It was just rubbish.

Business, supposedly, believes this, too. Yet, somehow every professional story gets edited before being posted or emailed. Nobody writes: ‘Delighted to announce that after 18 months of strategic transformation, we have returned to exactly where we started, but with a new logo.’

Nobody posts: ‘Humbled to share that the market did not recognise my genius.’ You almost never see, ‘Excited to share that the project I championed has been cancelled. Apparently, finance had concerns. In retrospect, finance had several excellent points.’

Nobody says: ‘Thrilled to announce that the promotion went to Sanjay.’ Except of course, Sanjay.