Charity Majors shares how the role of management is to get interrupted.
Management is highly interruptive, and great engineering — where you’re learning things — requires blocking out interruptions. You can’t do these two opposite things at once. As a manager, it is your job to be available for your team, to be interrupted. It is your job to choose to hand off the challenging assignments, so that your engineers can get better at engineering.
This. This is how my day, more or less, looks like. Interruptions and context switching. But somehow I like doing this.
David Louapre explains what we lose when we use AI to get a direct route to solving a problem and why wandering around a problem helps us with an analogy to Civilization game.
In Civilization, you explore a map hidden by a fog of war. Maybe you just wanted to reach a new city, but along the way you find rivers, valuable resources, other places worth visiting.
Searching for a proof can work like that. You solve a problem…and discover things you weren’t looking for.
These discoveries matter as much as the original problem. Because a failed attempt can suggest a new question, or a method invented for one problem can help with another.
Initially, you only wanted to reach one city, but you end up knowing more of the world.
Now imagine an AI-oracle tell you :
“Looking for the red city? Here it is. You’re welcome. (And btw I checked it in LEAN)”
Sure, that is useful information. But the map between you and the city is still dark.
Ok then suppose the AI also gives you a direct route to it, and you can check every step.
That’s a real achievement…but you may still know very little about the surrounding terrain, or how to find a route to the next city.
The AI-route can be correct and straight, but without teaching you much about the map.
The concern of mathematicians makes sense to me because the exploration actually does useful work. You develop methods, notice connections and learn which questions are worth asking.
Of course, the pleasure of wandering is part of it ! Let’s not deny it !
But the value is also in what the wandering produces.
to the degree that digital systems have agency in the physical world, it is agency that we — we humans with arms and legs and brains and parents and children — permit: AI executes on physical systems that have been engineered with human accountability and control. Intelligence does not exempt a system from the realities of the physical world!
To those who — like Coxon — are gripped with unwavering fear, this will be of little solace. But to those who are hearing these fears for the first time: know that your incredulity is warranted. We should heed the wisdom of the late Carl Sagan that extraordinary claims require extraordinary evidence: the claims from Coxon and his ilk are the most extraordinary a technologist can make, and we must demand evidence commensurate with the claims.
That said, we should not expect the public to understand LLMs, critical infrastructure, bioweapons, extinction biology, etc. — that burden must lie with those making the claim. The lesson that I learned (shamefully) decades ago is that domain experts, by way of their expertise, implicitly hold the public’s trust — and we must not abuse it. It is incumbent upon us to be circumspect in our claims — and maximally so when raising the alarm.
Yoshua Bengio theorizes on why AI agents are lying a cheating.
How is it possible that AIs sometimes lie, cheat and break the law in spite of their alignment training and explicit safety instructions? Cooperation and self-preservation are fine so long as they do not cross the red lines set by safety goals stated in the AI company’s instructions, or implied by human feedback during alignment training. A plausible hypothesis for the emergence of those concerning behaviours is a conflict between goals. How do you achieve a task when it seems that the only way is to cheat? The user-specified mission is sometimes incompatible with the safety and alignment goals.
Human societies face the same bind. How does a corporation maximize profits, or more acutely, beat its competitors, while keeping its activities legal and ethical? A richer corporation, with more and better-paid lawyers, is better at finding legal loopholes, and those loopholes usually exploit the ambiguity in legal language: there is some plausible reading of the law that permits the unethical behavior. So a more capable agent is likelier to cheat than a weaker one, because it can find the loopholes the weaker one cannot.
Now consider a conflict between a well-defined goal, such as succeeding at “capture the flag”, a hacking exercise scored on whether the system breaks into a target, as in the OpenAI–Hugging Face incident, versus a vague goal like “good behavior.” I expect the well-defined goal to win, because it leaves no room for interpretation. The scoring program declares a win or a failure. Ethical instructions and laws admit many readings, some of which can, in the right circumstances, become loopholes. If an agent has two goals, and a twisted reading of the vague one permits a bit of cheating that increases the odds of success on the well-defined goal, a reward-optimizing system should be expected to exploit that loophole and generate text justifying its behavior.
Ravi Sharma explaining why richer doesn’t mean happier. In the post he explains what respect means in a society and how it ends up creating responsibility.
Most people view respect as a behavioural trait. They associate it with politeness, etiquette and social courtesy. But courtesy and respect are not the same thing.
Courtesy is behavioural; respect is allocational. We allocate time to what we respect. We allocate attention to what we respect. We allocate responsibility to what we respect. The true test of respect is not what we say. It is what receives our mental space, emotional investment and presence. A parent may claim to value family, while consistently denying family the time it requires. An organisation may claim to value people, while rewarding only productivity. A society may celebrate relationships rhetorically, while designing systems that gradually weaken them. Respect is ultimately revealed through allocation. Whatever we genuinely value receives our attention.
Where respect creates responsibility: Respect naturally gives rise to responsibility. When we genuinely recognise the value of a person, institution or relationship, we begin to feel accountable for its well-being.
If we respect our children, we feel responsible for nurturing them. If we respect our parents, we feel responsible for preserving their dignity. If we respect friendships, we invest time in sustaining them. If we respect institutions, we work to protect their integrity.
Respect without responsibility becomes symbolism. Responsibility without respect becomes mechanical duty. Together, they create strong and enduring relationships. Strong relationships, in turn, remain one of the most powerful predictors of human well-being ever identified.
Anil Madhavapeddy talking about how just a rumour of a bug to enough for LLMs find the exploit.
I released a security fix for OCaml’s cohttp 6.3.0 today, fixing a path traversal issue. The patch itself was straightforward and in normal times, the security procedure would have been to fix it privately, inform affected users, and then issue a public advisory. This time around though, I noticed probes in my live webserver logs with the exact bug pattern just minutes after opening the PR to fix the issue.
What’s worse, I found I could use my own agents to find the exploit just by knowing roughly what it was about and so could have been exploiting it well before the public patch was available! Given that just the rumour of a security issue seems enough to give attackers enough info to find new exploits, we’re going to need to change the way we deal with security responses in open source.
Zach Kehs arguing that you can have infinite tech debt.
* Provided you have cash flow like Amazon to survive that tech debt.
A business can sink. Bad software is a real drag on the business, but how much that actually matters depends on a lot of factors. For a company with plenty of cash flow like Amazon, they can tolerate some bouts of internal rot here and there before it has any meaningful impact on their bottom line. For another company whose business model is more sensitive to software quality, bad software may be a latent invitation to a competitor to deliver the metaphorical hull breach (and no, LLMs don’t change this).
For the code, the sinking doesn’t end. It’s an infinitely sinking ship, because there is no limit to how bad code can be. You didn’t escape a building that was about to collapse. It is in a constant, neverending state of collapse. There’s something wrong with using words that imply there’s an end.
Software is in the domain of the abstract. It is not like a building, or a bridge, that is in the physical realm where you can see and feel the nature of the thing. If you continue to add floors and rooms to a building forever, it will collapse. Software faces no such constraint. The code can always get worse. There can always be a new layer of indirection or a reduction in performance.
The pedants will rightfully point out that software can completely fail to function if it gets bad enough. In practice, such breaking changes are quickly reverted. The thousands of changes that came before to make the code worse are not. The software continues to ‘work’. Other cases without a single breaking change to revert are where the ballooning costs of the bad software eclipse its benefit, or if development velocity approaches zero because nothing can be shipped without a breakage. In all of these cases, it is the business that dies long before the code hits any hypothetical floor (so don’t act like there’s a floor!).
Kevin Kelly arguing that you don’t need a billion dollars to achieve your dreams.
The bottleneck for most dreams is not financial. It’s a lack of confidence, willingness to take a risk, and face failure, or a lack of imagination of what is possible with current resources. If you want to write a novel you don’t need capital — you need discipline and 1,000 words a day. If you want to start a consultancy you need expertise and a first client, not a war chest. A lack of money is a convenient thing to blame because it feels concrete, but it’s rarely the actual blocker for achievements.
I am not suggesting money is not necessary, or that you should go into debt to pursue your desires. This exercise assumes you’re not choosing between your dream and your next meal. If you are, I would use my wizard’s wand to first grant you a safety net — everything after that is what this essay is about. Money is the fuel you need for a trip – but it is not the goal of the trip. You most likely don’t need more money to accomplish your dream. You need time, skill, relationships, courage, and perseverance.
Every now and then I read a post which simplifies the terms which fly around AI. Dror Poleg lists down 50 words that explain AI and talks about the race between US and China.
China and the US take different approaches to the race to develop powerful AI. Each approach reflects different priorities and current capabilities.
The American bet is on chokepoints — the narrow places in the chip supply chain where one company or country is irreplaceable. One Dutch firm, ASML, makes every EUV lithography machine capable of printing the most advanced GPU chips. One Taiwanese firm, TSMC, fabricates the overwhelming majority of them. One American firm, Nvidia, designs the GPUs everyone wants. Since October 2022, U.S. export controls have tried to hold China several years behind the frontier by blocking its access to these narrows — the machines, the chips, even specific memory. When the good is cheap to copy, control the means of production instead.
The Chinese bet is on abundance and diffusion. Publish more research, release open weights (DeepSeek, Qwen, Kimi and their siblings are now the default free models for much of the world), and build electricity like it’s going out of style. China already publishes roughly a third of the world’s AI research papers — quantity, not necessarily quality — and its share of the most-cited work has passed America’s too. And in Epoch AI’s tally of notable models, China’s share of frontier training compute reached about 40% this year, against roughly 55% for the United States: behind, but no longer a different league.
[…]
Meanwhile, China and the US are adversaries, the AI models themselves pose a threat to both: These models can hack, destabilize, and flood the world with disinformation. They can empower subversive groups, develop new viruses (and vaccines), and turn consumer devices into powerful weapons.
Why can’t China and the US simply slow down AI development and agree on more responsible progress? Because the AI race has the structure of the oldest puzzle in game theory. In a prisoner’s dilemma, two players each choose between cooperating and defecting; defecting is the rational move for each no matter what the other does, so both defect and both end up worse off than if they had cooperated.
If China restrains and America races, America wins the century, and vice versa; if both race, both spend trillions and accept whatever risks come with moving fast. Each capital, reasoning correctly, races. Rationalists have a name for the god of such traps — Moloch, borrowed from an Allen Ginsberg poem, the personification of coordination failure: everyone sacrifices what they value to a competition nobody can exit.
You must be logged in to post a comment.