Category: Artificial Intelligence

Jerry Neumann comparing AI with previous revolutionary technologies like microprocessor and containers—shipping containers, not software containers—and arguing that money will be made on the applications sitting on top of AI rather than on AI itself.

This doesn’t mean AI can’t start the next technological revolution. It might, if experimentation becomes cheap, distributed and permissionless—like Wozniak cobbling together computers in his garage, Ford building his first internal combustion engine in his kitchen, or Trevithick building his high-pressure steam engine as soon as James Watt’s patents expired. When any would-be innovator can build and train an LLM on their laptop and put it to use in any way their imagination dictates, it might be the seed of the next big set of changes—something revolutionary rather than evolutionary. But until and unless that happens, there can be no irruption.

Ashanty Rosario talking about the challenges AI presents for education.

Many homework assignments are due by 11:59 p.m., to be submitted online via Google Classroom. We used to share memes about pounding away at the keyboard at 11:57, anxiously rushing to complete our work on time. These moments were not fun, exactly, but they did draw students together in a shared academic experience. Many of us were propelled by a kind of frantic productivity as we approached midnight, putting the finishing touches on our ideas and work. Now the deadline has been sapped of all meaning. AI has softened the consequences of procrastination and led many students to avoid doing any work at all. As a result, these programs have destroyed much of what tied us together as students. There is little intensity anymore. Relatively few students seem to feel that the work is urgent or that they need to sharpen their own mind. We are struggling to receive the lessons of discipline that used to come from having to complete complicated work on a tight deadline, because chatbots promise to complete our tasks in seconds.

Noah Smith sharing his thoughts on the recent report about plunging job market among college graduates. He starts off with this.

The debate over whether AI is taking people’s jobs may or may not last forever. If AI takes a lot of people’s jobs, the debate will end because one side will have clearly won. But if AI doesn’t take a lot of people’s jobs, then the debate will never be resolved, because there will be a bunch of people who will still go around saying that it’s about totake everyone’s job. Sometimes those people will find some subset of workers whose employment prospects are looking weaker than others, and claim that this is the beginning of the great AI job destruction wave. And who will be able to prove them wrong? 

In other words, the good scenario for the labor market is that we continue to exist in a perpetual state of anxiety about whether or not we’re all about to be made obsolete by the next generation of robots and chatbots.

Ha!

If the good scenario is us being in perpetual anxiety, then I don’t want to imagine the bad scenario.

Ted Chiang’s post on The New Yorker about how ChatGPT—an LLMs in general—are the blurry JPEG of the web. This post is old and came out in Feb’23.

When an image program is displaying a photo and has to reconstruct a pixel that was lost during the compression process, it looks at the nearby pixels and calculates the average. This is what ChatGPT does when it’s prompted to describe, say, losing a sock in the dryer using the style of the Declaration of Independence: it is taking two points in “lexical space” and generating the text that would occupy the location between them. (“When in the Course of human events, it becomes necessary for one to separate his garments from their mates, in order to maintain the cleanliness and order thereof. . . .”) ChatGPT is so good at this form of interpolation that people find it entertaining: they’ve discovered a “blur” tool for paragraphs instead of photos, and are having a blast playing with it.

Reminds me of Venkatesh Rao’s analogy on LLMs as index funds.

Quoting Stephan H. Wissel.

LLMs with their dependency on well crafted prompts feels like the revenge of the English majors hurled towards computer science

Ha!

Kevin Collier reporting for NBC News on how a hacker vibe hacked their way into various industries. This information comes from Anthropic’s Threat Intelligence Report for Aug’25.

…one of Anthropic’s periodic reports on threats, the operation began with the hacker convincing Claude Code — Anthropic’s chatbot that specializes in “vibe coding,” or creating computer programming based on simple requests — to identify companies vulnerable to attack. Claude then created malicious software to actually steal sensitive information from the companies. Next, it organized the hacked files and analyzed them to both help determine what was sensitive and could be used to extort the victim companies.

The chatbot then analyzed the companies’ hacked financial documents to help determine a realistic amount of bitcoin to demand in exchange for the hacker’s promise not to publish that material. It also wrote suggested extortion emails.

The chatbot then analyzed the companies’ hacked financial documents to help determine a realistic amount of bitcoin to demand—What the…!

Since I have started following AI news, I read about how you should break down your problem statement into smaller chunks for AI, setup a plan, do periodic reviews of code generated, and then accept the changes. I never thought this approach would be effective in hacking also.

Craig McCaskill gives multiple example of bubbles and how they ultimately burst. But what if AI isn’t a bubble? What if this is the real deal?

The AI revolution is real, transformative, and probably unstoppable. Whether it unfolds through sustainable growth or boom-bust cycles depends largely on the choices we make in the next few years. The early signs (including voices like Altman’s warning about overexcitement) suggest we might actually be learning from history.

The AI bubble’s human impact could be fundamentally different. Previous bubbles destroyed jobs when they burst. AI might destroy jobs while it’s still inflating. If AI actually delivers on its automation promises, we could see the first bubble that eliminates more employment during its rise than its fall.

This creates an unprecedented social risk: a technology bubble that succeeds in its goals might cause more disruption than one that fails. The Railway Mania gave Britain train networks and industrial jobs. The dot-com bubble gave us e-commerce and digital careers. The AI bubble might give us unprecedented productivity and fewer jobs. That’s a social equation we haven’t solved.

Dwarkesh Patel arguing why he doesn’t think AGI is right around the corner.

But the fundamental problem is that LLMs don’t get better over time the way a human would. The lack of continual learning is a huge huge problem. The LLM baseline at many tasks might be higher than an average human’s. But there’s no way to give a model high level feedback. You’re stuck with the abilities you get out of the box. You can keep messing around with the system prompt. In practice this just doesn’t produce anything even close to the kind of learning and improvement that human employees experience.

The reason humans are so useful is not mainly their raw intelligence. It’s their ability to build up context, interrogate their own failures, and pick up small improvements and efficiencies as they practice a task.

How do you teach a kid to play a saxophone? You have her try to blow into one, listen to how it sounds, and adjust. Now imagine teaching saxophone this way instead: A student takes one attempt. The moment they make a mistake, you send them away and write detailed instructions about what went wrong. The next student reads your notes and tries to play Charlie Parker cold. When they fail, you refine the instructions for the next student.

This just wouldn’t work. No matter how well honed your prompt is, no kid is just going to learn how to play saxophone from just reading your instructions. But this is the only modality we as users have to ‘teach’ LLMs anything.

Dave DeGraw ranting about his frustration with vibe coded PRs and asking the most important question.

Is anyone making money on AI right now? I see a pipeline that looks like this:

• “AI” is applied to some specific, existing area, and a company spins up around it because it’s so much more “efficient”
• AI company gets funding from venture capitalists
• AI company give funding to AI service providers such as OpenAI in the form of paying for usage credits
• AI company evaporates

This isn’t necessarily all that different than the existing VC pipeline, but the difference is that not even OpenAI is making money right now.

Ha!

This comment from potatolicious on Hacker News about how AI has removed the deterministic expectations.

…I was trained as a classical engineer (mechanical), but pretty much just write code these days. But I did have a past life as a not-SWE.

Most classical engineering fields deal with probabilistic system components all of the time. In fact I’d go as far as to say that inability to deal with probabilistic components is disqualifying from many engineering endeavors.

Process engineers for example have to account for human error rates. On a given production line with humans in a loop, the operators will sometimes screw up. Designing systems to detect these errors (which are highly probabilistic!), mitigate them, and reduce the occurrence rates of such errors is a huge part of the job.

Likewise even for regular mechanical engineers, there are probabilistic variances in manufacturing tolerances. Your specs are always given with confidence intervals (this metal sheet is 1mm thick +- 0.05mm) because of this. All of the designs you work on specifically account for this (hence safety margins!). The ways in which these probabilities combine and interact is a serious field of study.

Software engineering is unlike traditional engineering disciplines in that for most of its lifetime it’s had the luxury of purely deterministic expectations. This is not true in nearly every other type of engineering.

If anything the advent of ML has introduced this element to software, and the ability to actually work with probabilistic outcomes is what separates those who are serious about this stuff vs. demoware hot air blowers.