❧ Cal Newport, whom I’ve long admired, recently wrote for the New York Times explaining why we shouldn’t be panicked by AI doom stories. I largely agree with his op-ed and yet I find it a bit strange.
His argument is basically this: The AI elite are Rationalists or heavily influenced by them. Rationalism is extreme, obsessive, overconfident, and weird. Therefore we should completely discount the alarms they raise.
Maybe he hedges enough so he’s not saying that the discounting should be complete, but it sure feels that way in this op-ed and his other posts and podcasts.
I have no problem stipulating that Rationalists and their extended universe seem like a weird sex cult, and that this is grounds enough not to take them completely seriously. But that just means we need to be more skeptical about their claims, not dismiss them altogether. If nothing else, the fact that they’ve built and are in charge of these impressive systems should recover for them some of the credibility they lose for being weirdos.
As exasperated as Newport and other non-apocalyptic technologists seem to be with the ASI-pilled Bay Area weirdos, the fact that those weirdos believe in X tells us nothing about the likelihood of X. The expert critics are better off patiently explaining why AI can’t technically lead to such doom (if they can). Indeed, just as much as I would like the AI doomers to rely less on parables and thought experiments and more on technical details, I would also like the critics to tell me what probabilities we should assign to catastrophic outcomes and why. [09.20.26]
❧ If you’re not someone who has coded much, I’m not sure you can appreciate how merited the more reasonable hype around AI is. Things that used to take me weeks to do now take me hours. And more importantly, it’s allowed me to customize my computing experience more than I ever imagined would be possible.
No matter how much you love a piece of software, there is always a thing or two you wish it would do differently. More typically the software available to us is nothing like what we would want for a task, especially in terms of a UI that makes sense to us.
Now that coding agents have gotten quite good, however, I don’t hesitate to just write from scratch the app I want to use, with the exact UI and functionality I want, even if I already have a good enough commercial alternative. Todo lists, editors, whatever. And I will also make bespoke tools that do something for which I’m the only customer.
It’s remarkable what I can get done, and it makes me think that this power is going to have to show up in the productivity stats eventually. Even if it’s just software that AI supercharges.
Yet I can also see the downsides. Since I didn’t write this code myself, I don’t really understand it and don’t know how good it is. That may come back to bite me. Given my coding proficiency, however, I can’t believe it’s doing worse than I would do. That said, the real alternative to a vibecoded app is not one I would write by hand, but instead simply using the hardened commercial app. I’m also taking on a maintenance and upkeep tax whose size I can’t now appreciate.
And then there’s the addiction piece. This can’t be an original insight, but using AI lately has reminded me of Addiction by Design, the book on machine gambling about which Stably and I recently podcasted. Sitting alone at a machine, dropping in ‘credits’ and ‘tokens’ of inscrutable value, pulling a lever, getting back variable rewards, repeating this turn after turn, and ultimately losing track of time and oneself. In a way all flow is like this, but the jury may still be out on whether in the long run vibecoding induces the ‘good’ kind of flow that produces mastery and a sense of agency, or the ‘bad’ kind that surrenders agency to the machine’s rhythm. [09.20.26]
❧ The Internet Archive says the Wayback Machine is having to block legitimate users to fend off a new wave of automated attacks. Coincidentally or not, working on a project this morning Claude suggested to me that we scrape the Wayback Machine for data that the original site was blocking us from accessing. [09.15.26]
❧ Dario Amodei argues AI companies must slow capability advancement to let safety research catch up. And he wants the government to give them an antitrust waiver so they can collude to do it. A few things strike me about his post.
He admits that Anthropic have has incidents similar to Hugging Face. I wonder if agrees there should be civil and criminal liability for such incidents, and why not?
He doesn’t mention existential risk or gray-goo scenarios. Instead his worry is “that in 6–12 months such a swarm could be capable of taking over the entire internet with a persistent botnet (potentially causing hundreds of billions of dollars in damage)”. That’s my concern as well. Such a disaster doesn’t require AGI or ASI or consciousness or anything more than an out of control but smart loop.
His argument is that we need a government-enforced cartel to slow down development before his company and others blow us all up, but let them develop fast enough to still get to a point where China can’t ‘beat’ us in the race to… he doesn’t quite say. He doesn’t make clear what ‘winning’ looks like. For the U.S. to win, do we have to indefinitely prevent China from gaining access to the state of the art models? If so that’s going to require a war. He says,
Some may believe these measures make it more difficult to cooperate with China, but I believe the opposite is true: these measures increase the leverage held by democracies and make an agreement more likely in the future.
Bob Wright is correct here. If this technology is as important to a nation’s sovereignty as Amodei says, then China won’t accept being shut out. And if it sees the U.S.’s lead growing to the degree Amodei wants it to, then why would China not resort to any means to stop us, including targeting Taiwan where all the chips are made?
Taking Amodei’s argument seriously, one would have to consider nationalizing and tightly controlling the technology the way we do with nuclear. [09.15.26]
❧ Police officers search license plate databases with absurd reasons like “LOL” and “idk.” This is a real thing. [09.15.26]
❧ This is a sample entry added to test the day-grouping headers with a second day in the feed. [09.14.26]