Something I never got to do in my past life as an AWS commentator is "predictions." It turns out that when you have access to confidential information, you can't very well make predictions that touch on those things without a serious lapse of ethics. Thus, any AWS prediction I make fell into two horrible failure modes: either I get it wrong, which is the least bad option, or else I get it right and everyone thinks I knew but somehow leaked other people's information, in which case I don't get to have a business anymore.
I don't have that problem in the AI space, because nobody's giving me their roadmaps. I'm not entirely sure I'd trust any of them, given how rapidly this space is evolving. And so, given that today's my birthday, I'm going to indulge myself by predicting things about the world of AI that may or may not come to pass, driven by overall trends.
The marginal cost of inference falls to zero
With open models rivaling frontier models, there’s a strong possibility that the cost of inference approaches a tiny premium over the cost to run its infrastructure. I'm not suggesting that you're going to run Kimi K3 on your laptop anytime soon, but folks like Baseten (who you should be paying attention to) and others are absolutely going to provide it to you on a cost-plus basis that makes things like Opus (let alone Fable) 5 pricing look like someone's retirement plan.
I'm also predicting that running a frontier lab whose primary business is selling inference against SOTA models will start to be shaped like a business whose margins are less "SaaS" shaped and more "suspiciously close to an airline's." Hence, I see the current scrabble as OpenAI and Anthropic both looking to love up the stack, and in so doing they start to look more like the AWS of yesteryear—backstabbing of "partners" and all.
Open harnesses won’t suck
Right now, both Codex and Claude Desktop alike are very similar, which is a polite way to say that they're both not great products, which is me desperately trying to upset friends at both companies by referring to their products as something scatological. I've lost count of the conversations I've had here in San Francisco with folks who're building their own agent harnesses. I think that this phase is going to pass (good god, it's not like we want to roll our own text editors either), and we're going to see a future iteration where the most common consumption pattern is an open harness or two around which the industry congeals like grease on a stove. Transparency around system prompts, token consumption, data exfiltration concerns, and more all point to this being increasingly important to the enterprise, whereas the developers want outcomes, not a weekend project just to wrangle the tools.
The fundamental nature of computers changes
The big shift that AI drives that I haven't heard articulated much is that for the first time since the dawn of computing, computers do what you mean instead of what you literally say. (Also somehow they are bad at math now.) That one shift reframes the entirety of computing from "know the magic incantation to make the computer do the specific thing you want" to "barely articulate the outcome you're after" as a viable path to an outcome.
We’re already seeing that developers aren’t getting tripped up on the syntactic nonsense they used to; in time, knowing how to write code by hand is going to be about as germane to software engineering as knowing assembly is today. Instead, the valuable marketable skill will be in how to coax the models into doing your bidding. That may not sound like a job (it is absolutely your boss’s job), and yet you’d also not think “knowing how to coax information out of a search engine” would have been a decades-long durable skill either.
Neoclouds are on a ticking clock
The reason the neoclouds are in business today is almost entirely due to supply constraints. There are virtually no enterprises who are pleased to be doing business with them; this is where they find themselves when they attempt to get GPU capacity from their usual hyperscale cloud providers, get told "get in line," and have business needs that won't wait on supply line physics. At some point in the future, supply constraints will ease and you'll be able to mostly get all the GPU you care to, from any provider you care to, much the way CPU bound instances work today. When that day comes, the neoclouds are going to have to have built a differentiated offering beyond pure GPU unless they want to follow in the wake of the swath of VPS providers that lost their market with the rise of cloud.
Model routing becomes a primitive
The current state of every piece of software chooses where to run its inference seems less sustainable to me than other paths. I can see a world where “inference” becomes an operating system interface that applications use, and based upon a variety of factors (how complicated is the task, is there currently an internet connection, has the user expressed a lack of caring about money) routes it accordingly. We’re already seeing model routers doing “smart routing;” bringing that decision tree closer to home yields many benefits.
SaaS will continue to thrive
The open source world learned previously that software is free as in puppy. We’re now learning that while vibe coding is fun, vibe maintaining sucks, and vibe maintaining someone else’s slop is just the absolute worst.
There’s strong business value to having a team of people care deeply about the problem domain, have the expertise to recognize when models are going off the rails, and can imagine your future needs in such a way that “rewrite the app when requirements change” isn’t your only path.
You will still have a place
Despite all the hoopla about automating folks’ jobs away, evidence of this happening at scale remains thin. Block claimed their 40% layoff was due to AI, but I’m a skeptic; I note they didn’t do big layoffs when all of their technical peers were doing them 1-2 years earlier, and “we’re the first of a new thing” absolutely plays better than “we’re staggering across the finish line hours after the race is over and the organizers have gone home.”
The shape of some jobs may change; the tools we use as technologists certainly will. But the two things that can’t be industrialized are wisdom and judgement, and it’s becoming increasingly clear that LLMs aren’t on a path to changing that. We’re in for some interesting times in the next few years, but we’ll still be here when the turmoil settles back down.
Hey - why did the updates stop? I like this one a lot more than your AWS blog!
Happy birthday, Corey!