Safetymogging and marginmaxxing the accelecels
Believing Anthropic and deploying physical AI
Thinking about AI safety which is ultimately inextricable from the underlying business. What’s happening now around text and coding models is obviously coming soon to physical AI.
Ant’s Financials
Two important things came out of Anthropic last week:
Dario made the claim that we need to “pace the frontier” and collectively slow down AI development through a series of controls and agreements
Anthropic leaked some pseudo-financials that claimed they’re profitable and high gross margin… if you exclude stock comp, training, and revenue share - so you know all their costs…
Even if we take everybody at their word and believe their best intentions, we still have to accept that these are basically the same story or at least the simultaneity means we have to treat them as such.
So if you believe that Dario and the Anthropic leadership genuinely does believe in AI Safety as a worthy cause, it is impossible for them to act on it unilaterally because of how quickly their core assets (models) depreciate. If everyone doesn’t slow down at once, Ant dies. Conveniently for them if everyone (foreign and domestic) were to slow down, they would win!
And this is of course inextricable from their finances.
Training is a cost of revenue, not an R&D cost. Stop training, everyone churns and moves on to your competitor or a distilled Chinese version. The labs are perpetually investing unbelievable sums in building these fast depreciating assets (capex and training). And each new SOTA model costs more than the last; we’re rapidly approaching >$1B training runs.
I think it’s worth believing the people worried about AI safety, or at least hearing them out. But until they do anything that runs counter to their self-interest it’s hard to trust their proposed remedies.
And of course every argument from every side (safety, open source, etc.) just sounds like motivated reasoning from partisans with a ton of stake. Because it is!
And from a purely financial perspective, “80% gross margins when we strip out training and add back revenue share” is obviously hilarious. Many have likened it to WeWork’s “community-adjusted EBITDA” but it’s actually even more ridiculous than that.
“Community-adjusted EBITDA” was an insane name for a dumb-but-not-insane concept (essentially four walls margins). If Adam Neumann had been a normal guy and just called it “same store gross profits” or something like that no one would have blinked. Ant’s margins are a sanely named but utterly insane concept.
New/made up metrics are fine. Finances are about telling a story and when you invent a metric you do so to highlight a part of that story that is otherwise un- or undertold.
A few decades ago EBITDA was considered a ridiculous made up metric (how could you exclude interest and taxes!) but the point was that it showed the debt load a business could support and helped indicate the cash a depreciation heavy business like telecom could throw off. So it was arguably specious but useful.
In the case of Anthropic, it's not clear what the metric is supposed to tell us other than “IPO incoming” except perhaps they don’t plan on needing to keep training frontier models because they think they can get everyone else to stop too.
Tired: AI Safety. Wired: Robot Safety!
The future is not going to be wholly discontinuous with the past. And in the past when a whole class fails, it hasn’t failed because of technology. It’s failed because of very human problems that get in the way of successful deployment.
The US is 20–30 years behind on solar/electrification, because people thought Carter was a little bitch for putting panels on the WH
We’re ≈50 years behind where we could be on nuclear because it was too scary to keep developing
We’re just now falling off the golden path of vaccination because of anti-science conspiracy theorists. This will cost the world millions of unnecessary deaths.
The ports will never be automated because of unions.
AVs will probably not launch in New York in the next decade because of fear and regulatory capture.
These are not technology problems and they are not unique.
They are a fear-based doom loop that drives underinvestment and slow deployment, and eventually makes social/political problems LOOK LIKE (or even cause) technology problems.
We can’t re-run the AI comms playbook (it’ll kill us all). Instead, it’s incumbent on technology leaders to make an affirmative, positive and humanist case for physical AI, robotics, automation etc. The credibility of our field is at an all time low. Datacenters and LLMs are on their way to becoming the most salient and negative issue areas since the Iraq War.
Rather than just trying to hill climb on success/viability, we need hill climb on acceptance or climb down the hill of scariness.
And as we/I look for bets to make in the pAI supply chain with a particular eye on system integrators (remember, deployment is the task of our time), it seems like there’s a whole universe of opportunities specifically around safety, compliance, and insurance to try to smooth off some of the sharp edges.
Safety will both drive regulatory requirements that need to be solved/scaled through and create softer requirements on the part of buyers, especially as robots interact in mixed environments with humans.
Otherwise
Last weekend I had dinner at Sam’s on Court Street, newly reopened by Georgia and the Long Island Bar crew. Really fun dinner and definitely worth a visit. The veal parm was a big hit.
The World’s Most Reliable AI Detector Has a Human Problem: Armed with a near-perfect detection tool, Pangram is fighting to convince the public AI cheating can be caught. Regular readers will know I’m a big Pangram fan but with time it’s clear that we really don’t have a good norms around AI writing. I absolutely maintain that if you didn’t take the time to write, I’m not gonna take the time to read it BUT right now we don’t have good models between “human” and “slop” and I think it’s fair to worry about witchhunts. “One drop counts” doesn’t strike me as the optimal outcome, nor does giving the AI total credit for execution of human ideas/thinking.
I’ve been saying that Fort Greene is becoming the new West Village while the West Village becomes the new Murray Hill... But then last night I had drinks in the West Village and I take it all back. Kind of a horror show! However I do stand behind the fact that TikTok is ruining my neighborhood.
This board game (which have never been successfully finished!) sounds like complete and utter insanity! Also kind of my personal hell. Makes cones of dunshire look like checkers.
While I was on vacation I built an email bot that read my email for me and texted me about anything important. This was great. I fully turned off notifications for the duration, explained what was important, and got to be mostly unplugged except when needed urgently. Even just having notifications on pulls you into the black box. This was a great little personal agent use case for me. Big fan and will do it again next time I’m away.