Andrej Karpathy had a really interesting post on X over the weekend: Here’s the tweet in a visual funnel format I think this diffusion question is a really important one. In particular, how quickly does AI diffuse? What Karpathy’s X post shows (in his opinion) is that diffusion is happening slower. PNC research shows that only ~2% of US households are paying for some sort of AI service. What is the diffusion gap? It’s the delta between what you can accomplish with models (ie how powerful they are) and how the average person uses models. Here’s why I think it matters. A big question in the market today is “will the revenues match the promised CapEx commitments.” It’s a big reason why all the semi-cap stocks got clobbered this week, and questions like “Will

In the early days of the cloud era, when dissecting costs most people just looked at the basic price of compute and storage (for services like EC2 and S3). Overtime, the real focus shifted to egress. In today’s AI world, the initial focus is on the input/output pricing per 1M tokens, but the real focus is on the cache read rate. What is the cache read rate? The cache read rate is what a model provider charges to essentially re-reat input tokens it processed / stored from a prior call. If you think about an agent loop with the same system prompt, same tool definitions, etc, all of this history / context doesn’t need to be resent and reprocessed every time (and this turns out to be a majority of input tokens). Here’s some data on the headline prices everyone looks at (just a blended input/ou

Lots of drama this week on math proofs! I’m sure most have seen the back and forth between folks at OpenAI and Anthropic about a Navier Stokes proof. The point of this post is not to dig in on all the details (too much commentary on that online already, and too much speculation…) Instead - here’s a very high level of one part of the saga (and then something that I found interesting). Producing a proof of the Navier Stokes existence and smoothness problem is one of seven Millennium Prize Problems that each have a $1m prize attached to it (ie solve it, get a $1m prize). And OpenAI said they did it. 10k agents working in parallel for 3-4 days, >100 pages of manuscript and a formalized proof. The drama around this are allegations that OpenAI piggybacked off research two professors had done

This week we got a peak into OpenAIs new model Astra. I thought this was an important model because it’s rumored to use a new technique called recurrent depth. In this post I want to discuss what that means and what the implications are. But first - a quick timeline of some of the larger breakthroughs in model architecture: 2017: The Transformer era. The leap here was self-attention mechanism. Before 2017 recurrent neural networks (RNNs) processed text word-by-word. The now famous paper Attention is Al You Need introduced the Transformer architecture which made parallel training possible 2018-2020: Decoder-Only Architectures. The leap here was autoregressive scaling and in-context learning. Early on transformers used a encoder-decoder setup (like BERT). OpenAI flipped this by scaling decod

$SpaceX(SPCX)$ lands another computing deal, this time with Reflection, an open source model development company. $150m / month for GB300s. SpaceX the Neocloud! Deal 1 with Anthropic Colossus 1 and Colossus 2. Anthropic took all of Colossus 1 $1.25b / month ~325k total chips, split roughly into 150k H100s, 50k H200s, 125k GB200s ~$5-$6 / hour blended Deal 2 with $Alphabet(GOOG)$ Colossus 2 $920m / month ~110k GB200s / GB300s (not clear which chip) ~$11-$12 / hour Deal 3 with Reflection Colossus 2 $150m / month ~18k GB300s Amount of chips not disclosed, but if we use the same hourly rate as the Google deal, the implied number of GB300s is ~18k In Summary $2.32B / month >$10 / hour for Blackwells (which

Workflows workflows workflows…I’ve probably written about this in the past (hard to keep track of everything I’ve written at this point, so I’m sure there will be some repetitive content!) but I wanted to circle back to it. For SaaS companies, the conventional wisdom was that you developed a moat if you built a system of record - a platform that stored data. If you owned / controlled the data, you had a moat! Data has gravity. True, but that wasn’t the real moat. The real moat was the hundreds of workflows that grabbed data from that system of record and then got work done. Sometimes those workflows originated from the system of record platform itself. Sometimes the workflows originated elsewhere, and one stop of the workflow grabbed data from the system of record. The real moat is owning