Recent Posts
Social•Nov 25, 2025
New Model Outperforms Gemini 3 with Greater Polish
My verdict is that it's significantly better than Gemini 3. It's at least as smart and just got more polish to it. Alignment on little details also significantly higher. Gemini 3 gets many things mixed up after a half-dozen messages, and completely confused after compaction.
By Alex J. Champandard
Social•Nov 25, 2025
Opus 4.5 Executes Tasks Seamlessly Beyond Token Limits
With Opus 4.5, it seems you don't need to ask multiple times or ORDER it to do work, it just gets stuff done — even beyond 50% the token limit and after chat compaction! This kind of message is a thing...
By Alex J. Champandard
Social•Nov 20, 2025
Benchmarks Mislead; Human Review Is the Real Bottleneck
These kinds of benchmarks are misleading without a joint metric showing much work was necessary by humans after the fact. How much time to clean up that 2h42m of code? Style and architecture need to make sense, not just passing tests. That's...
By Alex J. Champandard
Social•Nov 19, 2025
LLMs Lose Context After 100k Tokens, Need Frequent Resets
People working on basic code and reset their Agent chats every 4-5 replies I envy you. Having to work on deep context design work and at about 100k tokens, LLMs start to get lazy / confused. I resorted to giving them...
By Alex J. Champandard
Social•Nov 19, 2025
Gemini 3: Fast but Unreliable, Files Get Corrupted
Gemini 3 review: it's fast, it's not dumb, but it's completely unusable in practice. It will get lost after a few edits then completely trash the file: issuing patch commands that include line numbers at best, and at worst it will...
By Alex J. Champandard
Social•Oct 31, 2025
Mid-Tier Language Models only Hit 75‑85% on Basic Math
Language models perform poorly on high-school math? 🙄 You don't want to hear this, but the problems started in grade-school. The moment we (collectively) found acceptable that mid-tier models could score only 75%-85% on a GSM test set of 1.32k straightforward...
By Alex J. Champandard
Social•Oct 30, 2025
Fast Coding Model Feels Overpriced Despite Performance Gains
The speed of a faster coding model is worth it, but it seems mis-priced. C1 gobbles through files, reasons more, expect extra feedback to reach similar place as slower model do with less of everything. Intuitively it feels more expensive "the...
By Alex J. Champandard
Social•Oct 24, 2025
Better, Smaller Datasets Trump Massive Web Crawling
Great idea for a metric to further improve what datasets the models train on. It likely leads to an answer that is not web-scale crawling... Less data is often better, better data takes less.
By Alex J. Champandard
Social•Oct 23, 2025
Meta's Soft Tokens Yield Modest Gains on GSM8k
This paper from Meta about "Soft Tokens" in RL is interesting; it allows LLMs to invent their own non-discrete (recursive) representations in order to solve problems better... Results are mixed though: it's only a few percent better on GSM8k from pass@4...
By Alex J. Champandard
Social•Oct 23, 2025
AI Firms Launch Browsers to Dodge Scraping Liability
The reason AI companies are rushing to release browsers: they don't want the responsibility / liability of scraping on their servers. They need to push that to the users! We'll be moving into an ever more gated internet soon...
By Alex J. Champandard
Social•Oct 21, 2025
Only Path to Shippable Code From AI Agents?
Is this the only way to get coding agents to produce shippable quality code? https://t.co/bZvxMN6JEv
By Alex J. Champandard
Social•Oct 20, 2025
More Compute Trumps All Other AI Strategies
Without checking, what is the message behind the "Bitter Lesson", in your opinion? (a) all other things being equal, using more compute is better. (b) more compute is better than all the other things put together.
By Alex J. Champandard
Social•Oct 20, 2025
Internal AI May Replace Risky Open‑Source Contributions
Even though this particular example worked out as you'd expect today, Open Source dynamics will certainly change. Accepting and merging contributions is always a risk and has high cost, so trusting an internal AI system for minor codebase improvements may become...
By Alex J. Champandard
Social•Oct 20, 2025
Avoid the RL Hammer: Choose the Right Tool
If you sub-optimally define any problem to require RL, when most can be solved with different approaches, then of course the RL hammer looks like the right solution! Rather than defending RL thru semantics, better would be to ask: how can...
By Alex J. Champandard
Social•Oct 18, 2025
RL Fine‑tuning LLMs Caps at 61% Success Rate
It's hard to overstate how devastating this paper is, not only for reinforcement learning. They spent $4m of compute to find out that RL on LLMs basically taps out at 61% "asymptotic pass rate" (exact rate depends on context), but they...
By Alex J. Champandard