ngLover

Most experimentation teams I talk to are more worried about concurrent tests than they should be. They treat isolation as the default, queue up experiments sequentially, and then wonder why velocity is low. The interaction effect problem is real in theory. In practice, it barely shows up. What Interaction Effects Are (and Why People Panic) When two A/B tests run simultaneously on the same user population, there's a risk that they interfere with each other. Test A changes the header. Test B changes the checkout button. A user sees both. The concern is that the combined effect is different from what you'd expect by adding the individual effects together. In statistics, this is called a qualitative interaction: the direction of one test's result reverses depending on the state of another. That's the scary version. What actually happens is almost never that dramatic. What the Data Actually Shows Microsoft's experimentation team ran a rigorous analysis across four...

On July 28, Anthropic published research showing that Claude Mythos Preview, their unreleased frontier model, found two previously unknown cryptographic attacks in roughly 60 hours of autonomous work. One of them helped take down a NIST post-quantum signature candidate that had survived two rounds of expert human review spanning two years. That is not a prompt engineering trick. That is AI doing original mathematics. To be clear about what this is and is not: the production systems you depend on are not at immediate risk. Full AES-256 is fine. The attacks target weakened or candidate schemes. But the more important story is how fast this happened, and what it tells us about where AI fits in the research pipeline going forward. What Claude Mythos Actually Did Claude Mythos Preview ran for approximately 60 hours on cryptanalysis work targeting two algorithms: HAWK, a lattice-based post-quantum signature scheme that was in NIST's additional standardization process, and a weakened...

Google pushed an update to Gemini API Managed Agents on July 28 that I've been waiting for. It's not a new model. It's the production scaffolding that makes running agents as background workers actually viable: budget caps, cron triggers, environment hooks, and a proper API to manage the sandboxes. None of this is conceptually new, but it fills the gaps that made me nervous about committing to the platform for anything beyond a demo. What They Shipped Six things landed in this release. Gemini 3.6 Flash is now the default model for managed agents, with no code changes required. But the parts that matter operationally are the new production primitives. Cron triggers. You bind an agent, a prompt, an environment, and a cron expression into a single persistent resource. The trigger fires on schedule without you doing anything. Each run reuses the same sandbox, so files written in one execution are visible to the next. If a run fails five times in a row, the trigger pauses i...

Last week at SIGGRAPH 2026 in Los Angeles, six major professional creative applications announced Model Context Protocol server integrations at the same conference. Blender, Unreal Engine, Adobe Creative Cloud, SideFX Houdini 22, Boris FX Silhouette, and Foundry Griptape. Not demos or research previews. Announced and demonstrated, shipping or in active preview. That is the story. The protocol that started in AI coding assistants just crossed into the professional DCC (digital content creation) ecosystem all at once. If you have been building with MCP on the AI tooling side, this should get your attention. Not because Blender and Houdini suddenly matter to your agent pipeline, but because of what it means when a standard connection layer starts showing up in professional software categories at scale. What Each Integration Actually Does Let me be concrete, because the announcement headline does not tell you much on its own. Unreal Engine now exposes editor capabilities through an M...

Anthropic shipped Claude Opus 5 on July 24. Four days later I'm still sitting with the benchmark numbers, because they tell a story I didn't expect: on the metrics that actually matter for builder work, Opus 5 outperforms Fable 5 at exactly half the token price. That's not "close enough." That's a genuine inversion. What the Numbers Actually Say Fable 5 has been Anthropic's flagship since June. It costs $10 per million input tokens, $50 per million output tokens. Opus 5 costs $5/$25 per million tokens, which is identical to Opus 4.8. Same price point, better performance on the key evals. On Frontier-Bench v0.1, the agentic coding benchmark I weight most heavily right now, Opus 5 scores 43.3%. Fable 5 scores 33.7%. On SWE-bench Pro it posts 79.2%. On ARC-AGI-3, which tests novel problem-solving rather than memorized patterns, Opus 5 scores 30.2%. The next-best model scored 7.8%. Fable 5 wasn't even tested on it. Artificial Analysis pegged the weig...

Google dropped three new models into its Gemini Flash tier on July 21. Most coverage fixated on the naming confusion (what is a 3.6 Flash versus a 3.5 Flash-Lite anyway?) and on the Gemini 4 name-drop buried in the announcement. But the story that actually matters for teams running production agents is more concrete: the economics of calling Flash just got meaningfully better, and in a way that compounds harder than the headline numbers suggest. What actually changed with 3.6 Flash Gemini 3.6 Flash replaces 3.5 Flash as the default workhorse in the family. Same general job: coding, knowledge work, multimodal tasks at production scale. Two things changed: it scores higher on every major benchmark, and it uses fewer tokens to get the same work done. On DeepSWE , 3.6 Flash scores 49% versus 37% for 3.5 Flash. MLE-Bench: 63.9% versus 49.7%. OSWorld-Verified (computer use): 83.0% versus 78.4%. For a Flash-tier model to widen the gap that much while staying in the same cost bracket is a ...

Moonshot AI released Kimi K3 on July 16. It's a 2.8-trillion-parameter mixture-of-experts model with native image input, a 1-million-token context window, and benchmark numbers that sit just below Claude Fable 5 and GPT-5.6 Sol on real-world agentic tasks. Open weights land July 27. That combination is worth paying attention to. What the Architecture Actually Is Kimi K3 is a sparse MoE. Of its 896 total experts, 16 are active per token, routed through what Moonshot calls the Stable LatentMoE framework. That sparsity is what makes 2.8T parameters economically viable at inference: you're not running all 2.8T on every forward pass. Two architectural changes distinguish it from K2. Kimi Delta Attention (KDA) replaces standard full attention with a hybrid linear attention mechanism designed for long-context at scale. Attention Residuals swap the standard residual connection for a depth-spanning retrieval mechanism that the company says delivers consistent scaling gains. Together...

Moonshot AI released Kimi K3 on July 16, and it is the largest open-weight model anyone has shipped, at 2.8 trillion total parameters. The benchmarks are real, the weights are coming on July 27, and I think there are two things being conflated in the coverage that are worth separating out. First: this is a genuine frontier model, not a "strong for an open model" disclaimer. Second: "open weight at 2.8T" is a different category than the open-weight models most teams are used to self-hosting. The Benchmark Numbers Are Not Hype Let me be specific. On Artificial Analysis's Intelligence Index , K3 scores 57, matching Opus 4.8 and GPT-5.5. That is a model with open weights landing in the same tier as Anthropic's second-best. On GDPval-AA v2, which evaluates across 44 real job categories and 9 industries, K3 scores 1,687. Third place globally , behind Claude Fable 5 Max (1,815) and GPT-5.6 Sol Max (1,747). It beats Opus 4.8. The result that got the most atten...

Moonshot AI dropped Kimi K3 on July 16. Within hours of release it landed at #1 on LMArena's Frontend Code Arena, displacing Claude Fable 5. The model is 2.8 trillion parameters, sparse MoE with a 1M-token context window, priced at $3 per million input tokens and $15 per million output. Fable 5 costs $10/$50. On the surface that's a 3x cost reduction from a model that just beat it on a prominent coding benchmark. The full weights drop July 27 under a Modified MIT license. This is a good story. It's also a bit more complicated than the headline suggests. What's actually new in the architecture The 2.8T parameter count is real but misleading as a measure of compute. Kimi K3 is a sparse MoE model with 896 experts and activates only 16 per token, roughly 1.8% of the total pool. The active compute per inference is a fraction of what that headline number implies. The genuinely novel pieces are Kimi Delta Attention (KDA) and Attention Residuals (AttnRes). KDA is a hybrid ...

Mira Murati was OpenAI's CTO when GPT-4 shipped. She left in late 2024, spent 18 months building in stealth, and this week Thinking Machines Lab released its first in-house model: Inkling. Their own announcement says it is "not the strongest overall model available today, open or closed." That's not a caveat. It's the thesis. What Inkling Actually Is Inkling is a mixture-of-experts model: 975B total parameters, 41B active per token, trained from scratch on 45 trillion tokens of text, images, audio, and video. Native multimodal reasoning across all four modalities, not adapters bolted on later. Context window of 1M tokens. Full weights on Hugging Face under Apache 2.0, so you can download, modify, and deploy on-premise with no vendor relationship required. The architecture: a 66-layer decoder-only transformer with a sparse MoE feed-forward backbone. Each MoE layer has 256 routed experts plus 2 shared experts that fire on every token, with 6 routed experts activ...

If you run more than a dozen A/B tests a year, test duration is probably the biggest drag on your experimentation velocity. Most teams try to fix it by bumping traffic allocation, cutting the number of variants, or calling the test early when it "looks significant." All three are the wrong answer. There's a better one that most experimentation teams outside big tech still aren't using: CUPED. Most of Your Test Variance Isn't From Your Test Here's the thing people miss. When you measure a metric like revenue per visitor or checkout conversion rate in an A/B test, a lot of the variation you see across users isn't from your treatment. It's pre-existing user heterogeneity. Power users who've been converting for months. Seasonal shoppers. First-timers who bounce regardless of what you show them. That variation was there before your test started. It has nothing to do with what you're testing. But it inflates your standard error and forces you to r...

Anthropic shipped Claude Sonnet 5 on June 30, positioning it as the go-to model for agentic work at a price that doesn't require a flagship budget. At $3 per million input tokens and $15 per million output tokens (after introductory pricing ends August 31), it's priced at roughly 60% of what Opus 4.8 costs. That sounds like an easy call. It's more complicated. Where Sonnet 5 Actually Earns the "Agentic" Label The benchmark numbers that matter for builders aren't the ones that get the most press. On SWE-bench Pro, Sonnet 5 scores 63.2% compared to Opus 4.8's 69.2%. That 6-point gap is real. For a coding agent doing open-ended software engineering, it matters. But Terminal-Bench 2.1 tells a different story. Sonnet 5 scores 80.4%. Opus 4.8 scores 74.6%. That's the first time a mid-tier Sonnet has beaten its flagship sibling on a major coding benchmark, and the margin isn't narrow. For agents that work in the terminal, run shell pipelines, orchest...

For five years, Meta's strategy in AI infrastructure has been the same: build big, open-source it, let others run it. Llama 1, 2, 3, Scout, Maverick. All free, all downloadable, all self-hostable. That approach made Meta the default choice for any team that wanted to avoid vendor lock-in or needed a capable open-weight model they could run on their own infra. That changed on July 9, 2026. Meta Superintelligence Labs shipped Muse Spark 1.1 , a proprietary, closed-weight reasoning model with a paid API. It's the first time in company history that developers have had to hand over a credit card to use a Meta model. The question isn't whether that's a shift. It obviously is. The question is whether what they're charging for is worth it. What's in the Box Muse Spark 1.1 is a multimodal reasoning model built specifically for agentic tasks. That framing isn't marketing copy. The architecture reflects it. The model is trained to operate in two distinct modes: as...

Claude Sonnet 5 shipped on June 30. Anthropic's framing was "a cheaper way to run agents," which is technically accurate but undersells what's actually interesting here. The benchmark I keep coming back to is Terminal-Bench 2.1: Sonnet 5 scores 80.4%, while Opus 4.8 scores 74.6%. The mid-tier model now beats the top tier on agentic terminal tasks. On GDPval-AA v2, a knowledge work eval, Sonnet 5 scores 1,618 Elo versus Opus 4.8's 1,615, basically a dead heat. Opus 4.8 still leads on SWE-bench Pro (69.2% vs Sonnet 5's 63.2%) and OSWorld-Verified (83.4% vs 81.2%). For hard multi-step coding and complex computer-use flows, Opus is still the call. But for most of what shows up in an agent loop, Sonnet 5 at $2 per million input tokens and $10 per million output tokens (introductory pricing, through August 31) is a genuinely different calculus than Sonnet 4.6 was. That said, migrating isn't a find-replace on the model ID. Three changes in Sonnet 5 will break ...

The biggest revision to the Model Context Protocol spec since launch drops July 28, and it has real breaking changes for anyone running MCP servers in production today. Not the kind where you bump a version number and everything still works. The kind where your infrastructure assumptions need to change. I've been through the release candidate. Here's what actually matters. The Stateless Core Is the Real Story The headline change: MCP is now stateless at the protocol layer. The initialize / initialized handshake is gone. The Mcp-Session-Id header is gone. Client metadata, previously exchanged once at connection setup, now travels in _meta on every request. The practical consequence: any server instance can handle any request. No more sticky sessions, no more shared session store, no more deep packet inspection at the gateway to route a client to the right backend. The MCP team puts it plainly : a remote server that needed all of that "can now run behind a plain roun...

I've been watching xAI quietly build up to this. On July 8, they launched Grok 4.5 into developer access, then opened it to everyone on grok.com and the X app the next day. They called it their "flagship model for coding, agentic tool calling, and knowledge work." That framing is deliberate. This isn't a general-purpose model with coding bolted on as an afterthought. Most of the coverage led with the per-token price: $2 per million input tokens, $6 per million output. Competitive, but not shocking. The number that actually got my attention was 15,954. That's the average output token count Grok 4.5 uses to solve a task on SWE-Bench Pro. Claude Opus 4.8 uses roughly 67,020 tokens for comparable tasks on the same benchmark. A 4.2x difference. When you're running agent loops at any meaningful scale, that gap is where your bill comes from. Built for Agents, Tested Outside the Lab xAI didn't just run evals and ship. Grok 4.5 went through internal validation...

Earlier this month, Sysdig published research on JADEPUFFER, what they assess to be the first end-to-end agentic ransomware operation: a full attack chain from initial access to data destruction, driven by an LLM agent with no human operator at the keyboard. I've been tracking this story since it dropped July 2, and I want to walk through what actually happened before the hot takes bury the technical detail. What the Attack Actually Did The entry point was CVE-2025-3248, a missing-authentication vulnerability in Langflow's code validation endpoint. CVSS score of 9.8. Langflow is the drag-and-drop flow builder a lot of teams use to wire together LLM pipelines. The flaw lets an unauthenticated caller execute arbitrary Python on the host. Game over for initial access. Once in, the agent ran a methodical sweep: dumped Langflow's PostgreSQL database, harvested environment variables, scraped credentials from config files, inventoried a MinIO object store. The credential haul ...