ngLover

Supabase open-sourced their evals framework last week (supabase/evals, Apache-2.0), and I think it's the most useful thing published about AI coding agent evaluation in months. Not because of which model topped the leaderboard. Because of how they designed the measurement itself. What They're Testing and Why It's Hard Supabase built their evals around a three-axis grid: products (database, auth, storage, edge-functions, realtime, cron, queues, vectors, data-api), topics (RLS, security, migrations, SQL, SDK, observability, self-hosting, declarative-schema), and stages (build, deploy, investigate, resolve). That last axis is where it gets real. "Build" is the easy part. Any capable coding agent can scaffold a schema. "Deploy" and "Investigate" are where agents start to diverge. "Resolve" is where you find out if an agent can fix a broken RLS policy without silently breaking three others it didn't know existed. They're runni...

In early July, AWS raised EC2 Capacity Block prices 20% across its main GPU instance families. That is the second hike in six months, following a 15% increase on January 4. If you are on H100-based P5 instances, you are now paying $5.19 per GPU hour. B300 slots hit $14.04 per GPU hour. For a team running a p5e.48xlarge continuously, the January hike alone pushed the bill from $34.61 to $39.80 per hour. The July hike adds another layer on top of that. This matters because the compute story in 2026 has been running in two directions, and those directions are diverging fast. API Token Costs Fell. Reserved GPU Costs Did Not. If you are calling inference APIs, you are paying a fraction of what you paid two years ago. Per-token prices have dropped roughly 80% in 2026 alone and around 280-fold since 2022. Claude, Gemini Flash, and the latest DeepSeek models are all priced in a range that would have seemed impossible eighteen months ago. But that is API inference. If you are reserving GPU...

A 2026 NBER study quantified something a lot of engineering teams have already felt: AI coding agents increased commit rate by 180%, but software releases only grew by 30%. That gap is the story. You did not solve your velocity problem by adopting Claude Code or Cursor. You moved it downstream. On July 29, BrowserStack launched Test Companion , an agentic test automation tool built directly into the IDE. It's worth understanding why it exists and what it tells you about where AI tooling is headed. What the 180/30 Gap Actually Means When a coding agent can spin up a full feature in an afternoon, the constraint shifts. It's no longer "how fast can we write the code." It's "how fast can we trust that code enough to ship it." I've seen this play out on teams using Claude Code seriously. Output goes up fast. But PR queues get longer, QA cycles stretch out, and the release cadence barely moves. The agents didn't fix deployment velocity. They expo...

On August 1, 2026, OpenAI announced Astra, calling it their next major model family. They didn't release a product. They dropped a GitHub repo containing Lean 4 certificates formally verifying ten solutions to open problems in mathematics, some unsolved for over a decade. The standout: a construction proving non-sofic groups exist, a question Mikhail Gromov posed in 1999 that sat open for 27 years. The estimated token cost to find all ten solutions: roughly $2,000 at Sol API rates. That's about $200 per problem spanning group theory, von Neumann algebras, quantum complexity, and lattice cryptography. Sit with that number for a moment. What Astra Actually Is Astra is not a public product yet. OpenAI is positioning it as a model family built for long-horizon multi-agent work. The design is explicit: multiple agents working together on a single task for hours or days, not seconds. The system plans, tests its own output, revises, and keeps going without needing you to steer ea...

n8n shipped a significant update on July 29 that I've been waiting for since they announced MCP integration earlier this year. The headline: n8n now works as both an MCP client and an MCP server. That's not just a configuration option. It changes how you think about the workflows you've already built. What the July 29 Release Actually Shipped Four things worth paying attention to (full details in n8n's release notes ): Native MCP server mode. Any n8n workflow can now be exposed as a callable tool to external AI clients. Claude Desktop, a custom agent, your own LLM-powered interface. If the client speaks MCP, it can call your workflow. You toggle this at the workflow level, or bulk-enable it across an entire project folder from the new folder actions menu. Native MCP client mode. n8n's AI Agent nodes can now discover and call external MCP-compliant tools directly, without you writing a custom API wrapper or an HTTP request node. The agent resolves the tool lis...

Most experimentation teams I talk to are more worried about concurrent tests than they should be. They treat isolation as the default, queue up experiments sequentially, and then wonder why velocity is low. The interaction effect problem is real in theory. In practice, it barely shows up. What Interaction Effects Are (and Why People Panic) When two A/B tests run simultaneously on the same user population, there's a risk that they interfere with each other. Test A changes the header. Test B changes the checkout button. A user sees both. The concern is that the combined effect is different from what you'd expect by adding the individual effects together. In statistics, this is called a qualitative interaction: the direction of one test's result reverses depending on the state of another. That's the scary version. What actually happens is almost never that dramatic. What the Data Actually Shows Microsoft's experimentation team ran a rigorous analysis across four...