ngLover

Posts

Kimi K3: 2.8 Trillion Open Parameters and What the Weight Release Actually Changes

Moonshot AI released Kimi K3 on July 16. It's a 2.8-trillion-parameter mixture-of-experts model with native image input, a 1-million-token context window, and benchmark numbers that sit just below Claude Fable 5 and GPT-5.6 Sol on real-world agentic tasks. Open weights land July 27. That combination is worth paying attention to. What the Architecture Actually Is Kimi K3 is a sparse MoE. Of its 896 total experts, 16 are active per token, routed through what Moonshot calls the Stable LatentMoE framework. That sparsity is what makes 2.8T parameters economically viable at inference: you're not running all 2.8T on every forward pass. Two architectural changes distinguish it from K2. Kimi Delta Attention (KDA) replaces standard full attention with a hybrid linear attention mechanism designed for long-context at scale. Attention Residuals swap the standard residual connection for a depth-spanning retrieval mechanism that the company says delivers consistent scaling gains. Together...

Kimi K3 Just Reached the Frontier. What Open Weights at 2.8T Actually Mean.

Moonshot AI released Kimi K3 on July 16, and it is the largest open-weight model anyone has shipped, at 2.8 trillion total parameters. The benchmarks are real, the weights are coming on July 27, and I think there are two things being conflated in the coverage that are worth separating out. First: this is a genuine frontier model, not a "strong for an open model" disclaimer. Second: "open weight at 2.8T" is a different category than the open-weight models most teams are used to self-hosting. The Benchmark Numbers Are Not Hype Let me be specific. On Artificial Analysis's Intelligence Index , K3 scores 57, matching Opus 4.8 and GPT-5.5. That is a model with open weights landing in the same tier as Anthropic's second-best. On GDPval-AA v2, which evaluates across 44 real job categories and 9 industries, K3 scores 1,687. Third place globally , behind Claude Fable 5 Max (1,815) and GPT-5.6 Sol Max (1,747). It beats Opus 4.8. The result that got the most atten...

Kimi K3 Costs 3x Less Than Fable 5 and Won the Coding Arena. The Verbosity Tax Is Real.

Moonshot AI dropped Kimi K3 on July 16. Within hours of release it landed at #1 on LMArena's Frontend Code Arena, displacing Claude Fable 5. The model is 2.8 trillion parameters, sparse MoE with a 1M-token context window, priced at $3 per million input tokens and $15 per million output. Fable 5 costs $10/$50. On the surface that's a 3x cost reduction from a model that just beat it on a prominent coding benchmark. The full weights drop July 27 under a Modified MIT license. This is a good story. It's also a bit more complicated than the headline suggests. What's actually new in the architecture The 2.8T parameter count is real but misleading as a measure of compute. Kimi K3 is a sparse MoE model with 896 experts and activates only 16 per token, roughly 1.8% of the total pool. The active compute per inference is a fraction of what that headline number implies. The genuinely novel pieces are Kimi Delta Attention (KDA) and Attention Residuals (AttnRes). KDA is a hybrid ...

Thinking Machines Released Inkling. They Say It's Not the Best. That's the Strategy.

Mira Murati was OpenAI's CTO when GPT-4 shipped. She left in late 2024, spent 18 months building in stealth, and this week Thinking Machines Lab released its first in-house model: Inkling. Their own announcement says it is "not the strongest overall model available today, open or closed." That's not a caveat. It's the thesis. What Inkling Actually Is Inkling is a mixture-of-experts model: 975B total parameters, 41B active per token, trained from scratch on 45 trillion tokens of text, images, audio, and video. Native multimodal reasoning across all four modalities, not adapters bolted on later. Context window of 1M tokens. Full weights on Hugging Face under Apache 2.0, so you can download, modify, and deploy on-premise with no vendor relationship required. The architecture: a 66-layer decoder-only transformer with a sparse MoE feed-forward backbone. Each MoE layer has 256 routed experts plus 2 shared experts that fire on every token, with 6 routed experts activ...

Your A/B Tests Are Running Longer Than They Have To. CUPED Fixes That.

If you run more than a dozen A/B tests a year, test duration is probably the biggest drag on your experimentation velocity. Most teams try to fix it by bumping traffic allocation, cutting the number of variants, or calling the test early when it "looks significant." All three are the wrong answer. There's a better one that most experimentation teams outside big tech still aren't using: CUPED. Most of Your Test Variance Isn't From Your Test Here's the thing people miss. When you measure a metric like revenue per visitor or checkout conversion rate in an A/B test, a lot of the variation you see across users isn't from your treatment. It's pre-existing user heterogeneity. Power users who've been converting for months. Seasonal shoppers. First-timers who bounce regardless of what you show them. That variation was there before your test started. It has nothing to do with what you're testing. But it inflates your standard error and forces you to r...

Claude Sonnet 5: Cheaper Per Token, Not Always Cheaper Per Task

Anthropic shipped Claude Sonnet 5 on June 30, positioning it as the go-to model for agentic work at a price that doesn't require a flagship budget. At $3 per million input tokens and $15 per million output tokens (after introductory pricing ends August 31), it's priced at roughly 60% of what Opus 4.8 costs. That sounds like an easy call. It's more complicated. Where Sonnet 5 Actually Earns the "Agentic" Label The benchmark numbers that matter for builders aren't the ones that get the most press. On SWE-bench Pro, Sonnet 5 scores 63.2% compared to Opus 4.8's 69.2%. That 6-point gap is real. For a coding agent doing open-ended software engineering, it matters. But Terminal-Bench 2.1 tells a different story. Sonnet 5 scores 80.4%. Opus 4.8 scores 74.6%. That's the first time a mid-tier Sonnet has beaten its flagship sibling on a major coding benchmark, and the margin isn't narrow. For agents that work in the terminal, run shell pipelines, orchest...