#inference

Cloudflare said on X on March 19 that Kimi K2.5 is now available on Workers AI. The launch pairs a frontier open-source model with platform features aimed at lowering latency and cost for agent workloads.

Cloudflare said on March 20, 2026 that Kimi K2.5 is now available on Workers AI so developers can run agents end-to-end on its platform. The linked Cloudflare blog says the model ships with a 256K context window, multi-turn tool calling, vision, and structured outputs, and that one internal agent workload cut costs by 77% after the switch.

A Hacker News discussion highlighted Flash-MoE, a pure C/Metal inference stack that streams Qwen3.5-397B-A17B from SSD and reaches interactive speeds on a 48GB M3 Max laptop.

Cloudflare said on March 20, 2026 that Kimi K2.5 was available on Workers AI so developers could build end-to-end agents on Cloudflare’s platform. Its launch post says the model brings a 256k context window, multi-turn tool calling, vision inputs, and structured outputs, while an internal security-review agent processing 7B tokens per day cut costs by 77% after the switch.

A March 21, 2026 Hacker News discussion sent tinygrad's tinybox page back up the front page and put a shipping local AI workstation in front of builders looking beyond rented GPU time. The product pitch is notable because it pairs concrete specs with pricing that targets labs and startups trying to run bigger models on premises.

NVIDIA used GTC 2026 to describe how telecom operators are turning distributed network assets into AI grids. The pitch is that inference for low-latency, edge-heavy workloads should move closer to users, devices, and data.

A few weeks after release, r/LocalLLaMA is converging on task-specific sampler and reasoning-budget presets for Qwen3.5 rather than one default setup.

A LocalLLaMA thread on March 18, 2026 pushed fresh attention toward Mamba-3, a new state space model release from researchers at Carnegie Mellon University, Princeton, Cartesia AI, and Together AI. The project shifts its design goal from training speed to inference efficiency and claims prefill+decode latency wins over Mamba-2, Gated DeltaNet, and Llama-3.2-1B at the 1.5B scale.

Meta said on March 11, 2026 that it is developing and deploying four new generations of MTIA custom chips within the next two years. The company is positioning MTIA as a central part of its AI infrastructure strategy for ranking, recommendations, and GenAI inference workloads.

A March 15, 2026 Hacker News post about GreenBoost reached 124 points and 25 comments. The open-source Linux project combines a kernel module and CUDA shim to tier model memory across VRAM, DDR4, and NVMe so larger local LLMs can run without changing inference apps.

NVIDIA said on March 16, 2026 that Dynamo 1.0 is entering production as open source software for generative and agentic inference at scale. The company says the stack can raise Blackwell inference performance by up to 7x and is already supported across major cloud providers, inference platforms, and AI-native companies.

Meta said on March 11, 2026 that it is accelerating its in-house MTIA roadmap across four generations, from MTIA 300 through MTIA 500. The company is using custom silicon to push harder on ranking, recommendation, and especially GenAI inference economics at Meta scale.