AWS and Cerebras plan a disaggregated inference stack for Amazon Bedrock

What happened

On March 13, 2026, AWS and Cerebras announced a collaboration aimed at delivering much faster AI inference through Amazon Bedrock. The companies said the new service will be deployed in AWS data centers and launched in the coming months.

The centerpiece is a disaggregated inference design. Instead of treating generation as one monolithic workload, the system splits inference into prefill and decode. Prefill handles prompt processing and benefits from massively parallel compute, while decode generates output tokens sequentially and is far more dependent on memory bandwidth and low-latency data movement.

Key details

• AWS said Trainium-powered servers will handle prefill while Cerebras CS-3 systems handle decode, with the two stages connected by Elastic Fabric Adapter networking.
• The company described the result as potentially an order of magnitude faster than current alternatives for demanding generative AI workloads.
• AWS also said it will later offer leading open-source LLMs and Amazon Nova on Cerebras hardware.
• The companies framed AWS as the first cloud provider for Cerebras's disaggregated inference architecture, delivered exclusively through Amazon Bedrock.

The use case is straightforward: real-time coding assistants, interactive applications, and agent systems all suffer when token generation is slow. AWS argues that assigning different silicon to the two stages lets each processor focus on what it does best, increasing throughput without forcing every request through a single general-purpose path. The company also said the stack will inherit AWS Nitro-based security, isolation, and operational consistency.

Why it matters

This announcement matters because the AI infrastructure race is increasingly about inference economics, not just training scale. Reasoning-heavy models and agent workflows generate more tokens and spend more time in decode, so small latency gains can directly affect usability and cost at production scale.

For Insights readers, the larger signal is architectural. AWS and Cerebras are betting that the next competitive layer in AI cloud infrastructure is workload specialization: routing different parts of inference to different chips and connecting them with low-latency networking. If that approach works, platform differentiation will come not only from model access, but from how efficiently providers build the path from prompt to token.