Forge Framework Boosts 8B LLM from 53% to 99% on Agentic Tasks with Structured Guardrails

Small Models, Big Reliability Gains

Forge is an open-source Python framework that dramatically improves the reliability of self-hosted language models for agentic workflows. By applying structured guardrails rather than scaling to larger models, Forge demonstrates that small 8B models can punch far above their weight on tool-calling and multi-step agent tasks.

The Four Guardrail Mechanisms

Forge's reliability gains come from four lightweight components:

• Rescue Parsing: Catches and corrects malformed tool calls before they fail the agent loop.
• Retry Nudges: Guides the model toward correct outputs on retries with targeted prompts.
• Step Enforcement: Ensures required workflow steps execute in the correct order.
• Context Management: VRAM-aware tiered compaction keeps context within budget without losing critical information.

Benchmark Results

The top self-hosted configuration (Ministral-3 8B Q8 on llama-server) scores 86.5% across Forge's 26-scenario eval suite, and 76% on the hardest reasoning tier. On standard agentic tasks, the framework lifts accuracy from 53% to 99%.

Three Usage Modes

Forge can be used as a WorkflowRunner (full agentic loop), Guardrails middleware (composable with existing orchestration), or an OpenAI-compatible proxy server. It supports Ollama, llama-server, Llamafile, and Anthropic backends, requiring Python 3.12+.