GLM 5.2 tops Claude Code in Semgrep security benchmark
Original: GLM 5.2 beats Claude in our benchmarks View original →
Semgrep’s latest security benchmark puts Zhipu AI’s GLM 5.2 ahead of Claude Code on IDOR detection. Under the same dataset and prompt-only setup, GLM 5.2 reached 39% F1, while Claude Code scored 32%. Semgrep also estimated the GLM 5.2 run at roughly $0.17 per vulnerability found.
The result is not a claim that open models have solved application security. Semgrep’s own multimodal pipeline still scored higher at 53-61% F1. That gap matters because the pipeline is not just a raw model call; it combines model reasoning with static-analysis signals, rules, and a security-specific workflow.
What makes the post interesting is where the frontier moved. Security bug discovery has been a difficult area for smaller or open-weight models because it needs repository context, control-flow reasoning, and enough restraint to avoid false positives. GLM 5.2 doing well in a prompt-only setting gives teams a reason to test open models for internal code review and triage work, especially where data control and inference cost matter.
The HN discussion quickly shifted from the leaderboard to deployment reality. Some commenters described GLM 5.2 as a useful daily coding model; others asked what hardware can realistically serve a model of this size. That tension is the story: GLM 5.2 did not replace a purpose-built security system, but it did make the open-weight option harder to dismiss.
Related Articles
A LocalLLaMA discussion of SWE-rebench January runs reports close top-tier results, with Claude Code leading pass@1 and pass@5 while open models narrow the gap.
OpenAI and Paradigm launched EVMbench, a benchmark for AI agent performance on smart contract detection, patching, and exploitation tasks. OpenAI reports GPT-5.3-Codex scored 72.2% in exploit mode versus 31.9% for GPT-5.
The 936-point discussion focused less on a routine model refresh than on a three-week release cycle and introductory pricing at half the original 3.6 Flash cost. The published gains span coding, document reasoning, and workflow automation, but production teams still need to test total task cost.