LLM Reddit Apr 19, 2026 2 min read
LocalLLaMA cared about this eval post because it mixed leaderboard data with lived coding-agent pain: Opus 4.7 scored well, but the author says it felt worse in real use.
#benchmarks
RSS FeedLLM Apr 19, 2026 1 min read
A new arXiv preprint reports that LLM judges became meaningfully more lenient when prompts framed evaluation consequences, exposing a weak point in automated safety and quality benchmarks.
LLM Reddit Apr 19, 2026 1 min read
r/LocalLLaMA cared because the numbers were concrete: 79 t/s on an RTX 5070 Ti with 128K context, tied to one llama.cpp flag choice.
LLM Apr 18, 2026 2 min read
MM-WebAgent tackles a real flaw in AI-made webpages: models can generate pieces, but the page often loses visual coherence. The paper adds hierarchical planning, self-reflection, a benchmark, and released code/data so builders can test multimodal webpage agents beyond code-only output.
LLM Reddit Apr 18, 2026 2 min read
The r/singularity thread did not just react to Opus 4.7 scoring 41.0% where Opus 4.6 scored 94.7%. The interesting part was the community trying to separate real capability loss from refusal behavior, routing, and benchmark interpretation.
LLM Reddit Apr 18, 2026 1 min read
The LocalLLaMA thread cared less about a release headline and more about which Qwen3.6 GGUF quant actually works. Unsloth’s benchmark post pushed the discussion into KLD, disk size, CUDA 13.2 failures, and the messy details that decide local inference quality.
LLM Apr 17, 2026 2 min read
A new arXiv paper shows why low average violation rates can make LLM judges look safer than they are. On SummEval, 33-67% of documents showed at least one directed 3-cycle, and prediction-set width tracked absolute error strongly.
Sciences Reddit Apr 17, 2026 1 min read
r/MachineLearning reacted because the sample was small but painfully familiar: one user said 4 of 7 paper claims they checked this year did not reproduce, with 2 still sitting as unresolved GitHub issues. The comments moved from resignation about reviewers not running code to concrete demands for submission-time reproducibility reports.
LLM Apr 17, 2026 2 min read
HWE-Bench moves LLM agent evaluation from isolated HDL tasks to repository-scale hardware repairs. The best agent solved 70.7% overall, but performance fell below 65% on complex SoC-level projects.
LLM Apr 17, 2026 2 min read
A new arXiv paper puts a hierarchical agent system at the top of MLE-Bench with a 63.1% medal rate. The result matters because the agent handles design, coding, debugging, training, and tuning from a task description plus data.
AI X/Twitter Apr 17, 2026 2 min read
Why it matters: Anthropic is pushing Opus toward longer autonomous coding work without raising the premium model price. The linked launch page says Opus 4.7 reaches 70% on CursorBench versus 58% for Opus 4.6, while API pricing stays at $5 per million input tokens and $25 per million output tokens.
LLM Apr 17, 2026 2 min read
IBM Research’s VAKRA moves agent evaluation from static Q&A into executable tool environments. With 8,000+ locally hosted APIs across 62 domains and 3-7 step reasoning chains, the benchmark finds a gap between surface tool use and reliable enterprise agents.