#llm-training

LLM Reddit 2d ago 2 min read

LocalLLaMA Highlights a 356K-Row Human Code Review Dataset for Training Coding Models

A LocalLLaMA post pointed to a new Hugging Face dataset of human-written code reviews, pairing before-and-after code changes with inline reviewer comments and negative examples across 37 languages.

#code-review #datasets #github

LLM sources.twitter 3d ago 2 min read

Karpathy open-sources autoresearch for autonomous single-GPU nanochat experiments

Andrej Karpathy has published autoresearch, a minimal repo that lets AI agents iterate on a stripped-down nanochat training loop overnight. The project turns agent evaluation into a closed-loop research workflow with fixed 5-minute runs, Git branches, and validation-loss-based selection.

#karpathy #agents #open-source

LLM Hacker News Mar 5, 2026 2 min read

NanoGPT Slowrun community debate highlights data-efficient LLM training

A March 4, 2026 Hacker News thread elevated Q Labs’ Slowrun benchmark, which fixes training data at 100M FineWeb tokens and optimizes for data efficiency under large compute budgets.

#nanogpt #data-efficiency #llm-training

LLM Reddit Feb 21, 2026 2 min read

Reddit Discusses arXiv 2602.15322: Masked Adaptive Updates (Magma) for LLM Pretraining

A high-engagement r/singularity post pointed to arXiv 2602.15322, which reports that masked adaptive updates and the proposed Magma optimizer can improve 1B-model perplexity versus Adam and Muon with minimal overhead.

#llm-training #optimizers #rmsprop