Resource-Limited ML Researcher Rejected by Top Conference for Lacking Large-Scale Comparisons

A Researcher's Frustration Goes Viral

A post on r/MachineLearning garnered over 1,700 upvotes and sparked a wide-ranging discussion about the state of ML research. An independent researcher shared their experience developing a novel method to improve multimodal learning with limited resources.

The Research and Its Fate

The researcher ran experiments with a 500M parameter model, demonstrating that their method outperformed comparable contemporary methods at that scale. Unable to scale vertically — no access to larger models or bigger training runs — they scaled horizontally instead, producing a thorough analysis with multiple evaluations and insights others could reproduce at larger scales.

The paper was submitted to CVPR and received scores of 5/3/3 — above average. It was ultimately rejected, primarily for lack of comparisons against large-scale models.

The Structural Problem with Modern ML Research

Community response was overwhelming agreement. Many researchers argued that modern ML research has effectively become an engineering competition. Papers that tweak one component of a model pipeline, get marginally better results on benchmarks, and package it as novel work dominate top conferences.

Meanwhile, work that produces genuine insights but lacks the compute budget to run comparisons at scale gets filtered out — even when reviewers acknowledge the value of the contribution.

Democratization vs. Compute Arms Race

For AI research to truly democratize, evaluation frameworks need to value the novelty of ideas and the depth of insights over the scale of computational resources. As long as reviewers use "comparison to billion-parameter models" as a gating criterion, independent researchers and smaller institutions will continue to be systematically disadvantaged regardless of the quality of their ideas.