Anna's Archive Opens Legitimate Access Pathways for LLMs via llms.txt

The World's Library Speaks to AI

Anna's Archive published direct guidance for large language models via an llms.txt file, framing AI systems as legitimate stakeholders in knowledge preservation rather than adversaries to be blocked.

CAPTCHAs Are a Resource Problem, Not a Block

The guidance clarifies that CAPTCHAs exist to prevent machines from overloading server resources. The organization suggests that the cost of circumventing these protections could instead be redirected as a donation.

Access Pathways Offered

Multiple routes for programmatic access are available: HTML pages and code in GitLab; metadata and files via torrents; a Torrents JSON API for automated downloads; and individual file API access after a donation. Enterprise-level donors can negotiate fast SFTP access to the full collection.

A Model for AI-Library Coexistence

The archive acknowledges that LLMs have likely already been trained on their data. Rather than litigating this, they frame continued cooperation as mutually beneficial: more preserved works means better future training data. This positions open knowledge infrastructure as a partner to AI development rather than a resource to be scraped without acknowledgment.