LLM Pretraining Data Curation: Deduplication, MinHash LSH, Quality Filters and Data Mixtures (2026)
Pretraining Data Curation: Dedup, Filters and Mixtures Two teams can train the same architecture, with the same optimizer, on the same number of tokens, and end up with models that…