Nemotron Ultra is a open source and already gives DeepSeek R1 performance (admittedly not that good anymore). NVIDIA has open sourced the entire training process, including raw datasets and synthetic data generation. The raw data is mostly curated web crawl data.

Already happening for years.

Amazon mechanical turk, Outlier.ai, scale ai, are all gig platforms that pay people to produce real human content for LLM training.

At first they were paying people to actually produce content, like worked solutions to maths problems or translations, then they pivoted more to rating LLM output.

And as with every enshittification cycle, wages and work rapidly dried up, so people responded by asking LLMs for the answer just to meet deadlines and get by.