and fun fact: we landed nvfp4 in pretraining under a code name "yolo bundle"
When I first talked to @jackcookjack, he thought that although he was having fun working on nvfp4 pretraining in academia, he was skeptical that it would ever actually work for actual large-scale runs.
As it turns out, he was wrong :^) This blog post explains how it can be done