We’ve been cooking
Looped LLMs are heating up🔥. Open weights are becoming available, but the recipes to train these models remain hard to find.
Introducing ARLoop, a family of strong, fully-open pretrained looped transformers with Attention Residuals! Our strongest model, ARLoop-1.2B, outperforms