What Matters in Transformers? Not All Attention is Needed
Paper
•
2406.15786
•
Published
•
32
about Transformer
-
-
Breaking the Memory Barrier: Near Infinite Batch Size Scaling for Contrastive Loss
Paper • 2410.17243 • Published • 95 -
Forgetting Transformer: Softmax Attention with a Forget Gate
Paper • 2503.02130 • Published • 32 -
Transformers without Normalization
Paper • 2503.10622 • Published • 164 -
SageAttention2++: A More Efficient Implementation of SageAttention2
Paper • 2505.21136 • Published • 40