Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolation
Paper
•
2108.12409
•
Published
•
5
Papers to read - General
Papers I want to read, at some point.
-
-
YaRN: Efficient Context Window Extension of Large Language Models
Paper • 2309.00071 • Published • 71 -
MIMIC-IT: Multi-Modal In-Context Instruction Tuning
Paper • 2306.05425 • Published • 11 -
Music ControlNet: Multiple Time-varying Controls for Music Generation
Paper • 2311.07069 • Published • 45 -
Memory Augmented Language Models through Mixture of Word Experts
Paper • 2311.10768 • Published • 18 -
Positional Description Matters for Transformers Arithmetic
Paper • 2311.14737 • Published • 2 -
Mamba: Linear-Time Sequence Modeling with Selective State Spaces
Paper • 2312.00752 • Published • 143 -
QuIP: 2-Bit Quantization of Large Language Models With Guarantees
Paper • 2307.13304 • Published • 2