Why Thinking Machines Dropped RoPE

This essay examines a specific architectural change. Thinking Machines has decided to drop RoPE. The author argues that this was a positive decision. It explores the implications of removing Rotary Positional Embeddings. The piece analyzes the reasoning behind this shift. It provides a technical perspective on model architecture. The discussion centers on the effectiveness of different methods. It offers a critique of current positional encoding trends.