Rob and Cosmin go over "Full Bandwidth Transformer"

About this event

Full Bandwidth Transformer This Tuesday we're going over "Full Bandwidth Transformer" from MSR https://arxiv.org/abs/2608.08888 Resources twitter thread by author John Langford • https://x.com/JohnCLangford/status/2095849181919261168 he speculates it might be relevant to the new rumored astra arch from openai Other relevant papers Same MSR group, rnn/latent prediction ideas: • NextLat (2025) https://arxiv.org/abs/2511.05963 - an author presented it at Yacine's pod recently https://www.youtube.com/watch?v=coyYxJ26hlQ • followup - hierarchical NextLat https://arxiv.org/abs/2608.05806 • Almost Free State Prediction Separation https://arxiv.org/abs/2609.03807 Google: • recurrence and strong benchmark wins by Recirculation paper (got 15% win on gsm8k) https://arxiv.org/abs/2608.17981 MIT: • Pretraining Recurrent Nets Without Recurrence https://arxiv.org/abs/2606.06479