https://mradassaad.github.io/I write about recent things I learned and found interesting. I obtain a Ph.D. in eco-hydrology and environmental science from Duke University in 2020. I now work as an ML Engineer at Capital One Shopping. 2026-05-04T00:21:14-04:00 Assaad Mrad https://mradassaad.github.io/ Jekyll © 2026 Assaad Mrad /assets/img/favicons/favicon.ico /assets/img/favicons/favicon-96x96.png 2026-05-03T00:00:00-04:00 2026-05-04T00:11:39-04:00 https://mradassaad.github.io/posts/why-ssms-struggle-in-parameter-golf/ Assaad Mrad

TL;DR Over ~3 weeks of experimentation on an SSM-based submission to OpenAI’s Parameter Golf, I converged on a legal Mamba-3 hybrid at post-quant+TTT 1.1456 bpb, the best SSM submission in the 16MB track. Despite this, a persistent gap to the transformer SOTA remained. The key contribution of this writeup is not the submission itself but two structural handicaps that empirically cap SSMs in th...