-
https://arxiv.org/abs/2510.15103#facebook: “Continual Learning via Sparse Memory Finetuning”, Jessy Lin, Luke Zettlemoyer, Gargi Ghosh, Wen-Tau Yih, Aram Markosyan, Vincent-Pierre Berges, Barlas Oğuz -
https://arxiv.org/abs/2506.06105: “Text-To-LoRA (T2L): Instant Transformer Adaption”, Rujikorn Charakorn, Edoardo Cetin, Yujin Tang, Robert Tjarko Lange -
https://arxiv.org/abs/2505.11711: “Reinforcement Learning Finetunes Small Subnetworks in Large Language Models”, Sagnik Mukherjee, Lifan Yuan, Dilek Hakkani-Tur, Hao Peng -
https://arxiv.org/abs/2403.17844: “Mechanistic Design and Scaling of Hybrid Architectures”, Michael Poli, Armin W. Thomas, Eric Nguyen, Pragaash Ponnusamy, Björn Deiseroth, Kristian Kersting, Taiji Suzuki, Brian Hie, Stefano Ermon, Christopher Ré, Ce Zhang, Stefano Massaroli -
https://arxiv.org/abs/2311.10770: “Exponentially Faster Language Modeling”, Peter Belcak, Roger Wattenhofer -
https://www.sciencedirect.com/science/article/pii/S0893608023005051: “An Exact Mapping from ReLU Networks to Spiking Neural Networks”, Ana Stanojevic, Stanisław Woźniak, Guillaume Bellec, Giovanni Cherubini, Angeliki Pantazi, Wulfram Gerstner -
https://arxiv.org/abs/2310.17157: “Deja Vu: Contextual Sparsity for Efficient LLMs at Inference Time”, Zichang Liu, Jue Wang, Tri Dao, Tianyi Zhou, Binhang Yuan, Zhao Song, Anshumali Shrivastava, Ce Zhang, Yuandong Tian, Christopher Re, Beidi Chen -
https://arxiv.org/abs/2308.14711: “Fast Feedforward Networks”, Peter Belcak, Roger Wattenhofer -
https://arxiv.org/abs/2302.12441: “MUX-PLMs: Pre-Training Language Models With Data Multiplexing”, Vishvak Murahari, Ameet Deshpande, Carlos E. Jimenez, Izhak Shafran, Mingqiu Wang, Yuan Cao, Karthik Rajagopal Narasimhan -
https://arxiv.org/abs/2210.06313#google: “The Lazy Neuron Phenomenon: On Emergence of Activation Sparsity in Transformers”, Zonglin Li, Chong You, Srinadh Bhojanapalli, Daliang Li, Ankit Singh Rawat, Sashank J. Reddi, Ke Ye, Felix Chern, Felix Yu, Ruiqi Guo, Sanjiv Kumar -
https://arxiv.org/abs/2207.03620: “More ConvNets in the 2020s: Scaling up Kernels Beyond 51×51 Using Sparsity (SLaK)”, Shiwei Liu, Tianlong Chen, Xiaohan Chen, Xuxi Chen, Qiao Xiao, Boqian Wu, Mykola Pechenizkiy, Decebal Mocanu, Zhangyang Wang -
https://arxiv.org/abs/2205.03983#google: “Building Machine Translation Systems for the Next Thousand Languages”, Ankur Bapna, Isaac Caswell, Julia Kreutzer, Orhan Firat, Daan van Esch, Aditya Siddhant, Mengmeng Niu, Pallavi Baljekar, Xavier Garcia, Wolfgang Macherey, Theresa Breiner, Vera Axelrod, Jason Riesa, Yuan Cao, Mia Xu Chen, Klaus Macherey, Maxim Krikun, Pidong Wang, Alexander Gutkin, Apurva Shah, Yanping Huang, Zhifeng Chen, Yonghui Wu, Macduff Hughes -
https://arxiv.org/abs/2204.00595: “Monarch: Expressive Structured Matrices for Efficient and Accurate Training”, Tri Dao, Beidi Chen, Nimit Sohoni, Arjun Desai, Michael Poli, Jessica Grogan, Alexander Liu, Aniruddh Rao, Atri Rudra, Christopher Ré -
https://arxiv.org/abs/2203.06850: “Efficient Language Modeling With Sparse All-MLP”, Ping Yu, Mikel Artetxe, Myle Ott, Sam Shleifer, Hongyu Gong, Ves Stoyanov, Xian Li -
https://arxiv.org/abs/2202.07415#deepmind: “NeuPL: Neural Population Learning”, Siqi Liu, Luke Marris, Daniel Hennes, Josh Merel, Nicolas Heess, Thore Graepel -
https://arxiv.org/abs/2106.09685#microsoft: “LoRA: Low-Rank Adaptation of Large Language Models”, Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen -
https://arxiv.org/abs/1905.11946#google: “EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks”, Mingxing Tan, Quoc V. Le -
https://arxiv.org/abs/1803.10615: “SqueezeNext: Hardware-Aware Neural Network Design”, Amir Gholami, Kiseok Kwon, Bichen Wu, Zizheng Tai, Xiangyu Yue, Peter Jin, Sicheng Zhao, Kurt Keutzer -
https://arxiv.org/abs/1512.03385#microsoft: “Deep Residual Learning for Image Recognition”, Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun
‘NN sparsity’ directory
by Gwern Branwen· December 30, 2025· 2 min read