firstbacksecondback
69 Results
|
Workshop
|
Sat 13:54 |
One Initialization to Rule them All: Fine-tuning via Explained Variance Adaptation Fabian Paischer · Lukas Hauzenberger · Thomas Schmied · Benedikt Alkin · Marc Deisenroth · Sepp Hochreiter |
|
|
Workshop
|
SuperPos-Prompt: Enhancing Soft Prompt Tuning of Language Models with Superposition of Multi Token Embeddings Mohammad Ali Sadraei Javaheri · Ehsaneddin Asgari · Alice McHardy · Hamid Rabiee |
||
|
Workshop
|
Approximations may be all you need: Towards Pre-training LLMs with Low-Rank Decomposition and Optimizers Namrata Shivagunde · Mayank Kulkarni · Giannis Karamanolakis · Jack FitzGerald · Yannick Versley · Saleh Soltan · Volkan Cevher · Jianhua Lu · Anna Rumshisky |
||
|
Poster
|
Wed 11:00 |
LSH-MoE: Communication-efficient MoE Training via Locality-Sensitive Hashing Xiaonan Nie · Liu Qibin · Fangcheng Fu · Shenhan Zhu · Xupeng Miao · Xiaoyang Li · Yang Zhang · Shouda Liu · Bin CUI |
|
|
Poster
|
Thu 11:00 |
MOTE-NAS: Multi-Objective Training-based Estimate for Efficient Neural Architecture Search Yuming Zhang · Jun Hsieh · Xin Li · Ming-Ching Chang · Chun-Chieh Lee · Kuo-Chin Fan |
|
|
Poster
|
Fri 16:30 |
JiuZhang3.0: Efficiently Improving Mathematical Reasoning by Training Small Data Synthesis Models Kun Zhou · Beichen Zhang · jiapeng wang · Zhipeng Chen · Xin Zhao · Jing Sha · Zhichao Sheng · Shijin Wang · Ji-Rong Wen |
|
|
Workshop
|
Different Rates for Different Weights: Decoupled Relative Learning Rate Schedules Jan Ludziejewski · Jan Małaśnicki · Maciej Pióro · Michał Krutul · Kamil Ciebiera · Jakub Krajewski · Marek Cygan · Kamil Adamczewski · Sebastian Jaszczur |
||
|
Poster
|
Thu 16:30 |
Rethinking Memory and Communication Costs for Efficient Data Parallel Training of Large Language Models Hanxiao Zhang · Lin JU · Chan Wu · Jinjing Huang · Youshao Xiao · Zhenglei Zhou · Zhiming fan · Zhaoxin Huan · Siyuan Li · Fanzhuang Meng · Lei Liang · Xiaolu Zhang · Jun Zhou |
|
|
Workshop
|
Sat 11:54 |
OLMoE: Open Mixture-of-Experts Language Models Niklas Muennighoff · Luca Soldaini · Dirk Groeneveld · Kyle Lo · Jacob Morrison · Sewon Min · Weijia Shi · Evan Walsh · Oyvind Tafjord · Nathan Lambert · Yuling Gu · Shane Arora · Akshita Bhagia · Dustin Schwenk · David Wadden · Alexander Wettig · Binyuan Hui · Tim Dettmers · Douwe Kiela · Noah Smith · Pang Wei Koh · Amanpreet Singh · Hannaneh Hajishirzi |