Timezone: »

 
Poster
LIFT: Language-Interfaced Fine-Tuning for Non-language Machine Learning Tasks
Tuan Dinh · Yuchen Zeng · Ruisu Zhang · Ziqian Lin · Michael Gira · Shashank Rajput · Jy-yong Sohn · Dimitris Papailiopoulos · Kangwook Lee

Thu Dec 01 09:00 AM -- 11:00 AM (PST) @ Hall J #204
in Poster Session 5 »

Fine-tuning pretrained language models (LMs) without making any architectural changes has become a norm for learning various language downstream tasks. However, for non-language downstream tasks, a common practice is to employ task-specific designs for input, output layers, and loss functions. For instance, it is possible to fine-tune an LM into an MNIST classifier by replacing the word embedding layer with an image patch embedding layer, the word token output layer with a 10-way output layer, and the word prediction loss with a 10-way classification loss, respectively. A natural question arises: Can LM fine-tuning solve non-language downstream tasks without changing the model architecture or loss function? To answer this, we propose Language-Interfaced Fine-Tuning (LIFT) and study its efficacy and limitations by conducting an extensive empirical study on a suite of non-language classification and regression tasks. LIFT does not make any changes to the model architecture or loss function, and it solely relies on the natural language interface, enabling "no-code machine learning with LMs." We find that LIFT performs comparably well across a wide range of low-dimensional classification and regression tasks, matching the performances of the best baselines in many cases, especially for the classification tasks. We also report experimental results on the fundamental properties of LIFT, including inductive bias, robustness, and sample complexity. We also analyze the effect of pretraining on LIFT and a few properties/techniques specific to LIFT, e.g., context-aware learning via appropriate prompting, calibrated predictions, data generation, and two-stage fine-tuning. Our code is available at https://github.com/UW-Madison-Lee-Lab/LanguageInterfacedFineTuning.

Keywords: Regression · language models · Fine-tuning · non-language tasks · classification. · language interface · language-interfaced learning
[ Paper]  [ Poster]  [ OpenReview] 

Author Information

Tuan Dinh (University of Wisconsin-Madison)

Tuan Dinh is a final year Ph.D. student in Computer Sciences at the University of Wisconsin-Madison. He works on AI/ML area, with some focus on the efficient methods for adapting large pretrained/foundation models, and their applied methods in health/medical sciences/systems.

Yuchen Zeng (University of Wisconsin - Madison)
Ruisu Zhang (UW-Madison)
Ziqian Lin (UW-Madison)

Ph.D. student, Computer Science Department, University of Wisconsin Madison

Michael Gira (University of Wisconsin - Madison)
Shashank Rajput (University of Wisconsin - Madison)
Jy-yong Sohn (University of Wisconsin-Madison)
Dimitris Papailiopoulos (University of Wisconsin-Madison)
Kangwook Lee (UW Madison, Krafton)

More from the Same Authors