Skip to yearly menu bar Skip to main content

Workshop: MATH-AI: The 3rd Workshop on Mathematical Reasoning and AI

Learning Multi-Step Reasoning by Solving Arithmetic Tasks

Tianduo Wang · Wei Lu

Keywords: [ multi-step reasoning ] [ mathematical reasoning ]


The original version of this paper has been accepted at ACL 2023 as a main conference paper. This paper is modified based on the ACL verison to meet the requirement of NeurIPS 2023 Workshop MATH-AI.Mathematical reasoning is regarded as a neces- sary ability for Language Models (LMs). Recent works demonstrate large LMs’ impressive performance in solving math problems. The success is attributed to their Chain-of-Thought (CoT) reasoning abilities, i.e., the ability to decompose complex questions into step-by-step reasoning chains, but such ability seems only to emerge from models with abundant parameters. This work investigates how to incorporate relatively small LMs with the capabilities of multi-step reasoning. We propose to inject such abilities by continually pre-training LMs on a synthetic dataset MsAT which is composed of Multi-step Arithmetic Tasks. Our experiments on four math word problem datasets show the effectiveness of the proposed method in enhancing LMs’ math reasoning abilities.

Chat is not available.