Symbolic regression is the process of identifying mathematical expressions that fit observed output from a black-box process. It is a discrete optimization problem generally believed to be NP-hard. Prior approaches to solving the problem include neural-guided search (e.g. using reinforcement learning) and genetic programming. In this work, we introduce a hybrid neural-guided/genetic programming approach to symbolic regression and other combinatorial optimization problems. We propose a neural-guided component used to seed the starting population of a random restart genetic programming component, gradually learning better starting populations. On a number of common benchmark tasks to recover underlying expressions from a dataset, our method recovers 65% more expressions than a recently published top-performing model using the same experimental setup. We demonstrate that running many genetic programming generations without interdependence on the neural-guided component performs better for symbolic regression than alternative formulations where the two are more strongly coupled. Finally, we introduce a new set of 22 symbolic regression benchmark problems with increased difficulty over existing benchmarks. Source code is provided at www.github.com/brendenpetersen/deep-symbolic-optimization.
Terrell Mundhenk (Lawrence Livermore National Lab)
Mikel Landajuela (Lawrence Livermore National Labs)
Machine Learning Researcher at Lawrence Livermore National Laboratory (Computational Engineering Directorate), holding a Ph.D. from Université Pierre et Marie Curie and Inria.
Ruben Glatt (Lawrence Livermore National Laboratory)
With a background in Mechatronics and Mechanical Engineering, Ruben has turned to Artificial Intelligence where his main interest lies in Machine Learning (ML) research with a focus on Reinforcement Learning (RL), autonomous systems, and applications in energy efficiency. He received his Ph.D. in Computer Engineering in the area of ML at the University of Sao Paulo (USP), Brazil, holds a master degree in Mechanical Engineering in the area of controlling mechanical systems from the Universidade Estadual Paulista Julio de Mesquita Filho (UNESP), Brazil, and a Diplom-Ingenieur degree in Mechatronics in the area of sensors and robotics from the Karlsruhe Institute of Technology (KIT), Germany. Ruben has acquired years of professional experiences before and during his studies while working in the technology and energy sector, as well as in the organization of international ML conferences. After converting from a postdoctoral position at the Lawrence Livermore National Laboratory, USA, he is now working as a Machine Learning Researcher on a variety of RL projects to develop methods for collaborative autonomy in multi- agent systems, interpretable RL, and real-world applications. Ruben represented the postdocs at the Lab as Chair of the Lawrence Livermore Postdoc Association and member of the Institutional Postdoc Program Board. He also engages in community efforts and is currently the Vice-Chair of the IEEE Computer Society Oak land/Eastbay/San Francisco chapter and a voting member on the IEEE Computer Society Artificial Intelligence Standards Committee (C/AISC). Ruben’s long term research interest lies in successfully applying RL techniques to real-world challenges to accelerate and improve decision-making, autonomously or as a support tool for humans, preferably for applications in energy efficiency and smart mobility systems.
Claudio P Santiago (Lawrence Livermore National Laboratory)
Daniel faissol (Lawrence Livermore National Labs)
Brenden K Petersen (Lawrence Livermore National Laboratory)
More from the Same Authors
2022 Poster: A Unified Framework for Deep Symbolic Regression »
Mikel Landajuela · Chak Shing Lee · Jiachen Yang · Ruben Glatt · Claudio P Santiago · Ignacio Aravena · Terrell Mundhenk · Garrett Mulcahy · Brenden K Petersen
2019 : Lunch + Poster Session »
Frederik Gerzer · Bill Yang Cai · Pieter-Jan Hoedt · Kelly Kochanski · Soo Kyung Kim · Yunsung Lee · Sunghyun Park · Sharon Zhou · Martin Gauch · Jonathan Wilson · Joyjit Chatterjee · Shamindra Shrotriya · Dimitri Papadimitriou · Christian Schön · Valentina Zantedeschi · Gabriella Baasch · Willem Waegeman · Gautier Cosne · Dara Farrell · Brendan Lucier · Letif Mones · Caleb Robinson · Tafara Chitsiga · Victor Kristof · Hari Prasanna Das · Yimeng Min · Alexandra Puchko · Alexandra Luccioni · Kyle Story · Jason Hickey · Yue Hu · Björn Lütjens · Zhecheng Wang · Renzhi Jing · Genevieve Flaspohler · Jingfan Wang · Saumya Sinha · Qinghu Tang · Armi Tiihonen · Ruben Glatt · Muge Komurcu · Jan Drgona · Juan Gomez-Romero · Ashish Kapoor · Dylan J Fitzpatrick · Alireza Rezvanifar · Adrian Albert · Olya (Olga) Irzak · Kara Lamb · Ankur Mahesh · Kiwan Maeng · Frederik Kratzert · Sorelle Friedler · Niccolo Dalmasso · Alex Robson · Lindiwe Malobola · Lucas Maystre · Yu-wen Lin · Surya Karthik Mukkavili · Brian Hutchinson · Alexandre Lacoste · Yanbing Wang · Zhengcheng Wang · Yinda Zhang · Victoria Preston · Jacob Pettit · Draguna Vrabie · Miguel Molina-Solana · Tonio Buonassisi · Andrew Annex · Tunai P Marques · Catalin Voss · Johannes Rausch · Max Evans