With the advancement of modern technologies, programming becomes ubiquitous not only among professional software developers, but also for general computer users. However, gaining programming expertise is time-consuming and challenging. Therefore, program synthesis has many applications, where the computer automatically synthesizes programs from user-written descriptions. In this talk, I will discuss my research on neural program synthesis from semi-structured context, where the synthesized program is executed on structured input for data processing and analysis. In particular, I will present my work SpreadsheetCoder for spreadsheet formula prediction, which was integrated into Google Sheets. Our work demonstrates that modeling the tabular structure and learning from multi-modal input is important for inferring user intents, especially when the program specifications are implicit and ambiguous.