Applications critical to enterprise profitability such as customer churn prediction, fraudulent account detection, customer lifetime value estimation etc. are typically addressed by training dedicated supervised models using features engineered from tabular data containing customer information. Creating custom feature sets tuned to each applications has the overhead of development, operationalization as well as maintenance over time. Recent advances made in representation learning have the potential to simplify the feature engineering process across various applications. However, it is challenging to apply these methods to tabular data due to issues such as data heterogenity, variations in engagement history across customers and the large size of enterprise data. In this paper, we propose a novel approach to encode tabular data containing customer transactions, purchase history and other interactions into a generic representation of a customer's association with the business and use these embeddings as features to train multiple models spanning a variety of applications. CASPR, Customer Activity Sequence based Prediction and Representation, extends Transformer architecture to encode activity sequences to improve model performance and avoid bespoke feature engineering across applications. Our experiments with running CASPR at scale show it is suitable for both small & large enterprise data.