Machine Learning for Targeted Display Advertising: Transfer Learning in Action

This paper presents the design of a fully deployed multistage transfer learning system for targeted display advertising, highlighting the important role of problem formulation and the sampling of data from distributions different from that of the target environment.  Notably, the machine learning system itself is deployed and has been in continual use for years for thousands of advertising campaigns—in contrast to the more common case where predictive models are built outside the system, curated, and then deployed. In this domain, acquiring sufficient data for training from the ideal sampling distribution is prohibitively expensive.  Instead, data are drawn from surrogate distributions and learning tasks, and then transferred to the target task.  We present the design of the transfer learning system.  We then present a detailed experimental evaluation, showing that the different transfer stages indeed each add value.  We also present production results across a variety of advertising clients from a variety of industries, illustrating the performance of the system in use.  We close the paper with a collection of lessons learned from over half a decade of research and development on this complex, deployed, and intensely used machine learning system.