Tal Sharon (Software Architect), Aviel Buskila (DevOps Engineer) and Max Peres (Data Engineer) @ Nielsen:
At the Nielsen Marketing Cloud, we used to manage our data pipelines via AWS Data Pipeline. Over the years, we’ve encountered several issues with this tool, and a year ago we decided to embark on a journey to replace it with a tool more suitable for our needs.
In this session, we’ll discuss how we actually migrated to Airflow, what challenges we faced and how we mitigated them (and even contributed to the open-source project along the way). We’ll also provide some helpful tips for Airflow users