The evolution of a data platform
Designing a data platform is no easy task, particularly when there are new technologies, techniques, and approaches appearing every week. At Cox Auto UK we have been on a journey from manually deployed Hadoop clusters to a full platform as a service setup using Azure Databricks. This journey hasn’t always been smooth however and we’ve learned some things along the way! In this talk, we will examine how we have made design choices while evolving our platform, our decision to open source some of our work, and what our past, present, and future look like.
James Fielder, Senior Data Engineer @ Cox Automotive