Driving DataOps Culture with LinkedIn DataHub
Your data is not changing slowly, so why should your metadata?
LinkedIn DataHub was open-sourced to enable other organizations to harness the power of metadata and unleash excellent DataOps practices. Doing DataOps well requires bringing together multiple disciplines of data science, data analytics, and data engineering into a cohesive unit. However, this is complicated, because there are a wide variety of data tools that are in use by these different tribes. Shirshanka, who founded and architected DataHub at LinkedIn, will describe its journey in enabling DataOps use-cases on top of the metadata platform. He will also showcase the latest integrations and features in the tool and share the roadmap for the project.
Shirshanka Das, Co-Founder & CTO @ Acryl Data
My mission is to make engineers productive with data, ethically. I am a technical lead in the Data team at LinkedIn. I like solving large scale challenges in distributed data systems. I've built several data infrastructure projects at LinkedIn, some of which are open source: Apache Helix, Espresso and Databus.
I'm involved in the following projects aimed at simplifying the big data management space: Apache Gobblin (incubating), LinkedIn DataHub, Apache Pinot (incubating) and Dali.