The origins, purpose, and practice of data observability

Data Observability (DO) is an emerging category that proposes to help organizations identify, resolve, and prevent data quality issues by continuously monitoring the state of their data over time. This talk is a deep dive into DO, starting from its origins (why it matters), defining the scope and components of DO (what it is), and finally closing with actionable advice for putting observability into practice (how to do it).

We’ll rigorously define data observability to understand why it is different from software observability and existing data quality monitoring. We will derive the four pillars of DO (metrics, metadata, lineage, and logs) then describe how these pillars can be tied to common use cases encountered by teams using popular data architectures, especially on cloud data stacks.

Finally, we’ll close with pointers for how to put observability into practice, drawing from our experience helping teams across sizes, from fast-growing startups to large enterprises, successfully implement DO. Successfully implementing observability throughout an organization involves not only using the right technology, whether that be a commercial solution, an in-house initiative, or an open source project, but implementing the correct processes with the right people responsible for specific jobs. Talk participants can expect to leave with new concepts to understand how DO can help their organizations and ideas for how to implement DO.

Kevin-Hu-Metaplane.jpg

Kevin Hu

Co-founder

metaplane-logo-transparent.svg

What is the cost to attend and watch the virtual sessions?

Data Team Summit is always free and open for all to attend.

When is Data Teams Summit 2024?

Data Teams Summit 2024 was held on January 24, 2024.

What is Data Teams Summit?

Data Teams Summit is an annual peer-to-peer day of empowerment for data teams that reflects our focus on the teams and individuals running, managing, and monitoring data pipelines.

Data Teams Summit is a full-day virtual conference, led by real-world data practitioners and leaders at future-forward organizations about how they're establishing predictability, increasing reliability, and creating economic efficiencies with their data.

Who comes to the Data Teams Summit?

Data professionals and experts including data engineers, administrators, architects, analysts, AI/ML professionals, and relevant data technology leadership.

Join us for sessions on:

  • Data teams and the way they work together
  • Data pipeline, infrastructure & application performance
  • Data quality, management & data governance strategy
  • Methodology that allows teams to move faster and more efficiently
  • DataOps/MLOps/AIOps/FinOps
  • Data modernization & transformation

Interested in speaking at a future Data Teams Summit?

Submit your talk proposal at datateamssummit.com/cfp/ or send us a note to astronaut@solutionmonday.com.

That's a wrap! Thank you to all of those who attended, spoke, and sponsored Data Teams Summit 2024! To stay in the loop about future Data Teams Summits please fill out the form below.

Data Teams Summit Notifications

"*" indicates required fields

Name*
Privacy Policy*
zillow-logo.png
FirstMark-Capital-Logo-Transparent.png
capital-one-logo-transparent.png
Slack-Logo.svg
perpay-logo.png
Babylon-Health-Logo.svg
image001.png
ggv-capital-logo-transparent.png
Baker_Hughes_logo.svg.png
Lightspeed-Venture-Partners-Logo.svg
Squarespace-Logo.svg
menlo-ventures-logo.png
cisco-logo-300x107-1.png
IBM_logo.svg
google-logo.png
Dutchie-Logo.svg
7.png
Akamai-Logo.svg
johnson-johnson-logo.png
easypost-logo-transparent.svg
wistia-logo_color.png
Point-72-Ventures-Transparent-Logo.png
Agridigital-logo-transparent.png
cox.png
mastercard.png
8.png
equifax.png
gartner.png
alphacredit.png
ceridian.png
Canva-Data-Engineer-Logo.png
disney-streaming.svg
Hello_Fresh_Lockup.png
GoCardless-Data.png
oneweb-logo.png
GoCardless-Data.png
carta-logo.svg
Zwift-logo-breed-768x178-142066942.png
swinterton-logo.svg
SB_logo.webp
home-to-go-logo.png
Wattpad-Logo.png
Red_Ventures_logo.svg.png
Magenta-Care-Continuum-Pharmtech-Focus.png
logo.png
CoachArt-Logo.png
octo-logo.png
els-logo_landscape.png
qzonlplfavdnhzkr3vx7.png

Interested in speaking at Data Teams Summit or participating as a sponsor?

Please contact mike@solutionmonday.com.