A Dive into the AI Data Quality Revolution | Abe Gong, Great Expectations

Data quality issues can arise at various stages of the data pipeline, from data ingestion to model deployment. Common issues include null values, schema drift, and incorrect calculations. These seemingly small issues can have a significant impact on the accuracy and reliability of the data, leading to broken dashboards and loss of trust in the data system. In this episode, host Darius Gant interviews Abe Gong, the founder and CEO of Great Expectations, a leading data quality tool. Abe shares his insights into the world of data quality and how Great Expectations is solving the systemic problem of data quality in organizations. He explains the importance of building a robust testing system for data, similar to what software engineers do, in order to ensure accurate and reliable data. Abe discusses common data quality issues and how Great Expectations helps teams identify and fix these issues. He also explores the intersection of data quality and AI, highlighting the role of GX in ensuring the accuracy and trustworthiness of AI models. Throughout the conversation, Abe emphasizes the need for collaboration and communication in data teams to build trust and achieve data-driven success. If your company is looking to scale its AI initiatives, head over to Tesoro AI (www.tesoroai.com). We are experts in AI strategy, staff augmentation, and AI product development.  Founder Bio: Abe Gong is a founder and CEO at Great Expectations, the world’s leading open source tool for data quality. Prior to working on Great Expectations, Abe was Chief Data Officer at Aspire Health, the founding member of the Jawbone data science team, and lead data scientist at Massive Health. Abe has been leading teams using data and technology to solve problems in health, tech, and public policy for over a decade. He speaks and writes regularly on data, AI and entrepreneurship.   Time Stamps: 01:58 Abe Gong’s background and experience in data science 03:45 The pain point in the market that led to the creation of great expectations 05:00 Common errors and issues in data quality 06:47 Identifying and solving data quality issues 09:43 How great expectations support companies deploying AI models 12:45 Great Expectations involvement in generative AI use cases 16:34 Understanding the sensibilities and workflows of data developers 19:42 Building a remote-first team with a focus on open-source collaboration 22:11 Tips for running a remote team efficiently and effectively 24:41 Hiring independent and action-oriented individuals for remote work 27:24 Raising founds journey for Great Expectations. 30:08 Importance of technical leads on data teams 32:52 Difference between enterprise software sales and open source models 34:06 What is coming up for Great Expectations in the 2024   Resources Company website: https://greatexpectations.io/Twitter: https://twitter.com/expectgreatdata LinkedIn: https://www.linkedin.com/company/greatexpectations-data/