Four software engineering best practices to improve your data pipelines
Software and data engineering are very different, but you can apply the same best practices to both – if you’re smart about the context.
Data engineering
Why you don’t actually need a reverse ETL platform
If you set up your architecture right, you just need one, robust data pipeline system.
The complete change data capture guide for MySQL
A comprehensive guide to change data capture in MySQL for cross-functional data teams.
The complete change capture guide for PostgreSQL
A comprehensive guide to change data capture in PostgreSQL for cross-functional data teams.
3 reasons to rethink your approach to change data capture
As we navigate this rapidly evolving space, we should be familiar with the challenges of change data capture as it currently exists. But we should not expect them to remain the same for long.
A comprehensive introduction to change data capture (CDC)
Change data capture, or CDC, is the process of recognizing a change in a source data system so that a downstream system can act on that change, usually by updating a target system to reflect new information.
How new pipeline tools are changing data engineering in the 2020s
For the past several years, the workforce has been chronically short on data engineers. But what will happen as vendors build services that automate much of their daily work?
This article won’t tell you how to build a data mesh
Or, how to take an organization-first approach to modern data architecture
The power and implications of data materialization
The simple concept of materialization can tell us a lot about how (and why) data is stored and represented. In this post: logs and tables; data loading and queries; exports, and more.
Connector stories: Snowflake and BigQuery
Data warehouses like Snowflake and BigQuery are essential to your data stack. Today we’re talking with the two software engineers who built Estuary’s Snowflake and Bigquery connectors to get an inside look at their process.