Why TypeScript for real-time data transformation?
Using TypeScript for transformations allows us to prevent many of the common types of pipeline failures that might otherwise slip through the cracks.
Data engineering
Four software engineering best practices to improve your data pipelines
Software and data engineering are very different, but you can apply the same best practices to both – if you’re smart about the context.
How to prevent your data warehouse from becoming a data swamp
Your data storage system might be a data swamp masquerading as a data warehouse. Here’s how to tell, and what to do.
Why you don’t actually need a reverse ETL platform
If you set up your architecture right, you just need one, robust data pipeline system.
The complete change data capture guide for MySQL
A comprehensive guide to change data capture in MySQL for cross-functional data teams.
The complete change capture guide for PostgreSQL
A comprehensive guide to change data capture in PostgreSQL for cross-functional data teams.
3 reasons to rethink your approach to change data capture
As we navigate this rapidly evolving space, we should be familiar with the challenges of change data capture as it currently exists. But we should not expect them to remain the same for long.
A comprehensive introduction to change data capture (CDC)
Change data capture, or CDC, is the process of recognizing a change in a source data system so that a downstream system can act on that change, usually by updating a target system to reflect new information.
How new pipeline tools are changing data engineering in the 2020s
For the past several years, the workforce has been chronically short on data engineers. But what will happen as vendors build services that automate much of their daily work?
This article won’t tell you how to build a data mesh
Or, how to take an organization-first approach to modern data architecture