Posted on:
August 7th, 2025
Tips and Tricks #156: Use dbt for Maintainable Data Transformations
Build modular, tested, documented data transformations with dbt.
Tag: Data Engineering
All
All
General
Microsoft
Emerging Technologies
Tips and Tricks
KnowledgeBase
Community
OpenSource
HotFixes
Software/System Design
Posted on:
August 5th, 2025
Tips and Tricks #155: Partition Large Tables for Query Performance
Use table partitioning to dramatically speed up queries on large datasets.
Posted on:
August 3rd, 2025
Tips and Tricks #154: Implement Idempotent ETL with Merge Statements
Use MERGE (upsert) for safe, rerunnable data pipelines that handle duplicates gracefully.
Posted on:
August 1st, 2025
Tips and Tricks #153: Use Window Functions for Running Calculations
Calculate running totals, rankings, and moving averages efficiently with SQL window functions.
Posted on:
June 22nd, 2025
Building the Modern Data Stack: How Spark, Kafka, and dbt Transformed Data Engineering
The data engineering landscape has undergone a fundamental transformation over […]
- The Paradigm Shift: From Monolithic to Modular
- Apache Kafka: The Nervous System of Real-Time Data
- Apache Spark: The Processing Powerhouse
- dbt: Transforming How We Think About Transformations
- The Medallion Architecture: Bronze, Silver, Gold
- Delta Lake: ACID Transactions for the Lakehouse
- Orchestration and Monitoring
- Putting It All Together
Posted on:
June 4th, 2025
Tips and Tricks #124: Use dbt for Maintainable Data Transformations
Build modular, tested, documented data transformations with dbt.