In January 2026, Microsoft announced the general availability of Azure Databricks Agent Bricks—a native capability for creating, deploying, and managing AI agents directly within the Databricks platform. This integration unifies data engineering, machine learning, and agentic AI development in a single environment, enabling data teams to build intelligent agents that have native access to lakehouse […]
Read more →Tag: Data Engineering
Microsoft Acquires Osmos: Agentic Data Engineering Comes to Microsoft Fabric
In January 2026, Microsoft announced the acquisition of Osmos, an agentic AI data engineering platform that automates complex data transformation, integration, and quality tasks. This acquisition signals Microsoft’s commitment to bringing autonomous AI agents into the data engineering workflow within Microsoft Fabric. For data engineers struggling with repetitive ETL development, schema mapping, and data quality […]
Read more →Tips and Tricks – Use dbt for Maintainable Data Transformations
Build modular, tested, documented data transformations with dbt.
Read more →Tips and Tricks – Partition Large Tables for Query Performance
Use table partitioning to dramatically speed up queries on large datasets.
Read more →Tips and Tricks – Use Window Functions for Running Calculations
Calculate running totals, rankings, and moving averages efficiently with SQL window functions.
Read more →Data Quality for AI: Ensuring High-Quality Training Data
Data quality determines AI model performance. After managing data quality for 100+ AI projects, I’ve learned what matters. Here’s the complete guide to ensuring high-quality training data. Figure 1: Data Quality Framework Why Data Quality Matters Data quality directly impacts model performance: Accuracy: Poor data leads to poor predictions Bias: Biased data creates biased models […]
Read more →