Delta tables are the backbone of data storage in Microsoft Fabric. If you’re working with lakehouses, understanding Delta is essential. In this post, I’m going to explain what Delta tables are, why we need them, and how they work in Microsoft Fabric. This is part of a larger series where I’ll cover everything you need …
Today I’m going to show you how you can use Python notebooks inside Microsoft Fabric together with DuckDB to speed up your query process on delta tables—and maybe save you some money because you won’t need to start a Spark cluster anytime you want to do some simple querying. I’ve never worked with DuckDB before …
Microsoft recently made a change that caught a lot of people off guard: they disabled V-Ordering by default for new Fabric lakehouses. If you’re using Power BI Direct Lake mode, this matters—a lot. V-Ordering can dramatically improve your query performance, but it comes with tradeoffs. Today I’m going to show you how to check if …
Lakehouses are the foundation of data engineering in Microsoft Fabric. They’re where you store your raw data, your transformed tables, and everything in between. But if you’re new to Fabric, lakehouses can be a bit confusing. What’s the difference between tables and files? What are shortcuts? How do you actually work with the data once …
Recently, Microsoft came out with a private preview for OneLake Security in Microsoft Fabric lakehouses. This is amazing, it is a feature I have been waiting for now for a long time. The promise with Fabric would be that security in the most detailed grain (both row-level and column-level) would be implemented in OneLake. However, …
Materialized Lake Views in Fabric lakehouses are a very cool option to start introducing data virtualisation patterns within Microsoft Fabric. In today’s post I will show you how you can create MLVs and how you can manage them. Views vs Tables Let’s get this out of the way first. What is the difference between a …
Introduction to Delta Lake Liquid Clustering As your Delta tables grow in size, the need for performance tuning in Microsoft Fabric becomes essential. In this post, I’ll explore two powerful optimisation techniques — Delta Lake Partitioning and Liquid Clustering. Both can help improve query speed and reduce costs, but they work in very different ways. …
When managing large-scale data lakes with Microsoft Fabric, performance optimisation becomes crucial. One effective technique to achieve better performance is Delta Lake partitioning. Partitioning can significantly enhance query performance, reduce computational costs, and improve data management efficiency within Microsoft Fabric environments. In this blog post, we will explore what Delta Lake partitioning is, how it …