Sharing Microsoft Fabric Data With External Users

Merry Christmas from That Fabric Guy! I’m wearing my warm sweater because it’s getting cold outside, and it’s Christmas time. Christmas is all about sharing. And you know what else is all about sharing? Data. Today we’re going to learn how we can use Microsoft Fabric to share data with external parties. Let’s dive in. …

Read more

Using Power BI on an Apple Watch

Today we’re going to look at Power BI on the Apple Watch. Now, why would you want to use Power BI on your Apple Watch? I have no idea. And we’re going to find out. The premise is that with your smartwatch, you have a device with a screen and an internet connection on you …

Read more

Microsoft Fabric with a MacBook

People almost always come up to me at conferences and ask, “Bas, you’re presenting Microsoft stuff and technology on a MacBook. Why do you do that?” The joke that I make is: I love Microsoft, but I also love hardware that works. My MacBook is simply the most amazing laptop I’ve ever had. And because …

Read more

How to change Microsoft Fabric Python resources

A couple of weeks ago, I showed you how we can use DuckDB as a Python library in Microsoft Fabric to do Python notebooks on Delta Lake instead of using Spark notebooks. The benefits of using Python over Spark: Python is more lightweight, your session startup times are much lower, and these notebooks use less …

Read more

Vibe Coding in Microsoft Fabric

Can ChatGPT actually help you build a complete data platform in Microsoft Fabric if you pretend to know absolutely nothing? That’s exactly what I set out to test. I gave myself a simple scenario: I’m a data engineer with a SQL database that needs to be transformed into a proper medallion architecture with bronze, silver, …

Read more

Setting V-Order for delta tables in Microsoft Fabric

Microsoft recently made a change that caught a lot of people off guard: they disabled V-Ordering by default for new Fabric lakehouses. If you’re using Power BI Direct Lake mode, this matters—a lot. V-Ordering can dramatically improve your query performance, but it comes with tradeoffs. Today I’m going to show you how to check if …

Read more

All about Lakehouse in Microsoft Fabric

Lakehouses are the foundation of data engineering in Microsoft Fabric. They’re where you store your raw data, your transformed tables, and everything in between. But if you’re new to Fabric, lakehouses can be a bit confusing. What’s the difference between tables and files? What are shortcuts? How do you actually work with the data once …

Read more

Stop wasting money in Microsoft Fabric: Dataflows vs Notebooks

There’s a common debate in the Fabric community: should you use low-code tools like Dataflows Gen2, or should you write pro-code solutions with PySpark notebooks? Today I’m going to settle this debate with actual performance and cost data. Spoiler alert: low-code is easier, but you’re paying for that convenience with significantly higher compute costs. Let …

Read more

Case Sensitivity in Microsoft Fabric Spark

I recently ran into a sneaky issue while migrating a customer from SQL Server to Microsoft Fabric. We were doing what every lazy developer does—copy-pasting code—and suddenly our joins stopped working. Half the data was missing, relationships weren’t connecting, and it took me way too long to figure out why. The culprit? Case sensitivity. SQL …

Read more

OneLake Security Preview in Microsoft Fabric

Recently, Microsoft came out with a private preview for OneLake Security in Microsoft Fabric lakehouses. This is amazing, it is a feature I have been waiting for now for a long time. The promise with Fabric would be that security in the most detailed grain (both row-level and column-level) would be implemented in OneLake. However, …

Read more

VS Code Notebooks to Improve Your Microsoft Fabric Experience

When you’re using PySpark notebooks in Microsoft Fabric data engineering, you can develop straight from the web browser. While that is interesting, a browser is usually not the most perfect software development environment. In this article I will show you how you can use VS Code Notebooks to develop for Microsoft Fabric. Why Use VS …

Read more

Delta Lake Liquid Clustering vs Partitioning

delta lake liquid clustering schema

Introduction to Delta Lake Liquid Clustering As your Delta tables grow in size, the need for performance tuning in Microsoft Fabric becomes essential. In this post, I’ll explore two powerful optimisation techniques — Delta Lake Partitioning and Liquid Clustering. Both can help improve query speed and reduce costs, but they work in very different ways. …

Read more

Delta Lake Partitioning for Microsoft Fabric

When managing large-scale data lakes with Microsoft Fabric, performance optimisation becomes crucial. One effective technique to achieve better performance is Delta Lake partitioning. Partitioning can significantly enhance query performance, reduce computational costs, and improve data management efficiency within Microsoft Fabric environments. In this blog post, we will explore what Delta Lake partitioning is, how it …

Read more

Microsoft Fabric Costs Explained: A Complete Guide

Understanding the cost implications of implementing Microsoft Fabric is crucial for organisations transitioning to this unified analytics platform. This guide explores the fundamentals of Fabric’s billing model, helping you make informed decisions about resource allocation and cost management. Microsoft Fabric costs is a difficult topic, apparently, let’s make it a bit more understandable! The Basics …

Read more

Extracting Paginating APIs Without NextPage Metadata with Microsoft Fabric Notebooks

Most APIs these days will have some kind of pagination built into them. This is to make sure that queries against the underlying database are not returning too much data, compromising the database performance as well as sending too large messages across the network. Often, these APIs will tell you in their responses how many …

Read more

Implementing the DRY Principle in Microsoft Fabric

When we start implementing a data lake using Microsoft Fabric, we might be tempted to start creating pipelines and notebooks right away, without thinking about design principles. However, there’s one design principle I’d like you to consider from the beginning: DRY. DRY is an acronym that stands for Don’t Repeat Yourself. In this blog post, …

Read more

Notebook Orchestration in Microsoft Fabric

Coming from the ‘old school’ world of SSIS and SQL Server, and later Azure Data Factory and Azure SQL Database, I have always built my ETL orchestration processes using some kind of pipelines. In Fabric, we also have pipelines (the successor of ADF), but, we can now also create notebook orchestration using NotebookUtils and runMultiple(). …

Read more

How To Implement Medallion for Microsoft Fabric Lakehouses

medallion architecture in Microsoft Fabric

Introduction In this guide, we’ll dive straight into the practical steps of how to implement Medallion Lakehouses using Microsoft Fabric. If you’re unfamiliar with the Medallion Architecture, or need a refresher, check out our previous post on the theory behind the Medallion Architecture. This post focuses on hands-on implementation, guiding you through each step of …

Read more

Medallion Architecture for Microsoft Fabric, a Theoretical Guide

medallion architecture in Microsoft Fabric

The medallion architecture is maybe the most important concept to grasp when you start building lakehouse solutions on Microsoft Fabric. Data products should never be built ‘right away’ or on their own. There’s a structure and process to follow. Luckily, we don’t have to figure out that structure by ourselves. Rather, I’d suggest using what …

Read more