When implementing real-time ingestion, we usually implement an architecture called lambda. Using the lambda architecture, KustoDB in Microsoft Fabric is always recommended for the speed layer. Do you know why? Let’s analyze in detail. 1 – KustoDB uses SSD KustoDB uses an internal SSD storage. Lakehouses use ADLS as their backend. In this way, Kusto … Read more
In the first part of this two-part series, I covered the mostly non-technical aspects of building a data culture. While the lion’s share of the work will be getting people to work together and embrace ever deeper use of data, as a reader of Simple-Talk, a lot of this transition will be technical. In this … Read more
I have been talking about Data Exploration in Power BI on many of my sessions, specially the sessions about Data Marts. The new data exploration feature is one more feature on this expanding scenario for data exploration. This one brings some interesting details. We start using this feature from a query. The feature will allow … Read more
The most watched videos on my youtube channel are related to security. Azure SQL Networking Secrets and Cloud Security: Secure Access from your Applications to Azure SQL explain private endpoints from different points of views: Azure SQL and Function Apps. The concepts of Private Endpoints apply to most Azure features and even the relation of … Read more
One of the major trends in enterprise computing, and really in enterprises themselves is an increased emphasis on data. My career has always revolved around data, but this is a new focus for many parts of the organization. Even business units that traditionally don’t care about data realize that access to more, and better, data … Read more
Let’s consider a simple statement for partitioning and save a table in a lakehouse: df.write.mode("overwrite").format("delta").partitionBy("Year","Month","Day").save("Tables/" + table_name) Let’s consider we load the data daily, with all the transactions from the day. The table will save the transactions for each day in different partitions. We can expect the table to keep the partitions from previous day, … Read more
On the blog Fabric Notebook and Deployment Pipelines I explained a technique to keep notebooks configuration values in JSON files on lakehouses, a good solution from many different points of views. What if we need to provide maintenance to the JSON configuration file using notebooks? The first problem is the fact the typical statement to … Read more
When I was a brainstorming ideas for a series of posts, it took me back to a brainstorming session we had in a company I have worked for. As we got started, the leader said this phrase: “there are no dumb ideas.” Now, clearly none of us in the room thought that there really aren’t … Read more
Power BI Semantic Models have a property called Available In MDX. Each field in the model has this property and we can only see or change its value through XMLA. For example, we can use Tabular Editor for this. We can disable this property in the fields where it will not be needed (and in … Read more
Editor note: The call for volunteer and speakers is open at on the Pass Data Community Summit 2024 site for this year’s Summit coming up in Seattle November 4-8! Volunteering for local data events large and small is a great way to get into the data community and make connections that will last a lifetime. … Read more
Dataflows Gen 2 are the new version of Power BI dataflows. There are so many changes in relation to the previous version they are considered a new feature. The main difference is the possibility to set a target for the result of each query in the dataflow. In this way, it can be used as … Read more
On my article about Source Control with GIT, Power BI and Microsoft Fabric, I illustrate how to use the PBIP file format to include Power BI reports and semantic models in a source control process and stablish a SDLC (Software Development Lifecycle) for Power BI. However, the complete explanation is based on saving the development using … Read more
It is time again for the PASS Data Community Summit Call For Speakers. Some of you are very ready. You have a presentation idea that has percolated since at least last October. You have asked friends and coworkers if it is a good idea. You may have presented it at 10 SQL Saturdays to thunderous … Read more
One of the first things you learn when working with SQL Server, and other transactional based SQL systems, is that set based operations perform best. If you are querying data, a cursor pulling individual rows doesn’t perform as well as a single query. It doesn’t matter if that cursor is on the client side or … Read more
When organizing our SDLC (Software Development Lifecycle) in Power BI/Fabric, we use Deployment Pipelines and create rules to change connection configurations every time we promote an object from one environment (dev for example) to another (test, for example). Kusto connections, on the other hand, are not so simple. You can check more about Deployment Pipelines … Read more
Eventstream has many differences in relation to the technologies it proposes to replace. Event Hub, Stream Analytics, Streaming Dataflows and more. We can compare these technologies, but EventStream in Microsoft Fabric has some specific differences from all of them. One of the differences is how the transformation of the input data is linked to the … Read more
Power BI and Fabric are implementing source control support. It’s a long-awaited feature for Power BI. However, it’s important to highlight some basic principles which should be followed as source control best practices. Some of them apply to any project in source control, some are specific for this environment, and some are specific for this … Read more
You probably have heard the saying “if you fail to plan, you plan to fail”. I get the sentiment here, when you have a task to do, if you don’t plan how to do the task, you are far less likely to be successful at completing the task to the satisfaction of the person requesting … Read more
My motivation for writing this summary was an interaction with a project owner that didn’t understand why we couldn’t use feature flags directly in Power BI to control the user interface. This was different from our other deployments, so it took a few rounds of explanations to convince them that our use case didn’t support … Read more
I used to think plagiarism was my worst thing to deal with as an editor. That is only sort of true now. Now the biggest headache has become figuring out who has used AI to do their writing and to what extent. Often that is even less easy than it even sounds. While there are … Read more
Simple Talk Bloggers
Subscribe for more articles
Fortnightly newsletters help sharpen your skills and keep you ahead, with articles, ebooks and opinion to keep you informed.
