Category: Data Engineering

  • Implementing Fabric Eventstream & Data Activator With Real World Data

    Implementing Fabric Eventstream & Data Activator With Real World Data

    Understanding a tool’s utility is directly proportional to our knowledge of its use. This blog post aims to evaluate the advantages and disadvantages of using Fabric Eventstream & Data Activator with real-world data. We aim to achieve this while minimizing infrastructure requirements and costs. In essence, this post demonstrates how to implement continuous location data…

  • How-To Access Azure Key Vault Secrets from Fabric Notebook

    How-To Access Azure Key Vault Secrets from Fabric Notebook

    This blog post focuses on how to handle authentication within Fabric Notebooks using secrets without storing them in your code. To access Key Vault from a Fabric Notebook, use the PyTridentTokenLibrary to handle the call while using an access token. Specify the name of the secret in Key Vault and provide the URL of the…

  • Cultivating Intelligence: The Data-Backed Future of Vertical Farming

    Cultivating Intelligence: The Data-Backed Future of Vertical Farming

    Introduction: Data-Driven Cultivation – Where Science Meets Innovation In the realm of modern agriculture, a profound shift is underway — a harmonious blend of science and innovation that’s shaping the future of farming. Imagine a world where plant growth is deciphered through data, where technology orchestrates a seamless partnership with nature. In this blog, we…

  • Customizing ChatGPT: Answering Questions about Your Own Data with Embeddings

    Customizing ChatGPT: Answering Questions about Your Own Data with Embeddings

    ChatGPT is trained on a large set of data but what if you want it to answer questions about your own data, for example building a chatbot that answers questions about your knowledge base? The answer is embeddings and in this blog post I explain the underlying mechanisms and how to implement them. The code…

  • Incorrect Power BI Report Data After Refresh

    Incorrect Power BI Report Data After Refresh

    You just spent a lot of time designing your report and now it’s time to upload it to the Power BI Service. After setting up the credentials for the data source, you hit refresh to make sure the data can be pulled from the source. The refresh completes without error and you switch over to…

  • Nested ForEach loops in Azure Data Factory

    Nested ForEach loops in Azure Data Factory

    In a recent project, I tried to copy data from a SharePoint site to an Azure Blob Storage using Azure Data Factory (Copy files from SharePoint to Blob Storage using Azure Data Factory – Syntera). The goal was to enable employees to upload their expense receipts to their SharePoint site using OneDrive’s built-in scan function.…

  • Impact of renaming columns in Power BI

    Impact of renaming columns in Power BI

    Renaming columns in Power BI can have a severe impact on your reports. Initially, renaming columns in Power BI ensures that your end users don’t have to deal with cryptic system names from your source system. Over time, you may need to change a name of a column because of new terminology or to standardize…

  • Copy files from SharePoint to Blob Storage using Azure Data Factory

    Copy files from SharePoint to Blob Storage using Azure Data Factory

    Azure Data Factory is a great tool for ETL pipelines, and we love working with it. However, when it comes to the integration with the rest of the non-Azure Microsoft world (especially SharePoint) it can get a bit frustrating. In a recent project I wanted to build a solution, which allows employees to upload documents…