Data Warehouse solutions often include several copies of the same structure, I mean Development, Production and often Test and Pre-Production environments. When the Data Warehousing team works on the new piece of requirements, they don't want to put fully functioning...
Imagine for a minute, a conversation between a database administrator and a data warehouse developer. DBA asks: "I could never understand what you folks actually do..?" - "Well, we create a single database called a data warehouse where the data from multiple other...
The idea of moving into Cloud service takes over minds in organisations I work with. On my previous project we had our databases in Azure, and my current customer is starting the migration project next month. For most of people moving into cloud means everything stays...
With the rise of Digital era more and more organisations want to exchange the data. I'm not talking about Facebook, but mostly about the government organisations I usually work with. One could think that it would be implemented as some sort of API allowing machines to...
In my previous blog about SQL Server Master Data Services (MDS) I gave an overview of the MDS data management tool from Microsoft. In this blog, I'm going to give a couple of examples of the types of problems MDS could solve. The simple case is when a user uploads...
I think we can all agree that it is nice to get things for free when we can, and so I thought of this post about how to set up your local development machine to run U-SQL jobs for development and ad-hoc purposes rather than submitting them to Azure Data Lake Services...
Never has the phrase "a picture is worth a thousand words" been truer - well, in this case, a thousand words refers to lots of bits of code but you will soon see what I mean. In a previous post (Two Cans and a piece of string (Azure Data Factory)) I showed how to...
In my previous post about Azure Data Lake Store, I went over how to create one and what it can be used for. So assuming that you've created one and started to stock it with data and information - just how do we go about fishing out what we want? Let's talk about...
So it got near the end of the month for my "free" Azure trial and I need to change it up to a pay-as-you-go subscription if I want to keep playing with it. Switching to a Pay-as-you-Go subscription Assuming that you want to continue enjoying the azure goodies you will...
In my last post I plunged into the Azure pool and created a SQL Database. Now lets do what we always do with a database and start to fill it up. For this post I'm going to concentrate on the Azure Data Factory product and see how it helps us perform ETL (or ELT)...