I have started my career in Business Intelligence with SSIS. We had a reasonably small data warehouse, and my job was to maintain the existing SSIS packages and create new ones. Most of what I was doing was same again, so I could copy and paste many elements from one package into another...

It was the day when I finally fell in love with Postgres. I got a challenging task to generate data that haven’t existed for reporting. It turned out that it’s very easy to achieve in Postgres…

One of my customers has a very elegant multi-tenanted application with the flexible schema achieved by the extensive use of JSON. While base tables are the same, JSON allow a lot of customisation. Unfortunately, the tradeoff of this is a lot of manipulation...

photo by kolibri5 Some RDBMS support SQL statement MERGE, which also referred as upsert. This is a very common task in data warehousing: we compare the source dataset with what has been recorded in DW, insert new values and update those which have changed since the...

The time has arrived where I've needed to learn more Python for a project I'm working on.  Here, I share some resources, thoughts and tips in case you find yourself also needing to learn Python, too. Python is an interpreted object-oriented language. This means you...

At the end of last year, I attended a virtual conference organised by Snowflake for the Asia-Pacific region. The beauty of online event is that you can watch sessions on demand, which is handy as many interesting sessions were scheduled to be streamed at the same...

Most of my career I have spent in big organisations developing and maintaining enterprise-wide solutions. Therefore, so far I’ve mostly worked with MS SQL Server and only touched OracleDB a bit. But for a new, smaller project, I've uses PostgreSQL for the first time...

It is always a challenge to learn how to use new software products. Of course, all of them have documentation and how-to guides to explain the basics. However, many people would rather watch videos or find a teacher as this is the best way for our brains to consume...

Cloud solutions have little to do with the solutions we have on-premises. Cloud solutions are built with ambitious and monstrously huge goals in mind, so, often it feels like no previous technology has been reused. In addition to some understandable concepts, like...

On my first BI job, where I was an ETL developer, my team was using Microsoft SQL Server Reporting Service (SSRS) as a reporting tool. My job was to model the Data Marts and create the data flows into those tables. Our report analysts were good at SQL, I have even...

Almost two years ago I was a part of the BI team in an organization that decided to move to the Cloud. As a first step they followed the lift and shift approach, so they moved their Data Warehouse and all the existing processes into the Azure environment almost...

On one of my recent projects I was analysing my customer's data for migration. Their legacy application could handle documents. They were stored on the organisation's shared folder, and the file path was recorded in the database with the rest of data. As a part of the...

Recently I watched a webinar organised by Snowflake, it was called "Data Warehouse Automation, Ingestion and Industry Leading Analytics with Snowflake and Qlik". In the past I could never find time for webinars, and it's amazing how life can give you opportunities to...

Traditionally business intelligence is an enterprise solution, as only big businesses have multiple sources that require integration and analysis. It is assumed that the rest are small fishes that could be able to analyse their data with simpler accessible tools or...

Reporting and analytics computations are usually very resource-consuming, therefore they are never executed on the same server where the crucial business application is running. Instead, data is copied into dedicated servers to be dissected and studied for insights....

This month OptimalBI is hosting a Data Science Pioneers documentary screening. The first screening was crowded and the rest of the sessions are booking up very quickly, so you might have heard about it already. I'm guessing, the selling point (although it's completely...

Probably you know that our boss Victoria supports many digital initiatives across New Zealand. One day she has shared with us one of the latest Digital Future Aotearoa initiative called Electric Garden. This is an Internet of Things solution for garden monitoring...

The first I came across Snowflake was a few years ago when I was learning how to use Attunity Replicate, that time I established a data load into this cloud data warehouse. This time I have decided to explore what Snowflake is about. Terminology I watched some...