My colleague discussed in a recent post the different categorisations for visual exploration tools. Like most modern tools Qlik Sense does a bit of everything. These days the strength of the tools is not whether it can do something, more how hard is it to do it. With this in mind, you could say the strength of Qlik is as a Discovery Information Product. It can definitely be used as a guided information product but I think of this capability as a subset of the functionality of a Discovery Information Product. Yes, it can blend too, basic blending can be done via the GUI but generally, anything useful will require coding. Hence, why I say it’s more of a Discovery Information Product.
Qlik’s approach to data is to load it all into memory, it’s stored in a columnar data store where you can generally get 90% compression ratio. So you can load a lot of data into a few GB or Ram. However, there are other considerations which mean you shouldn’t try to load it all into memory. Most likely you’re in a shared environment which means your data isn’t the only set loaded in memory. Some things won’t compress well, such as unique values or the types of data you might expect from big data. There are still limits even if your data compresses well, if you have terabytes of data even with the best compression ratios you will still eventually run out of RAM.
So what should you do when you want to access a subset of all the data that you could possibly load?
- Become a coder and use variable and filters in your load scripts. Not really a nice option when you want to publish an app into a self-service world.
- You use on On Demand app generation. This is the idea that you dynamically build an app with a subset of the data available based on a user’s needs.
On demand app, generation is a step into the realm of the Blending information product, which is great if you want to do more with Qlik. However, it’s not a trivial task and involves a few moving parts to orchestrate. Thankfully, one organisation has posted their solution as open source. It provides an excellent starting point if you want to build your own On demand app.
If I was a robot I’d be Marvin – Ben
Ben writes blogs about the technical side of BI; the code, all the code and not much other than the code.
You can read all of Ben’s blog here.
Don’t forget, we can train your team in the art of agile business intelligence at any time!