Columnar Database Processing with DB2 and IBM Blu Technology


In memory processing of columnar-oriented databases has seen an increase in terms of popularity and use over the last couple of years. With the significant decrease in the cost of DRAM, a significant change is taking place when it comes to processing large data sets.

Last year Massimo Pezzini, VP and Gartner Fellow quoted the following.
“The relentless declines in DRAM and NAND flash memory prices, the advent of solid-state drive technology and the maturation of specific software platforms have enabled IMC to become more affordable and impactful for IT organizations,”

Some people have called it the “Holy Grail of Analytics” This means organisations can perform complex analyses in “real time”, and allows users to slice and dice large data sets in a quicker and in many cases more efficient manner. More vendors are strutting their stuff when it comes time to bringing their solutions to the market forefront.

What is a Columnar Database?

A columnar database is a database management system that stores data in columns instead of rows. The objective of a columnar database is to efficiently write and read data to and from hard disk storage in order to speed up the time it takes to return a query.

The biggest advantage and benefit is that data can be highly compressed. The compression allows columnar operations such as MAX, MIN, SUM, COUNT & AVG— to be executed very quickly. Columnar-based systems are self-indexing database management systems (DBMs), which essentially use less disk space than a relational database management system (RDBMs) which contain the same data.

What is In Memory Computing?

Although defined hundreds of times on the web, in simple English, it means that information is stored and processed in RAM rather than hard disks. The main objective is to reduce the time transferring data off and on slow hard disks and increase the performance and time to compute large instructions and process data sets.

What is IBM BLU Acceleration?

IBM BLU is a collection of technologies derived from IBM Research and development labs for analytical database workloads.

4 key points to know about IBM BLU

  • Dynamic In Memory Columnar processing with dynamic movement of data from storage
  • Actionable compression technique that preserves order so that it can be evaluated
  • Parallel Vector Processing
  • Data skipping which skips processing irrelevant data

If you want to learn more about it, please visit the official IBM BLU HUB

Here is a pic which describes the function of IBM BLU

ibm blu DB2

source: http://www.ibmbigdatahub.com/presentation/presentation-blu-acceleration-delivering-speed-thought-analytics

Here is a link I came across to a highly technical whitepaper which describes what IBM BLU is and how it functions.

IBM BLU for SMB

Since I work with customers of all sizes, my major concern was around the small and mid size business. Many of these in memory computing solutions are built and priced for large enterprises. I was happy to see that a version for SMBs was available.

Analytics Warehouse Service on Bluemix  ($50/month for a small instance) – Click here for details

If you are concerned about compatibility with DB2, it is built-in to version 10.5 and is backward compatible, which protects your current investments

IBM BLU Use Cases

When I look at a solution, I always look to see what customers have to say. I was able to find these 3 use cases where customers discuss their IBM BLU experience.

IBM Blu Home Page

Get social with IBM BLU on Twitter #ibmblu

Image Source - http://ibmbluhub.com/why-blu-acceleration/what-is-in-memory-computing/

«

Leave a Reply

Your email address will not be published. Required fields are marked *