The New Database Landscape

Just when everyone thinks the DBMS market is mature with 3-4 players (IBM DB2, Oracle, Microsoft SQLServer and  open source players like MySQL and Ingres), we see increased activity in both research and development. Why?

The traditional RDBMS market is 25 years old, the pioneering research foundation work being System R at IBM and Ingres at UC Berkeley during the 1970s. The design point during those days were slow processors, expensive and limited main memory, and expensive and slow disk storage. Hence these principles were adopted:

  • Disk oriented storage and indexing structure (B-tree)
  • User interaction via dumb terminals as I/O device
  • Multithreading to hide latency
  • Locking-based concurrency control mechanisms
  • Log-based recovery

Now look at the landscape. The Web is the client interface. Data is mostly text, video, maps, graphics, audio, etc.  While general purpose DBMS’s served the  business market well, specally in the OLTP  and data warehousing world, they are inadequate to address the new world of unstructured data. Even new challengers are outperforming them in Warehousing and OLTP maket.

According to Mike Stonebraker (founder of Ingres and now at MIT as a professor), there are at least 4 non-trivial markets where the traditional architecture (he calls it the row store) can be clobbered by specialized architecture. He lists them as follows

  • Warehouse (new start-ups Vertica, SybaseIQ,..)
  • Text (Google, Yahoo)
  • Scientific data (MATLab, ASAP prototype)
  • Streaming data (StreamBase, Coral8)

These speciallized architectures outperform their RDBMS brethren by factors as high as 50 to 100 times. A new prototype called H-Store from MIT is challenging the OLTP benchmarks TPC-C numbers (84 times faster than a RDBMS). The mantra is to exploit expanded memory and high-speed processing and bandwidth.

This industry thrives on new research and development challenging the incumbents. I am so excited  to see this happening.


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s