Big Data – Status

According to a Wall Street Journal article today by Rachael King and Steven Rosenbush, the market for new databases serving Big Data reached $1.22B last year and is expected to more than double by 2014 (according to research firm Wikibon). That is quite impressive.

Since relational databases using SQl are inefficient in handling data from social chatters, smartphones, and clicks (because of volume and variety), new databases are popping up over last 3-4 years. In the past two years 119 database software companies have been funded by VC’s for $1.17B (according to Venture Source, a Dow Jones company). This is remarkable, as not too long ago, the space was declared taken by 3 incumbents – IBM, Oracle, and Microsoft. However, the scene has changed dramatically now.

Thanks must go to Google for pioneering the start of new innovations in Big Table, GFS (Googel File System), and Map-Reduce algorithms for massively parallel processing using commodity hardware clusters. These technologies became part of Apache open source foundation and the result is Hadoop, HDFS, and several associated tools for the new ecosystem. Amazon, Yahoo and Facebook have also contributed good work here.

The article mentions a client Autozone using one of the new DBMS’s called NuoDB for better managing store inventory according to local shoppers. NuoDb like many others offers a cloud service with an annual subscription, cutting Capex for customers.

Another client Trulia (online real estate) was using MySQL, but has added Cassandra to better manage the listing of home foreclosures and apartment listings of its 100 million homes in the US.

Shutterstcok, a photo agency, stores 24 million images with 10,000 added each day. It uses HDFS (Hadoop) to find out user behavior (how long they hover over an image before purchasing).

The article suggests that large financial clients will stick to existing vendors such as Oracle for various reasons, but the threat of these newcomers is there. This is much like the cloud software  is shaking up Microsoft’s desktop software model.

We are in the data-intensive computing era now and the race will be fierce for leadership and market share.

Big Shift to Enterprise Cloud Software

One of the big shifts happening in the enterprise software business is the adoption of the Cloud. This is already the dominant model for the consumer Internet space (e.g. Google, Facebook). The overall myth was that large companies are so concerned about security and privacy of data that they will be reluctant to let go of running these applications in-house. Then about 3 years back, Oracle’s CEO ridiculed the cloud as vapor. But the landscape has surely changed and will change even faster going into 2012. Why do I say that?

Recently SAP bought Success Factors at a very high valuation and many saw that as a desperate move to get into the Cloud. Oracle also bought RightNow in the same vain and there is a strong possibility that Oracle will acquire NetSuite in the next twelve months (NetSuite addresses mostly the SMB market). SalesForce.com (SFDC) has been growing and its valuation has gone quite high. The new entrant Workday (founded by the founders of Peoplesoft) has been doing very well. Someone said it is valued at almost $2B with yearly booking rate of $300M. Not bad for a five year old company. Workday addresses the HR space, but is moving to financial applications as well. It wants to be known as an “ERP Replacement” company, not just an “HR Company”.

Most of Workday customers are the large enterprises such as Tyco, AIG, Flextronics, etc. These customers are replacing legacy applications from SAP and Oracle with Workday’s SaaS suite offered through the Cloud. The advantages are obvious for the customers – same version always, new functions added more frequently, vendor takes care of upgrades, less cost paid to SI’s. The traditional applications typically require a 5 to 6 times expense (over product license cost) on consultants (SI’s) who keep customizing the apps. Workday claims their customers have a 90% reduction on SI cost, which is a big deal.

The challenge for the enterprise Cloud vendors is the revenue model, where they do not get the hefty license dollars up front. Hence their investment (both capex and opex) is quite high. For example, Workday has taken investment of $250M to date, counting the last round. But if they can sustain like SFDC, then there is a thriving growth business. By the way, Google has a growing enterprise business and currently brings $1B revenue which is not bad at all. They clearly are focused to make this part grow faster in future.

This is not good news for legacy enterprise application companies like SAP and Oracle. They have to re-invent (or acquisitions like Success Factors and RightNow) quickly to catch up with this shift to the Cloud by large enterprises. Oracle is more likely to ride this bandwagon sooner than SAP. As per the SI’s, they need to re-invent also to at least keep their current clients happy. That means they have to shift from “customization” of code to more reconfiguration of business processes.

SaaScon 2009

I attended SaaScon 2009, a conference focusing on Software as a Service and Cloud computing as themes. This was hosted by ComputerWorld at the Santa Clara Convention center.

At the outset, host Jeff Kaplan and Ron Milton said that $6.8B was the SaaS revenue in 2008, according to Gartner group. IDC said 76% of US companies had at least one SaaS deployed. Top five applications in SaaS delivery model are (not surprisingly) – CRM, HR, Collaboration, Travel expenses, and Sales Incentives. Top five drawbacks are – integration with current in-house applications, potential security exposures, offline connectivity, vendor lock-in, and network bandwidth issues. As expected the top five benefits are – fast roll-out, monetary savings from a lack of capital investment, option of plain-vanilla vs. customized implementation, and lower TCO (total cost of ownership).

Under the current economic realities and tight IT budgets, SaaS makes lot of sense. One surprising fact is that 75% of all SaaS deals do not have SLA (Service Level Agreement) according to Gartner. The conference speakers also talked about PaaS (Platform as a Service)  and IaaS (Infrastructure as a Service). Examples of SaaS -  SalesForce.com (CRM), and NetSuite. Examples of PaaS – Force.com, and Google App. Engine. Examples of IaaS – Amazon EC2 (elastic computing cloud) and AT&T Synaptic Hosting.

Speakers said PaaS is a stepping stone to Cloud where development environment is offered to build own applications (similar to what Microsoft did 20 years ago, but on their platform). One future scenario could be your own information grid at the center interacting with several clouds such as data services, application services, hardware services, storage services, network services and others. This will include not only interaction with your internal cloud or grid, but also intra-cloud communication, something still at a very nascent stage.

The parallel to the commoditization of electricity was drawn a few times. How that grid (electricity) finally emerged with self-organizing, self-registration, and self-assembly features. Computing will eventually get to that stage – full compliment of services executed in a highly differentiated grid environment.

But the journey has started and there is no looking back to the good old days of huge capital expenditure of building in-house infrastructure with hundreds of humans managing it. We see the beginnings of that already, all of us using our iphones to access mail, calendar, maps, stock prices, airline information, and social networking like facebook, Twitter, and Linked-In.

One CIO explained how he reduced his IT spend from $607K in 2007 to $259K in 2008 by switching to SaaS (Google Apps for gmail, videochat, documents, spreadsheets, charts, etc.). He adopted Netsuite as the core ERP/CRM application and picked other packages        like WorkOasis (facility management), Halogen (performance management), ADP (payroll. HRIS), and admin tools like Webex, Echosign, eFax, Zoomerang, and VisualCV. His staff cost was reduced from $166K in 2007 to zero in 2008. Hardware cost was reduced from $52K to $8K. His is a small size company selling exercise equipments. No wonder the first adopters of Saas are the small to medium size companies.

IBM to buy Sun?

According to this morning’s WSJ article IBM is considering to acquire Sun for $6.5B.  To some people this will make sense and to others this is of little value. I think the benefit is more for Sun. IBM would gain Sun’s customer base plus $3B in cash reserve. But it will also face some product overlaps such as MySQL vs. EnterpriseDB. Sun Solaris proprietary systems will need to be migrated over to Linux.

Sun has been looking for a suitor for last few weeks. It continues to flounder in the market. The mighty “dot” in the dot-com business, or the creator of the slogan “network is the computer” has seen its glory days. It is a great company that starts good stuff, but somehow unable to monetize  them. Java started there, but IBM has more Java developers than Sun. Sun has been struggling to find its DNA and lately it has been focusing on being an open source (monetization?) company. Its acquisition of MySQL for a billion dollars has not resulted on any significant market leadership. It might have alienated older partners such as Oracle.

So what will IBM gain, other than reducing the “server” competitors by one? Let us see how it turns out.