The CAP theorem

Eric Brewer, professor at UC Berkeley came up with the CAP theorem. CAP stands for Consistency, Availability, and Partitioning or Distributability. He says out of the 3, the maximum you can achieve is 2.

For example, if you’re Amazon and you want various servers distributed all over the world for better availability, then you must forfeit consistency. If you buy a book in New York while I buy one in Singapore from two different servers, then these two databases are out of sync and not consistent. Eric calls this “eventually consistent”, as they will be synchronized later.

Another example is of a major bank, where billions of dollars get transferred between accounts. Consistency is so important besides availability that you forfeit “distributability”. Hence you centralize everything out of one data center.

I somehow like this “maximum 2 out 3″ theorem. A friend who went to MIT for undergraduate as well as graduate school had told me that between sleep, grades, and friends you can only have two. If you want to sleep and do well in studies, then forget friends. If you like to socialize and also sleep, then forget grades. If you like friends and want to keep good grades, then forget sleep.

Another “2 out of 3″ example is an entrepreneur who has to pick two out of the 3 variables – work, family, and socializing. It’s well-known that founders of start-ups with families have to give up socializing. Some do the mistake of too much work and socializing and neglect the family.

In the new world of Web 2.0, Eric’s CAP theorem is very insightful. He defined BASE as opposed to ACID (atomicity, consistency, isolation, durability) as another end of the spectrum. BASE stands for Basically Available, Soft state, Eventually consistent. One must examine where the application fits in the scale between ACID and BASE and decide accordingly.

Top IPO Contenders in 2010

As we start this year, hope is mounting on a vibrant IPO market, better than last couple of years. This article lists 20 companies that are hot candidates for IPO. The list has some well-known names like Facebook, Skype, LinkedIn, Twitter, Digg, Yelp, LiveOps, and Tesla Motors. The less known names are – Associated Content, BrightCove, Chegg, Demand Media Etsy, Exact Target, Gilt Group, Glam Media, Rearden Commerce, Workday, and Zynga.

Workday is the new company founded by Peoplesoft founder David Duffield. It’s a SaaS-based HR and ERP company.

Zynga is a hot company in the virtual gaming space on the Internet. It’s famous game Farmville is raking in good revenue from Facebook users. I hear the game is quite addictive.

Twitter is rumored to be valued at billion plus dollars, that with very little revenue. It has the publish-subscribe model where conversations-by-subject can be tracked. That should be a bonanza for marketers,  seeking specific target audiences.

Most of the companies in the list are addressing “content” either the discovery or the publishing of. We don’t see the old-fashioned enterprise applications anywhere, a reflection of the changing times. Workday is in that category, but purely cloud-based offering just like SalesForce.com few years back.

Companies like LinkedIn, Facebook, Twitter, and Skype brag huge number of eyeballs (users), bringing back memories of the dot.com days. Jeff Bezos in the height of the boom had said, “I spell profit as prophet”.

Let us get back to some crazy wealth-creation via IPO’s. It’s about time.

Top Ten Technologies for 2010

Every year around this time, Gartner Group hosts its big conference in Florida and invites some big names from our industry to talk about new trends. Gartner Group also presents what will be hot in the following year. Sometimes, it is like stating the obvious.

I remember my days at Oracle when certain Gartner group analyst will spend time listening to my views of database futures. Next thing I know I have paid $2000 conference fee to listen to the same analyst presenting back mostly what I had told him. It’s a good business – this “information selling”.

In fairness though, Gartner Group listens to many people like me from different companies and consolidates the data and presents them as future trends.  So this month at their conference,  Gartner Group said the following ten technology trends for 2010.

1.  Cloud computing – style of computing that characterizes a model in which providers deliver a variety of IT-enabled capabilities to consumers.

2. Advanced Analytics – last year it was BI. So this year it has to be advanced analytics, although it is unclear what exactly that is. It is simulation, prediction, optimization, etc.  Neil Raden wrote a blog on why we don’t need Ph.D’s for doing advanced analytics.

3. Client Computing – new ways of packaging client computing via virtualization.

4. IT for Green – ways IT can enhance your green credentials in terms of energy savings, reduction of paper, reduction of travel, etc.

5. Reshaping the Data Center – cutting operational expenses.

6.  Social Computing – use of social software and social media in the enterprise and participation and integration with externally facing enterprise sponsored and public communities.

7. Security-activity monitoring – going from the old model of putting up a security perimeter fence to the new world of monitoring activities and identifying patterns that would have been missed before).

8. Flash memory – moving up to a new tier in storage echelon.

9. Virtualization for Availability – same old notion of virtualization for better systems availability.

10. Mobile Applications – by year-end 2010, 1.2 billion people will carry handsets capable of rich, mobile applications.

The NoSQL Movement

Now there is a new community called NoSQL and they   (150 enthusiastic developers of the cloud era)   met for the first time  last June in San Francisco.

“Relational databases give you too much. They force you to twist your object data to fit a RDBMS [relational database management system],” said Jon Travis, principal engineer at Java toolmaker SpringSource.  NoSQL-based alternatives “just give you what you need,” Travis said.

So get used to names like Dynomite,  CassandraDB, Voldemort, CouchDB, BigTable, MongoDB, Hypertable, SimpleDB, etc. These are all members of this club which was started by Google’s BigTable and its clones such as Hypertable (ZVent). Amazon is doing SimpleDB on Dynamo (not called a database, but a highly available key-value data store). Facebook is doing CassandraDB. Apache CouchDB is a free, open source, document-oriented database written in Erlang programming language. MongoDB is a collection of JSON documents ( no rows or columns), an open source document-oriented DB written in C++ programming language.

So what are the common factors pushing this movement?

• They can blow through enormous volumes of data. For example, Google’s BigTable with its sister technology MapReduce processes as much as 20 petabytes of data per day. We have not seen this volume in RDBMSs.
• They run on clusters of cheap PC Servers. Google has said that one of BigTable’s bigger clusters manages as much as 6 petabytes of data across thousands of servers. Oracle’s RAC (Rapid Application Cluster) can get there but at a much higher cost.
• They beat performance bottlenecks.  The phrase used here is “eventually consistent”, trading off consistency to maximize availability and scalability.
• While conceding that relational databases offer an unparalleled feature set and a rock-solid reputation for data integrity, NoSQL proponents say this can be too much for their needs. Hence the mantra is “no overkill”.

I think this is exciting development. When people like me worked on early days of RDBMS, we could not imagine the kind of scalability and data volumes being talked about now. It’s only natural that new approaches must be innovated to handle the demands of the Internet era. Eric Brewer of UC Berkeley floated the idea of this in his research work at least 6-7 years ago.

Although NoSQL movement is not a threat to mainstream database community yet, this may change in next 3-4 years time.

Facing 2009 amidst all the gloom and doom!

We are into month 2 of the year 2009.  January was exciting with the new president Obama’s inauguration. The last two months since the election victory were painful with a lame-duck president and an economic crisis getting worse with each passing day. Someone said referring to the past president, “It’s like a dinner guest who refuses to quit.” With unemployment at a record high, there are hardly any signs of economic sunshine. Microsoft laid off 5000 people, a first in its history. Even Google is belt-tightening with cost cutting.

But then, you hear some encouraging signs here and there. IBM did well with its quarterly growth and income. Twitter got an infusion of cash ($35m) even if it was not looking for money and its revenue-producing business model not yet public. The arena of “Application Modernization” with the goal of reducing TCO (Total Cost of Ownership) and improving user experience seems to be getting more attention. The switch-over from the old client-server (read fat client) to a web-based architecture makes lot of sense. The consumer sector leads the way here. But enterprises continue to be laggards, mostly because of their dependence on “packaged software” such as SAP and Oracle.

The job market for software professionals is weak, based on anecdotal evidence. No one is recruiting right now. It’s a “wait-and-see” mentality. Prediction of a better second-half of 2009 is making the rounds. Start-up companies are following the mantra of “conserve cash”. Enterprise IT spending is at an all time low.

Let us hope this period ends soon.

Conference Season begins

As summer winds down in August (vacation month), the conference season begins in September. First there is Office 2.0 just after labor day (September 3-5) in San Francisco, organized by Ismael Ghalimi, for the third year in a row. Ismael does a great job. I have attended and participated in both the earlier conferences in 2006 and 2007. The focus is wider than originally planned. Now it covers three key areas -  Collaboration + Mobility + Productivity.

Then in September also, Web 2.0 Expo happens in New York. This is the east coast version of the larger show in San Francisco last April. The usual suspects will be there singing the glory of Web 2.0, social networking, tagging, and how these things are moving to the enterprise. I will be there discussing Enterprise RIA from our Curl booth.

Comes October  and we have AjaxWorld 2008 in its west coast incarnation at the San Jose convention center, after the New York event last March. You can see how they are marketing this event via speakers such as me. I will be speaking on real world examples of Rich Internet Application deployment by various large companies. While we can discuss the charm and beauty of Web 2.0 or Ajax, the proof is in the pudding of real usage. Curl ‘s RIA platform is used extensively in Asia by many global companies for business-critical applications. This is a switch from old-world client-server architecture (of which Oracle, SAP, etc are based) to a web-based architecture.

And there are many other conferences between September-December.

Which RIA Platform to choose?

This morning I read a very useful and interesting article written by my colleague Richard Monson-Haefel at Curl. He describes which RIA (Rich Internet Application) platform to chose for what purpose. In other words, there is no “one size fits all” solution. You would pick Ajax for simple consumer type applications such as Amazon.com. If you need animation, then Flex or Silverlight can be the right solution. For enterprise RIA applications which are stateful, transactional, with high scalability, security and performance needs, the MIT-Research-based product Curl is the right answer.

Since RIA is an emerging category, there exists a lot of confusion. Flash or Flex was the only choice and everyone thought that is the alternative to the first-generation (Web 1.0) technologies such as Javascript, CSS, DOM (the aggregate being termed Ajax). Ajax suffers from security and scalability issues for enterprise class RIA. Also, the complexity of code development is quite high in Ajax. Now we have choices outside Ajax and we would see these choices being adopted widely in the years to come.

Catalyst Conference 2008, San Diego

I had the opportunity to attend Burton Group’s annual conference called Catalyst 2008 at the Manchester Grand Hyatt in San Diego. There were probably 1200 people from all over the world.

This was my first time attending this conference. I am somewhat skeptical of conferences hosted by analysts firms such as Gartner, Forrester, etc. Having been a vendor writing software for over 25 years, I was always sought by the key analysts to understand what we were building and the future road map for products such as DB2 or Oracle DB. Then you pay to hear the same analyst tell you what you told them. But to be fair, there are good analysts who synthesize lots of information and give some insights. What I get very tired of is hearing the obvious. Tell me something I don’t know, please.

So at the Catalyst conference, I noticed many parallel tracks. One track that must be very good is security or identity management. Because whoever I ran into, mentioned that they are there for the identity management topics. I attended the track for collaboration, Rich Internet Applications, SOA, etc. Again, I was told that REST is the better way than SOAP.  Please!  This is at least 3 years old news. Amazon Web Services (AWS) had done this way back when. Just because the analyst thought it’s new does not make it new news.

Overall, there were many good sessions. Customers presenting real deployment stories were the most valuable. Paisley, a software company in the GRC space (Governance, Risk, Compliance), showed how they have used Curl RIA platform to build very attractive UI for the user. This was a good example of using Web as a platform over client-server to satisfy user demand at a much lower cost.

The evenings were packed with vendors “open house” and I did not even step out of the hotel for 3 days. There was a track on mobile computing and it always drew large audiences Burton Group seems much more technically focused and many attendees are their clients over the years. This was more like their annual user group meeting.

Few words from Japan – June2008

Writing this from Tokyo. I remember my first trip to Japan was 25 years ago, back in 1983 when I worked for IBM. I had come out to make several presentations on the then new technology of Relational Databases. I was part of the DB2 development team at IBM.

Now after many visits, I am here again. Always its amazing to me to see how simple things are so perfect here. Trains run on perfect time (so much so that you can adjust your watch per train departure times), taxi drivers wear white gloves and keep their taxis clean. There is no tipping anywhere. The politeness of the people is very visible and most of all, the work ethics amazes me the most. Everyone, be him a janitor or a construction worker or a systems programmer;  perform with utmost sincerity and efficiency. Everyone stays at the office for at least 10-12 hours a day.

As I met many customers and colleagues, I notice how efficient they are in the mobile technology. Of course my iPhone does not work here  (feel like coming from a third-world country these days). The Japanese use their mobile phone for credit card payment, checking maps and directions, full-function GPS, and much more. The taxi driver politely tells us the traffic ahead is going to be bad and suggests we take the subway instead, sacrificing his taxi fare.

There is much more use of Web 2.0 when it comes to RIA (Rich Internet Application). One customer is replacing all spreadsheet usage via a Curl-implemented web front-end simulatig the spreadsheet UI. Another client built a building security system using Curl, fully deployed on the web. They seem ahead in Enterprise adoption of RIA.

AjaxWorld 2008 – New York City, March 18-20

This was my first time at AjaxWorld conference, its fifth year since inception by Sys-con group. They conduct two of these, spring in the east coast(New York) and fall in the west coast (San Jose). Mostly web development and design folks come to these events. So what did I observe?

- Ajax is just a phrase to enable conversation on a set of technologies which are much older than the phrase, coined only in early 2006. The underlying technologies include the good old Javascript, Cascading Style Sheet (CSS), Document Object Model (DOM), and of course the Microsoft-created protocol XHR (XmlHttpRequest) for asynchronous messaging between the web client and the server. The take-off on usage has been great, mostly in the consumer space. Hence such a conference draws a lot of younger web-gen developers.

- However, it was clear from the March conference that RIA (Rich Internet Applications) is moving far beyond Ajax. The security risks of scripting language and browser host is well known and scares the enterprise world.  Plus, building transactional, complex, web applications via Ajax is highly non-trivial. Hence we have seen the newer platforms such as Curl (origin, MIT), Silverlight (Microsoft), and AIR (Adobe). There are hundreds of Ajax-based platforms like Jackbe, Nexaweb, Laszlo, etc., which are not getting the traction in the serious enterprise community which looks for superb performance, high security, scalability, and greater UI functionality.

- Web 1.0 era saw mostly graphic designers and serious programming folks did not participate. Now Adobe is trying to entice developers whereas Microsoft wants designers. A new term was used “devigners”. All new RIA platforms are bringing a two-language  strategy: a declarative GUI language and a serious OO language for business logic.  Adobe, for example has MXML/ActionScript. Microsoft has XAML/C#. Sun has JavaFX/Java. Interestingly Curl is the only platform that has one language that spans both, thanks to the forward-looking research at MIT. As a consequence, programmer productivity is significantly higher with Curl. There are over 300 large enterprise customers who have deployed Curl as the RIA platform. Bert Halstead from Curl presented at the AjaxWorld highlighting the user experience and technical superiority of Curl.

- Microsoft was very visible at the conference. Silverlight just announced their release 2.0 beta a week before. Someone said, if Silveright was like a black and white TV, Silverlight 2.0 will be a 1080P HDTV quality. But it’s too early to see the truth. The marketing machine was in full gear at the show. There was hardly any presence of Adobe.

- Some keynotes were rather hollow in content and tried to push their solution (you got to do some marketing as a sponsor to the event, a barter system).

Overall it was a good show and full of technical contents in this confusing space.