Design-thinking at IBM

The Sunday New York Times published this article on IBM’s new way of thinking that is worth reading.

The article states – The company is well on its way to hiring more than 1,000 professional designers, and much of its management work force is being trained in design thinking. “I’ve never seen any company implement it on the scale of IBM,” said William Burnett, executive director of the design program at Stanford University. “To try to change a culture in a company that size is a daunting task.”

If you ask people inside IBM for a design-thinking success story, they are likely to mention Bluemix, a software tool kit for making cloud applications. In just one year, Bluemix went from an idea to a software platform that has attracted many developers, who are making apps used in industries as varied as consumer banking and wine retailing. In the past, building that kind of technology ecosystem would have taken years.

Software developers are just as important as customers to IBM, since both groups create markets. “We wanted to redefine IBM for developers,” said Damion Heredia, an IBM vice president who leads the Bluemix operation. When a free test version of Bluemix was offered in February 2014, Mr. Heredia figured it might attract 2,000 developers in the first few months. It reached that number within a week, and a commercial version was introduced that July. Today, Bluemix is signing up 10,000 new users a week.

The new mantra at IBM is speed, speed and speed. The design-oriented approach is promised to yield the agility they need in re-inventing themselves. Being 104 years old, IBM has changed course and re-invented itself may times. Now the focus seems to be cloud computing, analytics, and big data. The emphasis from hardware has waned shifting more into software and service. They are also being successful in recruiting young graduates from top schools like Stanford, when these kids often think Google is an old company (and IBM is a historic relic). Design leadership from people like Phil Gilbert is bringing fundamental changes in this metamorphosis.

It’s worth watching how fast IBM switches its culture!

Dell + (EMC+VMW) = A $67B Gamble!

Yesterday, Dell announced the largest technology M&A in history with a proposed$67B buyout of EMC and VMware (via EMC’s 80% ownership of VMW). The combined company will have over $80B in revenue, employ tens of thousands of people around the world and sell everything from PCs, servers & storage to security software and virtualization software. Not to be overlooked is the fact that Dell and EMC will be private companies and free from the scrutiny of activist investors.

Dell has to borrow a ton of money to make this deal, like $40B debt. The annual interest payment will be $2.5B! This deal has three entities – Michael Dell’s own investment, Silverlake Partners, and Singapore based Tomasek. On paper this seems like the two companies bring complimentary values – Dell sells to small to medium size companies and EMC addresses the larger enterprise needs. The big attraction for Dell is the VMWare piece that revolutionized the desk-top virtualization market. Currently VMWare is 25% of EMC’s revenue, but 50% in valuation.

The concern is that as more corporations adopt cloud storage and cloud computing for their IT needs, there is less reason to spend money on the costly software and hardware upgrades typically offered by established IT companies like EMC. But by consolidating, they can better compete against the lower-cost cloud service companies – AWS (Amazon Web Services), IBM, Alphabet (Google), and Microsoft Azure.

This is going to be a big gamble. The HP CEO circulated an internal memo suggesting how this will be great opportunity for HP, as the combined company will create a lot of chaos and confusion. At the same time, being private, the new entity can execute radical restructure. But this will be a herculean task to make the combined company a winner in the highly competitive “IT infrastructure” market.

IBM’s big commitment to Apache Spark

Last June IBM made a serious commitment to the future of Apache Spark with a series of initiatives:  

  • It will offer Apache Spark as a service on Bluemix (Bluemix is an implementation of IBM’s Open Cloud Architecture based on Cloud Foundry, an open source Platform as a Service (PaaS). Bluemix delivers enterprise-level services that can easily integrate with your cloud applications without you needing to know how to install or configure them.
  • It committed to include 3500 researchers to work on Spark-related projects.
  • It will donate IBM SystemML (its machine learning language and libraries) to Apache Spark open source

The question is why this move by IBM?

First let us look at what is Apache Spark? Developed at UC Berkeley’s AMPLab, Spark gives us a comprehensive, unified framework to manage big data processing requirements with a variety of data sets that are diverse in nature (text data, graph data etc) as well as the source of data (batch v. real-time streaming data). Spark enables applications in Hadoop clusters to run up to 100 times faster in memory and 10 times faster even when running on disk. In addition to Map and Reduce operations, it supports SQL queries, streaming data, machine learning and graph data processing. Developers can use these capabilities stand-alone or combine them to run in a single data pipeline use case. In other words, Spark is the next-generation of Hadoop (came with its batch pedigree and high latency).

With other solutions for real-time analytics via in-memory processing such as RethinkDB, an ambitious Redis project or commercial in-memory SAP Hana, IBM needed a competitive offering. Other vendors betting on Spark range from Amazon to Zoomdata. IBM will run its own analytics software on top of Spark, including SystemML for machine learning, SPSS, and IBM Streams.

At this week’s Strata conference, several companies like Uber described how they have deployed Spark all the way for speedy real-time analytics.

Translytical Database

This is a new term I learnt this week, thanks to the Forrester analyst Mike Gualtieri. Terms like Translytics or Exalytics (Oracle’s phrase) do not roll off the tongue that easy. Mike defined Translytical as , “single unified database that supports transaction and analytics in real time without sacrificing transactional integrity, performance, and scale.”

[Transactions + Analytics = Translytical]

Those of us who saw the early days of Data Warehousing, we deliberately separated the two worlds, so that analytics workloads do not interfere with transaction performance. Hence snapshots of operational data were taken to do data warehousing for offline batch analysis and reporting. Mostly that was getting a retro-view of what happened. In the current scheme of things, where data is coming fast and furious from so many sources, there is need to look at trends in real time and take action. Some insights are perishable, therefore need to be acted on immediately. All data originate fast, but analytics usually done much later. Perishable insights can have exponentially more value that after-the-fact traditional historical analysis. Here is a classification of analytics:                     Past —- Learn (Descriptive Analytics),                                                                             Present —- Infer (Predictive Analytics), Detect (Streaming Analytics),                   Future —– Action (Prescriptive Analytics).

Streaming analytics (real time) require a database that can do in-memory streaming for near-zero latency for complex data and analytical operations. The traditional approach of moving data to analytics has created many silos  such as CRM stack, BI stack or Mobile stack. Translytical databases are transactional as well as analytical. Point solutions like Spark data streaming which does micro batch processing are not the answer. Such a unified database must do in-memory processing (use RAM for real-time), multi-modal and support compression and tiered data as well.  Customers are stitching together open source products such as Spark, Kafka, and Cassandra to achieve streaming analytics, but it becomes a non-trivial programming task.

The only database claiming to be Translytical is VoltDB with functions such as: in-memory processing, scale-out with shared nothing, ACID compliance for transactional integrity, reliability and fault tolerance. It also has real time analytics built in combined with integration with Hadoop ecosystem.   Such a unified database has to prove its worth in the market.

So we have come full circle – from single database to more than one to handle both transactions and analytics; now back to single database doing both.

It makes logical sense, but let us watch and see if that works.

Cassandra Summit 2015

I attended my first Cassandra Summit 2015 this week at the Santa Clara convention center. I was quite surprised to see more than 6000 people attend this event, much bigger than last year (2000 attendees) with 130 sessions. It was a proof to the growing popularity of Cassandra’s NoSQL database platform. CTO and cofounder  Jonathan Ellis (formerly from Rackspace) described new release 2.2 and 3.0 functions.

With its major addition of JSON support in 2.2, Cassandra basically eliminated the difference with MongoDB. They have their own query language called CQL, an SQL-like construct. Now with JSON support, the developer community will see some big advantages. They have functions like collections, udf (user defined types), and deeper nesting. Release 3.0 will see a brand new storage engine, a vast improvement to their previous key-value store engine with better space efficiency. Release 3.0 will include materialized views. Bragging about their fast performance and efficient distributed database functionality, Jonathan joked about MongoDB as the “snapchat for databases” (a reference to occasional data loss because of weak consistency). He emphasized three key elements: availability (onstage they dramatized the shutting down of many nodes in two data centers with Cassandra still running), scale, and performance (both read and write).

However, when it comes to streaming analytics, one Cassandra user explained how he combined Spark, Kafka, and Cassandra to achieve the same – a non-trivial programming feat. Cassandra CEO Billy Bosworth emphasized that they solve the transaction workload problem (always-on) and not geared for analytics. I understood that two key customers are Apple running their iTune application on Cassandra and Netflix. Majority of use cases were web-centric applications where speed and scale are key requirements. Several case studies indicated customers replacing MySQl with Cassandra.

In a panel discussion on monetizing open source software, the comment was made that Cassandra is foundational and customers are happy to pay for the fast performance and scale using their enterprise edition. In that sense, they are different than the RedHat model.

It was an interesting experience to see a new-generation database product gaining wider acceptance.

HP – Where is it headed?

HP (Hewlett Packard) is in the news today. The company (HPQ) is splitting itself into two separate companies on Nov 1st. – HP Enterprise (HPE) and HP Inc.(HPI). The first one will include the server line, consulting, and related software whereas the second entity will have the PC and printer business. In the process, 33,000 employees will be laid off (10% of its current workforce of 300,000). This will be on top of 55,000 earlier lay-offs under the current CEO Meg Whitman. The lay-offs will be carried out over next 3 years, mostly from the HP Enterprise. The restructuring will result in pretax charges of about $2.7 billion at HPE and $300 million at HP Inc, which has been hit hard by a relentless decline in sales of PCs.

HP has had a checkered history of many mis-steps. When the company’s growth stalled in the late 1990’s, then-CEO Lew Platt also opted for a split, spinning off HP’s original medical products and test and measurement devices unit. The next decade would bring a string of mega-mergers under a long line of various CEOs, starting with now-Presidential-hopeful Carly Fiorina and her $19 billion Compaq acquisition in 2002, then the $13.9 billion purchase of IT services company EDS under CEO Mark Hurd and finally the $11.1 billion acquisition of Autonomy under Leo Apotheker. The Compaq acquisition saw a lot of drama in the board room and arguably it did not yield the results HP had hoped. Fiorina was fired by the board at the end, but walked away with almost $100m in severance. It seems this will be brought to the debate by the GOP hopefuls.  Mark Hurd resigned unceremoniously and is now co-CEO at Oracle. Leo Apotheker did not last even for a year and was fired by the board. The subsequent acquisition of Autonomy was a disaster. Meg Whitman took over in 2011 as CEO and this major split is happening under her watch.

Whitman hopes with her latest move to make HPE  a leading player in cloud services. Still, that path is a challenging one: Rivals like Amazon, Microsoft, Google and IBM are already dominating the cloud market, and it won’t be easy for HP to break through. Given the poor state of the PC market, HPI will continue to face big challenges in terms of revenue and earnings. “HP Software” has always been an oxymoron with hardly any worthwhile product to mention. Openview was a success in the systems management arena. Now they are focusing on big data and analytics with acquisitions like Vertica, but it is yet to make a big impact in the market.

We will wait and see.

Another Big day at Apple

Apple continues to amaze. Excitement build-up. Total secrecy. Rumor mill in full works. Then the day comes like this morning! A perfectly scripted, 2-hour extravaganza packed with new products and features. The Steve Jobs tradition lives – perfect execution of a marketing event!

Tim Cook started off with the Apple Watch update.  The cool fashionable Apple Watch Hermes with great bands and watch faces. Another sports lineup with gold, rose gold, and anodized aluminum colors plus red band to go with. There were cool demos of new Watch apps, such as AirStrip where a pregnant mother can monitor baby’s heartbeat and transmit that to the doctor instantly. Other apps include Facebook messenger, GoPro, calendar, etc.

Next was the big news of iPad Pro, a large 12.9″ screen with multi-touch and 5.6m pixels. It comes with a full size smart keyboard that can get connected magnetically. The chip is A9X, a 64-bit powerhouse that doubles in performance compared to the A8X. Interestingly this iPad is 22 times faster than the original iPad. Apple also introduced a new smart pencil to draw sketches on the iPad Pro. Due to its large real estate, multiple applications can be parked, as shown by Microsoft Office guys (Excel, Word, and Powerpoint on the same screen). Adobe also demoed its comp, Photoshop Fix, and Photoshop Sketch on the iPad Pro. The iPad Pro comes at a hefty price of $799(32GB) and $949(128GB) and more for the 256GB one. The pencil costs $99 and keyboard is a hefty $169.

The other big news was the Apple TV, a revamped receiver and a brand new remote with cool features such as button for Siri and touchpad to scroll faster. Tim Cook said the future of TV is Apps such as iTunes, Netflix, Hulu, etc. The operating system is called tvOS based on iOS where developers can build new applications. It will be better user experience for sure. I like the remote control with a volume adjustment button. That way, you won’t need another remote control for the TV. This is great for Apple TV users like me who have abandoned cable for ever. This is the future for TV for sure.

Finally there was the iPhone. As predicted Apple introduced two new versions, iPhone 6s and iPhone 6s Plus. The big feature is the 3D Touch where apps can pop up for quick viewing and disappear. It has a better camera with 12 MP power with HD video or 4K video. Tim Cook mentioned that iPhone has grown 35% overall during last one year, but 75% in China. The new operating system iOS9 which is available on Sept. 12. The 6s ranges in price from $199 to $399, and 6s Plus costs $299 and up – no change from existing iPhone6 and 6Plus prices.

As usual, it was a delightful event with many cool innovations complimented with the baritone voice of Johny Ive on video for each new product design.