Facebook changing the game in messaging

This morning, at F8 developer’s conference in San Francisco, Marc Zukerberg announced “bots” for Facebook Messenger. Bots are humanlike robots built with AI algorithms that can interact between the user and business. For example, to order a Uber cab, you do not need to go into the Uber app, but directly ask for one via Facebook Messenger using a bot.

Facebook announced a new API today that will allow developers to build bots into Messenger. The idea is that brands and retailers like 1-800-Flowers or CNN can connect with users via messaging, and automate those conversations to eliminate the need for human-to-human interaction. That means you’ll soon be able to order flowers through private messaging without ever needing to chat with another human, for example. CNN and 1-800-Flowers were named onstage as early partners, but it’s likely that many more will be shown off throughout the conference. The more tasks you can accomplish within Messenger, the more likely you are to open the app every day. And the more purchases you make inside Messenger, the more potential revenue for Facebook. This is a brilliant stroke and a game changer.

There will be a way to search for bots in the app, and developers can advertise their bots on Facebook’s News Feed, driving users to Messenger to interact with the bot. For those who don’t know how to build bots (or may not have the technical artificial-intelligence chops to handle the language recognition challenges), Facebook is offering a “bot engine” to help developers build them. That engine will be powered by Wit.ai, an artificial intelligence startup it bought in January of 2015 to help people chat with robots.

Now the question is what will happen to the Apple store in the iPhone? Apple brings tons of business apps and makes a huge profit. Now Facebook users can bypass all that and directly interact with business via these bots. That will jeopardize the relationship between Apple (Facebook depends heavily on the iPhone for its mobile users) and Facebook. Will Apple be upset enough to cut Facebook off its iPhone partnership? Let us watch and see. The other question is – where is Google in all this?

This again shows the smart move by Zuck and team!

Top Five Digital Trends

BI Intelligence recently published the Top 5 Digital Trends that is worth looking at. It starts with a dramatic statement, “today, nearly 43% of the world is connected to the Internet, enabling us to talk, share photos, and conduct business halfway across the globe. As a result, we have seen more technological advancements in the past 10 years than we’ve witnessed in the past 10,000 years. And in the next five years, we’ll see even more”. Here are the key trend areas:

  1. Mobile – Messaging apps are the new OS. Facebook is winning this as it owns both the Facebook Messenger and WhatsApp. Many companies have apps on top of these messaging apps, like payments, video, taxi, etc.
  2. Digital Media – The rise of the bots. This morning, Marc Zukerberg showed how to use a bot to order flowers directly from the mobile device. Programmatic advertising — or the automatic buying and selling of ad impressions — has exploded in recent years, as the digital shift has led to an increase in ad inventory. This also brings in more fraud problems mimicking humans.
  3. Mobile Payments – Digital wallet is the future. Mobile in-store payments have gained traction over the past two years. The EMV (Europay Mastercard Visa) migration and the launch of mobile wallet solutions from smartphone providers have been the two largest enabling factors for in-store mobile payments. Consumers will be further incentivized by offer and loyalty programs, which will drive up volume.
  4. E-Commerce – Shipping innovation. Traditional shippers like Fedex and UPS have been increasing their fees, forcing mega-retailers like Amazon and Walmart to create their own shipping solutions. Amazon has been buying aircrafts and fleet of trucks since last year.
  5. IoT – The next Industrial revolution. Businesses are using the Internet of Things to lower their operating costs and increasing efficiency. Any device with an IP address (home devices like Nest, fitness devices like fitbit, etc.) becomes part of the IoT ecosystem. Eight industries are being transformed by IoT: Oil & Gas, Agriculture, Manufacturing, Insurance, Retail, Healthcare, Utilities, and Food services.

Hadoop, the next ten years

I attended a meetup yesterday evening at the San Jose Convention Center on the subject “Apache Hadoop, the next 10 years” by Doug Cutting, the creator of Hadoop while at Yahoo, who works at Cloudera now. That venue was chosen because of the ongoing Strata+Hadoop conference there.

It’s always fun listening to Doug recounting how Hadoop got created in the first place. Based on early papers from Google on GFS (Google File System) and Map Reduce computing algorithm, a project was launched called Nutch, subsequently renamed Hadoop (after Doug’s son’s toy elephant name). This all made sense as horizontal scaling via commodity hardware was coming to dominate the computing landscape. All the modules in Hadoop were designed with a fundamental assumption that hardware failures are common and should be automatically handled by the framework. That was all back in 2006. As an open source project, Hadoop gained momentum with community support for the overall ecosystem. Over the next seven years, we saw many new additions/improvements such as YARN, Hbase, Hive, Pig, Zookeeper, etc. Hence, Doug wanted to emphasize that there is a difference between just Hadoop and the Hadoop ecosystem.

The original Hadoop with its Map Reduce computing had its limitations and lately Spark is taking over the computing part. Spark provides an interface for programming entire clusters with implicit data parallelism and fault-tolerance. It originated at UC, Berkeley’s AMPlab and is gaining fast momentum with its added features for machine learning, streaming, graph and SQL interfaces. To a question from the audience, Doug replied that such enhancements are expected and more will come as the Apache Hadoop ecosystem grows. Cloudera has created Impala, a speedier version plus the SQL interface to meet customer needs. Another example of a key addition to the ecosystem is Kafka which originated from Linked-In. The Apache Kafka project is a message broker service and  aims to provide a unified, high-throughput, low-latency platform for handling real-time data feeds. To another question on whether another general-purpose platform will replace Hadoop, Doug suggested that projects like Spark will appear to handle parts of the ecosystem better. There may be many purpose-built software to address specific needs like Kafka. He eloquently praised the “open Source” philosophy of community of developers helping faster progress compared to the speed at older companies like Oracle in enhancing its DBMS software.

From the original Hadoop meant for batch processing of large volumes of data in a distributed cluster, we are moving towards the real-time world of streaming analytics and instant insights. The popularity of Hadoop can be gauged by the growth in attendance of the San Jose Hadoop Summit…from 2700 attendees in 2013, it more than doubled last year.

Doug is a good speaker and his 40 minute talk was informative and entertaining.

Hats Off, Andy Grove! RIP

I happen to be in Poland and just heard of Andy Grove’s passing away. If you go south from here, cross Slovakia, you will reach Hungary. That’s where Andy Grove grew up in a jewish family. Then came the Nazi occupation in the 1940s when jews were exterminated. Andy had to flee the country to Austria in very difficult circumstances. Then he got unto a boat and sailed for unknown land of America. He reached New York with $20 in his pocket and no idea what he was going to do. As he enrolled at City University of NY, he took up physics. He taught himself english and topped the class soon. A brilliant student, he continued his journey to the west coast to UC, Berkeley where he got a Ph.D. in chemistry/chemical engineering.

Andy started at Fairchild, but left to join Bob Noyce and Gordon Moore to start a new company called Intel. He basically built the company from scratch. From memory chips, Intel took a leap into microprocessors, considered a huge risk in those days. But Andy was at the helm, the CEO of a hugely successful company Intel, that almost defined the Silicon valley of today. Many would agree that Andy created the Silicon valley. He was Time’s man of the year in 1996. His straightforwardness and no-nonsense style of management is legendary. His famous book, “Only the Paranoid Survives” made big waves when it came out. You have to run scared to win in your business. Intel defined the computing era of 1980s and 1990s.

Andy taught many leaders of the valley as a mentor – Larry Ellison, Steve Jobs, John Doerr, Mark Zuckerberg, etc. etc. During my days at Oracle, I had the opportunity to listen to Andy a couple of times. He lead by example and was completely full of substance. People who worked for him, adored him. With all that success, he had the humility of very egoless person. He contributed significantly to many charities through his foundation. Oh, there is so much we can learn from Andy Grove.

As we mourn his death, let us remember his great leadership qualities combined with utter humility.

Stack Fallacy? What is it?

Back in January, Tech Crunch published an article on this subject called Stack Fallacy, written by Anshu Sharma of Storm Ventures. Then today I read this Business Insider article on the reason why Dropbox is failing and it is the Stack Fallacy.  Sharma describes Stack Fallacy as “the mistaken belief that it is trivial to build the layer above yours.”

Many companies trivialize the task of building layers above their core competency layer and that leads to failure. Oracle is a good example, where they thought it was no big deal to build applications (watching the success of SAP in the ERP layer initially built on the Oracle database). I remember a meeting with Hasso Plattner, founder of SAP back in the early 1990s when I was at Oracle. He said SAP was one of the biggest customers of Oracle at that time and now Oracle competes with them. For lack of any good answer, we said that we are friends in the morning and foes in the afternoon and welcomed him to the world of  “co-opetition”. Subsequently SAP started moving out of Oracle DB and was enticed by IBM to use DB2. Finally SAP built its own database (they bought Sybase and built the in-memory database Hana). Oracle’s applications initially were disasters as they were hard to use and did not quite meet the needs of customers. Finally they had to win the space by acquiring Peoplesoft and Siebel.

Today’s Business Insider article says, “…a lot of companies often overvalue their level of knowledge in their core business stack, and underestimate what it takes to build the technology that sits one stack above them.  For example, IBM saw Microsoft take over the more profitable software space that sits on top of its PCs. Oracle likes to think of Salesforce as an app that just sits on top of its database, but hasn’t been able to overtake the cloud-software space they compete in. Google, despite all the search data it owns, hasn’t been successful in the social-network space, failing to move up the stack in the consumer-web world. Ironically, the opposite is true when you move down the stack. Google has built a solid cloud-computing business, which is a stack below its search technology, and Apple’s now building its own iPhone chips, one of the many lower stacks below its smartphone device”.

With reference to Dropbox, the article says that it underestimated what it takes to build apps a layer above (Mailbox, Carousel), and failed to understand its customers’ needs — while it was investing in the unimportant areas, like the migration away from AWS. Dropbox is at a phase where it needs to think more about the users’ needs and competing with the likes of Google and Box, rather than spending on “optimizing for costs or minor technical advantages”.

Not sure, I agree with that assessment. Providing efficient and cost-effective cloud storage is Dropbox’s core competency and they are staying pretty close to that. The move away from AWS is clearly aimed at cost savings, as AWS can be a huge burden on operational cost, plus it has its limitations on effective scaling. In some ways, Dropbox is expanding its lower layers for future hosting. It’s focus on enterprise-scale cloud storage is the right approach, as opposed to Box or Google where the focus is on consumers.

But the Stack Fallacy applies more to Apple doing its own iPhone chips, or Dell wrongfully going after big data. At Oracle the dictum used to be, “everything is a database problem – if you have a hammer, then everything looks like a nail”.

Amazing Amazon

I remember back in 2003 when I had a meeting with the then CTO of Amazon for a couple of hours. He was narrating his vision of SOA (Service Oriented Architecture), where individual business or programming functions (called services) can be stacked up in libraries and get invoked as and when required. This notion of re-usable services was not new (remember subroutines from the mainframe era or stored procedures from the client-server days?).

Subsequently we called them “web services” because they were loosely coupled applications that can be exposed as services and easily consumed by other applications using Internet standard technologies. Phrases such as XML (EXtensible Markup Language), UDDI (Universal Discovery, Description, Integration), WSDL (Web Services Definition Language), and SOAP (Simple Object Access Protocol) were new lexicons then. These were URL addressable resources that could exchange information and execute processes automatically without human intervention. Oh yes, we talked about how the equivalent of a phone dial-tone is evolving to a personal digital dial-tone (Internet) to  an application digital dial-tone (web services).

I wondered then – why a book-selling company like Amazon was speaking this language, so far off its core business? Then the CTO explained to me that Jeff Bezos wanted to monetize the excess capacity from his massive data center investments which were idle like 70% of the time. Hence the starting point was S3 (simple shared storage), where savings of the order of one-hundredth could be achieved. No one believed such claims initially, but Amazon continued its journey into “cloud computing” with offering computing power as a utility with EC2 (elastic computing cloud). I left that meeting quite amazed, to say the least with skepticism in my mind.

Fast forward 12 years. Amazon’s AWS (Amazon Web Services) is the de-facto leader in cloud infrastructure provisioning business. It is both at IaaS (Infrastructure as a Service) and PaaS (Platform as a Service) levels. Amazon became the harbinger of “Cloud Computing”, taking that laurel away from leaders such as IBM, and HP. In 2015, the AWS business brought $8B in revenue. Others such as Microsoft’s Azure or IBM’s Bluemix or Oracle’s Cloud offering are all playing catch-up to AWS. Google’s cloud is yet to be a serious contender for enterprise computing. No wonder, they have hired Diane Green (VMWare founder) as the new cloud czar with a huge financial package.

Some predict that AWS will become the largest business unit at Amazon over time. Although I just read that they are after a $400B business, that of transportation, owning their own delivery services to replace Fedex and UPS (recently Amazon has been getting its own freight liners, trucking fleets, etc.). Amazon is secretive on its new business, just as back in 2002-2003, they were way ahead on their thoughts on cloud computing. Bezos is now part of the $50B+ club (top five richest people).

RocksDB from Facebook

I attended a HIVE-sponsored Meetup yesterday evening titled, “Rocking the database world with RocksDB”. Since I had never heard of RocksDB, I was curious to learn how it is rocking the database world.

Facebook built this key value store storage layer originally to use for MySQL (instead of InnoDB), as MySQL is used heavily at Facebook. They claim that was not the only motivation. Then in 2013, they decided to open source RocksDB. Last evening’s speaker in an earlier post on November, 2013 had said, “Storing and accessing hundreds of petabytes of data is a huge challenge, and we’re constantly improving and overhauling our tools to make this as fast and efficient as possible. Today, we are open-sourcing RocksDB, an embeddable, persistent key-value store for fast storage that we built and use here at Facebook.”

RocksDB is also ideal for SSD (Flash store) and claims fast performance. The team was excited when MongoDB opened up to other storage engines back in 2014 summer. For a period of time, MongoDB plus RocksDB was a fast combination. Then MongoDB decided to acquire WiredTiger ( a competitor) in December, 2014 to contribute to the performance, scalability, and hardware efficiency of MongoDB. That left RocksDB out of the official engagement with MongoDB. But they built something called MongoRocks that claims to be very fast. It seems several MongoDB users prefer MongoRocks over the native combo of MongoDB with WiredTiger.

Several users of RocksDB talked about their experience, specially in the IoT world where sensor data can be processed at the edge (ingestion, aggregation, and some transformation) before being sent to the cloud servers. The only issue I saw is the fact that there is no “real” owner of RocksDB as a deliverable solution. There is no equivalent of a Cloudera (For Hadoop) or Confluent (for Kafka) who can provide value-additions and support for the user base. It’s all open source download and do-your-own stuff till now. So serious production-level deployment is still a risky affair. For now, it’s a developer’s play tool.

Follow

Get every new post delivered to your Inbox.

Join 28 other followers