A provocative title for sure when everyone thinks we just started the era of cloud computing. I recently listened to a talk by Peter Levine, general partner at Andreessen Horowitz on this topic which makes a ton of sense. The proliferation of intelligent devices and the rise of IoT (Internet of Things) lead us to a new world beyond what we see today in cloud computing (in terms of scale).
I have said many times that the onset of cloud computing was like back to the future of centralized computing. We had IBM mainframes, dominating the centralized computing era during the 1960s and 1970s. The introduction of PCs created the world of client-server computing (remember the wintel duopoly?) from 1980s till 2000. Then the popularity of the mobile devices started the cloud era in 2005, thus taking us back to centralized computing again. The text message I send you does not go from my device to your device directly, but gets to a server somewhere in the cloud first and then to your phone. The trillions of smart devices forecasted to appear as sensors in automobiles, home appliances, airplanes, drones, engines, and almost any thing you can imagine (like in your shoe) will drastically change the computing paradigm again. Each of these “edge intelligent devices” can not go back and forth to the cloud for every interaction. Rather they would want to process data at the edge to cut down latency. This brings us back to a new form of “distributed computing” model – kind of back to a vastly expanded version of the “PC era”.
Peter emphasized that the cloud will continue to exist, but its role will change from being the central hub to a “learning center” where curated data from the edge (only relevant data) resides in the cloud. The learning gets pushed back to the edge for getting better at its job. The edge of the cloud does three things – sense, infer, and act. The sense level handles massive amount of data like in a self-driving car (10GB per mile), thus making it like a “data center on wheels”. The sheer volume of data is too much to push back to the cloud. The infer piece is all machine learning and deep learning to detect patterns, improve accuracy and automation. Finally, the act phase is all about taking actions in real-time. Once again, the cloud plays the central role as a “learning center” and the custodian of important data for the enterprise.
Given the sheer volume of data created, peer-to-peer networks will be utilized to lessen load on core network and share data locally. The challenge is huge in terms of network management and security. Programming becomes more data-centric, meaning less code and more math. As the processing power of the edge devices increases, the cost will come down drastically. I like his last statement that the entire world becomes the domain of IT meaning we will have consumer-oriented applications with enterprise-scale manageability.
This is exciting and scary. But whoever could have imagined the internet in the 1980s or the smartphone during the 1990s, let alone self-driving cars?
I moderated a panel of 3 CIOs last Sunday at the Solix Empower conference on the subject of data-driven enterprise. The three CIO’s came from different industries. Marc Parmet of the TechPar group spent many years at Avery Dennison after stints at Apple and IBM. Sachin Mathur leads the IT innovations at Terex Corp., a large company supplying cranes and other heavy equipments. PK Agarwal, currently dean at Northeastern University, used to be the CIO for the Government of California. Here are some of the points covered:
- I reminded the audience that we are at the fourth paradigm in science (as per the late Jim Gray). A thousand year ago, science was experimental, then few hundred years back science became theoretical (Newton’s law, Maxwell’s law..), fifty years ago, science became computational (simulation via a computer). Now the fourth paradigm is data-driven science where experiment, theory, and computation must be combined to one holistic discipline. Actually science hit the “big data” problem long before the commercial world.
- Top level management is starting to understand that data is the oxygen, but they are yet to fully make their organizations data-driven. Just having a data warehouse with analytics and reporting does not make it data-driven, but they do see the value of predictive analytics and deep learning for competitive advantage.
- While business-critical applications continue to run on-premise, newer, less critical apps such as collaboration and email (e.g. Lotus Notes) are moving to the public cloud. One said that they are evaluating migrating current Oracle ERP to a cloud version. Data security and reliability are critical needs. One panelist talked about not just private, public or hybrid cloud, but “scattered” cloud which will be highly distributed.
- Out of the 3V’s of big data (volume, variety, and velocity), variety seems to be of higher need – images, pictures, videos combined with sensors deployed in manufacturing and factory automation. For industries such as retail and telcos, volume dominates. The velocity part will become more and more critical as streaming of these data in real-time will need fast ingestion and analysis-on-the-fly for timely decision making. This is the emerging world of IoT where devices with an IP address will be everywhere – individuals, connected homes, autonomous cars, connected factories. They will produce huge amounts of data volume. Cluster computing with Hadoop/Spark will be the most economical technology to deal with this load. Much work lies ahead.
- There will be serious shortage of “big data” or “data science” skills, of the order of 4-5 million in next few years. Hence universities such as Northeastern is setting up new curriculum on data science. Today’s data scientist must have knowledge of the business, algorithms, comp. science, statistical modeling plus he/she must be good story teller. Unlike the past, it’s not just answering questions, but figuring out what questions to ask. Such skills will be at a premium as enterprises become more data-driven.
We discussed many other points. It was a fun panel.
After a few years gap, I attended this annual conference called TiEcon. TiE stands for The Indus Entrepreneurs, formed 23 years back by some of the valley technocrats originating from India. This is a non-profit organization to foster and help budding entrepreneurs. I helped organize the contents of this 15 years back. Now the scale has gone up and last week, there were almost 3000 attendees from the US and outside. Many attendees came from faraway places like India, Singapore, etc. Let me highlight some of the keynotes I attended.
- Shantanu Narayen, CEO of Adobe – This was the first keynote on day 1 where he narrated how far Adobe has come, from a desktop publishing company of the 1980s and 1990s to a cloud-based digital solutions company. He emphasized the challenge of transformation and said that some of the difficult ones are the antibodies inside the company averse to change. Hence he spent a lot of cycles convincing the troops on why change is so key for survival and growth. Now Adobe has a line of products called Creative Cloud (developers), Document Cloud (Acrobat, etc delivered in cloud), and Marketing Cloud (number of analytics products in cloud). Adobe has also been acquiring companies for non-organic growth, such as Omniture. They claim to be changing the digital experience for everyone, from emerging artists to global brands.
- Vishal Sikka, CEO of Infosys – I liked Vishal’s talk a lot. He has a Ph.D. in computer science from Stanford and his thesis was on AI, which was out of fashion for many years, but is emerging as the latest big trend. Vishal joined Infosys about 21 months back, after being CTO at SAP for many years. He described the tough transition from a product/technology company to a services company. But one can see his stamp of injecting AI technology into the services sector. He calls AI as Automation and Innovation. He announced a new solution called Infosys Mana, a platform that brings machine learning together with the deep knowledge of an organization, to drive automation and innovation – enabling businesses to continuously reinvent their system landscapes. Mana, with the Infosys Aikido service offerings, dramatically lowers the cost of maintenance for both physical and digital assets; captures the knowledge and know-how of people, and fragmented and complex systems; simplifies the continuous renovation of core business processes; and enables businesses to bring new and delightful user experiences leveraging state of the art technology. I was surprised to learn that Infosys has 200,000 employees and they educate something to the order of 17000 people every year in their huge facility in Mysore. Vishal is certainly transforming Infosys and their recent quarterly results have reflected that.
- Sanjay Mehrotra, CEO of SanDisk – This was a real treat as I was unfamiliar with the evolution of SanDisk as a company, built by 3 immigrants – Sanjay from India, Eli Harari from Israel, and Jack Yuan from Taiwan. Sanjay described how he got rejected 3 times for a US visa when he was planning to come to UC Berkeley for his undergraduate studies. He got his BS and MS in electrical engineering and started a career at Intel where he met the other two founders. The three started SanDisk, which created a new revolution in the flash memory business. After 27 years, SanDisk was acquired by Western Digital last October for $19B. I liked the candid answers Sanjay gave to the ups and downs of his journey and how he learned many lessons while going from an engineer to a business leader and growing a company to such scale. He narrated how Sequoia rejected them for the initial investment, suggesting that funding will happen only if they follow the Intel model. Of course they refused. He said that VC’s don’t always see the future and are risk-averse if you are charting a new path.
- Besides these keynotes, I also enjoyed listening to Diane Green, the new cloud czar at Google and how they are planning to compete with the de facto cloud king AWS. Sandy Carter from IBM described how IBM is moving towards building cognitive apps on its Watson platform.
There were several tracks on Cloud, IoT, Data Economy Social Entrepreneurship, etc. Overall it was a good 2-days experience.
BI Intelligence recently published the Top 5 Digital Trends that is worth looking at. It starts with a dramatic statement, “today, nearly 43% of the world is connected to the Internet, enabling us to talk, share photos, and conduct business halfway across the globe. As a result, we have seen more technological advancements in the past 10 years than we’ve witnessed in the past 10,000 years. And in the next five years, we’ll see even more”. Here are the key trend areas:
- Mobile – Messaging apps are the new OS. Facebook is winning this as it owns both the Facebook Messenger and WhatsApp. Many companies have apps on top of these messaging apps, like payments, video, taxi, etc.
- Digital Media – The rise of the bots. This morning, Marc Zukerberg showed how to use a bot to order flowers directly from the mobile device. Programmatic advertising — or the automatic buying and selling of ad impressions — has exploded in recent years, as the digital shift has led to an increase in ad inventory. This also brings in more fraud problems mimicking humans.
- Mobile Payments – Digital wallet is the future. Mobile in-store payments have gained traction over the past two years. The EMV (Europay Mastercard Visa) migration and the launch of mobile wallet solutions from smartphone providers have been the two largest enabling factors for in-store mobile payments. Consumers will be further incentivized by offer and loyalty programs, which will drive up volume.
- E-Commerce – Shipping innovation. Traditional shippers like Fedex and UPS have been increasing their fees, forcing mega-retailers like Amazon and Walmart to create their own shipping solutions. Amazon has been buying aircrafts and fleet of trucks since last year.
- IoT – The next Industrial revolution. Businesses are using the Internet of Things to lower their operating costs and increasing efficiency. Any device with an IP address (home devices like Nest, fitness devices like fitbit, etc.) becomes part of the IoT ecosystem. Eight industries are being transformed by IoT: Oil & Gas, Agriculture, Manufacturing, Insurance, Retail, Healthcare, Utilities, and Food services.
I attended a HIVE-sponsored Meetup yesterday evening titled, “Rocking the database world with RocksDB”. Since I had never heard of RocksDB, I was curious to learn how it is rocking the database world.
Facebook built this key value store storage layer originally to use for MySQL (instead of InnoDB), as MySQL is used heavily at Facebook. They claim that was not the only motivation. Then in 2013, they decided to open source RocksDB. Last evening’s speaker in an earlier post on November, 2013 had said, “Storing and accessing hundreds of petabytes of data is a huge challenge, and we’re constantly improving and overhauling our tools to make this as fast and efficient as possible. Today, we are open-sourcing RocksDB, an embeddable, persistent key-value store for fast storage that we built and use here at Facebook.”
RocksDB is also ideal for SSD (Flash store) and claims fast performance. The team was excited when MongoDB opened up to other storage engines back in 2014 summer. For a period of time, MongoDB plus RocksDB was a fast combination. Then MongoDB decided to acquire WiredTiger ( a competitor) in December, 2014 to contribute to the performance, scalability, and hardware efficiency of MongoDB. That left RocksDB out of the official engagement with MongoDB. But they built something called MongoRocks that claims to be very fast. It seems several MongoDB users prefer MongoRocks over the native combo of MongoDB with WiredTiger.
Several users of RocksDB talked about their experience, specially in the IoT world where sensor data can be processed at the edge (ingestion, aggregation, and some transformation) before being sent to the cloud servers. The only issue I saw is the fact that there is no “real” owner of RocksDB as a deliverable solution. There is no equivalent of a Cloudera (For Hadoop) or Confluent (for Kafka) who can provide value-additions and support for the user base. It’s all open source download and do-your-own stuff till now. So serious production-level deployment is still a risky affair. For now, it’s a developer’s play tool.
I have known Gartner for decades during my IBM and Oracle days. Even though I have observed how they invent new terms to stuff we already know (a bit annoying, but I guess that’s their business), they do a decent job in capturing key strategic trends.
In a recent article, I saw ten strategic technology trends and this is how they are grouped: the first 3 address merging the physical and the virtual worlds and the emergence of the digital mesh (their new phrase); The next 3 trends cover the algorithmic business, where much happens in the background in which people are not directly involved; the final 4 trends address the new architecture and platform trends needed to support the digital and algorithmic business.
The first 3 trends:
- The Device Mesh – In the postmobile world the focus shifts to the mobile user who is surrounded by a mesh of devices, each with an IP address always communicating.
- Ambient User Experience – Seamless flow of experience across a shifting set of devices. Think of shifting from IoT, to automobiles, smartphones, etc.
- 3D Printing Materials – This will necessitate the assembly line and supply chain processes to exploit 3D printing.
The next 3 trends:
- Information of Everything – This information goes beyond textual, audio and video and includes sensory and contextual stuff.How do you bring meaning to a chaotic deluge of information? Much work is needed here.
- Advanced Machine Learning – Deep Neural Networks (DNNs) go beyond classic computing and information management to create systems that can autonomously learn to perceive the world on their own. DNNs (an advanced form of machine learning applicable to large complex datasets) will make smart machines “intelligent”.
- Autonomous Agents & Things – Like robots, autonomous vehicles, virtual personal assistants and smart advisors.
The final 4 trends:
- Adaptive Security Architecture – how to combat the hacker industry beyond the perimeter defense and rule-based security?
- Advanced Systems Architecture – this is what Gartner said, “Fueled by field-programmable gate arrays (FPGAs) as an underlying technology for neuromorphic architectures, there are significant gains such as being able to run at speeds of greater than a teraflop with high-energy efficiency”.
- Mesh App and Service Architecture – Monolithic, linear application designs like the 3-tier architecture are giving way to loosely coupled integrative approach. Containers(e.g. Docker) are emerging as a critical technology for enabling agile development and microservice architectures. What is needed is a back-end cloud scalability and front-end device mesh experience.
- Internet of Things Platforms – The management, security, integration plus standards are needed for the IoT platform to succeed.
These are all known areas, but I liked the way Gartner grouped them in a logical sequence.