Category Archives: Blockchain

Big Data & Analytics – what’s ahead?

Recently I read somewhere this statement – As we end 2017 and look ahead to 2018, topics that are top of mind for data professionals are the growing range of data management mandates, including the EU’s new General Data Protection Regulation that is directed at personal data and privacy, the growing role of artificial intelligence (AI) and machine learning in enterprise applications, the need for better security in light of the onslaught of hacking cases, and the ability to leverage the expanding Internet of Things.

Here are the key areas as we look ahead:

  • Business owners demand outcomes – not just a data lake to store all kinds of data in its native format and API’s.
  • Data Science must produce results – Play and Explore is not enough. Learn to ask the right questions. Visualization of analytics from search.
  • Everyone wants Real Time – Days and weeks too slow, need immediate actionable outcomes. Analytics & recommendations based on real time data.
  • Everyone wants AI (artificial intelligence) – Tell me what I don’t know.
  • Systems must be secure – no longer a mere platitude.
  • ML (machine learning) and IoT at massive scale – Thousands of ML models. Need model accuracy.
  • Blockchain – need to understand its full potential to business – since it’s not transformational, but a foundational technology shift.

In the area of big data, a combination of new and long-established technologies are being put to work. Hadoop and Spark are expanding their roles within organizations. NoSQL and NewSQL databases bring their own unique attributes to the enterprise, while in-memory capabilities (such as Redis) are increasingly being utilized to deliver insights to decision makers faster. And through it all, tried-and-true relational databases continue to support many of the most critical enterprise data environments.

Cloud becomes the de-facto deployment choice for both users and developers. Serverless technology with FaaS (Function as a Service) is getting rapid adoption amongst developers. According to IDC, enterprises are undergoing IT transformation as they rethink their business operations, including how they use information and what technology to deploy. In line with that transformation, nearly 80% of large organizations already have a hybrid cloud strategy in place. The modern application architecture, sometimes referred to as SMAC (social, mobile, analytics, cloud) is becoming standard everywhere.

The DBaaS (database as a service) is still not as widespread as other cloud services. Microsoft is arguably making the strongest explicit claim for a converged database system with its Azure Cosmo DB as DBaaS. Cosmo DB claims to support four data models – key-value, column-family, document, and graph. However, databases have been slower to migrate to the cloud than other elements of computing infrastructure mainly for security and performance reasons. But DBaaS adoption is poised to accelerate. Some of these cloud based DBaaS systems – Cosmo DB, Spanner from Google, and AWS DynamoDB – now offer significant advantages over their on-premise counterparts.

One thing for sure, big data and analytics will continue to be vibrant and exciting in 2018.


Blockchain 101

There is a lot of noise on Blockchain these days. Back in May, 2015 The Economist wrote a whole special on Bockchain and it said, “The “blockchain” technology that underpins bitcoin, a sort of peer-to-peer system of running a currency, is presented as a piece of innovation on a par with the introduction of limited liability for corporations, or private property rights, or the internet itself”. It all started after the 2008 financial crisis, when a seminal paper written by Satoshi Nakamoto on Halloween day (Oct 31, 2008) caught the attention of many (the real identity of the author is still unknown). The name of the paper was “Bitcoin: A peer to peer electronic cash system”. Thus began a cash-less, bank-less world of money exchange over the internet using blockchain technology. Bitcoin’s value has exceeded $6000 and market cap is over $100B. VC’s are rushing to invest in cryptocurrency like never before.

The September 1, 2017 issue of Fortune magazine’s cover page screamed “Blockhain Mania”. The article said, “A blockchain is a kind of ledger, a table that businesses use to track credits and debits. But it’s not just any run-of-the-mill financial database. One of blockchain’s distinguishing features is that it concatenates (or “chains”) cryptographically verified transactions into sequences of lists (or “blocks”). The system uses complex mathematical functions to arrive at a definitive record of who owns what, when. Properly applied, a blockchain can help assure data integrity, maintain auditable records, and contracts into programmable software. It’s a ledger, but on the bleeding edge”.

So welcome to the new phase of network computing where we switch from “transfer of information” to “transfer of values”. Just as TCP/IP became the fundamental protocol for communication and helped create today’s internet with the first killer app Email (SMTP), blockchain will enable exchange of assets (the first app being Bitcoin for money). So get used to new terms like cryptocurrency, DLS (distributed ledger stack), nonce, ethereum, smart contracts, pseudo anonymity, etc. The “information internet” becomes the “value internet”. — Patrick Byrne, CEO of Overstock said, “Over the next decade, what the internet did to communications, blockchain is going to do to about 150 industries”. — In a recent article in Harvard Business Review, authors Joi Ito, Neha Narula, and Robleh Ali said, “The blockchain will do to the financial system what the internet did to media”.

The key elements of blockchain are the following:

  • Distributed Database – each party on a blockchain has access to entire DB and its complete history. No single party controls the data or the info. Each party can verify records without an intermediary.
  • Peer-to-Peer Transmission (P2P) – communication directly between peers instead of thru a central node.
  • Transparency with Pseudonymity – each transaction and associated value are visible to anyone with access to system. Each node/user has a unique 30-plus-character alphanumeric address. Users can choose to be anonymous or provide proof of identity. Transactions occur between blockchain addresses.
  • Irreversibility of records – once a transaction is entered in the DB, they can not be altered, because they are linked to every xaction record before them (hence the term ‘chain’).
  • Computational Logic – blockchain transactions can be tied to computational logic and in essence programmed.

The heart of the system is a distributed database that is write-once, read-many with a copy replicated at each node. It is transaction processing in a highly distributed network with guaranteed data integrity, security, and trust. Blockchain also provides automated, secure coordination system with remuneration and tracking. Even if it started with “money transfer” via Bitcoin, the underpinnings can be applied to any assets. The need for a central coordinating agency such as bank becomes unnecessary. Assets such as mortgages, bonds, stocks, loans, home titles, auto registries, birth and death certificates, passport, visa, etc. can all be exchanged without intermediaries. The Feb, 2017 HBR article said, “Blockchain is a foundational technology (not disruptive). It has the potential to create new foundations for our economic & social systems.”

We did not get into the depth of the technology here, but plenty of literature is available for you to read. Major vendors such as IBM, Microsoft, Oracle, HPE are offering blockchain as an infrastructure service for enterprise asset management.