As Cutter Senior Consultant San Murugesan and I noted in a previous Cutter article, “Innovation is the key to survival and prosperity in this new ‘disruptive’ age, which is dominated by technological advancement and obsolescence, as well as a changing business environment.” However, unlike a one-off, dramatic innovation that changes the world, most innovations in the business space are iterative, incremental, planned, and collaborative. Such innovations use technologies to achieve business goals. And since the business context is continuously changing, an organization’s efforts to innovate cannot be based on rigidly defined goals; instead, they must be driven by a set of guidelines for activities that can produce business value.

Innovators in such a dynamic business space need to be cognizant of the possibility of value coming from unanticipated directions. Correlating this value to business goals can be challenging because the innovation may not always demonstrate direct returns on investment. Big data, the Internet of Things (IoT), and the cloud add to the challenge as the direction of the innovating efforts may be dramatically different from the value that results. For example, an innovator could be trying to solve a security glitch in an IoT device, and the end result might be enhanced user experience. This situation is exacerbated by the complexities and vastness of data, the variety of IoT devices, and the uncertainty around the sourcing, storage, and sharing of data on the cloud by multiple parties.

Big data technologies (typically the distributed database architecture of Hadoop) and corresponding in-depth statistical analytics (e.g., descriptive, predictive, and prescriptive) create opportunities for innovative business applications. For example, big data analytics holds the promise of reengineering business processes that enable decentralized decision making. This requires new and creative ways of organizing the reporting hierarchies of a decentralized, data-driven business. Another example is the ubiquitous nature of cloud computing (despite certain privacy and security concerns), which has not only resulted in novel approaches within existing businesses, but has also spawned entirely new and innovative business models. Innovating with big data, IoT, and the cloud is challenging because by their very nature these technologies require:

• Due consideration of the security and privacy of data and its usage over the cloud
   
• Drafting and execution of service-level agreements (SLAs) regarding utilization of the cloud
   
• Understanding of the myriad sources of public and private data and a mechanism to use and pay for them
   
• Usability and overall user experience of IoT devices
   
• Upgradability of IoT devices, especially as many IoT devices may not have any contact with their manufacturers after they are released in the market
   
• Changing government rules and regulations around the use of these technologies
   
• Long latency in seeing the results from implementing the aforementioned technologies, as they require coordination amongst multiple organizational and user disciplines (e.g., technologies, business processes, usability, legality, security, privacy)
   
• Higher-than-usual risks due to uncertainty in the markets and dependency on factors beyond the control of the firm
   

Adopting an innovation in a business introduces risks emanating from changes in business practices, business operations, and business culture. Recognizing these risks and planning for the corresponding changes is as important as the innovation itself and vital for risk management. Successful implementations require a full understanding of the business, the domain in which it exists, and continuous improvements during the implementation process. Thus, the process of innovation also needs to keep pace with the way the industry — in particular, the big data–related industry — is going.

Creative Destruction and Disruptive Innovation

There are three types of innovation that an organization can choose to pursue: transformational, incremental, and breakthrough. Each type of innovation has its place in the world of big data, IoT, and the cloud:

1. Transformational innovation dramatically changes the organization and entire industries. Transforma­tional innovation requires organizations to deep-think their core values and offerings. For example, if railroads had understood that their core value was “people movement” (as against the efficient running of trains), then they would own the airlines now. Big data on the cloud spurs transformational innovation as it opens up opportunities for entire industries to collaborate across vast sets of data and generate actionable insights. Organizations like Google and Amazon are part of this new industry that is ­transformational.
    
2. Incremental innovation is meant to enable an organization to exist efficiently and effectively by creating new, small, yet significant improvements to its way of conducting its existing business (e.g., line or brand extensions, new bells and whistles, new packaging, new improved ingredients). For example, it might be said that the tobacco and alcohol industries have innovated more in packaging than the products themselves. Big data technologies are continuously being improved in terms of per­formance and security. For example, the distributed database technologies of Hadoop are evolving to in-memory Spark. Technical innovations help optimize business processes in an incremental manner rather than a dramatic one. Such innovations are multidisciplinary and collaborative and require excellence in managing risks.
    
3. Breakthrough innovation lies in between incremental and transformational innovation. According to the Northwestern Ontario Innovation Centre, “It requires significant change on the part of the innovating organization, both in terms of cultural and systems support.... Breakthrough ideas create new markets and business opportunities that did not exist before.” Big data analytics on the cloud helps foster breakthrough innovations that decentralize traditional, hierarchical organizational structures, as decision making moves to the point of contact with the customer.
    

In technology-based innovation, changes occur at the business and industry levels that can be destructive in a creative sense. In his book Capitalism, Socialism, and Democracy, Joseph Schumpeter coined the term “creative destruction” to denote a “process of industrial mutation that incessantly revolutionizes the economic structure from within, incessantly destroying the old one, incessantly creating a new one.” Once again, the aforementioned examples of Google and Amazon demonstrate the creativity in spinning off an entire industry based on big data analytics and the cloud. In “Technology, Jobs, and Creative Destruction,” Chrissie Deist talks about the power of innovative technologies to eliminate existing jobs and create new ones. For example, the position of “data scientist” did not exist a decade ago, whereas it’s one of the most coveted roles as we write this.

Creative destruction is accompanied by another idea — that of disruptive innovation. This term, conceived by Clayton Christensen, “describes a process by which a product or service takes root initially in simple applications at the bottom of a market and then relentlessly moves up market, eventually displacing established competitors.” This approach to innovation reduces the risks inherent in aiming for an innovative solution that might be too sophisticated, too expensive, and too complicated for many customers in the market.

[For more from the authors on this topic, see “Innovating with Big Data, IoT, and the Cloud.”]