CUTTER IT JOURNAL VOL. 29, NO. 6

Beyond Keywords

The term “big data” encompasses a wide variety of ­topics led by the two keywords “analytics” and “technologies.” Technically, big data implies Hadoop/HDFS, Spark, and, at the back end, NoSQL. From a business viewpoint, however, big data analytics command greater interest as they enable identification of patterns, facilitate predictions, and also provide prescriptive advice for better decision making.

When medium to large enterprises want to adopt big data, they need to go through the rigors of large-scale adoption through people, processes, and technol­ogies. For example, Guest Editor Barry Devlin’s 4 Ps (Preparation, People, Prediction, and Production) provide one such basis for analytics adoption. However, analytics need to be coupled with a proper understanding of technological capabilities provided by enterprise architectures. We thus find that multiple technical, ­analytical, and architectural elements come into play in big data adoption.

This article argues for an overarching framework that will not only facilitate adoption of analytics and technologies, but will also provide a solid foundation for taking a strategic approach to big data. This framework is called the Big Data Framework for Agile Business (BDFAB v1.5), and its development is based on a review of the relevant literature, experimentation, and practical application. The key elements composing this framework are:

• Agile values for business
   
• Organizational roles in big data
   
• Building blocks of big data strategies for busi­ness (including the role of analytics within those strategies)
   
• Key artifacts in big data adoption
   
• Business conditions and limitations
   
• Agile practices
   
• A compendium (repository)
   

The building blocks of big data strategies are themselves are made up of five modules:

1. Business decisions
    
2. Data: technology and analytics
    
3. User experience: operational excellence
    
4. Quality dimensions
    
5. People (capabilities)
    

In addition, this framework is accompanied by a 12-lane process for big data transformation, especially in large organizations. Exploration of BDFAB will be of practical benefit to organizations looking for a sensible pathway into big data. At the same time, the framework provides opportunity for refinement based on further experimentation.

BDFAB Overview

As noted above, BDFAB is a research-based framework that facilitates a strategic approach to the application of big data to business. Most contemporary big data approaches focus either on the Hadoop ecosystem (as a suite of technologies, programming, and management) or on the analytics (based around extensive statistical techniques such as predictive analytics, net promoter score [NPS], and so on). This represents a significant lacuna in the big data space, which requires a comprehensive and holistic approach to formulating a business strategy and synergizing the aforementioned technical and analytical elements. This lack is also felt in the Agile space, which predominantly constitutes a methodical approach to solutions development.

Elsewhere I have argued that Agile needs to transcend the solutions space and move into business strategy. The business technology domain finds itself awash in data and technology that can potentially be used to render a business Agile. Such strategic utilization of data requires deeper understanding of the current state of the business, its directions, and its capabilities (both architectural and people), as well as dynamic, smarter risk analysis. As the volume of data grows, the role of information architecture is changing from the passive structuring and managing of data to a smarter, more active role in ensuring effective use of information.

BDFAB builds on the technical and analytical aspects of big data in a holistic manner to understand and create new opportunities for business agility. Figure 1 highlights BDFAB’s core philosophy of bringing together analytics and technologies but then going beyond them into the realms of agility and business strategy.
 

Wedding Big Data and Agility

In his foreword to Big Data Analytics, IBM Fellow and Chief Scientist Jeff Jonas observes how big data maps to agility in business [emphasis mine]:

• “Organizations must be able to sense and respond to transactions happening now.” (Agility is the ability to spot the changes coming through — which are transactions at both the micro and macro levels.)
   
• They also “must be able to deeply reflect on what has been observed — this deep reflection is a necessary activity to discover relevant weak signal and emerging patterns.” (Agility requires the ability to take effective decisions; this effectiveness results from deep reflection, aided and impacted by big data ­analytics.)
   
• “As the feedback loop gets faster and tighter, it sig­nificantly enhances the discovery [from deep reflection].” (Agility requires rapid response, which in turn is based on analytical insights and leanness of organizational structure.)
   

Businesses can be helped to tap into Agile opportunities (ranging from expansion into new markets to enhanc­ing customer satisfaction and/or optimizing internal business processes) by incorporating vital elements of Agile values, principles, and practices in big data adoption. The translation of these values from the depths of software development to business processes is the result of combining the formality of planned approaches and the flexibility of Agile, as in the Composite Agile Method and Strategy (CAMS).

BDFAB incorporates agility in a strategic business context with the understanding that Agile has transcended software development and now plays a major role in business organizations. Agile is therefore a legitimate business goal in its own right, and a strategic approach to big data can go a long way in achieving that goal. Such an approach aims to make use of structured, semistructured, and unstructured data, and the velocity and volume of such data to generate an ongoing and significant amount of business intelligence to enable improved business decision making.

Core Elements of BDFAB

Table 1 summarizes the core elements of BDFAB. The table also shows examples of BDFAB’s core elements as well as key business considerations in utilizing a ­particular core element.

BDFAB starts with the business organization itself — its strengths, weaknesses, opportunities, and threats (SWOT) — and then moves into clarifying the vision and the capabilities the organization needs to satisfy that vision. Helping the business identify and exploit the existing and growing data capabilities (technologies and analytics) with a continuous focus on business decision making results in relevant insights. The output of this activity enables decision makers to either modify or develop new products and services, respond to individual customer issues, and rapidly change the business processes.

BDFAB helps position the technologies of Hadoop/MapReduce/Spark in a way that will assist in enhancing business agility. The decision makers need to see the correlation between big data (and its analytics) and making the business “Agile” as a result of those analytics. For example, online sellers (e.g., Amazon) use statistical analysis of hourly sales of books to recommend additional titles to readers. Airlines use hourly flight booking data to mark up (or down) their fares in a very Agile manner. Analytics thus continue to apply statis­tical techniques to generate rapid insights. Big data technologies, based around the Hadoop ecosystem (including HDFS, NoSQL, and MapReduce), support such rapid analytics by storing, sharing, and processing vast amounts of structured and unstructured data.

Furthermore, the analytics themselves are no longer ­static; they are themselves changing depending on the circumstances of an individual customer and/or the context in which the business finds itself (e.g., political uncertainty, changing legal structure, global collaborations). Thus, not only do the analytic processes support business agility, these analytical processes themselves need to be agile (or, in other words, continuously changing). BDFAB uplifts the capabilities of decision makers on an ongoing basis, resulting in business agility in a unique and holistic way.

The key innovative aspect of BDFAB is its focus on business strategies resulting from a balanced combination of big data technologies and analytics together with the concepts of composite Agile. This incorporation of composite Agile (CAMS) in BDFAB is based on the premise that Agile has transcended software development and now plays a major role in processes associated with the business.

BDFAB Modules

While analytics — including OLAP cubes, text and datamining, and dashboards — all add to and aid in decision making, what is even more interesting is the strategy for putting this whole process together. How does one get an organization to reach a stage where these analytics and the decision making they enable become the norm? To achieve that goal, BDFAB comprises five major modules, as summarized in Table 2.

Business Decisions

This module introduces the concept of big data to the business and starts positioning big data as a basis for business strategies. Therefore, this module focuses on the strategic/business value of big data analytics. It begins with a SWOT analysis and moves into the risks, cost advantages, and adoption approaches to big data. ROI (and cost-benefit analysis) for big data adoption forms part of this module. Agility is introduced to the organization as a business value (transcending Agile used in software development projects).

Data: Technology and Analytics

The second module focuses on data analytics, mapping volume, variety, and velocity with structured, unstructured, and semistructured data types. Each of these characteristics of big data is invaluable in supporting corresponding business strategies — if properly for­mulated. This module demonstrates the interplay between analysis of data and its impact on creating business strategies. This module helps the business understand and incorporate structured, semistructured and unstructured data in its analytics and decision-­making processes. It is underpinned by the technologies of Hadoop and the associated technical ecosystem.

User Experience: Operational Excellence

This module explains how data analytics can render a business Agile. Understanding customer (user) sentiments through a user experience analysis framework (UXAF) is the starting point for this work. Most UXAFs focus on time T0 to time T1 — when the user is in direct contact with the business through its systems and interfaces. Substantial data is generated, however, by user interactions with social and mobile networks that occur before T0 and after T1. Exploring the gen­eration and use of this data (based around the SMAC stack) is part of the discussion in this module. The “predictive” and “prescriptive” nature of ensuing ­analytics is discussed here.

Quality Dimensions and the SMAC Stack

Quality considerations in the big data domain assume prominence because of the direct impact they have on business decision making. This module focuses on this crucial quality aspect in big data solutions: data, information, analytics (intelligence), processes, usability, and reliability. Uniqueness of unstructured data and what can be done to enhance and validate its quality are part of this discussion. The challenges of contemporary testing (and the role of Agile practices, such as continuous testing) together with their application to big data are also explained and implemented in business through this module.

People (Capability)

Cutter Senior Consultant Larissa Moss and data ­strategies expert Sid Adelman have discussed the ever-growing importance of people and their capabilities in the big data space. Similarly, a McKinsey report on big data goes into the details of existing and needed capabilities in the big data technologies and analytics domains. BDFAB incorporates this important people issue in big data adoption by identifying and enhanc­ing the people capabilities at both the technical and ­analytical levels. The Skills Framework for the Information Age (SFIA) is used as a backdrop for ascertaining current skill levels and mapping an up-skilling path for the human resources in organizations adopting big data. This module can lead to formation of centers of excellence around big data and related disciplines.

BDFAB in Practice

When an enterprise adopts big data using the BDFAB, it reduces its risks and gains the following practical advantages.

Creating a Business Advantage

BDFAB is meant to help a business adopt big data in a way that will result in business agility. The advantage of this framework results from bringing together two important concepts of modern-day technology and ­business — agility and big data. This synergy demonstrates the value of analytics in rapid business decision making.

CAMS embodies Agile characteristics (e.g., flexible, change-welcoming, iterative, collaborative, and ready to fail fast) that provide value to business.¹ BDFAB expands on these concepts to provide the business with the values of Insight, Collaboration, Dynamicity, Leanness, Governance, and Environment (i.e., sustainability).

Adoption Process

A 12-lane adoption process (see Figure 2) is a crucial part of BDFAB. This detailed adoption process provides guidelines in terms of which aspects of agility and big data should be adopted first, how the adoption should iterate, and how to identify and overcome blockers. It offers a basis for risk reduction in big data adoption.

Risk Analysis

BDFAB addresses risk in two ways. First, and most important, is the embedding of business risk consid­erations in the framework itself. BDFAB starts with a SWOT analysis that enables assessment of early business decisions relating to big data. The adopting orga­nization considers the business priorities, risks, and budgets in detail when undertaking this risk analysis. Furthermore, BDFAB provides multiple angles to big data adoption (including technologies, analytics, user experience, and people up-skilling), which results in risk reduction. Risk analysis also includes due consideration of the needs of different types and sizes of businesses. For example, a medium-sized travel business may be more interested in cloud-based analytics and not much up-skilling of staff, whereas a large bank will be interested in both cloud and people up-skilling. Having Agile concepts and values embedded in the framework is helpful in reducing business risks thanks to the rapidly iterative nature of agility.

Consumer Dialogue

Consumer dialogue and user experience are integral parts of BDFAB. The second module of BDFAB focuses on the user experience to ensure that it is included as a crucial element of the organization’s big data strategy. For example, NPS statistical analysis provides a good basis for understanding what the consumer wants and embedding that in the business processes of the organization. Similarly, in optimizing internal business processes with big data–driven insights (e.g., anticipated production levels in a manufacturing plant, ­medical inventories in a hospital), BDFAB ensures ­continuous focus on the ongoing consumer dialogue.

Conclusion

In this article, I have presented BDFAB, a framework for adoption of big data by business. This comprehensive framework can help ensure that the end result of big data adoption is Agile business. BDFAB is unique in the sense that it elevates the current industry focus from technologies and analytics to business strategies. In due course, the framework will need to be accompanied by a corresponding process tool that will facilitate big data adoption in large organizations. Such a tool will not only help formulate organization-specific strategy, but also enable monitoring, tracking, reporting on, and optimizing the process of big data adoption for business agility.