Many organizations are opting to go with a commercial IoT platform provider in order to benefit from the convenience that using such offerings provide. The biggest reason is to take advantage of the flexibility, dynamic scalability, and performance offered by cloud-based architectures and services, including the capabilities for publishing APIs and Web services to facilitate the exchange and integration of machine data with enterprise data and to develop analytic capabilities that can be easily consumed by enterprise applications.

In Part I of this series, I explored the data lake and other metaphors for data storage in use today. Leaving technology platforms aside (which I strongly urge you to do, at least for now), understanding what a data lake could or should be starts, unsurprisingly, with knowing what the business needs from it and that it cannot get anywhere else. That understanding comes from a conceptual architecture, which therefore must have a much broader scope than its central topic. A conceptual architecture is a picture that forms the basis for conversation, understanding, and agreement between business and IT. It doesn't have enough detail for IT to build it. It must be simple enough for business people to take it in and understand what's going on.

No enterprise big or small should fail to think about what digital transformation can do for — or to — them. Within commonsense economic limits, current technologies should be exploited. Further, it's important to envision potential technological changes a few years out and think through how to prepare for and exploit them. Obviously, that's not easy.

In his keynote address on NoSQL key value stores at the 2014 Velocity Conference, Brian Bulkowski discussed "in-memory" mechanisms and how they can be handy in electronic advertisements and similar situations. He mentioned during the keynote that "in-memory key value is Agile" is significant. This very thought opens up opportunities to consider the technologies of big data as enablers of business agility.

Emerging technologies and digital disruption will transform the enterprise, but they will also transform the ways in which we architect. What will this mean for enterprise architecture in general and for the role of the enterprise architect? How will EA help enterprises to collaborate with one another? What will these changes mean for the nature of the enterprise and its architecture? In this issue of Cutter IT Journal, our authors provide their practical insights and guidance on disruption and emergence and what they mean for EA.

To achieve pervasive, sophisticated, wide, deep, joined-up architecture descriptions, we need to develop a new generation of EA tools and techniques. The fact is that in an enterprise of any complexity, the architecture of roles, processes, applications, flow of work and data, data stores, and data sources quickly exceeds the ability of any of us to hold in our minds effectively. We need powerful ways to capture, map, navigate, and trace linkages, interfaces, and change initiatives.

I have mixed feelings about the FBI/Apple iPhone standoff. On the one hand, due to all the data collection and spying activities revealed by former US National Security Administration (NSA) contractor Edward J. Snowden, it seems like Apple has a pretty good argument for not wanting to comply immediately with a federal court order demanding that the company develop a specialized version of its iPhone operating system (iOS) that would enable the FBI to unlock the on-device security features of the iPhone belonging to one of the San Bernardino, California, terrorists.

Perhaps the best example of an adaptive and integrated field of knowledge is IT. Yet, it is quite shocking to see how many companies fail to invest a meaningful amount of time and resources to thinking actively about the future, with teams based in transdisciplinary collaboration.

In the days of large waterfall projects, organizations made the assumption that a software budget was allocated one-third per major category: analysis and design, develop, test. This was the rule of thumb that was used to generate a high-level estimate (HLE). This rule of thumb was great when it was applied evenly to all three categories. However, what usually happened on software projects is that the first two categories needed more time and it inevitably came at the expense of testing in an effort to stay on budget.

We are rapidly moving to a world where individuals don’t switch off their technologies, and companies can’t switch off their technologies. Head-up displays, image recognition, wearable technologies, virtual reality, a revolution in manufacturing technologies, the Internet of Things (IoT), super-dense computer memory … the list goes on and on! But what does this mean for the future of enterprise architecture (EA) — as a discipline, as a process, and as it informs the nature of an enterprise?

API-based development, with its focus on functionality rather than user interface, is a way for organizations to future-proof their investments in software. Requirements for functionality and interfaces keep changing more and more quickly, but, largely, what is not changing is the basic analysis and requirements that run the project. By outsourcing the interface to interested parties through your API, you can broaden the reach of your project (just as companies like Facebook, eBay, and Google have done).

Whether it is data warehouses or marts, data lakes, or reservoirs, the IT industry has a penchant for metaphor. The subliminal images conjured in the human mind by the above terms are, in my opinion, of critical importance in guiding thinking about the fundamental meanings and architectures of these constructs. Thus, a data warehouse is a large, cavernous, but well-organized location for gathering and storing data prior to its final use and a place where consumers are less than welcome for fear of being knocked down by a forklift truck. A data mart, on the other hand, creates an image of something between your friendly corner store and Walmart.

Business continuity planning, information security governance, and IT governance are critical activities in managing the operations of today’s organizations. Traditionally, these areas have been handled as separate and distinct activities. Effective organizations can no longer afford to manage these activities separately if they wish to streamline planning, guarantee adequate response to business-impacting events, and control costs.

It's time we step back from benchmarking specific topics and industries to consider the process of performance measurement and benchmarking itself. How should you use benchmarking information? What information should you gather about your company's own internal operations? In general, how should you gather and use performance data? It's important to look inward at your processes, your own ways of thinking, to make sure they still make sense and that you haven't developed any bad habits.

When people use mobile apps they seldom think about the security implications. They sign up at an app store and agree to let the app have access to inbuilt sensors and communications capabilities; they also permit access to resources that may include personal data on the Web, corporate data and corporate applications, and/or network access to Web resources that might not be secure. Vulnerabilities can appear at any stage of the interaction between the mobile application and its data.

The enterprise players' IoT platforms are, for the most part, comprehensive IoT implementation environments providing platform-as-a-service (PaaS) capabilities. However, in addition to supporting the infrastructure requirements necessary for building, connecting, and managing IoT connected products and applications, they are also designed to integrate with, and take advantage of, the back-end, infrastructure, communications, process management/workflow, and analytics capabilities provided by the various business components of their respective vendors' ERP, CRM, BI, cloud, industrial control, and other enterprise software offerings.

Social business analytics is the most complex form of social media analysis because it involves analyzing unstructured social data in combination with structured data and other content maintained in enterprise sources. This requires an infrastructure for sourcing, managing, and analyzing social and enterprise data, and for integrating the findings back into the organization's enterprise data analysis and decision-support processes.

All friction is suicidal because it is your energy being wasted unnecessarily. We don’t have that much energy to waste in fighting with ourselves.

— Zen Master Osho

There is a new line separating human beings from each other and it is a great technocratic divide. Calling the reaction against this technocratic invasion a Luddite one doesn’t capture the significance of this scientific tsunami. All roads to the future lead through a forest of computing and scientific complexity. There are fewer mentally easy routes to economic prosperity. How the next generation sorts out their place in the 21st-century economy will be quite different than in the 20th-century economy.

The series of articles in the September 2015 issue of the Cutter IT Journal (“The Corporate Impact of Wearable Devices”) stimulated some reflection on the different perspectives taken by the authors. All of these articles, in different ways, discuss the impact of wearables on our relationship with technology.

The key point is that pace is relative — it is likely to be comparatively fast or comparatively slow, but there will always be some EA environments with a mixture of both fast and slow, and some that fluctuate between the two extremes.

When we talk about wearables, most of us have one or two specific devices in mind that we use to add tangibility to our thinking.

Disruptive innovation, a well-known business concept defined by Clayton Christensen, is changing. When he first defined this concept back in 1997, digital technologies already existed, but they were just beginning to make their impact on strategy and the process of disruption.

If you look closely at Agile, it is actually a huge advocate of building in quality and calls for everyone on the team to own quality. Agile/Scrum calls on the product owner to produce clear stories and acceptance criteria, the dev team to test their code, testing staff to be involved with the dev team from the start, and of course have customer involvement whenever possible.

Being true to architecture’s roots in business does a couple of things: it ensures that we stay grounded in things that matter, and it provides a framework of values that guides and validates everything that we do in the name of architecture.