Software strategy matters for digital success
Everything changes. We are in a period of significant shifts in companies—even entire industries—demonstrated in rankings, like the Fortune Global 500. For the last century, these periods of volatility have been driven by a combination of technological change and capital expansion.
There is obvious competition between direct, traditional market segments, but digital disruption also opens up the ability to compete and gain revenue in new areas. For example, a movie streaming service like Netflix also launches a community around the software it created to run its services, or an online retailer like Amazon also innovates with public cloud management. Innovation requires more than a slick customer user interface (UI).
There has to be a foundation of technology, standardized processes, and open culture that allow an organization to be flexible, to build on its existing knowledge, and to incorporate new ideas.
At a strategic level, today’s software is expected to deliver on a multitude of new and different business objectives, from big data, application programming interfaces (APIs), and edge computing initiatives to omnichannel experiences. In addition, software needs to work across multiple business functions, business models, engagement channels, and stakeholder ecosystems—and it needs to do it all at ever faster rates of change and innovation. What companies expect from their software and hardware systems is the ability to adapt to new market realities, realign to capitalize on opportunities, and do all of this without missing a beat in efficiency and uptime. An organization that can change pricing overnight and make new product options available to its global customers and staff rapidly has an enormous advantage over one that requires a three-month rollout with a cascade of manual verification steps.
The ability to integrate applications and data, known as enterprise integration, is key to realizing different business objectives and delivering competitive services. New, and increasingly difficult, demands are being placed on old approaches as digital innovation and disruption become the norm.
Internal business workflows and customer interactions still rely on core systems of record and their supporting IT infrastructure, but delivering solid internal solutions faster has become radically more difficult. New challenges like the increased adoption of cloud applications, hybrid cloud IT environments, and the need to extend systems to reach partners and customers generate demand for modern applications. This makes enterprise integration even more important and delivering services in a faster, continuous way even more critical.
We believe a better way to address these new and rapidly multiplying challenges is to integrate different applications and information systems using agile integration strategies.
What is agile integration, and how does it relate to CI/CD?
The term “agile integration” commonly relates to continuous integration/continuous delivery (CI/ CD), which represent the integration or combination of different development processes into a continuous process.
In this whitepaper, we define agile integration as an architectural approach that specifically applies to “integration” technologies and processes—taking advantage of agile methods and flexible microservices architectures—so that applications and data across multiple systems and services can be integrated and adapted faster to meet the rapidly changing demands of digital business. Agile integration relies on modern platforms, processes, and technologies that are suited for fast-paced and adaptive solutions. Agile integration approaches can help customers include their integration services as part of CI/CD processes.
Why agile integration?
Enterprise service bus (ESB) and other traditional integration technologies provide the essential capabilities (like transformation, routing, orchestration, and connectivity) required to integrate and connect different applications. ESBs coupled with architectural patterns, like service-oriented architecture (SOA) provide a platform to encapsulate integration logic as reusable services. SOA takes advantage of modular business functions and the ability to reuse these services. But it also comes with challenges: technology and governance complexity, time-consuming implementation cycles, and more emphasis on reusability than agility, for example.
For years, enterprises looking to mitigate the problems of monolithic application interoperability tried to make sense of the exponential increase in the number of connections between applications—expressed as x^(x-1), where x represented the number of applications in an enterprise. A solution to this problem was to integrate all applications into a single enterprise service bus. However, the multiple connection problem did not go away and instead, the complexity of those connections was constrained into a single box (the ESB), which could only scale vertically and itself became a monolithic application.
This architecture required centralized governance to control the connections inside the ESB. The goal of reducing complexity by forcing all applications into one central “connection box” failed. This solution reduced the agility of new application development. With a centralized ESB architecture and associated development processes, it became difficult to create, modify, and innovate with new services. This problem has become exponentially more complex with the advent of microservices architectures.
A microservices style architecture provides a more agile approach to developing applications by designing and developing application functionality as independently deployable services. Microservices architectures facilitate building agile business systems—systems that allow a business to change quickly, to build new functionality, to experiment, and to be more prepared to address disruptions.
However, as applications are broken into smaller, discrete services, integration capabilities like transformation, orchestration, and connection are still required. Integration is critical to microservices development, but centralized ESB technologies and deployment architectures do not support microservices architectures and the agile development processes that realize them. A different integration approach is required.
An agile integration approach relies on platforms, processes, and technologies that are more suited for adaptive solutions. With an agile integration approach, integrations can be part of application development processes, including microservices architectures, providing more agility. Integration should be a key capability of distributed teams tasked with delivering new systems of innovative solutions. Combining technological capabilities with different organizational and process approaches makes real change possible.
How to implement agile integration
The 3 key capabilities needed for an agile integration architectural approach are:
- Distributed integration
- Linux containers
Distributed integration— flexibility to adapt
With users increasingly engaging through digital channels (mobile, social, messaging, and web) software has shifted toward a more user-centric model with demand for features and services being pulled from the outside in, rather than being pushed from the inside out. This, combined with the ease of access to user-friendly software tools and cloud services, has created a shift in the role of IT to one that is more about collaboration and enablement of the business, and less about centralized control and gatekeeping.
A further outcome of this is the increased prominence of the line of business in technology strategy and purchases. This creates better alignment of business goals with technology decisions.
As a result of these market and organizational movements, IT’s hold on integration has to loosen into a more modular and distributed model while still adhering to the security and governance that remain core to business requirements. The integration competency centers that were the de facto centers of excellence for best practices for enterprise integration are now evolving to a more distributed integration and business-oriented model. Large integration teams in the IT organization are giving way to smaller and more flexible teams that can respond with greater agility.
In progressing toward a more connected world, agile integration needs to be supported by more modular, lightweight, and pattern-based approaches in order to meet the demand for faster and simpler integration of new services and applications. Changing front-end or client-side requirements demands equal flexibility on back-end integrations. However, lightweight approaches to integration are not always as easy as they sound. Traditional ESBs, usually built on proprietary technologies, are heavyweight and share the downsides of other monolithic applications.
A lightweight, flexible integration platform that enables rapid integration across multiple enterprise systems and services, on-premise or in the cloud, is necessary. An integration platform that enables developers to quickly create lightweight API-based integration services, deploy where required, and scale those services as needed is critical.
Take the example of a big brand retailer that sells products through traditional distribution channels that it largely dominates. As this retailer becomes disrupted by nimble online providers, it needs to adapt and develop its digital channels and do so while keeping the lights on in its traditional business models. It does this by creating small teams. These teams, as Amazon CEO Jeff Bezos famously noted, should be small enough that they can be fed by 2 pizzas—4 to 7 people. Teams this small can respond quickly to the business, creating full end-to-end applications enabled by more lightweight integration platforms and APIs.