One of the things I've noticed about aspiring Enterprise Architects from an application development background is that they tend to drill down into implementation details too soon. It's an understandable inclination. Typically, developers live in a detail-laden world of issues and bugs that can only be remedied by having an in-depth operational understanding of the technology at hand. One bad setting in a configuration file can cause a weekend's worth of grief trying to get a failing system back online.
Operating at the higher levels of abstraction that are part of the architectural design process can be difficult for people who have spent most of their careers achieving success by going "into the weeds." For the application developer, the devil is always in the details. Yet a good deal of the effort that goes into an architectural design requires working at the higher levels abstraction.
Descending down to concrete implementation before addressing higher-level concerns can lead to creating inherently flawed systems.
It's the analogical equivalent to a residential architect being so focused on finding just the right faucets for a client's kitchen that he or she fails to notice that the plot of land where the house is to be built is miles from the nearest water main. When it comes to design considerations, overlooking water access is a far greater problem than choosing the wrong set of faucets. As the analogy shows, problems created at the higher levels of the design process often are costly, or maybe impossible, to fix it later on.
Being able to design according to a certain level of abstraction is an important skill. To do so, you need to have a clear idea of the hierarchy of levels in the architectural design process. The following describes a model that's worked for me over the years.
Let's look at the details.
Understanding the four phases of architecture
As mentioned above, architectural design is not a one-shot deal. Instead, it's a progression through a hierarchy of phases. A model I use defines four design phases, starting from conceptual abstraction and ending with a concrete implementation. These four phases are named as follows:
Figure 1 below is an informal example of applying the four phases to a well-known building, the Eiffel Tower. The tower started as a concept imagined by two engineers, Maurice Koechlin and Émile Nouguier, who worked in the architecture firm of Gustave Eiffel. Eiffel bought the rights to the engineers' plans, and thus the tower came to be named after him.
Eiffel intended the tower to be the tallest structure of its time. To achieve this objective, the designers needed to develop a design that could withstand the significant wind force at high altitudes. The structure was also supposed to be a temporary centerpiece for the 1889 World's Fair in Paris. Thus, it needed to be not only assembled but also disassembled after the fair was over. It became a permanent fixture on the Paris landscape due to the popularity and utility that the structure garnered during the 1889 event and afterward. Still, the thing to remember is that the structure was initially to be taken apart at the end of the World's Fair. The decision to make the structure temporary influenced the logic of how the structure would be built. If the tower were supposed to be permanent, the resulting structure would have been very different.
Once the conceptual requirements for the tower are established, the next step in the architectural process is to create a logical way to move forward. Thus, Eiffel and his team decided to make the structure out of prefabricated components made of puddled iron that are to be assembled in sections and held together with rivets.
The next step is to move on to the structural design. The architects and engineers working on the project accomplished this with an amazing degree of accuracy. They produced 3,629 detailed drawings that described the 18,038 different parts of the structure. Every facet of the tower was accounted for.
The way that the structural drawings were transformed into concrete reality was to prefabricate the tower's sections offsite. The sections were then assembled at the tower's location on the World's Fair grounds at the Champ de Mars on the banks of the river Seine in Paris. Planning was so good that the rivet holes made offsite in the iron sections fit together perfectly when assembled on the main tower. No drilling was done onsite.
Outside of the historically fascinating aspects of making the Eiffel Tower, what's interesting from an architectural point of view is how well the four phases of architectural design map from creating a physical structure such as the Eiffel Tower onto creating enterprise software systems. Just as designing the tower needs to go through the conceptual, logical, structural, and concrete levels in sequence, so too does the pattern apply to enterprise architecture.
Allow me to elaborate.
A real-world example
Let's apply the four phases of architecture development I introduced earlier to a fictitious but realistic example. Figure 2 below shows the four phases of architectural design applied to an enterprise payroll system.
The sections following describe the details of the illustration shown above in Figure 2.
The conceptual level is where the definitions of purpose, scope, and constraints of the intended software architecture are created. For example, according to the illustration in Figure 2 above, the purpose of the intended architecture might be to allow employees in the Finance department of a company to conduct payroll activities for all divisions globally. The scope of the architecture is, as the name indicates, the Finance department, but what is not known is the number of current and projected employees in the Finance department. Neither is the number of employees company-wide known. In addition, the countries which will be subject to the payroll system need to be determined. Also, an important piece of information required is an estimated budget that the company is willing to spend on the architecture.
These items might seem like business considerations, but they're much more. Software designed for 100 users maximum over the system's anticipated lifetime is much different from one that needs to support 5,000 users. The same goes for the number of employees subjected to the system. A company that needs to handle payroll for 5,000 employees is different than a company that needs to run payroll for 50,000. Of course, the company might want a system to handle infinite scale over current and yet unknown geography. Achieving this goal is possible as long as the budget is there to support such an ambitious undertaking. If the funds are not available, the project's ambitions are moot. Every aspect of the conceptual definition of the intended architecture matters.
The long and the short of it is that decisions made in the conceptual level affect every aspect of the intended architecture moving forward.
The logical level of architectural design is where the purpose, scope, and constraints defined in the conceptual level are translated into a set of logical entities that can be converted into actual working software. Typically, enterprise architects use a combination of written descriptions and a diagramming language such as Unified Modeling Language (UML) to express the system's logical design. (See Figure 3, below)
The scope of logical design needs to be detailed enough that technical staff can be divided into distinct workgroups that focus on realizing the defined logical components, yet general enough that changes in the high-level design have minimal impact. An example of appropriate scope is changing the Banking and Human Resource components from a direct association to the Payroll System, as shown in Version 1 of Figure 4 below, to the Geography component subsystems, as shown in Version 2.
Once the logical phase has been completed, work begins in the structural phase.
The structural level is the place where the architectural design begins to take shape physically. The purpose of the structural phase is to create the system's physical blueprints. For example, this phase defines the data structures, protocols, and context boundaries associated with each component made earlier in the logical phase. High-level programming guidelines on a per-component basis are determined, too. For instance, logical components might be well-suited to using Object Oriented Programming or Functional Programming, while data storage components would use a SQL-based language.
In addition, the structural phase is where media support is identified. This phase is particularly important as voice and touch-screen activity continue to grow as a means of data generation and data exchange. (See Figure 5, below)
As stated above, the structural phase's work output provides the level of detail found in a blueprint. The blueprints produced in the structural phase describe precisely what is to be built and, to some degree, how the architecture is to be built. However, the final work of realizing the architecture is done in the concrete phase of implementation.
The concrete level is where the specific products, technologies, frameworks, and services necessary to realize the architectural design are identified and implemented. (See Figure 6, below)
Things get very specific in the concrete phase. The scope of the decisions made vary in impact. For example, choosing between Docker Swarm or Kubernetes as the container orchestration technology has broad implications throughout the architectural implementation. Choosing the programming language or database technology to use in a particular component has a narrower scope of impact. The trick is to organize decision-making about specific technologies according to the scope of influence.
Those decisions about products, services, and frameworks that affect the entire architecture are typically driven by consensus among senior engineering staff on a cross-functional basis. However, decisions that relate to the implementation of a specific component's internals, such as which programming language best suits the use case requirement defined in the structural phase previous, can be made by the particular development team.
Regardless of the scope of impact, whether it's system-wide or confined to the internals of a given component, the devil really is in the details. Thus, such decisions are best made by those with hands-on expertise. However, their work cannot be done in isolation. At the least, all decisions made at the concrete level need to be reviewed by the higher-level architects. It's a process very similar to building construction. Architects make the design, and construction crews implement those designs. However, all construction activity is subjected to rigorous inspection and subsequent certification by qualified personnel. Building standards must be met. Safety must be assured. Software architecture is but a newcomer to a practice that's been going on for hundreds of years.
Putting it all together
Making enterprise software requires structure and discipline. Small startups that employ a few dozen people can enjoy an improvisational approach to architectural design. However, large companies that employ thousands of people on a project require more business operations and enterprise software design organization. You can't put a person on the moon and bring him or her back again on a whim. Coordinating the efforts of thousands of people is very hard work.
The four design phases is but one of many approaches to creating software for very large projects. It might not be for you. There are a lot of schools of design and design methodology for Enterprise Architecture. The critical thing to understand is they all separate work into different phases of design activity. Each phase has distinct concerns and well-defined work products. None advocate vacillating up and down between levels. Being able to work within the boundaries of a particular level is a sign of professional discipline. Few people can do it right off the bat. It's a skill that, once acquired, ensures a design sensibility that delivers software that meets expectations on time and on budget.