Agile Data Architecture in Context

Many data professionals like to say that data is the life blood of your organization, and it is. Keeping the living organism metaphor going, there is also a skeleton, organs, muscles, fat, skin, hair, and more. When there is only life blood, then all you have is a crime scene. Data architecture is one of many aspects of an architecture, be it for a single system or of your entire enterprise. Agile data architecture is the act of defining and evolving data architecture in a collaborative, iterative, and incremental manner.

To put data architecture into context, we need to consider the following issues:

1. Agile Data Architecture: Scope

A given data architecture will address one or more levels of scope, overviewed in Figure 1. You can see that Figure 1 isn’t a clean stack diagram, nor is it an “onion diagram” where each layer is fully encompassed by the next one. For example, you see that component architecture is not completely encompassed by solution architecture. That is because a component may be part of a product line, or may be a reusable enterprise asset used by many solutions. Similarly, some aspects of your enterprise may be a subset of one or more industry architectures and some aspects of your enterprise may not be addressed by industry architectures at all.

Figure 1. Architectural scope (click to enlarge).

Data architecture at each of these levels focuses on the data issues pertinent to that level. These levels are:

Component. Component architecture addresses a portion of a solution, such as a data source, a micro-service, an application programming interface (API), or similar. Component architecture tends to be narrowly focused, and may not address the full range of breadth described in the next section. Component data architecture tends to focus on the physical data aspects of the implementation with mapping/traceability to applicable enterprise concepts.
Solution. Solution architecture – sometimes called system, application, or service architecture – addresses an offering (product or service) provided to your stakeholders. Solution architecture shows how various components fit together.
Product line. Product line architecture, sometimes called product family architecture, addresses a collection of related solutions and supporting components, showing how they fit together.
Enterprise. Enterprise architecture, sometimes called corporate architecture, addresses the overall architecture of your organization. Enterprise architecture explores how the solutions and product lines offered by your organization fit together and how they relate to any relevant industry architectures.
Industry. Industry architecture addresses common aspects of a business domain (such as insurance, automotive, or aviation) or a technical domain (such as security, connectivity, or data transport). They are often used as the basis for data transport between collaborating organizations.

Although I indicated that a given architecture addresses one or more levels of scope, my advice is to address only one level at a time. This enables you to focus your thinking, increases the chance that your work will be understood by others, and makes the architecture easier to maintain over time.

2. Agile Data Architecture: Breadth

One of the common characteristics of architecture frameworks – such as the Zachman Framework (ZF), The Open Group Framework (TOGAF), or the work of the Business Architecture Guild – are that they support multiple architectural viewpoints. Figure 2 overviews the ZF, with the What column focusing on data-oriented architecture, design, and implementation issues and the other 5 columns addressing other architectural viewpoints. Figure 3 overviews the architectural aspects addressed by TOGAF. Although these frameworks capture different approaches, the point is that data architecture proves to be a subset of the overall architecture strategy.

Figure 2. The Extended Zachman Framework (click to enlarge).

Figure 3. Viewpoints (aspects) of TOGAF (click to enlarge).

I cannot say this enough: Data architecture is a subset, or viewpoint, of your overall architecture. Other potential viewpoints include:

Business
Capabilities
Communications
Events
Process
Security
Technology
User experience (UX)
And others

When you focus on one architectural viewpoint to the exclusion of others you run the risk of local optimization. Your work may be well tuned tor that given viewpoint, but will be ineffective to the point of being damaging for other viewpoints. Agile architects need to find the combination of trade-offs that work best for the context that they face, and to do that they must take a broad, multi-viewpoint approach.

3. Agile Data Architecture: Process

What do agile architects, or more pertinently, agile data architects, do? Figure 4 depicts a flowchart overviewing how to choose between common activities required for successful data architects.

Figure 4. The agile architecture process (click to enlarge).

There are effectively four key activities performed by agile data architects:

Provide help to others. Agile data architects spend the majority of their time collaborating with their stakeholders to either apply, create, or evolve data architecture assets.
Agile architectural modeling. Agile data architects will apply Agile Modeling (AM) strategies to explore and capture data concerns.
Explore complex architectural issues. Sometimes you just don’t know how something will work in practice. Instead of just guessing, you can choose to reduce the risk of unknowns by making them known. Two common techniques for doing so are to build architecture spikes or run proof of concept (PoC) initiatives.
Invest in learning. An important responsibility for anyone in a senior role, including architects, is to share their skills and knowledge with others. And of course you should invest in your own learning.

I explore these activities in greater detail in the article The Agile Data Architecture Process

Agile Data Architecture in Context

1. Agile Data Architecture: Scope

2. Agile Data Architecture: Breadth

3. Agile Data Architecture: Process

4. Related Resources

Recommended Reading