Agile Data Logo

The Agile Data Architect

Follow @scottwambler on Twitter!

An agile data architect is someone who guides the development and support of the data-oriented aspects of something in a collaborative and evolutionary (iterative and incremental) manner. "Something" may be a component, a solution, a product line, or even your entire enterprise.

This article is organized into the following topics:

  1. The agile data architect role
  2. The responsibilities of an agile data architect
  3. Becoming an agile data architect
  4. Recommended resources

1. The Agile Data Architect Role

Agile data architects are considered specialists, or more accurately specialized architects. This can be a good stepping stone towards a more senior role, such as an architecture owner on an agile team or an enterprise architect. Or you may choose to remain specialized, it's really up to you. Having said that, as I indicate below agile data architects still need to work towards becoming a generalizing specialist in that minimally they will need agile database development skills.

Depending on need, agile data architects may focus on one or more architectural levels:

For more information about these architecture levels, see Agile Data Architecture Context.

2. The Responsibilities of an Agile Data Architect

There are effectively four key activities performed by agile data architects:

  1. Provide help to others. Agile data architects spend the majority of their time collaborating with their stakeholders to either apply, create, or evolve data architecture assets.
  2. Agile architectural modeling. Agile data architects will apply Agile Modeling (AM) strategies to explore and capture data concerns. More on this below
  3. Explore complex architectural issues. Sometimes you just don't know how something will work in practice. Instead of just guessing, you can choose to reduce the risk of unknowns by making them known. Two common techniques for doing so are to build architecture spikes or run proof of concept (PoC) initiatives.
  4. Invest in learning. An important responsibility for anyone in a senior role, including architects, is to share their skills and knowledge with others. And of course you should invest in your own learning.

I explore these activities in greater detail in the article The Agile Data Architecture Process.

3. Becoming An Agile Data Architect

Agile data architects need:

  1. To be generalizing specialists
  2. Agile modeling skills
  3. Data implementation skills

3.1 Agile Data Architects Need to be Generalizing Specialists

A generalizing specialist is someone who:

People who are generalizing specialists are in the "sweet spot" between the two extremes of being either just a specialist or just a generalist, enjoying the benefits of both but not suffering from their drawbacks.

But what does this mean for an agile data architect? As I indicated early, data architects are certainly an example of a specialized architect, are are security architects, user experience (UX) architects, business architects, and others. Although specialized architects can prove to be valuable, particularly when you face a very difficult problem, they can also prove to be detrimental in practice due to their tendency to locally optimize around their specialty. As an agile data architect, you usually need to be more than just a specialized data architect.

Figure 1 depicts representative skill ranges for various architectural roles, including developers. These are examples, not prescriptions, as everyone is different. It is the general pattern that is important. You see that a solution architect has a wide range of skills. This person has deep skills in a narrow specialty, let's assume data, from the implementation level all the way up to the enterprise level. They have broad skills applicable to solution architecture, hence their current role, but also deep skills in some aspects of product line, component architecture, and even implementation. They are well suited to be a solution architect given their broad skills in that, and their skills at other levels enables them to have more empathy for people working at those levels and to better interact with them as a result. Notice however that they don't have solid skills in all aspects of solution architecture, they have more to learn (as does everyone else). They're not just a specialized architect, however, someone with very deep skills in one area but weak in many others.

Figure 1. The agile architecture skillset (click to enlarge).

Agile Architecture Skillset

3.2 Agile Data Architects Need Agile Modeling Skills

Agile Modeling (AM) is a practice-based methodology for effective modeling and documentation. Agile modeling is an evolutionary (iterative and incremental) approach to creating models and supporting documents that is highly collaborative in nature. You can view Agile Modeling as a collection of core practices, as you see in Figure 2. These practices are applicable to all modeling domains, including both data and architecture.

Figure 2. The core practices of Agile Modeling (click on each practice for details).

Key AM concepts for data architects are:

3.3 Agile Data Architects Need Implementation Skills

Agile data architects are also familiar with, and ideally adept at, the agile database techniques stack overviewed in Figure 3 below. These are the critical skills required of anyone involved in the implementation aspects of an agile data initiative.

Figure 3. The agile database techniques stack.

Agile Database Techniques Stack

Because agile data architects are expected to be generalizing specialists, and with the exception of industry data architects having at least rudimentary implementation skills, they should be knowledgeable about the following techniques:

  1. Vertical slicing. Vertical slicing organizes functionality vertically into small, consumable pieces that may be potentially deployed into production quickly. These vertical slices are completely implemented - the analysis, design, programming, and testing are complete - and offer real business value to stakeholders.
  2. Clean data architecture and clean data design. A clean data architecture strategy enables you to develop and evolve your data assets at a pace which safely and effectively supports your organization - in short, to be agile. Similarly, a clean data design
  3. enables you to evolve specific data assets in an agile manner.
  4. Agile data modeling. With an evolutionary approach to data modeling you model the data aspects of a system iteratively and incrementally. With an agile data modeling approach you do so in a highly collaborative and streamlined manner.
  5. Database refactoring. A database refactoring is a small change to your database schema which improves its design without changing its semantics, enabling you to safely improve the quality of your data sources.
  6. Automated database regression testing. You should ensure that your database schema actually meets the requirements for it, and the best way to do that is via automated regression tests for your database schema to ensure data quality.
  7. Continuous database integration (CDI). Continuous integration (CI) is the automatic invocation of the build process of a system. As the name implies, continuous database integration (CDI) is the database version of CI.
  8. Configuration management. Your data models, database tests, test data, and so on are important artifacts which should be put under configuration management just like any other artifact.

4. Related Resources