An agile data architect is someone who guides the development and support of the data-oriented aspects of something
in a collaborative and evolutionary (iterative and incremental) manner. "Something" may be a component,
a solution, a product line, or even your entire enterprise.
This article is organized into the following topics:
- The agile data architect role
- The responsibilities of an agile data architect
- Becoming an agile data architect
- Recommended resources
1. The Agile Data Architect Role
Agile data architects are considered specialists, or more accurately specialized architects.
This can be a good stepping stone towards a more senior role, such as an
architecture owner
on an agile team or an enterprise architect. Or you may choose to remain specialized, it's really up to you.
Having said that, as I indicate below agile data architects still need to work towards becoming a
generalizing specialist
in that minimally they will need agile database development skills.
Depending on need, agile data architects may focus on one or more architectural levels:
- Component.
A component data architect will guide the development and support of the data aspects of a component,
such as a microservice or API. If the component is data intense, such as a collection of data services,
they may in effect be the architecture owner for that component.
- Solution.
A solution data architect will guide the development and support of the data aspects of a solution,
such as an application or system. If the solution is data intense, such as a data warehouse, they
may in effect be the architecture owner for the solution. In this case, they are likely to need
broader architectural skills than just data, for example security and user experience (UX) skills
for a
data warehouse/business intelligence solution.
- Product line.
A product line data architect will guide the development and support of the data aspects of a product line,
which is a collection of related solutions and services. Product line data architects will need to work
closely with the architecture owner for the product line as well as with the data architects (if any) on
teams working on the solutions and services
- Enterprise.
A enterprise data architect will guide the development and support of the data aspects of an entire organization,
working very closely with your enterprise architects to ensure that their work fits into the overall direction
of your enterprise.
For more information about these architecture levels, see
Agile Data Architecture Context.
2. The Responsibilities of an Agile Data Architect
There are effectively four key activities performed by agile data architects:
- Provide help to others.
Agile data architects spend the majority of their time collaborating with their
stakeholders to either apply, create, or evolve data architecture assets.
- Agile architectural modeling.
Agile data architects will apply
Agile Modeling (AM)
strategies to explore and capture data concerns. More on this below
- Explore complex architectural issues.
Sometimes you just don't know how something will work in practice.
Instead of just guessing, you can choose to reduce the risk of unknowns by making them known.
Two common techniques for doing so are to build architecture spikes or run proof of concept (PoC) initiatives.
- Invest in learning.
An important responsibility for anyone in a senior role,
including architects, is to share their skills and knowledge with others.
And of course you should invest in your own learning.
I explore these activities in greater detail in the article
The Agile Data Architecture Process.
3. Becoming An Agile Data Architect
Agile data architects need:
- To be generalizing specialists
- Agile modeling skills
- Data implementation skills
3.1 Agile Data Architects Need to be Generalizing Specialists
A generalizing specialist
is someone who:
- Has one or more specialized skills (e.g. User interface design, Project Management, Security Engineering, Marketing, ...).
- Has at least a general knowledge of software development.
- Has at least a general knowledge of the business domain in which they work.
- Actively seeks to gain new skills in both their existing specialties as well as in other areas.
People who are generalizing specialists are in the "sweet spot" between the two extremes
of being either just a specialist or just a generalist,
enjoying the benefits of both but not suffering from their drawbacks.
But what does this mean for an agile data architect? As I indicated early, data architects
are certainly an example of a specialized architect, are are security architects, user
experience (UX) architects, business architects, and others. Although specialized architects
can prove to be valuable, particularly when you face a very difficult problem, they can
also prove to be detrimental in practice due to their tendency to locally optimize around their
specialty. As an agile data architect, you usually need to be more than just a specialized data
architect.
Figure 1 depicts representative skill ranges for various
architectural roles, including developers. These are examples, not prescriptions, as
everyone is different. It is the general pattern that is important.
You see that a solution architect has a wide range of skills.
This person has deep skills in a narrow specialty, let's assume data, from the implementation
level all the way up to the enterprise level. They have broad skills applicable to
solution architecture, hence their current role, but also deep skills in some aspects of
product line, component architecture, and even implementation. They are well suited
to be a solution architect given their broad skills in that, and their skills at other levels
enables them to have more empathy for people working at those levels and to better interact
with them as a result. Notice however that they don't have solid skills in all aspects of
solution architecture, they have more to learn (as does everyone else). They're not just
a specialized architect, however, someone with very deep skills in one area but weak
in many others.
Figure 1. The agile architecture skillset (click to enlarge).
3.2 Agile Data Architects Need Agile Modeling Skills
Agile Modeling (AM)
is a practice-based methodology for effective modeling and documentation.
Agile modeling is an evolutionary (iterative and incremental) approach
to creating models and supporting documents that is highly collaborative in nature.
You can view Agile Modeling as a collection of
core practices,
as you see in Figure 2. These practices are applicable
to all modeling domains, including both data and architecture.
Figure 2. The core practices of Agile Modeling (click on each practice for details).
Key AM concepts for data architects are:
- Collaborate closely with stakeholders.
- Take an evolutionary (iterative and incremental) approach.
- Have the humility to realize that you don't have all of the answers.
- Tailor your way of working (WoW) to reflect your situation.
- Embrace change.
- Agile models are sufficient in that they are just barely good enough (JBGE) for the context in which they are used.
- Detailed specifications should be captured as executable tests whenever possible.
- High-level concepts should be captured visually and supported by concise descriptions whenever possible.
- Take a multi-view approach, data being just one view.
3.3 Agile Data Architects Need Implementation Skills
Agile data architects are also familiar with, and ideally adept at, the
agile database techniques stack
overviewed in Figure 3 below.
These are the critical skills required of anyone involved in the implementation aspects
of an agile data initiative.
Figure 3. The agile database techniques stack.
Because agile data architects are expected to be generalizing specialists,
and with the exception of industry data architects having at least
rudimentary implementation skills, they should be knowledgeable about the following techniques:
- Vertical slicing.
Vertical slicing
organizes functionality vertically into small,
consumable pieces that may be potentially deployed into production quickly.
These vertical slices are completely implemented - the analysis, design,
programming, and testing are complete - and offer real business value to stakeholders.
- Clean data architecture and clean data design. A
clean data architecture
strategy enables you to develop and evolve your data assets at a pace which safely and effectively
supports your organization - in short, to be agile. Similarly, a
clean data design
enables you to evolve specific data assets in an agile manner.
- Agile data modeling.
With an evolutionary approach to data modeling you model the data
aspects of a system iteratively and incrementally. With an
agile data modeling
approach you do so in a highly collaborative and streamlined manner.
- Database refactoring. A
database refactoring
is a small change to your database schema which improves its
design without changing its semantics, enabling you to safely improve the quality
of your data sources.
- Automated database regression testing. You should ensure that
your database schema actually meets the requirements for it, and the best way to
do that is via
automated regression tests for your database schema
to ensure data quality.
- Continuous database integration (CDI).
Continuous integration (CI) is the automatic invocation of the build process of a system.
As the name implies,
continuous database integration (CDI)
is the database version of CI.
- Configuration management.
Your data models, database tests, test data, and so on are important
artifacts which should be put under
configuration management
just like any other artifact.
4. Related Resources