Agile Data Roles: Data Professionals on Agile Teams
This article overviews the Agile Data roles. It is organized into the following topics:
- What does it mean for a role to be agile?
- The roles of the Agile Data method
- Important concepts about roles
- Is your organization short a few roles?
1. What Does It Mean For a Role To Be Agile?
Someone who is in the role of “Agile X” works in a manner that is highly collaborative and evolutionary in nature. To work in an evolutionary manner you work iteratively and incrementally. Below, when I describe each Agile X role, I focus on the X part of the role. An Agile X does that work in an agile manner.
2. The Agile Data Roles
Figure 1 summarizes the roles that are pertinent to the Agile Data method. They are organized into two categories:
- Primary roles. These roles are the focus of the Agile Data method.
- Supporting roles. People in these roles collaborate with, and support the work of, people in the primary roles. These roles are adopted from other methods such as Scrum or Agile Modeling.
Figure 1. The Roles of the Agile Data Method (click to enlarge).
2.1 The Primary Agile Data Roles
The Agile Data Method defines four primary roles:
- Backlog owner. A backlog owner prioritizes the needs and desires of stakeholders to a development team. This role addresses a subset of the responsibilities, in this case work prioritization, of the traditional Product Owner (PO) role. The PO role proves to be very difficult to fill in practice, and particularly so for teams where there is a high level of data knowledge required (data analysts are better sources of that).
- Data analyst. An agile data analyst identifies and describes how data can be used in order to answer questions and solve problems. They will often explore legacy data sources to better understand them, potentially reverse engineering them if existing documentation isn’t sufficient, as well as explore the details supporting stakeholder requirements.
- Data architect. A data architect is someone who guides the development and support of the data-oriented aspects of something. “Something” may be a component, a solution, a product line, or even your entire enterprise.
- Data engineer. An agile data engineer is anyone who is actively involved with the creation and evolution of the data aspects of one or more software-based solutions. Sometimes called an Agile data engineer.
- Data scientist. An agile data scientist works closely with stakeholders to understand their needs and identifies how data can be used to address those needs. They explore data sources, develop algorithms and predictive models to extract and manipulate the data, and analyze the data and share their insights.
- Data tester. A data tester is someone who validates that data, and how it is stored and transmitted, is of sufficiently high quality. They will perform data testing and support data engineers in adopting and applying appropriate data quality techniques to reduce data debt. They will also collaborate with data analysts to better understand data semantics and data ontologies.
2.2 The Supporting Agile Data Roles
There are several supporting roles that are important to recognize:
- Architecture owner. An architecture owner is responsible for facilitating the architectural modeling and evolution efforts. Architecture owners are often the most technically experienced person on a development team. This role is adopted from the Agile Modeling method.
- Auditor. An auditor validates that your work complies with organizational and regulatory requirements. Internal auditors work for your organization, focusing on ensuring that you are compliant and capable of passing an external audit. An external auditor works for, or at least represents, a regulatory authority that you must comply to.
- DataOps coach. A DataOps coach shares their knowledge and expertise about DataOps and agile data concepts and skills with others, guiding them on their learning and improvement journeys.
- Developer. A developer, also called a software developer, is anyone who is actively involved with the creation and evolution of a software-based solution.
- Enterprise architect. An enterprise architect is anyone who is actively involved in the creation, evolution, and support/communication of your enterprise vision.
- Governor. Governors provide organizational oversight. Sometimes governors are specialized, focusing on one oversight aspect such as data, financial, and architecture governance.
- Operations engineer. An operations engineer, sometimes called an IT operations enginer or (mistakenly) a DevOps engineer, is anyone involved with designing, implementation, or the execution of an organization’s IT infrastructure.
- Team lead. A team lead is responsible for guiding and leading a team. You can think of team lead as a metaclass that is instantiated in an appropriate manner reflecting your overall way of working (WoW). What I mean by this is that on a Scrum team the team lead role is often filled by a scrum master, on a project team by a project manager, on an architecture team by a chief architect, and so on.
- Stakeholder. A stakeholder, also known as a customer or business user, is anyone who is materially impacted by the outcome of a solution. A stakeholder could be a direct user, indirect user, manager of users, senior manager, operations staff member, the “gold owner” who funds the team, support (help desk) staff member, auditors, your governance/oversight team, people that rely on whatever you are working on, or maintenance professionals potentially affected by the development and/or deployment of your initiative.
3. Important Concepts About Roles
Any single individual can hold more than one of these roles, and several people may be in the same role. An important concept to understand is that although the responsibilities of each role vary widely that any individual in a given role may not fulfill them all, or at least not focus on all of them. For example an application developer may choose to focus on Java development on a specific team even though they also have significant modeling and testing skills. Agile IT professionals are generalizing specialists, and when they fulfill a role they will often focus on one or more responsibilities of that role, perhaps because they have been asked to trouble shoot those efforts or perhaps because they want to gain deeper experience in an existing or new specialty.
4. Is Your Organization Short a Few Agile Data Roles?
Your organization may not have people in these roles right now. Perhaps you’re a very small organization with only a few developers or perhaps you’re a mid-size organization that hasn’t yet come to grips with the architectural and operational issues experienced by larger or longer-lived firms. Don’t worry, you will. You’re actually in an enviable position because you can do it right from the start. You might not have anyone dedicated to the role of enterprise architect, let alone a group of people, but the fact still remains that you will need to come to grips with the issues that enterprise architects deal with. Perhaps enterprise architecture is something that your senior developers, and even your junior developers, discuss when they’re building new systems but never take the time to put it on paper. That’s all right, the important thing is that you think about it.