Adopting Agile Data Ways of Working and Thinking

The Agile Data method presents a wide range of ways of thinking (WoT) , proven ways of working (WoW), and one or two reasonable theories about effective ways to go about the data-oriented aspects of software development. What it has not done is discussed what you need to do to succeed at them. Until now. in this article I describe how your organization can adopt the agile database techniques described at this site.

To successfully adopt these WoT and WoW you must:

1. Change the Way You Look at Software Development

The WoT and WoW described at this site have several significant implications for your organization. My experience is that you must accept that:

Everyone needs to work closely together. Software development is a communication game, and as Cockburn (2002) argues, documentation is the worst form of communication available to you whereas face-to-face communication standing around a whiteboard is the best. Simple things such as co-locating the team in a single workspace, using simple tools such as whiteboards and paper, and having stakeholders as active members of your team will improve your productivity by at least an order of magnitude.
The models and documents are finished when the software ships. It is an iterative and incremental world now, not a serial one. The requirements are fully defined for the release of a system, if at all, only until the software is ready to go to final acceptance testing. Until then they might change. The same thing can be said of your logical data model, if you even create one, and your physical data model. These work products evolve as work progresses on the system and may even change just hours before the production baseline of your system is finalized.
Agile requires significantly greater discipline. Although this is not what the traditionalists want to hear, the reality is that Agile approaches require significant greater discipline on the part of practitioners compared with traditional approaches.
Power within your IT department will shift. Changes in process always entail shifts within the power structure of an organization, and Agile Data is no exception. Agile database techniques shift the way that enterprise concerns, particularly those pertaining to data, are considered. Architectural model reviews are a thing of the past because your models evolve over time and because reviews are a “process smell” in the agile development indicating that you’ve made an organizational mistake earlier in your initiative. One way to look at it is if someone is qualified to review a work product why didn’t you involve them in its initial development? Existing organizational structures, particularly those based on specialized skills or a command and control structure, need to be reworked in favor of a more organic structure.
Everyone needs to become actively involved. The day of the specialist who attends reviews, or has work products submitted to them for review and feedback, is over. The day of the bureaucrat who merely pushes paper and has no direct influence on the creation, maintenance, operation, or support of a system should never have come about in the first place. Bureaucrats can’t hide behind their onerous and prescriptive processes anymore, instead they must roll up their sleeves, work closely with development teams, and actually add value to your IT efforts. People who fight this concept are likely good candidates for re-education and in extreme cases may need to be made aware of opportunities in other organizations (if you get my meaning). We need generalizing specialists now.
Everyone needs to rethink their approach and beliefs. There are many thought provoking ideas presented at this site. It is possible to take an evolutionary approach to database design. There is more to modeling than the UML, or data for that matter. There are many different ways that you can approach development, one size does not fit all. Many technical issues often thought to be the sole domain of databases, including both referential integrity and transaction control, are also pertinent within your objects. You have many technical options available to you, and they all have their strengths and weaknesses. The point is that these ideas are likely to go against what you currently believe, or how you currently prefer to work. Get over it.

2. Understand the Challenges you Face

You also need to understand the challenges that you are likely to face when introducing agile data techniques into your organization. My experience is that the most difficult challenges that you will need to overcome are not technical in nature but instead are people oriented. At the risk of promoting stereotypes, I have found that experienced IT professionals, novice developers, and managers all seem to have their own unique challenges that need to be overcome. Let’s look at each group one at a time.

Experienced IT professionals, often people with twenty or more years in the industry, likely:

Have not invested the time to understand agile software development.
Haven’t taken, or been allowed, the time to try the WoW and WoT described at this site.
Aren’t comfortable with change, particularly change that dramatically shifts the political power structure within your IT department.
Are convinced that their existing approaches work (which they do in some situations) so don’t see the need to change their ways.
Have had bad experiences in the past with “code and fix” (CAF) approaches. Because they don’t understand agile software development they often equate it with CAF and therefore assume that agile techniques are a bad idea.
Have some very good points that aren’t well addressed by the agile community, such as data-oriented and enterprise issues, and therefore they feel that agile techniques aren’t sufficient for their needs.
Believe that their situation is unique, perhaps they work in a Fortune 50 company (although 49 other organizations are also in this situation) or a government agency, and therefore agile techniques won’t work for them.
Focus on symptoms, and not the root causes of problems within their IT organization, and therefore they haven’t questioned their preferred approach to development.
Haven’t coded in years and don’t realize the implications of the new techniques and tools currently being used by developers. Java/C# development is significantly different than COBOL development.
Have listened to myths and misconceptions promoted by other experienced IT professionals whom they respect, unfortunately people who are also struggling with agile concepts and whom are likely to tell them what they want to hear, and as a result feel that they don’t need to continue looking into agility.
Are narrowly focused specialists, often with years if not decades of experience, and therefore have difficulty understanding the bigger development picture.
Are likely scared that they don’t fit into agile development (and very likely don’t, given their current skillset).

Novice IT professionals are also struggling with learning agile techniques, although they face different challenges. They likely:

Perceive agility as meaning they don’t have to model and they don’t have to write documentation.
Have very narrow experience, if any, as a programmer.
Don’t have the experience to appreciate the bigger picture, or at least to appreciate its nuances.
Are focused on a single technology or programming language.

Managers within your organization have their own unique issues regarding agility. They often:

Can’t provide adequate resources for your team (or simply do not understand what they are).
Have had bad experiences in the past with new techniques and is unwilling to try yet another one.
Believe that agile software development is another fad and will go away after a few years.
Don’t realize that an agile development team requires other parts of the organization, including both IT groups such as enterprise architects and data management/administration , to work in an agile manner as well when they interact with the team.

Everyone needs to approach agility with an open mind, ideally without any preconceptions. They need to look at the big picture, recognize that they have serious problems that aren’t going to go away unless they act. Ideally everyone needs to work with, and be mentored by, experienced agile developers in order to learn these new approaches. Although it is possible to bootstrap your team, and even your entire organization, into agile software development you are much better advised to seek the help of people who have gone before you. Education and training in agile software development is also important although not as critical as good mentoring.

Experienced developers and managers need the opportunity to try these new ways, and more importantly be given the time it takes to break themselves of their “bad” non-agile habits. Novice developers need to focus on education/mentoring as well as simply gaining experience. Novice developers will likely learn agile techniques easier than experienced developers will as they have less baggage to discard along the way.

3. Actually Try Agile Data Ways of Working

There is a significant difference between theory and practice. You can read about something all you want, but until you try something you truly won’t understand it and its implications. At some point you’re going to have to get your feet wet and try this stuff out on a real initiative.

Remember the adage

“In theory, theory and practice are the same

In practice, they are not.”

4. Be Realistic about Agile Data Ways of Working

The following list describes several important factors that you need to consider when bringing agile database techniques into your organization.

Be patient, it will take a generation. I believe that the adoption curve for agile software development, including agile database techniques, will be similar to that of object technology. Object technology was first being considered within the business community in the late 1980s and at the time of this writing many organizations, often referred to as the Late Majority or Laggards on the technology adoption curve (Moore 2002), are still struggling with adopting object orientation (OO). That’s fifteen years by my watch, and I expect no different for agile software development (the implication is that it may be 2020 before your organization finally catches up).
Don’t be too fervent. The surest way to turn someone off of agile techniques is to try to convince them that agile is the only way. Remember the fit-for-purpose WoT of Agile Data – recognize that you should strive to find the appropriate way of working (WoW) for your context. The implication is that you might not become as agile as you’d like right away, but you very like could become more agile than you currently are.
Don’t go in blind. Reading this book is a good first step towards becoming more agile in your data-oriented activities, but it’s only the first step. Do some more reading before you try to adopt the WoW and WoT described at this site, www.agilealliance.com and AgileModeling.com are great resources that you should take advantage of. Talk with people who are following agile techniques on their own teams to discover what they’ve experienced. Get involved with the agile data community and share your own experiences.
Don’t underestimate the politics. Process is politics, and when you change the process as radically as I describe at this site
many people will obviously take issue with the necessary changes.
Be prepared to find work elsewhere. Your organization may not be ready for agility, and worse yet may not be for some time. The implication is that you have a very difficult decision to make – do you continue to wait and hope that an opportunity arises within your organization, do you try to create such an opportunity, or do you update your resume and start looking for work elsewhere. As Ron Jeffries likes to say “Change your organization or change your organization.” It’s your life, take control of it.

Adopting Agile Data Ways of Working and Thinking

1. Change the Way You Look at Software Development

2. Understand the Challenges you Face

3. Actually Try Agile Data Ways of Working

4. Be Realistic about Agile Data Ways of Working

Recommended Reading