Agile Data

Agile Data Home Page

Follow @scottwambler on Twitter!

The Agile Data (AD) method defines a collection of strategies that IT professionals can apply in a wide variety of situations to work together effectively on the data aspects of software systems.  This isn’t to say that AD is a “one size fits all” methodology.  Instead, consider AD as a collection of techniques and philosophies that will enable IT professionals within your organization to work together effectively when it comes to the data aspects of software-based systems. 

Agile Data Techniques


Modern software development processes - such as Disciplined Agile Delivery (DAD), Extreme Programming (XP), Scrum, and Team Software Process (TSP) -- are all iterative and incremental (evolutionary) in nature. Every single one of them. Some modern approaches, in particular XP and Scrum, are agile in nature (for the sake of simplicity, let's define agile software development as a highly collaborative and evolutionary approach). Traditional approaches to the data-oriented aspects of software development, however, tend to be serial, not evolutionary and certainly not agile, in nature. This is a serious problem.

Data has been an important aspect of every single business application which I have ever built. Then again, so have business rules, user interfaces, networks, and a slew of other issues. My experience is that left to their own devices software developers will usually struggle to get the data stuff right, and will often make questionable decisions from an enterprise data point of view. My experience is also that many data professionals are difficult to work with, often because they are stuck in their "serial ways" but also because they have little or no experience following modern software development techniques. These two observations reflect the cultural impedance mismatch between the two groups, a problem which is often over-shadowed by the technical impedance mismatch between the two technologies (object-based and relational) which the two groups work with. 


I believe that we need to find ways to work together effectively, and that there are five fundamental tasks which IT organizations must undertake:

  1. Adopt a new philosophical foundation. The Agile Data Method, and the supporting agile database techniques, describe an approach that has a chance at succeeding because it describes ways that people actually work in practice. It’s not just a collection of academic theories that sound great. The heart of the Agile Data method is in its philosophies and its roles.  The philosophies sound simple yet in practice have significant implications when you act on them: my experience is that rethinking your approach to software development and reworking your organization's culture to succeed at the software development game will prove to be difficult in practice.

  2. Help both developers and data professionals need to adopt evolutionary, if not agile, database techniques. This includes agile data modeling, database refactoring, database regression testing, test-driven development (TDD), and encapsulating database access appropriately. We must all recognize we need to become more agile, that we can take an agile approach to modeling and documentation, that data models don't drive object models (and vice versa), that we need new tools, and that we should work in our our separate development sandboxes. Adopting common agile database core practices is also a very good idea.

  3. Help developers learn fundamental data techniques.  This includes data modeling, mapping objects to RDBs (O/R mapping), and working with legacy data.  Developers should also understand how to choose a primary key strategy for a table, relational database fundamentals, XML, referential integrity and shared business logic, how to retrieve objects from an RDB, how to implement reports, security access control, transaction control, and concurrency control.

  4. Help data professionals to learn fundamental development techniques. This includes learning the basics of object orientation and the Unified Modeling Language (UML) and how to data model using the UML.

  5. Take an agile approach to enterprise efforts.  Your enterprise architecture and enterprise administration (including operational data administration) groups need to become more agile to succeed in modern IT environments.  It's not only possible to take an agile enterprise approach, I believe it's preferable to traditional traditional techniques because it's more responsive to the actual needs of your organization.


Other Important Links: