Would you take the same approach to create a web page describing
your family and the embedded software for a NASA space probe? Of
course you wouldn't. What about a
business application and a NASA space probe? Likely not. Would you take
the same approach with a team of six people that you would with a team of sixty
people? Or six hundred people? Once again, likely not. Different
situations obviously call for different approaches.
It's easy to see that you need to take a different approach
developing a web page as compared to developing a space probe. Your company likely doesn't develop both web pages and space probes, so
it's easy to stand back and recognize that there's a difference. It would be clearly foolish to follow similar approaches on these
projects - a very simple process is sufficient for the web page development
effort whereas a complex and rigorous process could be appropriate for the space
probe. You would put the space probe project at serious risk
following the web page process and you would incur significant costs following
the space probe software process to develop your web page. Clearly it is important to follow the right process.
But what about developing a
data warehouse as compared to an
online ordering system? Your
organization might do both of these things, and now suddenly it isn't so easy
for you to fairly judge the situation. Senior
management might want these two projects to follow the same method to
make it easier to manage, measure, and compare their entire portfolio of
projects. IT support groups, such
as enterprise
architecture and enterprise
administration, want to work with project teams in a consistent manner using
a consistent set of artifacts. Similarly
stakeholders who are involved with several IT projects will want to
follow a consistent approach with each team. Your human resources department doesn't have the budget to train and
educate people in every single IT method and technique. It is clear that
there is significant pressure to follow a single method within your
organization.
What
do you do? The
fit-for-purpose way of thinking (WoT) of the Agile Data method advises you to find the appropriate
way of working (WoW) for your context, and that is very often somewhere
betwee the extremes. In this case the extremes are following a single
method for all IT initiatives and following a different method for each.
The right approach may be found by adopting a handful of
methods that between them cover the majority of project types within your
organization. Each project team would choose one and then tailor it to
meet their specific needs. In organizations that tackle one type of
project, as is common in smaller firms and in specialized consulting firms, the
size of the handful may in fact be one method. A large organization, such as a
financial institution, telecommunication company, or government agency, that
tackles a wide range of project types will discover that they require several
methods.
In this article I discuss:
- The need for flexibility
- Categories of method
- Comparing development methods
- Categories of projects
- You may need several methods
To succeed at software development you need to be flexible
in your choice of software development method, to choose the right one for
the job. There are several reasons
why it is important to do so:
-
Different technologies require different techniques.
Object-oriented methods are best-suited for projects using
object-oriented technologies whereas data-oriented methods are
best-suited for data-oriented applications.
-
Every individual is unique.
People are not replaceable parts.
Each person has a different background, different preferences
for the way in which they work, and different cognitive style (e.g. are they
visual or textual thinkers?). An
approach that works incredibly well for you might be impossible for me to
grasp, and vice versa.
-
Every team is unique.
Because teams are made up of individuals, and because
individuals are unique, each team will require a unique way for them to work
in order to maximize their potential. Several
teams could follow DAD, but each will follow their own "flavor" of
DAD tailored to meet their exact needs.
-
Your external needs vary.
Some projects must conform to government regulations.
Other projects are highly-dependent on suppliers, such as technology
vendors or software development outsourcers, and therefore must tune their
process to reflect the ways that their suppliers need to work..
-
Project categories vary.
Different types of projects, see
below, require different approaches because each category has different
priorities and goals.
-
Method categories vary.
Each category of method,
for example agile and rigorous, has its strengths and weaknesses.
|
An organization gets the
process
that it deserves. |
|
2. Categories of Method
For the purpose of our discussion there are four different
categories into which a method, such as DAD or
Extreme Programming (XP), could fall into:
-
Code and fix. This approach is also known as
"hacking", "hack and slash", or "no-process at all". This approach to development is chaotic and often unplanned,
or when it is planned the plan is quickly abandoned. Estimates and schedules, when made at all, are rarely met in
practice.
-
Serial rigorous.
Software processes in this category are well defined and often
include detailed procedures that developers are expected to follow in a
more-or-less serial manner. For
example requirements are identified, reviewed, and accepted.
The analysis of those requirements is performed, reviewed, and
accepted. The design is
defined, reviewed, and accepted. And
so on. There is room for
feedback between phases, although that feedback is provided via a reasonably
defined procedure and the changes are then reviewed and accepted as before. Systems are typically delivered on an incremental basis where
the releases are on the order of several quarters or years in length.
ISO/IEC 12207 is an
example of a process which is typically instantiated in a serial and
rigorous manner (to be fair, it doesn't have to be instantiated like this
but it typically is). This
approach is sometimes called a "waterfall process" or a "big design up
front (BDUF)" process.
-
Iterative rigorous.
Software processes in this category are well defined and often
include detailed procedures that developers are expected to apply in an
iterative manner. For example
requirements may be
initially defined at a high-level with the detail later
identified on an as needed basis. Small
portions of your system are fleshed out, with software potentially delivered
on an incremental basis following short release cycles often on the order of
weeks or months.
Disciplined Agile Delivery (DAD) is an example
of an iterative rigorous process.
-
Agile. Agile
is an approach to
software development that is people oriented, that enables people to respond
effectively to change, and that results in the creation of working systems
that meets the needs of its stakeholders. Software processes in this category are defined at a
high-level, often presented as a collection of synergistic
ways of thinking (WoT) and ways of working (WoW).
Disciplined Agile Delivery (DAD)
and XP are both examples of agile software processes.
Each method category appeals to a different mindset. For example, code and fix methods are attractive to maverick developers, often referred to as
cowboys, that prefer to work alone with little guidance. Although great
software can sometimes be created this way, the reality is that the approach
doesn't scale well to teams of people nor does it succeed very often.
Serial rigorous processes often appeal to managers who are looking for an
approach that is straightforward (hopefully) to understand and to manage.
This is particularly true in heavily regulated industries, although my experience is that the regulations never seem
to stipulate that the process be serial nor rigorous; however, people choose
to interpret the regulations to mean that. Highly bureaucratic environments
also seem to prefer serial rigorous processes, often because this approach enables the politicians among us to build their empires easily. Iterative
rigorous processes work well in environments that are bureaucracy prone yet
still able to condone "risky" practices promoted by iterative development
(iterative development actually reduces most risks although it is harder for
"traditionalists" to manage). Agile processes are new to most
organizations and are therefore considered leading edge to many managers. It is interesting that those very same managers bemoan the fact that their
developers don't want to follow software processes yet will fight the adoption
of a new agile process by a project team that is clamoring to it.
|
Repeatable processes have
nothing to do with successful software development,
repeatable results have everything to do with it. |
|
Table 1 describes and compares
leading development methods. The focus is on high-level software
processes, not on detailed methods - therefore DAD is included but use case modeling is not. This list is not meant
to be exhaustive, instead it lists the development methods that appear to be in
reasonably common use (in my opinion). Table
1 includes advice for when to apply the method, some of the situations
overlap which reflects the fact that you have a methodological choice.
Figure 1 depicts a graph comparing the same processes.
Table 1 also indicates
primary types of modeling artifacts that you are likely to create following each
method. The artifacts are listed "in order" of creation although
the reality is that most models are created iteratively, even when you are
following so-called serial processes. There
are many secondary models that are not listed, such as business rules or user
interface prototypes on a DAD team, because those
artifacts are not the focus of the method.
The table isn't perfect, instead it is just barely good enough in that
it shows that there are many different approaches to development, each of which
has it's own collection of primary modeling artifacts.
Each method is applicable to certain types of situations, so it is to
your advantage to understand each method and to pick the one best suited for
your current situation.
It is important to understand that the advice regarding when to
use a method is fairly general and should be taken with a grain of salt.
For example, although XP is well suited for small teams people have been
successfully applying XP on medium-sized teams between 40 and 60 people.
Table 1.
Comparing software development methods.
Method
|
Description
|
When to Use It
|
Primary Modeling Artifacts
|
Agile Data (AD) |
A partial agile method which focuses on techniques which support
evolutionary (iterative and incremental) database development. |
Tailor the
Agile Data ways of thinking (WoT)
and ways of working (WoT) into other evolutionary processes. |
Agile
data models |
Agile Model Driven
Development (AMDD) |
A partial, practices-based method which describes techniques for
effective modeling and documentation of systems. |
Tailor the AM principles and practices into other agile or near-agile
processes. |
Apply the right artifact
for the situation at hand. |
Code and Fix |
A typically ineffective approach to development, usually followed by unskilled or
poorly skilled developers, where they simply write code without putting
much thought into it. Also called "hacking" or "immature
development". |
For throw-away prototypes. |
Source code |
Data-Driven Approach
|
This is a generic category of data-driven methods
popularized in the 1970s and 1980s with the emergence of structured
methods. This approach
is typical rigorous and serial.
For a humorous look, read
The
Glacial Methodology
|
Development of a
data warehouse,
although a usage-centered approach is still preferable.
Development of a simple "CRUD" (Create Read Update
Delete) business application.
|
Conceptual data model
Logical data model
Deployment architecture
Physical data model
|
Dynamic System Development Method (DSDM)
|
This is an agile method that has received ISO 9001
certification. In many ways
it is a formalization of the Rapid Application Development (RAD)
methods of the 1980s.
|
Development of a user interface intensive system.
Complex business application.
|
Functional prototype
Design prototype
|
Extreme Programming (XP)
|
An agile development method that focuses on the
critical activities required to build software.
|
Small, co-located project teams (4-10 people).
Requirements are uncertain.
Good relationship (potentially) exists with project
stakeholders.
|
User stories
Architectural metaphor/strategy
Class
responsibility collaborator (CRC) cards
|
ISO/IEC 12207
|
A multi-phase software process that includes development,
operation, and retirement of software-based systems.
|
Medium to large project teams (20+ people).
Mandated (e.g. by government) to follow this approach.
|
Shall statements
Logical data model
Logical process model
Physical data model
Physical process model
|
Object Oriented Software Process (OOSP) |
A rigorous, CMM Level 5 (when fully instantiated), process whose focus is
on the development, operations, support, and maintenance of systems
using object-oriented and/or component based technologies. |
Medium to large project teams (10+ people).
|
Use-case model
System architecture model
UML
Sequence diagrams
UML class model
Physical data model
|
Personal Software Process
(PSP) |
A prescriptive software process for individual developers. |
1 (see the TSP for the group version) |
TBD |
Scrum |
An agile method whose focus is on project management and
requirements management. Often combined with XP. |
Any size project |
Varies, depending on the development process (XP, ...) which you use Scrum
with. |
Team Software Process (TSP) |
TBD |
TBD |
TBD |
Test Driven Development
(TDD) |
An evolutionary approach to development where you must first write a test
that fails before you write new functional code. Also known as
test-first programming or test-first development |
Any size project |
Tests |
Figure 1. Comparing leading software processes.

It isn't enough to recognize that you have a selection of
software process methods to chose from.
Nor is advice presented in Table
1 sufficient to determine whether a given method is a good candidate
for your project because you must also take the type of project into account.
Table 2 summarizes the various
categories in which a project may fall, categories that are not mutually
exclusive (e.g. it is possible to have an outsourced data warehousing project).
Once again this table is meant to be representative but not exhaustive.
It is important to understand that:
-
Any given project may fall into several categories.
You may outsource a data warehouse for example, or a new application
development effort using C# and .NET technologies may involve integration
with several legacy systems, integration with a new commercial off-the-shelf (COTS) package, and the
retirement of an existing system that the new one replaces.
-
The suggested methods reflect my experiences and
opinion. I have no doubt that you could force-fit DAD to work for
the development of a new COBOL application but is this really your best
strategy when other well-defined and proven approaches already exist?
-
Each method is applicable in several scenarios yet
no single method is applicable to all. Just like you want to follow the practice "Apply the
Right Artifact(s)" when you are modeling, when you are working on a
project you want to "Apply the Right Methods".
Table 2
. Comparing project categories.
Project Category
|
Description
|
Suggested Methods
|
Commercial Off-The Shelf (COTS)
|
You have purchased or downloaded a system built externally
to your organization that may (or may not) be modified to meet your unique
needs and/or to integrate into your existing environment. Legacy analysis will comprise a significant portion of this
effort.
|
Data-driven approach
EUP
ISO/IEC 12207
TSP/PSP
|
Data warehouse
|
You are building a database, or collection of databases, to
be used for reporting purposes.
|
Agile Data
Data-driven approach
EUP
ISO/IEC 12207
TSP/PSP
Usage-driven approach
|
Emergency release
|
You have to quickly add a new feature, or fix a
high-priority defect, within a system that is currently in production.
|
Slimmed down version of the process used by the team to
develop the system
|
Integration/replacement
|
You are integrating two or more existing systems, to wrap
access to one or more existing systems, or simply to replace them.
|
AMDD
Data-driven approach
DSDM
EUP
DAD
ISO/IEC 12207
TSP/PSP
XP
|
New application development - Component/object
technologies (e.g. Java)
|
You are developing a new system.
|
AMDD
DSDM
EUP
DAD
OOSP
XP
|
New application development - Procedural technologies
(e.g. COBOL)
|
You are developing a new system.
|
Data-driven approach
ISO/IEC 12207
TSP/PSP
|
Ongoing maintenance
|
You are operating and evolving an existing system in
production.
|
The method used to develop the system originally, or a
newer method applicable to the technologies used and the nature of
your maintenance team.
|
Outsourced
|
You are managing an outsourcing effort.
|
The outsourcer's method.
Data-driven approach
EUP
ISO/IEC 12207
|
Retirement
|
You are retiring an existing legacy system.
|
Data-driven approach
EUP
ISO/IEC 12207
TSP/PSP
|
Safety Critical
|
Your system can affect the health or safety of people.
Examples include air traffic control systems, medical research data
analysis reports, and heart monitoring systems.
|
Data-driven approach
EUP
ISO/IEC 12207
XP
|
So what's the point? For the Agile Data method to
work within an organization everyone needs to be flexible when it comes to
leveraging a development method. One size clearly does not fit all,
particularly when the types of projects vary widely. Everyone involved with
software development needs to have an open mind, especially when working with
different methods than they are used to. For traditional data
developers the implication is that many projects may not have conceptual or
logical data models, instead they may use
Class Responsibility Collaborator
(CRC) cards or UML class models. For application developers with a strong
background in object-oriented development the implication is that sometimes they
may work from traditional requirements artifacts such as shall statements or
conceptual data models instead of the use cases or user stories that they may be
accustomed to. Different projects will work with different artifacts,
choosing the best options for their situation.
It is possible to easily adopt and support several
methods within your organization if you choose to do so. To succeed you should:
-
Adopt a small subset of the available methods.
You don't need to support every method listed in Table
1,
but you will need a few. For
example a combination of EUP, DAD, and a data-driven method should
prove sufficient for most organizations. A combination of ISO/IEC 12207, EUP, and DAD could also work for
many organizations. Organizations that tackle one type of project the
size of the subset may in fact be one method. However, adopt ONLY
ONE method if it makes sense to do so, not simply because it is
convenient for management.
-
Be prepared to tailor the process to each project
team. One team's version
of DAD will be slightly different than another team's version and
that's ok. You might also decide to
combine methods, for example DAD works very well with SAFe.
-
Define common goals, not common artifacts.
A common mistake that many organizations make is to insist on a
common set of artifacts that every project must develop, resulting in
make-work efforts that offer little actual value to the effort.
For example, don't insist on a detailed requirements model
comprised of a conceptual data model and a collection of shall statements
when many methods require different requirements artifacts (such as
use cases or feature statements). Instead
insist that the stakeholders come to an agreement with the project
team as to what is to be built and when it is to be built - How the team
and stakeholders come to that agreement is their business.
-
Define common terminology.
Using the same terminology and concepts will make it easier to
support several methods. One
source of common terminology is the
Software Process Engineering Metamodel (SPEM)
specification.
The SPEM is something that your enterprise
administrators may want to consider adopting to form the conceptual
foundation for their software process support efforts.
Having said this, I wouldn't want to force fit an existing process into an
unreasonable "standard" set of terms nor would I want to invest a
lot of time in an effort to standardize terminology. If you can easily
come to a terminology agreement then do it, otherwise move on to more
important things.
-
Be flexible.
Everyone involved with software development needs to recognize that
it is a complex endeavor and that there are many ways to succeed at it.
-
Prioritize.
A serious problem with supporting several methods is that many
people need to be prepared to work with different types of artifacts.
For example, quality assurance professionals who facilitate
requirements reviews will need to work with streamlined artifacts produced by an agile team to a collection of
acceptance tests written by an XP or DAD team to a 200-page system requirement
specification (SRS) written by a ISO/IEC 12207 team.
The take-home point from this article is simple: It may be completely unrealistic to expect every single project team within your
organization to follow the same software development process.
Different teams will create different types of artifacts, in different
manners, and in different orders. Software
development is complex and varied and as a result you need to have several
processes in your intellectual toolbox if you wish to succeed.
Yes, this makes your job as an IT professional harder than you would
likely prefer. Life's tough that
way.