Implementing Question Stories: User Stories for Data Teams

A question story is a specialized user story specific to data-oriented requirements. A question story represents a vertical/thin slice of value that is deployable into production. This article addresses the following topics:

1. What Are Question Stories?

The following are examples of question stories:

As a sales manager I want to know the level of sales by my team by the end of each day so that I know where we stand.
As an instructor I want to know the certification pass rate of my students so that I can update my seminar marketing message.
As a restaurant owner I want to know the common combinations of menu options being ordered so that I can identify potential specials.
As a city councilor I want to know the number of complaints about road quality so that I can determine where we need to focus our repair efforts.

2. Implementing a Question Story on an Agile/Scrum Team

Lets consider the case when a team is following a Scrum lifecycle. Figure 1 overviews an agile project lifecycle, although it is possible for this lifecycle to evolve into more of a continuous delivery lifecycle where sprint 0 disappears because it occurred in the distant past and the deploy phase disappears as the result of automation. Either way, the following advice still pertains.

Figure 1. The Agile Project Lifecycle (click to enlarge).

Figure 2 depicts the look-ahead data analysis work required for three question stories that are to be implemented during sprint #9 of an agile DW/BI initiative. Notice how each question story requires a different amount of data analysis effort due to the fact that every question has unique data needs.

Figure 2. Look-ahead data analysis on an agile team. Click to enlarge.

Figure 3 depicts the overall development effort for a single question story, including both the look-ahead analysis required for the data analytics as well as the implementation portion called out in Figure 2.

Figure 4. Implementing a question story on a Scrum team (click to enlarge).

3. Implementing a Question Story on a Continuous Delivery Team

Now lets consider the situation when a team team is following a continuous delivery lifecycle.

Figure 4. A Lean Continuous Delivery Lifecycle (click to enlarge).

Figure 5 depicts the look-ahead data analysis work for the same three question stories from Figure 2, the difference being that the work is done on a just-in-time (JIT) basis rather than scheduled into fixed-length sprints. Note that the same amount of data analysis is still required for each user story as in Figure 3, but that the implementation time is no longer tied to a two-week sprint.

Figure 5. Look-ahead data analysis on a continuous delivery team. Click to enlarge.

Figure 6 depicts the overall development effort for a single question story, including both the look-ahead analysis as well as the implementation portion called out in Figure 5.

Figure 6. Implementing a question story on a continuous delivery team (click to enlarge).

4. The Implementation Activities

As you saw in Figure 3 and Figure 5 there are several activities that the two flows have in common:

Data analytics – Explore question.
Data analytics – Explore data source(s).
Data analytics – Build answer prototype.
Data analytics – Validate answer.
Specify answer (Agile/Scrum only).
Review specification (Agile/Scrum only). Against DoR.
Implement answer. See the detailed answer in the following section.
Deploy answer. On a Scrum team you are likely to batch up several answer implementations, whereas on a CD team you hopefully deploy once the answer is ready to be released.

5. What Technical Work Needs to Occur?

Figure 7 depicts how a question story can be implemented as a thin vertical slice. The green boxes represent major components of a DataVault 2 approach to architecting a DW/BI solution and the blue line indicates the vertical slice that implements the given question story. In this case, new data elements from two sources, Data Source 1 and Data Source 5, are used. The new data elements from these two data sources are loaded into a raw vault, then into the data vault, into Data Mart 3, where it is then displayed on a dashboard. Other data that already existed in the raw vault and upwards was also used to implement the question story.

Figure 7. Implementing a question story as a vertical slice (click to enlarge).

The implied technical work to implement this question story includes:

Data analysis of the two data sources. This requires the data analytics activities mentioned above.
Extract and load data from the two sources.
Update of extract, transform, and load (ETL) from the raw vault to the data vault.
Update of the ETL from the data vault to Data Mart 3.
Update of the dashboard to display the information.