Data Warehouse Project Management
Information Management Magazine, March 1, 2006
Why is project management for a data warehouse different than most other
applications? The answer is - a data warehouse is never really a completed
project. Although there are phases with start and end dates, the data
warehouse itself never reaches an end state until it is terminated.
Data warehouses are a different breed. They are organic and need to be
nurtured to grow. Being organic, they are ever changing: dynamic, in other
words. This is what makes project management for a data warehouse so unique
There are two paths to project managing a data warehouse. The first is
to approach the project strictly from a project management perspective
by managing the project scope and timeline. The second route is to follow
through with the traditional responsibilities of project management and,
at the same time, do a deep dive into the inner workings of the data
Understanding in detail what is "under the hood" will allow you to speak
with authority on the capabilities and issues of the data warehouse.
Intimate knowledge of the data and its relationships is the distinction
between a generic project manager and data warehouse project manager.
A technical project manager is not the same as a data warehouse project
manager because a data warehouse does not function like any other system.
Only firsthand knowledge of what makes a data warehouse hum can brand
someone as a data warehouse project manager.
This is not to be disrespectful in any way to all the good project managers
out there, but the reality is, project managing a data warehouse is very
different than other applications. If you have a peer who is a data
warehouse project manager, take him to lunch and pick his brain to see
Whether you consider yourself a data warehouse project manager or not,
the following content is relevant to you as the project manager. You still
need a team, solid requirements and a roadmap to deliver a quality product
on time and within budget.
Project Team Members
As a project manager you sometimes have the luxury of putting your team
together. Often, your team "appears" because the prior project has
completed and these people are available. Keep in mind what was previously
mentioned about the data warehouse being a never-ending project. When
formulating your team, try to recruit people that you think will be around
for the long haul, not just a quick stint.
Ideally you would like all of your people to have experience in data
warehousing, but that is not always the case. At a minimum, you would like
to seed your group with key people who have the desired experience and
can spread their knowledge.
Project Manager Expertise
Because the project manager is the single point of contact and the
visionary of the team, being knowledgeable in data warehousing would be
very advantageous. As a competent project manager with no data warehousing
experience, it is still possible to deliver a data warehouse as planned,
but this requires strong reliance on the project team for subject matter
You may be thinking, "Well, I am the project manager, and I do not need
to know the details." This is true, but to ensure a quality product and
to be in a position to speak with authority about the data and workings
of the data warehouse, you will need to get your hands dirty. If you are
not intimate with data warehousing, at least ensure that a senior member
of your team is.
Data Warehouse Business Requirements
As with any system, requirements are a good indicator of success (or
failure). As the project manager who is overseeing the requirements
gathering and analysis, prior knowledge of data warehousing is again a
major plus. Knowing what questions to ask other than what you received
as requirements will not be apparent. There are significant unspoken words
that go into the design of a data warehouse that only prior experience
or a true visionary will bring to light. If members of the project team
have prior experience in data warehousing, make sure they participate in
the requirements review.
To flush out these hidden details, you will need to gather the requirements
from your users in bite-size chunks based on their attention span and
availability. This will be an iterative process because you too will have
work to do when gathering the business requirements. As the requirement
sessions progress, you should build a mockup of the application based on
what you have to date. This will become your platform for explaining how
you view what the customer has requested. This mockup will help your
customer make the leap from concept to reality by allowing him to visualize
Your platform mockup does not need to be a working model at this point.
A simple slide presentation or static Web pages will suffice. Don't be
surprised if your business requirements change drastically at this point
because your customers may now view what they were asking for in a whole
different light. Before you call the business requirements a wrap, have
a walk-through of the document to ensure all the content is still required
and that no ambiguity exists.
A tip for who should be included in the audience when gathering your
business requirements in addition to your users: representatives from
development and testing. You may not need/want them initially, but you
certainly want them involved once you get to the mockup phase. Getting
early buy-in from all participants is of the utmost importance for any
project to succeed.
Data Warehouse Technical Requirements
Now that you have a robust set of business requirements, it is time to
begin developing the technical requirements. Your technical requirements
document should maintain the same structure as the business requirements
wherever possible. If you need to deviate, ensure there is traceability
back to the business requirements.
One approach that can be used to tie the different documents together is
to derive a numbering system that can be inherited by the subsequent
documents as a reference, as shown in Figure 1. By devising a logical
structuring of the business document, the task of reconciling business
and technical requirements becomes much simpler and more effective. You
can almost guarantee yourself full test coverage if your test case
development follows the same methodology.
Figure 1: Requirements Traceability Outline
Once you have the technical requirements to the point where the system
flow is well defined, a working prototype should be developed. The
prototype does not need to do more than convey the screen logic flow and
basic functionality. The intent of presenting the prototype is to ensure
the technical requirements accurately reflect the business requirements.
The prototype will allow everyone on the development side of the house
to gain an understanding of what is to be built. You will most likely be
peppered with some tough questions from the crowd, but this is good
constructive criticism. If you follow this advice, fewer issues are likely
to surface during the design phase.
Data Warehouse Implementation
You've probably heard this before: you should not try to implement the
entire data warehouse at once. This is so true - no one would want to wait
a year or so before they see any results. The project plan should break
up the functionality to be delivered in phases of two to three months or
whatever works best for you.
Delivering in phases can be viewed as a rapid application development (RAD)
approach, which is very effective. Figure 2 depicts a single cycle using
RAD for delivering your data warehouse. Each phase, therefore, would be
a new cycle.
Figure 2: System Development Lifecycle
By delivering in phases, you not only deliver something tangible for your
users, but you may also flush out issues that can be quickly corrected.
Issues that would normally have a downstream impact can be addressed at
this time before the added functionality is developed.
Project Manager's Litmus Test for Success
Now that you have rolled your data warehouse into production, how do you
know if you've done a good job? Finishing on time and on budget is good,
but that alone doesn't guarantee the project's success.
You know you have delivered a quality product that produces actionable
results for the business when:
Users are constantly knocking on your door for more information than
the data warehouse currently contains. This tells you the thirst
for knowledge is starting to spread, and people have faith in the
The buzz in the hallways mentions the data warehouse, or meetings
make reference to it as the source of data.
The data warehouse becomes the heartbeat of the business, where
decisions are made from the data intelligence it provides.
You will want to build on this success by continuing to nurture the data
warehouse. If you emphasized data integrity and flexibility in the design,
momentum will keep rolling along as new data and functionality are added
to the data warehouse.
Ken Pohl is the president and founder of Endorse Consulting
(www.EndorseConsulting.com) which specializes in minimizing the risk to
a project's ROI by mitigating potential problems before they arise. His
many years of developing and implementing data warehousing, business
intelligence and CRM systems has provided a wealth of knowledge as proven
techniques. You can reach Pohl at KenP@EndorseConsulting.com
For more information on related topics, visit the following channels:
Project Management/Tool Selection