Waterfall - Communication Problems

Last updated: 2007-10-18 00:00:00

Most of the communication done in a Waterfall managed project is through documents. Let's take a look at the problems this introduces.

Communications issues - particularly the over-reliance on communication by documents

It seems so correct and natural to capture into documents all the details of what we are working on that it is almost unthinkable to question this practice. But I am going to try.

Here is a short list of some of the problems of communication by documentation in a Waterfall project. I'm going to try to expand on these sometime soon.

• No way to test for correctness or completeness
• Its "Lossy" - the most important and valuable information remains in the brain of the author
• Becomes outdated quickly and is often not kept up to date
• No easy way to discover stuff not thought about
• Especially hard to do in a high pressure, strict deadline situation
• Wasteful repetition from phase-to-phase
• Suffers from the "telephone game" syndrome
• Documents are essentially "one-way" devices
• Causes lengthy feedback loops
• Changes are difficult or impossible to inject

No way to test for correctness or completeness

No matter how careful we are, the only way we can gain confidence about our documents is to scrutinize them carefully as part of our process. We try to get as many eyes as possible going over the documents and making sure everything is correct. We have meetings where people discuss the document. We have documents with change audit trails built in so we can see what has changed over time. And so on.

The problem is, until we have working code of some type there is no way to know that we've done a decent job with any intermediate deliverable - be it documents, diagrams, charts, checklists, screen shots, and so on. There is no way to put those documents into a test harness or automated user acceptance test suite and verify things are good.

We can test a prototype, but it is rare to see prototypes used - at least in my experience. And they introduce their own set of issues.

How do we deal with this? Start coding as early in the project as possible. Preferably on the first day. Communication that becomes working code directly is the most effective communication. When a real user or "user proxy" starts testing and clicking around on something, or when real user acceptance tests can be run, we can get to the real needs quickly.

The "Lossy" nature of communication by documentation

Capturing everything we know about something into a document is a very difficult task. Even under the best conditions the typical person is going to miss some stuff, make it ambiguous, or have trouble putting important details into words.

We might have a deep understanding of what we are trying to communicate, but we still find it difficult to translate that understaning into written word (along with charts, graphs, images, and lots of etc.) It is rare that a true domain expert is also an expert at communicating via documents. Even the best writers have trouble capturing technical and business knowledge into a document.

In other words, much of the important stuff gets captured in the documents (at some less-than-perfect level of correctness), and a smattering of the details gets captured (and usually too much detail that clouds the real issues), but almost none of the real thinking can get communicated this way - so a lot of useful thinking gets lost.

Documentation becomes outdated quickly and is often not kept up to date

If we have a document that is not in sync with the code or other work that depends on that documents we've got a problem. To be useful a document must be kept up to date at all times, otherwise people who are using it will be doing the wrong work - and this is a compounding problem as that bad and out of date work cascades down the waterall.

Of course, if no one is using the document then the question is "why have it at all?". So, if a document is useful and being used it must be kept up to date. Makes sense to me.

Even in the most strict Waterfall project where it is expected that everything gets thought about and documented "up-front", the discovery and knowledge that is gained during later phases almost always uncovers things no one thought about or properly understood "up-front", and the earlies work needs to changed.

Okay then - I hope we can agree that we might need to change the documents whenever we discover something that changes our thinking, and on many projects that is either explicity or implicity accepted - but it is rare that it gets done. Especially when we get into the "crunch time" when there's hardly enough time to get our current work done let alone updating some documents we might not even know exist, and probably don't have the rights to change, or where the does have the rights and understanding has been rolled off to some other project. So... documentation becomes outdated quickly and is often not kept up to date.

There is no easy way to discover stuff not thought about, or not thought about enough

This might not seem like a documentation problem, but I think it is. See if you agree with me.

On the projects I've worked on, I've never seen a case where important things were discovered late in the process. A number of things have always been discovered that weren't documented and probably weren't even thought about in the earlier phases. Also, simple things end up being not nearly as simple as originally thought.

The reason this is a communication problem is because documents cannot be tested for completness or correctness. The only time we can be confident that we didn't leave something out or have "under-thought" things is once the program is in the hands of the end users, or at least in the hands of a competent user proxy or testing team.

This introduces the concept and value of short feedback loops. Feedback from the customer or customer proxy can guide and steer our efforts towards the best result. The sooner we can get this feedback the better. My suggestion is that we must directly convert communication and thinking into working software and get it into the customers hands as quickly as possible. In an ideal world, this would be almost instantaneous. We can't do that perhaps, but we should strive to get as close as possible. (Also see "Causes lenghty feedback loops below).

Good documentation is especially hard to do in a high pressure, strict deadline situation

Dang. I have enough trouble thinking things through and being complete and correct when my brain has enough time to think and explore. When the crunch is on, all bets are off.

All the other problems of communication by documentation are exacerbated when the pressure is on. More stuff gets left out, the wrong things get thought about, short-cuts are taken, things are scrutinized less... and so on.

Wasteful repetition from phase-to-phase

Once something is introduced in a phase, it becomes work for people in the following phases. Those people in the pool below have to read the documents, understand them, do their work based on them and then pass them on to the next phase with the work they have.

This gets done over and over until the last phase (sometimes know as "bankruptcy") is reached. In other words, differnt groups of people have to re-learn stuff someone else has already learned and tried to document. The sad news is that not all of the really important stuff ever gets captured into the documents, and even worse, since most of the original thinking never made it into the documents in the first place there is a good chance we've misunderstood huge chunks of stuff. Not only is everyone re-learning what someone earlier had to learn, they are learning it differently than what everyone probably thinks.

There are ways to address this, and the best way I know is to have everyone learn and discover as much as possible at the same time instead of having one small subset of people try to learn everything, then passing it on to someone else who has to learn everything, then passing it on... and etc.

Suffers from the "telephone game" syndrome

Causes lengthy feedback loops

Documents are essentially "one-way" devices

Changes are difficult or impossible to inject