Comprehensive Documentation - Bad Assumptions

Agile is easy to dismiss. Many people hear something that doesn't sound right, something that sounds different than what they are accustomed to, and they develop misconceptions. They share their misconceptions with others. They dismiss agile without any more thought. A common misconception concerns documentation.

This came across my inbox recently:

I spoke to …, and he doesn’t have a requirement for Agile. He is familiar with the methodology, and his concern is comprehensive documentation. He believes that is a critical part of development that becomes less critical with the Agile methodology.  If his assumptions are incorrect, please let me know.

Are his assumptions incorrect? There are likely a lot of assumptions in that short statement:

• "Comprehensive documentation is a critical part of development." That may be true for his project. But it is not true for many projects. If he assumes that comprehensive documentation is too critical to do without on any project, his assumption is incorrect.
• "Comprehensive documentation becomes less critical with the Agile methodology." Often this is true. But that's a good thing. So, he's probably concerned about this…
• If he assumes you can't write comprehensive documentation on agile projects, his assumption is incorrect.
  You can write all the documentation you want.

Agile simply warns against using documentation as a measure of progress. Instead, agile favors working software as a measure of progress. If comprehensive documentation helps you get to working software, or adds value after the software is written, then create comprehensive documentation. If you have a regulatory environment you can't get around, then meet your regulatory requirements, but be creative and try to find an automated approach to generating it. If we're talking end user documentation and you can't make your application usable without end-user documentation, then write end-user documentation. If we're talking about installation instructions, agile favors automation, but if you can't automate and can't make it easy, then document it. If you have published APIs, create them with great care, provide great documentation and treat them as contracts.

If we're talking about requirements specifications, agile prefers requirements documentation in the form of user stories, constant collaboration between the givers of the requirements and the implementers of the requirements, and automated story tests. If you aren't there yet, if you don't know how to do that yet, then keep writing requirements documents for now, and strive for continuous learning and continuous improvement.

If we're talking about comprehensive technical documentation of the implementation, agile favors the software craftsmanship approach of refactoring towards small methods with very clear and descriptive method, variable, and class names. This leads to readable and self-documenting code. Agile also favors high code coverage through automated unit tests, which documents how to use the implementation and the intent of it.

Every instance of agile is unique. Each adoption of agile must be tailored for the context in which it will live -- the organization.

Usually, however, comprehensive documentation is waste. If the documentation doesn't match the source code, the documentation is wrong. Documentation is almost always wrong. Using misleading documentation is wasteful. Maintaining documentation is expensive and it rarely achieves a defect free state. More waste.

At the very least, this person shouldn't throw the baby out with the bath water. A lot of value remains in agile even if he disagrees with its warnings about documentation. I'm not saying that agile is suitable in his environment -- it in fact might not be. He may have no change or uncertainty in his environment. He may have a very repeatable and predictable situation, ideally suited for a plan driven approach.


In any event, this person is most assuredly concerned about how something will be handled, be it support (handled by pair-programming and a coded test suite) or design (handled by design-all-the-time, pair-programming, test suite, refactoring) or coordination (handled by collaborative space, published and documented APIs as contracts), or etc... I could assuage his fears if I new what was underlying them. That would require a face to face conversation. I'd welcome that opportunity. With him, or with any of you, my readers, or with any of your colleagues. My email address and phone number are noted at the top of my blog. Contact me at any time.

Scrum Fundamental: Planning and Iterating

This is my last post, for now, in this series on common issues teams have with fundamental Scrum concepts. Here are a few tips, reminders actually, for planning and iterating.

Make story estimates visible to the Product Owner. Understanding the relative costs of features helps make better trade-offs.

Product Owners reprioritize work not started. Always work on the highest value story and risk reduction stories first.

Plan a second release from the start. This takes the pressure off. “We’ve just got to have feature X!” becomes “ok, we’ll put feature X in the second release and not worry about it now.” You may never do feature X, or release two, but it helps the first release move along unhindered.

If you need to know when you’ll be done, make a build up chart for the release. See Alistair Cockburn’s article on this topic.

Minimize the amount of work in process (WIP). Limit this in two ways: In each iteration, have the whole team work on just a few stories (like, three to six, depending on size). I put some rules of thumb on this in an earlier post. Don’t start work on something that cannot be completed in the iteration. With traditional project management using Gantt charts, you identify and care for the critical path. With agile projects, you limit WIP and increase cross-training, teamwork, team goals and collaboration to care for the critical path.

In addition to limiting the number of stories in the iteration, limit how many stories can be in progress at any one time. Limiting WIP to, say, one-third of the number of stories committed to the iteration might be about right. Then work towards limiting it to just one.

That’s it for now. But there is so much more that can be said. I welcome your own thoughts and lists and links to other relevant information in the comments below.

Scrum Fundamental: Communication, Delegation

I’ve been blogging lately on common issues in teams practicing Scrum or otherwise trying to be agile, and not having complete success. Yesterday's post covered a hodgepodge of items very loosely related to measuring progress. Last week's post covered user stories. Today I thought I’d talk about communication and delegation.

Communication

The purpose of the daily stand-up/scrum is communication. This is very important. The idea is to keep the meeting short (so make everyone stand) and have everyone share just the info that most everyone else needs to know. This can often be done in five or ten minutes.

Some teams cover what they did yesterday, what they plan to do today and roadblocks. It’s useful to mention some significant change that they are about to commit to source control.

Teams that are co-located and that pair-program with collective code ownership and limited Work In Process don’t typically need to discuss those topics because everyone already knows what everyone else is working on. Those teams just talk about less about what they are doing and more about their impediments – things that are slowing them down.

Decide what is most important for your situation. Whatever your standard topic is, don’t go into much detail or resolve anything during the meeting. Those that need to follow-up or work together or have a deeper discussion get together afterwards.

Whatever your standard topic is, vary it. Don’t get stuck in a rut. If you usually go in order by person, once in a while go in order by story. Throw in a different question to answer, perhaps related to something you are trying to improve upon identified in a retrospective.

The standup is a start, but don’t stop there. Stop using email. Get out of your seat and speak to someone face to face. And working with the whole team in an open space is a really really really good idea.

Delegation

Let go of assigning tasks to people. Let the team manage task assignment themselves. People will choose the correct tasks most of the time. When they don’t, let that be a learning experience.

Let the team estimate the stories.

Let the team come up with the tasks.

Make time for yourself. Take time to think about and to improve the system. By ‘system’ I’m referring to the process and the organization, not the software you are developing.

Next

Next up should be a post on planning.

What is on your short list of common problems related to some fundamental aspect of Scrum?

Scrum Fundamental: Measures of Progress

In my user stories blog post last week I claimed that problems on Scrum teams are often due to lack of knowledge about a Scrum fundamental. Today I’ll tackle the next item in my short-list of common problem areas:

• measures of progress,
• delivering value,
• demos,
• getting feedback,
• keeping it working,
• continuous integration,
• and testing.

Yeah, so, that’s more than one item, but one kind of leads to the next.

Requirements and Design

One way, the traditional way, to try to build trust and understanding between a development team and the business or client is to spend a lot of time on designs, analysis, and wirefames. The problem is that it’s difficult to predict the future with much accuracy. You will likely miss details and misunderstand how the app will be used in practice. Requirements and priorities will change.

The agile prescription, therefore, is to not spend much time figuring out details for some story that you may never implement or that may change drastically in the ensuing months. Understand the stories at a high level. Get into development and begin delivering working software as soon as possible. Get feedback sooner and begin reaping the benefits of the software sooner.

Defer any decision and any work that can be deferred. You’ll be smarter next month, you’ll understand the system and the requirements better. Let the design speak to you, refactoring as you go.

This would be a good place to say something about lean and systems thinking and Pillar Technology’s “Business value over working software” appendage to the Agile Manifesto and about not building software when it would be better to redesign the system, without IT, then automate what makes sense. But that’s another whole post.

One good thing about working software is that you can have demos.

Demos

Favor demos over status reports. After completing each small story, demo it at least internally. Get to where you can have frequent (weekly or bi-weekly) demos to your clients/users. That early feedback is very important. This also holds staff accountable for quality, for “keep it working”, and for getting to done.

The demo isn’t for the Product Owner. Your Product Owner should have been involved with the team while the story was being implemented. Your Product Owner should have seen the story before the demo. There was an interesting discussion on the Agilistas LinkedIn group about who should do the demo, and why.

Keep it Working

Employ the “keep it working” principle. Build a usable system early on and keep it working continuously. The source in the version control system must always be usable. Set up an automated build/test process using a tool like CruiseControl, Cruise, or Hudson. Have it inform everyone when the build breaks. You should have a demoable product at all times. At the end of each iteration you should have a shippable product.

This requires automated testing. Pick a level with which to start coding test cases. Most often this is at the lowest level, unit tests. I recommend the xUnit framework or RSpec, but there are other good choices. [Hmmm. I feel compelled to say something about cucumber or BDD, but I intend this to be a fundamentals post. BDD ought to be fundamental, but it isn't as fundamental as this other stuff, particularly if I'm talking about Scrum. Admittedly, this post has left Scrum and is off in generic agile-land. But I digress.]

Sometimes, however, your methods are too large and have too many dependencies to be tested well at this level without first doing much refactoring. In those cases, automate your tests at a higher layer, then refactor and employ unit testing as you go. I’ve used Selenium and Rational Functional Tester for UI-based testing, but there are other choices out there. Working Effectively With Legacy Code by Michael Feathers comes highly recommended.

If you do a good job here, you’ll reduce the amount of manual testing you have to do. You can limit that to usability, break-the-box, and exploratory testing rather than regression testing.

Just a short word about testers. QA professionals think differently and approach testing in ways that developers do not. I like getting QA involved with the developers, pairing. I like getting development involved with anything that QA is taking the lead on automating. And I really like getting QA involved with the Product Owner to help define good stories and good acceptance criteria.

Parting Note

I almost abandoned this series of posts. There’s really nothing new here. I’ve said nothing better than it has been said before. I’ve just said it differently. Sometimes ‘different’ resonates with someone who hadn’t understood the way it was put to them before. So, I’ll press on with these posts. Unless I change my mind.

Scrum Fundamental: User Stories

In my line of work as an agile coach I get to help people who are just starting to use Scrum. Some common problems crop up with many of these teams. If often comes down to some Scrum fundamentals. Fundamental doesn’t mean easy. Professional athletes work at the fundamentals and so must we.

If I had a short list of common problem areas, the neglected fundamentals might be: good stories, “keep it working”, getting to done, collaboration, and planning. That’s a bit much for one blog post, so, maybe over the next few days I can address these in a series of posts. Here’s the first.

User Stories

Analysis of a business problem yields features that may either be “stories” or that may be broken down into stories. The first pass will have stories that have some very important parts as well as some less important parts. Identify and implement the most important stories first and address risk first. Defer as much as possible to the end of the project.

That sounds obvious, but it’s so easy for a developer to “just add this in while we’re working in this area.” It’s seductive to the Product Owner to want to “round out” some feature set. So many useless features and dead code gets bought and paid for that way.

Maximize the amount of work not done. Be ruthless in deferring what is less essential until later; you might not ever have to do it. Don’t build frameworks in advance or build support in for features you plan to work on later because You Aren’t Going to Need It – the well known YAGNI principle.

Split Stories

The approach here is to split your stories. The “User Management” story becomes “Add Users”, “Delete Users”, and “Modify Users”. You might never implement Delete, and Modify isn’t needed in the first release. You may further split Add User into basic and wiz-bang versions. This can likely be split further to deal with different types of users, assorted permissions, error cases, cascading, and more complex attributes of the user. After splitting, implement the most valuable stories first, of course.

Each story should have some value to your business/client/customer, particularly to an end-user. Each story should deliver some function from top to bottom. It should be a complete vertical slice through your application. Don’t divide stories along architectural lines. Avoid technical/horizontal slices (the DB framework, the complete object model, all the UI components). For the “user management” story, don’t have a story for the UI and a story for the backend.

Spike Stories

Use spike stories when you need to figure out how to implement or estimate something. A spike is a short time-boxed experiment or research, with a specific objective to address unknowns and risks. The conclusion of a spike may beget more stories and spikes.

Rules of Thumb

Some rules of thumb for story size:

• One story should not take the whole team more than half the iteration.
• The team should be able to finish any story in one to two days for a one-week iteration.
• The team should be able to finish any story in one to four days for a two-week iteration.
• Aim to have 3 to 8 stories for a one-week iteration.
• Aim to have 3 to 8 stories for a two-week iteration.
• If most of your stories can be done by a person in half a day, they are too small.
• If your team is doing more than 8 stories per iteration, your stories are too small.
• If your team is doing more than 9 stories in a three-week iteration, your iterations are too big.

Remember, these are just my rules of thumb. There are valid exceptions.

Summary

Poor stories will make Scrum painful. I’m not the first to write about good user stories, not the most eloquent, and won’t be the last. If what I’ve written describes your problem, read more about it or find a good coach. I’m available.

Question for the day

Disagree with my recommendation? Have a better way to word this? Hate my rules of thumb? Have a better blog post on user stories? Post your thoughts or post a link.

Scrum Master? Or coach?

I've seen many more open positions for Scrum Masters than I have agile coaches. I've been asked to be a Scrum Master more than I've been asked to be a coach. Is it just marketing? Scrum has market share; it's widely known and understood. Does the market distinguish between a Scrum Master and a coach?

I often find that companies looking for a Scrum Master want someone to run their standups, planning and retrospectives. They want an agile project manager to manage the projects, to report status, to track budget.

They seem to not want the change agent and process improvement aspects of the Scrum Master role. Or, they want a little change agent and process improvement to happen as long as it's directed towards the team and not management. And anything directed toward the team mustn't involve facilities or pair programming. Or HR. Or empowerment.

But isn't that what a Scrum Master is supposed to be, a coach?

I dislike this dichotomy.

There was a recent discussion on the Agile Alliance discussion group on LinkedIn about the career growth path of a Scrum Master. The question was, is there a growth path? I liked Irene Michlin's response:

Have they all brought their teams to "performing" stage? Process is perfect, as in "nothing left to improve" a constant motive in retrospectives?

They are either all very brilliant and need to go and become trainers and coaches, or they don't get what it means to be a Scrum Master.

If there is something left to improve, then your job as a Scrum Master is not done -- you haven't mastered Scrum Mastering. Or, at least you aren't done coaching. But there is that distinction again.

I dislike this distinction between Scrum Master and coach. Unfortunately, I find it useful. It's a tell, as in poker. It helps evaluate an opportunity. It's how the conversation goes. "No, I won't be your Scrum Master, but I'll gladly train your team to handle this role. I'll be your coach. Are you ready for a change agent?" Sigh. I find it useful, therefore I tend to perpetuate the distinction.

What should we do? Not use the term coach and elevate the term Scrum Master? Should we live with it because it is what it is? Once the dichotomy is there it's not possible to kill it. Is there a useful distinction between the two?