If It's Painful, Do It More Often

I've worked with a number of teams who couldn't deliver. They could write code all day long for months on end, and they'd do just that. But then they'd spend a month or three shaking bugs out and still not be able to get the product into production. In many of these cases, the programmers just want to program. They don't want to or know how to do the messy task of deployment. Install and upgrade is not their core competency.

It's instinctive to cram more features into the release when it is just so hard to deliver. If you get few chances to deliver improvement, you want to cram as much in the release as possible. But that just makes it harder. More beautiful garbage to push over the goal line. This raises the inventory of unfinished work, the amount of work in process, which further delays deployment and hides the real problem.

I'm reminded of the Israelites when Moses instructed them to not save the manna that God provided from one day to the next. Do not keep an inventory of the stuff. But some of them didn’t listen and kept some of it until morning. But by then it was full of maggots and had a terrible smell. Software development with too much WIP is exactly like that.

Anyway, Lean software development reduces work in process in order to surface problems and reduce costs.

I like this illustration from page 7 in Mary Poppendieck's book, Implementing Lean Software Development. When you first look at it, your reaction might be "No! RAISE the water because there are rocks!" But the rocks are still there. Some day you are going to hit those rocks. Better to hit the rocks and deal with it in a dingy than in a yacht. Better to deal with the inability to deploy using a small release than a big one. It's better to lower the inventory, surface the issues and work to resolve them. Don't leave them there, lingering.

In eXtreme Programming we say that if is something is painful, do it more often. What we mean is that we should not avoiding dealing with problems. Solve them right away.

Getting Your Team to Take Short-Cuts

I was once in a software product company teetering on the edge of financial problems. And who in this business hasn't been? We had only one customer of consequence and they were unhappy with the cost of what was essentially custom development. This turned into pressure to deliver more features per dollar of cost. I.E., to develop the features faster. Right away. Pronto. How do you make a team of programmers immediately faster?

I had already looked at our process and had evaluated our use of best practices and our quality. We were following XP well, so if you subscribe to XP being a good set of best practices, we had that covered. Our development speed was average for enterprise software vendors using Java/Swing. But our quality was best in class. I am basing my evaluation mainly on the work of Capers Jones' recent books and metrics gathered from the team's history. That's about as objective as I could get. I also considered what I know of the industry and other groups, a more subjective measure. From what I've seen since, that team's velocity was actually quite good. To get the performance boost, there was only one thing left to do.

You may be able to get a short boost of speed at the expense of incurring technical debt, or, more technical debt than normal as your case may be. But to do that, to get a short boost in the rate of delivered features, you’ve got to paint a good picture of the situation to development. Why? Because you’ve got to get them to understand the seriousness of the situation and give them a vision of how long or how much or how many of what is needed so that they can buy-in to changing their habits and their preferred, ingrained and natural method of working. Saying "preferred way of working" makes this sound like a choice. A team's preferred way of working is indeed a preference, but it isn't much of a choice. It's a compulsion. You must realize when you demand speed, when you ask the team to take short-cuts, you are asking them to work-differently. You are asking them to work in an unnatural manner.

To illustrate: You are most likely right handed. Pick up your pen with your left hand and write a long letter to your spouse or your mom. Get the picture? The unnatural is difficult to achieve.

In Complete Control and Not Controlling

My regular followers know I post on lean/agile IT topics as well as bicycling and faith-based views. You agile IT guys – hang with me and I’ll tie a little agile into this before the end.

One of my favorite bloggers is Joe Gibson, co-author of the Ant Developer’s handbook. I like his opinionated writing on topics of interest to me: Java, Scala, Groovy, Ruby, Lisp, Koine Greek and religion. His own translation of old Greek manuscripts and commentary on other translations are quite interesting.

Joey posted a few days ago on how those who use the Bible too often "selectively edit scripture to fit a particular message, or to fit nicely with a pithy saying." This is essentially eisegesis (to draw in) rather than proper exegesis (to draw out). That is, reading stuff into the text rather than more appropriately getting out of it what the author intended. I was passionately into his post, in complete agreement.

Now Joey and I don’t see eye to eye on a number of matters of faith. Yet even when we strongly disagree there is usually some point in his writing that I strongly support. So I was with him right up until he said this:

...the minister was going on about how we need to "stop worrying" and just know that "God is in complete control" and blah blah blah… I’ve always hated that way of thinking. I despise the Sandy Patty song, "God Is In Control," because I do not believe that is true. (Bold emphasis added.)

So I crafted a nice and timely, intelligent, thorough and sound response. Joey’s blog seems to have a multi-step click-post, preview, click-post-again are-you-sure, captcha phrase kind of process that I obviously flubbed because my response never made it to the blog. Time has passed and now I’m here rambling on, many times over what I so eloquently put in my initial response.

So back to Joey. Joey’s next statement shows a misunderstanding of "complete control" and he has made quite a leap.

...if you believe that he is in "complete control" then you have to take that to its logical conclusion. If you believe that every good thing that happens to you is because God wanted it to be that way, then you must also accept that every bad thing that happens to you is because God wanted it that way.

The error here is the assumption that to believe that God is in complete control is to believe that everything that happens to you is because God wanted it to be that way. Let me explain. My kids run wild at times. This is not what I want. "Hey kids! Run around the house, fight some, and drive your mother and I crazy." It’s not what I want, but that’s what happens. I give them some room to exercise their free will. Notice I said some room. I have boundaries and my kids test them. I’m not controlling my kids, at least not most of the time. At least I try to not be controlling. Anyway, I assure you I’m in complete control of the situation at home and am quick hand out corporal punishment when boundaries are reached. In control, but not controlling.

A good agile manager or agile architect does likewise. They aren’t controlling. Rather, they set some boundaries, and otherwise let the team self-organize. They give the team much latitude to decide how to get the work done within some boundaries. Rather than hand out tasks, they encourage the team to select their own work tasks. Yet they remain in complete control.

[Sorry – I have no analogy for you bicyclists. I control my bicycle, but am never fully in control, much to the disappointment of my Dunwoody Cycling buddies.]

Notice that God told Adam that he is free. Free to eat of any tree in the garden but one. God gave man free will. But God also set some boundaries. So Adam was hanging out in this cool garden in complete fellowship with God Himself. But Adam poked God’s boundary so Adam and all of us now suffer the consequences. God’s will is for man to love God and obey his boundaries. God sends no one to hell. God’s will is that no one perishes, but that everyone accepts his free gift of salvation by faith in Jesus Christ alone. But for love and obedience to be real, there has to be a choice. A choice to disobey. Therefore, God is not controlling our every action and decision. And we will all suffer the consequences of my sin and your sin and Adam’s sin and the sin of others. God does not wish this on any of us.

Witness Job. God allowed Satan to bring disaster on Job and his family. But God set boundaries for Satan. "Go this far and no further" in effect. So God certainly allows bad things to happen, within boundaries. I wouldn’t wish Job’s trials on anyone, but I’m sure glad it happened. There are a lot of lessons for us to learn at Job’s expense, in particular, that God is in control.

Boundaries – God is not going to allow us to interfere with his plans. We can’t blow up the earth, for example, and kill everyone. We can’t wreck the environment such that we all die. Unless, of course, that is God’s plan for how the "first earth" will pass away (Revelation 21). But in that case, there is nothing we can do to stop it from happening either. But in any case, we’ve got boundaries and God is in control. We can certainly kill many and do big-time damage. But we can’t mess up his plans. God will intervene.

God does directly intervene at times. See 1 Kings 22:23 -- "You see, the LORD has put a lying spirit into the mouth of all these prophets of yours, and the LORD has pronounced disaster against you." The Bible says that what Joseph’s brothers meant for harm, God used for good. And there are the New Testament miracles.

This is getting long and a bit rambling, and my argument is beginning (or perhaps continuing) to trail off, so rather than wrap it up and put a nice bow on it like Joey does, I’m just going to quit here.

It’s The Values That Matter. Or Maybe It’s The Culture.

Mechanical Agile^[1] is when one tries to follow the process with no understanding of the theory behind the practices, with no understanding of why it is designed as it is. Daryl Kulak wrote the most excellent post "Five Symptoms of Mechanical Agile". Any team exhibiting the conditions represented there, certainly need help. But truly, any team having those problems is pretty far along. They are trying to be agile but are misfiring on some point. Many teams don’t get that far because they just don’t get it.

As with XP, Scrum can be followed mechanically, without any understanding of why. Have some iterations, estimate in points, make a pretty burn down chart, do a demo, and maybe a retrospective every other sprint “because we just don’t see the value of doing it every sprint”.

And they totally miss the point of agile.

Ken Schwaber said that teams implementing Scrum need to "...recognize the even higher degree of control, risk management, and transparency required to use Scrum successfully. I estimate that 75% of those organizations using Scrum will not succeed in getting the benefits that they hope for from it." (Emphasis in original.)

I agree with Ken. More is needed than is evident to most teams. In the context of the quote, Ken is pointing out what is needed to supplement Scrum in lieu of "controls and safeguards of waterfall and predictive processes." And there is more needed beyond just that.

Same thing with Kanban: model the process, put some swim lanes on the wall, and maybe put some WIP limits in. Visualize the workflow and break a log-jam from time to time.

And totally miss the point of continuous improvement and the metrics that enable such.

In his article, Daryl wrote that if you fall into mechanical agile, "[only] addressing the attitude of thinking of people as machines will help you." Daryl touches briefly on the topic of culture in his article, suggesting that cultural issues can be a problem and the need to change the culture as a solution.

Jeff Patton says that "Agile development is more culture than process". That is, if you ain’t got the culture, your Scrum/Kanban team may not be agile. Pay close attention to agile values and mismatches between those and the organization’s values.

That’s the key ingredient: the agile values. Or maybe it’s the agile culture. Not sure if values drive the culture or whether it’s the culture that influences the values. Either way, they go together.

When I teach XP in a time-limited situation I’m always torn between focusing on the four core values versus the practices. The practices are easier for people to understand. They add some value by themselves. They are concrete and give the audience something they can grab on to. But picking up the practices don’t make one agile. It’s the XP or Scrum or agile values that give you a fighting chance of understanding what you are doing the practices for. But starting with the values leaves most people scratching their heads. They don’t resonate with the average software practitioner.

The best chance you have of changing a culture, then, is with successful agile projects, mechanical or otherwise, punctuated with judicious and timely explanations of the values.

1. Dr. Hong Li coined the term Mechanical Agile which was first used publically by his co-author in an upcoming book, Daryl Kulak.