Sunday, April 18, 2010

Technical Debt - A non-technical managers guide

There are a few concepts that in my opinion any manager in a software company must understand, whether they are themselves of a technical background or not, and one of these is Technical Debt.

Technical debt is a phrase coined by the computer scientist Martin Fowler a few years ago. It surmises that just as when one borrows money they incurr a financial penalty that must be serviced later, as technology product grows in features and complexity it proportionaly becomes more difficult to extend and build upon unless the appropriate software design standards and architecture are maintained.

Occasionally it may be necessary to short change these practices in order to ship on time, but the "debt" incurred in this process must be "repaid" with time to re-factor and clean up the mess later, else suffer an ongoing cost via a higher defect rate, longer development time for new features, and ultimately developer attrition.

The analogy is an apt one - like financial debt, technical debt isn't fundamentally a bad thing. However conventional wisdom would have it, just as a financial debt should be paid down as soon as practical. to minimise the long term cost, so too should unnecessary complexity be stripped from software whenever possible through good architecture and regular refactoring in order to minimise the ongoing cost of managing future releases of the software. If technical debt is not accounted for, the result can be "bankrupt" projects that require complete re-writes in order to remain viable. This situation can be devastating for a startup that is rapidly iterating over just one one or two products.

Do you have techincal debt in your project? Here's a few Symptoms...

Progress is Slow. After a few cycles of feature packed releases, your software is now too buggy and/or difficult to maintain that no progress is being made, there's a good sign you've got debt on your hands. If you're using Scrum to manage your team, this is often manifest as a plateau on your burn-down charts.

Your defect rate is increasing. Often this is a corollary of the first symptom. As your product is burdened with unnecessary complexity due to rushed work or poor design decisions, mistakes are increasingly made. This same complexity means the mistakes are less likely to be detected until after they are in production.

Nobody wants to work on your product. This is often more noticeable in larger organisations, as the ambitious and talented developers look to younger, less tainted projects that are more fun to work on. Even in smaller companies, good developers have been known to resign over bad code.

How worried should I be about technical debt?

Just like financial debt, accruing technical debt can be a necessary and even appropriate strategy in product development, as long as it is in fact a strategy.

There's often a trade off that must be made between shipping a feature quickly and shippping one with minimal debt attached, and the right answer is not simply the latter. The right answer it will depend on the buisness as well as technical context. A prototype or early beta might reasonably trade high technical debt in return for quick implementation of new features. Getting these decisions right is what separates the wheat from the chaff of developers, and is absolutely crucial to the long term success of any software product and team.

Get it wrong and you'll end up with a product so loaded with technical debt it will be impossible to pay it off, or at the other end of the spectum an even worse fate - a product so beautifully engineered that it ran out of cash before it could even be shipped! Great teams therefore don't seek simply to minimise technical debt so much as manage it for appropriate conditions.

But a great team can only do this under the right conditions. Crucially, the whole team should have an appreciation for the significance of what they are doing, and the impact of, say dropping features or missing a deadline. Further, since technical debt unlike financial debt is difficult to quantify, the decisions in managing it must be made or at least informed by competent software engineers. Their opinions need to be understood and respected by anyone managing such a team. In a perfect world, every dev team would instincivley know when to pick the quick and dirty solution over the beatifully engineered one, or when to slow feature development to focus on fixing legacy dirty code. And in turn, a management team that measures progress by more than just features.

This last point is difficult for non-technical managers to follow, especially if they are inexperienced. Features are tangible positive outcomes, with obvious benefits. Technical debt by contrast isn't a tangibly obvious problem often until it is too late to do anything about it.

If you are not a technical manager then there's a few things you can do.

1. Make sure your engineers are accountable to business goals not software goals. Make sure your devs understand why they are building the features they are. Include them as much as possible in feature design and prioritisation. In a really well functioning team, developers will understand the buisness and it's customers so well that they can do these themselves.

2. Create a culture where it's okay for the guys on the ground to push back. While great developers recognize debt, nobody likes saying no to the boss. Good developers will argue that minimizing technical debt should be an eternal priority for the buisness, and (rightly) ensure they have the time and tools to do it. Make sure your team knows they are responsible not just for shipping features, but for informing you on the technical debt incuured in doing so. If they don't think something should be shipped, make sure a genuine conversation results around that.

3. Worry about it. It's a real problem, and your development team are probbably facing it - even if they are not telling you. If they aren't facing it, maybe they need more work to do :) Management must genuinely understand and buy in to the problem before it can be solved.

4. If you must incurr technical debt now, make sure you have a plan in place to pay it off at some suitably near point in the future, and stick to it.

Ultimately, it comes down to good management, and trust. Awesome projects projects are inevitably built on trust and a mutual understanding of the often competing priorities between a great development team, and the business they are working for.

1 comment: