Tuesday, August 4, 2009

Effective Software Organization

As a software development organization, we have to be effective. We owe effectiveness to our employers who pay our salaries and we owe effectiveness to our communities in order to ensure that NDS continues to prosper in Southampton, Jerusalem, London, Bangalore, and Costa Mesa for many years. But first and foremost we owe effectiveness to ourselves. Spending nine or more hours per day, five days per week on fruitless activities would mean we are willing to squander our time and health for nothing. To be ineffective is simply against our dignity.
In reality, no sane person would sincerely want to be ineffective. We just differ in our opinions on what it means to be effective and how to attain effectiveness.
The textbook definition of effectiveness is fairly simple: do the right things reasonably well. Doing wrong things in the most optimal way is a recipe for disaster. “What is right and what is wrong” is too often treated as a philosophical or even theological question. NDS, as a commercial software development organization, could use a more concrete indicator of effectiveness: either we sell or we don’t sell. We either sell for profit, or we go bankrupt. If we do sell for profit without jeopardizing our long-term capabilities, we do things right, we are effective, and if don’t - we are ineffective.
How much profit do we need? Following is a simple calculation which is very inaccurate, and cannot be used for the company finance management, but will help us understand what is effective, and what is not. Let’s start with some numbers: Ninety percent of startups fail during the first five years. Nine out of ten startups that survive the first stage (from five to ten years) die during the second stage in a so-called chasm where despite the number of successful sales, they still cannot acquire enough of a market share. Therefore, in order to barely break even, we ideally have to get a 100-fold ROI on successful projects. Consider $100K as a company cost per man-year (again, a very rough number, but convenient for educational purposes). For every man-year invested in development, we have to get $10M in return. This sounds like a lot, right? However, keep in mind that Google, with all its billons today, started from a ~$30M initial investment. This high ROI is required in order to compensate initiatives and trials, which will fail for one reason or another. Look at how much money Microsoft spent trying to penetrate the consumer electronics market with such little success. What would happen to the richest man in the world if MS Windows would not be so profitable? Without experimentation and endlessly trying new things, we cannot survive in the long run. Any product, regardless of how successful, has a limited life span and one day will stop bringing in money.
Ok, we have to sell, but to whom? Who is the customer? We are in the digital media business, and we collectively provide an information and entertainment service to the consumer. The consumer is THE customer, she pays the bill, and she must be served in the best possible way. However, this information and entertainment service is not a tangible object but rather an emerging property of collaboration among numerous hardware, software, and human elements. No single element could claim the ultimate credit for providing this service. It’s a systemic effect, which cannot be easily dissected into separate units without losing some essential properties of the whole. As a corollary, any attempt to optimize a local technological silo at the expense of other system elements would be plain wrong - and a lack of effectiveness.
It is very honorable to keep the consumer as our topmost priority, but we, R&D, most likely do not have any opportunity to contact her directly. We do not supply our products to consumers, and we seldom meet with TV operators, who are the closest possible approximation of the target market. It’s indeed a challenge, but we are hardly unique in this situation. In reality, very few people could optimize globally any reasonably complex commercial system. What usually happens is that all participants optimize communication with their neighbors, letting the whole system self-organize. Modern science states that open systems do have a good chance of self-organizing provided that enough energy and matter is supplied and internally produced waste is removed at proper time intervals1. This is the best advice we could get from modern science: optimize your “metabolic” delivery process rather than structure, strive to a constant smooth flow of exchange between closest neighbors and remove any obstacles that prevent it.
We are now approaching the focal point of this article. Effective development organization is a delivery-oriented organization that optimizes its “metabolic” process of exchange on its’ internal market. If we do not deliver early and often, the software products we develop will poison us on the inside. And, as with cholesterol, they will completely block our vital arteries. Without a clear delivery plan and commitment we should not even start developing. That is not to say that we should avoid research and experimentation in our professional field, but that’s another story. The bottom line is, when we start developing, we deliver.
How and who decides what to deliver? If we try to provide everything for everybody, we would never be effective. Remember the 100-fold desired target? This means that we need as many deployments as possible for the least effort possible. Who can decide this? We all know that “if everybody decides, nobody decides”. Does management decide? Yes, to a certain extent. The top managers are accountable for ROI and therefore have a real stake in deciding what to develop and how many resources to allocate. However, making day-to-day prioritization decisions requires certain professional skills and a huge amount of time. Release planning by functional management is hardly an effective practice. There is a special role for this activity, sometimes called “Product Owner” or “Product Manager”, whose primary responsibility is to decide which features to deliver to whom and when. Feature is a capability of our system under development, which satisfies some needs of some stakeholders. The most important features are those that satisfy the most pressing needs of the most important stakeholders. Under this model everything is negotiable, and everyone is a stakeholder. Today, effective product management is the number one success factor of the whole organization.
Initial prioritization is necessary, but not sufficient. Without addressing the systemic effect of multiple feedback loops, even the best product management will still be ineffective. Once we satisfy the most pressing needs of the most important stakeholders, the whole system changes around and the importance of needs and stakeholders needs to be re-evaluated. Therefore, proper time boxing of delivery is a matter of survival.
How often should we deliver in order to be effective? How long should we plan for? How can we ensure that we develop what we decide and deliver what we develop? These are not easy questions. Each one deserves a separate post of itsown. May be some day ...

No comments:

Post a Comment