Back to newsletter 121 contents | All Javva's articles
Given Jack decided to focus on how centralizing performance management has become a trend, he asked me to write a bit on my experiences here. If you've been reading my columns you'll already have a taste of how this works: you have to be aware of lots of different systems and technologies, you have to be prepared to troubleshoot existing live production systems as well as pitch in to development teams for profiling and testing and explain (diplomatically) to architects and designers where they need to consider performance implications. And you have to market your service and successes to the company-wide development teams and the business and watch out for any attempt to slice the whole you are providing into slices that are more acceptable and those that are less acceptable.
So we're a business within a business. We need to sell internally. But we can't charge internally like a business, so we're not really a business. I guess we're more like a tax. Hmmm, I need to explain that I guess. Well, we looked at charging internally, after all we were definitely saving the business money and improving quality, so why not? Well, it turns out that if you start charging for an "optional" service where you can't provide the up front costs and benefits, then it can easily get dropped from downward shifting budgets. Of course we know that they'll need us sooner or later, but what happens next is that someone internally to the project basically gets assigned to performance duties. Not explicitly, of course, as the requirement was dropped (or "delayed") from the budget so they can't turn around and say this person will do performance management. No, it just happens implicitly. So now the project is paying a person who doesn't have as much expertise as we do, to do slower and more inconsistently what we could do better and faster. And cheaper.
So we don't charge internally. But we know how much we cost the overall business, and we keep detailed records of the improvements we've made and periodically go back to determine how much those improvements have benefitted the business in terms of reduced costs or increased profits. And that is probably the most difficult aspect of the whole thing. Improvements which save capacity can be costed relatively easily, but how about when potential throughput has been increased? The estimate can range from nothing (after all, it is just potential throughput, nothing is gained until the previous maximum throughput is exceeded) to an estimate based on future business projections (how much would handling the extra load have cost if extra capacity was required instead of us doing what we did) and on through to how much we saved by avoiding regulatory fines that we would have been levied had that old throughput been overwhelmed by some business overload and transactions were consequently queued and delayed beyond the regulatory limits.
In fact, I did just such an estimate for one of our improvements. Weevil was outraged at the higher cost saving estimates and fought ever so hard to push back to a level of zero benefits now, but future benefits calculated based on actual increased volumes within the next year. I agreed entirely to his proposal for two reasons. Firstly, he agreed that if volumes increased in the next year to above the previous capacity I could include an "avoidance of regulatory fine" calculation - and he didn't know that peak volumes had already increased to 80% of the previous maximum by the time we were having the discussion (we have increased capability five-fold beyond that with our tuning); but more importantly, the cost-benefit analysis for this year was already showing that we had generated more than 10x benefits compared to our costs, so I was delighted to push the benefits of this project into next year's cost-benefit calculation. Even in the extremely unlikely case that it never exceeds the old capacity over the next year, I'm not really down anything; but in the far more likely case that it does, I get the benefit into next year's analysis, and it's always nice to start with something in the bank - the sad fact is that nowadays you have to re-justify your extistence every year, no matter what past glories you have to show.
So, to summarize my centralized performance managing unit, we do what every project needs to do, performance-wise, but we also have to handle all the politics that goes with being an independent unit that needs to justify it's existence. It's hard work and benefits the company but am I better off? Well, I suppose being the person people come to company-wide for advice is fun. And the better money is good too. And the varied performance tasks we face are interesting. And the fact that I'm keeping us from being a sub-team of Weevil's QA unit is nice.
BCNU - Javva The Hutt. (LinkedIn profile http://www.linkedin.com/in/javathehutt feel free to connect me, my email as javva@ this java performance domain).
Back to newsletter 121 contents