Back to newsletter 101 contents | All Javva's articles
Have improved the throughput on the claims processing system five-fold, using all the usual tuning techniques, caching, elminating contention points, pooling, a bit of code speedup. All is in the next release, I have put all the tasks into the release build, so in QA now.
QA cycle complete, now going to release in 2 days, it's a big jump in capability, not just increased throughput, but existing throughput is faster too. Have only noted there are 'some' performance improvements in the release, am awaiting post-release measurements before I send out a nice 'look what I did' circular.
Release went out yesterday. The proverbial hit the fan. I'm not entirely sure yet, all I know is that it was disastrous, we all have to check our release items, there's a total change freeze on everything until the issues have been resolved. The freeze order came from top management, so stuff is serious!
Things are clearer now. The release caused a serious issue. Phantom claims of some kind were generated. Actual money was lost. Enough money that it went to the board level. Mind you, nowadays that probably isn't that much money anyway, but rumours here are of millions. Anyway, turned out to be in a piece of code that sneaked in, only gets exercised in the deployed system and never in QA due to lack of coverage. Board level meetings on our QA coverage, people buzzing around looking very downcast, someone got a serious telling off. It's mostly explained, except for one puzzling aspect. The post-mortem analysis explained everything except for one detail - the money flew out the door a lot faster than they would have expected it to flow, given the usual processing rate of the system. It's apparently very puzzling. People are jokingly calling it the credit crunch system because it seems able to process losing money four times faster than it can when the money was flowing in.
The system was reverted to the prior release, and now all items in the future releases are on an absolutely only-definitely-needed-and-reviewed-multiple-times-and-QAed-thoroughly procedure, so all of my changes have been left for much later. I shall feed them in, bit by bit. There will be no big bang speedup, it will be a slow trickle of improvements over time. Probably best all round, I think. Making a system faster at losing money is probably not something I want on my CV.
BCNU - Javva The Hutt.
Back to newsletter 101 contents