|
|
|
|
Back to newsletter 029 contents
Extraordinarily, the Java JIT Compiler optimization which enabled last month's benchmark to work as listed there has been backed out in version 1.4.2 of the JDK. So sadly, that benchmark only works with 1.4.1 JDK, which unfortunately makes that already controversial page somewhat useless.
Last month, our question of the month on the performance of Java against other languages showed an empty loop benchmark (Note that if you tested the initially published version, there was an editing mistake that meant it didn't work as advertised, but the version listed now should). This benchmark lead to some expressions of annoyance, disbelief and downright disgust. This month we'll consider those concerns.
The empty loop benchmark from last month's QOTM is NOT an April fool joke. It is completely serious (sort of). Try it, you'll see it works. Read the code, it is simple and comprehensible in all the languages. There is no coding trick involved.
The source of any confusion you might have about it is that, like all benchmarks, this benchmark is misleading. It is not measuring quite what you expect. If you read the code, you probably assume that the benchmark measures the time taken for any language to iterate a loop counter. In fact the benchmark measures the ability of a language to optimize an application by eliminating inefficient instructions. The Java server mode JIT compiler eliminates the empty loops completely from the JIT-compiled code. Had I chosen to use the client mode 1.4.1 JVM (client is the default mode), the loops would not be optimized away. Had I chosen to use Microsoft's C++ compiler, the C version would take no time. Clearly, the choice of compiler is critical.
The infamous PetStore .Net vs J2EE benchmark actually measured whether Microsoft's technicians or those from TheMiddlewareCompany were better at optimizing a toy version of an enterprise application. The empty loop benchmark effectively measures the maturity of Java's HotSpot compiler. It shows how Java's HotSpot compiler is now mature enough to start competing with the mature compilers of other older languages.
You might say that this is a trick, that empty loops are a special case which Java specially recognizes. Certainly it is true that in the past C compilers have been built to recognize well known benchmarks and produce specially optimized code for those benchmarks. Has Java done the same here? No the HotSpot compiler does not have a "special case" optimization for this benchmark. At a high level HotSpot can look for code which does not affect the state of a program, and can optimize that code to eliminate redundant instructions. For example, calls to empty methods are quickly eliminated by HotSpot. Instructions which simply assign values to variables can be eliminated by HotSpot if those values are not subsequently used. The empty loop benchmark demonstrates a general class of optimizations that HotSpot can apply. Note that the empty loop is not a No-op. There is an integer increment. The compiler needs to recognize that the variable does not affect the state of program and can safely eliminate it.
Can other languages apply these optimizations? Yes of course. Every programming language ultimately converts source code from that language into instructions that some computer understands. One way of characterizing the efficiency of a language is to take comparable source codes, and determine the efficiency of the subsequent machine level instructions issued by the language runtime execution. Comparable source codes could produce identical machine level instructions if the languages have support for the optimizations applied. In the benchmark that is what I did. The results are that some languages with some compilers do not provide optimal machine level instructions. Java with server mode HotSpot does. Microsoft C++ compiler does too, and since this optimization is fairly basic I would actually expect many other compilers have the capability too.
So does all this mean that Java is really faster than other languages? Hundreds of times faster? Well, that was just being cheeky. The truth is that Java is faster for some things. I have no doubt that there are other things it may not be faster for. Personally, I now find Java fast enough for everything (though like any program in any language you may need to do some tuning to get there). And I expect that sooner or later, it will be faster for everything that matters. Which sounds pretty good to me.
Finally, there were some of you who wanted a complex application comparison of Java vs C/C++ for some reason or other. You can find one over at http://www.rolemaker.dk/articles/evaljava/. But you're fooling yourself if you think such a comparison makes any difference. If you are trying to justify your language (whichever) based on it's speed compared to other languages, then have fun. The only language you can justify is Assembler (which is basically prettified machine code). Which is why game coders have traditionally written their tightest loops in Assembler. If you intend to switch to become an Assembler programmer on the basis of it's speed advantage, good luck to you. Perhaps I can recommend SIMPLE instead:
SIMPLE is an acronym for Sheer Idiot's Programming Linguistic Environment. This language, developed at the Hanover College for Technological Misfits, was designed to make it impossible to write code with errors in it. The statements are therefore confined to BEGIN, END, and STOP. No matter how you arrange the statements, you cannot make a syntax error. Programs written in Simple do nothing useful. They thus achieve the results of programs written in other languages without the tedious, frustrating process of testing and debugging.
From an article which appeared in the November 2, 1984 edition of the Waterloo mathNEWS, Author unknown.
The JavaPerformanceTuning.com team
Back to newsletter 029 contents
