Back to newsletter 071 contents

http://blogs.sun.com/fatcatair/entry/micro_benchmarks_just_say_no
Microbenchmark Adventure (Page last updated August 2006, Added 2006-10-31, Author Steve Goldman, Publisher Sun). Tips:

• Microbenchmarks are easily wrong as often the compiler is able to remove work that could never be removed in a real app and so the benchmark predicts more performance than you actually get. In some cases it also does the opposite.
• HotSpot can use prediction to assume a single type at a potentially polymorphic call is used, and compile for that (it can even inline it). If the usage pattern changes going from monomorphic to bi-morphic hostpost will back out the compile and recompile. Later still id that gets proven incorrect HotSpot can go mega-morphic with a v-table call. Such behavior can directly impact microbenchmarks.
• A switch with static monomorphic calls is not object-oriented, but can outperform a polymorphic call [not recommended except in the most extreme cases].
• Sanity check the results of any performance test: your stats reporting code could be buggy!
• Doing calls from JNI to Java has a similar cost to doing them from Java to JNI, so count calls happening both ways when you are counting Java/JNI transitions.
• Repack 2-d arrays into 1-d arrays prior to moving it to JNI if you are suing 2-d arrays.
• [The optimized Java benchmark here shows pure Java performing more than twice as fast as a C implementation]
• Because a Java compiler can optimize the code based on its running characteristics, it s possible to apply optimizations that cannot be determined from a static analysis.
• Calling from Java->native is more expensive than calling from C->C (unsurprisingly).

http://www.ociweb.com/jnb/jnbSep2006.html
Discover the Elegant Simplicity of JSR 166 (Page last updated September 2006, Added 2006-10-31, Author Nathan Tippy, Publisher OCI). Tips:

• The concurrency package is not intended to replace the traditional use of synchronize, wait and notify etc., it gives the developer an additional set of synchronization choices.
• Compare and Swap is an atomic operation that checks to ensure that the existing value is what it expects it to be and then replaces it with the new value unless it has already been changed by another thread.
• AtomicLong is a thread safe sequence number generator that uses Compare and Swap to avoid synchronization.
• Compare and Swap is usually used in a loop like this: while(true){currentVal = get();if (compareAndSet(currentVal, newVal)) return currentVal;} - if the value matches what the real current value is then the update is done but if they do not match then a new snapshot value is retrieved and the update is attempted again.
• Compare and Swap is less desirable under conditions where there are large numbers of threads contending on relatively few resources but in most applications this is not the case.
• Runnables should be designed to exit on their own after receiving some sort of predefined message or flag, their thread should not be killed.
• CountDownLatch is designed for cases where every thread needs to wait until the count goes down to zero.
• CyclicBarrier does the same thing as a CountDownLatch but it can be used multiple times.
• Using an iterator without synchronization on a List is not normally thread safe, but it will work on a CopyOnWriteArrayList.

http://www-128.ibm.com/developerworks/java/library/j-jtp09196/index.html
Instrumenting applications with JMX (Page last updated September 2006, Added 2006-10-31, Author Brian Goetz, Publisher IBM). Tips:

• JMX is a standardized means for providing a remotely accessible management interface and is an easy way to add a flexible and powerful management interface to an application. JMX components are called managed beans or MBeans.
• The jconsole tool can act as a universal JMX client.
• From Java SE 5.0 the JVM is instrumented to provide easy visibility into the state of memory management, class loading, active threads, logging, and platform configuration. Monitoring and management for most of the platform services are turned on by default (the performance impact is minimal), so it is just a matter of connecting to the application with a JMX client.
• JMX specifies multiple protocols that can be used to communicate between the MBeanServer and the JMX client, built-in transports are available for local connections, RMI, and SSL are provided, and it is possible to create new transports through the JMX Connector API.
• To create a managed service, you need to create an MBean interface specifying the management methods. You can then create an MBean that implements that interface, instantiates it, and registers it with the MBeanServer.
• JMX imposes some naming restrictions; for instance, MBean interface names must end with MBean, and the MBean class for the FooMBean interface must be called Foo. (You can lift this restriction by using a more advanced JMX feature, dynamic MBeans.)
• Registering an MBean with the built-in JMX implementation is straightforward: MBeanServer server = ManagementFactory.getPlatformMBeanServer();server.registerMBean(myMbean, new ObjectName("myapp:type=myMbean,name=foo thingy"));
• JMX figures out which properties are read-only and which are read-write, and jconsole allows you to modify the read-write properties. If the setter for a read-write property throws an exception (such as IllegalArgumentException), JMX reports the exception back to the client.
• [Article shows a TrackingThreadPool implementation, a subclass of ThreadPoolExecutor that keeps track of which tasks are in progress, as well as timing statistics for tasks that have been completed, by overriding the beforeExecute() and afterExecute() hooks and providing getters to retrieve the collected data].

http://conferences.oreillynet.com/presentations/os2006/secoske_matt.ppt
Open Source Performance Monitoring Tools and Tricks for Java (Page last updated July 2006, Added 2006-10-31, Author Matt Secoske, Publisher O'Reilly). Tips:

• Plan for Performance: Determine your performance goals; Create testing scenarios; Determine monitoring/profiling needs; Integrate into development process; Integrate into production environment.
• What to monitor (Hardware): CPU; Memory; Cache Hits/Misses; Disk/Network speed.
• What to monitor (Java Specific): GC; Application specific metrics.
• Profiling Tools: Load Testing / Driving; Logging / Log Analysis; Contained Profiling (Profiler wraps Application); External Profiling (JVMPI); java.lang.instrumentation.
• JUnitPerf is great for running benchmarks against a particular test case (or cases) while refactoring, not so great as an over-all monitoring solution.
• Load Testers: Apache JMeter; The Grinder.
• Use the GlassBox inspector in jconsole to monitor MBeans.
• NetBeans profiler is a good 5.0+ profiler; Eclipse TPTP is also usable.
• Put in just enough metrics to get your performance measurements
• Performance Test != Production
• Keep some monitoring in production
• Know when and what to optimize ? comes from experience and profiling
• Make performance a part of your development process

http://www.ddj.com/dept/windows/192700235
Application Responsiveness (Page last updated September 2006, Added 2006-10-31, Author Joe Duffy, Publisher DrDobbsJournal). Tips:

• Positive user experiences are linked to application responsiveness; frustrating user experiences are often caused by poor responsiveness.
• A responsive UI is snappy, responds to input promptly, and doesn't leave users hanging.
• Being responsive to the user does not necessarily mean completely processing user input, but rather simply giving the sender some acknowledgment of receipt. In cases where full processing takes time, it might mean giving users an estimated completion time and ongoing progress using, say, progress bars.
• UI blocked time should be predicatble - the maximum processing time for any event that might occur should not severely impact the user's happiness with your UI
• Variable response times are problematic and can be almost as frustrating as applications that are all around sluggish.
• Anything the UI thread does impacts its ability to keep up with incoming events.
• UI event handlers execute on the UI thread, and to avoid blocking the UI it is often prudent to offload processing to a separate thread.
• The general cause of event-processing bottlenecks is executing unbounded or high-latency operations on the GUI thread.
• Testing with small sample data sets may lead to acceptable performance, but often users operate on more data than you anticipated, and this can lead to significantly longer compute times than expected, and a blocked UI.
• Device I/O has a higher cost than most CPU- and memory-based activities, usually by several orders of magnitude; Doing I/O on the UI thread is almost always a mistake.
• Under system-wide memory pressure ordinary memory operations can suddenly turn into disk I/O due to page faulting.
• Data- and compute-intensive tasks should be done on a separate thread from the UI, even if that incurs overhead from worker synchronization.
• Locking in the GUI thread can easily lead to bad performance, the thread holding the lock might be performing I/O or waiting for an event. This essentially looks as if the GUI thread itself were performing such operations because it has to wait the same amount of time.
• Deadlocks are the worst type of lock contention, especially on the GUI thread.
• It is often important to support cancellation.
• A blocked GUI, even with an active progress bar can lead to user frustratation.
• A responsive application should provide a way to cancel the operation if the user decides they do not wish to wait for it to complete.

http://www.informit.com/articles/article.asp?p=517378&rl=1
Using Scatter Charts to Recognize Patterns in Performance Test Data (Page last updated July 2006, Added 2006-10-31, Author Michael Kelly, Publisher informit). Tips:

• Filter the data by turning different variables on and off.
• Filter the data by looking at one variable at a time.
• Filter the data by looking at different combinations of variables.
• Filter the data by changing the colors of the variables to help make sense of the data.
• Changing the scale: increase or decrease the scale; use a log scale; Change your units of measure.
• Try inverting the axes of the graphs.
• Correlate multiple sources of data for the same run: Overlay your performance test transactions with data from other monitoring tools (What?sUp, Interscope, sysmon tools, etc.).
• Correlate multiple sources of data for the same run: Overlay your performance test transactions with difference types/sizes of data than your test used (if possible).

Jack Shirazi Back to newsletter 071 contents