JProfiler Get rid of your performance problems and memory leaks! Modern Garbage Collection Tuning Java Performance Training Courses COURSES AVAILABLE NOW. We can provide training courses to handle all your Java performance needs Java Performance Tuning, 2nd ed The classic and most comprehensive book on tuning Java Java Performance Tuning Newsletter Your source of Java performance news. Subscribe now! Enter email: Training online Threading Essentials course JProfiler Get rid of your performance problems and memory leaks!

Back to newsletter 185 contents

http://www.infoq.com/presentations/throughput-latency-memory-footprint
Explorations of the Three Legged Performance Stool (Page last updated February 2016, Added 2016-04-30, Author Charlie Hunt, Publisher QCon). Tips:

• Improving one or two out of throughput, latency and footprint will usually make one of the three worse. It may be possible to improve all three simultaneously, with a lot of work.
• GC pause time is mostly due to live objects, but object graph structure, reference objects and memory locality also have effects (old generation GC pause is also affected by the amount of fragmentation).
• Reduce pause time by reducing the size of Eden - but this will make pauses more frequent, reduce throughput and increase promotion rate. (You could also change the code to reduce the allocation rate).
• Young garbage collection frequency is determined by object allocation rate and the size of Eden. For the old generation, it's the promotion rate and the size of the old space heap.
• Java 9 has byte strings which seems to be better for all three of throughput, latency and footprint. In case it isn't for your app, you can turn it off with -XX:-CompactStrings.

http://engblog.yext.com/post/why-async
Why Async? (Page last updated March 2016, Added 2016-04-30, Author Jacob Fancher, Publisher Yext). Tips:

• An synchronous implementation of some function will only move the execution to another thread, not magically make the work disappear. However it let's you have more flexibility over scheduling and parallelism (though at the cost of readability).
• Some threads are special: UI threads, request handlers, message handlers, remote input; these should not be blocked for any significant length of time or you dramatically impact the system (freezing the UI or reducing throughput). So any blocking work for these types of threads should be spun out to other threads.
• Threads and thread pools generally have a purpose. If you want to rate-limit requests do so explicitly, not implicitly by blocking handler threads.
• Where available, nio (eg AsynchronousFileChannel) will use OS primitives to do I/O efficiently without requiring you to handle the threading.
• Rather than block threads waiting on external requests to complete, you can have a request handler which dispatches results as callbacks to a thread pool.
• An asynchronous interface lets you change the underlying asynchronous implementation without changing the callers of the asyncronous methods. This lets the underlying asynchronous implementation be optimised for the particular use case (considering thread pooling, nio, parallelism, rate limiting, system limitations).

http://techblog.bozho.net/general-performance-tips/
General Performance Tips (Page last updated December 2015, Added 2016-04-30, Author Bozhidar Bozhanov, Publisher Bozho's tech blog). Tips:

• Always monitor, measure and try to find bottlenecks in the running system.
• Anything that gets accessed many times but doesn't change often should be cached. Not just objects and data, but routines, eg database queries, requests for static resources, etc. Cache invalidation needs to be thought out carefully.
• Queuing provides a generic solution to many types of performance issues that involve limiting the use of shared resources.
• If it takes a non-short time, do it in the background so that your handler (or UI) remains responsive.
• Use EXPLAIN to see how queries are executed, identify unnecessary full table scans, create secondary indexes.
• Different marshalling/unmarshalling schemes have different costs. A 20% higher marshalling cost adds enormous overheads at scale.

https://plumbr.eu/blog/memory-leaks/native-memory-leak-example
Native memory leak example (Page last updated March 2016, Added 2016-04-30, Author Gleb Smirnov, Publisher plumbr). Tips:

• The continual growth of used heap memory after Major GCs is a signature of a memory leak.
• A stable size for all monitorable JVM memory pools but a continually increasing process size indicates a native memory leak.
• The JVM caches the original bytecodes of classes in native memory, not any of the monitored (heap) memory pools.

https://dzone.com/articles/architecting-for-failure
Architecting for Failure (Page last updated April 2016, Added 2016-04-30, Author Gaurav Purandare, Publisher DZone). Tips:

• Consider failures as an integral part of any system.
• Isolation is a powerful aid in anticipating and mitigating failures, providing clarity on resources utilized.
• Maxed-out resources can be an indicator for (imminent) failure.
• Component isolation combined with replication prevents applications from having single points of failure, so adding resilience and making failures less likely overall.
• Applications should always be developed to be prepared for a hardware failure.
• Sticky sessions imply that when a server goes down, all the sessions it is handling will fail; while the system overall might be resilient, it isn't for those sessions (and users).
• All services should have regular health checks - industry best practice is to do this at most every five minutes.
• If failure occurs with a database server (you should always assume the worst case scenario, the primary or master server are impacted), there needs to be a configuration in place to elect a new primary or master node. Application libraries need to adjust their database connections, which ideally should be an automated process.
• Following major industry blogs and key influencers on social media so that you can discover and address new industry issues swiftly after they have been identified.
• Having proper monitoring in place: daily sanity tests, automated port scanning and access controls, etc. Be sure to also monitor the monitoring scripts themselves.
• Try to determine the maximum scalability of different components in your system, at the very least all mission critical ones.
• Update your checklists with failure to make yourself smarter and your systems more resilient.

https://blog.twitter.com/2016/resilient-ad-serving-at-twitter-scale
Resilient ad serving at Twitter-scale (Page last updated March 2016, Added 2016-04-30, Author Sridhar Iyer, Publisher Twitter). Tips:

• If you have a computationally (or resource) expensive routine that needs to be processed as quickly as possible, break down the routine into less expensive but less accurate subroutines that can be run first which will make the more expensive full routine run more efficiently. Additionally, you can tune for speed versus overall accuracy by deciding how accurate the first subroutine is. For example a slow accurate filter will perfor faster with fewer things to filter, so pre-filtering with a fast but approximate filter to eliminate the obvious candidates makes the overall routine faster [EDITOR: also, from my book, an expensive but accurate slow sort will be faster if the elements are first partially sorted, eg by an innacurate but approximately correct fast sort].
• You can auto-tune a system if you have a parameter which can be adjusted that you can control, a measure that the parameter affects, and a target for that measure (and assuming the parameter and measure react within reasonable times). Then using a controller with a feedback loop, the parameter can be auto-adjusted to minimize deviations from the target. This can allow your system to adapt to dynamically changing resource contention (eg request surges).
• To make your service CPU bound rather than latency bound, make all operations asynchronous, reduce lock contention, etc, ie eliminate all blocking.
• Provisioning techniques for balancing cost vs service capability are: over-provisioning compared to average request rates in order to handle spikes; dynamically add provisioning for spikes and remove later; provisioning statically for average request rates but building your service to degrade but continue under spikes.

Jack Shirazi

Back to newsletter 185 contents