JProfiler Get rid of your performance problems and memory leaks! Modern Garbage Collection Tuning Java Performance Training Courses COURSES AVAILABLE NOW. We can provide training courses to handle all your Java performance needs Java Performance Tuning, 2nd ed The classic and most comprehensive book on tuning Java Java Performance Tuning Newsletter Your source of Java performance news. Subscribe now! Enter email: Training online Threading Essentials course JProfiler Get rid of your performance problems and memory leaks!

Back to newsletter 242 contents

https://netflixtechblog.com/keeping-netflix-reliable-using-prioritized-load-shedding-6cc827b02f94
Keeping Netflix Reliable Using Prioritized Load Shedding (Page last updated November 2020, Added 2021-01-30, Author Manuel Correa, Arthur Gonigberg, and Daniel West, Publisher Netflix). Tips:

• Some common failures are: misbehaving clients that trigger a retry storm, an under-scaled service in the backend, a bad deployment, a network blip, issues with the cloud provider.
• Prioritized load shedding works by: Consistently prioritize requests across device types into NON_CRITICAL (does not affect customer experience), DEGRADED_EXPERIENCE (affects customer experience but doesn't prevent primary customer operations), CRITICAL, and score between 1 to 100 for each request given its individual characteristics; progressively throttle requests based on priority; validate by chaos testing (deliberate fault injection) for requests of specific priorities.
• Requests with higher priority get preferential treatment. Under failure situations, the higher priority requests get served, while the lower priority ones will not. Request throttling is done at the system entry point (the gateway).
• The gateway functionality is critical: if it goes down, no traffic can get through to backend services, resulting in a total outage, Consequently the gateway throttles aggressively using CPU utilization, concurrent requests, and connection count metrics. When any of the thresholds for those metrics are crossed, the gateway will aggressively throttle traffic to keep itself up and healthy while the system recovers.
• When traffic is dropped, a signal is sent to clients to let them know to back off. The signal states how many retries clients can perform and what kind of time window they can perform them in.
• Priority load shedding is repeatedly re-tested using A/B testing so that new features are categorized correctly and bugs caught.

https://www.youtube.com/watch?v=1LUae4wSZyM
Performance Costs of Synchronized Methods in Java: JDK Mission Control & JVM Flight Recorder Example (Page last updated December 2020, Added 2021-01-30, Author Cameron McKenzie, Publisher Mission Control). Tips:

• An example of starting with Flight Recorder profiling on is to use -XX:+FlightRecorder -XX:StartFlightRecording=delay=2s,duration=60s,name=Latency,filename=recording.jfr,settings=profile -XX:FlightRecorderOptions=stackdepth=128
• JMC will process JFR recordings and open with some auto-analysis to indicate obvious problems that might exist - this is a good place to start.
• The "Lock Instances" tab will show you when the locks were blocking the application from proceeding, and selecting any lock will give you the stack trace where the lock was blocked
• In the "Java Application" tab you can highlight a set of threads and view just that set in a graphical coloured view which shows which threads are running and which are blocked all together in a combined timeline.

https://sematext.com/blog/jvm-performance-tuning/
JVM Tuning: How to Prepare Your Environment for Performance Tuning (Page last updated October 2019, Added 2021-01-30, Author Rafal Ku?, Publisher Sematext). Tips:

• Before tuning a JVM you need to eliminate significant memory leaks, as they will otherwise always lead to garbage collection issues regardless of tuning.
• Main JVM tuning parameters: -Xms, -Xmx, -XX:+AlwaysPreTouch, -XX:MaxMetaspaceSize, -XX:MetaspaceSize, -XX:MinMetaspaceFreeRatio, -XX:MaxMetaspaceFreeRatio, -XX:NewSize, -XXMaxNewSize, -Xmn, -XX:+Use*GC; and other useful parameters include: -XX:+UseStringDeduplication, -XX:+UseLWPSynchronization, -XX:LargePageSizeInBytes, -XX:MaxHeapFreeRatio, -XX:MinHeapFreeRatio, -XX:SurvivorRatio, -XX:+UseLargePages, -XX:+UseStringCache, -XX:+UseCompressedStrings, -XX:+OptimizeStringConcat
• Before you start tuning the JVM, set performance goals for latency, throughput and footprint, but be aware that you will tradeoff these against each other, eg high throughput and low latency results in higher memory usage; high throughput and low memory usage results in higher latency; low latency and low memory usage results in lower throughput.

Jack Shirazi Back to newsletter 242 contents