JProfiler Get rid of your performance problems and memory leaks! Modern Garbage Collection Tuning Java Performance Training Courses COURSES AVAILABLE NOW. We can provide training courses to handle all your Java performance needs Java Performance Tuning, 2nd ed The classic and most comprehensive book on tuning Java Java Performance Tuning Newsletter Your source of Java performance news. Subscribe now! Enter email: Training online Threading Essentials course JProfiler Get rid of your performance problems and memory leaks!

Back to newsletter 307 contents

Generic third-party API tuning can be summarized as "batch, cache and minimize data retrieved, and measure latency". DB comms tuning is a specialized version of this, adding specifically to avoid query loops, check SQL efficiency and optimize for/with indexes, stored procedures and query plans. LLM calls are a new specialized version, and there are huge efficiency and cost benefits from tuning.

The tuning required is extensive but can be fully automated: Pre-compute context by extracting useful facts ahead of time, indexing them, and let the model retrieve compact, answer-ready context instead of sending raw documents through the API. This reduces tokens (LLM bandwidth), context needed (LLM memory), round trips, and results in faster retrieval, lower cost, and actually produces better outcomes - the model applies reasoning to the right facts instead of to eliminating irrelevant facts!

Now on to all the usual newsletter list of links, tips, tools, news and talks, and as usual I've extracted all the tips into this month's tips page.

A note from this newsletter's sponsor

News

Java performance tuning related news

• LLMs waste effort searching, reading, re-searching, and only then reasoning. Pre-computed context provides huge efficiencies
• Analysis and alternatives for TLS stack object management
• Serverless AWS performance tuning

Tools

Java performance tuning related tools

• JFR Shell - An interactive CLI for exploring and analyzing Java Flight Recorder (JFR) files with a query language
• The mathematics of backlogs
• K8s zero-downtime: rolling updates, readiness/liveness probes, graceful shutdown

Articles

• The Next Evolution of Java: Achieving Hyper Performance and Efficiency in Cloud Native Workloads (Daniel Oh/Devoxx)
• Achieving Microsecond Latencies with Java (Stefan Angelov/Devoxx)
• How to trace a Java Memory Leak (Victor Rentea/Devoxx)

Jack Shirazi

Back to newsletter 307 contents