Published September 2003, Author Scot Mcphee
Page 2 of 2
previous page: Timers and audio performance
Therefore, how was the ‘elapsed time’ or the audio recording being measured? Even if your computer provided to the JVM a millisecond-resolution implementation (and some computers/JVM combinations, especially PCs, aren’t anywhere near that good), 44.1 samples could pass you by before you can do anything about it.
This scenario was compounded by the use of our old friend, Thread.sleep(). The author was presumably kicking off the recording, putting a timer thread to sleep() for x seconds, recording some audio, waking the timer, and measuring the number of samples recorded. The audio at that point would be ‘time-stamped’ with the assumed value of milliseconds elapsed and the whole cycle repeated.
Naturally, this is a wildly inaccurate design for this purpose. As well as the sample-rate being much more finely grained than the timer, we have the compounding issue that when a sleep() timer expires, it doesn’t actually mean that the thread will immediately start executing. It only places the thread on a runnable queue. It should therefore come as no surprise that that the time interval could not be accurately measured.
In the discussion, the recommendation was to put the thread to sleep, wake it, and measure the number of sample frames recorded, and stamp the end of the cycle with the number of samples, rather than the stamping the end sample with the so-called ‘elapsed’ time. That is, the frame rate of the sample card is a much more accurate timer source than anything else in Java.
As the mailing list thread progresses, we come to learn that the application in question was for demonstration purposes. The program captures screen actions just as mouse movements, key presses, screen transitions, and mouse clicks, along with an audio recording of the accompanying lecture. Each screen event was time stamped, along with the audio that accompanied it. It was during playback that the audio was to be adjusted to meet the alleged synchronisation time of the screen event. However, the correct design here in these circumstances, is not to adjust the audio playback. It is accurate enough for this purpose, with far better resolution than the available system clock. Instead, screen events should be ‘time-stamped’ with the sample frame number, and on playback the screen events should be adjusted to fit the audio samples.
Although games need accurate timers and there are good techniques for achieving these within that context, what this problem highlights is the precision (or lack there of) of the timers available to us within Java. Typically a game timer might tick off every quarter-second (250ms) to synchronise gaming action to a human’s movements that is accurate enough, if you can count off a quarter second with a good degree of accuracy (say under 10 ms). But this is not accurate enough for good semi-professional sound application purposes, where stable sample-accurate timing is necessary.
In Java the most frequently used 'timer', used by most programmers, is the System.currentTimeMillis() method which experienced users know lacks precision. Further, this is compounded by inexperienced programmers confusing the thread control mechanisms available to them as a substitute for correct timing.
If Java is to be a serious contender as a ‘desktop operating system’ it requires some improved low-level services such as a microsecond accurate timer. Java from 1.4.2 beta now has a high-precision timer available to programmers. This is the undocumented class sun.misc.Perf. This singleton class has two methods of interest to the programmer requiring high precision time measurement:
sun.misc.Perf p = sun.miscPerf.getPerf(); long frequency = p.highResFrequency(); long tick = perf.highResCounter();
To illustrate the relationship bewteen the two values, frequency and tick, the System.currentTimeMillis() value can be calculated as:
Java 1.5 will also have System.nanoTime(), as part of JSR 166, which also introduces a java.util.concurrent package for concurrent programming, including a TimingUnit class that can represent timing units down to nanoseconds. The nanoTime() method, is "distinct from and uncoordinated with System.currentTimeMillis". There is also a java.util.PriorityQueue class and some related others that will greatly aid programmers looking for improved timing precision. See more information about JSR 166 at http://gee.cs.oswego.edu/dl/concurrent/dist/docs/index.html and also http://gee.cs.oswego.edu/dl/concurrency-interest/index.html. This is similar to the existing util.concurrent package that is authored by Doug Lea, who is the lead on the JSR. Many audio applications require good concurrency to work well so this is a definite improvement.
Professional audio, to be correctly synchronised, requires much greater precision than that available under Java. In C++, audio programmers are blessed with a whole variety of clocks they can obtain from their processor (and damned with having to know how to deal with the particularities of each of these processors). If the programmer chooses to use the usual thread control mechanisms, the methods outlined above will still suffer from the problems that inflict System.currentTimeMillis(). I have seen it claimed that the current mechanisms would be adequate if the Windows implementation of them wasn't so wildly inaccurate. However I doubt that 1 millisecond of precision, no matter how precisely ticked off, is of benefit to applications requiring very good audio timing (such as for example, a Java-based competitor to Steinberg Cubase or Emagic Logic would have to have). The Java Sound API provides a few tricks, and the sample frames from the sound card may in itself be enough to achieve the good stable timing (at least as accurate as your sound card clock let's say). There are some improvements coming in the 1.5 release, and Sun has recently doubled it's developers working on the Sound API (from one to two), and they are making improvements to the API. There is still a little way to go however. For example, in my opinion the addition of a MediaTimer class capable of firing future timer events in the way that java.util.Timer does now, only with better precision, would be a very good thing indeed.
One problem with a lot of professional audio programs is that they are very tied down to particular platforms, especially as some services move into the domain of the operating system. Java can help solve this problem, which will can also go a long way to lowering the prohibitive cost of such programs (because of their small market), by allowing developers to reach a large audience with it's cross-platform ability. Also with it's varied platforms many new applications spring to mind, such as PDA MP3 multi-track recorders, and the like.
I'd like to thank the people on the java sound mailing list for their stimulating discussion which allowed this (usually) lurking programmer to get the ideas to write this article.
Sound API Home Page: http://java.sun.com/products/java-media/sound/
Java Sound mailing list archive: http://archives.java.sun.com/archives/javasound-interest.html
Scot Mcphee is a senior programmer (Java Certified) with over thirteen years I.T. experience including the last five years exclusively designing and programming with the Java and J2EE platforms. His experience in several industry sectors (which include Government, Education, Telecommunications, Media, and Finance) have resulted in a wide-ranging background in technologies from naval radar & weapons systems, Unix and network protocols to the latest enterprise technologies which include human factors and web systems. In recent times, he has made somewhat of a specialisation of re-factoring J2EE systems to improve or meet reliability and performance requirements and mentoring developers in design patterns and performance enhancement. He has also been responsible for the design and implementation of software development life cycle systems. These systems assist developers by providing clear guidelines so they may deliver the business requirements within a framework that can be measured and improved. He has a Bachelor of Communications from the University of Technology, Sydney.
Page 2 of 2
previous page: Timers and audio performance