|
|
|
|
Back to newsletter 142 contents
The Masstree paper I extracted tips from this month (see articles below) achieves the same performance as memcached, yet with the addition of persistency. This is a pointer to the future application use of storage: we'll increasingly be using in-memory storage with persistency for all our data rather than the older paradigm of persistent storage with some of the data cached in memory. After all, if Google feels it's beneficial to put their entire web index into memory (which they had already started doing as early as 2001 according to Jeff Dean's WSDM09 presentation), it's highly likely that your application storage requirements can cost-effectively fit into memory too.
Of course if you need to manage many terabytes of stored data, we may not be there yet, but the vast majority of applications have not more than multi-gigabyte storage requirements, and that can now comfortably fit into memory.
The simplest way to store data in-memory is just to make your caches big enough to store all your persistent data. This is effectively how your database server can work as an in-memory system. The database vendors work to ensure that this configuration works effectively, they're fully aware that servers are increasingly coming with hundreds of gigabytes of memory. Naturally database caches are very specific types of caches optimized for database management, not the same sort of things you plug into your application. But the principle is the same.
However, that's the old "persistent storage + cache" technique, where the persistent storage is primary and the cache an optimization. If you are designing your application now, you should really be thinking the other way round - how do I design so that the in-memory storage is my primary working storage, and I have additionally a mechanism to ensure that any changes are persisted so that I can recover if necessary.
This is the future world of applications - parallel, multicore enabled, efficiently highly concurrent, with in-memory storage and persistence for recovery. Now on to all our usual links to Java performance tools, news, articles and, as ever, all the extracted tips from all of this month's referenced articles.
Java performance related news.
Java performance related tools.
Back to newsletter 142 contents
