|
|
|
|
Back to newsletter 042 contents
This month we interviewed Matt Tucker and Bill Lynch, founders of Jive software. If Jive doesn't sound familiar, then that may be because of how successfully their product works, quietly powering as it does a number of very popular discussion forums across the web, not least Sun's and Javalobby. Bill and Matt give away details about how to scale heavily interactive sites with large amounts of dynamic content. So if you are building a portal, this is possibly one of the most useful set of performance tips you may see.
JPT: Can you tell us a bit about yourself and what you do?
We're the founders of Jive Software and lead technical guys. We both wear many hats, but most of our time is spent working on Jive Software's flagship product, Jive Forums.
JPT: I only recently noticed that the Sun discussion forums are run using your software (Jive Forums). Are they your largest customer, or are there even bigger sites out there using Jive Forums?
Sun was our first large customer and it was actually the professional services relationship with them that let us start Jive Software three years ago. Although their forums are fairly large, the actual traffic is middle of the road compared to our other customers. For example, community-focused sites like Babycenter and StarWars.com have many more messages and page views. A "large" Jive Forums deployment is a community with one million or more page views a day and millions of messages in the database at any one time. The ability to scale to this type of traffic is a big differentiator -- the free PHP forum packages simply can't do it since they lack an adequate caching infrastructure.
JPT: I imagine you have seen some pretty interesting scalability issues cropping up. Can you tell us about a couple?
Serving up entirely dynamic pages with constantly changing data millions of times a day has definitely presented many scaling challenges. Mostly, scaling comes down to our caching architecture. Luckily, discussion forums are well suited for caching since users generally read more messages than they post. So, caching basically takes two forms:
1) Cache objects in the system -- forums, categories, messages, threads, users. These types of caches are long-lived and size-bound. We calculate the size of each object in memory so that users can allocate 4 MB to one cache, 32 MB to another, etc. Items are expired from the caches using an least frequently used (LFU) algorithm. This is pretty necessary with caches like the message cache -- if you have millions of messages that are each about 50K, there is no way you'll fit them all into RAM. However, LFU is perfect in this case since people generally read the most recent messages over and over again, while old messages are accessed infrequently.
2) Cache the relationship between objects, such as the list of messages in a thread, the list of threads in a forum ordered by modification date, etc.
The first cache is very simple; it's the second type of cache that presents all the interesting challenges. For example, on sites with very heavy traffic, users post new messages multiple times a second. If you're trying to cache the list of messages in a forum, that list gets expired multiple times a second (faster than the database can do the query), so you very quickly become database bound. To solve this problem, we have a special caching mode that forces certain items to stay in cache for at least N seconds (where N is typically 5 or 10 seconds). This puts a cap on the number of cache expirations that happen even as traffic can continue to increase. We also do the cache update on a separate thread so that users can continue to read slightly stale data until the new data has been read from the database and put back into cache. Basically, it's far better for a user to not see messages posted within the last 5-10 seconds than it is for them to not be able to view any content at all because the database is getting slammed and can't keep up with queries.
The other interesting scaling issue we deal with is clustering. Because we support app-servers like Tomcat and Resin we don't use EJB. That means we've had to roll our own clustering support based on a product called Coherence from Tangosol. In its simplest form, Coherence is a clustered implementation of a Map that we build caching structures on top of. It actually gets quite a bit more complicated than that with our advanded caching modes, and Coherence also provides many other useful clustering services like an invocation service (you pass it a Runnable and it executes it across all cluster members and returns results).
Wow, this feels like you are giving away the secrets of a scaling a highly successful product. Thanks, I'm sure our readers enjoyed that, especially those that might be having performance problems with portals which have many of the same issues cropping up. I'll be a little cheeky, and ask if there are even more performance considerations that you could tell us about?
Yes, we thought of a few more interesting scalability issues that have come up
1) Getting good performance out of databases despite the fact that the product is mostly database-neutral and we support everything from MySQL to Oracle and DB2.
2) We have to strictly limit the number of JSP taglibs we use since the overhead is too great on a huge number of dynamic page views for most app-servers.
Yet more excellent performance tips, thanks. Could you expand on how you achieve the database neutral high performance, or is it a business advantage you would rather not detail further?
We use several strategies to increase performance while still maintaining a codebase that works with a huge variety of databases. We have a ConnectionManager class that we use to get connections. It maintains quite a few pieces of meta-data about databases, some of it reported by JDBC drivers and other items that we've hand-coded. Examples of meta-data fields:
// True if the database support transactions.
private static boolean transactionsSupported;
// True if the database supports the Statement.setMaxRows() method.
private static boolean maxRowsSupported;
// True if the database supports the Statement.setFetchSize() method.
private static boolean fetchSizeSupported;
// True if the database supports correlated subqueries.
private static boolean subqueriesSupported;
// True if the database supports scroll-insensitive results.
private static boolean scrollResultsSupported;
// True if the database supports batch updates.
private static boolean batchUpdatesSupported;
So, we might have one path for databases that support sub-queries and another code path for MySQL since it doesn't support sub-queries. In some cases, we also want to know which specific database is being used:
/**
* A class that identifies the type of the database that Jive is connected
* to. In most cases, we don't want to make any database specific calls
* and have no need to know the type of database we're using. However,
* there are certain cases where it's critical to know the database for
* performance reasons.
*/
public static class DatabaseType {
public static final DatabaseType ORACLE = new DatabaseType();
public static final DatabaseType POSTGRES = new DatabaseType();
public static final DatabaseType MYSQL = new DatabaseType();
public static final DatabaseType OTHER = new DatabaseType();
//
private DatabaseType() {
/* do nothing */
}
}
We use that information in just a few critical places to make database-specific optimizations like passing in hints to Oracle. This combination of stategies lets 95% of our database code be totally generic while still allowing us to optimize where we need to.
Thank you very much! That is really useful information, I'm sure quite a few of our readers are going to benefit from this interview. Congratulations on your product, it seems even more impressive after what you have told us.
The JavaPerformanceTuning.com team
Back to newsletter 042 contents