Tread carefully before rolling out dynamic content caching products.

Web caching can easily accelerate sites composed of static pages or dynamic pages that are generated into static ones, such as those held by many content management systems. But if you add significant personalization or highly variable page elements to a site, the benefit of Web caching can be lost, since most page requests will not result in a cache hit. A new class of Web caching products - we'll call them dynamic content caching - attempts to address the challenge of accelerating dynamic content. (See 'How we did it'). We looked at four such products to determine if acceleration could be achieved and if so, with what difficulty. We tested Cache Technologies' XCache, Chutney Technology's PreLoader, SpiderCache's self-named product and Persistence Software's Dynamai. SpiderCache won based on its rich set of features at a very attractive price. We tested a fifth product, FineGround Condenser, but decided to not include it in this roundup (click here for more details).

SpiderCache 1.5: The little cache that could

SpiderCache had a much richer set of caching control features than would be expected for a product at this price.

SpiderCache installed very easily, successfully installing on the first try. Administration of SpiderCache is similar to XCache, relying on a familiar Microsoft Management Console-style interface to indicate which pages you want to cache. SpiderCache in particular focuses on caching or ignoring content based on file extension. We found a very annoying bug when adding a new file extension to cache, such as .cfm - the product lost the configuration change whenever we started or stopped caching on the site.

A missing feature in this release of SpiderCache was built-in performance monitoring. The current version requires outside monitoring facilities, such as NT's Performance Monitor. However, the product provides many more configuration options than XCache, and we found it very easy to change and monitor caching parameters.

Without any programming changes to a site, users should see immediate improvement. On the test with some dynamic content mixed with static content there was a performance gain of about two times. There appeared to be little gain from static files, because SpiderCache does not support any form of white space compression or GZIPing to text files. Instead, the product offers image optimization. As many Web designers tend to optimize images already, the value of this feature seems somewhat limited.

While SpiderCache is similar to XCache, for detailed cache control and partial page caching it is the clear winner. To control caching within a page you include HTML-style comments, which makes it more language-independent than XCache or PreLoader. You can also control caching via URL strings, cookie values or HTTP header values. This includes user agent strings, letting you do browser-specific caching. The product also allows for control over cache expiration on a URL-by-URL basis, so pages can explicitly expire at various times and under programmed conditions.

SpiderCache is very programming-friendly, integrating with scripts using the HTML comment approach and with databases through a very useful set of database triggers. The triggers come in the form of SQL Server- or Oracle-stored procedures that let you clear or refresh cached entries directly from the database. Considering where dynamic content is actually first changing, it is much more appropriate to use the database to interact with the cache rather than site script code. The product also includes SpiderCache Data Objects, which provide a COM/_ActiveX programming interface. This lets developers control the caches from their own programs. For developers with custom content management systems, this is very useful.

SpiderCache requires 128M bytes of RAM and 1G byte of disk space to operate, although most sites will probably have higher RAM requirements if they need this product. The product runs on Windows NT or 2000, with versions also available for HP-UX, Linux and Solaris.

For the price, SpiderCache is a very well-rounded product. It is a great place to start if you're looking to see what dynamic caching products can do and don't want to recode your site to reap some initial benefits.

Xtremely easy caching

XCache 1.4.5 was by far the easiest of the products to use and install. Installation on our Win 2000 test server was so easy that no options needed to be specified. XCache requires Windows NT 4.0 with Service Pack 6a or Win 2000 with SR1, but has no defined RAM or disk space requirements.

Configuration was also very easy. You can specify the size of the cache as well as which pages you want to cache, all through a Microsoft Management Console plug-in. By default, XCache focuses on dynamic pages only, such as .asp, .cfm and .dll files, but you can have it cache static site objects if desired. Of course, it might be better for overall site performance to employ a content delivery network or an external cache to deal with such files.

One minor deployment problem we had using XCache with ColdFusion was that it forced us to remove debugging information from our test site to get the appropriate caching performance. However, once configured, the product worked admirably with our ColdFusion-based test cases, unlike other products such as Chutney, which could not run these cases at all.

Beyond the change made to handle ColdFusion, we didn't have to make changes to our Web site code to achieve initial site acceleration. On the initial tests of XCache we saw one heavy query page in our test site containing more than 50 queries with average uncached response times well over 10 seconds become nearly as fast as static pages, averaging around 1 to 2 seconds in our test environment. To achieve better caching with dynamic pages using XCache, you can employ partial page caching by indicating some portions built at cache time and some at request time. These indicators are language-specific and support only ASP- and ColdFusion-based pages. However, they are quite easy to add.

Another very attractive feature of XCache was its performance monitoring, which gave feedback on performance, scalability and bandwidth gains. Tests with some dynamic content mixed with static content revealed a performance gain of 2.5 times. Finally, a simple test with no dynamic content revealed an increase of 1.5 times.

XCache also supports simple page compression through white space reduction as well as stripping out HTML comments. The product also offers URL rewriting rules to fix dynamic URLs filled with question mark characters, so Web search spiders will index them.

A potential downside of XCache Version 1.4 is it generates pages with one particular user agent, Internet Explorer, by default. If certain types of browser-specific dynamically generated pages are created, much of the caching benefits could be lost, and it may be possible to even send the wrong page to an end user. A segmented cache by browser type would solve this. The new version of XCache, 2.0 (currently in beta test and not tested for this review), should take care of this problem. It adds many features, including segmented caching (letting users define different cached pages based on browser type), cookie, HTTP headers, language, integration with content delivery networks and improved caching support (such as absolute expiration and precaching of selected documents).

For midsize NT sites with a few servers and numerous dynamically generated pages, XCache is a real bargain. It's easy to use and produces page improvements with relatively little work.

Chutney PreLoader: Price and power?

In some ways Chutney takes the exact opposite approach to the dynamic page-caching problem taken by XCache. Chutney's PreLoader 3.0 requires developers to make changes to their sites to see any acceleration benefits. The detailed caching control provided by PreLoader lends itself mostly to sites with a heavy degree of personalized content. Sites lacking per-user, per-page changes in content will find the amount of work required to accelerate a page using this product unacceptable and would be better off recoding their sites directly to improve performance. However, for those interested in this power, Chutney shows promise.

PreLoader requires significant system resources to do its job, with 1G byte of RAM suggested and a minimum of 512M bytes of RAM required. Armed with the appropriate hardware, the installation process was straightforward, although it did not work initially. A few installs later, we could get the product to work properly with Internet Information Server. We also encountered some initial confusion using the product in our test environment that had multiple virtual Web servers on each machine. Configuration was not obvious because the product always assumed to connect to the base server for the machine (it didn't expect other sites on the same box). Unfortunately, the documentation did not help us figure out this mistake, but Chutney technical support helped. In a normal deployment, Chutney generally provides professional services to ease such problems and assist with deployment. But at $100,000 per server plus $5,000 per CPU for these services, it will cost.

PreLoader's administration is performed via a Java application. The tool was somewhat cumbersome, and on a few occasions we were unconvinced our changes were being applied. We ended up editing configuration files directly at a few points during testing.

Once set up, preparing a Web site for caching using PreLoader was more like recoding a Web site for caching. In effect you have to manually add a portion of a dynamic page to cache if it is not already cached and pull it out if it already is. Unless you go through this you will see no advantage to using PreLoader. This is a tedious process, but you can control all aspects of caching this way.

In some of our basic tests, we didn't see any discernible improvement using the technology because our basic test pages were too simple in their calculations, and the cache control code required by PreLoader effectively negated any gains. However, when we changed our tests to use more complex calculations with large sections of dynamic content taking longer generation times, we saw modest but acceptable improvement in page delivery time - an improvement of about 1.5 times on average delivery for single pages.

PreLoader provides support for ASP- and JSP-based pages through its API, although it did not include direct support for ColdFusion. Chutney said this will be included in a future release. However, the product does provide more advanced integration with application servers such as WebSphere and BEA's WebLogic, which is missing from some of the lower-priced products.

While the control provided by PreLoader may suit large-scale personalized sites, the cost/benefit ratio for the product is a little troublesome. If your site needs such a solution, it is probably best to architect the site around PreLoader rather than applying support for it later on.

Dynamai: Super Squid redux

We tested Persistence Software's Dynamai 2.0 under RedHat Linux 6.2, and found it should be right at home in the open source environment. Under the hood it was heavily based on the popular Squid Web proxy cache. Given that Squid can be somewhat difficult to use, we hoped that Persistence had smoothed out the problems and made the product more powerful and accessible. However, in the end we were happy only with its raw power and performance, and disappointed with the product's execution.

Installation was easy, but we found the product's administration relied on certain specific XWindows facilities and did not react properly without them. Fortunately, the manual helped us get beyond such problems and get the product working.

The most significant problem with Dynamai was its poorly implemented browser-based administration system. We found the administration interface would not refresh to reflect changes made. In many cases it became completely inoperable. On more than a few occasions the interface would literally explode, with strange frame sets spawning over and over with portions of the tool bar in it. A great deal of frustration is certainly in store for Dynamai users who encounter the bugs we did. When the administration did work, we found it provided comprehensive control of what content to cache, the invalidation schemes to use, and a fairly complete set of reporting and monitoring tools that the other products went without.

After we got the product working adequately, it provided immediate improvement without programming because of its basic caching facilities. We saw a performance improvement of about 10 times in the basic test cases without programming changes. In some cases we saw processor usage drop significantly, because the cache was doing all the work on a separate box. However, these types of gains should be expected when using any form of Web cache and mostly static content. The real test, of course, is dynamic content.

Like the other products, Dynamai lets you control cached content via query string parameters, form parameters, cookies and URLs. It also allows for segmented caching by caching different content depending on the browser sending the page request. You can also create different browser-specific content and map user agents to various forms of content, such as frame, no frame, Java and no Java. Similar to SpiderCache, the programming interface for segmenting pages for caching uses an HTML comment-style approach. Dynamai also provides a set of APIs for C++, Java and ActiveX for issuing cache control events, similar to Chutney's PreLoader. Under these tests we saw only modest improvements, generally averaging only about 1.3 times better in page delivery time.

Dynamai certainly has more than enough features for all but the most demanding developers. Unfortunately, giving it higher marks really isn't possible given the freeware-tied-together feeling of the product and its extremely problematic administration interface. These issues need to be cleaned up in a future release.

Worth the trouble?

While some reported gains seem meager, consider how they affect a site when significant numbers of visitors hit you simultaneously. A small performance gain, even 50%, may translate into a much larger number of possible visitors on the same hardware. So if you have a dynamically generated site with an increasing user load, you might consider looking into these caching products before throwing more hardware at the problem. However, proceed with caution - spending more money may not result in major savings. You should perform a return-on-investment analysis to determine the savings gained from performance and scalability improvements. And don't forget to consider implementation and integration costs when performing such calculations, because with many of the more advanced dynamic caching products, successful use of the product will certainly require significant professional services or extreme patience.

In the end, we found that the lower-end products certainly could teach a thing or two to some of the more advanced caching products on the market. If you just want to get an idea of how dynamic caching products can improve performance, try SpiderCache or XCache first to understand what these products can do for your site before looking into higher-end products.

Originally published on Network World, Published: June 11, 2001.