Magento, ESI, Varnish and performance

I have been doing a little playing with Magento over the past couple of days.  I’ve been helping out Ebay/Magento by delivering some of their performance training over the past few months.  I’m by no means the world’s best Magento person at the moment, but I know the architecture pretty well.

One of the things I’ve wanted to do is play more with Varnish.  There’s lots of hype, and a lot of the hype is true.  It really is as fast as they say it is and worth looking at for a full-page caching solution.

But what about when you “can’t” do full page caching?  Enter Magento (or any ecommerce platform).  Most of the time you pages are fully cacheable.  Right up until you click the “Add To Cart” button.  At that point full page caching doesn’t work and so the default behavior for most platforms is to simply not cache output at that point.

But the problem is that you still have 90% of the page (or more) is still cacheable.  So you are doing full execution on a page that has a bare minimum of actual dynamic content.

This is where ESI comes in.  ESI, or Edge Side Includes, allow servers on the edge of a CDN do full page caching but do dynamic callbacks to your website to fill in certain parts of the page.  You lose a lot of the benefit of having a full page cache in that the many order of magnitude performance improvements you gain with full page caching is reduced to how fast you can get the backend dynamic content.

But who says that ESI only needs to be on the edge?  I decided to take a look at how I might implement ESI, with partially dynamic content in Magento, using Varnish as a processor.

My results are in and they look pretty good, though they are preliminary.  As I write this blog post I’m dealing with some wildly erratic response times from Varnish.  Varnish as a full page cache is consistent.  The ESI pages are consistent.  But put them together and I have response times that vary by two orders of magnitude when I do a benchmark that uses concurrent connections.

But if I do the benchmark with sequential HTTP calls you can definitely see an improvement.

First the results without a full page cache.  This is the way you do it by default now once someone adds something to their shopping cart.

Concurrency Level: 1
Time taken for tests: 1.171843 seconds
Complete requests: 10
Failed requests: 0
Write errors: 0
Total transferred: 81030 bytes
HTML transferred: 76210 bytes
Requests per second: 8.53 [#/sec] (mean)
Time per request: 117.184 [ms] (mean)

Now the results when using Varnish with ESI.

Concurrency Level: 1
Time taken for tests: 0.591032 seconds
Complete requests: 10
Failed requests: 0
Write errors: 0
Total transferred: 86500 bytes
HTML transferred: 81000 bytes
Requests per second: 16.92 [#/sec] (mean)
Time per request: 59.103 [ms] (mean)

That is roughly a 50% improvement with no loss of functionality.

In this example I simply removed the call to the Magento sidebar which renders the shopping cart and replaced it with an ESI tag.

<esi:include src=”http://magento.loc/eschrade/esi/sidebar” />

That routes to a controller where I manually spit out the sidebar contents.

Now, you might be thinking “OK, that’s one HTTP call.  What happens if you have 20 ESI calls?  Won’t your server be overloaded?”  Well, perhaps (though not definitely).  But what you need to remember is that performance and scalability are two different things.  Yes, you could overload that ONE server with 20 ESI calls.  But what if you have 20 servers running behind a load balancer?  Yes, you will be using much more by way of server resources.  But if your response time from each of those backend servers is 50 ms you still only have a total response time of 50ms, give or take a few nanoseconds to render the page.  So while CPU time will be greatly increased across your cluster, wall time will be greatly reduced because the processing is being done asynchronously with Varnish aggregating the results.

This, of course, assumes that Varnish does its ESI processing asynchronously, which I will confirm.  (I’d be surprised if it were synchronous)

[UPDATE]

As noted in the first comment before, it does look like Varnish processes ESI synchronously, meaning one at a time.  So these numbers would seem to hold if you only have one ESI call to make, but they drop off significantly as you add ESI calls.  So I will be looking at different options since the whole premise depends on asynchronous ESI.

Crap.

[/UPDATE]

But like I said, these are preliminary results.  I will do a fuller blog post on the subject when I have the inconsistent performance issue worked out.