*** Personal note: It has been suggested that this may reflect an official Magento position. It does not. I like doing research and I like publishing what I find as long as I find it interesting and it isn’t tied to a customer. So if you’re wondering if this reflects a change in Magento or some kind of future roadmap, it does not. ***
I was recently asked by a client about Magento and HHVM support. While the answer is “don’t touch it in production and there’s no guarantee of support” it did spur in me the insatiable desire to create charts. I’m curious about it and I’m sure you are too.
One of the things I’ve been curious about is how much juice could we squeeze out of the HHVM/Magento combo? In other words, how fast could Magento Enterprise 1.14 actually be?
My test setup is my crappy local VM. 8GB of RAM with 4x Intel(R) Core(TM) i5-4440 CPU @ 3.10GHz. I had intended to test with Redis (boy do I have a fascinating blog post coming up there!) but there seems to be an odd thing that occurs to HHVM when dealing with reading from sockets that have buffered data 16k or greater. Since HHVM uses FastCGI that meant that for a proper comparison I needed to use Nginx with PHP-FPM. So I used the Cm_Cache_Backend_File instead. While my goal was to see if HHVM was faster than standard PHP for Magento I also wanted to see how many more requests per second I could squeeze out of it.
The answer to the latter question is; a whole shitload.
The test was simple. Hit the front page of a FPC-enabled site with JMeter and see what happened (0 – 25 concurrent connections and down again). Here’s what happened.
PHP-FPM peaks at just under 200 requests per second
HHVM Peaks at, wait for it, over 300 requests per second.
That is 300 freaking MAGENTO requests per second on a 4 CPU box!
Now, lest you think I’m suggesting you use HHVM, I’m not. It was not easy getting it to work and I wouldn’t trust it with a full workload. PHP 7 is also supposed to have similar improvements. And this was a test of one page, to see what was possible with one piece of functionality. But it’s an interesting indication of the improvements that are being made in the PHP ecosystem as a whole thanks to the introduction of HHVM. Perhaps PHP 7 will render HHVM moot. If that’s what happens, it’s fine by me. I’m just glad that we are finally starting to see some forward momentum on addressing some of the performance issues that have been inherent in PHP for quite some time. A little bit of creative destruction, now and then, is a good thing.
Comments
Allan MacGregor
Great article, thanks for sharing!
Magento-Neuigkeiten #32
[…] Schroder stellte einen kurzen Performance-Vergleich von PHP und HHVM mit einer Magento Enterprise Edition mit Full Page Cache […]
Gnomezero
Hello,
Great article!
in the same environment (nginx + php-fpm 1.6.2 + Magento 1.14.1) I migrated to HHVM (currently in 3.5.1) The performance gain is awsome!
And many performance improvements are announced for the 3.6 LTS.
I’m testing HHVM + Magento for couple of month, and I did not encounter much difficulty to install on Debian 7. Everything is functionnal for my store.
I’d like to know what has been complicated for you?
The only bad thing for me is the lack of MongoDB driver … wait & see.
Thank you again for your feedback!
Kevin Schroeder
There are a few complications. The first is that it is still very new for Magento and as such there are not a lot of techniques for troubleshooting in production. PHP has 20 years of practice behind it and lots of visibility and lots of mindshare. HHVM is not there yet. Nowhere near. I don’t yet have the confidence of putting it into production and being able to diagnose problems as quickly as with PHP. That might just be me, but it will be a while before HHVM has those kinds of practices solidified.
I also had some issues compiling it. I run on CentOS as do a lot of enterprises (usually RHEL). Until there is a drop-in, supportable, implementation for RHEL/CentOS it will still be on the bleeding edge for most organizations.
Bryan Veloso
Do you have benchmarks for Magento 2 + HHVM? 😀
Simone
Form my point of view, benchmarking with FPC is a completly wrong benchmark. If your goal is to benchmark “php” execution, you must turn ANY cache off, and then benchmark. At the end of the day, it does not matter the real number per se, if you get 30 / 300/ 3000 / 30000 per seconds it’s just a number. you need to understand if HHVM , as a php alternative, it’s faster then plan ol’ php-fpm.
Why don’t you repeat the test with cache off and see the results? Becouse to cache there is always time 🙂
Kevin Schroeder
You are right. If my goal is to benchmark PHP execution then this is the completely wrong benchmark. However, if you were to read the title of this post you will note that I am not benchmarking PHP, I am benchmarking Magento Enterprise Edition Full Page Cache with HHVM.