Magento-based asynchronous execution

Working with an off-the-shelf shopping cart usually requires a little bit of patience. Scaling an e-commerce site does have its share of problems. There is a LOT of interactivity that needs to be implemented. This can be things along the lines of generating targeted ads, sending email or charging a credit card.

To charge a credit card, the ecommerce software will usually take the credit card information from the end user, put it into some form of web service request and the request is submitted to a remote system.  While that web service request is taking place the process handling PHP is unable to take additional require requests to serve regular pages.

One option that you have is to complain that PHP doesn’t have threading.  That’s not the best thing to do.  As Marco Tabini said recently on Twitter “Every time someone mentions threading in PHP, an angel’s wings enter a race condition”.  Threading solves some problems.  However, chances are that while you may want threading you probably don’t need it.

However, while you probably don’t need threading, there are plenty of times when being able to do things asynchronously would be beneficial.  The example that I started looking at was a credit card request.  While waiting for the credit card transaction to occur you have one of two options.  1) Let the screen be blank while you’re waiting for the transaction or, 2) use some kind of output buffering and progressive rendering to let the end user know that the transaction is, in fact, being processed.

However, there is another, better, option. Rather than either spending loads of CPU time to process loads of logic, such as personalized ads, or have long wait times, such as processing a credit card, you can have this processed “behind the scenes” so you can immediately respond to your customer.

A simple example of what a Job Queue architecture can look like is almost like a hub and spoke architecture except that instead of the hub being the center it is actually the outside.  Ok, so a simple Job Queue architecture is exactly the opposite of a hub and spoke architecture.  Sue me.

The way it works is that there is a backend server, or cluster of servers, that handle servicing Job Queue requests. The requests are made from your front end web servers which is sent to a URL on the backend.  The URL is where the logic is that needs to be run.

Using a simple architecture you can just have that URL be a simple script that is run.  However, I prefer a more structured solution if I am going to integrate asynchronous processing in my application.

This is where the Magento connection starts.  I have already written about how to implement a structured asynchronous mechanism.  This is the same implementation that I use on this blog site.  What I’ve done is take that implementation and re-implement it so that it works within the context of a Magento application.  I have placed this implementation on Github.  It is not yet part of Magento Connect, though I intend to do it and I intend for it to be provide free of charge.  However, what I also wanted to do was give others the chance to look at it and improve it prior to putting it on Magento Connect.

Implementing your own task, be it pre-processing advertisements or processing a credit card is very easy.  Processing a credit card, however, should be done with the addition of encrypting the data is that data is stored “as-is” in the Job Queue database.

Defined in this library is a class called ZendServer_JobQueue_Job_Abstract. This is the base class for defining a task.  There is only one method that you need to implement, though you can implement as many of your own method as you want, such as getters and setters.  The method is called _execute() and this is where you would implement the logic that you want to implement.  However, it is important to note that because this is run on a completely different machine once the task has been set to execute no changes that you make will be reflected in the job if it has started running already.

In the code download there is an example of how to implement this class.  It is called ZendServer_JobQueue_Model_Mock.  All it does is write to the PHP error log, but does so asynchronously from the Job Queue URL.  The code looks like this

class ZendServer_JobQueue_Model_Mock extends ZendServer_JobQueue_Job_Abstract
{
            protected function _execute()
            {
                        error_log('Mock Model run');
            }         
}

One thing to note.  It’s freaking easy to implement this!  If you want to run this, here is your code.

$task = new ZendServer_JobQueue_Model_Mock();
$task->execute();

Wham.  Bam.  Done.  It is now running on your Job Queue server.  I won’t get into all of the details on how it’s done, though.   You can take a look at the abstract class and understand the details yourself.  It is open source after all.

But if you were to run this code right now you would probably get an exception thrown.  That is because you have not configured your Job Queue yet.  In order to do that you need to look etc/config.xml file.  You need to edit the element config/modules/ZendServer_JobQueue/jobqueue/url and specify the URL of the job queue entry point.  Since there is an index controller for the ZendServer_JobQueue extension and I just used the standard router, the URL would be $HOST/jobqueue.  It is recommended (highly recommended) that you make this URL available over the localhost or private.  It is not by default, so I recommend that you set this up using either a virtual host that only listens on 127.0.0.1 or on a machine that is behind a firewall.

So, that’s pretty much it.  Though I suppose you’ll need Zend Server as well. 

To install Zend Server you can go to zend.com and set up your system to install or download (for Windows) Zend Server.  It comes with a 30 day free trial.  Give it a shot.  If you have trouble feel free to post on the forums at forums.zend.com or you can post a comment here and I can try to answer it.

Happy coding!