[TL;DR] Go to the conclusion at the end, get the links for the GitHub repo, contribute.  Follow MagiumLib on Twitter (nothing posted as of writing)

[update] The Magium site is live.  Lots of stuff there.  More coming.

This past summer (2015) I was tasked with building out automated testing for a Magento 1 module.  I had not really built out a lot of automated testing before and so I was in for a massive shocker. It sucked.  It royally sucked.

I had my list of user stories and I started out as everyone starts out: I downloaded the Selenium IDE.  It worked really well.  I built out about 10-20 test cases and was impressed at my progress.

Then I had to make a simple change to all of the test cases.  That turned all of my progress into a useless pile of crap.  I had wanted do it with as little developer skills as necessary and failed miserably.

So I went to Google to try and find out what was available that suited my needs.  My needs were simple:

1. I needed repeatable functionality, such as being able to easily execute the checkout process
2. I needed that repeatable functionality to be modifiable – the module was tightly integrated with checkout and had many options to tests against.  So I needed to be able to modify the checkout consistently.
3. I needed to do all that in the admin area too
4. It needed to be written in PHP.  Magento IS written in PHP and it’s silly to require yet-another-programming-language

There were several different options available but they often seemed basic.  Good, but basic.  Or good, but general.  In other words, there was still a lot of work that I needed to do to make it work.

Plus, I didn’t have an existing toolset that I was familiar with enough to force it to make it work.

So I decided to take some things that I was decent at (PHPUnit and PHP in general) and make it work.

Lo and behold there was a PHPUnit Selenium2 library.  So I started writing tests using that library.  I really kind of liked it.  I was under the gun and so didn’t have to time research and architect it well.  What I wrote broke about every SOLID guidance there was but it largely worked.

Except it didn’t do it very well.  Things like category navigation worked, except when I had to switch themes.  Then I had to create conditional statements to see which Xpath query would work.  Checkout would work, except when it didn’t.  Ajax was not happy with the library.  And don’t even get me started on the admin area.  It was quite difficult.  Add on top of that the module had a lot of different configuration settings that needed to be tested.

Which brings me to my last point.  The Selenium2 library for PHPUnit is sssslllllooooowwwwwww!.

At the end of the day I made it work, but there were a lot of lessons learned and a lot of bad testing approaches that I would not have done had I had more experience.

Enter Magium.

Magium is a testing framework that I’ve started work on that originally combined the things I needed to have working together.  Selenium, PHPUnit, and Magento.  However, as I’ve been working on it I’ve found that it could actually be used for other projects as well.  But my work is with Magento and so I’m building out the framework with that in mind.  Hence the name, despite the fact that the core functionality could actually be used in any web-based application.

I looked at some options for a PHP-based Selenium driver and there were basically 3 that came up.  PHPUnit_Selenium2, Instaclick, and Facebook.

1. PHPUnit_Selenium2 – Good.  Nice integration with PHPUnit.  Slow as molasses in Alaska.
2. Instaclick – Definitely faster, but I had problems with consistent test results
3. Facebook – No problems.  Nice, fast, and consistent

So I opted to use the Facebook WebDriver driver.

Now I have the WebDriver driver decided on.  Next up is to decide how to best use it.

Now, “best” is a hard term to quantify.  What is the best way of doing browser testing?  There are a plethora of browser testing tools out there.  Several are available for Magento.  Many of them are good.  But I found that what was available didn’t solve the problems the way I felt they needed to be solved.

Because of that there are a number of choices I made to build a testing foundation that solved the 4 problems I noted earlier.

Repeatable Functionality

This is probably one of the easiest problems to solve.  Create classes that utilize SOLID principles.

Single Responsibility – A class is (generally) responsible for one thing.  For example, filling out the billing form or navigating to a category.

Open/Closed – Checkout may have some customization that need to be tested.  Because each step in the checkout process follows Single Responsibility the checkout can be modified by providing different objects to do different things.  But the checkout class, itself, is only a container for that functionality and does not require any changes in order to handle customization.

Liskov Substitution – Using checkout as an example again.  The checkout may have different payment methods.  You are able to swap them out because they are based off of sub-class that defines how they should work.

Interface Segregation – Payment in checkout is a good example.  It is both a payment method and a checkout step and both are defined separately.

Dependency Inversion – I kind of break this one a little, though not too badly.  The way I break it is by depending on base-level functionality, not abstractions.  That is because what I needed was repeatable functionality that works with the core Magento implementation, but can also be customized.  The proper way, I know, is to provide the abstractions, or interfaces, as the dependency and then configurably provide concrete implementations.  I commit a minor sin in its usage for the purpose of having a working test system out of the box that requires only minor configuration.

Modifiable, Repeatable Functionality

My implementation of this requirement is probably what will give most people the most agida.  I used the SOLID principles not for architecting an application, but architecting tests.  I mis-use them intentionally because it is the principles that are useful, not the implementation.  The test is the application, in the case of building a testing library.

My most egregious assault on software development principles is in how I fill the need for making the repeatable functionality modifiable.  I do that through the use of a Dependency Injection Container as a critical part of the test case.

I have very good reasons for this.

Test code must be configurable

A big, big part of my original problem was that the module I was working on needed to work on CE 1.7, CE 1.8, CE 1.9, EE 1.12, EE 1.13, and EE 1.14.  You might be tempted to think that not much has changed, or, rather, that the change has been iterable.  But that’s not true.  Checkout HTML code is quite different between versions.

As an example consider this code I had to write to place the order.

1
2
3
4
5
6
7
8
9
10
11

  $this->waitUntilElementDisplayed('review-buttons-container');
  if ($this->elementExists('//button[@title="Place Order"]', self::BY_XPATH)) {
    $this->waitUntilElementDisplayed('//button[@title="Place Order"]', self::BY_XPATH);
    $this->byXPath('//button[@title="Place Order"]')->click();
  } else if ($this->elementExists('//*[@onclick="review.save();"]', self::BY_XPATH)) {
    $this->waitUntilElementDisplayed('//*[@onclick="review.save();"]', self::BY_XPATH);
    $this->byXPath('//*[@onclick="review.save();"]')->click();
  } else if ($this->elementExists('review_button')) {
     $this->waitUntilElementDisplayed('review_button');
     $this->byId('review_button')->click();
  }

That is ghastly code that I never want to write again.  Doesn’t it make you want to throw up?  And what happens if another change is made?  Yet Another If Statement.

Instead, I want to be able to make changes where the behavior is the same, but the selectors might be different.  Using Magium there are two ways of doing this (which I won’t get into here).  Either by setting type preferences for the DIC or by having some classes configurable (extending an AbstractConfigurableElement class).

Test code must be repeatable

One of the things I’ve not liked about many of the libraries I’ve looked at is that they look fine in their examples but that they don’t take you beyond the basics.  I did find some other code that was similar to what I needed but most of it looked pretty old and was less complete than what I’ve written so far.

But I’ve gone a little too far.  What do I mean by “repeatable?”

I mean that I can make one method call to do a checkout.  I don’t have to write it out over and over again.

For example, if I need to do a guest checkout in Magium I can simply run

1
2

$guestCheckout = $this->getAction('Checkout\GuestCheckout');
$guestCheckout->execute();

But it’s not enough to just run the same thing over and over.  What if one of your requirements is that you can sell to the US and Canada?

Handling that kind of scenario is really easy in Magium.

1
2
3

$this->getIdentity()->setBillingCountryId('CA');
$guestCheckout = $this->getAction('Checkout\GuestCheckout');
$guestCheckout->execute();

Test code must be readable

Which brings me to my next point.  A good number of people, i.e. virtually all of them, who run user acceptance testing are not particularly good programmers.  I wanted a library that was both powerful and accessible.

One of the core intentions of Magium is to make tests look clean so they are easy to read.

To see what I mean, consider the Selenium log to navigate to a category and assert that a product exists on that page.

[get: http://magento19.loc/])
[find element: By.xpath: //nav[@id="nav"]/ol/descendant::li[contains(concat(" ",normalize-space(@class)," ")," level0 ")]/a[.="Accessories"]/../a])
[mousemove: 0 false])
[find element: By.xpath: //nav[@id="nav"]/ol/descendant::li[contains(concat(" ",normalize-space(@class)," ")," level0 ")]/a[.="Accessories"]/../descendant::li[contains(concat(" ",normalize-space(@class)," "),"level1 ")]/a[.="Jewelry"]/../a])
[mousemove: 1 false])
[find element: By.xpath: //body[contains(., "Blue Horizons Bracelets")]])

Now compare that to the Magium code.

1
2

$this->commandOpen('http://magento19.loc/]');
$this->getNavigator()->navigateTo('Accessories/Jewelry');

Which makes more sense to you?  And you’re probably a programmer.

It needs to work in the admin area

Seeing that ^^^ header tag alone probably makes you want to go home and cry.  Have you ever tried to script placing an order in the admin?  You’ll be Nietzsche and Freddy Mercury rolled into one.  You will be sorrowfully singing the Bohemian Rhapsody all the way home from work in your car around when the clock ticks over to Sunday and praying that the priest will accidentally throw some holy water in your eyes.  (being Protestant I have no such hope)

Let’s say you have a feature that turns the site into maintenance mode via a configuration switch.  You gotta test that, right?  You want to automate it, right?

Well because I have such fondness for people who want to automate their Magento testing as much as possible Magium has an action called an Enabler that allows you to enable and disable “enablement” switches in the system configuration.  Let’s say that your hypothetical maintenance mode is under System / Maintenance mode and has switch called “Enable”.

1
2

$enabler = $this->getAction('Admin\Configuration\Enabler');
$enabler->enable('System/Maintenance');

Magium will log in, navigate to the System Configuration page, click on the System Tab, make sure that the Maintenance section is displayed, clicking the section header if it isn’t visible, switches the “Enable” switch to “Yes”, and clicks “Save” (which is surprisingly hard in Magento).

Conclusion

So what is Magium?  It is a fancy batching tool for Selenium tests.

How should you start?  Read the tutorials.  (Very much a work in progress at the time of writing)

How can you contribute?  Check out Magium and MagiumMagento and contribute.  I’m one of the nice open source software maintainers and I promise I won’t be an ass about contributions.

Follow MagiumLib on Twitter (nothing posted as of writing).

Important notes

• This is not an official Magento project – it is done entirely on my own time to solve my own problems
• I have no idea how or even if this will fit into Magento 2. I have barely touched M2 and, more importantly, the Magento Test Framework.  M2 MTF may render this whole thing moot.  I’ll let you know when I actually have time to get to know Magento 2.
• This is nowhere near a v1 release.