A few weeks back I asked a question on Twitter about browser testing. There weren’t a ton of responses but the responses I got were about what I expected. Around 80% of Magento developers (and I would probably say web-developers in general) do NOT do any significant browser testing automation.
The reason that doesn’t surprise me is because browser testing is kind of hard, time consuming, and often requires a fairly significant investment to do it. I know this because I spent a modest portion of my summer last year trying to do it for an important project. If you are a really good developer, or a professional QA tester, then this may sound like absolute nonsense to you. But I think that when you look at the market at large this is a truth that you will see.
Note that I don’t believe that this is a symptom of developer competency or anything like that. My perception is that this is largely due to the cost/investment required to automate browser testing.
For that reason I started working on a project creatively called Magium. I won’t speak much about it here (check out the website for more information). My goal was to remove many of the barriers to doing effective browser testing on Magento. I believe that is something I am achieving, though have not achieved yet. As it stands Magium supports a lot of things
- Add-to-cart functionality (automated (“best guess”) and specific)
- Guest Checkout
- Registered Customer Checkout
- New Customer Checkout
- Frontend Navigation
- Customer Login
- Customer Creation
- Customer Navigation
- Customer Order Extraction
- Search (including search suggestions)
- Layered Navigation
- Product Extraction (Category List, Category Grid, Search List, Search grid)
- Admin Navigation
- System Configuration Changes (this was not easy to do)
- Admin Order Handling (this was even harder to do)
plus a bunch more
If all of this interests you I would invite you to take a look at this video. It’s a little long, but it is a tutorial that will take you through the basics of building Magium tests through doing a full checkout all the way through managing the order in the admin.
Using Selenium.
In about 20 lines of code.
Yeah.
Comments
Bob
Have you thought about using http://codeception.com/ ?
Kevin Schroeder
I have and it looks nice and clean, though I have not used it myself. The problem is that, based off of what I had seen on the site, the way I would be using it would be YAAL (Yet Another Abstraction Layer) that I would have to debug. Then at the end of the day, based off of the approach I’ve used for Magium, the end user wouldn’t see much of it.
Bryan Veloso
Your title is very true, I think it’s 90-ish %. We do manual browser testing / QA and outsource it to 10 or more different freelancers, to test. And yes, it’s relatively expensive, but it does the job.
Currently watching your Magium video now to see if we can use it as an alternative.