Introduce self, what it takes to build and maintain a web automation system.
My story!
I work at Sauce Labs, we are trying to make your life less painful.
This is my cat, he’s a nice guy, and yeah - I am a real person.
How many people are doing any sort of web automation?
Of you how many have to deal with flash?
Awesome red wrench.
When designing your automation system, be very careful about the tools you choose.
General
Boxes - Great starting place, don’t scale that well
VM’s - Expensive hardware and licensing, no recurring costs ++
Cloud - hard to test internal apps without tunnels, recurring costs, slow
You need it, evaluate, pick one and go from there.
I’m not going to go into the “best one”, they all have strengths and weaknesses and you need to decide for yourself which fits your needs.
If you can get away with not having a JS enabled browser, run your tests without one.
Demo FlexPilot in Sel IDE
Eventually you will need to expand your setup
Upgrading VM servers gets expensive
You should decide what configurations you really care about, because the matrix gets immense
The cloud stays pay per machine per minute
JavaScript is a great platform for testing the web
However it has some shortfalls when it comes to integration with various services
You need to focus on what makes your QA/Dev teams the most productive
What are your users using? That plays a huge role in your technology choices
If you can test just safari, IE, chrome and FF on windows, you should focus on that
Smoke test macosx and linux as they are probably a tiny portion of your userbase
Hudson running a Selenium test
There are a few schools of thought behind how testing should be done, lets explore those
Functional test your user experience
Unit test your backend code, integrate as much as you need and run those tests often
I would advise you to separate those efforts functional testing and unit testing are important for different reasons, and different points in your process
True/False can tell you if 100% of your functional tests are passing (which is important), but its also incredibly important to know exactly which aren't passing
Know all the time the status of your system, and be dedicated to it.
Automation is at a point where it has proven itself
All elements of your company should be involved
Take it seriously, if your testing suites aren’t stabilized, releases shouldn't go out
Start with what matters, your money path
Improve that small set of tests to the point where failures really mean something serious
Shamless
Running a test in the Cloud
This does not scale, do I really have to talk about this?
I have told you all the different kinds of solutions that you can chose and how to tailor each of them to your specific needs, please go make it happen.