I have heard, in many organisations, speed being cited as a primary benefit of automation. In most cases the use of tools will make things quicker. But faster doesn't necessarily mean good. It's important to assess automation against other criteria.
The development team, not only the testers, should regularly reflect upon and discuss their automation. One approach to this conversation is to have each team member indicate on a scale whether they agree or disagree with the following statements:
I understand what is being checked by our automation
I feel confident that a failure indicates an important problem in the application
I do not manually repeat those checks that are automated
I find it easy to diagnose the cause of a failure
The collated answers will help to uncover whether people understand and agree with coverage, whether they believe the chosen implementation of automated checks is robust, and whether they are capable of investigating problems that are discovered.
Automation may execute quickly, but without assessing its value it may be quickly doing nothing useful.
At the cost side of the equation - another very important considiration may be the amount of repetetions (reuse of test automation artifacts) this may change all test automation ROI.
ReplyDeleteYour last sentence pointed the whole thing out. Test Automation is for 'execution' of those automated tests that were built by a human. All a machine does is take what it is told and runs it. No AI here folks. Automation is a way of 'automating' the process of executing tests (or checks as some so stubbornly like to call them, whatever).
ReplyDeleteBy leverage a machine you get efficiency gains that relate back improved performance, or perceived performance. The true benefit is to get to the stage where you have multiple machines running tests in parallel. I like to call this the "Illusion of Speed". The total time to run all tests is the same but by dividing up and running sets of tests in parallel you get a perceived compression of time. So if a set of automated tests takes 8 hours to run on one machine then if you had 8 machines and can run all the tests independently (no interdependencies) then you can divide them up across the machines and make it appear that it all ran in an hour.
That is the Illusion of Speed, leveraging economies of scale.
Jim Hazen
But only code reusability is very less It seems work only for project from similar domain.
ReplyDeletePavan T