Tuesday, February 7, 2012

Transforming the way testing is performed...part one

Glossary: Test Strategy - High level / abstract test plan, usually less than a page.
Caveat 1: This is how testing is done where I currently work. Is it the way testing should be done? For you, maybe not. For us, it works out pretty well.

SETTING:
We use scrum, 2 week sprints, and mostly agile :)

Each 'feature' is a story in scrumworks, or multiple stories depending on feature size. Features can be broad or narrow, depending on the area under work and the PO / team. We have 8 active feature development scrum teams.

As a story is brought into a sprint the QA person writes an initial test strategy: thoughts about how to test the feature, what is the target, different ways to think about the feature, what areas are we NOT going to test, etc. The strategies should be specific to the work for that story and be related to other work in subsequent stories if the feature span’s multiple sprints.

The story is implemented by development and tested by QA until conditions of satisfaction are met. Then the test strategy is updated to reflect what was actually done. Notes about things that might be useful and/or not immediately apparent to the next person are added to the test strategy.

This next part is the one we are currently working on. The automation team takes over. Between interpreting the test strategies and having a conversation with the QA person, the SDETs come up with an automation effort that includes specific test cases. This allows the manual testers to not have to manually regress old features that we are not actively testing. (More of this will be in Part 3)

EXCEPTIONS:
Some features never get a test strategy. Example: "correct the link on page X to now be Y". There is limited usefulness in agonizing over this trivial change. Examine the new page...verify functionality that might be directly affected by the change, but truth be told it's most likely a five minute testing effort. If it takes you longer to write the test strategy than it did to actually test it...don't.

If writing a state diagram is easier then writing up a test strategy do. IE, if there are 3 states that can have 4 results, those 12 possibilities are easiest to just write out, rather then try and abstract them into a test strategy.

Sometimes just writing the test cases out is the quickest method. I frown on this method cause it tends to limit the thinking of the tester as they are performing their software testing. We have seen some cases where X, Y, Z are the test cases and thinking outside those really are just 'trying' to complicate matters in an area that is minimally useful.

REASONINGS (behind this madness):
We hire smart, intelligent people with good judgement. I leave it up to those smart people to know when any reasonably trained QA person could test a new feature with no previous knowledge. Or those same people could test something with the limited knowledge included in a test strategy.

We have the standard feature creep, emergency injections and other areas of software reality. This is where the judgement of the people I've hired comes into play. I trust them to make the right choices and I trust them to be able to defend those choices.

Do people make mistakes? Yep, probably more then I'm aware of, but the system works pretty well given our environment, our systems, our people and the speed at which we move. This allows us the flexibility to provide minimal documentation (given a feature and it's importance), and not have that documentation slow us down anymore than is necessary.

2 comments:

  1. The flow is well explained. I agree with your statements about trusting those that you work with. (or that work for you) With trust comes initiative and creativity which in my opinion vastly outweighs the few mistakes that are made along the way.

    ReplyDelete
  2. Mad Scientist Goes QA Test Lead.

    Film at 11.

    ReplyDelete