Pillars of software testing.

You can't stack another deck...

---

...when your pillars are not okay.

To my opinion, and I admit it's a bit of my vision, there are a couple of pillars in software testing. One of the pillars is unit testing, the other one is test automation, another one is what I call UAT and the last one is "the process".

All (at least) 4 have to be in place to proceed to another level of software testing within a company. You don't want a wobbly deck, right?

Okay so let's zoom into those 4 pillars.
Many times the question arises are those test techniques that TMap, ISTQB, ISEB and others school really needed?
Well yes and no.
To understand this we'll have to take a closer look on where they can be applied.
Let's just say -for the convenience- , that there are 9 test techniques and one of them is the decision table
-
The technique the decision table can be applied on the pillar unit testing so for instance you build a piece of software code and the verify if all the correct behavior represents as desired you test all the cases.

Writing UNIT tests based on programming code

Remember, the decision table is "just one of the 9 techniques" that  can help you in the process of verifying if the software is build according to plan...

Okay next up, so you used the decision table and verified that the code you made produced the correct outcome in all cases. (and you build some nice automic unit tests for it as well.) Does that mean the software is bug free? no, not necessarily.

Let's go to the next pillar, 'Test automation (Graphical User Interface)'. Remeber you wrote the code, it runs in the right ways and you verified it with a test technique.
-
Now we take a closer look at that other pillar, we are gonna test the results of the code on screen. Because we want to verify that the form/page (text on printed page,  ... etc, whatever) produces the correct results.
One of the methods we can use is a test technique ofcourse (or common sense just for fun), we pick the decision table again... but we could have easily switched to another technique like the decision tree  in case we 'mind-mapped & brainstormed' that one was more appropriate, keep that in mind.

Graphical User Interface, test automation (RobotFrameWork, Tosca, Selenium Fitnesse, etc)

So at this point you're testing a flow and use a test technique to verify if the results on screen match te requirements. In one of those flows the code that you tested in the pillar unit testing is also "touched".  (So in the flow of screens and input fields you're implicitly testing something that was tested, (@Michael and the other one: sorry I meant...implicitly checking something that was checked on another level) make sure that you understand this before you continue reading. It's important). (I warned for cryptical explanations and otherwise out-of-the-box references)
-
Okay let's just assume so far you understand my cryptic way of explaining things somehow.

We discussed 2 pillars and we used test techniques.
-
Did we apply the right test technique? That's hard to tell, hard to predict.

§§ Do you cover (test coverage) everything if you apply a test technique? probably not. It is very hard to explain how this works but will you always win a fight if you apply material art techniques? well no. Sometimes it might even be best not to use a learned technique and improvize. So are certificates in software testing usefull? well yes and no...
They might come in handy, but it's no guarantee that someone without it is a lesser software tester! keep that in mind!
-
Next level...

**The code changes**

because the customer had a desire, dus the code had to be changed. What has to be retested?

Do you have to retest (recheck) the code if it works as desired in all cases?
Well yes, looks like to me you want to search for bugs.
(It would be a bit strange to check for laws and regulations accordance first and with a change of the software code just let it be, right?)
-
Is the same test technique that you used before still the one that is most suitable?
This might not be the case, the code got more complex and changed bigtime, now it's more suitable for another test technique.



Is thi§§ still applicable? Well yes why wouldn't it be?



Does the change-of-code effect the results of the other pillar 'test automation'?
Well yes, just because the results of the input fields and "final page" where okay at one point in time.
It still can easily be the case that the flow that touches that part of the code, now will result in a failure. test nok
-
might it be the case that improvizing is now better then a test technique?
yes might be the case
-
might it be the case that another test technique is now better?
yes, or a combination of both
-
okay, fast forward >> into the future...

All flows (chronological order of input fileds) are test- automated on the gui.
It produces all the desired results.
all unit tests pass as well.
-
**The GUI/website, front-end changes**
Customer sayin':
I'd like to order of the input fields re-arranged, I'd like the input fields 3 6 and 9 to be appeared on website page 1 and the input fields 10 and 11 on website page 4.
-
What is the effect on the pillars you think?
-
Does the test automation of the GUI have to be altered?
yes.
Do the unit tests have to be checked again?
yeah (why build them, to never run them ever again, right? )
-
so far...
We have mentioned only 2 pilars, 1 test technique.
And the fact that improvizing can be better sometimes then a technique at all.
-
Does your company have "all" (at least) 4 pillars I mentioned in this blog in place? (unit tests as wel as... as well as... as well as - at least an attempt to understand test process-
(Most companies don't to my opinion...)
-
Do you have any idea how many scenario's, cases, user story's have to possibly be re-checked (and test cases/techniques reconsidered) when 1 tiny bit is changed? (and that is only the checking part of testing, not even the "creatively search for a bug by means of improvizing or...." etc)
-
all in all...

Grosso modo: when 1 pillar changes, all pillars have to be checked.
-
Take this into account the next time you hear someone say:
'Is testing done yet?'
(sadly, some people just don't have any clue how complex it can get to (in their eyes, and what they call) "test everything".)