In the previous chapter I was talking about the 'Test Pyramid'. The Test Pyramid is about the 'HOW' of software testing.

But there is also something to say about the 'WHAT' of software testing.

In order to speak sensible about the what of software testing, you first got to understand that programming language (code) is mostly put in a code base.
Suppose you would see that stream of 0's and 1's as a fluid.

(All programming code basically is a stream of ON/OFF switches to the processor of the computer anyway, right?)

Keeping that as a thought.
You (as a project or company) have got either one of both systems....
namely:

One is like a system where every new code is going through a process where is monitored what is put in the new release. I always see this kind of system as a perfect funnel where not 1 bit (literally) is spilled. Nothing is uncontrolled put into the new release. It is completely clear what is going into the new version.

The other version is a system where some persons have the right(s) (for some reason) to commit and push something into the new version of the build. And if you think 'that never happens', trust me I have seen it too many times! It is untested, still it ended up in production.

You can visualize it something like this:

In the beginning of a Agile Scrum Sprint it is determined what parts of that new code (the 0's and 1's... the... FLUID) is going to be tested.

The "fluid is on the radar" you could say haha.. ;)

A selection of the code that is going through the funnel is within scope, some of that fluid is going to get tested.

And you see, in order to deliver high quality software code and thus a high quality software end product. You and your company, your Agile team is in need of certain rules.

Those rules can also state or write something and clarify  if anyone is able to commit and push something in the Master, any code at all, that was not on the plan board (and part of the test effort) of the current Sprint.


Be honest about it. I have seen it too many times, yeah that can't happen. But in reality... it's things like:
Oh he? He is a really REALLY senior developer and he doesn't make any mistakes, continue with the Sprint please!


It's either a spilling funnel, or it isn't. It's one of both!
(I don't want it to make it any more complex then needed, but just remember for a moment where there also a situation where it is not clear who is developing on the team and maybe who is not. check for that blog but let's not over-complicate.)

Why is it important? Well if you do NOT take it into account, sooner or later bugs will pop-up in production that are related to the code that was spilling alongside the funnel!!

Okay now you know about the importance of that (WHAT (previous blog)). And you also know about the checking (HOW (this blog)), now it's time to indulge yourself in the..... WHEN software testing is complete....

To be more precise: WHEN is software testing done?
(after all we stated, the Sprint is DONE when software testing is DONE.
It is in the Definition Of Done. RIGHT?)

All team members committed to it!
(click here if you want to continue reading about one of the best real life oriented anecdotes regarding committing to Sprint Poker planning).

In case you're interested, gaining knowledge about
(link 1)'QA Sign Off'
(link 2)'documents'
is a very good starting point for your next step in this chain of knowledge blogs about software test processes.
I'd advice you to read until you understand why they exist. Why do people use them?

If you're at that point of understanding, then you are ready for Blog #3, which will be soon ready on one of the days of the up-coming weeks...

credits to Bing a.i. for the possibility to create such wonderful generated images!

If you want to talk about software testing, there are a couple of things you need to understand.

A tester on a project is in need of certain skills and knowledge. If you want to learn more about it, take a look at this page.

Notice where the ' Tools' sections is located!

Those tools are used at several layers of the test pyramid.

They can be aimed at UI level (Tosca, Selenium, Robot FrameWork, etc)
or Service Tests level (SoapUi. etc)
or unit tests level (..etc...)

Those tools have different kind of 'Programming experience levels needed'

Some tools you DON'T need any programming language experience at all.

Some tools you can work with if you have a little bit of programming experience.

Some tools you can only work with if you are very EXPERIENCED with programming languages.

You also got to understand, with some (framework) tools where you need a little-bit more then average programming experience, they can be used with several different programming languages as the basis:

That is not all. The knowledge you need about using the (framework) tools can be divided into sections:

first:
You need knowledge to know how to find the web elements, by means of RegExp, name, ID, Title, or CSS or XPath.

second:
(in some cases) You need knowledge about OOP.

third:
You need knowledge about the used programming language in general.
(Which is in it's turn actually a whole different world of knowledge on it's own and a "15 levels deep presentation" in Prezi.com worthy... click here for a basic overview in this case C# as an example,
And if you know all these basic C# things, then that doesn't mean that you know what .NET is, which is in it's turn.. well you guessed it... but click here if you still wanna dive deeper and get a headache because of an information overload.)

fourth:
And then you need the specific knowledge that is test automation related like how to give arguments, how to use the web-driver, how to use Docker with the automation etc....


Tip! as a recruiter/job interviewer: specifically ask questions in those 4 sections!! example: what is your knowledge about locating web-elements with test automation? (or OS winforms elements in case of a Windows application of-course)

So you see... although 'Tools' is only a small sub-section of what a software tester comes across on a daily basis (the complete picture). The whole understanding of using those 'Tools' is a complete world on it's own regarding the knowledge about it.
(Imagine: if it was a Prezi.com presentation we would have been 10 levels deep under/into the green arrow section by now...)

So in case you are providing a job opportunity to a software tester, make sure you prevent them from having to ask questions like:

-What are the programming languages used on the project (in general)?

• Can I view the current code(base) as a software tester? Can I also push commits and contribute to the codebase?

-What are the tools currently being used? oh is it Selenium? is Selenium used with Kotlin?

-Oh only a low-code automation tool is used? What tool is it? So does this mean I don't have to have programming language experience?

-Oh Okay Selenium with C# is used? Do I have to script taken into account OOP?

-Are there currently tools used on the integration level of the test pyramid?-

What is the balance between the test effort I have to put into 'Explorative testing' vs the 'automation' part? oh 75% Explorative testing aha okay.

-How many other testers are on the project? What is their level of test automation knowledge? oh very inexperienced, they don't have any Kotlin knowledge... aha okay...



Software testing is already difficult as it is,
so please provide as much information as possible in a (online/digital) test job opportunity!
("even if it looks obvious to you, but not every other company also uses Java!")

After-all 'time=money' and if a candidate comes to a job interview and it was completely for nothing then it's a waste if the answers of those questions could have been given by forehand.

Or even worse: they don't get asked at all and when time passes by it gets clear that a candidate has too little of the knowledge that is actually needed at some or several levels.

Also: (and this can be a little-bit contra-dictionary after reading the text above) remember that not everything is about the tooling aspects. If you have someone with super duper test automation knowledge, but he can't find a descent bug... I'd rather have testers that are great in discovering the BIG bugs, right?

So now you have some guidelines for the job description and the interview and hopefully a bit better understanding about software testing in general.

Thank you!

In software testing it is important to understand that changing something in one place, can create a bug somewhere else. I always see the 'Newton's cradle' in front of me when it's about this topic somehow. 

Lately I saw videos on YouTube about the UK post office Horizon scandal. I watched 2 actually.
This one and this one.

​I had a lot of thoughts when watching these two videos. 

We life in a digital world and there is a lot of digital crime helaas.
When we go to the airport our luggage's are being scanned. There is a lot of monitoring going on. Of course, this is needed, and we all understand that. 

We also know that in this digital area we are in, there might be some monitoring and control going on that is not just as easily visible to civilians as can be seen on the airport. 

Still those systems have to be tested. (as a whole, or as a test object)

So the question is: is in all the cases that you, as a software test consultant, are on a project, all of the code presented to you? Do you know about all of the functionalities that exist?
Are some (parts of) the code maybe restricted (for only certain people)? Or maybe not even visible at all?

I think that with some software it is the case that not every developer can view the code, not every developer can make adjustments, inherently not every software tester can test with the full knowledge, or full scope.

This means that it is possible to create a test plan, or a test strategy based on the code or functionalities that you do know of that it exists. 
But you cannot create test cases based on functionality or code that you don't know of.​

I think, sometimes when you want to do a bug root cause analysis, it is important not to have a tunnel vision (or narrow minded). But try to keep thinking outside the box. Maybe there is some info missing? People tend to have biases. 

In case there is some code that can only be viewed by certain people, and in case that there is a bug found in specific that section.
Then who's responsibility is it to check that there is not a 'Newton's Cradle' effect in the "visible for everyone" code section?
Who is going to do the regression testing of the complete system?
Who is going to do the integration testing of the complete system?

What happens if the not-hidden code is ready to go into production and online, but changes in the hidden section actually cause some counters to malfunction?

I think it can make the testing (process) very complex from time to time. Even opaque.

You don't want that the testcases of the "visible for everyone to see code section" turn up green(passed).
Whilst if the "confidential section is also taken into account they actually turn up red(failed).

Is it even possible to communicate clearly about all these things always?