Test Engineer

I took time to comment on James Whittaker’s excellent blog post on the roles Google has defined within it’s testing organization and I wanted to share it here.  The basic concept is that the development and testing roles can be defined as Software Engineer (SWE), Software Engineer in Test (SET) and Test Engineer (TE).  The breakdown of tasking is described as SWE’s write functional code, SET’s write unit test code and modify function code to make it more testable & TE’s write integration & user scenario test code as well as coordinate the testing work of the SET’s & SWE’s.  Wow, that’s a lot of code being written by everybody & only one of those roles is writing code that customers will actually use.  I commented to disagree with the idea that all of these roles should be focused primarily on cranking out code, particularly the Test Engineer.  I’d love to hear what your thoughts and experiences with the matter are.

“My experience as a titled Test Engineer at an aerospace company aligns well with James’ description with the exception of the statement that I spent most of my time writing automation. A majority of my time was spent working with the customers on how they use the system, researching test methodology & learning/testing the system I was working on in the lab. I typically performed those tasks in that order, over & over again, week by week until the project was complete. The automation I used as part of my testing was to simulate a random input generating live system that communicated with our system via an API. I needed to be able to control the input so that I could test different user scenarios & that’s where automation became useful. However, I did not write the base test code, the code was written by an experienced developer who modified an existing script in a matter of minutes to do the basics of what I need. It was a fairly simple program so I was able to expend it to simulate exactly what I needed. The customers were impressed that I was able to not only create scenarios to test individual events & functions but that I was also able to create interesting real world scenarios to show how the functions integrated. I was able to design a large volume of user scenario tests because I was I had the benefit of a development expert modifying a piece of code for my own testing use that I could further expand as my test ideas expanded. A TE should be focused on how the customers use their system first & foremost. Automation is a tool to help them achieve the best possible scenarios in the smallest amount of time.”

Why Electronic Intelligence?

Electronic Intelligence or ELINT  is the gathering of intelligence by using sensors to detect electromagnetic emissions from a system for use in locating, identifying and classifying the source.  I think this description is a good abstraction on what we do as software testers.  The sensors we use are automation, exploratory methods, error logs and our judgement to locate and isolate faults.  Once the defect is found and documented we can then begin the task of: identifying it’s taxonomy, figuring out how it was introduced and adding it to the regression list.  Gathering intelligence and classifying threats within the system is just one part of the equation.  The tests we write have to be targeted to create the most harmful or frequent failure conditions, often with a limited amount of time and manpower.  Compounding the problem, our production software exists in a dynamic environment full of different types of users: friendly, hostile and somewhere in-between.  These users have different expectations and ways of interacting with the system that can have subtle yet malicious effects over time.

There are many sources of intelligence about what you are testing.  The primary and often most elusive is the developers themselves.  I would have to say that most of the empathy and user advocacy skills that a tester can develop could most effectively be directed towards the developer.  Software developers have the most difficult and abstract interaction with the code and system as a whole, and often, the narrowest.  This is where the tester’s ability to use research, problem solving and heuristics comes in to play.  Empathy with the end user should be the default state of mind for the tester as they as are the user advocate in the development & testing process.  Testers can be thought of like spies, wearing the disguise of the user to trick the system under test to showing  it’s weakness.  Once that weakness is understood, you can bring in the main force of testing assets to bear on the vulnerability, mitigating it to a pre-integration automated check or a piece of intel that can be used to find other weaknesses in the design.

This is a loose & fun association that we’re playing with but it is also with a purpose; to use one context to help understand another.  Please leave a comment if you can expand on this abstraction or have thoughts on cross-context explanation.