Testing is Rocket Science Not Brain Surgery

Almost all of the World's Greatest Accomplishments Were The Result of Great Planning!

Testing is Rocket Science Not Brain Surgery header image 1

A Side Note: “Faith in the Machinery” by Ed Cook

August 29th, 2010 · A Side Note

After the explosion of the Space Shuttle Challenger, Congress formed the Rogers Commission to investigate the causes of the accident, both technical and systemic, that led to the accident. One of the members of that commission was Nobel Prize winning physicist Richard Feynman. In his report, he wrote:

“It appears that there are enormous differences of opinion as to the probability of a failure with loss of vehicle and of human life. The estimates range from roughly 1 in 100 to 1 in 100,000. The higher figures come from the working engineers, and the very low figures from management. What are the causes and consequences of this lack of agreement? Since 1 part in 100,000 would imply that one could put a Shuttle up each day for 300 years expecting to lose only one, we could properly ask “What is the cause of management’s fantastic faith in the machinery?””
[

→ No CommentsTags: ·····

A Side Note: “To Pull the Plug or Not, Who Knew The Life And Death Of a Computer Would Depend On A Tester?” by Howard Clark

August 25th, 2010 · A Side Note

To Pull the Plug or Not? Who knew the life or death of a computer would depend on a tester

“To be, or not to be, that is the question:
Whether ’tis nobler in the mind to suffer
The slings and arrows of outrageous fortune,
Or to take arms against a sea of troubles
And by opposing end them.” – Shakespeare’s Hamlet

Does a company choose to start a fire to prove that their fire evacuation plan is valid; something most might consider an extreme measure? Should a tester look to use extreme methods to exercise a particular test case, say an application’s data will not be lost in the event of power failure for instance? As a firefighter must practice saving lives without putting lives in danger, a tester must attempt to simulate real world scenarios, but look to do so in a controlled environment. Recently I participated in a System Test effort for a client implementing a Citrix based deployment of a client server application. While my role during this effort was to serve strictly as a human test robot under the direction of the client and not that of a test analyst I decided to err on the side of caution and question the test script based on my intuition confirmed by the advice of my peers. This effort involved testing several system requirements related to disaster recovery. One step within the script I was assigned was for the human test robot; me, to unplug the power cord from the computer while it was powered on. Yes, you read it correctly, UNPLUG the power cord. The expected result I was instructed to observe was zero data loss, when really the more likely outcome would be a hard drive crash, which would definitely lead too loss of data, just not data in the AUT.
[

→ No CommentsTags: ·······

A Side Note: “Fighting To Maintain A Tester’s Integrity” by Bobby Washington

July 6th, 2010 · A Side Note

“Oh, what a tangled web we weave, when first we practice to deceive.”

The operative word here is practice. As testers credibility is just as important a skill to possess as any. However, anyone involved with software development for any length of time can agree that sometimes pressures from management can try to force us to alter or misconstrue our results, often met with grave consequences if we do not comply. Recently my wife applied for a part time job. The hiring company provides the contact information of executives and other high profile individuals of a particular organization to other companies that are interested in marketing to them. Needless to say this privileged information is not easy to come by. During the week of her training my wife noticed people essentially telling “little white lies” to garner this information. Nothing really damaging, but it would seem that professing to be a college student working on a terms paper was a tried and tested technique for acquiring the necessary contact information. My wife explained to the hiring manager that however innocent, this approach made her uncomfortable. Fortunately the hiring manager understood and still proceeded to hire my wife. Continuing on this theme, I just finished reading an article on www.stickyminds.com by Fiona Charles titled “No Compromise”. In this article she explains the fear testing professionals have with communicating less than desirable information and how to overcome them. Many times we read articles that detail different techniques and technical skills we can utilize to aid in performing more efficient, consistent, and robust testing. However once we have our results, how do we communicate this information in hostile environments? How do we communicate our findings to individuals with the professional title and influence to “spin” our findings in a way to protect themselves and/or further personal agendas by attempting to put lipstick on the pig?
[

→ No CommentsTags: ·······

A Side Note: “Do More with Less…Can Someone Tell Me How?” by Bobby Washington

June 3rd, 2010 · A Side Note

Testrocket.org is very pleased to open up to Bobby Washington as it’s newest contributing author. Bobby brings a fresh perspective from the viewpoint of the intended reader of this site, the practitioner. He is a trusted colleague and valued contributor!

“Do more with less…”,less resources, less time, less money. If you have worked in IT for any length of time you will have experienced this mindset. In software development, testing is typically considered an afterthought. Typically, when there is a need to bring your project schedule in, cutting the time spent testing is the solution most often chosen. To which I say, “It sucks and I hate it.” Strong words I know, there I got the venting out of the way. There is always going to be less time and money to test to the real extent necessary. Now as career software testers what we must realize is that we ca not allow this mindset to force us to test with less passion, less creativity, and less skill. Since the odds of us receiving more time to test before a production release are small, it seems logical to find a way to conduct additional and/or continuous testing after the production release.

Just as IT management must justify the cost and value gained for a new change or system, we too must justify the cost and benefit of executing a test or suite of tests after a production release separate from any regression effort. When you think about it, developers embrace the concept of “Continuous Integration” .
[

→ 3 CommentsTags: ·······

From the Field: LoadRunner and Citrix

March 1st, 2010 · Testing From The Field


You need to understand that any project requiring a performance test of an application deployment via Citrix is not going to be your casual record and playback affair. So begin by digging up your old C reference manuals, actually they should be close by at all times when working with this tool, but for the neophyte a good reference is a must. Next get a lay of the land for what you can monitor over the network and what key Citrix performance counters you need. Then start to steel yourself for the onslaught of maintenance, you’re going to be using snapshots of defined screen areas to validate whether something has loaded on the screen or finished. What this means is any GUI changes or minor differences in the way the GUI renders (something that happens even when there are no screen changes, go figure) will require at least another run through of all your scripts and making the necessary bitmap hash code updates. LoadRunner provides these for you; it’s just a labor-intensive affair. Factor in that each load injector may render a little differently and you have the potential for a lot of debug pass throughs.

My colleague and friend Chetan Rahul of Fusion Alliance based in Indianapolis was nice enough to take me up on putting together a PowerPoint outlining some of the issues and lessons learned when we began our LoadRunner Citrix engagement. Enjoy!

NOTE: HP has provided some enhancements through a runtime setting that allows for the tool to accept some differential in what is rendered versus the snapshot originally taken, unfortunately despite patch 9.52 I’m still not seeing any benefit from changing the image tolerance.

→ No CommentsTags: ·········