December 30, 2009

QA Q and A - Release Tests



Question:  What are Release Tests?

Answer:  Tests designed to ensure that the code (which has already been thoroughly tested and approved for release) is correctly installed and configured in Production.  This is often a fairly quick test, but may involve some migration tests as well.

In my company, we develop applications that we host on the web.  When the development and testing cycle is complete, we must release the new version to Production.  Part of that release involves a different kind of testing that we call "Release Testing".

Because of our Service Level Agreements with our customers, we often have to perform these releases after hours - when most users would not be using the system.  For some of our systems, that means any time outside of weekday business hours.  For other systems, that means Sunday evenings.  For still other systems, they are required to stay up essentially 24 x 7.  Those systems require special architectures that let us make most changes while the system stays running.

Our developers create a "build package" containing all the code, scripts, etc that must be moved to Production, along with the instructions for doing so.  The IT team actually make the changes.  And QA performs the Release Tests to make sure everything went as planned.

It wouldn't make sense to test all the features of these applications in Production - they have already been tested extensively in the QA Environment (and sometimes in a Staging Environment), and approved for release.

Instead, we test with a specific goal in mind - to check that the new release arrives safely in Production, and that the application is up and running correctly again.

When designing a Release Test, the QAer must consider how she/he will know that the new code is now running in Production, and hasn't been missed.  Often, this means quickly checking whatever changed in this release - perhaps new features, perhaps modifications to existing features.  In addition, we want to ensure that things that shouldn't have changed remain running as before.

When problems are found during a Release Test, it sometimes means the abandonment of the release, and a roll-back to the previous state.

Some of the attributes of our Release Tests:
  • planned
  • quick
  • often after-hours
  • often under time pressure
  • sometimes coordinated with customers
  • designed solely to check that the code we have been testing all along is safely in Production, without adverse side-effects
  • we don't re-run all our prior tests
  • sometimes, must test each server in a load-balanced group individually
  • check that configurations are correct
  • sometimes, check migrations from the prior version
  • sometimes, must check that all active users are first drained from the system
  • sometimes, must check that the "Under Maintenance" page and process are working correctly first
  • pretty much any new bug is a showstopper bug, requiring roll-back
For other QA and Testing Terms, see:

WinTask - Using ODBC Functions with an Oracle Database

Some Oracle datatypes are not directly supported by WinTask.

For example you cannot use DbGetFieldNumeric() with Oracle Number fields.  When you try, you will get an execution error that indicates the field is not a numeric field:
Error at line 13 : XXX is not a numeric field
WinTask Tech Support indicates that they don't support this type of numeric field.

Here are some Oracle datatypes and how to deal with them:

Then, use ToInteger() or ToNumber() below to convert to a number
DbSelect("select CAST(MyCol as VARCHAR2(x)) from MyTable t",SNAPSHOT)
Then, use ToInteger() or ToNumber() below to convert to a number
Then, use ToInteger() or ToNumber() below to convert to a number
DbSelect("select CAST(MyCol as VARCHAR2(x)) from MyTable t",SNAPSHOT)
Then, use ToInteger() or ToNumber() below to convert to a number
DbSelect("select CAST(MyCol as VARCHAR2(x)) from MyTable t",SNAPSHOT)
Then, use ToInteger() or ToNumber() below to convert to a number
Note: Will lose trailing zeros (e.g. 1.000 -> "1" and 2.220 -> "2.22")
DbSelect("select CAST(MyCol as VARCHAR2(y)) from MyTable t",SNAPSHOT)  'where y = x*2

Some helper functions:

Function ToNumber(test$)
    Local i
    Local new$
    Local work$
    While i<= len(test$)
        Select Case Mid$(test$,i,1)
            Case "0", "1", "2", "3", "4", "5", "6", "7", "8", "9"
            Case Else

Function ToInteger(test$)    Local i
    Local new$
    Local work$
    While i<= len(test$)
        Select Case Mid$(test$,i,1)
            Case "0", "1", "2", "3", "4", "5", "6", "7", "8", "9"
            Case "."
            Case Else

November 23, 2009

A New Version of WinTask - 3.7

The good folks at TaskWare have released a new version of the favorite tool in my toolbox - WinTask.

There are two major new features in this release. 

First, WinTask 3.7 now supports Windows 7 and Windows Server 2008. 
And WinTask 3.7 includes the following FTP functions:

  • #FTPTimeout - Specifies the number of seconds which WinTask should wait before reporting a runtime error when it tries to execute a FTP function
  • FTPChDir - Specifies the new FTP current folder
  • FTPConnect - Makes a connection to the specified FTP server
  • FTPCurrentDir - Returns the FTP current folder
  • FTPDisconnect - Terminates the connection to a FTP server
  • FTPExistDir - Checks if the specified FTP folder exists or not
  • FTPExistFile - Checks if the specified FTP file exists or not
  • FTPGetFile - Downloads one or several files to the local PC from the FTP server
  • FTPKill - Deletes one or several files from the FTP server
  • FTPMkDir - Creates a folder on the FTP server
  • FTPName - Renames one or several files in the FTP server
  • FTPPutFile - Uploads one or several files from the local PC to a FTP folder
  • FTPRmDir - Deletes a folder and its contents on the FTP server
I'm particularly pleased with the new FTP functions. Our systems use FTP a lot - these new features will make it much easier to set up test conditions, and verify output.

Here are the release notes for version 3.7:

Check this tool out at

You can see some of my WinTask utility scripts here, using the WinTask tag.

November 20, 2009

WinTask - DayOfDate$()

' DayOfDate$ - Return the day of the week for a given date
' Author - Joe Strazzere

Function DayOfDate$(TargetDate$)
    Local DayOfDate

    DayOfDate =((Weekday() + DateBetween("d",Date$(),TargetDate$)) mod 7)
    If DayOfDate < 1 then
        DayOfDate = DayOfDate + 7
    Select Case DayOfDate
        Case 1
            DayOfDate$ = "Sunday"
        Case 2
            DayOfDate$ = "Monday"
        Case 3
            DayOfDate$ = "Tuesday"
        Case 4
            DayOfDate$ = "Wednesday"
        Case 5
            DayOfDate$ = "Thursday"
        Case 6
            DayOfDate$ = "Friday"
        Case 7
            DayOfDate$ = "Saturday"
        Case Else
            DayOfDate$ = "I'm lost. I don't know what day "+Str$(DayOfDate)+" is."

MsgBox(TargetDate$+" was a "+TargetDay$)

WinTask - DayOfDate()

' DayOfDate - Return the day number for a given date
' Author - Joe Strazzere

Function DayOfDate(TargetDate$)
    Local DayNumber
    DayNumber =((Weekday() + DateBetween("d",Date$(),TargetDate$)) mod 7)
    If DayNumber < 1 then
        DayOfDate = DayNumber + 7
        DayOfDate = DayNumber

MsgBox(TargetDate$+" was a "+Str$(TargetDay))

November 14, 2009

WinTask - ToDollarAndCent()

' ToDollarAndCent()
' Author: Joe Strazzere
' Removes all non-numerics from a string
' Returns two numbers
'    - the dollars portion
'    - the cents portion
Function ToDollarAndCent(test$,Dollars,Cents)
    Local i
    Local new$
    Local work$
    While i<= len(test$)
        Select Case Mid$(test$,i,1)
            Case "0", "1", "2", "3", "4", "5", "6", "7", "8", "9"
            Case "."
            Case Else
    If Dollars=-1 Then

ToDollarAndCent(" $123,456.789 ",D,C)

November 13, 2009

WinTask - ToNumber()

' ToNumber()
' Author: Joe Strazzere
' Removes all non-numerics from a string and returns a number
Function ToNumber(test$)
    Local i
    Local new$
    Local work$
    While i<= len(test$)
        Select Case Mid$(test$,i,1)
            Case "0", "1", "2", "3", "4", "5", "6", "7", "8", "9"
            Case Else

WinTask - ToDollar()

' ToDollar()
' Author: Joe Strazzere
' Removes all non-numerics from a string and returns a number,
' stopping at the first decimal point.
Function ToDollar(test$)
    Local i
    Local new$
    Local work$
    While i<= len(test$)
        Select Case Mid$(test$,i,1)
            Case "0", "1", "2", "3", "4", "5", "6", "7", "8", "9"
            Case "."
            Case Else

November 5, 2009

Test Data Graduates Into Production - WBZ

My good friend Dan Bourret recently heard that his son's school was being closed due to a high rate of illness among the teachers and students. Just to be sure, he checked the local television station's website and saw this:

Now, Dan also happens to be a terrific QAer, so he immediately recognized what happened - some test data had made its way into production. Oops!

We've all heard horror stories of test data gone astray, and this one seems rather benign.

Still, I'm wondering if there's actually a School Name A (Test) Elementary somewhere in Massachusetts that had an unanticipated day off?

October 31, 2009

It's a Shame - Original Software's Software Testing Hall of Shame

My good friend Jim Hazen has been involved in Software Quality Assurance for a long time.

So noticing mistakes in websites comes naturally to him. And when he recently visited Original Software's corporate blog, he found a good one.

If you go to Original Software's Blog, you'll see that they periodically post about public software failures that they call the "Software Testing Hall of Fame".

Here's one:

These are good articles, and worth reading.

However, if you click the Original Software logo in the upper left corner, you see that they have improperly linked this logo, so you are presented with their 404 page:

When I visited, it said:

Sorry!, No Page Was Found

The page you where looking could not be found

So not only do they have a bad link, but bad grammar, bad spelling, and dicey punctuation as well.

Such a shame!

October 22, 2009

Bugs In The Wild - SQE Training

While reading an email from SQE, I clicked a link and ended up here:

You can see it for yourself at

When I looked closely, I happened to notice two typos.  I have highlighted them in red.
Course Guarantee:Course attendees who do not pass the ISTQB Software Tester Certification-Foundation Level exam within 60 days of completing the course, will be provided with and additional 45 days of free access to the course.
Who's Behind the Training?
SQE Training is affilated with Software Quality Engineering, the publisher of and Better Software magazine.
I guess it wouldn't be so bad if this weren't the registration page for a Software Tester Certification course!

October 13, 2009

Perhaps They Should Have Tested More - Apple

I’m a PC.

I’m a Snow Leopard.

Hi there, Snowy! How are things?

I’m a Snow Leopard.

Oops. Looks like someone is stuck in “out-of-box” condition.   

I’m a Snow Leopard.

Perhaps they should have tested more?

I’m a Snow Leopard.

A critical bug in Apple's Snow Leopard OSX 10.6 reportedly can wipe out users' account information when they open and close "guest" accounts.

  • Snow Leopard has apparently suffered this problem from the beginning
  • Critical bug can wipe out users' account information
  • Data deleted upon logout
  • Complete loss of user data
  • Home directory is replaced with a new, empty copy
  • Was reported at least one month ago
  • Not corrected in the OSX 10.6.1 update
  • All the standard folders have reverted to an "out-of-box" condition

“Users start their Macs up as normal only to find they’ve logged in as ‘Guests’ on their machine – with all the files and data held on their Mac in their own user account seemingly deleted.”

Perhaps they should have tested more.

See also:;txt 

October 11, 2009

Book: Exploratory Software Testing

A few years back, I read How to Break Software by James Whittaker.  I liked it.  It wasn't wonderful, but it had a good batch of practical, useful tips.  Then I read How to Break Software Security and How to Break Web Software.  I liked them as well, but not as much.  Still, I figured I'd read James Whittaker's newest book Exploratory Software Testing.  Sadly, the downward progression of his writing continues.  This book is by far the worst of the bunch.

Chapter 1 - The Case for Software Quality is nothing more than "software is terrific, but it has bugs".   That's it, nothing more here.

Chapter 2 - The Case for Manual Testing talks a bit about testing, and tries to define exploratory testing.  Whittaker's definition has apparently caused some controversy among some well-known practitioners of exploratory testing, so here is his perhaps unique definition:
When the scripts are removed entirely (or as we shall see in later chapters, their rigidness relaxed), the process is called exploratory testing.
Whittaker then divides exploratory testing into two sections.  Exploratory testing in the small is that which guides the tester to make small, distinct decisions while testing.  Exploratory testing in the large guides the tester in how an application is explored more than how a specific feature is tested.

Chapter 3 - Exploratory Testing in the Small was, to me, the only useful chapter in the whole book.  Here Whittaker offers practical advice with examples for thinking about constructing test data, software state, and test environment.

Chapter 4 - Exploratory Testing in the Large is where Whittaker dives into what appears to be the point of the whole book - his Tourist Metaphor.  Apparently this is a big hit at Microsoft, but I found it pointless.  Think about every type of testing you have ever performed.  Now try to torture it into a phrase that ends with the word Tour.  There you go - that's the chapter. 

Just to give you a flavor, here's a list of all these Tours, and their variations:
  • The Guidebook Tour
    • Blogger's Tour
    • Pundit's Tour
    • Competitor's Tour
  • The Money Tour
    • Skeptical Customer Tour
  • The Landmark Tour
  • The Intellectual Tour
    • Arrogant American Tour
  • The FedEx Tour
  • The After-Hours Tour
    • Morning-Commute Tour
  • The Garbage Collector's Tour
  • The Bad-Neighborhood Tour
  • The Museum Tour
  • The Prior Version Tour
  • The Supporting Actor Tour
  • The Back Alley Tour
    • Mixed-Destination Tour
  • The All-Nighter Tour
  • The Collector's Tour
  • The Lonely Businessman's Tour
  • The Supermodel Tour
  • The TOGOF Tour
  • The Scottish Pub Tour
  • The Rained-Out Tour
  • The Couch Potato Tour
  • The Saboteur Tour
  • The Antisocial Tour
    • Opposite Tour
    • Crime Spree Tour
    • Wrong Turn Tour
  • The Obsessive-Compulsive Tour
Perhaps the idea of calling UI Testing a Supermodel Tour appeals to you, and will make for a richer, more productive set of tests.  I don't get it.  I just don't see any value here.  Doesn't testing have enough variation in language and definitions already, without adding this silliness?

Chapter 5 - Hybrid Exploratory Testing Techniques tells us that it's acceptable to combine scenario testing with exploratory testing.  Then it spends time rehashing each of the tours from Chapter 4 and tries to suggest a side trip for each. 

Chapter 6 - Exploratory Testing in Practice presents essays written by several Microsoft testers describing how they each used one or more of the tours in a testing situation.  It appears as if Whittaker instructed his charges to write a "What I did this summer"-style  essay, in the form of "How I used Tours to do my testing". 

Chapter 7 - Touring and Testing's Primary Pain Points tries to tell us (in a few paragraphs) how to avoid five pain points - Aimlessness, Repetiveness, Transiency, Monontony, and Memorylessness.  There's little real instruction here.  For example, we are told that in order to avoid repetitiveness, we must know what testing has already occurred, and understand when to inject variation.  Uhm, ok.

Chapter 8 - The Future of Software Testing has nothing at all to do with the other chapters, or exploratory testing.  It's basically Whittaker's gee-whiz vision of what might be possible (some day) in the future.  Perhaps.  Whittaker has given this talk in several webinars - it's simply rehashed here.

Since these chapters take up only 136 pages, and obviously aren't enough to fill out a real book, three unrelated appendices are bolted on.  A few pages about Testing as a career, and a bunch of pages lifted directly from Whittaker's blogs fill out the book to over 200 pages.

If you really want to learn about Exploratory Testing, this is probably not the place.  Exploratory Software Testing is fluff - stretched and tortured out barely to book-length.  There's not much in the way of learning here.

And if Microsoft testers are really instructed to "Tell me what kind of testing you did today, and make sure it ends with the word Tour", then I feel very sorry for them.

Also see:
And here's my Amazon review of this book, along with comments about my review:

October 8, 2009

Checking a List of Sites Using Xenu Link Sleuth

A new member at SQAForums asked:

"I test an online booking website which links to over 60000 client websites.

I am looking for a tool that will allow me to check their URL's to ensure that they are still valid.

I am trying to find a tool that will allow me to create a script that will reference the URL from a spreadsheet and send a query to that URL and get the HTTP Header response (to see if it returns a 404).

I have been looking for tools and the ones that I have found seem to check the link then expands out to check links on that page. I don't need (or want) that to happen. I just need a tool that checks the header response for the URL provided then moves on to the next one in the list."
I use Xenu Link Sleuth for this sort of thing. It's free, but not open source. It's also easy to use, and very fast.

Xenu can be set to check a Maximum Level of 0 - indicating that it should not spider the site, but just check the top-level URLs.

Here's how to do that:
  1. Create a text file containing all the URLs you wish to check, with each on a separate line
  2. In Xenu, select Option, Preferences... and set Maximum Level = 0 in the Options dialog
  3. Set any other Options you choose copyrightjoestrazzere
  4. In Xenu, select File, Check URL List (Test)...
  5. In the Open URL List dialog, open the text file containing the URLs that you created in Step 1
  6. Your test runs

You can find Xenu Link Sleuth at:

October 1, 2009

Book: Even Faster Web Sites: Performance Best Practices for Web Developers

After really liking Steve Souders' High Performance Web Sites: Essential Knowledge for Front-End Engineers, I really wanted to like his latest offering Even Faster Web Sites: Performance Best Practices for Web Developers.  I liked it, but not nearly as much.

While High Performance Web Sites is centered around Souders' "14 Rules" for better web performance, Even Faster Web Sites isn't rule-oriented.  Instead, it is organized into the three areas of JavaScript performance, network performance, and browser performance.  And six of the fourteen chapters were written by contributing authors, rather than by Sounders himself.

All this adds up to a somewhat uneven, and less widely applicable, set of ideas.

Here's the list of chapters:
  • Understanding Ajax Performance - written by Douglas Crockford
  • Creating Responsive Web Applications - written by Ben Galbraith and Dion Almaer
  • Splitting the Initial Payload
  • Loading Scripts Without Blocking
  • Coupling Asynchronous Scripts
  • Positioning Inline Scripts
  • Writing Efficient JavaScript - written by Nicholas C. Zakas
  • Scaling with Comet - written by Dylan Schiemann
  • Going Beyond Gzipping - written by Tony Gentilcore
  • Optimizing Images - written by Stoyan Stefanov and Nicole Sullivan
  • Sharding Dominant Domains
  • Flushing the Document Early
  • Using Iframes Sparingly
  • Simplifying CSS Selectors
A good book, but still a minor disappointment.

September 25, 2009

Some Testing and QA Blogs You Should Read

Blogs are a great way to quickly read what others in the Testing and QA field have to say about our profession.

Here are some good ones.  I read them often.  Perhaps you should check them out:

Matt Heusser's Blog - Testing at the Edge of Chaos
Matt is a software craftsman with an interest in testing, project management, development, how people learn and systems improvement
Matt Heusser

I.M. Testy
Treatises on the practice of software testing
Bj Rollison

Tooth of the Weasel
Notes and rants from Alan Page.
Alan Page

Test Obsessed
Elisabeth Hendrickson’s Thoughts on Testing, Agile, and Agile Testing
Elisabeth Hendrickson

Google Testing Blog
If it ain't broke, you're not trying hard enough

DevelopSense Blog
Observations on software testing and quality, by Michael Bolton
Michael Bolton

James Bach’s Blog
The Consulting Software Tester
James Bach

Practical QA
Common Sense Quality for the Rest of Us
Linda Wilkinson

I specialize in tactical QA - preventing, finding and fixing software problems no matter what the circumstances. No excuses, no problem.
Catherine Powell

Steve Rowe's Blog
Ruminations on Computing - Programming, Test Development, Management and More
Steve Rowe

Exploration Through Example
Example-driven development, Agile software development, testing, Ruby, and other things of interest to Brian Marick
Brian Marick

Collaborative Software Testing
Jonathan Kohl's blog on software investigation
Jonathan Kohl

Test This Blog - Eric Jacobson's Software Testing Blog
Refinements on the art of software testing
Eric Jacobson

Expected Results
Testing, managing, consulting, quality and the art of motorcyle maintenance
Phil Kirkham

PractiTest QA Blog
Testing Tools & Methodologies for the Practical QA Tester
Joel Montvelisky

Adam Goucher
Quality through Innovation
Adam Goucher

Steve Souders - High Performance Web Sites
Steve works at Google on web performance and open source initiatives...
Steve Souders

QA Hates You
You suspected it.  Now you know it.
The Director

(last updated December 30, 2009)

September 22, 2009

Testing Virus Recognition - The EICAR Anti-Virus Test File

We are currently building a product that allows users to upload a file that we will then place on a publicly-available website.  One of the requirements is that we check the uploaded file for viruses first.  This presented an interesting testing opportunity.

Our enterprise anti-virus software can scan the uploaded files, and delete them if a virus is detected before they are moved to the externally-accessible location.  But how to test this?  We couldn't use a real virus - that's far too dangerous.

Fortunately, there's a nice solution.  Eicar, the European Institute for Computer Antivirus Research, in conjunction with most major anti-virus vendors, has created a file that is not a virus itself, but will cause most anti-virus software to react as if it were a virus.

If you copy the following 68-character string into notepad, and save it to a text file, your anti-virus software will treat that file as if it contained a virus: copyrightjoestrazzere
(Or download one of the files from

For my tests, I simply had to take this file to a machine that had its own anti-virus software temporarily turned off, then submit it to the new product.
  • Test completed. 
  • The product reacted as expected.
  • The appropriate message was written to the event log.
  • The appropriate warning message was displayed to the user.
  • The "pseudo-infected" file was deleted and not made public.
  • Test Passed!
It's fun to learn a new technique.

See also:

August 23, 2009

The Sea Was Angry That Day My Friends

The sea was angry that day my friends. Like an old man trying to send back soup at a deli.
- George Costanza, Seinfeld

 Jenness State Beach - Rye, NH

We drove to Rye, NH today, to Jenness Beach.

Hurricane Bill had passed by the coast and left behind some much larger than normal waves - a treat for surfers like my son. Unfortunately, shortly after we arrived, the incoming tide and the undertow made it too treacherous, and the life guards closed the beaches.

We stayed and watched the rising tide, and the debris washing in. My son and his girlfriend spent the time grabbing buoys that were snapped loose from their lobster traps (they snagged two nice ones).

And we saw some fools decide to ignore the lifeguards and go out with their surfboards anyway. New Hampshire State Police arrived to deal with them as they exited the water.

I understand wanting to see the big waves. The ocean certainly looked strong and beautiful. And it was exciting to see it crashing up and over the sea barriers.

But I don't understand breaking the law, and risking your life, by going out into waves declared unsafe. Foolish, and setting a bad example for the children watching.

August 16, 2009

Villanova - Saving The Country Billions... One Certification At a Time

Have you been receiving a series of emails from Villanova?  I have.  I've been getting emails fairly regularly for the past year or so.  And the Subject is getting increasingly (how do I put this?) dramatic.

This month's edition: "Software Testers Can Save Economy Billions".

Now, I'm not one to shirk my duty as a citizen, so if there's a way for me and a few of my tester friends to save this economy, I'm all for it.  Somehow, I'm guesssing that completing three courses (even courses "presented by Rex Black"), won't make much of a dent in an our economy, much less a Billion Dollar Dent.

Here are some of the claims in this month's email:
Recession-Proof Yourself With New Credentials Online!  I know some certified folks.  And I know some who are not certified.  I cannot say I see a difference in their recession-proofedness.  Perhaps the laid off friends who own credentials can tell me otherwise?

Software testers can earn up to $135,000.  Oddly, the source they appear to cite has a salary survey that is over five years old, and appears to reference exactly one outlier who made that much.  It does make you wonder what that one person is doing these days?

Software bugs, or errors, are so prevalent and so detrimental that they cost the U.S. economy an estimated $59.5 billion annually, or about 0.6 percent of the gross domestic product.  If this isn't an indication of a certification-starved country, I don't know what is!

An estimated $22.2 billion worth of error costs could be eliminated by an improved testing infrastructure that enables earlier and more effective identification and removal of software defects. Hmm, does this certification improve my testing infrastructure?  I thought people did that?

Currently, over half of all errors are not found until "downstream" in the development process or during post-sale software use.  Over half?  I've never worked any place where that was even remotely true.  Maybe I've just been lucky all these years, one of the fortunate uncertified few.

By becoming an ISTQB-certified tester, you will distinguish yourself as someone who knows how to apply sound software testing techniques and principles to reduce defects, improve quality and enhance business success.  Must be all those "indistinguishable" testers dragging the economy down, huh?

Certification acknowledges that you have mastered the newly acquired subject matter and demonstrates your commitment to professional excellence.  Well, it acknowledges that I have memorized enough to pass the certification exam.  Oh, and paid the fee, too.  (I've been fairly committed to my profession for a while now, thank you.)

By earning your Master Certificate in Software Testing from worldwide leader Villanova University and obtaining the stature of an industry-recognized certification like ISTQB’s Certified Tester, you will come away with a feeling of prestige, respect and accomplishment, along with the potential to instantly increase your earnings.   Stature is good.  So are feelings of prestige, respect and accomplishment.  I wonder - are these feelings the same thing as actually having prestige, respect and accomplishment?  Perhaps not.  But everyone always has the potential to increase their earnings, right?

All these things sound quite wonderful.  But I'm still guessing that the economy will go whichever way it goes without being much affected by marching hordes of newly certified testers.

Thanks, 'Nova - but I'll pass.

August 9, 2009

Patriots Training Camp 2009

I took a trip down to Foxborough to see this year's New England Patriots Training Camp.

They held both a morning and afternoon session. It was a warm, humid day weather-wise, and a lot of fun.

A few observations:
  • Tom Brady looked fine coming off of his knee injury - no limp, no problems throwing. Other than the black brace on his left knee, he looks just like the Tom Brady of the 2007 season.
  • Fred Taylor looks terrific. He was faster than I expected, and was making some terrific cuts in one-on-one drills versus the linebackers. He' going to be a great addition to an already good running game.
  • Jerod Mayo is a beast. Perhaps because of the year of experience, or just some fantastic offseason conditioning, but he looks even faster and stronger than last year.
  • Adalius Thomas is amazing. In drills versus the running backs he was able to either run around them or just run through them. He couldn't be stopped.
  • Randy Moss was just being Randy Moss. When he catches a football, it seems to touch only his fingertips and hands - never his body. He is so smooth to watch.
  • The offense looks really solid. With the exception of the tight ends, virtually every position is covered with both high-end talent, and depth. They could be quite special again this year.
  • The defense, and particularly the defensive backfield, will look very different this year with all the acquisitions and draftees. If these two practices were any indication, it will be much better.
Football season starts soon. It should be fun. Go Pats!

July 31, 2009

The Wow Factor - Puffery for Your QA Department

I was reading a free, ad-supported trade magazine for the QA and Tester market, and was struck by the sheer volume of puffery in nearly every ad.

According to Wikipedia, "puffery as a legal term refers to promotional statements and claims that express subjective rather than objective views, such that no reasonable person would take literally. Puffery is especially featured in testimonials."

I understand that this is what ad-writers and marketers do.  I understand that you aren't supposed to take it literally.  But this seemed even more over-the-top than usual.

From an Application Lifecycle Management provider:
  • 50% faster to market
  • 25% more productive
  • You can avoid cutting heads
  • Ensure your Agile success
From a test tool vendor:
  • Cut testing time in half
  • Test faster while protecting quality
From a testing services vendor:
  • Compete with confidence
  • Optimizing complex test enviroments
  • Accelerated results unmatched in the industry
From an automation design tool vendor:
  • A New Era of Automated Web Testing
  • Unprecedented productivity
  • Boost productivity and communication!
From a training vendor:
  • Accelerate Your Career
  • World-Class Expert Instructors
  • Perfect solution
  • One of the best investments you can make
This all sounds great.  Who wouldn't want to ensure success, protect quality, have optimized, unmatched results, and find a perfect solution in a new era?  Very exciting.  Very - Wow!  Should I go right out and purchase all of them?

So what if we started talking about our QA efforts within our own companies using similar language?  From now on, let's add a bunch of Wow Factor in how we speak:
  • A New Era in Quality Software Releases!
  • 99% fewer bugs in the field!
  • Ensure corporate success
  • Perfect bug reports
  • Unprecedented quality
Will we get a higher salary?  Would we get more time to test?  Might we get more resources?
Maybe not.

July 29, 2009

Optimists, Pessimists, and a Terrific Comic Strip

If you've read my article comparing Optimistic Developers to Pessimistic Testers (, then you know that the topic interests me.

And you can imagine how pleased I was to stumble across this particular comic strip. *
Urban Jungle by David Wilborn

The main protagonist of Urban Jungle is a developer named Zack.  According to the website:
Zack is the only human being in an office full of animals. He does software development for a large pharmaceutical company. Single, thirty-something, and hanging on to the last shreds of his optimism. He was raised by wolves.
Good stuff!  You should definitely check it out.

Note that I haven't yet figured out what the QA Tester character is supposed to be:
  • an opossum?
  • a rat?
  • a weasel?
I guess any of them could be appropriate!

I've asked the artist.  I'll post his reply once I get it.

Update:  According to David (the artist) her name is Sharon, and "she's a possum".  Apparently, Sharon is a somewhat pessimistic possum.  I like it!

* My friend "The Director" at points out that I "stumbled across" this excellent comic strip through his site. I think he's right, but in my defense I look at lots of blogs/sites in the early morning hours.  Sometimes I'm lucky if I can remember if I drank my coffee or not.  Don't hate on a fellow QAer, now!

July 26, 2009

The Blue Screen of Jeff

My QA Team gets together every two weeks to informally talk about QA and testing. We often take turns presenting something we think will be of interest to the entire team.

One of the Senior QAers has recently been presenting a series of 10-15 minute slideshow talks full of useful hints and tips.

He wanted to come up with a good name for the series, and eventually settled on:

The Blue Screen of Jeff

How cool is that?

July 13, 2009

Bad Vacation Weather - Good Vacation News

We took our annual vacation to York Beach, Maine.

(York Beach, Maine - The best day of the week!)

The bad news was the weather.  This was by far the worst weather vacation I can recall.  It rained every day.

Our last day of vacation was the best.  We had a nice morning on the beach.  But mid-afternoon, the clouds rolled in again, and severe thunderstorms in the area forced the lifeguards to empty the beach.

While we always have a few activities ready for the inevitable rainy day, my patience with shopping, eating, and walking in the rain was wearning thin.

For the entire eight-day vacation, we had a total of six hours of sun!  Oh well.

This year, instead of the usual three or so books, I decided to bring just one book for summer reading: 'Red and Me - My Coach, My Lifelong Friend' by Bill Russell.  Not bad, but a bit short and light, even for summer reading.  In addition I had loaded my iPod with some podcasts, to see how that would work out.  Not bad!  While I tried a bunch, the three that I particularly liked ended up being:
  • Patriots Football Weekly in Progress
  • NPR Science Friday
  • Scientific American
As bad as the weather was, we did receive two bits of great family news.

My older son, who had been searching for months, finally got approval on a mortgage and was able to close and move into his new condominium.  With the economy the way it is, the condo prices were good, but despite having been pre-approved, the approval process was very painful.  We spent several days during vacation going to banks, faxing statements to loan officers, etc.  Fortunately, it all worked out, and he's happily moved into a lovely condo, in a terrific area.

My younger son had graduated college and was looking for a job.  He had been getting just a few interviews, and no offers.  During vacation, he got two offers, and accepted one!  He'll be a (gasp!) Software Engineer, developing systems marketed to travel agencies.  It's with a small division of a large global company.  It seems like a great job, right up his alley, with lots of potential.  I'm really happy for him, and very proud.

So while not the most relaxing vacation, and certainly not the best weather, it was a really good week family-wise.

June 24, 2009

WinTask - CleanString$

' CleanString$ - replace all occurences of < and > in a string,
'                with < Lower> and < Greater> respectively
'                so it can be safely used with SendKeys()
' Author - Joe Strazzere


Function CleanString$(StringIn$)
    StringIn$=Replace$(StringIn$,">","< Greater>")
    StringIn$=Replace$(StringIn$,"{{lt}}","< Lower>")
    CleanString$ = StringIn$


' For example
KeyString$="a < b and b > c"


UseWindow("NOTEPAD.EXE|Edit|Untitled - Notepad|1",1)

June 13, 2009

Perhaps They Should Have Tested More - Starbucks

For a couple of days in May, coffee giant Starbucks ended up double charging about 1 million of their U.S. and Canadian customers.
  • More than 1 million transactions at 7,000 Starbucks locations on May 22 and May 23 were double billed
  • The one-million Starbuck transactions involved both credit and debit cards
  • The POS charges—and the receipts given to customers—were perfectly in order. The problems kicked in hours later, in the settlement processing area
  • Starbucks has declined to say how the glitch happened
  • Starbucks has declined to say how they plan to prevent it from happening again
  • Starbucks says they've had some customers call whose accounts were not fixed
  • Any customers with questions should contact the company's customer relations hotline at 1-800-23-LATTE.
Starbucks spokeswoman Trina Smith didn't seem to think the double charges were a big deal, just a minor confusion:
"We sincerely apologize to customers who were confused,". But, she adds, in a world of digital transactions, "Consumers are accustomed to these kinds of things."
Others disagree, and think the Starbucks image may have been tarnished by the error:
Starbucks patrons pay more not just for a better product, but for a perception that it treats consumers better, says Bradford Hudson, assistant marketing professor at Boston University. "You expect a company like Starbucks to do everything flawlessly, or to do something extra special to recover." He says it could have given a free drink to all affected.
Perhaps they should have tested more.

Just wondering - On the Starbucks Annual Report, will this screwup be categorized as a "Short" bug?  A "Tall" or "Grande" bug?  Or is it actually a "Venti" bug?

See also:

June 10, 2009

Project Completion Lunch and Awards

After many months of hard work, we recently completed a big project.

We released a new version of our system with support for 529 Plans (tax-advantaged college savings plans).

And we were able to time it so that our 529 release occurred on 5/29! (Ok, so while we started releasing the 529 code on 5/29, it took so long that we actually finished on 5/29 at around 54 o'clock - it was still quite a nice accomplishment.)

Recently, we have started celebrating some major releases by taking the project team out to lunch and handing out awards. They are meant to be humorous, and are usually accompanied by a bit of ceremony and a little story about why that particular award was chosen.

For this release, the awards were selected by the VP of Development. She did a great job.

There were some very funny awards. The graphics designer got a mini Etch-A-Sketch. The lead developer got a toy Iron Man. I was the sole QAer on the project. When it came my turn, I was presented with an "Extreme Bug Vacuum". Very nice!

I like this new tradition.

May 28, 2009

The Barbara Test

(not actually Barbara)

About 10 years ago, I was lucky enough to be Employee #9 at a startup software company. 

It was a terrific experience.  I got to do a bunch of interesting things.  I learned a lot.

Like many startups, we did a lot of things wrong.  Although none of the original members work there any longer, the company is still around, so I suspect we did at least a few things right as well.

One of the few things we did right was something I like to call "The Barbara Test".

As we developed the first version of our first product, we had a very strong sense of the target user.  Our CEO was a domain expert in our product area.  She knew the problems faced by the target user.  In fact she knew the target user - her name was Barbara.

Early on in the development cycle, a bunch of us (most of the company, actually) flew out to visit Barbara on her home turf in the midwest.  We were able to hear and see what Barbara knew, what she did on a daily basis, how she did it, and the tools she used to do it.  We could see what was easy for Barbara, what was hard, and what was nearly impossible.  We asked Barbara what would make her job easier, what she could live with, and what she couldn't live without.

This was terrific insight!  It was a living, breathing set of User Scenarios.

Armed with this knowledge, we had a great way to judge our design, to assess our decisions, to triage our bugs, to set our priorities, to focus our efforts.  Whenever we had a question, we could always ask ourselves:
  • How would Barbara do it?
  • Is this easy enough for Barbara?
  • Can Barbara get her work done in time, using this feature?
  • Does this report have enough detail for Barbara?
  • Would Barbara prefer this, or that?
  • What kind of Help system does Barbara need?
  • Could Barbara live with this bug, or must we fix it now?
In the end Barbara was happy, and so were we (her company purchased our product).

As you test, it's nice to have an understanding of the target user for your product.  If you can't actually meet her, at least be able to imagine her.  Ask your Project Manager: "Tell me about our Barbara."

May 25, 2009

Think First, Then Test

(I have this statue on my desk)

You've just been given a project to test.  It's a big deal - important to the company, and important to you.  So you are ready to jump in and get going.  Should you just play with the first build?  Should you do some exploratory testing?  Should you check for that first bug you know is almost certainly waiting for you?  Should you start creating some automation?  Should you write a test plan first?

While people seem to naturally have a preference for action, I believe it's most important to think first, then test.

As Cem Kaner, James Bach and Bret Pettichord say in their excellent book Lessons Learned in Software Testing:
The difference between excellent testing and mediocre testing is how you think: your test design choices, your ability to interpret what you observe, and your ability to tell a compelling story about it.  The rest of testing is ordinary office work, for the most part.
They indicate four main categories of thinking that figure into the practice of testing:
Technical thinking.  The ability to model technology and understand causes and effects.  This includes things like knowledge of relevant technical facts and the ability to use tools and predict the behavior of systems.

Creative thinking.  The ability to generate ideas and see possibilities.  You will test only in ways that you can imagine testing.  You will look only for problems that you imagine can exist.

Critical thinking.  The ability to evaluate ideas and make inferences.  This includes the ability to detect and eliminate errors from our thinking, to relate product observations to quality criteria, and to build a compelling case for a particular belief or suggested course of action.

Practical thinking.  The ability to put ideas into practice.  This ability includes such skills as applying test tools and making test techniques and effort fit within the scope of the project.
Here are a few things to think about.

Before you test...
  • Who are the relevant stakeholders for this project?
  • What do they need from me most?
  • How high should the "quality bar" be for this project?
  • What are the risks?
  • What could go wrong?
  • What is the most important thing to test?
  • In what order should things be tested?
  • What tools do I need?
  • What skills do I need to learn before testing this?
  • Who could help?
  • How much time do I have?
While you are testing...
  • What have I learned during testing that I didn't know before?
  • What seems riskier?
  • What kind of overall quality am I seeing?
  • What should I tell the stakeholders about it?
  • Should I change my planned strategy?
  • Do I need additional tools?
  • Do I need more help?
  • Do I need more time?
  • What is most important for me to do right now?
After you test...
  • What worked well?
  • What didn't work as well?
  • How could I have done that more efficiently?
  • What didn't I catch, and why not?
  • What should I do differently next time?
  • What notes/documentation should I leave for next time?

May 22, 2009

Messages from Me

In my Spam filter, I've been seeing more and more messages from "me".

For example, right now, I see these:
From: me
Subject: What is your floor?

From: me
Subject: About your results

From: me
Subject: 2 u

From: me
Subject: Let's dine together tonight

From: me
Subject: Our group dismissed

From: me
Subject: Hilarious party, come

From: me
Subject: Your card found

From: me
Subject: Read through this

From: me
Subject: Order #462803

From: me
Subject: Drive me home

From: me
Subject: Re: Your Hyrdoc0done Order #009642
I have to admit I was intrigued by this one:
From: me
Subject: Work is cancelled today
After all, who would know better if work is really cancelled than me?  So I clicked it.

Thoughtfully, Gmail provides these warnings when I actually decide to peek at one of these messages from me, and even invites me to learn more:
Warning: This message may not be from whom it claims to be. Beware of following any links in it or of providing the sender with any personal information.  Learn more
This message was likely forged and did not originate from your account. Learn More
That's quite kind of Gmail.  Even so, I'm guessing that I would know if it actually originated from me.

Sadly, it didn't originate from me.  Work isn't actually cancelled today - but if you still want to skip work on a Friday before a long holiday weekend, you didn't hear it from me.

Now, I know me. 
I've known me for more than 50 years. 
In fact, I know me almost as well as I know myself.
And I like me, but I guess I just don't like so many messages from me. 

If I tell myself not to send so many silly emails, will that actually stop me?

May 17, 2009

WinTask versus WinTask Lite

P>If you are considering WinTask as a test automation tool, is it necessary to purchase the full WinTask, or can you get by with the lower-cost WinTask lite?

The simple answer - if you want to do any real test automation, then Yes - you should purchase WinTask.  WinTask Lite is missing several functions that are very, very handy for the tester.
The logging functions are useful for writing a results log
The response time functions are useful for checking the duration of tasks
Some of the system functions are necessary for creating extensions to WinTask (to call into system DLLs, for example)
The web functions are essential for testing web-based applications
In some cases, the OCR functions can get text that is otherwise not accessible
Here's what you will NOT get in WinTask Lite (based on version 3.5a):

Window management functions
  • CheckedW - Retrieves the state of a checkbox/radiobutton
  • ClickOnTextOCR - Clicks mouse button on a text recognized by the OCR engine inside a window
  • SelectedItem$ - Returns the selected item in a listbox/combobox
  • SendKeysEncrypted - Sends encrypted keyboard input to the last specified Window
  • UseWindowRegEx - Specifies, using Regular Expression, the window where the script will now send its keystrokes
  • WriteEditEncrypted - Writes an encrypted string into the specified edit field  
Capture functions
  • CaptureAreaOCR$ - Captures the text that appears in the specified area of a window using OCR
  • HardCopy - Saves a screenshot  
Synchronization functions
  • #ErrorFunction$ - Gives the name of the function which causes the error
  • #ErrorMsg$ - Gives the error message sent by the function which causes the error
  • #ErrorScript$ - Gives the name of the script where the error occurred.
  • Disable - Disables the management of a specified event
  • Enable - Reactivates the management of a specified event
  • OnAction ... EndAction - Manages events
  • OnAction Error ... EndAction - Manages error-handling
  • Sleep - Makes the current script sleep whereas the events are still managed  
User dialog
  • BeginDialog...EndDialog - Defines a dialog box with its controls
  • CallDialog - Displays a dialog box defined previously
  • InputboxSecret$ - Displays a simple dialog box where the user can type a hidden value
  • SelectDir - Returns the name of the directory selected by the user in the standard Browse for Folder dialog
  • SelectFile - Returns the name of the file selected by the user in the standard dialog box File Open
  • SelectMultipleFile - Returns the name of the files selected in the standard dialog box File Open 
File management functions
  • AppendXMLNode - Adds a node in the specified XML file
  • EnumXMLAttributes - Retrieves the attribute names and attribute values for the specified XML node
  • EnumXMLChildren - Enumerates the child node descriptors for the specified XML node
  • GetXMLAttribute - Retrieves the content of an attribute in the specified XML file
  • ReadIni$ - Reads a parameter in the specified INI file
  • SetXMLAttribute - Modifies or adds an attribute in the specified XML file
  • WriteIni - Writes to the specified INI file
Flow control functions
  • #ErrorCode - Gives the error code sent by the function which causes the error
  • #ExecTimeout - After this timeout, execution of the script is stopped
  • #LastErrorLine$ - Gives the script line number where the error occurred
  • #ScriptAfterTimeout$ - Specifies the script to run after timeout  
String Management functions
  • Encrypt - function encrypts the specified string
System functions
  • Allocate - Reserves a memory area for data used by external DLL
  • DeleteRegKey - Deletes the specified key in Registry
  • DeleteRegValue - Deletes a value in Registry
  • External - Calls an external DLL
  • External$ - Calls a Windows DLL
  • GetCpuLoad - Returns the CPU load percentage
  • GetMemUsage - Returns the memory used percentage
  • GetProcessCpuLoad - Returns the CPU percentage used by a process
  • GetProcessList - Gives the list of active process and their attributes
  • GetWindowsList - Gives the list of parent window names present on desktop
  • ImpersonateUser - Allows WinTask to acquire additional rights
  • KillApp - Terminates the specified application
  • KillAppChildren - Terminates the specified application and its child processes
  • KillProcess - Kills the specified process
  • PeekInteger - Reads one or several bytes in memory and returns an integer
  • PeekString$ - Reads a string in memory
  • PokeInteger - Writes in memory a value of type integer or Unsigned
  • PokeString - Writes in memory a string
  • ReadIni$ - Reads a parameter in the specified .INI file
  • ReadReg - Reads an integer or a string from Registry
  • RevertToSelf - Cancels an impersonation made previously by ImpersonateUser
  • SendEmail - Sends an email using the defined SMTP server
  • WriteIni - Writes in the specified .INI file
  • WriteReg - Creates or modifies a string or numeric value in Registry
Clipboard and log functions
  • Comment - Writes a comment in the log file
  • LogFile - Forces the script to log its actions in the specified logfile
  • StopLog - Stops recording in the logfile 
Com port management functions
  • CloseCom - Closes the specified com port
  • OpenCom - Opens the specified com port
  • ReadCom - Reads the data in com port buffer
  • WriteCom - Writes data in the com port buffer
Services management
  • IsServiceStarted - Tells if the specified service is started or not
  • StartService - Starts the specified service
  • StopService - Stops the specified service
Web functions
  • CopyLink - Copies the link associated with the specified HTML element into the specified string
  • ExistHTMLElement - Checks if the specified HTML element exists or not
  • ExtractLink - Returns all the links of the child elements of the specificed HTML element
  • GetFrameSource - Retrieves the source of the specified frame
  • GetPageSource$ - Returns the source of the current web page
  • SelectedHTMLItem$ - Returns the selected item in an HTML list/combo
  • WriteHTMLEncrypted - Writes encrypted text in a Web form 
Response time functions
  • ResetTimer - Resets the specified clock
  • StartTimer - Starts the specified clock
  • StopTimer - Stops the specified clock
  • Timer - Returns the value of the specified clock
ODBC functions
  • #DbDateFormat - Controls the date fields format
  • DbBof - Tells if recordset contains no records
  • DbClose - Closes the recordset
  • DbConnect - Establishes the data source connection through the odbc driver
  • DbDisconnect - Closes the data source connection
  • DbEof - Tells if pointer is at the end of the recordset
  • DbExecute - Executes an SQL command on the opened data source
  • DbGetFieldNumeric - Retrieves the value of a numeric field in a recordset
  • DbGetfieldString - Retrieves the value of a string field in a recordset
  • DbMove - Moves the current record pointer within the recordset at specified position
  • DbMoveFirst - Positions the current record on the first record in the recordset
  • DbMoveLast - Positions the current record on the last record in the recordset
  • DbMoveNext - Positions the current record on the next record in the recordset
  • DbMovePrev - Positions the current record on the previous record in the recordset
  • DbRecordCount - Returns the number or records in the recordset
  • DbSelect - Fills the recordset by retrieving in the table the records matching the sql query
OCR functions
  • CaptureOCR$ - captures what is seen in the specified window, OCRizes the content and returns the text as analysed by the OCR engine
  • CaptureAreaOCR$ - captures what is seen in the specified area of the specified window, OCRizes the content and returns the text as analysed by the OCR engine
  • UseOCREngine - specifies which OCR engine will be used by subsequent OCR function calls
Additionally, there are a few differences in internal limits between WinTask and WinTask Lite.

For WinTask:
  • Maximum number of procedures/functions: 500.
  • A script cannot contain more than 32 local variables.
  • A script cannot contain more than 1000 variables.
  • A script cannot contain more than 128 arrays.
  • Maximum size for an array is 65535 elements (so from 0 to 65534).
  • Maximum number of parameters that can be passed to a procedure/function is 16.
  • A script cannot contain more than 64 OnAction statements.
  • A maximum of 5 Include levels is allowed.

For WinTask Lite:
  • Maximum number of procedures/functions: 100.
  • A script cannot contain more than 32 local variables.
  • A script cannot contain more than 200 variables.
  • A script cannot contain more than 64 arrays.
  • Maximum size for an array is 65535 elements (so from 0 to 65534).
  • Maximum number of parameters that can be passed to a procedure/function is 16.
  • A maximum of 5 Include levels is allowed.