Take the 2-minute tour ×  I recently had a test assignment where
I was testing a pretty well solidified API. The team started
phase 1 by creating basic functional, boundary and error tests
for each API call. It turned out to be three or four tests per
API call. The error checking was really basic. Things like row
counts for inserts and so on. We found a majority of the bugs in
this phase during test case creation.  Once that cycle was done
and all the bugs found were fixed, phase 2 used a data driven
approach. We exercised tens of thousands of variations using a
fairly robust set of test data with "poison pills" in the data
wherever we could. To my surprise, the number and severity of
bugs found in phase 2 were quite low compared to the number of
person hours invested. I had expected to find some outlying
killer bugs or at least a fair number of medium priority bugs.
Based on the number and severity of bugs found, the team's time
could have been better spent with ad-hoc testing or moving on to
other product