Justin posted this to the Hexawise Twitter feed

cave-question

It sparked some interesting and sometimes humorous discussion.

cave-exploration

The parallels to software testing are easy to see. In order to learn useful information we need to understand what the purpose of the visit to the cave is (or what we are seeking to learn in software testing). Normally the most insight can be gained by adjusting what you seek to learn based on what you find. As George Box said

Always remember the process of discovery is iterative. The results of each stage of investigation generating new questions to answered during the next.

Some "software testing" amounts to software checking or confirming that a specific known result is as we expect. This is important for confirming that the software continues to work as expected as we make changes to the code (due to the complexity of software unintended consequences of changes could lead to problems if we didn't check). This can, as described in the tweet, take a form such as 0:00 Step 1- Turn on flashlight with pre-determined steps all the way to 4:59 Step N- Exit But checking is only a portion of what needs to be done. Exploratory testing relies on the brain, experience and insight of a software tester to learn about the state of the software; exploratory testing seeks to go beyond what pre-defined checking can illuminate. In order to explore you need to understand the aim of the mission (what is important to learn) and you need to be flexible as you learn to adjust based on your understanding of the mission and the details you learn as you take your journey through the cave or through the software. Exploratory software testing will begin with ideas of what areas you wish to gain an understanding of, but it will provide quite a bit of flexibility for the path that learning takes. The explorer will adjust based on what they learn and may well find themselves following paths they had not thought of prior to starting their exploration.

 

Related: Maximizing Software Tester Value by Letting Them Spend More Time Thinking - Rapid Software Testing Overview - Software Testers Are Test Pilots - What is Exploratory Testing? by James Bach

By: John Hunter on Apr 8, 2014

Categories: Exploratory Testing, Scripted Software Testing, Software Testing, Testing Checklists

Screen-Shot-2013-01-15-at-7.28.42-PM-250x300

 

Elisabeth Hendrickson's new book, Explore It!, will begin shipping from Amazon in a week. If you're interested in software testing, I highly recommend it without reservation. It's outstanding. It is currently available for sale on Prag Prog and for pre-order on Amazon. The paper version will be published on January 22nd. Since Amazon apparently doesn't allow people to review books until they officially go on sale, I can't yet post my review on Amazon, but here, one week early, is my glowing review:

Explore It! is one of the very best software testing books ever written. It is packed with great ideas and Elisabeth Hendrickson's writing style makes it very enjoyable to read. Elisabeth Hendrickson has a well-deserved global reputation in the software testing community as someone who has the enviable ability to clearly communicate highly-practical, well-thought-out ideas. Tens of thousands of software testers who have already read her "Test Heuristics Cheat Sheet" no doubt already appreciate her uncanny ability to clearly convey an impressive number of actionable ideas with a minimal use of ink and paper. A pdf download of the cheat sheet is available here. If you're impressed by how much useful stuff Hendrickson can pack into one double-sided sheet of paper, you should see what she can do with 160 pages.

Testers at all levels of experience will benefit from this book. Like the best TED talks, Explore It! contains advanced ideas, yet those ideas are presented in way that is both interesting and accessible to a broad audience. Beginning testers will benefit from learning about the fundamentals of Exploratory Testing (an important and incredibly useful approach to software testing that is increasingly getting the respect it deserves). Experienced testers will benefit from practical insights, frameworks for thinking about challenges that bedevil all of us, and Hendrickson's unmatched ability to clearly explain important aspects of testing (including her superb explanations of test design principles).

Chapter 4 "Find Interesting Variations" in itself is worth far more than the price of the book. It is my favorite chapter in any software testing book I have ever read. A large part of the reason I have so much appreciation for this chapter is that I have personally been teaching software testers how to create interesting variations in their testing efforts for the last six years and know from experience that it can be a challenging topic to explain. I was excited to see how thoroughly Hendrickson covered this important topic because relatively few software testing books address it. I was humbled by how effortlessly Hendrickson seemed to make this complex topic easy to understand.

Buy it. You won't regret it. I'm buying multiple copies to give to developers and testers at my company as well as multiple copies to give to our clients.

  • Justin Hunter

By: Justin Hunter on Jan 15, 2013

Categories: Book Review, Exploratory Testing, Software Testing, Testing Checklists, Testing Strategies, Training

A combinatorial explosion is when the configuration settings and user actions and data entered etc. makes it impossible to test everything. The number of tests required to individually test every single possibility is many thousands of times greater than could realistically be tested.

When faced with taking over an existing software applications without a good test suite (or any test plan) often is daunting. And the problems of creating an unfathomable number of tests face you due to combinatorial explosion. Hexawise is a software as a service that aids in dealing with this dilemma for software testers. Software test plans are created that provide far better coverage than is seen in practice with a tiny fraction of the test required for complete combinatorial coverage (that is testing every possible combination [pairwise or 3, 4, 5... way] individually).

The Google Maps test plan provides a good example of combinatorial explosion faced by the testers (in this case, those who tested Google Maps). Take a look at the Google Maps test plan by login to your Hexawise account (creating a demo account is free and simple). The Google Maps test plan is one of 9 samples currently provided in Hexawise.

For creating your own test plan, while you are exploring the software application and testing it out to find "where the weak points are," you will probably find it useful to vary things as much as possible, repeat your actions as little as possible. Those points are true whether you're doing relatively informal lightly documented exploratory testing or more heavily documented test scripts. It addition, since a large percentage of defects can be triggered by the interaction of just two test inputs, it would be nice, if you had time, to test every single possible combination involving two test inputs; that's the rationale behind allpairs, pairwise and orthogonal array-based test case prioritization methods.

To recreate a similar - very early draft - plan for yourself, I'd suggest going through the following steps to put together a relatively small number of highly informative end-to-end-ish tests:

Ask what can change as users go through the system? Think about configuration settings, user actions, data formats, data ranges, etc. even throw in more "creative" ideas like user personas. Let your creativity and common sense guide you. Enter those in as parameters.

Ask how those parameters can change? (for the parameter "Browser" enter IE7, IE8, FF, etc.) Put those in as values under each parameter (entering constraints as required)

Ask does that variation matter? When possible (when it doesn't matter as much) use equivalence classes and be biased towards fewer values - at least for your early draft tests.

Ask what special paths thorough the system do you want to be sure to include? (Most common happy path, paths to trigger certain business rules, etc.)

Click the Create Tests button in Hexawise and you'll instantly get a very nice draft starter set of highly varied tests. If they look like they're relatively interesting and don't miss hugely important things, start informally executing them and you'll be sure to learn some more things as you do about the system's weak points that would result in you going back to those draft tests and iterating them to make them stronger and cover more.

To get a bit more on using this approach see our case studies. Hexawise TV provides narrated videos online showing how to make your life easier as a software tester.

 

Related: Hexawise Tip: Using Value Expansions and Value Pairs to Handle Dependent Values - 3 Strategies to Maximize Effectiveness of Your Tests - Pairwise and Combinatorial Software Testing in Agile Projects

By: John Hunter and Justin Hunter on Jan 2, 2013

Categories: Combinatorial Software Testing, Exploratory Testing, Multi-variate Testing, Pairwise Software Testing, Software Testing, Testing Strategies

I can be too verbose with some of my posts. This will be quick.

I recommend that you read this. It is the Exploratory Testing Dynamics document, a tightly condensed list of useful testing heuristics authored by three of the most thoughtful and experienced software testers alive today: James Bach, Michael Bolton, and Jon Bach. Using it will help you improve your software testing capabilities. http://www.satisfice.com/blog/wp-content/uploads/2009/10/et-dynamics22.pdf

Told you it'd be brief. Go. Now. Read.

By: Justin Hunter on Jul 13, 2010

Categories: Exploratory Testing, Software Testing