Kathleen Poulsen of Fidelity Investments gave a presentation at STAREAST 2017 sharing her experience using Hexawise to improve their software testing performance. Watch a 10 minute video with highlights of that talk:

We didn't really have what I'd call a scientific methodology to approaching the tests...

Our regression test suites were continuously expanding... We found there was a repition of tests.

We had 3 different projects that I will talk about that I feel like combinatorial or pairwise testing was the key to answering all of those problems.

Hexawise allows you to harness the power of combinatorial software testing with test plans designed to provide thourogh testing of interaction impacts on the software being testing. Hexawise provides more coverage with fewer tests.

All the teams that are using Hexawise can use that same file, they can talk to each other. [Another] thing I liked about Hexawise was the coverage chart... I go back to my business partners and say I am not running these tests. If they are important to you I add them back in with the click of a button. I love that... it was a game changer for me.

Using the Hexawise exporting options

the tests that we produced were converted into the given then when type scenarios automatically and when they are exported into excel you can use them to drive the Sellenium test automation framework. No additional work from us involved.

Using Hexawise's ability to create highly optimized test plans Fidelity was able to greatly reduce the number of tests while also greatly improving coverage.

We were able to reduce from 12,000 tests down to 600.

This type of result sounds amazing, and it is. But it is also what we find consisently from clients over and over. There are certain things people just cannot do well and designing test plans to cover incredible large numbers of interactions between test values and conditions is one of those things. Using highly optimized alogorithms to create test plans to cover these interactions in order to reliably create software customers will love is key. This also frees people to do what they do best.

Kathleen also discussed the significant improvement in communication within Fidelity that was brought about by using Hexawise.

The common language has become the test plan that comes out of Hexawise today.

Improving communiction is an area many organization see as important but finding concrete ways to achieve better communication is often difficult. We have designed Hexwise to aid the communication between stakeholders, including: software developers, software testers, product owners, help desk support staff and senior management.

The simplicity of this tool along with the way you can enter your parameters using the mind map tool, getting that coverage chart automatically out of it, having it export your data into a pretty commonly usable format - those are things that were teribly important to me. They gave me real value... I love that.

I can accomodate many differnt types of testing. We are testing at the class method level, at the services interface level, at the UI level...

Related: 84% of Software Defects Found in Production Could Have Been Found Using Pairwise Testing - Create a Risk-based Testing Plan With Extra Coverage on Higher Priority Areas - 2 Minute Introduction to Hexawise Software Testing Solution

By: John Hunter on Jun 9, 2017

Categories: Combinatorial Software Testing, Combinatorial Testing, Efficiency, Hexawise, Hexawise test case generating tool, Multi-variate Testing, Pairwise Software Testing, Pairwise Testing, Recommended Tool, Software Testing, Software Testing Presentations, Software Testing Efficiency, Testing Case Studies, User Experience

Hexawise has had another great year and we owe it to you, our users. Thank you! As a result of your input and word-of-mouth recommendations, in 2013, the Hexawise test design tool:

  • Added powerful new features,

  • Became even easier to use,

  • Introduced lots of new practical instructional content, and

  • Doubled usage again.

If you haven’t visited Hexawise lately, please login now to see all the improvements we've made (register for free).

Ease-of-Use Enhancements

Instructional Guides for Hexawise Features
We’ve added illustrated step-by-step instructions describing how to use Hexawise features.

Find them at help.hexawise.com. For our advanced features, like creating risk-based test plans, auto-generating complete scripts, and using Value Expansions, we’ve gone beyond “how?” to explain “why?” you would want to use these features.

Practical Test Design Tips
Want to see tips and tricks for creating unusually powerful tests? Want to learn about common mistakes (and how to avoid them)? Want to understand whether pairwise test design really works in practice? These topics and more can now be found at training.hexawise.com.

Frog-Powered Self-Paced Learning

frogs next

Want to become a Hexawise guru? Listen to the frogs. If you complete the achievements recommended by your friendly amphibian guides, you will level up from a Novice to Practitioner to Expert to Guru.

frogs become expert

You’ll complete two kinds of achievements on your way to guru-ness. To complete some achievements you only need to use certain Hexawise features. Completing the other achievements requires learning important test design concepts and demonstrating that you understand them. The frogs, ever-ready to guide you towards test design mastery, will greet you immediately upon logging into your account.

Powerful New Features

Recently-added features that will make you a more powerful and speedy test designer, include:

Coverage of Specific High-Priority Scenarios
You can now force specific scenarios to appear in the tests you generate using the Requirements feature.

Requirements Traceability
Requirements traceability is easier to manage than ever with the Requirements feature.

Generation of Detailed Test Scripts
The Auto-Scripting feature allows you to automatically transform sets of optimized test conditions into test scripts that contain detailed instructions in complete sentences.

Auto-Population of Expected Results in Test Scripts
If you want to, you can even automatically generate rules-based Expected Results to appear as part of your test steps by using the Expected Results feature.

To find out more about these features Hexawise added in 2013, please check out these cool slides: "Powerful New Hexawise Features".

 

Public Recognition and Rapid Growth

Kind Words
As a five-year old company working as hard as we can to make Hexawise the best damn tool it can be, hearing input from you keeps us motivated and headed in the right direction. Once a week or so, we hear users say nice things about our tool. Here are some of the nice things you guys said about Hexawise this past year:

 

“Working coaching session with customer today. Huge data/config matrix was making them weep. Stunned silence when I showed them @Hexawise :)”

-Jim Holmes (@aJimHolmes)

 

“That would be @Hexawise & combinatorial testing to the rescue once again. #Thanks”

-Vernon Richards (@TesterFromLeic)

 

Freaking awesome visualisation of test data coverage. Kind courtesy of @Hexawise at Moolya!”

-Moolya Testing (@moolyatesting)

 

“Using @Hexawise combinatorial scenarios for e-commerce basket conditions. Team suitably impressed by speed and breadth of analysis. #Win”

-Simon Knight (@sjpknight)

 

“Just discovered Hexawise today, brilliant tool for creating test cases based on coverage of many variables.”

-Stephen Blower (@badbud65)

 

“This changes everything.”

-Dan Caseley (@Fishbowler)

 

Using Hexawise is one of the highest ROI software development practices in the world.

-Results, paraphrased, of independent study by industry expert Capers Jones and colleagues.

 

Rapid Growth
Throughout 2013, Hexawise continued to be piloted and adopted at new companies week after week. Hexawise is currently being used to generate tests at:

More than 100 Fortune 500 firms More than 2,000 smaller firms

Hexawise office

New offices
Having moved into our new offices in October, the Hexawise team now gets together to do all our best stuff at this swanky new location in Chapel Hill, North Carolina.

What's Next?

Constant Improvements
We keep a public running tally of all of our enhancements. As you can see, we’re making improvements at a rate of more than once a week.

Want us to Add a New Feature?
If you have an additional feature, please let us know. We listen. There’s not much that makes us happier than finding ways to improve Hexawise.

Please Tell Your Friends
Our growth has been almost purely due to word-of-mouth recommendations from users like you. So if you find Hexawise to be helpful, please help spread the word. We appreciate it more than you know.

You can even let your friends in on a bit of a testing industry secret: while company-wide Hexawise licenses start at $50,000 per year, we allow the first five users to use fully-featured Hexawise accounts at no cost!

Thank you for all of your help and support this past year. And have a great 2014!

By: Justin Hunter on Jan 28, 2014

Categories: Hexawise test case generating tool, Recommended Tool

20100302-dnpabhucxq6ucrs5hqdnhwnbku

Design of Experiments in Software Testing - Pairwise and Combinatorial - Hexawise

Justin Hunter @Hexawise:

 

Removing inefficiency is good, sure, but it is not why Design of Experiments is so friggin' powerful. Saying DoE is interesting to know about because it can help identify and remove specific inefficiencies is a bit like saying Canada is a good country to visit because you can sometimes find a good cup of coffee there. To my mind, saying DoE is primarily about removing inefficiency misses the main point.

Design of Experiments is so powerful because it allows practitioners to predictably, systematically, and consistently find out more useful, actionable information in much less time than they would otherwise take to obtain this information (if they could find it at all with their less-structured approaches).

In manufacturing circles (e.g., when engineers produce new prototypes), DoE's ability to do this is no longer questioned. This is because leaders like George Box taught people in industry how to apply DoE and they gathered conclusive evidence that DoE allowed manufacturers to learn much faster through techniques like applying factorial designs. Box and other DoE experts (Taguchi, Montgomery, my dad, etc.) dealt with skeptical manufacturing engineers for four decades by showing them the facts and using DoE on the skeptics' own projects right under their noses. The evidence that DoE allows manufacturers to learn much faster (about a wide variety of learning goals) than the other methods they used prior to 1960 is incontrovertible.

In 2010, in the gradually maturing field of software testing, Design of Experiments-based methods of test case design has not caught on much at all yet. As an industry, it's adoption of DoE-based approaches is roughly where manufacturing was in 1960. Most software testers, even very good ones, don't know anything at all about how DoE can help them. Many other software testers have heard a bit about pairwise but mistakenly think pairwise and related, structured, DoE-based, test case selection method can't help them.

Even some of the best testers in the world who have written some of the most clearly-written and well-reasoned articles about pairwise approaches do not (in my view) seem to fully-understand: (a) how powerful the benefits are, (b) how often the approach can be applied / in how many diverse kinds of testing situations they can be utilized, and/or (c) how consistently the efficiency and effectiveness benefits are be generated when they are used properly. DoE methods, including pairwise and n-wise and mixed strength automatic test condition generation (made possible by tools like our Hexawise tool and also, to a great extent by James Bach's free AllPairs tool) allow software testers to learn much faster about critically important questions like: (1) where are the bugs?, (2) what is causing the bugs to appear?, (3) am I confident I have efficiently tested for a huge range of combinations of values in the System Under Test that might trigger defects? (4) am I succeeding in avoiding redundant repetition of steps in many test cases?, (5) how many bugs would be likely to find if we were to continue to run the next 100 tests?, etc.

In summary, the reason for the existence of Design of Experiments methods (whether we're talking about their applicability to testing software as efficiently and effectively as possible, or DoE methods' applicability to a huge variety of other objectives) - and, for that matter, the reason that they have been continuously refined and improved for 40+ years - is that DoE methods consistently and predictably allow users to learn actionable results as quickly as possible.

 

Related: Maximize Test Coverage Efficiency And Minimize the Number of Tests Needed - Pairwise and Combinatorial Software Testing in Agile Projects - Video Highlight Reel of Hexawise

By: Justin Hunter on Feb 11, 2013

Categories: Design of Experiments, Multi-variate Testing, Recommended Tool, Testing Strategies