A large global IT and Consulting firm uses Hexawise to improve the thoroughness and efficiency of their software testing activities. They have been happy with the results, especially the large number of users who are using Hexawise, the responsiveness we have demonstrated whenever questions arise, and our willingness to go to great lengths to help make them successful. A year into their licensing arrangement, the firm has more than 6,000 Hexawise users in more than 35 countries, and hundreds of clients who are impressed with the quality of Hexawise-enabled testing the firm is providing to them. These are the highlights of the first year of their experiences.
The firm evaluated Hexawise to assess how Hexawise could help their testers and to confirm that Hexawise would deliver significant benefits on the wide variety of testing projects they regularly encounter. During their evaluation:
We supported and guided a Proof of Concept pilot project to demonstrate the benefits Hexawise routinely delivers, We helped gather benefits data from the pilot project by comparing the efficiency and thoroughness of business as usual tests to test cases generated with Hexawise, and We helped create a customized, fact-based business case that quantified the financial benefit of rolling out a broader adoption of Hexawise (using pilot project results together with estimates of loaded cost rates, costs of defects, etc.). Hexawise received very high marks from the evaluators, including especially strong marks related to: Ease-of-use of the tool, Low enterprise-wide license costs (with no incremental costs per new user), Powerful features (with our mixed-strength solutions, and elegant constraint-handling being particular favorites of the evaluators), and Our willingness to quickly add a couple features they requested to our tool.
We agreed an enterprise-wide license cost with the firm that was far less than the firm’s own estimated of the benefits they would likely receive from using Hexawise.
The firm secured a global enterprise-wide license of Hexawise to make it possible for all of their software test designers, software developers, Business Analysts, Project Managers, and other stakeholders to access Hexawise and collaborate (without any added incremental costs per user).
This enterprise-wide licensing arrangement was more attractive to the firm than a per seat license because they knew from experience that decentralized testing teams would be discouraged from using a software testing tool if they had to pay to do so (regardless of whether the value far exceeded the cost). A better solution for them was to pay for all the licensing costs centrally with one fixed cost, then promote broader adoption. If adoption spread to a hundred users or so, the firm-wide license would more than pay for itself; otherwise, their investment could lead to disappointing results.
As an adjunct to the enterprise-wide license, we agreed to provide several hundred hours worth of instructor-led training, consulting and rollout support.
The following chart shows the cumulative number of registered Hexawise users, by month. Eleven months into the license, we are on track to have almost 1,000 users before they completion of their first year in using Hexawise. The firm has been pleased with both the pace that testing teams have begun to use Hexawise and with the enthusiasm that testing groups have embraced it.
Adoption Challenges that we helped the Firm Overcome
There were several challenges that had to be overcome for broad adoption to succeed, including these:
Decentralization – First, the firm’s testing organization is largely decentralized, with testers in more than 30 countries, so a simple top down mandate to “use this new tool” would not have been nearly enough to drive adoption on its own. Local Autonomy – Second, testing teams are largely autonomous (and had opted out of using tools recommended by the central “tools team” in the past); unless testing teams had quickly seen how Hexawise would deliver immediate value on their projects, adoption would probably have been slower. Client-imposed Restrictions – Third, testing teams work closely with the firm’s clients (who often dictate usage of their own preferred testing tools for functions like test management and test automation); unless Hexawise integrated well with other tools and provided additional value not found in other testing tools, adoption would have been slower than it has been. Actions we took to help the firm generate awareness and broader adoption:
We customized training materials and instructor-led training sessions using industry-specific training examples (which we have found tends to be far more effective than general, abstract training examples).
We provided clear training for beginning, intermediate, and advanced users in a variety of formats including computer-based training, instructor-led web-based training, and instructor-led face-to-face training.
Instructor-led training was conducted face-to-face in the United States, India, United Kingdom, and Germany for the following industries:
United States and Canada: Insurance, banking, healthcare payments, telecommunications, government agencies, SAP testing, retail products, software firms India: Insurance, banking, healthcare payments, telecommunications, retail products, software firms, the firm’s proprietary applications and centrally maintained applications United Kingdom: Banking, retail products Germany: Insurance We conducted instructor-led web-based training for the firm for testers in numerous countries. Training examples included all of the above industries plus additional industries and topics.
Awareness and Rollout
We helped the firm create a communication program to increase awareness and promote adoption.
We contribute an article each month on test design related topics to their Testing Community of Practice newsletter.
Listening to Users and Making Hexawise Easier to Use
Some users initially found it confusing to generate tests without relevant training examples in Hexawise. We listened and made our tool better. We pre-populated Hexawise with customized, re-usable, template test plans specific to the firm’s needs.
We proactively shared explanatory example plans with new users based on what industry they were working in. Users now use these template plans both to understand our tool as the basis to generate tests of their own.
Attentive Customer Support with “Telephone Monogamy”
We practice “telephone monogamy” which means that, unlike many firms who make it difficult, if not impossible, to talk to an actual human:
We actively welcome the opportunity to talk to you and to answer your questions; we want to help you succeed We give everyone in your firm full contact details (including cell phone number) of a single Senior Test Design Consultant who strives to understand your firm’s testing needs inside and out Full Support and Backing by the Firm’s Leadership
The firm made Hexawise an important part of their standard tools and methods
The firm’s testing leadership recommends that all testing teams at the firm use Hexawise to design test cases on every project
The firm has used Hexawise on hundreds of testing projects from the following industries:
Banking Insurance Capital Markets Government Products Telecommunications Technology Resources The Firm’s Own Internal Testing Projects On projects where Hexawise has been used, the firm has consistently seen the following measurable improvements:
Less time to design test cases More defects found per tester hour More defects found overall The firm has used Hexawise as a competitive differentiator to successfully win testing outsourcing business because it makes them more efficient and effective testers than their competitors.
The firm decided to renew their annual license long before it was up for renewal; the benefits received from using Hexawise have, by all accounts, exceeded the licensing costs.