75% savings delivered across 15 projects at top U.S. health insurance company

75% savings delivered across 15 projects at top U.S. health insurance company

This health insurance company is the second largest provider in the U.S., holding 11% of market share. It serves more than 45 million members in 14 states.

Problem

This company was experiencing too many defects found late in production and spending uncontrollable amounts of money to address them. In addition, the company used outdated approaches to identifying what it believed were the right software tests to provide sufficient coverage. On their own, teams did their best to apply a risk-based approach to test design that relied too heavily on application and SME knowledge. Tests were manually selected, which inherently led to critical gaps in test coverage, costly defects to address, and test suites that were too large to manually execute and maintain.

Challenges
Too many gaps in coverage across deliveries
Many costly defects found late in production
Impractical test suite sizes for most business domains
Solution

The health insurance provider enlisted the support of one of the largest testing services providers in the world. With the provider’s guidance and support, stakeholders identified 15 projects across the organization that were experiencing the most significant testing challenges. The project team planned three phases of implementing Hexawise across these projects to optimize their regression suites:

Once trainings and pilots were completed to develop expertise and demonstrate the value of Hexawise for a smaller scope of projects, the expertise and approaches were applied to the broader regression suites. From there, with Hexawise, teams generated fewer tests to maintain for ongoing regression efforts and tests that would provide complete coverage of every system critical interaction using Hexawise’s superior, optimized test generation methods. Hexawise systematically eliminated every single wasteful idea and test within the manually-written test suites, while also ensuring that every critical interaction within their requirements were now covered with the new Hexawise tests.

The team approached this broad optimization effort in “waves” - strategically choosing similar sets of tests within the suite until the entire effort was complete. Once optimized tests were generated within these waves, Hexawise Automate was used to restructure the bank’s approaches to test automation within its center of excellence. With Automate, the team took a data-driven approach to designing automated tests using the tool’s Gherkin, “Given-When-Then”, Behavior-Driven Development-style scripting methods. Since Hexawise takes a one-to-many approach to automated test generation, fewer scripts would inherently need to be maintained in the future when changes were necessary. In addition, the scripts were designed to provide reusability in the future - when new requirements would be released, teams could pull from existing assets to create new scripts. This would further accelerate the time and effort savings during the test planning and documentation phases.

Outcomes
Twice as many defects found across 15 projects
75% fewer tests for regression efforts
Clear coverage insights for stakeholders

The project team measured the effectiveness of Hexawise against their previous, traditional approaches to designing and documenting tests. These measurements provided results for 15 unique projects - all demonstrating the fact that Hexawise was twice as effective at identifying defects compared to manual approaches, while also doing so in significantly fewer tests:

In order to find twice as many defects across these 15 projects, the project teams could execute 75% fewer tests with Hexawise. The teams realized the significant value that Hexawise provided in generating as scientifically few test scenarios as possible, while also providing objectively more superior test sets than any human could derive on his or her own using traditional methods. In addition, teams were now able to articulate the precise coverage that was achieved in these optimized test sets by using Hexawise’s clear and concise coverage reporting capabilities. Stakeholders immediately realized the cost savings (in test planning, execution, and maintenance) that would come out of applying Hexawise to significantly more projects across the organization.

The healthcare industry continues to be a unique and prosperous space for Hexawise implementations given the complexity of applications and business processes used for claims, care, pharmacies, and benefits at these firms. The more complex the systems are, the more value Hexawise will provide in designing powerful, fewer tests to run and maintain over time.

Other case studies you may like
Top P&C Insurance Company

Top P&C insurance company on pace to achieve $25 million in savings for multi-year testing modernization project
Leading U.K. Bank
Chandan Nakar
Head of QA
Hexawise reduces regression suite by 85% at a leading U.K. bank
Global Bank

Global bank achieves enormous savings in large regression testing suite optimization effort