Leading global bank radically improves defect containment
Leading global bank radically improves defect containment
A global financial institution with over $1 trillion in assets under management and more than 16 million clients across 37 countries. This company has had a Hexawise license for its wealth management operations since 2015.
Problem
The bank’s stakeholders recognized there were significant issues within its testing organization - outdated, manual methods test case creation were leading to costly defects found in User Acceptance Testing on high-profile applications. The bank was spending significant portions of its budget addressing the issues without much improvement. Stakeholders began exploring test design solutions to optimize their suites and provide much more thorough critical coverage. In addition, they identified the business areas that needed it most - beginning with its wealth management and trading operations.
The testing organization began with a set of regression tests for a high-profile, Order Entry Processing application containing hundreds of repetitive test cases - leading to unknown, inadequate test coverage and a high percentage of defects found too late during testing. This was considered the riskiest application at the company due to its high level of complexity and high cost of defects in production.
Challenges
"People hearing about large Hexawise benefits wonder whether it would be applicable to their applications. Yes. You can apply Hexawise to any application you are testing."
Solution
The bank discovered Hexawise’s test design solution and was particularly interested in Hexawise’s unique capabilities that directly addressed their challenges of optimizing their current test cases by improving thoroughness, covering more in significantly fewer tests, and clearly communicating coverage achieved to stakeholders and regulators.
Once the bank procured Hexawise, business analysts and testers at the bank collaborated to generate new sets of tests using Hexawise within the trading space that quickly proved to be objectively superior to any tests created in the past using previous, manual, more guess-work-based scenario selection methods. Hexawise mind maps were used to clearly visualize and communicate the critical test inputs for the application, which led to a streamlined, thoughtful decision on how to proceed with the testing.
The new Hexawise-generated test sets were much more thorough than their previous sets of manually-selected tests. Hexawise systematically, and instantly, filled every critical gap in testing coverage that was previously seen in the bank’s manually-selected tests. Hexawise coverage charts also made the thoroughness improvements immediately obvious to stakeholders and testers, which allowed them to make precise decisions about the sufficient level of testing needed for the application moving forward.
Outcomes
On a separate project in the wealth management and trading space, the Order Entry application team set up their Hexawise plan in a way that offered them 3 optimized coverage options to choose from. As compared to their existing set of 85 manually-selected tests, optimized Hexawise-generated test sets provided a variety of instantly-adjustable options:
Using Hexawise coverage reports to quickly evaluate options, the team ultimately chose Option 3 - to execute (more tests than they were originally planning to with the manually-selected tests they would have executed without Hexawise) in order to achieve dramatically more critical testing coverage.
Automatic coverage analysis reporting capabilities in Hexawise made it clear to the team that their traditional approaches had resulted in undertesting this particular system. Their “business as usual” set of tests left far more coverage gaps than the team had realized. Becoming aware that they had been accidentally under-testing, the team made a conscious decision to dramatically increase their testing thoroughness even though this choice resulted in increasing the number of tests that would be executed.
However, the percentage of defects slipping through into UAT decreased by 95% due to Hexawise’s objectively more thorough intelligent augmentation-based test generation methods. Before adopting Hexawise, the bank’s manually-selected tests allowed 33% of defects found to slip into UAT (due to the significant coverage gaps from manual test creation methods used), when addressing defects becomes significantly more costly for the organization. With Hexawise, only 1.5% of defects found were slipping into UAT (due to filling in previously-seen coverage gaps with objectively better Hexawise tests), ultimately saving the organization significant costs in the development of this high-profile application.
“Hexawise allowed us to achieve the highest quality release we’ve ever had... My boss’s boss’s boss visited from out of town and congratulated our team!”
Within the bank's wealth management space, Hexawise is currently used for...
- Functional testing
- User acceptance testing
- Systems integration testing
- Non-functional testing
- Regression testing
- Agile sprints
- Test automation efforts (as well as manual testing efforts)
- Facilitating BDD-style stakeholder conversations
- Refining requirements, eliminating ambiguities
- Identifying previously-unknown edge cases
- And many other areas
The bank ultimately spread the use of Hexawise across the entire U.S. wealth management business area, transforming the ways in which they approach their testing efforts with Hexawise’s intelligently augmented test generation. Executives have even praised their Quality Engineers for contributing more to their test automation efforts - Hexawise developers partnered with the client to build a customized solution to seamlessly integrate Hexawise-generated scenarios with their automation framework of choice, iCAP. Stakeholders continue to realize transformational savings with the Hexawise approach for their widespread test design and automation efforts.
