Improved testing thoroughness leads to 0 production defects at top 5 asset management firm
Improved testing thoroughness leads to 0 production defects at top 5 asset management firm
The video testimonial link for this case study can be found here.
As one of the largest asset management companies in the world, this Hexawise client has over $4 trillion in assets under management. Since 2017, more than 600 resources within the personal investing space at this company have used Hexawise to design much more powerful tests for automation.
Problem
One of the firm’s key applications allowed users to enter their personal financial information and investment preferences before receiving an investment plan tailored to their needs. Production defects in this application opened the company up to significant monetary and legal risk.
This customer-facing, high-visibility app quickly experienced defects in production due to significant amounts of under-testing at the services layer, leading to potential lawsuits against the organization. In addition, the project team had minimal visibility into test coverage achieved which led to a smaller-than-necessary testing effort with minimal focus on testing the critical system interactions. Lawsuits from customers were imminent, so the firm needed to quickly find a new approach to increasing software testing thoroughness for its high-impact applications.
Challenges
Solution
Executives immediately reached out to Hexawise for help knowing that the widely-known test design solution would directly address these costly issues. Stakeholders bypassed most of the standard procurement processes in order to immediately get started with the tool and train its resources accordingly.
The firm recognized that Hexawise represented an improved mindset and worked with the Hexawise team to train key project resources on test case design with Hexawise. With the help of Hexawise test design experts, these resources used Hexawise to generate a new set of tests that completely replaced the existing application test cases.
The process began with training the project’s testing team, evaluating the system under test, and identifying the right set of variation ideas to drive optimal test creation with Hexawise. Using Hexawise’s mind map visualizations, testers were quickly able to receive feedback on their application input models from business partners and stakeholders.
With the help of developers and automation engineers, Hexawise was then integrated into the team’s services testing toolchain including SQLite and Parasoft SOAtest.
Using its coverage dial, Hexawise provided the team with several options for the level of thoroughness that they would like to achieve for these new test sest. Due to the high-risk nature of the application processes and their histories of dreadful defects, the team ultimately decided to select an abnormally-high level of combinatorial coverage for its Hexawise-generated tests.
Outcomes
“We have not seen a defect in production for this [previously problematic] mission critical system in more than four years since starting to use Hexawise.”
For these specific personal investment applications, the team generated 1,800 optimized Hexawise tests scenarios that achieved 100% coverage of every 4-way interaction in the system with the simple click of a button. For reference, this is a similar level of thoroughness to tests run by NASA in preparation for a space shuttle launch:
By doing so, since the creation and implementation of these 1,800 Hexawise tests in early 2017, there have been 0 production defects for this highly visible, customer-facing application (which has also led to 0 lawsuits against the asset management company). This set of 1,800 automated test scenarios still executes each night in just under 30 minutes, and they are easily-maintained due to Hexawise’s model-based approach to making scenario updates.
Within the investment management company’s personal investing space, Hexawise is currently used for...
- Functional testing
- User acceptance testing
- Systems integration testing
- Non-functional testing
- Services layer testing
- Regression testing
- Agile sprints
- Test automation efforts, and many other projects
Hexawise ensured the personal investing teams were taking a precise, risk-based approach to designing objectively better tests for their systems. In turn, executives were able to resolve their high-profile production defects and ensure they never slipped through again, avoiding millions of dollars in lawsuits from customers due to the increase in delivery quality. Hexawise continues to be used across all personal investing projects to this day.
