CA/Nimsoft QA Case Study

Contact Us
Contact Us
CA/Nimsoft: XBOSoft’s 3rd Party Validation Service ensures the highest product quality

XBOSoft’s tag line is “Software Quality Improvement”. Software quality is what we are all about. In over ten years of providing software QA & testing services, we’ve learned a thing or two about “Quality”. Foremost, you can’t improve your software quality without a structured QA process. Structure provides the framework for understanding, evaluating, and ultimately improving product quality. It’s the foundation for establishing, collecting, and tracking key test metrics that can be used to set performance baselines and provide input into test plans and strategies for reaching quality assurance goals.

CA Technologies logo.svg

We’ve recently spoken with John Smith, former VP of Product Development at CA/Nimsoft, to see what he had to say about his multi-year relationship with XBOSoft, and XBO’s value as a 3rd Party Validation Service.

CA Unified Infrastructure Management (CA UIM), formerly CA/Nimsoft, provides a single, analytics-driven solution for proactively and efficiently managing modern cloud and hybrid IT infrastructures. CA UIM is the only IT monitoring solution that provides intelligent analytics, comprehensive coverage, and an open, extensible architecture. The product is consistently the #1 rated infrastructure management solution as ranked by IT Central Station.

Here’s what John had to say about XBOSoft as a trusted QA & Testing Partner.

Q: What were the main two or three reasons you selected XBOSoft over the other candidates for the job?

John: Testing expertise, competitive pricing, and trust in the company developed during the sales process. The professionalism and personality of their team during the contract discussion and early stages of the negotiations were very positive.

Q: Do you recall when the company began doing QA for Nimsoft?

John: We started working with XBOSoft in 2008. We had a 2-tier QA process with the developers doing unit testing, followed by our internal QA team for functional, performance, and reliability testing among others. Because of the nature of our business and a critical role our product played in the operations of our customers, we strived for a “zero defect” goal. For that reason, we looked for an independent company to provide 3rd party customer validation of our product.

Just to jump ahead here; eventually, the XBOSoft QA team was reporting more than 60% of the functional defects. In several cases, they unearthed Level 3 defects that would have been very problematic for us.

Q: Can you cite any specific example of instances where XBOSoft identified a bug or defect in a product that either saved a considerable amount of money or solved a particular vexing issue with a product?

John: Xbosoft has been a key piece in CA’s release process with critical input to go-no-go decisions mainly based on their testing report. For many releases, they have worked closely with our team to prevent serious defects from escaping and being discovered by our customers. Some examples include:

  • Installer corrupting data
  • Significant performance degradation
  • Legacy functionalities missing in the current release
  • Security issues in access privileges for data
  • Usability issues that could confuse users into making mistakes

Q: Were there any drawbacks to working with an off-shore team?

John: No, it was never a problem for us. When we started with Agile development, I was a little concerned in the beginning, but the XBO team managed it like a champ.

We had an XBO test engineer on-site in Fort Collins who worked directly with our QA team, and he then managed the off-shore team, serving as the conduit between the two sites. This was a very important part of the methodology and process that enabled our success.

Q: If you were talking about XBOSoft to a company looking for a QA consultant, what would be the main points you would want to make about the value XBOSoft brought to your engagement?

John: The trust we developed over time, their flexibility to work with our evolving development methodology, and their overall professionalism and dependability. We truly considered them as part of the CA/Nimsoft development and QA Team.

Q: Did the partnership with XBOSoft evolve over time? For instance, did XBOSoft move from testing into a higher level of consulting services for CA?

John: Yes, we started with a small number of testers and grew over time. Starting with only 3 engineers in 2008, we began working with the Nimsoft team in Norway. I went over to Oslo regularly to visit our team and to ensure that we were satisfied with XBO. By 2011, we had grown the XBO team to 12 engineers as we continued to add services such as performance automation. The XBO engineers continued to master the application very deeply and served as the last round of regression testing before release to the end-users. We add four XBO engineers in 2013, and three more in 2014. So, in total, we had 19 XBO engineers working with CA. The CA dev team has always been satisfied and happy with their contribution and has expanded XBOSoft’s responsibilities to include customer validation testing and platform testing.

Platform or Tools usedWhat We DidResults
OS

  • AIX PowerPC
  • HPUX
  • CentOS
  • Debian
  • OpenSuse
  • RHEL
  • SUSE
  • Ubuntu
  • Solaris SPARC
  • Solaris x86
  • Windows Server
  • zLinux

Database

  • Oracle
  • Oracle RAC
  • MS SQL Server
  • MySQL

Source Code Management

  • Git/Github

Life Cycle Management Tool

  • CA Agile Central (Rally)
Browser

  • Chrome
  • Firebox
  • IE

Mobile Devices

  • iPhone iPad Android
  • Continuous Integration
  • Jenkins
  • Team City

Automation

  • Groovy
  • Java
  • Perl
  • Selenium
  • Serenity
Installation/upgrade testing

  • Validated installation/migration process for end-users

Functional Regression Testing

  • Converted features/user stories to test cases
  • Implemented test case/defect writing and management

Load/Performance Testing

  • Monitored software performance as a whole and addressed the bottlenecks in early stages

End to End User Experience Testing

  • Tested product from end-user perspective to help product owners to better design the functions
  • Set up and test customer-specific environments

Integration testing with Test Automation

  • Tested product as an integrated system, filling the gaps between different dev teams
  • Write/execute test automation; both Blackbox, UI and API level

Pre-Beta Testing and Customer issue resolution

  • Replicated and analyzed end-user issues
  • Conducted end-user defect analysis with triad summary
Increased client’s customer satisfaction by:

  • More than 60% of functional defects were filed by XBOSoft team
  • Discovered user experience issues to raise user satisfaction
  • Found security issues including data and password exposure to ensure client’s and client’s customer’s data and information remained uncompromised
  • Resolved customer escalations to increase customer satisfaction

Download This Case Study!

Download