2009 Top IT Software Testing and Quality Assurance Outsourcing Vendors, Black Book Survey 2009 Results
In 2009, the Black Book software testing and QA client industry user survey investigated over 800 contracts held by 2,400 of the top spending banking corporations and organizations globally.
In order to rank the organizations, 18 key performance indicators (KPIs) or criteria are employed, scored on each respective vendor by client type and ranked on a 0–10 scale per KPI.
Key findings
Key finding: in 2008, Russian software testing and QA outsourcers surpassed Indian and other offshore firms in client experience. In the last 12 months, the top honored firms included more Indian and Chinese suppliers
In 2008, six of the top 10 software testing and QA firms were located in Russia and Eastern Europe. In 2009, several Chinese firms skyrocketed to the top ranks, surpassing both Indian and Russian competitors.
Key finding: most important customer satisfaction KPIs
Innovation, trust, reliability and customization are the most important attributes influencing software testing and QA client’s satisfaction with their 2009 outsourcing providers.
Key finding: vendor dissatisfaction is uncommon in the software testing and QA services industry among top ranked suppliers
Strong dissatisfaction is uncommon in this niche information technology outsourcing (ITO) sector, occurring in only 8.1% of technology-related industry’s client types, 16.3% in software and high tech company clients directly, and 21.3% of non-tech industry clients globally. However, this is a significant decrease in satisfaction from 2008 and 2007 results.
US clients are among the most satisfied with offshore software testing and QA services delivery from India and increasingly China (satisfaction with Chinese suppliers up 87.5% since 2007).
Strong dissatisfaction with offshore outsourcing vendors was less than 8.8% of all surveyed clients with 2009 projects involving testing, QA and related outsourced software functionalities.
Key finding: comprehensive services vendor arrangements from a comprehensive/end-to-end software testing vendor produce the highest satisfaction rates
Single vendors offering comprehensive QA services to software testing clients ranked highest in the overall survey by clients.
In order to rank the organizations, 18 key performance indicators (KPIs) or criteria are employed, scored on each respective vendor by client type and ranked on a 0–10 scale per KPI.
Key findings
Key finding: in 2008, Russian software testing and QA outsourcers surpassed Indian and other offshore firms in client experience. In the last 12 months, the top honored firms included more Indian and Chinese suppliers
In 2008, six of the top 10 software testing and QA firms were located in Russia and Eastern Europe. In 2009, several Chinese firms skyrocketed to the top ranks, surpassing both Indian and Russian competitors.
Key finding: most important customer satisfaction KPIs
Innovation, trust, reliability and customization are the most important attributes influencing software testing and QA client’s satisfaction with their 2009 outsourcing providers.
Key finding: vendor dissatisfaction is uncommon in the software testing and QA services industry among top ranked suppliers
Strong dissatisfaction is uncommon in this niche information technology outsourcing (ITO) sector, occurring in only 8.1% of technology-related industry’s client types, 16.3% in software and high tech company clients directly, and 21.3% of non-tech industry clients globally. However, this is a significant decrease in satisfaction from 2008 and 2007 results.
US clients are among the most satisfied with offshore software testing and QA services delivery from India and increasingly China (satisfaction with Chinese suppliers up 87.5% since 2007).
Strong dissatisfaction with offshore outsourcing vendors was less than 8.8% of all surveyed clients with 2009 projects involving testing, QA and related outsourced software functionalities.
Key finding: comprehensive services vendor arrangements from a comprehensive/end-to-end software testing vendor produce the highest satisfaction rates
Single vendors offering comprehensive QA services to software testing clients ranked highest in the overall survey by clients.
INTRODUCTION
SUMMARY
Survey overview
Key findings
Software testing and QA outsourcing vendor – top honors overall 2009
BLACK BOOK METHODOLOGY
How the data are collected
Understanding the statistical confidence of black book data
Who participates in the Black Book ranking process
2009 SOFTWARE TESTING AND QA INDUSTRY OUTSOURCING VENDOR RANKINGS AND RESULTS
REGIONAL TRENDS IN THE OUTSOURCED TESTING INDUSTRY
India
Eastern Europe
China
Latin America
THE NEXT GENERATION
Software risk management and analysis
Full lifecycle testing
Black box testing
White box testing
Test automation
Security Testing
VERTICAL GROWTH PREDICTED TO INCREASE THROUGH 2011
TOP 20 SOFTWARE TESTING AND QA OUTSOURCING VENDORS
STOP LIGHT SCORING KEY
Operational excellence of OST and QA vendors
OVERALL KPI LEADERS
Summary of criteria outcomes
Top score per individual criteria
INDIVIDUAL KEY PERFORMANCE
INDIVIDUAL KEY PERFORMANCE
APPENDIX
Orbys consulting
Disclaimer
SUMMARY
Survey overview
Key findings
Software testing and QA outsourcing vendor – top honors overall 2009
BLACK BOOK METHODOLOGY
How the data are collected
Understanding the statistical confidence of black book data
Who participates in the Black Book ranking process
2009 SOFTWARE TESTING AND QA INDUSTRY OUTSOURCING VENDOR RANKINGS AND RESULTS
REGIONAL TRENDS IN THE OUTSOURCED TESTING INDUSTRY
India
Eastern Europe
China
Latin America
THE NEXT GENERATION
Software risk management and analysis
Full lifecycle testing
Black box testing
White box testing
Test automation
Security Testing
VERTICAL GROWTH PREDICTED TO INCREASE THROUGH 2011
TOP 20 SOFTWARE TESTING AND QA OUTSOURCING VENDORS
STOP LIGHT SCORING KEY
Operational excellence of OST and QA vendors
OVERALL KPI LEADERS
Summary of criteria outcomes
Top score per individual criteria
INDIVIDUAL KEY PERFORMANCE
INDIVIDUAL KEY PERFORMANCE
APPENDIX
Orbys consulting
Disclaimer
