Quality Engineering Services - NEAT EVALUATION FOR CAPGEMINI
←
→
Page content transcription
If your browser does not render page correctly, please read the page content below
NEAT EVALUATION FOR CAPGEMINI:
Quality Engineering Services
Market Segment: Overall
Introduction
This is a custom report for Capgemini presenting the findings of the NelsonHall NEAT vendor
evaluation for Quality Engineering Services in the Overall market segment. It contains the NEAT
graph of vendor performance, a summary vendor analysis of Capgemini for quality engineering
services, and the latest market analysis summary for quality engineering services.
This NelsonHall Vendor Evaluation & Assessment Tool (NEAT) analyzes the performance of
vendors offering quality engineering services (formerly referred to as software testing
services). The NEAT tool allows strategic sourcing managers to assess the capability of vendors
across a range of criteria and business situations and identify the best performing vendors
overall, and with a specific focus on AI testing, continuous testing, and UX testing.
Evaluating vendors on both their ‘ability to deliver immediate benefit’ and their ‘ability to meet
client future requirements’, vendors are identified in one of four categories: Leaders, High
Achievers, Innovators, and Major Players.
Vendors evaluated for this NEAT are: Amdocs, Capgemini, Cigniti, DXC Technology, EPAM
Systems, Expleo, Hexaware Technologies, Infostretch, Infosys, LTI, NTT DATA, Qualitest Group,
TCS, Tech Mahindra, TestingXperts, and Virtusa.
Further explanation of the NEAT methodology is included at the end of the report.
© NelsonHall 2020 1 March 2020
August 2020 1
Licensed for distribution © NelsonHall 2020
Licensed for distributionNEAT Evaluation for Capgemini: Quality Engineering Services NEAT Evaluation: Quality Engineering Services (Overall) NelsonHall has identified Capgemini as a Leader in the Overall market segment, as shown in the NEAT graph. This market segment reflects Capgemini’s overall ability to meet future client requirements as well as delivering immediate benefits to its quality engineering clients. Leaders are vendors that exhibit both a high ability relative to their peers to deliver immediate benefit and a high capability relative to their peers to meet client future requirements. Buy-side organizations can access the Quality Engineering Services NEAT tool (Overall) here. August 2020 2 Licensed for distribution © NelsonHall 2020
NEAT Evaluation for Capgemini: Quality Engineering Services
Vendor Analysis Summary for Capgemini
Overview
Capgemini's testing activities involve ~22.5k career testers. The company has 900 testing
clients and markets its services under the Digital Assurance and Quality Engineering (DAQE)
brand.
Since 2016, Capgemini has had a strategic business unit (SBU)-centric model for its software
testing services. Under this model, SBUs have responsibility for go-to-market, bids, and
delivery. The company believes this organization approach helps to win small, digital- and
agile-centric contracts that require an onshore reactivity to proposals.
Capgemini also highlights that SBUs are in a better position to conduct personnel reskilling
locally than a more centralized structure. Within SBUs, testing practices have full P&L
responsibility. The SBUs include Sogeti, which primes in Europe in the respective geographies;
the Financial Services SBU, which include sub-business units, i.e. banking, insurance,
Continental Europe, and APAC; and LatAM, APAC, and North America.
Nevertheless, Capgemini wants to maintain a high level of coordination across its different
software testing units, and in 2016 created its Unified Technology Office (UTO). UTO is a center
of expertise which has responsibility for:
• Harmonizing technologies, methodologies, best practices, and partnerships with testing
ISVs across the different SBUs of Capgemini Group
• Setting up TCoEs: both vertical (telecom, media and entertainment, financial services,
energy and utilities, public sector, automotive, healthcare and life sciences, and high-tech)
and technology (process, automation, performance engineering, specialized services,
digital, and agile/DevOps)
• Accelerator and platform creation and creating an ecosystem with Silicon Valley-based
startups
• Co-leadership on personnel reskilling with SBUs
• Marketing
• Partnership relationships.
UTO is an essential element of Capgemini Group's testing activities. It has ~200 dedicated
personnel and a budget which is 2% to 2.5% of Capgemini's testing revenues (~€25m).
Capgemini has structured its testing portfolio priorities around:
• CX and business performance, notably through its OPTIK and DigiAssure IP investments
• DevOps, agile, and continuous testing automation. Capgemini highlights that two-thirds of
its QA work is now related to agile projects: the company continues to invest in this area,
notably through IP creation:
− Smart QA, an IP that is the core element of the portfolio
− Adapt, for automated test script creation and maintenance, going beyond the
traditional continuous testing "platforms" that are now common
• AI in QA. In the past 12 months, Capgemini has accelerated its roll-out of AI-based IPO:
August 2020 3
Licensed for distribution © NelsonHall 2020NEAT Evaluation for Capgemini: Quality Engineering Services
− An example is Adapt, for accelerating test script creation from BDD-standardized test
cases. Adapt goes beyond the traditional NLP and relies on identifying components
and objects for test script creation and maintenance
− Another IP is ADA in the file of test data management. Capgemini generates synthetic
data beyond text and numbers to include images, and provide metadata and data
patterns
• Re-skill with a focus on expanding capabilities of testers from functional and manual
testing to quality engineering and SDETs.
Alongside these priorities, Capgemini continues to push its portfolio around RPA and test
automation, test data and test environment management, SAP migration around S/4HANA and
SAP Cloud migration, and Salesforce.
Financials
NelsonHall estimates that Capgemini’s 2019 software testing revenues were $1.6bn. This
number reflects the activities of Capgemini's career testers and does not include revenues
from the 14.5k application engineers with testing capabilities. NelsonHall estimates the
breakdown of 2019 testing services revenues by geography to be:
• North America: ~40%
• EMEA: 49%
• RoW: 11%.
Strengths
• UX testing: unlike some competitors, Capgemini has invested in automating several of its
UX testing activities, with DigiAssure. Also, its offering benefits from Capgemini Invent,
which is also active in UX, end-user research, UX testing, and product design
• AI use cases: Capgemini is among the leaders in developing AI use cases for automating
testing services, beyond analytics-centric use cases. An example is Adapt, to further
shorten automated test creation and maintenance. Another example is ADA around
synthetic data creation
• RPA use cases: Capgemini is one of the early vendors active in deploying RPA for testing
purposes
• The delivery engine is well-balanced between large offshore presence and significant
onshore personnel.
Challenges
• Lack of proven AI system testing capabilities
• Lack of proven RPA software testing. Capgemini highlights that it has led a large-scale bot
testing project for a large CPG firm
• Overall cognitive: competition in AI has increased significantly, and Capgemini will need to
invest in developing AI use cases, both in terms of analytics and automation (such as
Adapt) further.
August 2020 4
Licensed for distribution © NelsonHall 2020NEAT Evaluation for Capgemini: Quality Engineering Services Strategic Direction Overall, Capgemini wants to be the leader in testing services and a trusted partner to clients. The company is happy to start small with strategic clients and, over time, expand the relationship. To become a trusted partner, Capgemini relies on the quality of execution and investment in innovation in terms of thought leadership, portfolio, and IP. As a result, it has focused on smaller discretionary projects than in the past: it has aligned its portfolio towards continuous testing, AI, and automation overall. IP creation is a significant element of Capgemini's QA value proposition. The company continues to invest in accelerators and IP, with AI its priority. Alongside this, Capgemini is also transitioning its QA personnel (and the workforce of its clients) to new skills. The company is promoting self-learning, with an adoption acceleration during the COVID-19 pandemic. It is also making self-learning mandatory for its QA teams and part of individual evaluation processes. Finally, Capgemini has focused, from a commercial perspective, on bundled ADM opportunities that also include QA. The company highlights that agile projects increasingly bundle development, enhancements, and testing services, and works with its peers in ADM to address the changing client demand. Outlook Capgemini has a balanced QA service portfolio with investments across UX testing, AI and automation, and continuous testing. This balanced portfolio positions Capgemini among the leaders in QA. We would like Capgemini to further invest in UX testing automation. While demand for UX testing has not been as significant as for continuous testing, NelsonHall still thinks the potential for UX testing is immense, both from a client demand perspective and automation effort. Finally, Capgemini has significant engineering and R&D services capabilities across sectors, including medical devices, high-tech, and aerospace. The company has developed several accelerators to automate (in the lab) QA of bundled hardware and software-based devices/equipment. With the acquisition of Altran, NelsonHall expects an acceleration in its hardware and software QA automation, initially around product engineering services. Over time, Capgemini will expand its QA focus to digital manufacturing/"intelligent industry". August 2020 5 Licensed for distribution © NelsonHall 2020
NEAT Evaluation for Capgemini: Quality Engineering Services
Quality Engineering Services Market Summary
Buy-Side Dynamics
The three primary quality engineering (software testing) services client segments are:
• "Agile Mainstreams": organizations that are transitioning to hybrid agile (with digital
projects adopting agile and non-digital remaining on waterfall methodologies). They are
currently implementing DevOps tools (i.e. continuous testing) to increase their level of
automation. Agile Mainstreams are reskilling their manual testers
• "Advanced Automation”: organizations engaged in an agile and continuous testing
transformation like Agile Mainstreams. They look at emerging automation opportunities
(e.g. AI-based automated test script creation, RPA tools) and reach new levels of
automation, initially in functional testing
• "Digital Matures": organizations that have several digital programs and look to automate
digital technologies (e.g. chatbots, IoT) and UX research and testing to overcome
challenges such as volume (IoT). Clients of managed testing services are mostly IT
departments and continue to be a large testing service segment. Efficiency organizations
are currently working on making their TCoEs relevant to digital. They are deploying
automation through continuous testing approaches, AI and other cognitive technologies.
Market Size & Growth
NelsonHall expects a sharp decline (-5%) in 2020 in testing services spending, followed by a
moderate rebound in 2021 (+2%) as clients reconsider their expenses with the economic
recession. Spending will reach $38bn in 2024, representing a +2% CAGR in 2019-2024.
In terms of geographies, NelsonHall is expecting:
• North America will rebound fast, despite a substantial recession
• Europe will improve more slowly and will be impacted by Brexit
• APAC will bounce back in China, India, and Australia, with Japan flat
• In Latin America, Mexico's recovery will be led by the U.S. economy, while Brazil and
Argentina will struggle.
By activity:
• Functional testing will be flat during the 2019-24 period. Within functional testing, manual
testing spending will accelerate its decline (with 2019-24 CAGR of -5%) impacted by the
recession, and the adoption of functional automation (+8%). COTS and digital testing will
be affected by the downturn initially and resume their spending driven by upgrades and
digital transformation (respectively 4% and 5%)
• Outside of functional testing, specialized testing will be up by 6% during 2019-24, driven
by security (+9%), cognitive (mostly AI, +26%), and test support services (+4%).
August 2020 6
Licensed for distribution © NelsonHall 2020NEAT Evaluation for Capgemini: Quality Engineering Services
Success Factors
Critical criteria for selecting a software testing services vendor are somewhat different by
client segment:
"Agile Mainstream" clients look for the ability to:
• Deploy DevOps software or platforms to drive automation, to serve agile projects
• Expand automation outside of functional execution to continuous testing, and experiment
with new functionality such as test support services (e.g. test data and environment
management) and AI use cases
• Reskill manual testers towards technical services.
"Advanced Automation” organizations select testing vendors based on their ability to
demonstrate:
• Their investment in AI use cases, initially around AI-based analytics and expanding to
automation
• Best practices and sharing a clear view of the art of the possible.
For "Digital Mature" clients, vendors must demonstrate:
• They have capabilities ranging from change management, business consulting, UX
research and testing and QA, and can make it work effectively
• They are investing in joint offerings spanning UX and QA and have client references.
Outlook
Over the next few years, the main challenges of the software testing services industry are:
• Continuing investment in AI use cases and expanding to have AI involved in test
automation, initially around making test script creation more automated
• Including RPA in testing considerations for both automating testing of workflows and bots,
and using RPA software for automating testing in 2E2 scenarios across different
programming languages and technologies
• UX testing (apart from accessibility testing) lacks automation, inhibited by fragmentation
of tools and human-intensive activities. A few leading vendors are investing ahead of client
demand
• Reorganizing their clients’ TCoEs. A major issue will be the reskilling of manual testers
towards becoming testing software specialists. This transition of skills will have divergent
success among testing personnel. At this point, it is not clear whether testing service
vendors will be able to retrain manual testers effectively or will have to turn to lay-offs
• Also, TCoEs in the long term will need to provide an increasing share of specialized testing
services, provide software tools, and roll out best practices. TCoEs will become shared
services and move out of functional testing services.
August 2020 7
Licensed for distribution © NelsonHall 2020NEAT Evaluation for Capgemini: Quality Engineering Services
NEAT Methodology for Quality Engineering Services
NelsonHall’s (vendor) Evaluation & Assessment Tool (NEAT) is a method by which strategic
sourcing managers can evaluate outsourcing vendors and is part of NelsonHall's Speed-to-
Source initiative. The NEAT tool sits at the front-end of the vendor screening process and
consists of a two-axis model: assessing vendors against their ‘ability to deliver immediate
benefit’ to buy-side organizations and their ‘ability to meet client future requirements’. The
latter axis is a pragmatic assessment of the vendor's ability to take clients on an innovation
journey over the lifetime of their next contract.
The ‘ability to deliver immediate benefit’ assessment is based on the criteria shown in Exhibit
1, typically reflecting the current maturity of the vendor’s offerings, delivery capability,
benefits achievement on behalf of clients, and customer presence.
The ‘ability to meet client future requirements’ assessment is based on the criteria shown in
Exhibit 2, and provides a measure of the extent to which the supplier is well-positioned to
support the customer journey over the life of a contract. This includes criteria such as the level
of partnership established with clients, the mechanisms in place to drive innovation, the level
of investment in the service, and the financial stability of the vendor.
The vendors covered in NelsonHall NEAT projects are typically the leaders in their fields.
However, within this context, the categorization of vendors within NelsonHall NEAT projects is
as follows:
• Leaders: vendors that exhibit both a high ability relative to their peers to deliver immediate
benefit and a high capability relative to their peers to meet client future requirements
• High Achievers: vendors that exhibit a high ability relative to their peers to deliver
immediate benefit but have scope to enhance their ability to meet client future
requirements
• Innovators: vendors that exhibit a high capability relative to their peers to meet client
future requirements but have scope to enhance their ability to deliver immediate benefit
• Major Players: other significant vendors for this service type.
The scoring of the vendors is based on a combination of analyst assessment, principally around
measurements of the ability to deliver immediate benefit; and feedback from interviewing of
vendor clients, principally in support of measurements of levels of partnership and ability to
meet future client requirements.
Note that, to ensure maximum value to buy-side users (typically strategic sourcing managers),
vendor participation in NelsonHall NEAT evaluations is free of charge and all key vendors are
invited to participate at the outset of the project.
August 2020 8
Licensed for distribution © NelsonHall 2020NEAT Evaluation for Capgemini: Quality Engineering Services
Exhibit 1
‘Ability to deliver immediate benefit’: Assessment criteria
Assessment Category Assessment Criteria
Continuous testing
AI use cases: analytics
AI use cases: automation
RPA-based automation use cases
Offerings Testing of AI systems
Testing of RPA software and systems
UX testing: usability
UX testing: accessibility
UX testing: other
Indian delivery capability
US onshore capability
Delivery UK onshore capability
CE onshore capability
Offshore leverage
Customer presence globally
Customer presence in NA
Presence Customer presence in UK
Customer presence in CE
Customer presence In RoW
Level of cost savings achieved
Increased application quality/reduced production downtime
Increased speed-to-market for digital initiatives
Benefits Achieved
Increased end-user/business satisfaction/UX
Other benefits achieved
Pricing approach
August 2020 9
Licensed for distribution © NelsonHall 2020NEAT Evaluation for Capgemini: Quality Engineering Services
Exhibit 2
‘Ability to meet client future requirements’: Assessment criteria
Assessment Category Assessment Criteria
Investment in continuous testing
Investment in AI use cases
Investment in RPA use cases
Investments Investment in testing of cognitive technology
Investment in usability testing
Investment in accessibility testing
Investment in other UX testing activities
Market Momentum Quality engineering (software testing) market momentum
Mechanisms in place to deliver client automation innovation
Extent to which client perceives that automation innovation has
been delivered
Suitability of vendor to meet future continuous testing needs of
Ability to Deliver Improved clients
Outcomes Suitability of vendor to meet future cognitive testing needs of
clients
Suitability of vendor to meet future UX testing needs of clients
Perception of suitability to meet future needs for other
technologies
Financial Security Financial rating
For more information on other NelsonHall NEAT evaluations, please contact the NelsonHall
relationship manager listed below.
Sales Enquiries
NelsonHall will be pleased to discuss how we can bring benefit to your
organization. You can contact us via the following relationship manager:
research.nelson-hall.com Guy Saunders at guy.saunders@nelson-hall.com
Important Notice
Copyright © 2020 by NelsonHall. All rights reserved. NelsonHall exercises its best efforts in preparation of the information
provided in this report and believes the information contained herein to be accurate. However, NelsonHall shall have no
liability for any loss or expense that may result from incompleteness or inaccuracy of the information provided.
August 2020 10
Licensed for distribution © NelsonHall 2020You can also read
