SMART BDD TESTING USING CUCUMBER AND JACOCO - AUTHOR: ARISH ARBAB IS A SOFTWARE ENGINEER AT HCL SINGAPORE PTE LIMITED, HAVING EXPERTIZE ON AGILE ...
←
→
Page content transcription
If your browser does not render page correctly, please read the page content below
www.hcltech.com
Smart BDD Testing
Using Cucumber
and JaCoCo
Business
assurance
$ Testing
AuthOr:
Arish Arbab is a Software Engineer at HCL
Singapore Pte Limited, having expertize on Agile
GUI/API Automation methodologies.
WHITEPAPER April 2015SMART BDD TESTING USING CUCUMBER AND JACOCO | April 2015
TABLE OF CONTENTS
INTRODUCTION 3
PROBLEM FACED 3
SOLUTION APPROACH 4
BENEFITS 6
IMPROVEMENTS 7
APPLICABILITY TO OTHER PROJECTS 7
UPCOMING FEATURES 7
REFERENCES 8
APPRECIATIONS RECEIVED 8
ABOUT HCL 9
© 2015, HCL TECHNOLOGIES. REPRODUCTION PROHIBITED. THIS DOCUMENT IS PROTECTED UNDER COPYRIGHT BY THE AUTHOR, ALL RIGHTS RESERVED. 2SMART BDD TESTING USING CUCUMBER AND JACOCO | April 2015
INTRODUCTION
Behavioral Driven Development (BDD) testing uses natural language to describe
the “desired behavior” of the system that can be understood by the developer, tester
and the customer. It is a synthesis and refinement of practices stemming from TDD
and ATDD. It describes behaviors in a single notation that is directly accessible to
domain experts, testers and developers for improving communication. It focuses on
implementing only those behaviors, which contribute most directly to the business
outcomes for optimizing the scenarios.
PROBLEM FACED
By using the traditional automation approach, and if given a summary report of
automated tests to the business analysts, then it is guaranteed that the report would be
met with a blank expression. This makes it tricky to prove that the tests are correct—
do they match the requirement and, if this changes, what tests need to change to
reflect this? The whole idea behind BDD is to write tests in plain English, describing
the behaviors of the thing that you are testing. The main advantage of this is that
the tests reflect the user stories and the business requirements of the application.
Therefore, the results generated can be read by a non-technical person, e.g., a project
sponsor, a domain expert, a business analyst, and the tests can be validated against the
requirements for better operational efficiency.
To achieve the BDD approach, there is an open source tool—Cucumber—that
parses the scenarios/tests that are written in Gherkin language (GWT). This tool also
interacts with the API/Junit/Source code of an application, using its step definitions
test code. The best option to get the business intent into our tests is to have clear
deliverables and acceptance criteria in the user stories. These can be then translated
into feature tests containing scenarios written in Gherkin Language.
BDD is the “outside-in” software development methodology, which has changed
the tester’s role dramatically in the recent years and bridges the communication gap
between business and technology. This innovative approach has the potential to shape
a new organization standard with new SDET roles.
Problems faced while implementing the new framework:
yy Justifying the coverage done by this framework becomes difficult as the complexity
of the application code increases
yy Exposing APIs/Source Code was difficult as developers disagreed
yy Converting the software requirement into GWT format at the later stages of the
software development utilized more efforts
yy Requiring testers with good technical/coding knowledge
yy Extending the Junit by using the mocking techniques or spring-based test was a
hindrance when the complexity of the code increased
yy Convincing the BTMs to review the scenarios on every day was difficult
© 2015, HCL TECHNOLOGIES. REPRODUCTION PROHIBITED. THIS DOCUMENT IS PROTECTED UNDER COPYRIGHT BY THE AUTHOR, ALL RIGHTS RESERVED. 3SMART BDD TESTING USING CUCUMBER AND JACOCO | April 2015
SOLUTION APPROACH
Traditionally, software requirement was written in simple English sentences, but to
convert those requirements into GWT format required an extra effort that should be
properly handled. As GWT approach results in better quality of product, requirement
should be written in GWT format by the business analyst at the early stages of software
development rather than writing the requirement in domain-specific language. Set of
scenarios should depict a particular user story. Implementing this approach in a later
stage may increase the cost of a project.
Example of writing the scenarios in the abovementioned format would be:
Traditional software requirement: As a user, I want to login to the application with
valid credentials.
Simple Cucumber Feature File in Gherkin Format:
Feature: Login to the system,
User should be allowed to login with valid credentials
Scenario: Login Success
Given The login page is opened
When I input valid username and password in the textboxes
And I click the login button
Then I logged in successfully.
Step definitions for matching scenarios:
Similarly for “When” and “Then,” step definitions are created.
Corresponding step definitions are then written, and the result is displayed in the
above format that assures developer/tester/BA and customers that a particular software
requirement has been successfully implemented.
Common method of testing an application at the API level requires its API or web
services. However, if the application has neither the API available nor the developers
who want to create it for testing purposes, then the problem arises to test the code
written by developer without any extra development effort. To overcome this problem,
Junit classes that are written by developers can be used to call the respective service
methods. However, if the unit testing is being done using mocking technique, then
it would be difficult to test the actual functionality of the application. So, in order
to use the Junit methods effectively, dependency injections techniques can be used
at the testing end. Such usage would call the Junit method by passing a particular
dependency, which, in turn, calls the real service method.
© 2015, HCL TECHNOLOGIES. REPRODUCTION PROHIBITED. THIS DOCUMENT IS PROTECTED UNDER COPYRIGHT BY THE AUTHOR, ALL RIGHTS RESERVED. 4SMART BDD TESTING USING CUCUMBER AND JACOCO | April 2015
BDD approach—detailed explanation of using Cucumber by leveraging Junit
The pictorial representation of Cucumber automation framework:
App Service
Methods Model Service Methods
Junit Classes Junit Classes containing unit test methods
Cucumber Step Definitions
Cucumber
Cucumber Main(Driver)
Feature File(Scenarios)
DB, Test Data Test Result
And Reports Data(.xlsx) Database HTML Reports, Screen shots, logs of failed
scenarios etc.
Flow of the BDD framework using Cucumber is as follows-
yy The test data, which is to be used in the scenarios, is stored in Excel sheet or MySql
Database.
yy The test data is then passed to feature file, containing scenarios written in GWT
format. Scenarios are denoted by certain tags.
yy Driver class decides which scenario among the entire features files should be executed
by proving a proper tag value. Before running a particular feature, dependency used
in application is injected to avoid mocking unit testing framework.
yy Appropriate matching method defined in the Cucumber step definition file is
called.
yy From the step definition, appropriate unit test method of the parameterized Junit
class is called and, thus, test data is passed into it.
yy Unit test method, which, in turn, calls the real service method by passing the test
data to it.
yy Response is then returned from the service method to unit test method, which, in
turn, passes the information to Cucumber step definition file.
yy Expected response is validated against the returned (actual) response in the step
definition.
yy Logs are created upon failure or success and an auto HTML report is created,
depicting the passed and failed scenarios in appropriate colors.
© 2015, HCL TECHNOLOGIES. REPRODUCTION PROHIBITED. THIS DOCUMENT IS PROTECTED UNDER COPYRIGHT BY THE AUTHOR, ALL RIGHTS RESERVED. 5SMART BDD TESTING USING CUCUMBER AND JACOCO | April 2015
Now, if we have the API available, then, instead of calling the Junit test method, the
API method is called directly and rest of the flow remains the same.
Then the question arises, “Whether or not we are covering all the areas of an application
with our test as the scenarios written are very behavioral specific?” Here the need
for the code coverage arises. JaCoCo is one of the tools, which can be integrated
with Cucumber Test, and the code coverage report generated can be easily read by
customers/developers/business analysts and testers. The traditional way of testing may
work out with testers, having less technical knowledge. Nevertheless, as we can clearly
point out by seeing the above solutions and exposure of code to tester. The need for a
tester with good coding skills is prerequisite to implement this approach successfully.
JaCoCo report
Cucumber test execution can be integrated into CI (Jenkins) server and can be
executed depending upon the need. Now, when we have everything in place—
scenarios, test code and test results—it becomes difficult to convince the BTMs to
review the scenarios on a daily basis. There is another open-source tool available that
overcomes these difficulties: Pickles, open-source living document generator. This tool
generates the document consisting of the scenarios, which can be easily reviewed by
the customer as it supports advance search capabilities.
BENEFITS
Key Benefits
yy Cucumber and JaCoCo are open-source tools, which provide equal benefits that
other paid tools provide.
yy Around 80% reduction in the test execution time as compared to manual testing.
yy Cucumber can be integrated with existing tools like Selenium, TestComplete and
QTP.
yy Quality of an application increased as the defect is found at an early stage of
development.
yy Cucumber can be used to test REST and SOAP Web services, thereby, easily
adjusting in traditional way of API automation.
© 2015, HCL TECHNOLOGIES. REPRODUCTION PROHIBITED. THIS DOCUMENT IS PROTECTED UNDER COPYRIGHT BY THE AUTHOR, ALL RIGHTS RESERVED. 6SMART BDD TESTING USING CUCUMBER AND JACOCO | April 2015
yy Unlike other API testing tools, it produces more detailed, easy-to-understand test
execution report for client, developer, BA and tester regardless of their technical
knowledge.
yy One of the eminent features of Cucumber is that the separation on three levels of
feature, steps and world makes it easier to reuse certain parts of the code.
yy Scenarios can be written in different languages, including French, German, Chinese
and Hindi.
yy Cucumber with Code Coverage tool (JaCoCo) provides efficient and advanced way
of testing applications.
Available for nine programming languages, Cucumber is an open-source tool for BDD.
The tool is available in languages such as Ruby, JVM-based languages, JavaScript and
.NET (.NET languages are supported through the SpecFlow tool.).
IMPROVEMENTS
The key learning is as follows:
yy Team gained knowledge on API automation using smart BDD tools (Cucumber
and JaCoCo).
yy Team was able to understand the requirements better due to the constant interaction
with BAs, and the interaction with developers helped the team increase its technical
knowledge.
yy Detailed test results report was generated in the UI automation by integrating
Selenium with Cucumber.
yy Exposure to continuous integration servers increased, which helped in scheduling
the overnight batch execution.
yy Team collaboration and communication techniques—used for involving
customers—such as Pair Programming and Code Reviews became frequent.
APPLICABILITY TO OTHER PROJECTS
yy BDD testing is not restricted to any domain, and, hence, can be used in any domain.
yy It can be easily adjusted/merged to the traditional automation approach.
yy By leveraging Cucumber, API-based projects can also use these techniques.
yy Junit/Nunit test method can be used along with Cucumber to achieve BDD.
yy Cucumber dependencies are openly available, which can be easily integrated with
any project.
UPCOMING FEATURES
yy Collaborate across the globe with a real-time shared editor
yy Access the latest version of your executable specifications anytime
© 2015, HCL TECHNOLOGIES. REPRODUCTION PROHIBITED. THIS DOCUMENT IS PROTECTED UNDER COPYRIGHT BY THE AUTHOR, ALL RIGHTS RESERVED. 7SMART BDD TESTING USING CUCUMBER AND JACOCO | April 2015
yy Link to issue trackers
yy Edit incorrect specifications instantly, and see the results as they appear
REFERENCES
http://en.wikipedia.org/wiki/Cucumber_(software)
http://cukes.info/
http://www.ibm.com/developerworks/library/a-automating-ria/
https://github.com/cucumber/cucumber/wiki/Cucumber-JVM
http://en.wikipedia.org/wiki/Behavior-driven_development
http://guide.agilealliance.org/guide/bdd.html
APPRECIATIONS RECEIVED
Proof of Concept was accepted and greatly appreciated by the client. Appreciation for
this practice was also provided by the head and the project manager.
ABBREVIATIONS
Abbreviation Expansion
TDD Test Driven Development
ATDD Acceptance Test Driven Development
JaCoCo Java Code Coverage
UI User Interface
SDET Software Development Engineer in Test
GWT Given-When-Then
BA Business Analyst
HTML Hyper Text Markup Language
CI Continues Integration
API Application Program Interface
JVM Java Virtual Machine
BTM Business Technology Manager
© 2015, HCL TECHNOLOGIES. REPRODUCTION PROHIBITED. THIS DOCUMENT IS PROTECTED UNDER COPYRIGHT BY THE AUTHOR, ALL RIGHTS RESERVED. 8ABOUT HCL
About HCL Technologies
HCL Technologies is a leading global IT services company working with clients in the
areas that impact and redefine the core of their businesses. Since its emergence on the
global landscape, and after its IPO in 1999, HCL has focused on ‘transformational
outsourcing’, underlined by innovation and value creation, offering an integrated
portfolio of services including software-led IT solutions, remote infrastructure
management, engineering and R&D services and business services. HCL leverages
its extensive global offshore infrastructure and network of offices in 31 countries to
provide holistic, multi-service delivery in key industry verticals including Financial
Services, Manufacturing, Consumer Services, Public Services and Healthcare &
Life sciences. HCL takes pride in its philosophy of ‘Employees First, Customers
Second’ which empowers its 100,240 transformers to create real value for customers.
HCL Technologies, along with its subsidiaries, had consolidated revenues of
US$ 5.7 billion, for the Financial Year ended as on 31st December 2014 (on LTM
basis). For more information, please visit www.hcltech.com
About HCL Enterprise
HCL is a $6.8 billion leading global technology and IT enterprise comprising two
companies listed in India – HCL Technologies and HCL Infosystems. Founded
in 1976, HCL is one of India’s original IT garage start-ups. A pioneer of modern
computing, HCL is a global transformational enterprise today. Its range of offerings
includes product engineering, custom & package applications, BPO, IT infrastructure
services, IT hardware, systems integration, and distribution of information and
communications technology (ICT) products across a wide range of focused industry
verticals. The HCL team consists of over 105,699 professionals of diverse nationalities,
who operate from 31 countries including over 505 points of presence in India.
HCL has partnerships with several leading global 1000 firms, including leading
IT and technology firms. For more information, please visit www.hcl.com
Hello there! I am an Ideapreneur. I believe that sustainable business outcomes are driven by relationships nurtured through
values like trust, transparency and flexibility. I respect the contract, but believe in going beyond through collaboration,
applied innovation and new generation partnership models that put your interest above everything else. Right now 105,000
Ideapreneurs are in a Relationship Beyond the Contract™ with 500 customers in 31 countries. How can I help you?You can also read
