Effective automation reporting for Selenium using ExtentReport

Posted by on in Technology
  • Font size: Larger Smaller
  • Hits: 584
  • 3 Comments
  • Print

In testing, Test Summary report is an important deliverable.  It represents the quality of a product.  As automation testing is mostly carried out in the absence of human, I recommend that test results should be presented in a good way.

Automation test report should be useful to people of all levels like automation experts, manual tester who is not aware of a code, high-level management. 

 

In an ideal case test automation report should comprise of following:

  • Statistical data like number of test cases passed, failed, skipped
  • Cause of test failure
  • Evidence (like screenshots indicating success/failure conditions)

Additional to above if we have following things in our test report then it will be impressive and useful:

  • Pass and fail percentage of tests
  • Test execution time for individual test case and a test suite
  • Test environment details
  • Representation of statistical data in the form of charts
  • Grouping of test cases as per the type like Functional, Regression etc.

TestNG or JUnit does not provide good reporting capabilities. TestNG default reports are not attractive. So for that we have to develop the customized reports.

I suggest using ExtentReport for automation test reporting will be more effective. This library allows us to accomplish the above mentioned things.

About ExtentReport:

It is an open-source test automation reporting API for Java and .NET  developers. The report is generated in HTML form.

Following are some features of ExtentReport:

  • Easy to use
  • Results are displayed in the form of pie charts
  • Provides passed test case percentage
  • Displays test execution time
  • Environment details can be added in an easy way
  • Screenshots can be attached to the report
  • Test reports can be filtered out based on the test results (Pass/Fail/Skip etc.)
  • Filtering stepwise results like info/pass/fail etc.
  • Categorized report  for Regression/Functional etc. testing
  • Test step logs can be added
  • Can be used with JUnit/TestNG
  • It can be used as a listener for TestNG
  • We can create parallel runs as well. So single report can be created for the parallel runs
  • We can add the configuration to report
  • Results from multiple runs can be combined to single report

Downloading and installation:

Download ExtentReport jar from http://extentreports.relevantcodes.com/index.html and add it as a dependency to your java project.

 

ExtentX:

ExtentX is a report server and project-wise test analysis dashboard for ExtentReports.

 

How ExtentReport works:

To see how ExtentReport exactly works, here is a simple example – One test case will pass and another will fail.

 

import org.openqa.selenium.WebDriver;

import org.openqa.selenium.firefox.FirefoxDriver;

import org.testng.Assert;

import org.testng.annotations.AfterTest;

import org.testng.annotations.BeforeTest;

import org.testng.annotations.Test;

import com.relevantcodes.extentreports.ExtentReports;

import com.relevantcodes.extentreports.ExtentTest;

import com.relevantcodes.extentreports.LogStatus;

 

public class ExtentReportTest{

     private WebDriver driver;

     ExtentReports extent;

     ExtentTest test;

     StringBuffer verificationErrors = new StringBuffer();

    

     @BeforeTest

     public void testSetUp() {

           driver = new FirefoxDriver();

           extent = new ExtentReports(".\\TestAutomationReport.html", true);    //Report initializing

           extent.addSystemInfo("Product Version", "3.0.0")   //System or environment info

                 .addSystemInfo("Author", "Sachin Kadam");

     }

    

     @Test

     public void TC1() {

           test = extent

                     .startTest("Test case 1", "Check the google home page title")  //Start test case

                     .assignAuthor("Sachin Kadam")   

                     .assignCategory("Regression", "Functional");

           String appURL = "http://google.com";

           driver.get(appURL);

           test.log(LogStatus.INFO, "Navigating to URL : "+appURL);   //Log info

           customVerify(driver.getTitle(), "Google");

           extent.endTest(test);   //End test case

           checkForErrors();

     }

    

     @Test

     public void TC2() {

           test = extent

                   .startTest("Test case 2", "Check the wikipedia home page title") //Start test case

                   .assignCategory("Functional")

                   .assignAuthor("Sachin Kadam");

           String appURL = "https://www.wikipedia.org";

           driver.get(appURL);

           test.log(LogStatus.INFO, "Navigating to URL : "+appURL); //Log info

           customVerify(driver.getTitle(), "Google"); //Incorrect expected title to fail test case

           extent.endTest(test);   //End test case

           checkForErrors();

     }

    

     //custom assertion method for string comparison

     public void customVerify(String actual, String expected){

        try{

           Assert.assertEquals(actual, expected);

           //Log pass results

           test.log(LogStatus.PASS, "Expected title:"+expected + " :: Current title:" + actual); 

           }catch(Error e){

                 //Log fail results along with error

                   test.log(LogStatus.FAIL, "Expected title:"+expected + " :: Current title:" + actual +" :: "+ e.toString());                                                 

                verificationErrors.append(e);

          }

     }

 

     @AfterTest

     public void tearDown(){

           driver.quit();

           extent.flush();

     }

     

    //Method for logging correct results to TestNG report in case of failure

     public void checkForErrors(){ 

           if(!"".equals(verificationErrors.toString())){

           Assert.fail(verificationErrors.toString());

           verificationErrors = new StringBuffer();

           }

     }

}

 

Finally generated HTML report looks like:

 ExtentReport01

 

ExtentReport02

 

ExtentReport03

 

I hope you will find ExtentReport very useful, easy to use, impressive and productive.

For more reference: http://extentreports.relevantcodes.com/index.html

 

- Sachin Kadam

 

Last modified on
Rate this blog entry:

Sachin Kadam is Senior software engineer at GS Lab.

Comments

  • Guest
    vinod borole Thursday, 01 December 2016

    very interesting

    I liked the way it is integrated with Junit and TestNG

  • Guest
    Shriniwas Thursday, 01 December 2016

    Perfect..

    This is very important information those who wants to use extent report. Nice blog. Keep it up..

  • Guest
    Shilpa Agrawal Thursday, 01 December 2016

    Nice option to make reports more interesting

    I am using the third party tool for reporting , as'TestNG' . Tried 'ExtentReport with TestNG' and my reports are now more attractive with pie charts/Pass-Fail Result/ExecutionTime..

Leave your comment

Guest Sunday, 22 January 2017
Very low screen size go to mobile site instead

Click Here