Once you have executed a tests suite, you could integrate the results into continuous integration tool. For this, you can generate a TAP report or an XML report.
(v2) TAP report
TAP is a protocol to integrate tests executed in different programming languages, and then generate the results in a single report. The frameworks that create reports are called TAP Producers, and these reports are used by TAP Consumers to aggregate the results and generate a general output. This framework provides the functionality to create a TAP report and its generated output is in TAP v13.
The generated report could help to integrate the test units into a continuous integration system, like Jenkins.
You can use the tests suite Example to generate a report. First, install and run the tests suite:
db2 -tf Tests_DB2UNIT_EXAMPLE.sql
db2 "CALL DB2UNIT.RUN_SUITE('DB2UNIT_EXAMPLE')"
There are two ways to create the TAP report:
- By calling the stored procedure that generates the report and returns an opened cursor, with a user temporary table with the data (
DB2UNIT_x.TAP_REPORT
table). - By calling the shell script that calls the stored procedure and then creates a file with the content.
Both methods are shown here:
db2 -x "CALL DB2UNIT.CREATE_TAP_REPORT()"
. ./tap_report
tap_report.bat
The scripts will call the following line, that retrieves the values from a declared global temporary table.
db2 "EXPORT TO tap.report OF DEL MODIFIED BY NOCHARDEL
SELECT MESSAGE
FROM DB2UNIT_2_BETA.TAP_REPORT
ORDER BY NUMBER"
After calling the report generator, you will get an output like this:
TAP version 13
1..9
not ok 1 TEST_9
---
Message: 'Exception(2): SQLCode -438, SQLState TEST9-SQL0438N Application raised error or warning with diagnostic text: "Normal error test 9". SQLSTATE=TEST9'
...
not ok 2 TEST_2
---
Message: 'Exception(2): SQLCode -438, SQLState 12345-SQL0438N Application raised error or warning with diagnostic text: "". SQLSTATE=12345'
...
not ok 3 TEST_8
---
Message: 'STRING_EQUALS'
Message: 'Different strings. The content of both strings is different'
Message: 'Expected : "A"'
Message: 'Actual : "B"'
...
not ok 4 TEST_5
---
Message: 'STRING_EQUALS'
Message: 'Strings have different lengths'
Message: 'Expected : "A"'
Message: 'Actual : "AB"'
...
not ok 5 TEST_3
---
Message: 'Exception(2): SQLCode -438, SQLState 12345-SQL0438N Application raised error or warning with diagnostic text: "Test signal". SQLSTATE=12345'
...
ok 6 TEST_4
ok 7 TEST_1
ok 8 TEST_6
ok 9 TEST_7
You can test the generated output in the following site: http://instanttap.appspot.com/ If the output is valid, you can integrate your tests with a TAP consumer.
(v2) XML report
The XML report allows to integrate db2unit into the tests phase of a software lifecycle building. Maven is a software with this characteristics, and db2unit generate an XML report that is compatible with the tests phase. Each tests suite generates a XML report, and this will be integrated into a global test report.
The structure of the XML schema is based on the documentation at http://llg.cubic.org/docs/junit/.
<?xml version="1.0" encoding="UTF-8"?>
<testsuites disabled="" <!-- The total number of unexecuted tests from all testsuites. -->
errors="" <!-- The total number of tests with error result from all testsuites. -->
failures="" <!-- The total number of failed tests from all testsuites. -->
name="" <!-- The name of the database. -->
tests="" <!-- The total number of successful tests from all testsuites. -->
time="" <!-- TODO The time in seconds to execute all test suites. -->
>
<testsuite name="" <!-- The schema name - test suite name. -->
tests="" <!-- The total number of tests in the test suite. -->
disabled="" <!-- the total number of unexecuted tests in the test suite. -->
errors="" <!-- The total number of tests in the test suite that errored. An errored
test is one that had an unanticipated problem, for example an
uncatched signal; or a problem with the implementation of the
test. -->
failures="" <!-- The total number of tests in the test suite that failed. A failure is
a test which the code has explicitly failed by using the mechanisms
for that purpose. e.g., via an assertStringEquals. -->
hostname="" <!-- Host on which the tests were executed. -->
id="" <!-- Starts at 0 for the first test suite and is incremented by 1 for each
following testsuite -->
time="" <!-- Time taken (in seconds) to execute the tests in the test suite.
Optional -->
timestamp="" <!-- When the test was executed (2014-01-21-16:17:18). Optional -->
>
<!-- Properties (e.g., environment settings) set during test execution.-->
<properties>
<!-- Property can appear multiple times. -->
<property name="" value=""/>
</properties>
<!-- testcase can appear multiple times. -->
<testcase name="" <!-- Name of the procedure. -->
assertions="" <!-- TODO Number of assertions in the test case. -->
classname="" <!-- Full procedure name for the procedure the test method is in
(Schema.Procedure). -->
time="" <!-- Time taken (in fraction of seconds) to execute the test. -->
>
<!-- skipped can appear 0 or once. -->
<skipped/>
<!-- Indicates that the test errored. An errored test is one that had an unanticipated
problem. -->
<error message="" <!-- The error message. -->
type="" <!-- The type of error that occurred. The SQL code -->
/>
<!-- Indicates that the test failed. A failure is a test which the code has explicitly failed by
using the mechanisms for that purpose. -->
<failure message="" <!-- The message specified in the assert. -->
type="" <!-- The type of the assert. -->
></failure>
<!-- Extra data from Report test. -->
<system-out></system-out>
</testcase>
<!-- Data from the Execution report. -->
<system-out></system-out>
</testsuite>
</testsuites>
Let's suppose you have compiled the DB2UNIT_EXAMPLE tests suite (included in the sources) into the database. You can generate the corresponding XML report like the following:
db2 -tvf src/examples/sql-pl/Tests_DB2UNIT_EXAMPLE.sql
db2 "call db2unit.run_suite('DB2UNIT_EXAMPLE')"
... (execution report)
db2 "call db2unit.XML_REPORT_ONE('DB2UNIT_EXAMPLE')"
db2 -x "select varchar(document, 2000) from db2unit_2_beta.XML_REPORT"
<testsuite name="DB2UNIT_EXAMPLE" tests="9" disabled="1" errors="2" failures="2" hostname="localhost" id="0" time="17" timestamp="2015-01-21T21:28:43.018638">
<properties/>
<testcase name="TEST_1" assertions="0" classname="DB2UNIT_EXAMPLE.TEST_1" time="182">
<system-out/>
</testcase>
<testcase name="TEST_2" assertions="0" classname="DB2UNIT_EXAMPLE.TEST_2" time="779">
<error type="SQLError"/>
<system-out/>
</testcase>
<testcase name="TEST_3" assertions="0" classname="DB2UNIT_EXAMPLE.TEST_3" time="721">
<error type="SQLError"/>
<system-out/>
</testcase>
<testcase name="TEST_4" assertions="0" classname="DB2UNIT_EXAMPLE.TEST_4" time="305">
<system-out/>
</testcase>
<testcase name="TEST_5" assertions="0" classname="DB2UNIT_EXAMPLE.TEST_5" time="1729">
<failure type="AssertionError"/>
<system-out/>
</testcase>
<testcase name="TEST_6" assertions="0" classname="DB2UNIT_EXAMPLE.TEST_6" time="163">
<system-out/>
</testcase>
<testcase name="TEST_7" assertions="0" classname="DB2UNIT_EXAMPLE.TEST_7" time="171">
<system-out/>
</testcase>
<testcase name="TEST_8" assertions="0" classname="DB2UNIT_EXAMPLE.TEST_8" time="1828">
<failure type="AssertionError"/>
<system-out/>
</testcase>
<testcase name="TEST_9" assertions="0" classname="DB2UNIT_EXAMPLE.TEST_9" time="0">
<skipped/>
<system-out/>
</testcase>
</testsuite>
Note: The produced XML document is not formatted. The previously shown was modified to better explain how it works.