Overview
This article illustrates how to generate Boost.Test XML reports, capture those reports as a build artifact and post-process the test results. This will integrate your Boost.Test results into the pulse™ build result. The process differs for built-in projects as opposed to pulse file, both are described below.
New to Boost.Test?
If you're new to Boost.Test, you might also find our blog posts on the topic to be helpful:
Generating XML Reports With Boost.Test
Boost.Test has built-in support for XML output that can be turned on with command-line arguments if you are using the provided main function:
--log_format=XML --log_level=test_suite
These arguments both enable the XML formatter and increase the detail of the log output. Note that the output goes to stdout by default, so you will need to redirect it to a file somehow. To apply these arguments in your build, refer to the details below.
Building With Make
If you are building and running your tests with make, adding the extra command line arguments to your Makefile should be straight forward. As an example, consider the Makefile below:
TARGETS=footests bartests
all: $(TARGETS)
test: $(TARGETS) $(addprefix run-,$(TARGETS))
%: %.cpp
$(CXX) -o$@ -lboost_unit_test_framework $^
run-%: %
mkdir reports
-./$^ --output_format=XML --log_level=test_suite > reports/$(^).xml
clean:
rm $(TARGETS) reports/*.xml
This Makefile assumes two test source files "footests.cpp" and "bartests.cpp" that will both compile to Boost.Test executables. The "test" target runs the "footests" and "bartests" binaries, capturing their output in XML format in the files "reports/footests.xml" and "reports/bartests.xml".
Building With Boost.Test
Boost.Build has support for building and running Boost.Test binaries in its "testing" module. You can include this module in your Jamfile and use the "run" rule to specify which tests to execute:
using testing ;
lib boost_unit_test_framework ;
run footests.cpp boost_unit_test_framework : --output_format=XML --log_level=test_suite ;
The syntax for the run rule is, more generally:
rule run (
sources + :
args * :
input-files * :
requirements * :
target-name ? :
default-build *
)
In this case we just specify the source file, the framework library and the additional flags as args. No redirection of the output is necessary as Boost.Test will capture the output for you to a file in a configuration-specific subdirectory. In this case the output will be something like:
bin/footests.test/gcc-4.4.1/debug/footests.output
Capturing the Report
Now you need to tell pulse™ to capture the XML reports as an artifact, and to post-process them with a Boost.Test XML report post-processor. This is achieved differently for built-in versus pulse file projects. Note in the examples below we assume you have captured your test reports in a "reports" subdirectory, as illustrated in the example Makefile above. You may need to adjust the exact paths/patterns in your artifact configuration to match your setup.
Built-in Projects
Add a new directory artifact to your project to capture the XML report generated. From the project's configuration view, expand the "recipes, commands and artifacts" tree item and navigate down to the "artifacts" child item of the command that runs your tests. Click the "add" link at the bottom of the "artifacts" table. Then:
- Select "directory artifact" as the type.
- Name your artifact "Boost.Test XML Reports", set the base directory to "reports", add the include pattern "*.xml" and add the "boost.test XML report processor".
This configuration is illustrated below:
Pulse File Projects
Capture the XML report in your pulse™ file where you execute the make command:
<?xml version="1.0" encoding="UTF-8"?>
<project default-recipe="default">
<recipe name="default">
<make name="build" targets="test">
<dir-artifact name="Boost.Test XML Reports" base="reports">
<include pattern="*.xml"/>
<process processor="${boost.test XML report processor}"/>
</dir-artifact>
</make>
</recipe>
</project>
The dir-artifact element is used to capture the report files, and the post-processor is applied to it. Note that the "boost.test XML report processor" is defined in the global project template by default, so you do not need to define a processor yourself. Now your test results will show up as part of the build results in the pulse™ web interface!
