BullseyeCoverage
BullseyeCoverage is a code coverage analyzer for the C and C++ languages. With BullseyeCoverage, you can:
The steps involved in using BullseyeCoverage are:
BullseyeCoverage cannot measure coverage of assembly code.
When coverage build is enabled, BullseyeCoverage intercepts compiler invocations to perform instrumentation.
Information about the code is written to a coverage file.
The instrumented code is compiled using your compiler.
At run-time,
the coverage file is updated with coverage measurements.
The COVFILE
environment variable names the coverage file.
You can view the coverage file using the
Coverage Browser
or the
command line report programs.
BullseyeCoverage measures condition/decision coverage and function coverage. The following specific events are monitored.
if
,
if
-else
, while
, do
-while
,
and for
statements.
The first operand of the ?:
operator is a decision if the ?:
occurs outside a controlling expression.
In the example code below, the decisions are marked by yellow boxes.
if (v1 == v2 && (v3 || v4)) v5 = v6 ? v7 : v8; if (v6 ? v7 : v8) v9 = (v10 || v11) ? v12 : v13; if ((v10 || v11) ? v12 : v13) v14 = v15 || v16;
&&
and ||
that does not contain logical operators.
The ?:
operator is also a logical operator.
The first operand of the ?:
operator is a condition
if it does not contain logical operators and the ?:
occurs in a decision.
In the example code below, the conditions are marked by blue boxes.
if (v1 == v2 && (v3 || v4)) v5 = v6 ? v7 : v8; if (v6 ? v7 : v8) v9 = (v10 || v11) ? v12 : v13; if ((v10 || v11) ? v12 : v13) v14 = v15 || v16;
case
and default
label was selected.
When the default
label is omitted,
the branch to the end of the switch
statement is indicated.
For example, BullseyeCoverage indicates whether control flowed to each of the pink boxed areas below.
switch (a) { case 1: case 2: statements break; } // Implicit default
__try
, __except
) is indicated similarly.
For example, BullseyeCoverage indicates whether control flowed to the pink boxed area below.
try { statements } catch (exception-declaration) { statements } catch (...) { statements }
void function()
{
statements
}
[] { statements }
for
statement body was entered.
for (for-range-declaration: expression) statement
By default, BullseyeCoverage does not measure coverage of code produced by macros.
BullseyeCoverage does not measure coverage of variable initializers at file scope.
BullseyeCoverage does not measure coverage of static
variable initializers in C.
These initializers are evaluated at compile-time.
BullseyeCoverage does not measure coverage of functions declared consteval
or constexpr
.
These functions might be evaluated at compile-time.
Likewise,
BullseyeCoverage does not measure coverage of initializers declared constexpr
or constinit
.
BullseyeCoverage measures coverage in template functions the same way as regular functions. Coverage is measured by instrumenting the source code. Coverage in all template instantiations is combined and reported together.
These are general guidelines for usage of C++ features and tools that can have an impact on coverage measurement.
&&
and ||
.
Defining these operators with operand type or return type other than bool
conflicts with condition/decision coverage at a basic level.
In addition to the conflict with condition/decision coverage,
overloading operators &&
and ||
is not recommended due to these semantic differences:
Use the alternative tokens 'and
' and 'or
' rather than the token spellings '&&
' and '||
'.
The token &&
is used for multiple purposes,
which can be difficult to distinguish.
The token &&
serves as both the logical-and operator as well as the rvalue reference qualifier.
With GCC and Clang, &&
is also a unary operator taking the address of a label.
BullseyeCoverage recognizes 'and
' as the logical-and operator unambiguously.
To enable the alternative tokens with Microsoft C++,
use option /permissive-
or
use C++20 or later, for example with /std:c++20
.
/Zc:preprocessor
,
available in Visual Studio 2019 version 16.5 and later.
The Microsoft traditional preprocessor is buggy and idiosyncratic.
Although we make effort to mimic this incorrect behavior,
it is not always possible and in rare instances the differences lead to a failed build.
The Microsoft conforming preprocessor behaves properly,
eliminating the possibility of a difference in behavior.
/permissive-
or enable C++20 or later.
If this is not feasible,
use covc option --strict
.
These options ensure that latent errors in uninstantiated templates and other standards violations
result in straightforward error messages that are easy to understand and fix.
Instrumenting with BullseyeCoverage typically increases execution time to 1.2x, code size to 1.4x and compile time to 1.7x with most Microsoft C++ projects, 3-8x with other compilers. Your results may vary. Compiler optimization settings greatly affect performance of instrumented code.
BullseyeCoverage may modify compiler options and directives affecting performance. These changes may include, but are not limited to, the following.
for
directive (#pragma omp parallel for) is disabled
Updated: 29 Aug 2023
Copyright © Bullseye Testing Technology. All Rights Reserved.