Code coverage - tutorial

Page content

Code coverage

Definition from Wikipedia:

In computer science, test coverage is a measure (in percent) of the degree to which the source code of a program is executed when a particular test suite is run.

Types:

  • Function coverage – has each function (or subroutine) in the program been called?
  • Statement coverage – has each statement in the program been executed?
  • Edge coverage – has every edge in the control-flow graph been executed?
    • Branch coverage – has each branch (also called the DD-path, Decision-to-decision path) of each control structure (such as in if and case statements) been executed? For example, given an if statement, have both the true and false branches been executed? (This is a subset of edge coverage.)
  • Condition coverage – has each Boolean sub-expression evaluated both to true and false? (Also called predicate coverage.)

Profiling

Definition from Wikipedia:

In software engineering, profiling (“program profiling”, “software profiling”) is a form of dynamic program analysis that measures, for example, the space (memory) or time complexity of a program, the usage of particular instructions, or the frequency and duration of function calls. Most commonly, profiling information serves to aid program optimization, and more specifically, performance engineering.

Tools around coverage

  • gcov: GNU coverage. https://en.wikipedia.org/wiki/Gcov
  • lcov: LTP GCOV extension. LTP stands for “Linux Test Project”. A graphical GCOV front-end.
  • grcov: GNU Rust coverage. A coverage tool written in Rust by Mozilla.

Apply grcov to Rust projects

Install

cargo install grcov

Quickstart

You need to write some tests before check coverage (by definition of coverage). I skipped this topic.

https://github.com/mozilla/grcov#usage

rustup component add llvm-tools-preview
export RUSTFLAGS="-Cinstrument-coverage"
cargo build
export LLVM_PROFILE_FILE="coverage-%p-%m.profraw"

cargo test
# After the test, created profiling info are created, for example:
# 	default.profraw
# 	coverage-823742-14124375359717810450_0.profraw
# 	coverage-823742-201223823317921146_0.profraw
# 	coverage-823742-7575863756035616307_0.profraw
# 	coverage-823742-8281542370250289214_0.profraw
# 	coverage-823792-12232369502980225749_0.profraw
# 	coverage-823793-14124375359717810450_0.profraw
# 	coverage-823793-201223823317921146_0.profraw
# 	coverage-823793-7575863756035616307_0.profraw
# 	coverage-823793-8281542370250289214_0.profraw

# Generate HTML report based on profiling files
grcov . -s . --binary-path ./target/debug/ -t html --branch --ignore-not-existing -o ./target/debug/coverage/

If you want to used lcov

Change the last command as follows:

grcov . -s . --binary-path ./target/debug/ -t lcov --branch --ignore-not-existing -o ./target/debug/coverage
genhtml -o ./target/debug/coverages/ --show-details --highlight --ignore-errors source --legend ./target/debug/coverage