Testing and debugging

Each class must be thoroughly tested with an independent unit test in the test directory. For complete coverage, each function of the class should have at least as many tests to cover all parts of code, and possibly as many as the number of code flow paths. So, if your function has one if statements, it should have at least two tests (to make sure each branch is tested); and if it has three if statements, it may need up to eight different tests to ensure that all combinations are tested. (For further discussion, read about cyclomatic complexity.) It’s useful in such cases to define helper functions to better isolate conditionals from each other.

Running CTest

When configured with CELERITAS_BUILD_TESTS (see Configuration options), CTest will be automatically configured. Running through CTest sets special environment variables for data, disabling GPUs, or testing the code itself. CTest can be run either through ninja test or by manually invoking ctest. Two useful ways to run are ctest -V -R <test regex>, which will run one or more tests that match the given regular expression (such as corecel/math/), and ctest -j --output-on-failure which runs in parallel and prints only test failures.

Using GoogleTest

Google test is very well documented <https://google.github.io/googletest/>. Celeritas defines a base class test harness with some utility functions:

class Test : public testing::Test

Googletest test harness for Celeritas codes.

The test harness is constructed and destroyed once per subtest. It contains helper functions and data commonly needed in Celeritas tests.

Subclassed by celeritas::testdetail::test::JsonComparerTest

as well as several macros that simplify testing with floating-point data (and vectors thereof):

EXPECT_VEC_EQ(expected, actual)

Container equality macro.

EXPECT_REAL_EQ(expected, actual)

Single-ULP floating point equality macro.

EXPECT_SOFT_EQ(expected, actual)

Soft equivalence macro.

EXPECT_SOFT_NEAR(expected, actual, rel_error)

Soft equivalence macro with relative error.

EXPECT_VEC_SOFT_EQ(expected, actual)

Container soft equivalence macro.

EXPECT_VEC_NEAR(expected, actual, rel_error)

Container soft equivalence macro with relative error.

EXPECT_JSON_EQ(expected, actual)

JSON string equality (soft equal for floats)

PRINT_EXPECTED(data)

Print the given container as an array for regression testing.

For more details on the test harnesses, especially the hierarchy used for setting up physics problems for testing, see the celeritas::test namespace in the Doxygen developer documentation.

You can run most tests manually from the build directory and filter so that only a subset of tests run:

$ ./test/celeritas/global_Stepper --gtest_filter=SimpleComptonTest.host

Using LLDB

LLVM’s built-in debugger is a very helpful tool for understanding what may be going wrong (or right!) in the code. It’s best if you can reduce a bug to the simplest form that will run in a unit test with a debug assertion failure. Then you can run lldb, telling it to break on C++ exception throws and perform a backtrace after running, while telling GoogleTest to filter on the failing test:

$ lldb -o "break set -E c++" -o "run" -o "bt" -- ./test/celeritas/optical_Cerenkov --gtest_filter=CerenkovTest.generator
(lldb) target create "./test/celeritas/optical_Cerenkov"
Current executable set to '/Users/seth/Code/celeritas/build/test/celeritas/optical_Cerenkov' (arm64).
(lldb) settings set -- target.run-args  "--gtest_filter=CerenkovTest.generator"
(lldb) break set -E c++
Breakpoint 1: no locations (pending).
(lldb) run
2 locations added to breakpoint 1
Celeritas version 0.5.0-dev.209+dc984b0d8
Note: Google Test filter = CerenkovTest.generator
[==========] Running 1 test from 1 test suite.
[----------] Global test environment set-up.
[----------] 1 test from CerenkovTest
[ RUN      ] CerenkovTest.generator
Process 67474 stopped
* thread #1, queue = 'com.apple.main-thread', stop reason = breakpoint 1.1
    frame #0: 0x182ef4158 libc++abi.dylib`__cxa_throw
libc++abi.dylib`:
->  0x182ef4158 <+0>:  pacibsp
    0x182ef415c <+4>:  stp    x22, x21, [sp, #-0x30]!
    0x182ef4160 <+8>:  stp    x20, x19, [sp, #0x10]
    0x182ef4164 <+12>: stp    x29, x30, [sp, #0x20]
Target 0: (optical_Cerenkov) stopped.
Process 67474 launched: '/Users/seth/Code/celeritas/build/test/celeritas/optical_Cerenkov' (arm64)
(lldb) bt
* thread #1, queue = 'com.apple.main-thread', stop reason = breakpoint 1.1
  * frame #0: 0x182ef4158 libc++abi.dylib`__cxa_throw
    frame #1: 0x100017f98 optical_Cerenkov`celeritas::RejectionSampler<double>::RejectionSampler(this=0x16fdfcda8, f=-0.0062093880005715963, fmax=0.17188544207007173) at RejectionSampler.hh:87:5
    frame #2: 0x10001714c optical_Cerenkov`celeritas::RejectionSampler<double>::RejectionSampler(this=0x16fdfcda8, f=-0.0062093880005715963, fmax=0.17188544207007173) at RejectionSampler.hh:86:1
    frame #3: 0x100014c64 optical_Cerenkov`celeritas::Span<celeritas::optical::Primary, 18446744073709551615ul> celeritas::optical::CerenkovGenerator::operator()<celeritas::test::DiagnosticRngEngine<std::__1::mersenne_twister_engine<unsigned int, 32ul, 624ul, 397ul, 31ul, 2567483615u, 11ul, 4294967295u, 7ul, 2636928640u, 15ul, 4022730752u, 18ul, 1812433253u>>>(this=0x16fdfd1f8, rng=0x16fdfdc48) at CerenkovGenerator.hh:165:18
    frame #4: 0x10000ed60 optical_Cerenkov`celeritas::test::CerenkovTest_generator_Test::TestBody()::$_0::operator()(this=0x16fdfdb20, pre_step=0x16fdfd910, particle=0x16fdfd8d0, sim=0x16fdfd8a8, pos=0x16fdfd890, num_samples=64) const at Cerenkov.test.cc:361:28
    --8<-- snip --8<--

Many classes in Celeritas store complex structures of data. Normally LLDB does not understand the various data pointers, so “collection groups” (such as Params data) are unintelligible:

(lldb) print params->host_ref()
(const celeritas::ParamsDataInterface<celeritas::optical::CerenkovData>::HostRef) {
  angle_integral = {
    storage_ = {
      data = {
        s_ = {
          data = 0x600001e2faa0
          size = 1
        }
      }
    }
  }
  reals = {
    storage_ = {
      data = {
        s_ = {
          data = 0x00014282ac00
          size = 202
        }
      }
    }
  }
}

You can execute these commands (note that this assumes the working directory is one below the source, as it would if running in build):

command script import ../scripts/dev/celerlldb.py --allow-reload
type synthetic add -x "^celeritas::Span<.+>$" --python-class celerlldb.SpanSynthetic
type synthetic add -x "^celeritas::ItemRange<.+>$" --python-class celerlldb.ItemRangeSynthetic

Then the “spans” of data will print their actual contents:

(lldb) print params->host_ref()
(const celeritas::ParamsDataInterface<celeritas::optical::CerenkovData>::HostRef) {
  angle_integral = {
    storage_ = {
      data = {
        [0] = {
          grid = (begin = 0, end = 0)
          value = (begin = 0, end = 0)
        }
      }
    }
  }
  reals = {
    storage_ = {
      data = {
        [0] = 0.0000010981771340407463
        [1] = 0.0000011070017717250021
        [2] = 0.0000011169747606594615
    --8<-- snip --8<--

For large data structures , you can prevent LLDB from eliding the deep/long data:

set set target.max-children-depth 16
set set target.max-children-count 1024

When trying to debug a failure on CPU in the main Celeritas stepping loop, you can call a global function to print the full state of the current track:

(lldb) call celeritas::debug_print(track)
{
 "geo": {
  "dir": [
   0.9998302826766889,
   0.010529089939196719,
   0.015117675340624488
  ],
  "is_on_boundary": false,
  "is_outside": false,
  "pos": [
   -2.135075225174846,
   0.0,
   0.0
  ],
  "volume_id": "inner@0x60000350ada0"
 },
 ...

If the stepping loop “hangs” (i.e., the number of steps seems unbounded) and you have access to a debugger, you can call the Stepper::kill_active method to kill all active tracks and (on CPU) print detailed debug information about them.