Environment variables

Note

Many of these environment variables will eventually be replaced by Control and System options in Celeritas v1.0 and beyond.

Some pieces of core Celeritas code interrogate the environment for variables to change system- or output-level behavior. These variables are checked once per execution, and checking them inserts the key and user-defined value (or empty) into a diagnostic database saved to Celeritas’ JSON output, so the user can tell what variables are in use or may be useful.

Environment variables used by Celeritas.

Variable	Component	Brief description
CELER_COLOR	corecel	Flag: log with ANSI colors
CELER_DEBUG_DEVICE	corecel	Flag: check device ID consistency
CELER_DISABLE_DEVICE	corecel	Flag: disable CUDA/HIP support
CELER_DISABLE_PARALLEL	corecel	Flag: disable MPI support
CELER_DISABLE_ROOT	corecel	Flag: disable ROOT I/O calls
CELER_DEVICE_ASYNC	corecel	Flag: allocate memory asynchronously
CELER_ENABLE_PROFILING	corecel	Flag: use NVTX/ROCTX profiling [1]
CELER_LOG	corecel	Set “global” logger verbosity [2]
CELER_LOG_LOCAL	corecel	Set “local” logger verbosity [2]
CELER_MEMPOOL… [3]	corecel	Set cudaMemPoolAttrReleaseThreshold
CELER_PERFETT… [4]	corecel	Set in-process tracing buffer size
CELER_PROFILE_DEVICE	corecel	Flag: record kernel launch counts
CUDA_HEAP_SIZE	geocel	Set cudaLimitMallocHeapSize (VG)
CUDA_STACK_SIZE	geocel	Set cudaLimitStackSize for VecGeom
G4ORG_EXPORT	orange	Filename base to save converted G4 geo
G4ORG_VERBOSE	orange	Flag: verbose Geant4-to-ORANGE conversion
G4VG_COMPARE_VOLUMES	geocel	Check G4VG volume capacity when converting
HEPMC3_VERBOSE	celeritas	HepMC3 debug level integer
VECGEOM_VERBOSE	celeritas	VecGeom CUDA verbosity integer
CELER_DISABLE	accel	Flag: disable Celeritas offloading entirely
CELER_LOG_ALL_LOCAL	accel	Flag: log more messages from all threads
CELER_KILL_OFFLOAD	accel	Flag: kill offloaded tracks [5]
CELER_NONFATAL_FLUSH	accel	Flag: print and continue on failure [6]
CELER_STRIP_SOURCEDIR	accel	Flag: clean exception output

[1]

See Performance profiling

[2] (1,2)

See Logging

[3]

CELER_MEMPOOL_RELEASE_THRESHOLD changes the amount of memory used for the device asynchronous memory pool.

[4]

CELER_PERFETTO_BUFFER_SIZE_MB

[5]

This flag, used by accel, omits Celeritas setup and instead kills tracks that would have otherwise been offloaded. This capability can be used to estimate a maximum speedup by using Celeritas.

[6]

Normally, exceeding the “maximum steps” or interrupting the stepping loop will call G4Exception, which normally kills the code. (In external frameworks this usually causes a stack trace and core dump.) Instead of doing that, kill all the active tracks and print their state. If more tracks are buffered, those will continue to transport.

Some of the Celeritas-defined environment variables have prefixes from other libraries because they directly control the behavior of that library and nothing else. The CELER_DEVICE_ASYNC may be needed when running HIP 5.7 or later due to the “beta” nature of hipMallocAsync: it defaults to “true” except for HIP less than 5.2 (where it is not implemented) or greater than 5.6.

Environment variables from external libraries can also be referenced by Celeritas or its apps:

Environment variables used by relevant external libraries.

Variable	Library	Brief description
CUDA_VISIBLE_DEVICES	CUDA	Set the active CUDA device
HIP_VISIBLE_DEVICES	HIP	Set the active HIP device
G4LEDATA	Geant4	Path to low-energy EM data
G4FORCE_RUN_MANAGER_TYPE	Geant4	Use MT or Serial thread layout
G4FORCENUMBEROFTHREADS	Geant4	Set CPU worker thread count
OMP_NUM_THREADS	OpenMP	Number of threads per process

Note

For frameworks integrating Celeritas, these options are configurable via the Celeritas API. Before Celeritas is set up for the first time, on a single thread access the celeritas::environment() struct (see System), and call insert for the desired key/value pairs.

Logging

The Celeritas library writes informational messages to stderr. The given levels can be used with the CELER_LOG and CELER_LOG_LOCAL environment variables to suppress or increase the output. The default is to print diagnostic messages and higher.

Logging levels in increasing severity.

Level	Description
debug	Low-level debugging messages
diagnostic	Diagnostics about current program execution
status	Program execution status (what stage is beginning)
info	Important informational messages
warning	Warnings about unusual events
error	Something went wrong, but execution can continue
critical	Something went terribly wrong: program termination imminent