Environment variables

Note

Many of these environment variables will eventually be replaced by Control and System options in Celeritas v1.0 and beyond.

Some pieces of core Celeritas code interrogate the environment for variables to change system- or output-level behavior. These variables are checked once per execution, and checking them inserts the key and user-defined value (or empty) into a diagnostic database saved to Celeritas’ JSON output, so the user can tell what variables are in use or may be useful.

Environment variables used by Celeritas.

Variable

Component

Brief description

CELER_COLOR

corecel

Flag: log with ANSI colors

CELER_DEBUG_DEVICE

corecel

Flag: check device ID consistency

CELER_DISABLE_DEVICE

corecel

Flag: disable CUDA/HIP support

CELER_DISABLE_PARALLEL

corecel

Flag: disable MPI support

CELER_DISABLE_ROOT

corecel

Flag: disable ROOT I/O calls

CELER_DEVICE_ASYNC

corecel

Flag: allocate memory asynchronously

CELER_ENABLE_PROFILING

corecel

Flag: use NVTX/ROCTX profiling [1]

CELER_LOG

corecel

Set “global” logger verbosity [2]

CELER_LOG_LOCAL

corecel

Set “local” logger verbosity [2]

CELER_MEMPOOL… [3]

corecel

Set cudaMemPoolAttrReleaseThreshold

CELER_PERFETT… [4]

corecel

Set in-process tracing buffer size

CELER_PROFILE_DEVICE

corecel

Flag: record kernel launch counts

CUDA_HEAP_SIZE

geocel

Set cudaLimitMallocHeapSize (VG)

CUDA_STACK_SIZE

geocel

Set cudaLimitStackSize for VecGeom

G4ORG_EXPORT

orange

Filename base to save converted G4 geo

G4ORG_VERBOSE

orange

Flag: verbose Geant4-to-ORANGE conversion

G4VG_COMPARE_VOLUMES

geocel

Check G4VG volume capacity when converting

HEPMC3_VERBOSE

celeritas

HepMC3 debug level integer

VECGEOM_VERBOSE

celeritas

VecGeom CUDA verbosity integer

CELER_DISABLE

accel

Flag: disable Celeritas offloading entirely

CELER_LOG_ALL_LOCAL

accel

Flag: log more messages from all threads

CELER_KILL_OFFLOAD

accel

Flag: kill offloaded tracks [5]

CELER_NONFATAL_FLUSH

accel

Flag: print and continue on failure [6]

CELER_STRIP_SOURCEDIR

accel

Flag: clean exception output

Some of the Celeritas-defined environment variables have prefixes from other libraries because they directly control the behavior of that library and nothing else. The CELER_DEVICE_ASYNC may be needed when running HIP 5.7 or later due to the “beta” nature of hipMallocAsync: it defaults to “true” except for HIP less than 5.2 (where it is not implemented) or greater than 5.6.

Environment variables from external libraries can also be referenced by Celeritas or its apps:

Environment variables used by relevant external libraries.

Variable

Library

Brief description

CUDA_VISIBLE_DEVICES

CUDA

Set the active CUDA device

HIP_VISIBLE_DEVICES

HIP

Set the active HIP device

G4LEDATA

Geant4

Path to low-energy EM data

G4FORCE_RUN_MANAGER_TYPE

Geant4

Use MT or Serial thread layout

G4FORCENUMBEROFTHREADS

Geant4

Set CPU worker thread count

OMP_NUM_THREADS

OpenMP

Number of threads per process

Note

For frameworks integrating Celeritas, these options are configurable via the Celeritas API. Before Celeritas is set up for the first time, on a single thread access the celeritas::environment() struct (see System), and call insert for the desired key/value pairs.

Logging

The Celeritas library writes informational messages to stderr. The given levels can be used with the CELER_LOG and CELER_LOG_LOCAL environment variables to suppress or increase the output. The default is to print diagnostic messages and higher.

Logging levels in increasing severity.

Level

Description

debug

Low-level debugging messages

diagnostic

Diagnostics about current program execution

status

Program execution status (what stage is beginning)

info

Important informational messages

warning

Warnings about unusual events

error

Something went wrong, but execution can continue

critical

Something went terribly wrong: program termination imminent