rocSHMEM environment variables#
This section describes the important environment variables used to control the behavior of rocSHMEM.
Environment variable |
Default value |
Value |
|---|---|---|
ROCSHMEM_HEAP_SIZEDefines the size of the rocSHMEM symmetric heap in bytes.
|
|
Size in bytes.
Note the heap is on the GPU memory.
|
ROCSHMEM_MAX_NUM_CONTEXTSDefines the number of contexts an application can use.
|
|
Maximum number of contexts. |
ROCSHMEM_MAX_NUM_TEAMSDefines the number of teams an application can use.
|
|
Maximum number of teams. |
ROCSHMEM_BACKENDWhen rocSHMEM is compiled for all backends, this enviroment variable
selects which backend to execute. The default value is an empty string and rocSHMEM auto-selects the most appropriate backend.
|
`` `` |
ipc: IPC Backendro: Reverse Offload Backendgda: GPU Direct Async Backend |
ROCSHMEM_UNIQUEID_WITH_MPIDefines whether rocSHMEM is expected to use MPI when using the uniqueId based initialization.
|
|
0: Do not use MPI.1: Use MPI. |
ROCSHMEM_DISABLE_MIXED_IPCDefines whether to force using the network conduit even when IPC is available.
|
|
0: Use IPC when available.1: Force network conduit. |
ROCSHMEM_USE_IB_HCADefines which NIC that this PE should be bound to. The default value is an empty string and rocSHMEM auto-detects the most appropriate NIC.
|
`` `` |
Example value:
bnxt_re0 |
ROCSHMEM_BOOTSTRAP_SOCKET_IFNAMEChooses the interface to bootstrap rocSHMEM with.
Only valid when not using MPI.
The default value is an empty string and rocSHMEM auto-detects the most appropriate interface.
|
`` `` |
Example value:
eno8303 |
ROCSHMEM_GDA_PROVIDERWhen rocSHMEM is compiled with support for multiple NIC vendors,
the enviroment variable selects the desired provider.
The default value is an empty string and rocSHMEM auto-detects the most appropriate NIC.
|
`` `` |
bnxt: Broadcom Thor 2pensando: AMD Pensando Pollaraionic: AMD Pensando Pollara (alias)mlx5: Mellanox ConnectX-7 |
ROCSHMEM_GDA_ALTERNATE_QP_PORTSEnables or disables alternating QP mappings across rocSHMEM contexts.
|
|
0: Disabled.1: Enabled. This helps saturate bandwidth on multiport bonded interfaces. |
ROCSHMEM_GDA_TRAFFIC_CLASSWhen using an NIC with an Ethernet link layer, this sets the traffic class for the QPs.
|
|
The traffic class number. |
ROCSHMEM_GDA_PCIE_RELAXED_ORDERINGEnables PCIe Relaxed Ordering when registering the symmetric heap with the RDMA NICs.
|
|
0: Disabled.1: Enabled. |