<no title> — dask-hip 1.0.0 documentation

Size of the CUDA device LRU cache, which is used to determine when the worker starts spilling to host memory. Can be an integer (bytes), float (fraction of total device memory), string (like "5GB" or "5000M"), "auto" or 0 to disable spilling to host (i.e. allow full device memory usage). Another special value "default" (which happens to be the default) is also available and uses the recommended Dask-CUDA’s defaults and means 80% of the total device memory (analogous to 0.8), and disabled spilling (analogous to auto/0) on devices without a dedicated memory resource, such as system on a chip (SoC) devices.

Default:: 'default'

--enable-cudf-spill, --disable-cudf-spill#

Enable automatic cuDF spilling. WARNING: This should NOT be used with JIT-Unspill.

Default:: False

--cudf-spill-stats <cudf_spill_stats>#: Set the cuDF spilling statistics level. This option has no effect if --enable-cudf-spill is not specified.

--rmm-pool-size <rmm_pool_size>#: RMM pool size to initialize each worker with. Can be an integer (bytes), float (fraction of total device memory), string (like "5GB" or "5000M"), or None to disable RMM pools.

Note

This size is a per-worker configuration, and not cluster-wide.

--rmm-maximum-pool-size <rmm_maximum_pool_size>#: When --rmm-pool-size is specified, this argument indicates the maximum pool size. Can be an integer (bytes), float (fraction of total device memory), string (like "5GB" or "5000M") or None. By default, the total available memory on the GPU is used. rmm_pool_size must be specified to use RMM pool and to set the maximum pool size.

Note

When paired with --enable-rmm-async the maximum size cannot be guaranteed due to fragmentation.

Note

This size is a per-worker configuration, and not cluster-wide.

--rmm-managed-memory, --no-rmm-managed-memory#

Initialize each worker with RMM and set it to use managed memory. If disabled, RMM may still be used by specifying --rmm-pool-size.

Warning

Managed memory is currently incompatible with NVLink. Trying to enable both will result in failure.

Default:: False

--rmm-async, --no-rmm-async#

Initialize each worker with RMM and set it to use RMM’s asynchronous allocator. See rmm.mr.CudaAsyncMemoryResource for more info.

Warning

The asynchronous allocator is incompatible with RMM pools and managed memory, trying to enable both will result in failure.

Default:: False

--set-rmm-allocator-for-libs <rmm_allocator_external_lib_list>#

Set RMM as the allocator for external libraries. Provide a comma-separated list of libraries to set, e.g., “torch,cupy”.

Options:: cupy | torch

--rmm-release-threshold <rmm_release_threshold>#: When rmm.async is True and the pool size grows beyond this value, unused memory held by the pool will be released at the next synchronization point. Can be an integer (bytes), float (fraction of total device memory), string (like "5GB" or "5000M") or None. By default, this feature is disabled.

Note

This size is a per-worker configuration, and not cluster-wide.

--rmm-log-directory <rmm_log_directory>#: Directory to write per-worker RMM log files to. The client and scheduler are not logged here. Can be a string (like "/path/to/logs/") or None to disable logging.

Note

Logging will only be enabled if --rmm-pool-size or --rmm-managed-memory are specified.

--rmm-track-allocations, --no-rmm-track-allocations#

Track memory allocations made by RMM. If True, wraps the memory resource of each worker with a rmm.mr.TrackingResourceAdaptor that allows querying the amount of memory allocated by RMM.

Default:: False

--pid-file <pid_file>#: File to write the process PID.

--resources <resources>#: Resources for task constraints like "GPU=2 MEM=10e9".

--dashboard, --no-dashboard#

Launch the dashboard.

Default:: True

--dashboard-address <dashboard_address>#

Relative address to serve the dashboard (if enabled).

Default:: ':0'

--local-directory <local_directory>#: Path on local machine to store temporary files. Can be a string (like "path/to/files") or None to fall back on the value of dask.temporary-directory in the local Dask configuration, using the current working directory if this is not set.

--shared-filesystem, --no-shared-filesystem#: If --shared-filesystem is specified, inform JIT-Unspill that local_directory is a shared filesystem available for all workers, whereas --no-shared-filesystem informs it may not assume it’s a shared filesystem. If neither is specified, JIT-Unspill will decide based on the Dask config value specified by "jit-unspill-shared-fs". Notice, a shared filesystem must support the os.link() operation.

--scheduler-file <scheduler_file>#: Filename to JSON encoded scheduler information. To be used in conjunction with the equivalent dask scheduler option.

--protocol <protocol>#: Protocol like tcp, tls, or ucx

--interface <interface>#: External interface used to connect to the scheduler. Usually an ethernet interface is used for connection, and not an InfiniBand interface (if one is available). Can be a string (like "eth0" for NVLink or "ib0" for InfiniBand) or None to fall back on the default interface.

--preload <preload>#: Module that should be loaded by each worker process like "foo.bar" or "/path/to/foo.py".

--death-timeout <death_timeout>#: Seconds to wait for a scheduler before closing

--dashboard-prefix <dashboard_prefix>#: Prefix for the dashboard. Can be a string (like …) or None for no prefix.

--tls-ca-file <tls_ca_file>#: CA certificate(s) file for TLS (in PEM format). Can be a string (like "path/to/certs"), or None for no certificate(s).

--tls-cert <tls_cert>#: Certificate file for TLS (in PEM format). Can be a string (like "path/to/certs"), or None for no certificate(s).

--tls-key <tls_key>#: Private key file for TLS (in PEM format). Can be a string (like "path/to/certs"), or None for no private key.

--enable-tcp-over-ucx, --disable-tcp-over-ucx#: Set environment variables to enable TCP over UCX, even if InfiniBand and NVLink are not supported or disabled.

--enable-infiniband, --disable-infiniband#: Set environment variables to enable UCX over InfiniBand, implies --enable-tcp-over-ucx when enabled.

--enable-rocm-ipc, --disable-rocm-ipc#: Set environment variables to enable rocm_ipc TL for UCX, implies --enable-tcp-over-ucx when enabled.

--enable-nvlink, --disable-nvlink#: Same effect as --enable-rocm-ipc/--disable-rocm-ipc, as NVLink in not valid/supported for AMD. Only supported for easy portability. A warning will be shown unless suppressed using the ‘DASK_HIP_SUPPRESS_NVLINK_WARNING’ environment variable. Please use --enable-rocm-ipc/--disable-rocm-ipc instead.

--enable-rdmacm, --disable-rdmacm#: Set environment variables to enable UCX RDMA connection manager support, requires --enable-infiniband.

--enable-jit-unspill, --disable-jit-unspill#: Enable just-in-time unspilling. Can be a boolean or None to fall back on the value of dask.jit-unspill in the local Dask configuration, disabling unspilling if this is not set.

Note

This is experimental and doesn’t support memory spilling to disk. See proxy_object.ProxyObject and proxify_host_file.ProxifyHostFile for more info.

--worker-class <worker_class>#: Use a different class than Distributed’s default (distributed.Worker) to spawn distributed.Nanny.

--pre-import <pre_import>#: Pre-import libraries as a Worker plugin to prevent long import times bleeding through later Dask operations. Should be a list of comma-separated names, such as “cudf,rmm”.

--multiprocessing-method <multiprocessing_method>#

Method used to start new processes with multiprocessing

Options:: spawn | fork | forkserver

Arguments

SCHEDULER#: Optional argument

PRELOAD_ARGV#: Optional argument(s)

Contents