cuvs.neighbors.vamana

cuvs.neighbors.vamana#

5 min read time

Applies to Linux

Submodules#

cuvs.neighbors.vamana.vamana

Classes#

`Index`	Vamana index object. This object stores the trained Vamana index state
`IndexParams`	IndexParams(metric=u'sqeuclidean', *, graph_degree=32, visited_size=64, vamana_iters=1, alpha=1.2, max_fraction=0.06, batch_base=2.0, queue_size=127, reverse_batchsize=1000000)

Functions#

`build`(*args[, resources])	build(IndexParams index_params, dataset, resources=None)
`save`(*args[, resources])	save(filename, Index index, bool include_dataset=True, resources=None)

Package Contents#

class cuvs.neighbors.vamana.Index#

Vamana index object. This object stores the trained Vamana index state which can be used to perform nearest neighbors searches.

static __reduce__(*args, **kwargs)#: Index.__reduce_cython__(self)

static __repr__(*args, **kwargs)#

static __setstate__(*args, **kwargs)#: Index.__setstate_cython__(self, __pyx_state)

class cuvs.neighbors.vamana.IndexParams#

IndexParams(metric=u’sqeuclidean’, *, graph_degree=32, visited_size=64, vamana_iters=1, alpha=1.2, max_fraction=0.06, batch_base=2.0, queue_size=127, reverse_batchsize=1000000)

Parameters for building a Vamana index

metricstr, default=”sqeuclidean”
String denoting the metric type. Supported metrics include: - “sqeuclidean” - “l2”

graph_degreeint, default=32
Maximum degree of graph; corresponds to the R parameter of Vamana algorithm in the literature.

visited_sizeint, default=64
Maximum number of visited nodes per search during Vamana algorithm. Loosely corresponds to the L parameter in the literature.

vamana_itersfloat, default=1
Number of Vamana vector insertion iterations (each iteration inserts all vectors).

alphafloat, default=1.2
Alpha for pruning parameter. Used to determine how aggressive the pruning will be.

max_fractionfloat, default=0.06
Maximum fraction of dataset inserted per batch. Larger max batch decreases graph quality, but improves speed.

batch_basefloat, default=2.0
Base of growth rate of batch sizes.

queue_sizeint, default=127
Size of candidate queue structure - should be (2^x)-1.

reverse_batchsizeint, default=1000000
Max batchsize of reverse edge processing (reduces memory footprint).

static __reduce__(*args, **kwargs)#: IndexParams.__reduce_cython__(self)

static __setstate__(*args, **kwargs)#: IndexParams.__setstate_cython__(self, __pyx_state)

cuvs.neighbors.vamana.build(*args, resources=None, **kwargs)#

build(IndexParams index_params, dataset, resources=None)

Build the Vamana index from the dataset for efficient search.

The build utilities the Vamana insertion-based algorithm to create the graph. The algorithm starts with an empty graph and iteratively inserts batches of nodes. Each batch involves performing a greedy search for each vector to be inserted, and inserting it with edges to all nodes traversed during the search. Reverse edges are also inserted and robustPrune is applied to improve graph quality. The index_params struct controls the degree of the final graph.

The following distance metrics are supported:

L2Expanded

index_params : IndexParams object dataset : CUDA array interface compliant matrix shape (n_samples, dim)

Supported dtype [float32, int8, uint8]

resourcesOptional cuVS Resource handle for reusing CUDA resources.
If Resources aren’t supplied, CUDA resources will be allocated inside this function and synchronized before the function exits. If resources are supplied, you will need to explicitly synchronize yourself by calling resources.sync() before accessing the output.

index: cuvs.vamana.Index
>>> import cupy as cp
>>> from cuvs.neighbors import vamana
>>> n_samples = 50000
>>> n_features = 50
>>> dataset = cp.random.random_sample((n_samples, n_features),
...                                   dtype=cp.float32)
>>> build_params = vamana.IndexParams(
...     metric="sqeuclidean", graph_degree=64, visited_size=128)
>>> index = vamana.build(build_params, dataset)
>>> # Serialize index to file for later use with CPU DiskANN
>>> vamana.save("my_index.bin", index)

cuvs.neighbors.vamana.save(*args, resources=None, **kwargs)#

save(filename, Index index, bool include_dataset=True, resources=None)

Saves the index to a file.

Matches the file format used by the DiskANN open-source repository, allowing cross-compatibility.

filenamestring
Name of the file.

indexIndex
Trained Vamana index.

include_datasetbool
Whether or not to write out the dataset along with the index. Including the dataset in the serialized index will use extra disk space, and might not be desired if you already have a copy of the dataset on disk.

resourcesOptional cuVS Resource handle for reusing CUDA resources.
If Resources aren’t supplied, CUDA resources will be allocated inside this function and synchronized before the function exits. If resources are supplied, you will need to explicitly synchronize yourself by calling resources.sync() before accessing the output.
>>> import cupy as cp
>>> from cuvs.neighbors import vamana
>>> n_samples = 50000
>>> n_features = 50
>>> dataset = cp.random.random_sample((n_samples, n_features),
...                                   dtype=cp.float32)
>>> # Build index
>>> index = vamana.build(vamana.IndexParams(graph_degree=64,
...                                         visited_size=128), dataset)
>>> # Serialize and save the vamana index
>>> vamana.save("my_index.bin", index)