SubwordTokenizer

2025-05-05

20 min read time

Applies to Linux

Constructor

SubwordTokenizer(hash_file[, do_lower_case])	Run CUDA BERT subword tokenizer on cuDF strings column.
SubwordTokenizer.__call__(text, max_length, ...)	Run CUDA BERT subword tokenizer on cuDF strings column.