WordPieceTokenizer#
2026-02-27
19 min read time
Applies to Linux
Constructor#
|
A vocabulary object used to tokenize input text. |
|
Produces tokens for the input strings. |
2026-02-27
19 min read time
|
A vocabulary object used to tokenize input text. |
|
Produces tokens for the input strings. |