JSON

2026-03-11

28 min read time

Applies to Linux

pylibhipdf.io.json.chunked_read_json(JsonReaderOptions options, int chunk_size=100000000, Stream stream=None) → tuple

Reads chunks of a JSON file into a TableWithMetadata.

Parameters

optionsJsonReaderOptions

Settings for controlling reading behavior

chunk_sizeint, default 100_000_000 bytes.

The number of bytes to be read in chunks. The chunk_size should be set to at least row_size.

streamStream | None

CUDA stream used for device memory operations and kernel launches

Returns

tuple

A tuple of (columns, column_name, child_names)

pylibhipdf.io.json.read_json(JsonReaderOptions options, Stream stream=None) → TableWithMetadata

Read from JSON format.

The source to read from and options are encapsulated by the options object.

For details, see read_json().

Parameters

options: JsonReaderOptions

Settings for controlling reading behavior

streamStream | None

CUDA stream used for device memory operations and kernel launches

Returns

TableWithMetadata

The Table and its corresponding metadata (column names) that were read in.

pylibhipdf.io.json.read_json_from_string_column(Column input, Scalar separator, Scalar narep, list dtypes=None, compression_type compression=compression_type.NONE, json_recovery_mode_t recovery_mode=json_recovery_mode_t.RECOVER_WITH_NULL, Stream stream=None) → TableWithMetadata

Joins a column of JSON strings into a device buffer and reads it into a table using the JSON reader.

The source to read from is a string column of JSON records.

For details, see join_strings() and read_json().

Parameters

input: Column

String column with json-like strings as rows

separator: Scalar

String scalar used to join the input strings

narep: Scalar

String scalar used to replace null values during join

dtypes: List

Set data types for columns to be read.

compression: CompressionType

Set compression type of the string column contents

recovery_mode: JSONRecoveryMode

Set recovery option for corrupted JSON input in string column

streamStream | None

CUDA stream used for device memory operations and kernel launches

Returns

TableWithMetadata

The Table and its corresponding metadata (column names)

pylibhipdf.io.json.write_json(JsonWriterOptions options, Stream stream=None) → void

Writes a set of columns to JSON format.

Parameters

optionsJsonWriterOptions

Settings for controlling writing behavior

stream: Stream

CUDA stream used for device memory operations and kernel launches

Returns

None

class pylibhipdf.io.json.JsonReaderOptions

Bases: object

The settings to use for read_json

For details, see :cpp:class:`cudf::io::json_reader_options

allow_nonnumeric_numbers(self, bool val) → void

allow_numeric_leading_zeros(self, bool val) → void

allow_unquoted_control_chars(self, bool val) → void

static builder(SourceInfo source)

Create a JsonReaderOptionsBuilder object

For details, see cudf::io::json_reader_options::builder()

Parameters

sinkSourceInfo

The source to read the JSON file from.

Returns

JsonReaderOptionsBuilder

Builder to build JsonReaderOptions

enable_dayfirst(self, bool val) → void

enable_experimental(self, bool val) → void

enable_keep_quotes(self, bool keep_quotes) → void

Set whether the reader should keep quotes of string values.

Parameters

keep_quotesbool

Boolean value to indicate whether the reader should keep quotes of string values

Returns

None

enable_lines(self, bool val) → void

Set whether to read the file as a json object per line.

Parameters

valbool

Boolean value to enable/disable the option to read each line as a json object

Returns

None

enable_mixed_types_as_string(self, bool mixed_types_as_string) → void

Set whether to parse mixed types as a string column. Also enables forcing to read a struct as string column using schema.

Parameters

mixed_types_as_stringbool

Boolean value to enable/disable parsing mixed types as a string column

Returns

None

enable_normalize_single_quotes(self, bool val) → void

enable_normalize_whitespace(self, bool val) → void

enable_prune_columns(self, bool prune_columns) → void

Set whether to prune columns on read, selected based on the set_dtypes option.

Parameters

prune_columnsbool

When set as true, if the reader options include set_dtypes, then the reader will only return those columns which are mentioned in set_dtypes. If false, then all columns are returned, independent of the set_dtypes setting.

Returns

None

set_byte_range_offset(self, size_t offset) → void

Set number of bytes to skip from source start.

Parameters

offsetsize_t

Number of bytes of offset

Returns

None

set_byte_range_size(self, size_t size) → void

Set number of bytes to read.

Parameters

sizesize_t

Number of bytes to read

Returns

None

set_delimiter(self, unicode val) → void

set_dtypes(self, list types) → void

Set data types for columns to be read.

Parameters

typeslist

List of dtypes or a list of tuples of column names, dtypes, and list of tuples (to support nested column hierarchy)

Returns

None

set_na_values(self, list vals) → void

set_strict_validation(self, bool val) → void

class pylibhipdf.io.json.JsonReaderOptionsBuilder

Bases: object

build(self)

Create a JsonReaderOptions object

byte_range_offset(self, size_t byte_range_offset) → JsonReaderOptionsBuilder

Set number of bytes to skip from source start.

Parameters

byte_range_offsetsize_t

Number of bytes of offset

Returns

Self

byte_range_size(self, size_t byte_range_size) → JsonReaderOptionsBuilder

Set number of bytes to read.

Parameters

byte_range_sizesize_t

Number of bytes to read

Returns

Self

compression(self, compression_type compression) → JsonReaderOptionsBuilder

Sets compression type.

Parameters

compressionCompressionType

The compression type to use

Returns

Self

dayfirst(self, bool val) → JsonReaderOptionsBuilder

Set whether the reader should parse dates as DD/MM versus MM/DD.

Parameters

valbool

Boolean value to indicate whether the reader should enable/disable DD/MM parsing

Returns

Self

delimiter(self, unicode delimiter) → JsonReaderOptionsBuilder

Set delimiter character separating records in JSON lines inputs

Parameters

delimiterstr

Character to be used as delimiter separating records

Returns

Self

dtypes(self, list types) → JsonReaderOptionsBuilder

Set data type for columns to be read

Parameters

typeslist

List of dtypes or a list of tuples of column names, dtypes, and list of tuples (to support nested column hierarchy)

Returns

Self

experimental(self, bool val) → JsonReaderOptionsBuilder

Set whether to enable experimental features. When set to true, experimental features, such as the new column tree construction, utf-8 matching of field names will be enabled.

Parameters

valbool

Boolean value to enable/disable experimental features

Returns

Self

keep_quotes(self, bool val) → JsonReaderOptionsBuilder

Set whether the reader should keep quotes of string values.

Parameters

valbool

Boolean value to indicate whether the reader should keep quotes of string values

Returns

Self

lines(self, bool val) → JsonReaderOptionsBuilder

Set whether to read the file as a json object per line.

Parameters

valbool

Boolean value to enable/disable the option to read each line as a json object

Returns

Self

mixed_types_as_string(self, bool val) → JsonReaderOptionsBuilder

Set whether to parse mixed types as a string column. Also enables forcing to read a struct as string column using schema.

Parameters

valbool

Boolean value to enable/disable parsing mixed types as a string column

Returns

Self

na_values(self, list vals) → JsonReaderOptionsBuilder

Sets additional values to recognize as null values.

Parameters

valslist

Vector of values to be considered to be null

Returns

Self

nonnumeric_numbers(self, bool val) → JsonReaderOptionsBuilder

Set whether unquoted number values should be allowed NaN, +INF, -INF, +Infinity, Infinity, and -Infinity. Strict validation must be enabled for this to work.

Parameters

valbool

Boolean value to indicate whether leading zeros are allowed in numeric values

Returns

Self

normalize_single_quotes(self, bool val) → JsonReaderOptionsBuilder

Sets whether to normalize single quotes around strings.

Parameters

valbool

Boolean value to enable/disable the option to normalize single quotes around strings

Returns

Self

normalize_whitespace(self, bool val) → JsonReaderOptionsBuilder

Sets whether to normalize unquoted whitespace characters

Parameters

valbool

Boolean value to enable/disable the option to normalize unquoted whitespace characters

Returns

Self

numeric_leading_zeros(self, bool val) → JsonReaderOptionsBuilder

Set whether leading zeros are allowed in numeric values. Strict validation must be enabled for this to work.

Parameters

valbool

Boolean value to indicate whether leading zeros are allowed in numeric values

Returns

Self

prune_columns(self, bool val) → JsonReaderOptionsBuilder

Set whether to prune columns on read, selected based on the @ref dtypes option. When set as true, if the reader options include @ref dtypes, then the reader will only return those columns which are mentioned in @ref dtypes. If false, then all columns are returned, independent of the @ref dtypes setting.

Parameters

valbool

Boolean value to enable/disable column pruning

Returns

Self

recovery_mode(self, json_recovery_mode_t recovery_mode) → JsonReaderOptionsBuilder

Specifies the JSON reader’s behavior on invalid JSON lines.

Parameters

recovery_modejson_recovery_mode_t

An enum value to indicate the JSON reader’s behavior on invalid JSON lines.

Returns

Self

strict_validation(self, bool val) → JsonReaderOptionsBuilder

Set whether strict validation is enabled or not

Parameters

valbool

Boolean value to indicate whether strict validation is to be enabled

Returns

Self

unquoted_control_chars(self, bool val) → JsonReaderOptionsBuilder

Set whether in a quoted string should characters greater than or equal to 0 and less than 32 be allowed without some form of escaping. Strict validation must be enabled for this to work.

Parameters

valbool

Boolean value to indicate whether unquoted control chars are allowed

Returns

Self

class pylibhipdf.io.json.JsonWriterOptions

Bases: object

The settings to use for write_json

For details, see cudf::io::json_writer_options

static builder(SinkInfo sink, Table table)

Create a JsonWriterOptionsBuilder object

Parameters

sinkSinkInfo

The sink used for writer output

tableTable

Table to be written to output

Returns

JsonWriterOptionsBuilder

Builder to build JsonWriterOptions

set_compression(self, compression_type comptype) → void

Sets compression type to be used

Parameters

comptypeCompressionType

Compression type for sink

Returns

None

set_false_value(self, unicode val) → void

Sets string used for values == 0

Parameters

valstr

String to represent values == 0

Returns

None

set_rows_per_chunk(self, size_type val) → void

Sets string to used for null entries.

Parameters

valsize_type

String to represent null value

Returns

None

set_true_value(self, unicode val) → void

Sets string used for values != 0

Parameters

valstr

String to represent values != 0

Returns

None

class pylibhipdf.io.json.JsonWriterOptionsBuilder

Bases: object

build(self) → JsonWriterOptions

Create a JsonWriterOptions object

compression(self, compression_type comptype) → JsonWriterOptionsBuilder

Sets compression type of output sink.

Parameters

comptypeCompressionType

Compression type used

Returns

Self

include_nulls(self, bool val) → JsonWriterOptionsBuilder

Enables/Disables output of nulls as ‘null’.

Parameters

valbool

Boolean value to enable/disable

Returns

Self

lines(self, bool val) → JsonWriterOptionsBuilder

Enables/Disables JSON lines for records format.

Parameters

valbool

Boolean value to enable/disable

Returns

Self

metadata(self, TableWithMetadata tbl_w_meta) → JsonWriterOptionsBuilder

Sets optional metadata (with column names).

Parameters

tbl_w_metaTableWithMetadata

Associated metadata

Returns

Self

na_rep(self, unicode val) → JsonWriterOptionsBuilder

Sets string to used for null entries.

Parameters

valstr

String to represent null value

Returns

Self

utf8_escaped(self, bool val) → JsonWriterOptionsBuilder

Sets whether to write UTF-8 characters in string fields without escaping them.

Parameters

valbool

If False, disables escaping of UTF-8 characters in output

Returns

Self