Parquet Metadata

2026-03-11

20 min read time

Applies to Linux

class pylibhipdf.io.parquet_metadata.ParquetColumnSchema

Bases: object

Schema of a parquet column, including the nested columns.

Parameters

parquet_column_schema

__init__(*args, **kwargs)

child(self, int idx) → ParquetColumnSchema

Returns schema of the child with the given index.

Parameters

idxint

Child Index

Returns

ParquetColumnSchema

Child schema

children(self) → list

Returns schemas of all child columns.

Returns

list[ParquetColumnSchema]

Child schemas.

name(self) → unicode

Returns parquet column name; can be empty.

Returns

str

Column name

num_children(self) → int

Returns the number of child columns.

Returns

int

Children count

class pylibhipdf.io.parquet_metadata.ParquetMetadata

Bases: object

Information about content of a parquet file.

Parameters

parquet_metadata

__init__(*args, **kwargs)

columnchunk_metadata(self) → dict

Returns a map of leaf column names to lists of total_uncompressed_size metadata from all column chunks in the file footer.

Returns

dict[str, list[int]]

Map of leaf column names to lists of total_uncompressed_size metadata from all their column chunks.

metadata(self) → dict

Returns the key-value metadata in the file footer.

Returns

dict[str, str]

Key value metadata as a map.

num_rowgroups(self) → int

Returns the total number of rowgroups in the file.

Returns

int

Number of row groups.

num_rowgroups_per_file(self) → list

Returns the number of rowgroups in each file.

num_rows(self) → int

Returns the number of rows of the root column.

Returns

int

Number of rows

rowgroup_metadata(self) → list

Returns the row group metadata in the file footer.

Returns

list[dict[str, int]]

Vector of row group metadata as maps.

schema(self) → ParquetSchema

Returns the parquet schema.

Returns

ParquetSchema

Parquet schema

class pylibhipdf.io.parquet_metadata.ParquetSchema

Bases: object

Schema of a parquet file.

Parameters

parquet_schema

__init__(*args, **kwargs)

root(self) → ParquetColumnSchema

Returns the schema of the struct column that contains all columns as fields.

Returns

ParquetColumnSchema

Root column schema

pylibhipdf.io.parquet_metadata.read_parquet_metadata(SourceInfo src_info) → ParquetMetadata

Reads metadata of parquet dataset.

Parameters

src_infoSourceInfo

Dataset source.

Returns

ParquetMetadata

Parquet_metadata with parquet schema, number of rows, number of row groups and key-value metadata.