AMD rocAL - Data Loaders API

AMD rocAL - Data Loaders API#

rocAL: API: AMD rocAL - Data Loaders API
API: AMD rocAL - Data Loaders API

The AMD rocAL data loader functions. More...

Functions

RocalTensor ROCAL_API_CALL rocalJpegFileSource (RocalContext context, const char *source_path, RocalImageColor rocal_color_format, unsigned internal_shard_count, bool is_output, bool shuffle=false, bool loop=false, RocalImageSizeEvaluationPolicy decode_size_policy=ROCAL_USE_MOST_FREQUENT_SIZE, unsigned max_width=0, unsigned max_height=0, RocalDecoderType rocal_decoder_type=RocalDecoderType::ROCAL_DECODER_TJPEG, RocalShardingInfo rocal_sharding_info=RocalShardingInfo())
 Creates JPEG image reader and decoder. It allocates the resources and objects required to read and decode Jpeg images stored on the file systems. It has internal sharding capability to load/decode in parallel is user wants. If images are not Jpeg compressed they will be ignored. More...
 
RocalTensor ROCAL_API_CALL rocalJpegFileSourceSingleShard (RocalContext context, const char *source_path, RocalImageColor rocal_color_format, unsigned shard_id, unsigned shard_count, bool is_output, bool shuffle=false, bool loop=false, RocalImageSizeEvaluationPolicy decode_size_policy=ROCAL_USE_MOST_FREQUENT_SIZE, unsigned max_width=0, unsigned max_height=0, RocalDecoderType rocal_decoder_type=RocalDecoderType::ROCAL_DECODER_TJPEG, RocalShardingInfo rocal_sharding_info=RocalShardingInfo())
 Creates JPEG image reader and decoder. It allocates the resources and objects required to read and decode Jpeg images stored on the file systems. It accepts external sharding information to load a singe shard. only. More...
 
RocalTensor ROCAL_API_CALL rocalSequenceReader (RocalContext context, const char *source_path, RocalImageColor rocal_color_format, unsigned internal_shard_count, unsigned sequence_length, bool is_output, bool shuffle=false, bool loop=false, unsigned step=0, unsigned stride=0, RocalShardingInfo rocal_sharding_info=RocalShardingInfo())
 Creates JPEG image reader and decoder. Reads [Frames] sequences from a directory representing a collection of streams. More...
 
RocalTensor ROCAL_API_CALL rocalSequenceReaderSingleShard (RocalContext context, const char *source_path, RocalImageColor rocal_color_format, unsigned shard_id, unsigned shard_count, unsigned sequence_length, bool is_output, bool shuffle=false, bool loop=false, unsigned step=0, unsigned stride=0, RocalShardingInfo rocal_sharding_info=RocalShardingInfo())
 Creates JPEG image reader and decoder. Reads [Frames] sequences from a directory representing a collection of streams. It accepts external sharding information to load a singe shard only. More...
 
RocalTensor ROCAL_API_CALL rocalJpegCOCOFileSource (RocalContext context, const char *source_path, const char *json_path, RocalImageColor color_format, unsigned internal_shard_count, bool is_output, bool shuffle=false, bool loop=false, RocalImageSizeEvaluationPolicy decode_size_policy=ROCAL_USE_MOST_FREQUENT_SIZE, unsigned max_width=0, unsigned max_height=0, RocalDecoderType rocal_decoder_type=RocalDecoderType::ROCAL_DECODER_TJPEG, RocalShardingInfo rocal_sharding_info=RocalShardingInfo())
 JPEG image reader and decoder. It allocates the resources and objects required to read and decode COCO Jpeg images stored on the file systems. It has internal sharding capability to load/decode in parallel is user wants. If images are not Jpeg compressed they will be ignored. More...
 
RocalTensor ROCAL_API_CALL rocalJpegCOCOFileSourcePartial (RocalContext p_context, const char *source_path, const char *json_path, RocalImageColor rocal_color_format, unsigned internal_shard_count, bool is_output, std::vector< float > &area_factor, std::vector< float > &aspect_ratio, unsigned num_attempts, bool shuffle=false, bool loop=false, RocalImageSizeEvaluationPolicy decode_size_policy=ROCAL_USE_MOST_FREQUENT_SIZE, unsigned max_width=0, unsigned max_height=0, RocalShardingInfo rocal_sharding_info=RocalShardingInfo())
 JPEG image reader and partial decoder. It allocates the resources and objects required to read and decode COCO Jpeg images stored on the file systems. It has internal sharding capability to load/decode in parallel is user wants. If images are not Jpeg compressed they will be ignored. More...
 
RocalTensor ROCAL_API_CALL rocalJpegCOCOFileSourcePartialSingleShard (RocalContext p_context, const char *source_path, const char *json_path, RocalImageColor rocal_color_format, unsigned shard_id, unsigned shard_count, bool is_output, std::vector< float > &area_factor, std::vector< float > &aspect_ratio, unsigned num_attempts, bool shuffle=false, bool loop=false, RocalImageSizeEvaluationPolicy decode_size_policy=ROCAL_USE_MOST_FREQUENT_SIZE, unsigned max_width=0, unsigned max_height=0, RocalShardingInfo rocal_sharding_info=RocalShardingInfo())
 Creates JPEG image reader and partial decoder. It allocates the resources and objects required to read and decode COCO Jpeg images stored on the file systems. It has internal sharding capability to load/decode in parallel is user wants. If images are not Jpeg compressed they will be ignored. More...
 
RocalTensor ROCAL_API_CALL rocalJpegCOCOFileSourceSingleShard (RocalContext context, const char *source_path, const char *json_path, RocalImageColor color_format, unsigned shard_id, unsigned shard_count, bool is_output, bool shuffle=false, bool loop=false, RocalImageSizeEvaluationPolicy decode_size_policy=ROCAL_USE_MOST_FREQUENT_SIZE, unsigned max_width=0, unsigned max_height=0, RocalDecoderType rocal_decoder_type=RocalDecoderType::ROCAL_DECODER_TJPEG, RocalShardingInfo rocal_sharding_info=RocalShardingInfo())
 Creates JPEG image reader. It allocates the resources and objects required to read and decode COCO Jpeg images stored on the file systems. It has internal sharding capability to load/decode in parallel is user wants. If images are not Jpeg compressed they will be ignored. More...
 
RocalTensor ROCAL_API_CALL rocalJpegCaffeLMDBRecordSource (RocalContext context, const char *source_path, RocalImageColor rocal_color_format, unsigned internal_shard_count, bool is_output, bool shuffle=false, bool loop=false, RocalImageSizeEvaluationPolicy decode_size_policy=ROCAL_USE_MOST_FREQUENT_SIZE, unsigned max_width=0, unsigned max_height=0, RocalDecoderType rocal_decoder_type=RocalDecoderType::ROCAL_DECODER_TJPEG, RocalShardingInfo rocal_sharding_info=RocalShardingInfo())
 Creates JPEG image reader and decoder for Caffe LMDB records. It allocates the resources and objects required to read and decode Jpeg images stored in Caffe LMDB Records. It has internal sharding capability to load/decode in parallel is user wants. If images are not Jpeg compressed they will be ignored. More...
 
RocalTensor ROCAL_API_CALL rocalJpegCaffeLMDBRecordSourceSingleShard (RocalContext p_context, const char *source_path, RocalImageColor rocal_color_format, unsigned shard_id, unsigned shard_count, bool is_output, bool shuffle=false, bool loop=false, RocalImageSizeEvaluationPolicy decode_size_policy=ROCAL_USE_MOST_FREQUENT_SIZE, unsigned max_width=0, unsigned max_height=0, RocalDecoderType rocal_decoder_type=RocalDecoderType::ROCAL_DECODER_TJPEG, RocalShardingInfo rocal_sharding_info=RocalShardingInfo())
 Creates JPEG image reader and decoder for Caffe LMDB records. It allocates the resources and objects required to read and decode Jpeg images stored in Caffe2 LMDB Records. It has internal sharding capability to load/decode in parallel is user wants. More...
 
RocalTensor ROCAL_API_CALL rocalJpegCaffe2LMDBRecordSource (RocalContext context, const char *source_path, RocalImageColor rocal_color_format, unsigned internal_shard_count, bool is_output, bool shuffle=false, bool loop=false, RocalImageSizeEvaluationPolicy decode_size_policy=ROCAL_USE_MOST_FREQUENT_SIZE, unsigned max_width=0, unsigned max_height=0, RocalDecoderType rocal_decoder_type=RocalDecoderType::ROCAL_DECODER_TJPEG, RocalShardingInfo rocal_sharding_info=RocalShardingInfo())
 Creates JPEG image reader and decoder for Caffe2 LMDB records. It allocates the resources and objects required to read and decode Jpeg images stored in Caffe2 LMDB Records. It has internal sharding capability to load/decode in parallel is user wants. If images are not Jpeg compressed they will be ignored. More...
 
RocalTensor ROCAL_API_CALL rocalJpegCaffe2LMDBRecordSourceSingleShard (RocalContext p_context, const char *source_path, RocalImageColor rocal_color_format, unsigned shard_id, unsigned shard_count, bool is_output, bool shuffle=false, bool loop=false, RocalImageSizeEvaluationPolicy decode_size_policy=ROCAL_USE_MOST_FREQUENT_SIZE, unsigned max_width=0, unsigned max_height=0, RocalDecoderType rocal_decoder_type=RocalDecoderType::ROCAL_DECODER_TJPEG, RocalShardingInfo rocal_sharding_info=RocalShardingInfo())
 Creates JPEG image reader and decoder for Caffe2 LMDB records. It allocates the resources and objects required to read and decode Jpeg images stored on the Caffe2 LMDB Records. It accepts external sharding information to load a singe shard. only. More...
 
RocalTensor ROCAL_API_CALL rocalMXNetRecordSource (RocalContext context, const char *source_path, RocalImageColor rocal_color_format, unsigned internal_shard_count, bool is_output, bool shuffle=false, bool loop=false, RocalImageSizeEvaluationPolicy decode_size_policy=ROCAL_USE_MOST_FREQUENT_SIZE, unsigned max_width=0, unsigned max_height=0, RocalDecoderType rocal_decoder_type=RocalDecoderType::ROCAL_DECODER_TJPEG, RocalShardingInfo rocal_sharding_info=RocalShardingInfo())
 Creates JPEG image reader and decoder for MXNet records. It allocates the resources and objects required to read and decode Jpeg images stored in MXNet Records. It has internal sharding capability to load/decode in parallel is user wants. If images are not Jpeg compressed they will be ignored. More...
 
RocalTensor ROCAL_API_CALL rocalMXNetRecordSourceSingleShard (RocalContext p_context, const char *source_path, RocalImageColor rocal_color_format, unsigned shard_id, unsigned shard_count, bool is_output, bool shuffle=false, bool loop=false, RocalImageSizeEvaluationPolicy decode_size_policy=ROCAL_USE_MOST_FREQUENT_SIZE, unsigned max_width=0, unsigned max_height=0, RocalDecoderType rocal_decoder_type=RocalDecoderType::ROCAL_DECODER_TJPEG, RocalShardingInfo rocal_sharding_info=RocalShardingInfo())
 Creates JPEG image reader and decoder for MXNet records. It allocates the resources and objects required to read and decode Jpeg images stored on the MXNet records. It accepts external sharding information to load a singe shard. only. More...
 
RocalTensor ROCAL_API_CALL rocalFusedJpegCrop (RocalContext context, const char *source_path, RocalImageColor rocal_color_format, unsigned num_threads, bool is_output, std::vector< float > &area_factor, std::vector< float > &aspect_ratio, unsigned num_attempts, bool shuffle=false, bool loop=false, RocalImageSizeEvaluationPolicy decode_size_policy=ROCAL_USE_MOST_FREQUENT_SIZE, unsigned max_width=0, unsigned max_height=0, RocalShardingInfo rocal_sharding_info=RocalShardingInfo())
 Creates JPEG image reader and partial decoder. It allocates the resources and objects required to read and decode Jpeg images stored on the file systems. It has internal sharding capability to load/decode in parallel is user wants. If images are not Jpeg compressed they will be ignored and Crops t. More...
 
RocalTensor ROCAL_API_CALL rocalFusedJpegCropSingleShard (RocalContext context, const char *source_path, RocalImageColor color_format, unsigned shard_id, unsigned shard_count, bool is_output, std::vector< float > &area_factor, std::vector< float > &aspect_ratio, unsigned num_attempts, bool shuffle=false, bool loop=false, RocalImageSizeEvaluationPolicy decode_size_policy=ROCAL_USE_MOST_FREQUENT_SIZE, unsigned max_width=0, unsigned max_height=0, RocalShardingInfo rocal_sharding_info=RocalShardingInfo())
 Creates JPEG image reader and partial decoder. It allocates the resources and objects required to read and decode Jpeg images stored on the file systems. It accepts external sharding information to load a singe shard. only. More...
 
RocalTensor ROCAL_API_CALL rocalJpegTFRecordSource (RocalContext context, const char *source_path, RocalImageColor rocal_color_format, unsigned internal_shard_count, bool is_output, const char *user_key_for_encoded, const char *user_key_for_filename, bool shuffle=false, bool loop=false, RocalImageSizeEvaluationPolicy decode_size_policy=ROCAL_USE_MOST_FREQUENT_SIZE, unsigned max_width=0, unsigned max_height=0, RocalDecoderType rocal_decoder_type=RocalDecoderType::ROCAL_DECODER_TJPEG, RocalShardingInfo rocal_sharding_info=RocalShardingInfo())
 Creates TensorFlow records JPEG image reader and decoder. It allocates the resources and objects required to read and decode Jpeg images stored on the file systems. It has internal sharding capability to load/decode in parallel is user wants. If images are not Jpeg compressed they will be ignored. More...
 
RocalTensor ROCAL_API_CALL rocalJpegTFRecordSourceSingleShard (RocalContext context, const char *source_path, RocalImageColor rocal_color_format, unsigned shard_id, unsigned shard_count, bool is_output, bool shuffle=false, bool loop=false, RocalImageSizeEvaluationPolicy decode_size_policy=ROCAL_USE_MOST_FREQUENT_SIZE, unsigned max_width=0, unsigned max_height=0, RocalDecoderType rocal_decoder_type=RocalDecoderType::ROCAL_DECODER_TJPEG, RocalShardingInfo rocal_sharding_info=RocalShardingInfo())
 Creates TensorFlow records JPEG image reader and decoder. It allocates the resources and objects required to read and decode Jpeg images stored on the file systems. It accepts external sharding information to load a singe shard. only. More...
 
RocalTensor ROCAL_API_CALL rocalRawTFRecordSource (RocalContext p_context, const char *source_path, const char *user_key_for_raw, const char *user_key_for_filename, RocalImageColor rocal_color_format, bool is_output, bool shuffle=false, bool loop=false, unsigned out_width=0, unsigned out_height=0, const char *record_name_prefix="", RocalShardingInfo rocal_sharding_info=RocalShardingInfo())
 Creates Raw image loader. It allocates the resources and objects required to load images stored on the file systems. More...
 
RocalTensor ROCAL_API_CALL rocalRawTFRecordSourceSingleShard (RocalContext p_context, const char *source_path, RocalImageColor rocal_color_format, unsigned shard_id, unsigned shard_count, bool is_output, bool shuffle=false, bool loop=false, unsigned out_width=0, unsigned out_height=0, const char *record_name_prefix="", RocalShardingInfo rocal_sharding_info=RocalShardingInfo())
 Creates Raw image loader. It allocates the resources and objects required to load images stored on the file systems. More...
 
RocalTensor ROCAL_API_CALL rocalVideoFileSource (RocalContext context, const char *source_path, RocalImageColor color_format, RocalDecodeDevice rocal_decode_device, unsigned internal_shard_count, unsigned sequence_length, bool is_output=false, bool shuffle=false, bool loop=false, unsigned step=0, unsigned stride=0, bool file_list_frame_num=true, RocalShardingInfo rocal_sharding_info=RocalShardingInfo())
 Creates a video reader and decoder as a source. It allocates the resources and objects required to read and decode mp4 videos stored on the file systems. More...
 
RocalTensor ROCAL_API_CALL rocalVideoFileSourceSingleShard (RocalContext context, const char *source_path, RocalImageColor color_format, RocalDecodeDevice rocal_decode_device, unsigned shard_id, unsigned shard_count, unsigned sequence_length, bool shuffle=false, bool is_output=false, bool loop=false, unsigned step=0, unsigned stride=0, bool file_list_frame_num=true, RocalShardingInfo rocal_sharding_info=RocalShardingInfo())
 Creates a video reader and decoder as a source. It allocates the resources and objects required to read and decode mp4 videos stored on the file systems. It accepts external sharding information to load a singe shard only. More...
 
RocalTensor ROCAL_API_CALL rocalVideoFileResize (RocalContext context, const char *source_path, RocalImageColor color_format, RocalDecodeDevice rocal_decode_device, unsigned internal_shard_count, unsigned sequence_length, unsigned dest_width, unsigned dest_height, bool shuffle=false, bool is_output=false, bool loop=false, unsigned step=0, unsigned stride=0, bool file_list_frame_num=true, RocalResizeScalingMode scaling_mode=ROCAL_SCALING_MODE_DEFAULT, std::vector< unsigned > max_size={}, unsigned resize_shorter=0, unsigned resize_longer=0, RocalResizeInterpolationType interpolation_type=ROCAL_LINEAR_INTERPOLATION, RocalShardingInfo rocal_sharding_info=RocalShardingInfo())
 Creates a video reader and decoder as a source. It allocates the resources and objects required to read and decode mp4 videos stored on the file systems. Resizes the decoded frames to the dest width and height. More...
 
RocalTensor ROCAL_API_CALL rocalVideoFileResizeSingleShard (RocalContext context, const char *source_path, RocalImageColor color_format, RocalDecodeDevice rocal_decode_device, unsigned shard_id, unsigned shard_count, unsigned sequence_length, unsigned dest_width, unsigned dest_height, bool shuffle=false, bool is_output=false, bool loop=false, unsigned step=0, unsigned stride=0, bool file_list_frame_num=true, RocalResizeScalingMode scaling_mode=ROCAL_SCALING_MODE_DEFAULT, std::vector< unsigned > max_size={}, unsigned resize_shorter=0, unsigned resize_longer=0, RocalResizeInterpolationType interpolation_type=ROCAL_LINEAR_INTERPOLATION, RocalShardingInfo rocal_sharding_info=RocalShardingInfo())
 Creates a video reader and decoder as a source. It allocates the resources and objects required to read and decode mp4 videos stored on the file systems. Resizes the decoded frames to the dest width and height. It accepts external sharding information to load a singe shard only. More...
 
RocalTensor ROCAL_API_CALL rocalRawCIFAR10Source (RocalContext context, const char *source_path, RocalImageColor color_format, bool is_output, unsigned out_width, unsigned out_height, const char *filename_prefix="", bool loop=false)
 Creates CIFAR10 raw data reader and loader. It allocates the resources and objects required to read raw data stored on the file systems. More...
 
RocalStatus ROCAL_API_CALL rocalResetLoaders (RocalContext context)
 reset Loaders More...
 
RocalTensor ROCAL_API_CALL rocalJpegCaffeLMDBRecordSourcePartialSingleShard (RocalContext p_context, const char *source_path, RocalImageColor rocal_color_format, unsigned shard_id, unsigned shard_count, bool is_output, std::vector< float > &area_factor, std::vector< float > &aspect_ratio, unsigned num_attempts, bool shuffle=false, bool loop=false, RocalImageSizeEvaluationPolicy decode_size_policy=ROCAL_USE_MOST_FREQUENT_SIZE, unsigned max_width=0, unsigned max_height=0, RocalShardingInfo rocal_sharding_info=RocalShardingInfo())
 Creates JPEG image reader and partial decoder for Caffe LMDB records. It allocates the resources and objects required to read and decode Jpeg images stored in Caffe2 LMDB Records. It has internal sharding capability to load/decode in parallel is user wants. More...
 
RocalTensor ROCAL_API_CALL rocalJpegCaffe2LMDBRecordSourcePartialSingleShard (RocalContext p_context, const char *source_path, RocalImageColor rocal_color_format, unsigned shard_id, unsigned shard_count, bool is_output, std::vector< float > &area_factor, std::vector< float > &aspect_ratio, unsigned num_attempts, bool shuffle=false, bool loop=false, RocalImageSizeEvaluationPolicy decode_size_policy=ROCAL_USE_MOST_FREQUENT_SIZE, unsigned max_width=0, unsigned max_height=0, RocalShardingInfo rocal_sharding_info=RocalShardingInfo())
 Creates JPEG image reader and partial decoder for Caffe2 LMDB records. It allocates the resources and objects required to read and decode Jpeg images stored in Caffe22 LMDB Records. It has internal sharding capability to load/decode in parallel is user wants. More...
 
RocalTensor ROCAL_API_CALL rocalJpegExternalFileSource (RocalContext p_context, RocalImageColor rocal_color_format, bool is_output=false, bool shuffle=false, bool loop=false, RocalImageSizeEvaluationPolicy decode_size_policy=ROCAL_USE_MOST_FREQUENT_SIZE, unsigned max_width=0, unsigned max_height=0, RocalDecoderType rocal_decoder_type=RocalDecoderType::ROCAL_DECODER_TJPEG, RocalExternalSourceMode external_source_mode=RocalExternalSourceMode::ROCAL_EXTSOURCE_FNAME, RocalShardingInfo rocal_sharding_info=RocalShardingInfo())
 Creates JPEG external source image reader. More...
 

Detailed Description

The AMD rocAL data loader functions.

Function Documentation

◆ rocalFusedJpegCrop()

RocalTensor ROCAL_API_CALL rocalFusedJpegCrop ( RocalContext  context,
const char *  source_path,
RocalImageColor  rocal_color_format,
unsigned  num_threads,
bool  is_output,
std::vector< float > &  area_factor,
std::vector< float > &  aspect_ratio,
unsigned  num_attempts,
bool  shuffle = false,
bool  loop = false,
RocalImageSizeEvaluationPolicy  decode_size_policy = ROCAL_USE_MOST_FREQUENT_SIZE,
unsigned  max_width = 0,
unsigned  max_height = 0,
RocalShardingInfo  rocal_sharding_info = RocalShardingInfo() 
)

Creates JPEG image reader and partial decoder. It allocates the resources and objects required to read and decode Jpeg images stored on the file systems. It has internal sharding capability to load/decode in parallel is user wants. If images are not Jpeg compressed they will be ignored and Crops t.

Parameters
[in]contextRocal context
[in]source_pathA NULL terminated char string pointing to the location on the disk
[in]rocal_color_formatThe color format the images will be decoded to.
[in]num_threadsDefines the parallelism level by internally sharding the input dataset and load/decode using multiple decoder/loader instances. Using shard counts bigger than 1 improves the load/decode performance if compute resources (CPU cores) are available.
[in]is_outputDetermines if the user wants the loaded images to be part of the output or not.
[in]area_factorDetermines how much area to be cropped. Ranges from from 0.08 - 1.
[in]aspect_ratioDetermines the aspect ration of crop. Ranges from 0.75 to 1.33.
[in]num_attemptsMaximum number of attempts to generate crop. Default 10
[in]shuffleDetermines if the user wants to shuffle the dataset or not.
[in]loopDetermines if the user wants to indefinitely loops through images or not.
[in]decode_size_policy
[in]max_widthThe maximum width of the decoded images, larger or smaller will be resized to closest
[in]max_heightThe maximum height of the decoded images, larger or smaller will be resized to closest
[in]rocal_sharding_infoThe members of RocalShardingInfo determines how the data is distributed among the shards and how the last batch is processed by the pipeline.
Returns
Reference to the output tensor

◆ rocalFusedJpegCropSingleShard()

RocalTensor ROCAL_API_CALL rocalFusedJpegCropSingleShard ( RocalContext  context,
const char *  source_path,
RocalImageColor  color_format,
unsigned  shard_id,
unsigned  shard_count,
bool  is_output,
std::vector< float > &  area_factor,
std::vector< float > &  aspect_ratio,
unsigned  num_attempts,
bool  shuffle = false,
bool  loop = false,
RocalImageSizeEvaluationPolicy  decode_size_policy = ROCAL_USE_MOST_FREQUENT_SIZE,
unsigned  max_width = 0,
unsigned  max_height = 0,
RocalShardingInfo  rocal_sharding_info = RocalShardingInfo() 
)

Creates JPEG image reader and partial decoder. It allocates the resources and objects required to read and decode Jpeg images stored on the file systems. It accepts external sharding information to load a singe shard. only.

Parameters
[in]contextRocal context
[in]source_pathA NULL terminated char string pointing to the location on the disk
[in]rocal_color_formatThe color format the images will be decoded to.
[in]shard_idShard id for this loader
[in]shard_countTotal shard count
[in]is_outputDetermines if the user wants the loaded images to be part of the output or not.
[in]area_factorDetermines how much area to be cropped. Ranges from from 0.08 - 1.
[in]aspect_ratioDetermines the aspect ration of crop. Ranges from 0.75 to 1.33.
[in]num_attemptsMaximum number of attempts to generate crop. Default 10
[in]decode_size_policy
[in]max_widthThe maximum width of the decoded images, larger or smaller will be resized to closest
[in]max_heightThe maximum height of the decoded images, larger or smaller will be resized to closest
[in]rocal_sharding_infoThe members of RocalShardingInfo determines how the data is distributed among the shards and how the last batch is processed by the pipeline.
Returns
Reference to the output tensor

◆ rocalJpegCaffe2LMDBRecordSource()

RocalTensor ROCAL_API_CALL rocalJpegCaffe2LMDBRecordSource ( RocalContext  context,
const char *  source_path,
RocalImageColor  rocal_color_format,
unsigned  internal_shard_count,
bool  is_output,
bool  shuffle = false,
bool  loop = false,
RocalImageSizeEvaluationPolicy  decode_size_policy = ROCAL_USE_MOST_FREQUENT_SIZE,
unsigned  max_width = 0,
unsigned  max_height = 0,
RocalDecoderType  rocal_decoder_type = RocalDecoderType::ROCAL_DECODER_TJPEG,
RocalShardingInfo  rocal_sharding_info = RocalShardingInfo() 
)

Creates JPEG image reader and decoder for Caffe2 LMDB records. It allocates the resources and objects required to read and decode Jpeg images stored in Caffe2 LMDB Records. It has internal sharding capability to load/decode in parallel is user wants. If images are not Jpeg compressed they will be ignored.

Parameters
[in]contextRocal context
[in]source_pathA NULL terminated char string pointing to the location on the disk
[in]rocal_color_formatThe color format the images will be decoded to.
[in]internal_shard_countDefines the parallelism level by internally sharding the input dataset and load/decode using multiple decoder/loader instances. Using shard counts bigger than 1 improves the load/decode performance if compute resources (CPU cores) are available.
[in]is_outputDetermines if the user wants the loaded images to be part of the output or not.
[in]shuffleDetermines if the user wants to shuffle the dataset or not.
[in]loopDetermines if the user wants to indefinitely loops through images or not.
[in]decode_size_policy
[in]max_widthThe maximum width of the decoded images, larger or smaller will be resized to closest
[in]max_heightThe maximum height of the decoded images, larger or smaller will be resized to closest
[in]rocal_decoder_typeDetermines the decoder_type, tjpeg or hwdec
[in]rocal_sharding_infoThe members of RocalShardingInfo determines how the data is distributed among the shards and how the last batch is processed by the pipeline.
Returns
Reference to the output tensor

◆ rocalJpegCaffe2LMDBRecordSourcePartialSingleShard()

RocalTensor ROCAL_API_CALL rocalJpegCaffe2LMDBRecordSourcePartialSingleShard ( RocalContext  p_context,
const char *  source_path,
RocalImageColor  rocal_color_format,
unsigned  shard_id,
unsigned  shard_count,
bool  is_output,
std::vector< float > &  area_factor,
std::vector< float > &  aspect_ratio,
unsigned  num_attempts,
bool  shuffle = false,
bool  loop = false,
RocalImageSizeEvaluationPolicy  decode_size_policy = ROCAL_USE_MOST_FREQUENT_SIZE,
unsigned  max_width = 0,
unsigned  max_height = 0,
RocalShardingInfo  rocal_sharding_info = RocalShardingInfo() 
)

Creates JPEG image reader and partial decoder for Caffe2 LMDB records. It allocates the resources and objects required to read and decode Jpeg images stored in Caffe22 LMDB Records. It has internal sharding capability to load/decode in parallel is user wants.

Parameters
[in]rocal_contextRocal context
[in]source_pathA NULL terminated char string pointing to the location on the disk
[in]rocal_color_formatThe color format the images will be decoded to.
[in]shard_idShard id for this loader
[in]shard_countTotal shard count
[in]is_outputDetermines if the user wants the loaded images to be part of the output or not.
[in]shuffleDetermines if the user wants to shuffle the dataset or not.
[in]loopDetermines if the user wants to indefinitely loops through images or not.
[in]decode_size_policy
[in]max_widthThe maximum width of the decoded images, larger or smaller will be resized to closest
[in]max_heightThe maximum height of the decoded images, larger or smaller will be resized to closest
[in]rocal_sharding_infoThe members of RocalShardingInfo determines how the data is distributed among the shards and how the last batch is processed by the pipeline.
Returns
Reference to the output tensor

◆ rocalJpegCaffe2LMDBRecordSourceSingleShard()

RocalTensor ROCAL_API_CALL rocalJpegCaffe2LMDBRecordSourceSingleShard ( RocalContext  p_context,
const char *  source_path,
RocalImageColor  rocal_color_format,
unsigned  shard_id,
unsigned  shard_count,
bool  is_output,
bool  shuffle = false,
bool  loop = false,
RocalImageSizeEvaluationPolicy  decode_size_policy = ROCAL_USE_MOST_FREQUENT_SIZE,
unsigned  max_width = 0,
unsigned  max_height = 0,
RocalDecoderType  rocal_decoder_type = RocalDecoderType::ROCAL_DECODER_TJPEG,
RocalShardingInfo  rocal_sharding_info = RocalShardingInfo() 
)

Creates JPEG image reader and decoder for Caffe2 LMDB records. It allocates the resources and objects required to read and decode Jpeg images stored on the Caffe2 LMDB Records. It accepts external sharding information to load a singe shard. only.

Parameters
[in]p_contextRocal context
[in]source_pathA NULL terminated char string pointing to the location on the disk
[in]rocal_color_formatThe color format the images will be decoded to.
[in]shard_idShard id for this loader
[in]shard_countTotal shard count
[in]is_outputDetermines if the user wants the loaded images to be part of the output or not.
[in]shuffleDetermines if the user wants to shuffle the dataset or not.
[in]loopDetermines if the user wants to indefinitely loops through images or not.
[in]decode_size_policy
[in]max_widthThe maximum width of the decoded images, larger or smaller will be resized to closest
[in]max_heightThe maximum height of the decoded images, larger or smaller will be resized to closest
[in]rocal_decoder_typeDetermines the decoder_type, tjpeg or hwdec
[in]rocal_sharding_infoThe members of RocalShardingInfo determines how the data is distributed among the shards and how the last batch is processed by the pipeline.
Returns
Reference to the output tensor

◆ rocalJpegCaffeLMDBRecordSource()

RocalTensor ROCAL_API_CALL rocalJpegCaffeLMDBRecordSource ( RocalContext  context,
const char *  source_path,
RocalImageColor  rocal_color_format,
unsigned  internal_shard_count,
bool  is_output,
bool  shuffle = false,
bool  loop = false,
RocalImageSizeEvaluationPolicy  decode_size_policy = ROCAL_USE_MOST_FREQUENT_SIZE,
unsigned  max_width = 0,
unsigned  max_height = 0,
RocalDecoderType  rocal_decoder_type = RocalDecoderType::ROCAL_DECODER_TJPEG,
RocalShardingInfo  rocal_sharding_info = RocalShardingInfo() 
)

Creates JPEG image reader and decoder for Caffe LMDB records. It allocates the resources and objects required to read and decode Jpeg images stored in Caffe LMDB Records. It has internal sharding capability to load/decode in parallel is user wants. If images are not Jpeg compressed they will be ignored.

Parameters
[in]contextRocal context
[in]source_pathA NULL terminated char string pointing to the location on the disk
[in]rocal_color_formatThe color format the images will be decoded to.
[in]internal_shard_countDefines the parallelism level by internally sharding the input dataset and load/decode using multiple decoder/loader instances. Using shard counts bigger than 1 improves the load/decode performance if compute resources (CPU cores) are available.
[in]is_outputDetermines if the user wants the loaded images to be part of the output or not.
[in]shuffleDetermines if the user wants to shuffle the dataset or not.
[in]loopDetermines if the user wants to indefinitely loops through images or not.
[in]decode_size_policy
[in]max_widthThe maximum width of the decoded images, larger or smaller will be resized to closest
[in]max_heightThe maximum height of the decoded images, larger or smaller will be resized to closest
[in]rocal_sharding_infoThe members of RocalShardingInfo determines how the data is distributed among the shards and how the last batch is processed by the pipeline.
Returns
Reference to the output tensor

◆ rocalJpegCaffeLMDBRecordSourcePartialSingleShard()

RocalTensor ROCAL_API_CALL rocalJpegCaffeLMDBRecordSourcePartialSingleShard ( RocalContext  p_context,
const char *  source_path,
RocalImageColor  rocal_color_format,
unsigned  shard_id,
unsigned  shard_count,
bool  is_output,
std::vector< float > &  area_factor,
std::vector< float > &  aspect_ratio,
unsigned  num_attempts,
bool  shuffle = false,
bool  loop = false,
RocalImageSizeEvaluationPolicy  decode_size_policy = ROCAL_USE_MOST_FREQUENT_SIZE,
unsigned  max_width = 0,
unsigned  max_height = 0,
RocalShardingInfo  rocal_sharding_info = RocalShardingInfo() 
)

Creates JPEG image reader and partial decoder for Caffe LMDB records. It allocates the resources and objects required to read and decode Jpeg images stored in Caffe2 LMDB Records. It has internal sharding capability to load/decode in parallel is user wants.

Parameters
[in]rocal_contextRocal context
[in]source_pathA NULL terminated char string pointing to the location on the disk
[in]rocal_color_formatThe color format the images will be decoded to.
[in]shard_idShard id for this loader
[in]shard_countTotal shard count
[in]is_outputDetermines if the user wants the loaded images to be part of the output or not.
[in]area_factorDetermines how much area to be cropped. Ranges from from 0.08 - 1.
[in]aspect_ratioDetermines the aspect ration of crop. Ranges from 0.75 to 1.33.
[in]num_attemptsMaximum number of attempts to generate crop. Default 10
[in]shuffleDetermines if the user wants to shuffle the dataset or not.
[in]loopDetermines if the user wants to indefinitely loops through images or not.
[in]decode_size_policy
[in]max_widthThe maximum width of the decoded images, larger or smaller will be resized to closest
[in]max_heightThe maximum height of the decoded images, larger or smaller will be resized to closest
[in]rocal_sharding_infoThe members of RocalShardingInfo determines how the data is distributed among the shards and how the last batch is processed by the pipeline.
Returns
Reference to the output tensor

◆ rocalJpegCaffeLMDBRecordSourceSingleShard()

RocalTensor ROCAL_API_CALL rocalJpegCaffeLMDBRecordSourceSingleShard ( RocalContext  p_context,
const char *  source_path,
RocalImageColor  rocal_color_format,
unsigned  shard_id,
unsigned  shard_count,
bool  is_output,
bool  shuffle = false,
bool  loop = false,
RocalImageSizeEvaluationPolicy  decode_size_policy = ROCAL_USE_MOST_FREQUENT_SIZE,
unsigned  max_width = 0,
unsigned  max_height = 0,
RocalDecoderType  rocal_decoder_type = RocalDecoderType::ROCAL_DECODER_TJPEG,
RocalShardingInfo  rocal_sharding_info = RocalShardingInfo() 
)

Creates JPEG image reader and decoder for Caffe LMDB records. It allocates the resources and objects required to read and decode Jpeg images stored in Caffe2 LMDB Records. It has internal sharding capability to load/decode in parallel is user wants.

Parameters
[in]rocal_contextRocal context
[in]source_pathA NULL terminated char string pointing to the location on the disk
[in]rocal_color_formatThe color format the images will be decoded to.
[in]shard_idShard id for this loader
[in]shard_countTotal shard count
[in]is_outputDetermines if the user wants the loaded images to be part of the output or not.
[in]shuffleDetermines if the user wants to shuffle the dataset or not.
[in]loopDetermines if the user wants to indefinitely loops through images or not.
[in]decode_size_policy
[in]max_widthThe maximum width of the decoded images, larger or smaller will be resized to closest
[in]max_heightThe maximum height of the decoded images, larger or smaller will be resized to closest
[in]rocal_decoder_typeDetermines the decoder_type, tjpeg or hwdec
[in]rocal_sharding_infoThe members of RocalShardingInfo determines how the data is distributed among the shards and how the last batch is processed by the pipeline.
Returns
Reference to the output tensor

◆ rocalJpegCOCOFileSource()

RocalTensor ROCAL_API_CALL rocalJpegCOCOFileSource ( RocalContext  context,
const char *  source_path,
const char *  json_path,
RocalImageColor  color_format,
unsigned  internal_shard_count,
bool  is_output,
bool  shuffle = false,
bool  loop = false,
RocalImageSizeEvaluationPolicy  decode_size_policy = ROCAL_USE_MOST_FREQUENT_SIZE,
unsigned  max_width = 0,
unsigned  max_height = 0,
RocalDecoderType  rocal_decoder_type = RocalDecoderType::ROCAL_DECODER_TJPEG,
RocalShardingInfo  rocal_sharding_info = RocalShardingInfo() 
)

JPEG image reader and decoder. It allocates the resources and objects required to read and decode COCO Jpeg images stored on the file systems. It has internal sharding capability to load/decode in parallel is user wants. If images are not Jpeg compressed they will be ignored.

Parameters
[in]rocal_contextRocal context
[in]source_pathA NULL terminated char string pointing to the location on the disk
[in]json_pathPath to the COCO Json File
[in]rocal_color_formatThe color format the images will be decoded to.
[in]shard_countDefines the parallelism level by internally sharding the input dataset and load/decode using multiple decoder/loader instances. Using shard counts bigger than 1 improves the load/decode performance if compute resources (CPU cores) are available.
[in]is_outputDetermines if the user wants the loaded images to be part of the output or not.
[in]decode_size_policy
[in]max_widthThe maximum width of the decoded images, larger or smaller will be resized to closest
[in]max_heightThe maximum height of the decoded images, larger or smaller will be resized to closest
[in]rocal_decoder_typeDetermines the decoder_type, tjpeg or hwdec
[in]rocal_sharding_infoThe members of RocalShardingInfo determines how the data is distributed among the shards and how the last batch is processed by the pipeline.
Returns
Reference to the output tensor

◆ rocalJpegCOCOFileSourcePartial()

RocalTensor ROCAL_API_CALL rocalJpegCOCOFileSourcePartial ( RocalContext  p_context,
const char *  source_path,
const char *  json_path,
RocalImageColor  rocal_color_format,
unsigned  internal_shard_count,
bool  is_output,
std::vector< float > &  area_factor,
std::vector< float > &  aspect_ratio,
unsigned  num_attempts,
bool  shuffle = false,
bool  loop = false,
RocalImageSizeEvaluationPolicy  decode_size_policy = ROCAL_USE_MOST_FREQUENT_SIZE,
unsigned  max_width = 0,
unsigned  max_height = 0,
RocalShardingInfo  rocal_sharding_info = RocalShardingInfo() 
)

JPEG image reader and partial decoder. It allocates the resources and objects required to read and decode COCO Jpeg images stored on the file systems. It has internal sharding capability to load/decode in parallel is user wants. If images are not Jpeg compressed they will be ignored.

Parameters
[in]rocal_contextRocal context
[in]source_pathA NULL terminated char string pointing to the location on the disk
[in]json_pathPath to the COCO Json File
[in]rocal_color_formatThe color format the images will be decoded to.
[in]shard_countDefines the parallelism level by internally sharding the input dataset and load/decode using multiple decoder/loader instances. Using shard counts bigger than 1 improves the load/decode performance if compute resources (CPU cores) are available.
[in]is_outputDetermines if the user wants the loaded images to be part of the output or not.
[in]area_factorDetermines how much area to be cropped. Ranges from from 0.08 - 1.
[in]aspect_ratioDetermines the aspect ration of crop. Ranges from 0.75 to 1.33.
[in]num_attemptsMaximum number of attempts to generate crop. Default 10
[in]decode_size_policy
[in]max_widthThe maximum width of the decoded images, larger or smaller will be resized to closest
[in]max_heightThe maximum height of the decoded images, larger or smaller will be resized to closest
[in]rocal_sharding_infoThe members of RocalShardingInfo determines how the data is distributed among the shards and how the last batch is processed by the pipeline.
Returns
Reference to the output tensor

◆ rocalJpegCOCOFileSourcePartialSingleShard()

RocalTensor ROCAL_API_CALL rocalJpegCOCOFileSourcePartialSingleShard ( RocalContext  p_context,
const char *  source_path,
const char *  json_path,
RocalImageColor  rocal_color_format,
unsigned  shard_id,
unsigned  shard_count,
bool  is_output,
std::vector< float > &  area_factor,
std::vector< float > &  aspect_ratio,
unsigned  num_attempts,
bool  shuffle = false,
bool  loop = false,
RocalImageSizeEvaluationPolicy  decode_size_policy = ROCAL_USE_MOST_FREQUENT_SIZE,
unsigned  max_width = 0,
unsigned  max_height = 0,
RocalShardingInfo  rocal_sharding_info = RocalShardingInfo() 
)

Creates JPEG image reader and partial decoder. It allocates the resources and objects required to read and decode COCO Jpeg images stored on the file systems. It has internal sharding capability to load/decode in parallel is user wants. If images are not Jpeg compressed they will be ignored.

Parameters
[in]rocal_contextRocal context
[in]source_pathA NULL terminated char string pointing to the location on the disk
[in]json_pathPath to the COCO Json File
[in]rocal_color_formatThe color format the images will be decoded to.
[in]shard_idShard id for this loader
[in]shard_countDefines the parallelism level by internally sharding the input dataset and load/decode using multiple decoder/loader instances. Using shard counts bigger than 1 improves the load/decode performance if compute resources (CPU cores) are available.
[in]is_outputDetermines if the user wants the loaded images to be part of the output or not.
[in]decode_size_policy
[in]max_widthThe maximum width of the decoded images, larger or smaller will be resized to closest
[in]max_heightThe maximum height of the decoded images, larger or smaller will be resized to closest
[in]area_factorDetermines how much area to be cropped. Ranges from from 0.08 - 1.
[in]aspect_ratioDetermines the aspect ration of crop. Ranges from 0.75 to 1.33.
[in]rocal_sharding_infoThe members of RocalShardingInfo determines how the data is distributed among the shards and how the last batch is processed by the pipeline.
Returns
Reference to the output tensor

◆ rocalJpegCOCOFileSourceSingleShard()

RocalTensor ROCAL_API_CALL rocalJpegCOCOFileSourceSingleShard ( RocalContext  context,
const char *  source_path,
const char *  json_path,
RocalImageColor  color_format,
unsigned  shard_id,
unsigned  shard_count,
bool  is_output,
bool  shuffle = false,
bool  loop = false,
RocalImageSizeEvaluationPolicy  decode_size_policy = ROCAL_USE_MOST_FREQUENT_SIZE,
unsigned  max_width = 0,
unsigned  max_height = 0,
RocalDecoderType  rocal_decoder_type = RocalDecoderType::ROCAL_DECODER_TJPEG,
RocalShardingInfo  rocal_sharding_info = RocalShardingInfo() 
)

Creates JPEG image reader. It allocates the resources and objects required to read and decode COCO Jpeg images stored on the file systems. It has internal sharding capability to load/decode in parallel is user wants. If images are not Jpeg compressed they will be ignored.

Parameters
[in]rocal_contextRocal context
[in]source_pathA NULL terminated char string pointing to the location on the disk
[in]json_pathPath to the COCO Json File
[in]rocal_color_formatThe color format the images will be decoded to.
[in]shard_idShard id for this loader
[in]shard_countTotal shard count
[in]is_outputDetermines if the user wants the loaded images to be part of the output or not.
[in]decode_size_policy
[in]max_widthThe maximum width of the decoded images, larger or smaller will be resized to closest
[in]max_heightThe maximum height of the decoded images, larger or smaller will be resized to closest
[in]rocal_decoder_typeDetermines the decoder_type, tjpeg or hwdec
[in]rocal_sharding_infoThe members of RocalShardingInfo determines how the data is distributed among the shards and how the last batch is processed by the pipeline.
Returns
Reference to the output tensor

◆ rocalJpegExternalFileSource()

RocalTensor ROCAL_API_CALL rocalJpegExternalFileSource ( RocalContext  p_context,
RocalImageColor  rocal_color_format,
bool  is_output = false,
bool  shuffle = false,
bool  loop = false,
RocalImageSizeEvaluationPolicy  decode_size_policy = ROCAL_USE_MOST_FREQUENT_SIZE,
unsigned  max_width = 0,
unsigned  max_height = 0,
RocalDecoderType  rocal_decoder_type = RocalDecoderType::ROCAL_DECODER_TJPEG,
RocalExternalSourceMode  external_source_mode = RocalExternalSourceMode::ROCAL_EXTSOURCE_FNAME,
RocalShardingInfo  rocal_sharding_info = RocalShardingInfo() 
)

Creates JPEG external source image reader.

Parameters
[in]rocal_contextRocal context
[in]rocal_color_formatThe color format the images will be decoded to.
[in]is_outputDetermines if the user wants the loaded images to be part of the output or not.
[in]shuffleDetermines if the user wants to shuffle the dataset or not.
[in]loopDetermines if the user wants to indefinitely loops through images or not.
[in]decode_size_policyis the RocalImageSizeEvaluationPolicy for decoding
[in]max_widthThe maximum width of the decoded images, larger or smaller will be resized to closest
[in]max_heightThe maximum height of the decoded images, larger or smaller will be resized to closest
[in]rocal_decoder_typeDetermines the decoder_type, tjpeg or hwdec
[in]external_source_modeDetermines the mode of the source passed from the user - file_names / uncompressed data / compressed data
[in]rocal_sharding_infoThe members of RocalShardingInfo determines how the data is distributed among the shards and how the last batch is processed by the pipeline.
Returns
Reference to the output tensor

◆ rocalJpegFileSource()

RocalTensor ROCAL_API_CALL rocalJpegFileSource ( RocalContext  context,
const char *  source_path,
RocalImageColor  rocal_color_format,
unsigned  internal_shard_count,
bool  is_output,
bool  shuffle = false,
bool  loop = false,
RocalImageSizeEvaluationPolicy  decode_size_policy = ROCAL_USE_MOST_FREQUENT_SIZE,
unsigned  max_width = 0,
unsigned  max_height = 0,
RocalDecoderType  rocal_decoder_type = RocalDecoderType::ROCAL_DECODER_TJPEG,
RocalShardingInfo  rocal_sharding_info = RocalShardingInfo() 
)

Creates JPEG image reader and decoder. It allocates the resources and objects required to read and decode Jpeg images stored on the file systems. It has internal sharding capability to load/decode in parallel is user wants. If images are not Jpeg compressed they will be ignored.

Parameters
[in]contextRocal context
[in]source_pathA NULL terminated char string pointing to the location on the disk
[in]rocal_color_formatThe color format the images will be decoded to.
[in]shard_countDefines the parallelism level by internally sharding the input dataset and load/decode using multiple decoder/loader instances. Using shard counts bigger than 1 improves the load/decode performance if compute resources (CPU cores) are available.
[in]is_outputDetermines if the user wants the loaded tensors to be part of the output or not.
[in]shuffleDetermines if the user wants to shuffle the dataset or not.
[in]loopDetermines if the user wants to indefinitely loops through images or not.
[in]decode_size_policy
[in]max_widthThe maximum width of the decoded images, larger or smaller will be resized to closest
[in]max_heightThe maximum height of the decoded images, larger or smaller will be resized to closest
[in]rocal_decoder_typeDetermines the decoder_type, tjpeg or hwdec
[in]rocal_sharding_infoThe members of RocalShardingInfo determines how the data is distributed among the shards and how the last batch is processed by the pipeline.
Returns
Reference to the output tensor

◆ rocalJpegFileSourceSingleShard()

RocalTensor ROCAL_API_CALL rocalJpegFileSourceSingleShard ( RocalContext  context,
const char *  source_path,
RocalImageColor  rocal_color_format,
unsigned  shard_id,
unsigned  shard_count,
bool  is_output,
bool  shuffle = false,
bool  loop = false,
RocalImageSizeEvaluationPolicy  decode_size_policy = ROCAL_USE_MOST_FREQUENT_SIZE,
unsigned  max_width = 0,
unsigned  max_height = 0,
RocalDecoderType  rocal_decoder_type = RocalDecoderType::ROCAL_DECODER_TJPEG,
RocalShardingInfo  rocal_sharding_info = RocalShardingInfo() 
)

Creates JPEG image reader and decoder. It allocates the resources and objects required to read and decode Jpeg images stored on the file systems. It accepts external sharding information to load a singe shard. only.

Parameters
[in]contextRocal context
[in]source_pathA NULL terminated char string pointing to the location on the disk
[in]rocal_color_formatThe color format the images will be decoded to.
[in]shard_idShard id for this loader
[in]shard_countTotal shard count
[in]is_outputDetermines if the user wants the loaded tensor to be part of the output or not.
[in]shuffleDetermines if the user wants to shuffle the dataset or not.
[in]loopDetermines if the user wants to indefinitely loops through images or not.
[in]decode_size_policy
[in]max_widthThe maximum width of the decoded images, larger or smaller will be resized to closest
[in]max_heightThe maximum height of the decoded images, larger or smaller will be resized to closest
[in]rocal_decoder_typeDetermines the decoder_type, tjpeg or hwdec
[in]rocal_sharding_infoThe members of RocalShardingInfo determines how the data is distributed among the shards and how the last batch is processed by the pipeline.
Returns
Reference to the output tensor

◆ rocalJpegTFRecordSource()

RocalTensor ROCAL_API_CALL rocalJpegTFRecordSource ( RocalContext  context,
const char *  source_path,
RocalImageColor  rocal_color_format,
unsigned  internal_shard_count,
bool  is_output,
const char *  user_key_for_encoded,
const char *  user_key_for_filename,
bool  shuffle = false,
bool  loop = false,
RocalImageSizeEvaluationPolicy  decode_size_policy = ROCAL_USE_MOST_FREQUENT_SIZE,
unsigned  max_width = 0,
unsigned  max_height = 0,
RocalDecoderType  rocal_decoder_type = RocalDecoderType::ROCAL_DECODER_TJPEG,
RocalShardingInfo  rocal_sharding_info = RocalShardingInfo() 
)

Creates TensorFlow records JPEG image reader and decoder. It allocates the resources and objects required to read and decode Jpeg images stored on the file systems. It has internal sharding capability to load/decode in parallel is user wants. If images are not Jpeg compressed they will be ignored.

Parameters
[in]contextRocal context
[in]source_pathA NULL terminated char string pointing to the location of the TF records on the disk
[in]rocal_color_formatThe color format the images will be decoded to.
[in]internal_shard_countDefines the parallelism level by internally sharding the input dataset and load/decode using multiple decoder/loader instances. Using shard counts bigger than 1 improves the load/decode performance if compute resources (CPU cores) are available.
[in]is_outputDetermines if the user wants the loaded images to be part of the output or not.
[in]shuffleDetermines if the user wants to shuffle the dataset or not.
[in]loopDetermines if the user wants to indefinitely loops through images or not.
[in]decode_size_policy
[in]max_widthThe maximum width of the decoded images, larger or smaller will be resized to closest
[in]max_heightThe maximum height of the decoded images, larger or smaller will be resized to closest
[in]rocal_decoder_typeDetermines the decoder_type, tjpeg or hwdec
[in]rocal_sharding_infoThe members of RocalShardingInfo determines how the data is distributed among the shards and how the last batch is processed by the pipeline.
Returns
Reference to the output image

◆ rocalJpegTFRecordSourceSingleShard()

RocalTensor ROCAL_API_CALL rocalJpegTFRecordSourceSingleShard ( RocalContext  context,
const char *  source_path,
RocalImageColor  rocal_color_format,
unsigned  shard_id,
unsigned  shard_count,
bool  is_output,
bool  shuffle = false,
bool  loop = false,
RocalImageSizeEvaluationPolicy  decode_size_policy = ROCAL_USE_MOST_FREQUENT_SIZE,
unsigned  max_width = 0,
unsigned  max_height = 0,
RocalDecoderType  rocal_decoder_type = RocalDecoderType::ROCAL_DECODER_TJPEG,
RocalShardingInfo  rocal_sharding_info = RocalShardingInfo() 
)

Creates TensorFlow records JPEG image reader and decoder. It allocates the resources and objects required to read and decode Jpeg images stored on the file systems. It accepts external sharding information to load a singe shard. only.

Parameters
[in]contextRocal context
[in]source_pathA NULL terminated char string pointing to the location of the TF records on the disk
[in]rocal_color_formatThe color format the images will be decoded to.
[in]shard_idShard id for this loader
[in]shard_countTotal shard count
[in]is_outputDetermines if the user wants the loaded images to be part of the output or not.
[in]shuffleDetermines if the user wants to shuffle the dataset or not.
[in]loopDetermines if the user wants to indefinitely loops through images or not.
[in]decode_size_policy
[in]max_widthThe maximum width of the decoded images, larger or smaller will be resized to closest
[in]max_heightThe maximum height of the decoded images, larger or smaller will be resized to closest
[in]rocal_decoder_typeDetermines the decoder_type, tjpeg or hwdec
[in]rocal_sharding_infoThe members of RocalShardingInfo determines how the data is distributed among the shards and how the last batch is processed by the pipeline.
Returns
Reference to the output tensor

◆ rocalMXNetRecordSource()

RocalTensor ROCAL_API_CALL rocalMXNetRecordSource ( RocalContext  context,
const char *  source_path,
RocalImageColor  rocal_color_format,
unsigned  internal_shard_count,
bool  is_output,
bool  shuffle = false,
bool  loop = false,
RocalImageSizeEvaluationPolicy  decode_size_policy = ROCAL_USE_MOST_FREQUENT_SIZE,
unsigned  max_width = 0,
unsigned  max_height = 0,
RocalDecoderType  rocal_decoder_type = RocalDecoderType::ROCAL_DECODER_TJPEG,
RocalShardingInfo  rocal_sharding_info = RocalShardingInfo() 
)

Creates JPEG image reader and decoder for MXNet records. It allocates the resources and objects required to read and decode Jpeg images stored in MXNet Records. It has internal sharding capability to load/decode in parallel is user wants. If images are not Jpeg compressed they will be ignored.

Parameters
[in]contextRocal context
[in]source_pathA NULL terminated char string pointing to the location on the disk
[in]rocal_color_formatThe color format the images will be decoded to.
[in]internal_shard_countDefines the parallelism level by internally sharding the input dataset and load/decode using multiple decoder/loader instances. Using shard counts bigger than 1 improves the load/decode performance if compute resources (CPU cores) are available.
[in]is_outputDetermines if the user wants the loaded images to be part of the output or not.
[in]shuffleDetermines if the user wants to shuffle the dataset or not.
[in]loopDetermines if the user wants to indefinitely loops through images or not.
[in]decode_size_policy
[in]max_widthThe maximum width of the decoded images, larger or smaller will be resized to closest
[in]max_heightThe maximum height of the decoded images, larger or smaller will be resized to closest
[in]rocal_decoder_typeDetermines the decoder_type, tjpeg or hwdec
[in]rocal_sharding_infoThe members of RocalShardingInfo determines how the data is distributed among the shards and how the last batch is processed by the pipeline.
Returns
Reference to the output tensor

◆ rocalMXNetRecordSourceSingleShard()

RocalTensor ROCAL_API_CALL rocalMXNetRecordSourceSingleShard ( RocalContext  p_context,
const char *  source_path,
RocalImageColor  rocal_color_format,
unsigned  shard_id,
unsigned  shard_count,
bool  is_output,
bool  shuffle = false,
bool  loop = false,
RocalImageSizeEvaluationPolicy  decode_size_policy = ROCAL_USE_MOST_FREQUENT_SIZE,
unsigned  max_width = 0,
unsigned  max_height = 0,
RocalDecoderType  rocal_decoder_type = RocalDecoderType::ROCAL_DECODER_TJPEG,
RocalShardingInfo  rocal_sharding_info = RocalShardingInfo() 
)

Creates JPEG image reader and decoder for MXNet records. It allocates the resources and objects required to read and decode Jpeg images stored on the MXNet records. It accepts external sharding information to load a singe shard. only.

Parameters
[in]p_contextRocal context
[in]source_pathA NULL terminated char string pointing to the location on the disk
[in]rocal_color_formatThe color format the images will be decoded to.
[in]shard_idShard id for this loader
[in]shard_countTotal shard count
[in]is_outputDetermines if the user wants the loaded images to be part of the output or not.
[in]shuffleDetermines if the user wants to shuffle the dataset or not.
[in]loopDetermines if the user wants to indefinitely loops through images or not.
[in]decode_size_policy
[in]max_widthThe maximum width of the decoded images, larger or smaller will be resized to closest
[in]max_heightThe maximum height of the decoded images, larger or smaller will be resized to closest
[in]rocal_decoder_typeDetermines the decoder_type, tjpeg or hwdec
[in]rocal_sharding_infoThe members of RocalShardingInfo determines how the data is distributed among the shards and how the last batch is processed by the pipeline.
Returns
Reference to the output tensor

◆ rocalRawCIFAR10Source()

RocalTensor ROCAL_API_CALL rocalRawCIFAR10Source ( RocalContext  context,
const char *  source_path,
RocalImageColor  color_format,
bool  is_output,
unsigned  out_width,
unsigned  out_height,
const char *  filename_prefix = "",
bool  loop = false 
)

Creates CIFAR10 raw data reader and loader. It allocates the resources and objects required to read raw data stored on the file systems.

Parameters
[in]contextRocal context
[in]source_pathA NULL terminated char string pointing to the location on the disk
[in]rocal_color_formatThe color format the images will be decoded to.
[in]is_outputDetermines if the user wants the loaded images to be part of the output or not.
[in]out_widthoutput width
[in]out_heightoutput_height
[in]filename_prefixif set loader will only load files with the given prefix name
[in]rocal_sharding_infoThe members of RocalShardingInfo determines how the data is distributed among the shards and how the last batch is processed by the pipeline.
Returns
Reference to the output tensor

◆ rocalRawTFRecordSource()

RocalTensor ROCAL_API_CALL rocalRawTFRecordSource ( RocalContext  p_context,
const char *  source_path,
const char *  user_key_for_raw,
const char *  user_key_for_filename,
RocalImageColor  rocal_color_format,
bool  is_output,
bool  shuffle = false,
bool  loop = false,
unsigned  out_width = 0,
unsigned  out_height = 0,
const char *  record_name_prefix = "",
RocalShardingInfo  rocal_sharding_info = RocalShardingInfo() 
)

Creates Raw image loader. It allocates the resources and objects required to load images stored on the file systems.

Parameters
[in]rocal_contextRocal context
[in]source_pathA NULL terminated char string pointing to the location on the disk
[in]rocal_color_formatThe color format the images will be decoded to.
[in]is_outputDetermines if the user wants the loaded images to be part of the output or not.
[in]shuffleto shuffle dataset
[in]looprepeat data loading
[in]out_widthThe output_width of raw image
[in]out_heightThe output height of raw image
[in]rocal_sharding_infoThe members of RocalShardingInfo determines how the data is distributed among the shards and how the last batch is processed by the pipeline.
Returns
Reference to the output tensor

◆ rocalRawTFRecordSourceSingleShard()

RocalTensor ROCAL_API_CALL rocalRawTFRecordSourceSingleShard ( RocalContext  p_context,
const char *  source_path,
RocalImageColor  rocal_color_format,
unsigned  shard_id,
unsigned  shard_count,
bool  is_output,
bool  shuffle = false,
bool  loop = false,
unsigned  out_width = 0,
unsigned  out_height = 0,
const char *  record_name_prefix = "",
RocalShardingInfo  rocal_sharding_info = RocalShardingInfo() 
)

Creates Raw image loader. It allocates the resources and objects required to load images stored on the file systems.

Parameters
[in]rocal_contextRocal context
[in]source_pathA NULL terminated char string pointing to the location on the disk
[in]rocal_color_formatThe color format the images will be decoded to.
[in]shard_idShard id for this loader
[in]shard_countTotal shard count
[in]shuffleto shuffle dataset
[in]looprepeat data loading
[in]out_widthThe output_width of raw image
[in]out_heightThe output height of raw image
[in]record_name_prefix: if nonempty reader will only read records with certain prefix
[in]rocal_sharding_infoThe members of RocalShardingInfo determines how the data is distributed among the shards and how the last batch is processed by the pipeline.
Returns
Reference to the output tensor

◆ rocalResetLoaders()

RocalStatus ROCAL_API_CALL rocalResetLoaders ( RocalContext  context)

reset Loaders

Parameters
[in]contextRocal Context
Returns
Rocal status value

◆ rocalSequenceReader()

RocalTensor ROCAL_API_CALL rocalSequenceReader ( RocalContext  context,
const char *  source_path,
RocalImageColor  rocal_color_format,
unsigned  internal_shard_count,
unsigned  sequence_length,
bool  is_output,
bool  shuffle = false,
bool  loop = false,
unsigned  step = 0,
unsigned  stride = 0,
RocalShardingInfo  rocal_sharding_info = RocalShardingInfo() 
)

Creates JPEG image reader and decoder. Reads [Frames] sequences from a directory representing a collection of streams.

Parameters
[in]contextRocal context
[in]source_pathA NULL terminated char string pointing to the location on the disk
[in]rocal_color_formatThe color format the images in a sequence will be decoded to.
[in]internal_shard_countDefines the parallelism level by internally sharding the input dataset and load/decode using multiple decoder/loader instances.
[in]sequence_lengthThe number of frames in a sequence.
[in]is_outputDetermines if the user wants the loaded sequences to be part of the output or not.
[in]shuffleDetermines if the user wants to shuffle the sequences or not.
[in]loopDetermines if the user wants to indefinitely loops through images or not.
[in]stepFrame interval between each sequence.
[in]strideFrame interval between frames in a sequence.
[in]rocal_sharding_infoThe members of RocalShardingInfo determines how the data is distributed among the shards and how the last batch is processed by the pipeline.
Returns
Reference to the output tensor.

◆ rocalSequenceReaderSingleShard()

RocalTensor ROCAL_API_CALL rocalSequenceReaderSingleShard ( RocalContext  context,
const char *  source_path,
RocalImageColor  rocal_color_format,
unsigned  shard_id,
unsigned  shard_count,
unsigned  sequence_length,
bool  is_output,
bool  shuffle = false,
bool  loop = false,
unsigned  step = 0,
unsigned  stride = 0,
RocalShardingInfo  rocal_sharding_info = RocalShardingInfo() 
)

Creates JPEG image reader and decoder. Reads [Frames] sequences from a directory representing a collection of streams. It accepts external sharding information to load a singe shard only.

Parameters
[in]contextRocal context
[in]source_pathA NULL terminated char string pointing to the location on the disk
[in]rocal_color_formatThe color format the images in a sequence will be decoded to.
[in]shard_idShard id for this loader
[in]shard_countTotal shard count
[in]sequence_lengthThe number of frames in a sequence.
[in]is_outputDetermines if the user wants the loaded sequences to be part of the output or not.
[in]shuffleDetermines if the user wants to shuffle the dataset or not.
[in]loopDetermines if the user wants to indefinitely loops through images or not.
[in]stepFrame interval between each sequence.
[in]strideFrame interval between frames in a sequence.
[in]rocal_sharding_infoThe members of RocalShardingInfo determines how the data is distributed among the shards and how the last batch is processed by the pipeline.
Returns
Reference to the output tensor

◆ rocalVideoFileResize()

RocalTensor ROCAL_API_CALL rocalVideoFileResize ( RocalContext  context,
const char *  source_path,
RocalImageColor  color_format,
RocalDecodeDevice  rocal_decode_device,
unsigned  internal_shard_count,
unsigned  sequence_length,
unsigned  dest_width,
unsigned  dest_height,
bool  shuffle = false,
bool  is_output = false,
bool  loop = false,
unsigned  step = 0,
unsigned  stride = 0,
bool  file_list_frame_num = true,
RocalResizeScalingMode  scaling_mode = ROCAL_SCALING_MODE_DEFAULT,
std::vector< unsigned >  max_size = {},
unsigned  resize_shorter = 0,
unsigned  resize_longer = 0,
RocalResizeInterpolationType  interpolation_type = ROCAL_LINEAR_INTERPOLATION,
RocalShardingInfo  rocal_sharding_info = RocalShardingInfo() 
)

Creates a video reader and decoder as a source. It allocates the resources and objects required to read and decode mp4 videos stored on the file systems. Resizes the decoded frames to the dest width and height.

Parameters
[in]contextRocal context
[in]source_pathA NULL terminated char string pointing to the location on the disk. source_path can be a video file, folder containing videos or a text file
[in]color_formatThe color format the frames will be decoded to.
[in]rocal_decode_deviceEnables software or hardware decoding. Currently only software decoding is supported.
[in]internal_shard_countDefines the parallelism level by internally sharding the input dataset and load/decode using multiple decoder/loader instances.
[in]sequence_lengthThe number of frames in a sequence.
[in]dest_widthThe output width of frames.
[in]dest_heightThe output height of frames.
[in]shuffleto shuffle sequences.
[in]is_outputDetermines if the user wants the loaded sequence of frames to be part of the output or not.
[in]looprepeat data loading.
[in]stepFrame interval between each sequence.
[in]strideFrame interval between frames in a sequence.
[in]file_list_frame_numDetermines if the user wants to read frame number or timestamps if a text file is passed in the source_path.
[in]rocal_sharding_infoThe members of RocalShardingInfo determines how the data is distributed among the shards and how the last batch is processed by the pipeline.
Returns
Reference to the output tensor

◆ rocalVideoFileResizeSingleShard()

RocalTensor ROCAL_API_CALL rocalVideoFileResizeSingleShard ( RocalContext  context,
const char *  source_path,
RocalImageColor  color_format,
RocalDecodeDevice  rocal_decode_device,
unsigned  shard_id,
unsigned  shard_count,
unsigned  sequence_length,
unsigned  dest_width,
unsigned  dest_height,
bool  shuffle = false,
bool  is_output = false,
bool  loop = false,
unsigned  step = 0,
unsigned  stride = 0,
bool  file_list_frame_num = true,
RocalResizeScalingMode  scaling_mode = ROCAL_SCALING_MODE_DEFAULT,
std::vector< unsigned >  max_size = {},
unsigned  resize_shorter = 0,
unsigned  resize_longer = 0,
RocalResizeInterpolationType  interpolation_type = ROCAL_LINEAR_INTERPOLATION,
RocalShardingInfo  rocal_sharding_info = RocalShardingInfo() 
)

Creates a video reader and decoder as a source. It allocates the resources and objects required to read and decode mp4 videos stored on the file systems. Resizes the decoded frames to the dest width and height. It accepts external sharding information to load a singe shard only.

Parameters
[in]contextRocal context
[in]source_pathA NULL terminated char string pointing to the location on the disk. source_path can be a video file, folder containing videos or a text file
[in]color_formatThe color format the frames will be decoded to.
[in]rocal_decode_deviceEnables software or hardware decoding. Currently only software decoding is supported.
[in]shard_idShard id for this loader.
[in]shard_countTotal shard count.
[in]sequence_lengthThe number of frames in a sequence.
[in]dest_widthThe output width of frames.
[in]dest_heightThe output height of frames.
[in]shuffleto shuffle sequences.
[in]is_outputDetermines if the user wants the loaded sequence of frames to be part of the output or not.
[in]looprepeat data loading.
[in]stepFrame interval between each sequence.
[in]strideFrame interval between frames in a sequence.
[in]file_list_frame_numDetermines if the user wants to read frame number or timestamps if a text file is passed in the source_path.
[in]rocal_sharding_infoThe members of RocalShardingInfo determines how the data is distributed among the shards and how the last batch is processed by the pipeline.
Returns
Reference to the output tensor

◆ rocalVideoFileSource()

RocalTensor ROCAL_API_CALL rocalVideoFileSource ( RocalContext  context,
const char *  source_path,
RocalImageColor  color_format,
RocalDecodeDevice  rocal_decode_device,
unsigned  internal_shard_count,
unsigned  sequence_length,
bool  is_output = false,
bool  shuffle = false,
bool  loop = false,
unsigned  step = 0,
unsigned  stride = 0,
bool  file_list_frame_num = true,
RocalShardingInfo  rocal_sharding_info = RocalShardingInfo() 
)

Creates a video reader and decoder as a source. It allocates the resources and objects required to read and decode mp4 videos stored on the file systems.

Parameters
[in]contextRocal context
[in]source_pathA NULL terminated char string pointing to the location on the disk. source_path can be a video file, folder containing videos or a text file
[in]color_formatThe color format the frames will be decoded to.
[in]rocal_decode_deviceEnables software or hardware decoding. Currently only software decoding is supported.
[in]internal_shard_countDefines the parallelism level by internally sharding the input dataset and load/decode using multiple decoder/loader instances.
[in]sequence_lengthThe number of frames in a sequence.
[in]shuffleto shuffle sequences.
[in]is_outputDetermines if the user wants the loaded sequence of frames to be part of the output or not.
[in]looprepeat data loading.
[in]stepFrame interval between each sequence.
[in]strideFrame interval between frames in a sequence.
[in]file_list_frame_numDetermines if the user wants to read frame number or timestamps if a text file is passed in the source_path.
[in]rocal_sharding_infoThe members of RocalShardingInfo determines how the data is distributed among the shards and how the last batch is processed by the pipeline.
Returns
Reference to the output tensor

◆ rocalVideoFileSourceSingleShard()

RocalTensor ROCAL_API_CALL rocalVideoFileSourceSingleShard ( RocalContext  context,
const char *  source_path,
RocalImageColor  color_format,
RocalDecodeDevice  rocal_decode_device,
unsigned  shard_id,
unsigned  shard_count,
unsigned  sequence_length,
bool  shuffle = false,
bool  is_output = false,
bool  loop = false,
unsigned  step = 0,
unsigned  stride = 0,
bool  file_list_frame_num = true,
RocalShardingInfo  rocal_sharding_info = RocalShardingInfo() 
)

Creates a video reader and decoder as a source. It allocates the resources and objects required to read and decode mp4 videos stored on the file systems. It accepts external sharding information to load a singe shard only.

Parameters
[in]contextRocal context
[in]source_pathA NULL terminated char string pointing to the location on the disk. source_path can be a video file, folder containing videos or a text file
[in]color_formatThe color format the frames will be decoded to.
[in]rocal_decode_deviceEnables software or hardware decoding. Currently only software decoding is supported.
[in]shard_idShard id for this loader.
[in]shard_countTotal shard count.
[in]sequence_lengthThe number of frames in a sequence.
[in]shuffleto shuffle sequences.
[in]is_outputDetermines if the user wants the loaded sequence of frames to be part of the output or not.
[in]looprepeat data loading.
[in]stepFrame interval between each sequence.
[in]strideFrame interval between frames in a sequence.
[in]file_list_frame_numDetermines if the user wants to read frame number or timestamps if a text file is passed in the source_path.
[in]rocal_sharding_infoThe members of RocalShardingInfo determines how the data is distributed among the shards and how the last batch is processed by the pipeline.
Returns
Reference to the output tensor