/home/docs/checkouts/readthedocs.org/user_builds/advanced-micro-devices-rocal/checkouts/docs-6.1.1/rocAL/include/api/rocal_api_data_loaders.h File Reference

/home/docs/checkouts/readthedocs.org/user_builds/advanced-micro-devices-rocal/checkouts/docs-6.1.1/rocAL/include/api/rocal_api_data_loaders.h File Reference#

rocAL: /home/docs/checkouts/readthedocs.org/user_builds/advanced-micro-devices-rocal/checkouts/docs-6.1.1/rocAL/include/api/rocal_api_data_loaders.h File Reference
rocal_api_data_loaders.h File Reference

The AMD rocAL Library - Data Loaders. More...

#include "rocal_api_types.h"

Go to the source code of this file.

Functions

RocalTensor ROCAL_API_CALL rocalJpegFileSource (RocalContext context, const char *source_path, RocalImageColor rocal_color_format, unsigned internal_shard_count, bool is_output, bool shuffle=false, bool loop=false, RocalImageSizeEvaluationPolicy decode_size_policy=ROCAL_USE_MOST_FREQUENT_SIZE, unsigned max_width=0, unsigned max_height=0, RocalDecoderType rocal_decoder_type=RocalDecoderType::ROCAL_DECODER_TJPEG)
 Creates JPEG image reader and decoder. It allocates the resources and objects required to read and decode Jpeg images stored on the file systems. It has internal sharding capability to load/decode in parallel is user wants. If images are not Jpeg compressed they will be ignored. More...
 
RocalTensor ROCAL_API_CALL rocalJpegFileSourceSingleShard (RocalContext context, const char *source_path, RocalImageColor rocal_color_format, unsigned shard_id, unsigned shard_count, bool is_output, bool shuffle=false, bool loop=false, RocalImageSizeEvaluationPolicy decode_size_policy=ROCAL_USE_MOST_FREQUENT_SIZE, unsigned max_width=0, unsigned max_height=0, RocalDecoderType rocal_decoder_type=RocalDecoderType::ROCAL_DECODER_TJPEG)
 Creates JPEG image reader and decoder. It allocates the resources and objects required to read and decode Jpeg images stored on the file systems. It accepts external sharding information to load a singe shard. only. More...
 
RocalTensor ROCAL_API_CALL rocalSequenceReader (RocalContext context, const char *source_path, RocalImageColor rocal_color_format, unsigned internal_shard_count, unsigned sequence_length, bool is_output, bool shuffle=false, bool loop=false, unsigned step=0, unsigned stride=0)
 Creates JPEG image reader and decoder. Reads [Frames] sequences from a directory representing a collection of streams. More...
 
RocalTensor ROCAL_API_CALL rocalSequenceReaderSingleShard (RocalContext context, const char *source_path, RocalImageColor rocal_color_format, unsigned shard_id, unsigned shard_count, unsigned sequence_length, bool is_output, bool shuffle=false, bool loop=false, unsigned step=0, unsigned stride=0)
 Creates JPEG image reader and decoder. Reads [Frames] sequences from a directory representing a collection of streams. It accepts external sharding information to load a singe shard only. More...
 
RocalTensor ROCAL_API_CALL rocalJpegCOCOFileSource (RocalContext context, const char *source_path, const char *json_path, RocalImageColor color_format, unsigned internal_shard_count, bool is_output, bool shuffle=false, bool loop=false, RocalImageSizeEvaluationPolicy decode_size_policy=ROCAL_USE_MOST_FREQUENT_SIZE, unsigned max_width=0, unsigned max_height=0, RocalDecoderType rocal_decoder_type=RocalDecoderType::ROCAL_DECODER_TJPEG)
 JPEG image reader and decoder. It allocates the resources and objects required to read and decode COCO Jpeg images stored on the file systems. It has internal sharding capability to load/decode in parallel is user wants. If images are not Jpeg compressed they will be ignored. More...
 
RocalTensor ROCAL_API_CALL rocalJpegCOCOFileSourcePartial (RocalContext p_context, const char *source_path, const char *json_path, RocalImageColor rocal_color_format, unsigned internal_shard_count, bool is_output, std::vector< float > &area_factor, std::vector< float > &aspect_ratio, unsigned num_attempts, bool shuffle=false, bool loop=false, RocalImageSizeEvaluationPolicy decode_size_policy=ROCAL_USE_MOST_FREQUENT_SIZE, unsigned max_width=0, unsigned max_height=0)
 JPEG image reader and partial decoder. It allocates the resources and objects required to read and decode COCO Jpeg images stored on the file systems. It has internal sharding capability to load/decode in parallel is user wants. If images are not Jpeg compressed they will be ignored. More...
 
RocalTensor ROCAL_API_CALL rocalJpegCOCOFileSourcePartialSingleShard (RocalContext p_context, const char *source_path, const char *json_path, RocalImageColor rocal_color_format, unsigned shard_id, unsigned shard_count, bool is_output, std::vector< float > &area_factor, std::vector< float > &aspect_ratio, unsigned num_attempts, bool shuffle=false, bool loop=false, RocalImageSizeEvaluationPolicy decode_size_policy=ROCAL_USE_MOST_FREQUENT_SIZE, unsigned max_width=0, unsigned max_height=0)
 Creates JPEG image reader and partial decoder. It allocates the resources and objects required to read and decode COCO Jpeg images stored on the file systems. It has internal sharding capability to load/decode in parallel is user wants. If images are not Jpeg compressed they will be ignored. More...
 
RocalTensor ROCAL_API_CALL rocalJpegCOCOFileSourceSingleShard (RocalContext context, const char *source_path, const char *json_path, RocalImageColor color_format, unsigned shard_id, unsigned shard_count, bool is_output, bool shuffle=false, bool loop=false, RocalImageSizeEvaluationPolicy decode_size_policy=ROCAL_USE_MOST_FREQUENT_SIZE, unsigned max_width=0, unsigned max_height=0, RocalDecoderType rocal_decoder_type=RocalDecoderType::ROCAL_DECODER_TJPEG)
 Creates JPEG image reader. It allocates the resources and objects required to read and decode COCO Jpeg images stored on the file systems. It has internal sharding capability to load/decode in parallel is user wants. If images are not Jpeg compressed they will be ignored. More...
 
RocalTensor ROCAL_API_CALL rocalJpegCaffeLMDBRecordSource (RocalContext context, const char *source_path, RocalImageColor rocal_color_format, unsigned internal_shard_count, bool is_output, bool shuffle=false, bool loop=false, RocalImageSizeEvaluationPolicy decode_size_policy=ROCAL_USE_MOST_FREQUENT_SIZE, unsigned max_width=0, unsigned max_height=0, RocalDecoderType rocal_decoder_type=RocalDecoderType::ROCAL_DECODER_TJPEG)
 Creates JPEG image reader and decoder for Caffe LMDB records. It allocates the resources and objects required to read and decode Jpeg images stored in Caffe LMDB Records. It has internal sharding capability to load/decode in parallel is user wants. If images are not Jpeg compressed they will be ignored. More...
 
RocalTensor ROCAL_API_CALL rocalJpegCaffeLMDBRecordSourceSingleShard (RocalContext p_context, const char *source_path, RocalImageColor rocal_color_format, unsigned shard_id, unsigned shard_count, bool is_output, bool shuffle=false, bool loop=false, RocalImageSizeEvaluationPolicy decode_size_policy=ROCAL_USE_MOST_FREQUENT_SIZE, unsigned max_width=0, unsigned max_height=0, RocalDecoderType rocal_decoder_type=RocalDecoderType::ROCAL_DECODER_TJPEG)
 Creates JPEG image reader and decoder for Caffe LMDB records. It allocates the resources and objects required to read and decode Jpeg images stored in Caffe2 LMDB Records. It has internal sharding capability to load/decode in parallel is user wants. More...
 
RocalTensor ROCAL_API_CALL rocalJpegCaffe2LMDBRecordSource (RocalContext context, const char *source_path, RocalImageColor rocal_color_format, unsigned internal_shard_count, bool is_output, bool shuffle=false, bool loop=false, RocalImageSizeEvaluationPolicy decode_size_policy=ROCAL_USE_MOST_FREQUENT_SIZE, unsigned max_width=0, unsigned max_height=0, RocalDecoderType rocal_decoder_type=RocalDecoderType::ROCAL_DECODER_TJPEG)
 Creates JPEG image reader and decoder for Caffe2 LMDB records. It allocates the resources and objects required to read and decode Jpeg images stored in Caffe2 LMDB Records. It has internal sharding capability to load/decode in parallel is user wants. If images are not Jpeg compressed they will be ignored. More...
 
RocalTensor ROCAL_API_CALL rocalJpegCaffe2LMDBRecordSourceSingleShard (RocalContext p_context, const char *source_path, RocalImageColor rocal_color_format, unsigned shard_id, unsigned shard_count, bool is_output, bool shuffle=false, bool loop=false, RocalImageSizeEvaluationPolicy decode_size_policy=ROCAL_USE_MOST_FREQUENT_SIZE, unsigned max_width=0, unsigned max_height=0, RocalDecoderType rocal_decoder_type=RocalDecoderType::ROCAL_DECODER_TJPEG)
 Creates JPEG image reader and decoder for Caffe2 LMDB records. It allocates the resources and objects required to read and decode Jpeg images stored on the Caffe2 LMDB Records. It accepts external sharding information to load a singe shard. only. More...
 
RocalTensor ROCAL_API_CALL rocalMXNetRecordSource (RocalContext context, const char *source_path, RocalImageColor rocal_color_format, unsigned internal_shard_count, bool is_output, bool shuffle=false, bool loop=false, RocalImageSizeEvaluationPolicy decode_size_policy=ROCAL_USE_MOST_FREQUENT_SIZE, unsigned max_width=0, unsigned max_height=0, RocalDecoderType rocal_decoder_type=RocalDecoderType::ROCAL_DECODER_TJPEG)
 Creates JPEG image reader and decoder for MXNet records. It allocates the resources and objects required to read and decode Jpeg images stored in MXNet Records. It has internal sharding capability to load/decode in parallel is user wants. If images are not Jpeg compressed they will be ignored. More...
 
RocalTensor ROCAL_API_CALL rocalMXNetRecordSourceSingleShard (RocalContext p_context, const char *source_path, RocalImageColor rocal_color_format, unsigned shard_id, unsigned shard_count, bool is_output, bool shuffle=false, bool loop=false, RocalImageSizeEvaluationPolicy decode_size_policy=ROCAL_USE_MOST_FREQUENT_SIZE, unsigned max_width=0, unsigned max_height=0, RocalDecoderType rocal_decoder_type=RocalDecoderType::ROCAL_DECODER_TJPEG)
 Creates JPEG image reader and decoder for MXNet records. It allocates the resources and objects required to read and decode Jpeg images stored on the MXNet records. It accepts external sharding information to load a singe shard. only. More...
 
RocalTensor ROCAL_API_CALL rocalFusedJpegCrop (RocalContext context, const char *source_path, RocalImageColor rocal_color_format, unsigned num_threads, bool is_output, std::vector< float > &area_factor, std::vector< float > &aspect_ratio, unsigned num_attempts, bool shuffle=false, bool loop=false, RocalImageSizeEvaluationPolicy decode_size_policy=ROCAL_USE_MOST_FREQUENT_SIZE, unsigned max_width=0, unsigned max_height=0)
 Creates JPEG image reader and partial decoder. It allocates the resources and objects required to read and decode Jpeg images stored on the file systems. It has internal sharding capability to load/decode in parallel is user wants. If images are not Jpeg compressed they will be ignored and Crops t. More...
 
RocalTensor ROCAL_API_CALL rocalFusedJpegCropSingleShard (RocalContext context, const char *source_path, RocalImageColor color_format, unsigned shard_id, unsigned shard_count, bool is_output, std::vector< float > &area_factor, std::vector< float > &aspect_ratio, unsigned num_attempts, bool shuffle=false, bool loop=false, RocalImageSizeEvaluationPolicy decode_size_policy=ROCAL_USE_MOST_FREQUENT_SIZE, unsigned max_width=0, unsigned max_height=0)
 Creates JPEG image reader and partial decoder. It allocates the resources and objects required to read and decode Jpeg images stored on the file systems. It accepts external sharding information to load a singe shard. only. More...
 
RocalTensor ROCAL_API_CALL rocalJpegTFRecordSource (RocalContext context, const char *source_path, RocalImageColor rocal_color_format, unsigned internal_shard_count, bool is_output, const char *user_key_for_encoded, const char *user_key_for_filename, bool shuffle=false, bool loop=false, RocalImageSizeEvaluationPolicy decode_size_policy=ROCAL_USE_MOST_FREQUENT_SIZE, unsigned max_width=0, unsigned max_height=0, RocalDecoderType rocal_decoder_type=RocalDecoderType::ROCAL_DECODER_TJPEG)
 Creates TensorFlow records JPEG image reader and decoder. It allocates the resources and objects required to read and decode Jpeg images stored on the file systems. It has internal sharding capability to load/decode in parallel is user wants. If images are not Jpeg compressed they will be ignored. More...
 
RocalTensor ROCAL_API_CALL rocalJpegTFRecordSourceSingleShard (RocalContext context, const char *source_path, RocalImageColor rocal_color_format, unsigned shard_id, unsigned shard_count, bool is_output, bool shuffle=false, bool loop=false, RocalImageSizeEvaluationPolicy decode_size_policy=ROCAL_USE_MOST_FREQUENT_SIZE, unsigned max_width=0, unsigned max_height=0, RocalDecoderType rocal_decoder_type=RocalDecoderType::ROCAL_DECODER_TJPEG)
 Creates TensorFlow records JPEG image reader and decoder. It allocates the resources and objects required to read and decode Jpeg images stored on the file systems. It accepts external sharding information to load a singe shard. only. More...
 
RocalTensor ROCAL_API_CALL rocalRawTFRecordSource (RocalContext p_context, const char *source_path, const char *user_key_for_raw, const char *user_key_for_filename, RocalImageColor rocal_color_format, bool is_output, bool shuffle=false, bool loop=false, unsigned out_width=0, unsigned out_height=0, const char *record_name_prefix="")
 Creates Raw image loader. It allocates the resources and objects required to load images stored on the file systems. More...
 
RocalTensor ROCAL_API_CALL rocalRawTFRecordSourceSingleShard (RocalContext p_context, const char *source_path, RocalImageColor rocal_color_format, unsigned shard_id, unsigned shard_count, bool is_output, bool shuffle=false, bool loop=false, unsigned out_width=0, unsigned out_height=0, const char *record_name_prefix="")
 Creates Raw image loader. It allocates the resources and objects required to load images stored on the file systems. More...
 
RocalTensor ROCAL_API_CALL rocalVideoFileSource (RocalContext context, const char *source_path, RocalImageColor color_format, RocalDecodeDevice rocal_decode_device, unsigned internal_shard_count, unsigned sequence_length, bool is_output=false, bool shuffle=false, bool loop=false, unsigned step=0, unsigned stride=0, bool file_list_frame_num=true)
 Creates a video reader and decoder as a source. It allocates the resources and objects required to read and decode mp4 videos stored on the file systems. More...
 
RocalTensor ROCAL_API_CALL rocalVideoFileSourceSingleShard (RocalContext context, const char *source_path, RocalImageColor color_format, RocalDecodeDevice rocal_decode_device, unsigned shard_id, unsigned shard_count, unsigned sequence_length, bool shuffle=false, bool is_output=false, bool loop=false, unsigned step=0, unsigned stride=0, bool file_list_frame_num=true)
 Creates a video reader and decoder as a source. It allocates the resources and objects required to read and decode mp4 videos stored on the file systems. It accepts external sharding information to load a singe shard only. More...
 
RocalTensor ROCAL_API_CALL rocalVideoFileResize (RocalContext context, const char *source_path, RocalImageColor color_format, RocalDecodeDevice rocal_decode_device, unsigned internal_shard_count, unsigned sequence_length, unsigned dest_width, unsigned dest_height, bool shuffle=false, bool is_output=false, bool loop=false, unsigned step=0, unsigned stride=0, bool file_list_frame_num=true, RocalResizeScalingMode scaling_mode=ROCAL_SCALING_MODE_DEFAULT, std::vector< unsigned > max_size={}, unsigned resize_shorter=0, unsigned resize_longer=0, RocalResizeInterpolationType interpolation_type=ROCAL_LINEAR_INTERPOLATION)
 Creates a video reader and decoder as a source. It allocates the resources and objects required to read and decode mp4 videos stored on the file systems. Resizes the decoded frames to the dest width and height. More...
 
RocalTensor ROCAL_API_CALL rocalVideoFileResizeSingleShard (RocalContext context, const char *source_path, RocalImageColor color_format, RocalDecodeDevice rocal_decode_device, unsigned shard_id, unsigned shard_count, unsigned sequence_length, unsigned dest_width, unsigned dest_height, bool shuffle=false, bool is_output=false, bool loop=false, unsigned step=0, unsigned stride=0, bool file_list_frame_num=true, RocalResizeScalingMode scaling_mode=ROCAL_SCALING_MODE_DEFAULT, std::vector< unsigned > max_size={}, unsigned resize_shorter=0, unsigned resize_longer=0, RocalResizeInterpolationType interpolation_type=ROCAL_LINEAR_INTERPOLATION)
 Creates a video reader and decoder as a source. It allocates the resources and objects required to read and decode mp4 videos stored on the file systems. Resizes the decoded frames to the dest width and height. It accepts external sharding information to load a singe shard only. More...
 
RocalTensor ROCAL_API_CALL rocalRawCIFAR10Source (RocalContext context, const char *source_path, RocalImageColor color_format, bool is_output, unsigned out_width, unsigned out_height, const char *filename_prefix="", bool loop=false)
 Creates CIFAR10 raw data reader and loader. It allocates the resources and objects required to read raw data stored on the file systems. More...
 
RocalStatus ROCAL_API_CALL rocalResetLoaders (RocalContext context)
 reset Loaders More...
 
RocalTensor ROCAL_API_CALL rocalJpegCaffeLMDBRecordSourcePartialSingleShard (RocalContext p_context, const char *source_path, RocalImageColor rocal_color_format, unsigned shard_id, unsigned shard_count, bool is_output, std::vector< float > &area_factor, std::vector< float > &aspect_ratio, unsigned num_attempts, bool shuffle=false, bool loop=false, RocalImageSizeEvaluationPolicy decode_size_policy=ROCAL_USE_MOST_FREQUENT_SIZE, unsigned max_width=0, unsigned max_height=0)
 Creates JPEG image reader and partial decoder for Caffe LMDB records. It allocates the resources and objects required to read and decode Jpeg images stored in Caffe2 LMDB Records. It has internal sharding capability to load/decode in parallel is user wants. More...
 
RocalTensor ROCAL_API_CALL rocalJpegCaffe2LMDBRecordSourcePartialSingleShard (RocalContext p_context, const char *source_path, RocalImageColor rocal_color_format, unsigned shard_id, unsigned shard_count, bool is_output, std::vector< float > &area_factor, std::vector< float > &aspect_ratio, unsigned num_attempts, bool shuffle=false, bool loop=false, RocalImageSizeEvaluationPolicy decode_size_policy=ROCAL_USE_MOST_FREQUENT_SIZE, unsigned max_width=0, unsigned max_height=0)
 Creates JPEG image reader and partial decoder for Caffe2 LMDB records. It allocates the resources and objects required to read and decode Jpeg images stored in Caffe22 LMDB Records. It has internal sharding capability to load/decode in parallel is user wants. More...
 
RocalTensor ROCAL_API_CALL rocalJpegExternalFileSource (RocalContext p_context, RocalImageColor rocal_color_format, bool is_output=false, bool shuffle=false, bool loop=false, RocalImageSizeEvaluationPolicy decode_size_policy=ROCAL_USE_MOST_FREQUENT_SIZE, unsigned max_width=0, unsigned max_height=0, RocalDecoderType rocal_decoder_type=RocalDecoderType::ROCAL_DECODER_TJPEG, RocalExternalSourceMode external_source_mode=RocalExternalSourceMode::ROCAL_EXTSOURCE_FNAME)
 Creates JPEG external source image reader. More...
 
RocalTensor ROCAL_API_CALL rocalAudioFileSource (RocalContext context, const char *source_path, unsigned shard_count, bool is_output, bool shuffle=false, bool loop=false, bool downmix=false)
 
RocalTensor ROCAL_API_CALL rocalAudioFileSourceSingleShard (RocalContext p_context, const char *source_path, unsigned shard_id, unsigned shard_count, bool is_output, bool shuffle=false, bool loop=false, bool downmix=false)
 

Detailed Description

The AMD rocAL Library - Data Loaders.

Function Documentation

◆ rocalAudioFileSource()

RocalTensor ROCAL_API_CALL rocalAudioFileSource ( RocalContext  context,
const char *  source_path,
unsigned  shard_count,
bool  is_output,
bool  shuffle = false,
bool  loop = false,
bool  downmix = false 
)

Creates Audio file reader and decoder. It allocates the resources and objects required to read and decode audio files stored on the file systems. It has internal sharding capability to load/decode in parallel if user wants. If the files are not in standard audio compression formats they will be ignored, Currently wav format is supported

Parameters
[in]contextRocal context
[in]source_pathA NULL terminated char string pointing to the location of files on the disk
[in]shard_countDefines the parallelism level by internally sharding the input dataset and load/decode using multiple decoder/loader instances. Using shard counts bigger than 1 improves the load/decode performance if compute resources (CPU cores) are available.
[in]is_outputBoolean variable to enable the audio to be part of the output.
[in]shuffleBoolean variable to shuffle the dataset.
[in]loopBoolean variable to indefinitely loop through audio.
[in]downmixBoolean variable to downmix all input channels to mono. If downmixing is turned on, the decoder output is 1D. If downmixing is turned off, it produces 2D output with interleaved channels incase of multichannel audio.
Returns
Reference to the output audio

◆ rocalAudioFileSourceSingleShard()

RocalTensor ROCAL_API_CALL rocalAudioFileSourceSingleShard ( RocalContext  p_context,
const char *  source_path,
unsigned  shard_id,
unsigned  shard_count,
bool  is_output,
bool  shuffle = false,
bool  loop = false,
bool  downmix = false 
)

Creates Audio file reader and decoder. It allocates the resources and objects required to read and decode audio files stored on the file systems. It has internal sharding capability to load/decode in parallel is user wants. If the files are not in standard audio compression formats they will be ignored.

Parameters
[in]contextRocal context
[in]source_pathA NULL terminated char string pointing to the location of files on the disk
[in]shard_idShard id for this loader
[in]shard_countDefines the parallelism level by internally sharding the input dataset and load/decode using multiple decoder/loader instances. Using shard counts bigger than 1 improves the load/decode performance if compute resources (CPU cores) are available.
[in]is_outputBoolean variable to enable the audio to be part of the output.
[in]shuffleBoolean variable to shuffle the dataset.
[in]loopBoolean variable to indefinitely loop through audio.
[in]downmixBoolean variable to downmix all input channels to mono. If downmixing is turned on, the decoder output is 1D. If downmixing is turned off, it produces 2D output with interleaved channels incase of multichannel audio.
Returns
Reference to the output audio