/home/docs/checkouts/readthedocs.org/user_builds/advanced-micro-devices-rocal/checkouts/latest/rocAL/include/api/rocal_api_data_loaders.h File Reference#
rocal_api_data_loaders.h File Reference
The AMD rocAL Library - Data Loaders. More...
#include "rocal_api_types.h"Go to the source code of this file.
Functions | |
| RocalTensor ROCAL_API_CALL | rocalJpegFileSource (RocalContext context, const char *source_path, RocalImageColor rocal_color_format, unsigned internal_shard_count, bool is_output, bool shuffle=false, bool loop=false, RocalImageSizeEvaluationPolicy decode_size_policy=ROCAL_USE_MOST_FREQUENT_SIZE, unsigned max_width=0, unsigned max_height=0, RocalDecoderType rocal_decoder_type=RocalDecoderType::ROCAL_DECODER_TJPEG) |
| Creates JPEG image reader and decoder. It allocates the resources and objects required to read and decode Jpeg images stored on the file systems. It has internal sharding capability to load/decode in parallel is user wants. If images are not Jpeg compressed they will be ignored. More... | |
| RocalTensor ROCAL_API_CALL | rocalJpegFileSourceSingleShard (RocalContext context, const char *source_path, RocalImageColor rocal_color_format, unsigned shard_id, unsigned shard_count, bool is_output, bool shuffle=false, bool loop=false, RocalImageSizeEvaluationPolicy decode_size_policy=ROCAL_USE_MOST_FREQUENT_SIZE, unsigned max_width=0, unsigned max_height=0, RocalDecoderType rocal_decoder_type=RocalDecoderType::ROCAL_DECODER_TJPEG) |
| Creates JPEG image reader and decoder. It allocates the resources and objects required to read and decode Jpeg images stored on the file systems. It accepts external sharding information to load a singe shard. only. More... | |
| RocalTensor ROCAL_API_CALL | rocalSequenceReader (RocalContext context, const char *source_path, RocalImageColor rocal_color_format, unsigned internal_shard_count, unsigned sequence_length, bool is_output, bool shuffle=false, bool loop=false, unsigned step=0, unsigned stride=0) |
| Creates JPEG image reader and decoder. Reads [Frames] sequences from a directory representing a collection of streams. More... | |
| RocalTensor ROCAL_API_CALL | rocalSequenceReaderSingleShard (RocalContext context, const char *source_path, RocalImageColor rocal_color_format, unsigned shard_id, unsigned shard_count, unsigned sequence_length, bool is_output, bool shuffle=false, bool loop=false, unsigned step=0, unsigned stride=0) |
| Creates JPEG image reader and decoder. Reads [Frames] sequences from a directory representing a collection of streams. It accepts external sharding information to load a singe shard only. More... | |
| RocalTensor ROCAL_API_CALL | rocalJpegCOCOFileSource (RocalContext context, const char *source_path, const char *json_path, RocalImageColor color_format, unsigned internal_shard_count, bool is_output, bool shuffle=false, bool loop=false, RocalImageSizeEvaluationPolicy decode_size_policy=ROCAL_USE_MOST_FREQUENT_SIZE, unsigned max_width=0, unsigned max_height=0, RocalDecoderType rocal_decoder_type=RocalDecoderType::ROCAL_DECODER_TJPEG) |
| JPEG image reader and decoder. It allocates the resources and objects required to read and decode COCO Jpeg images stored on the file systems. It has internal sharding capability to load/decode in parallel is user wants. If images are not Jpeg compressed they will be ignored. More... | |
| RocalTensor ROCAL_API_CALL | rocalJpegCOCOFileSourcePartial (RocalContext p_context, const char *source_path, const char *json_path, RocalImageColor rocal_color_format, unsigned internal_shard_count, bool is_output, std::vector< float > &area_factor, std::vector< float > &aspect_ratio, unsigned num_attempts, bool shuffle=false, bool loop=false, RocalImageSizeEvaluationPolicy decode_size_policy=ROCAL_USE_MOST_FREQUENT_SIZE, unsigned max_width=0, unsigned max_height=0) |
| JPEG image reader and partial decoder. It allocates the resources and objects required to read and decode COCO Jpeg images stored on the file systems. It has internal sharding capability to load/decode in parallel is user wants. If images are not Jpeg compressed they will be ignored. More... | |
| RocalTensor ROCAL_API_CALL | rocalJpegCOCOFileSourcePartialSingleShard (RocalContext p_context, const char *source_path, const char *json_path, RocalImageColor rocal_color_format, unsigned shard_id, unsigned shard_count, bool is_output, std::vector< float > &area_factor, std::vector< float > &aspect_ratio, unsigned num_attempts, bool shuffle=false, bool loop=false, RocalImageSizeEvaluationPolicy decode_size_policy=ROCAL_USE_MOST_FREQUENT_SIZE, unsigned max_width=0, unsigned max_height=0) |
| Creates JPEG image reader and partial decoder. It allocates the resources and objects required to read and decode COCO Jpeg images stored on the file systems. It has internal sharding capability to load/decode in parallel is user wants. If images are not Jpeg compressed they will be ignored. More... | |
| RocalTensor ROCAL_API_CALL | rocalJpegCOCOFileSourceSingleShard (RocalContext context, const char *source_path, const char *json_path, RocalImageColor color_format, unsigned shard_id, unsigned shard_count, bool is_output, bool shuffle=false, bool loop=false, RocalImageSizeEvaluationPolicy decode_size_policy=ROCAL_USE_MOST_FREQUENT_SIZE, unsigned max_width=0, unsigned max_height=0, RocalDecoderType rocal_decoder_type=RocalDecoderType::ROCAL_DECODER_TJPEG) |
| Creates JPEG image reader. It allocates the resources and objects required to read and decode COCO Jpeg images stored on the file systems. It has internal sharding capability to load/decode in parallel is user wants. If images are not Jpeg compressed they will be ignored. More... | |
| RocalTensor ROCAL_API_CALL | rocalJpegCaffeLMDBRecordSource (RocalContext context, const char *source_path, RocalImageColor rocal_color_format, unsigned internal_shard_count, bool is_output, bool shuffle=false, bool loop=false, RocalImageSizeEvaluationPolicy decode_size_policy=ROCAL_USE_MOST_FREQUENT_SIZE, unsigned max_width=0, unsigned max_height=0, RocalDecoderType rocal_decoder_type=RocalDecoderType::ROCAL_DECODER_TJPEG) |
| Creates JPEG image reader and decoder for Caffe LMDB records. It allocates the resources and objects required to read and decode Jpeg images stored in Caffe LMDB Records. It has internal sharding capability to load/decode in parallel is user wants. If images are not Jpeg compressed they will be ignored. More... | |
| RocalTensor ROCAL_API_CALL | rocalJpegCaffeLMDBRecordSourceSingleShard (RocalContext p_context, const char *source_path, RocalImageColor rocal_color_format, unsigned shard_id, unsigned shard_count, bool is_output, bool shuffle=false, bool loop=false, RocalImageSizeEvaluationPolicy decode_size_policy=ROCAL_USE_MOST_FREQUENT_SIZE, unsigned max_width=0, unsigned max_height=0, RocalDecoderType rocal_decoder_type=RocalDecoderType::ROCAL_DECODER_TJPEG) |
| Creates JPEG image reader and decoder for Caffe LMDB records. It allocates the resources and objects required to read and decode Jpeg images stored in Caffe2 LMDB Records. It has internal sharding capability to load/decode in parallel is user wants. More... | |
| RocalTensor ROCAL_API_CALL | rocalJpegCaffe2LMDBRecordSource (RocalContext context, const char *source_path, RocalImageColor rocal_color_format, unsigned internal_shard_count, bool is_output, bool shuffle=false, bool loop=false, RocalImageSizeEvaluationPolicy decode_size_policy=ROCAL_USE_MOST_FREQUENT_SIZE, unsigned max_width=0, unsigned max_height=0, RocalDecoderType rocal_decoder_type=RocalDecoderType::ROCAL_DECODER_TJPEG) |
| Creates JPEG image reader and decoder for Caffe2 LMDB records. It allocates the resources and objects required to read and decode Jpeg images stored in Caffe2 LMDB Records. It has internal sharding capability to load/decode in parallel is user wants. If images are not Jpeg compressed they will be ignored. More... | |
| RocalTensor ROCAL_API_CALL | rocalJpegCaffe2LMDBRecordSourceSingleShard (RocalContext p_context, const char *source_path, RocalImageColor rocal_color_format, unsigned shard_id, unsigned shard_count, bool is_output, bool shuffle=false, bool loop=false, RocalImageSizeEvaluationPolicy decode_size_policy=ROCAL_USE_MOST_FREQUENT_SIZE, unsigned max_width=0, unsigned max_height=0, RocalDecoderType rocal_decoder_type=RocalDecoderType::ROCAL_DECODER_TJPEG) |
| Creates JPEG image reader and decoder for Caffe2 LMDB records. It allocates the resources and objects required to read and decode Jpeg images stored on the Caffe2 LMDB Records. It accepts external sharding information to load a singe shard. only. More... | |
| RocalTensor ROCAL_API_CALL | rocalMXNetRecordSource (RocalContext context, const char *source_path, RocalImageColor rocal_color_format, unsigned internal_shard_count, bool is_output, bool shuffle=false, bool loop=false, RocalImageSizeEvaluationPolicy decode_size_policy=ROCAL_USE_MOST_FREQUENT_SIZE, unsigned max_width=0, unsigned max_height=0, RocalDecoderType rocal_decoder_type=RocalDecoderType::ROCAL_DECODER_TJPEG) |
| Creates JPEG image reader and decoder for MXNet records. It allocates the resources and objects required to read and decode Jpeg images stored in MXNet Records. It has internal sharding capability to load/decode in parallel is user wants. If images are not Jpeg compressed they will be ignored. More... | |
| RocalTensor ROCAL_API_CALL | rocalMXNetRecordSourceSingleShard (RocalContext p_context, const char *source_path, RocalImageColor rocal_color_format, unsigned shard_id, unsigned shard_count, bool is_output, bool shuffle=false, bool loop=false, RocalImageSizeEvaluationPolicy decode_size_policy=ROCAL_USE_MOST_FREQUENT_SIZE, unsigned max_width=0, unsigned max_height=0, RocalDecoderType rocal_decoder_type=RocalDecoderType::ROCAL_DECODER_TJPEG) |
| Creates JPEG image reader and decoder for MXNet records. It allocates the resources and objects required to read and decode Jpeg images stored on the MXNet records. It accepts external sharding information to load a singe shard. only. More... | |
| RocalTensor ROCAL_API_CALL | rocalFusedJpegCrop (RocalContext context, const char *source_path, RocalImageColor rocal_color_format, unsigned num_threads, bool is_output, std::vector< float > &area_factor, std::vector< float > &aspect_ratio, unsigned num_attempts, bool shuffle=false, bool loop=false, RocalImageSizeEvaluationPolicy decode_size_policy=ROCAL_USE_MOST_FREQUENT_SIZE, unsigned max_width=0, unsigned max_height=0) |
| Creates JPEG image reader and partial decoder. It allocates the resources and objects required to read and decode Jpeg images stored on the file systems. It has internal sharding capability to load/decode in parallel is user wants. If images are not Jpeg compressed they will be ignored and Crops t. More... | |
| RocalTensor ROCAL_API_CALL | rocalFusedJpegCropSingleShard (RocalContext context, const char *source_path, RocalImageColor color_format, unsigned shard_id, unsigned shard_count, bool is_output, std::vector< float > &area_factor, std::vector< float > &aspect_ratio, unsigned num_attempts, bool shuffle=false, bool loop=false, RocalImageSizeEvaluationPolicy decode_size_policy=ROCAL_USE_MOST_FREQUENT_SIZE, unsigned max_width=0, unsigned max_height=0) |
| Creates JPEG image reader and partial decoder. It allocates the resources and objects required to read and decode Jpeg images stored on the file systems. It accepts external sharding information to load a singe shard. only. More... | |
| RocalTensor ROCAL_API_CALL | rocalJpegTFRecordSource (RocalContext context, const char *source_path, RocalImageColor rocal_color_format, unsigned internal_shard_count, bool is_output, const char *user_key_for_encoded, const char *user_key_for_filename, bool shuffle=false, bool loop=false, RocalImageSizeEvaluationPolicy decode_size_policy=ROCAL_USE_MOST_FREQUENT_SIZE, unsigned max_width=0, unsigned max_height=0, RocalDecoderType rocal_decoder_type=RocalDecoderType::ROCAL_DECODER_TJPEG) |
| Creates TensorFlow records JPEG image reader and decoder. It allocates the resources and objects required to read and decode Jpeg images stored on the file systems. It has internal sharding capability to load/decode in parallel is user wants. If images are not Jpeg compressed they will be ignored. More... | |
| RocalTensor ROCAL_API_CALL | rocalJpegTFRecordSourceSingleShard (RocalContext context, const char *source_path, RocalImageColor rocal_color_format, unsigned shard_id, unsigned shard_count, bool is_output, bool shuffle=false, bool loop=false, RocalImageSizeEvaluationPolicy decode_size_policy=ROCAL_USE_MOST_FREQUENT_SIZE, unsigned max_width=0, unsigned max_height=0, RocalDecoderType rocal_decoder_type=RocalDecoderType::ROCAL_DECODER_TJPEG) |
| Creates TensorFlow records JPEG image reader and decoder. It allocates the resources and objects required to read and decode Jpeg images stored on the file systems. It accepts external sharding information to load a singe shard. only. More... | |
| RocalTensor ROCAL_API_CALL | rocalRawTFRecordSource (RocalContext p_context, const char *source_path, const char *user_key_for_raw, const char *user_key_for_filename, RocalImageColor rocal_color_format, bool is_output, bool shuffle=false, bool loop=false, unsigned out_width=0, unsigned out_height=0, const char *record_name_prefix="") |
| Creates Raw image loader. It allocates the resources and objects required to load images stored on the file systems. More... | |
| RocalTensor ROCAL_API_CALL | rocalRawTFRecordSourceSingleShard (RocalContext p_context, const char *source_path, RocalImageColor rocal_color_format, unsigned shard_id, unsigned shard_count, bool is_output, bool shuffle=false, bool loop=false, unsigned out_width=0, unsigned out_height=0, const char *record_name_prefix="") |
| Creates Raw image loader. It allocates the resources and objects required to load images stored on the file systems. More... | |
| RocalTensor ROCAL_API_CALL | rocalVideoFileSource (RocalContext context, const char *source_path, RocalImageColor color_format, RocalDecodeDevice rocal_decode_device, unsigned internal_shard_count, unsigned sequence_length, bool is_output=false, bool shuffle=false, bool loop=false, unsigned step=0, unsigned stride=0, bool file_list_frame_num=true) |
| Creates a video reader and decoder as a source. It allocates the resources and objects required to read and decode mp4 videos stored on the file systems. More... | |
| RocalTensor ROCAL_API_CALL | rocalVideoFileSourceSingleShard (RocalContext context, const char *source_path, RocalImageColor color_format, RocalDecodeDevice rocal_decode_device, unsigned shard_id, unsigned shard_count, unsigned sequence_length, bool shuffle=false, bool is_output=false, bool loop=false, unsigned step=0, unsigned stride=0, bool file_list_frame_num=true) |
| Creates a video reader and decoder as a source. It allocates the resources and objects required to read and decode mp4 videos stored on the file systems. It accepts external sharding information to load a singe shard only. More... | |
| RocalTensor ROCAL_API_CALL | rocalVideoFileResize (RocalContext context, const char *source_path, RocalImageColor color_format, RocalDecodeDevice rocal_decode_device, unsigned internal_shard_count, unsigned sequence_length, unsigned dest_width, unsigned dest_height, bool shuffle=false, bool is_output=false, bool loop=false, unsigned step=0, unsigned stride=0, bool file_list_frame_num=true, RocalResizeScalingMode scaling_mode=ROCAL_SCALING_MODE_DEFAULT, std::vector< unsigned > max_size={}, unsigned resize_shorter=0, unsigned resize_longer=0, RocalResizeInterpolationType interpolation_type=ROCAL_LINEAR_INTERPOLATION) |
| Creates a video reader and decoder as a source. It allocates the resources and objects required to read and decode mp4 videos stored on the file systems. Resizes the decoded frames to the dest width and height. More... | |
| RocalTensor ROCAL_API_CALL | rocalVideoFileResizeSingleShard (RocalContext context, const char *source_path, RocalImageColor color_format, RocalDecodeDevice rocal_decode_device, unsigned shard_id, unsigned shard_count, unsigned sequence_length, unsigned dest_width, unsigned dest_height, bool shuffle=false, bool is_output=false, bool loop=false, unsigned step=0, unsigned stride=0, bool file_list_frame_num=true, RocalResizeScalingMode scaling_mode=ROCAL_SCALING_MODE_DEFAULT, std::vector< unsigned > max_size={}, unsigned resize_shorter=0, unsigned resize_longer=0, RocalResizeInterpolationType interpolation_type=ROCAL_LINEAR_INTERPOLATION) |
| Creates a video reader and decoder as a source. It allocates the resources and objects required to read and decode mp4 videos stored on the file systems. Resizes the decoded frames to the dest width and height. It accepts external sharding information to load a singe shard only. More... | |
| RocalTensor ROCAL_API_CALL | rocalRawCIFAR10Source (RocalContext context, const char *source_path, RocalImageColor color_format, bool is_output, unsigned out_width, unsigned out_height, const char *filename_prefix="", bool loop=false) |
| Creates CIFAR10 raw data reader and loader. It allocates the resources and objects required to read raw data stored on the file systems. More... | |
| RocalStatus ROCAL_API_CALL | rocalResetLoaders (RocalContext context) |
| reset Loaders More... | |
| RocalTensor ROCAL_API_CALL | rocalJpegCaffeLMDBRecordSourcePartialSingleShard (RocalContext p_context, const char *source_path, RocalImageColor rocal_color_format, unsigned shard_id, unsigned shard_count, bool is_output, std::vector< float > &area_factor, std::vector< float > &aspect_ratio, unsigned num_attempts, bool shuffle=false, bool loop=false, RocalImageSizeEvaluationPolicy decode_size_policy=ROCAL_USE_MOST_FREQUENT_SIZE, unsigned max_width=0, unsigned max_height=0) |
| Creates JPEG image reader and partial decoder for Caffe LMDB records. It allocates the resources and objects required to read and decode Jpeg images stored in Caffe2 LMDB Records. It has internal sharding capability to load/decode in parallel is user wants. More... | |
| RocalTensor ROCAL_API_CALL | rocalJpegCaffe2LMDBRecordSourcePartialSingleShard (RocalContext p_context, const char *source_path, RocalImageColor rocal_color_format, unsigned shard_id, unsigned shard_count, bool is_output, std::vector< float > &area_factor, std::vector< float > &aspect_ratio, unsigned num_attempts, bool shuffle=false, bool loop=false, RocalImageSizeEvaluationPolicy decode_size_policy=ROCAL_USE_MOST_FREQUENT_SIZE, unsigned max_width=0, unsigned max_height=0) |
| Creates JPEG image reader and partial decoder for Caffe2 LMDB records. It allocates the resources and objects required to read and decode Jpeg images stored in Caffe22 LMDB Records. It has internal sharding capability to load/decode in parallel is user wants. More... | |
| RocalTensor ROCAL_API_CALL | rocalJpegExternalFileSource (RocalContext p_context, RocalImageColor rocal_color_format, bool is_output=false, bool shuffle=false, bool loop=false, RocalImageSizeEvaluationPolicy decode_size_policy=ROCAL_USE_MOST_FREQUENT_SIZE, unsigned max_width=0, unsigned max_height=0, RocalDecoderType rocal_decoder_type=RocalDecoderType::ROCAL_DECODER_TJPEG, RocalExternalSourceMode external_source_mode=RocalExternalSourceMode::ROCAL_EXTSOURCE_FNAME) |
| Creates JPEG external source image reader. More... | |
| RocalTensor ROCAL_API_CALL | rocalAudioFileSource (RocalContext context, const char *source_path, unsigned shard_count, bool is_output, bool shuffle=false, bool loop=false, bool downmix=false) |
| RocalTensor ROCAL_API_CALL | rocalAudioFileSourceSingleShard (RocalContext p_context, const char *source_path, unsigned shard_id, unsigned shard_count, bool is_output, bool shuffle=false, bool loop=false, bool downmix=false) |
Detailed Description
The AMD rocAL Library - Data Loaders.
Function Documentation
◆ rocalAudioFileSource()
| RocalTensor ROCAL_API_CALL rocalAudioFileSource | ( | RocalContext | context, |
| const char * | source_path, | ||
| unsigned | shard_count, | ||
| bool | is_output, | ||
| bool | shuffle = false, |
||
| bool | loop = false, |
||
| bool | downmix = false |
||
| ) |
Creates Audio file reader and decoder. It allocates the resources and objects required to read and decode audio files stored on the file systems. It has internal sharding capability to load/decode in parallel if user wants. If the files are not in standard audio compression formats they will be ignored, Currently wav format is supported
- Parameters
-
[in] context Rocal context [in] source_path A NULL terminated char string pointing to the location of files on the disk [in] shard_count Defines the parallelism level by internally sharding the input dataset and load/decode using multiple decoder/loader instances. Using shard counts bigger than 1 improves the load/decode performance if compute resources (CPU cores) are available. [in] is_output Boolean variable to enable the audio to be part of the output. [in] shuffle Boolean variable to shuffle the dataset. [in] loop Boolean variable to indefinitely loop through audio. [in] downmix Boolean variable to downmix all input channels to mono. If downmixing is turned on, the decoder output is 1D. If downmixing is turned off, it produces 2D output with interleaved channels incase of multichannel audio.
- Returns
- Reference to the output audio
◆ rocalAudioFileSourceSingleShard()
| RocalTensor ROCAL_API_CALL rocalAudioFileSourceSingleShard | ( | RocalContext | p_context, |
| const char * | source_path, | ||
| unsigned | shard_id, | ||
| unsigned | shard_count, | ||
| bool | is_output, | ||
| bool | shuffle = false, |
||
| bool | loop = false, |
||
| bool | downmix = false |
||
| ) |
Creates Audio file reader and decoder. It allocates the resources and objects required to read and decode audio files stored on the file systems. It has internal sharding capability to load/decode in parallel is user wants. If the files are not in standard audio compression formats they will be ignored.
- Parameters
-
[in] context Rocal context [in] source_path A NULL terminated char string pointing to the location of files on the disk [in] shard_id Shard id for this loader [in] shard_count Defines the parallelism level by internally sharding the input dataset and load/decode using multiple decoder/loader instances. Using shard counts bigger than 1 improves the load/decode performance if compute resources (CPU cores) are available. [in] is_output Boolean variable to enable the audio to be part of the output. [in] shuffle Boolean variable to shuffle the dataset. [in] loop Boolean variable to indefinitely loop through audio. [in] downmix Boolean variable to downmix all input channels to mono. If downmixing is turned on, the decoder output is 1D. If downmixing is turned off, it produces 2D output with interleaved channels incase of multichannel audio.
- Returns
- Reference to the output audio