/home/docs/checkouts/readthedocs.org/user_builds/advanced-micro-devices-rocal/checkouts/docs-6.1.1/rocAL/include/api/rocal_api_data_loaders.h File Reference

/home/docs/checkouts/readthedocs.org/user_builds/advanced-micro-devices-rocal/checkouts/docs-6.1.1/rocAL/include/api/rocal_api_data_loaders.h File Reference#

rocAL: /home/docs/checkouts/readthedocs.org/user_builds/advanced-micro-devices-rocal/checkouts/docs-6.1.1/rocAL/include/api/rocal_api_data_loaders.h File Reference

The AMD rocAL Library - Data Loaders. More...

#include "rocal_api_types.h"

Go to the source code of this file.

Functions
RocalTensor ROCAL_API_CALL	rocalJpegFileSource (RocalContext context, const char *source_path, RocalImageColor rocal_color_format, unsigned internal_shard_count, bool is_output, bool shuffle=false, bool loop=false, RocalImageSizeEvaluationPolicy decode_size_policy=ROCAL_USE_MOST_FREQUENT_SIZE, unsigned max_width=0, unsigned max_height=0, RocalDecoderType rocal_decoder_type=RocalDecoderType::ROCAL_DECODER_TJPEG)
	Creates JPEG image reader and decoder. It allocates the resources and objects required to read and decode Jpeg images stored on the file systems. It has internal sharding capability to load/decode in parallel is user wants. If images are not Jpeg compressed they will be ignored. More...

RocalTensor ROCAL_API_CALL	rocalJpegFileSourceSingleShard (RocalContext context, const char *source_path, RocalImageColor rocal_color_format, unsigned shard_id, unsigned shard_count, bool is_output, bool shuffle=false, bool loop=false, RocalImageSizeEvaluationPolicy decode_size_policy=ROCAL_USE_MOST_FREQUENT_SIZE, unsigned max_width=0, unsigned max_height=0, RocalDecoderType rocal_decoder_type=RocalDecoderType::ROCAL_DECODER_TJPEG)
	Creates JPEG image reader and decoder. It allocates the resources and objects required to read and decode Jpeg images stored on the file systems. It accepts external sharding information to load a singe shard. only. More...

RocalTensor ROCAL_API_CALL	rocalSequenceReader (RocalContext context, const char *source_path, RocalImageColor rocal_color_format, unsigned internal_shard_count, unsigned sequence_length, bool is_output, bool shuffle=false, bool loop=false, unsigned step=0, unsigned stride=0)
	Creates JPEG image reader and decoder. Reads [Frames] sequences from a directory representing a collection of streams. More...

RocalTensor ROCAL_API_CALL	rocalSequenceReaderSingleShard (RocalContext context, const char *source_path, RocalImageColor rocal_color_format, unsigned shard_id, unsigned shard_count, unsigned sequence_length, bool is_output, bool shuffle=false, bool loop=false, unsigned step=0, unsigned stride=0)
	Creates JPEG image reader and decoder. Reads [Frames] sequences from a directory representing a collection of streams. It accepts external sharding information to load a singe shard only. More...

RocalTensor ROCAL_API_CALL	rocalJpegCOCOFileSource (RocalContext context, const char source_path, const char json_path, RocalImageColor color_format, unsigned internal_shard_count, bool is_output, bool shuffle=false, bool loop=false, RocalImageSizeEvaluationPolicy decode_size_policy=ROCAL_USE_MOST_FREQUENT_SIZE, unsigned max_width=0, unsigned max_height=0, RocalDecoderType rocal_decoder_type=RocalDecoderType::ROCAL_DECODER_TJPEG)
	JPEG image reader and decoder. It allocates the resources and objects required to read and decode COCO Jpeg images stored on the file systems. It has internal sharding capability to load/decode in parallel is user wants. If images are not Jpeg compressed they will be ignored. More...

RocalTensor ROCAL_API_CALL	rocalJpegCOCOFileSourcePartial (RocalContext p_context, const char source_path, const char json_path, RocalImageColor rocal_color_format, unsigned internal_shard_count, bool is_output, std::vector< float > &area_factor, std::vector< float > &aspect_ratio, unsigned num_attempts, bool shuffle=false, bool loop=false, RocalImageSizeEvaluationPolicy decode_size_policy=ROCAL_USE_MOST_FREQUENT_SIZE, unsigned max_width=0, unsigned max_height=0)
	JPEG image reader and partial decoder. It allocates the resources and objects required to read and decode COCO Jpeg images stored on the file systems. It has internal sharding capability to load/decode in parallel is user wants. If images are not Jpeg compressed they will be ignored. More...

RocalTensor ROCAL_API_CALL	rocalJpegCOCOFileSourcePartialSingleShard (RocalContext p_context, const char source_path, const char json_path, RocalImageColor rocal_color_format, unsigned shard_id, unsigned shard_count, bool is_output, std::vector< float > &area_factor, std::vector< float > &aspect_ratio, unsigned num_attempts, bool shuffle=false, bool loop=false, RocalImageSizeEvaluationPolicy decode_size_policy=ROCAL_USE_MOST_FREQUENT_SIZE, unsigned max_width=0, unsigned max_height=0)
	Creates JPEG image reader and partial decoder. It allocates the resources and objects required to read and decode COCO Jpeg images stored on the file systems. It has internal sharding capability to load/decode in parallel is user wants. If images are not Jpeg compressed they will be ignored. More...

RocalTensor ROCAL_API_CALL	rocalJpegCOCOFileSourceSingleShard (RocalContext context, const char source_path, const char json_path, RocalImageColor color_format, unsigned shard_id, unsigned shard_count, bool is_output, bool shuffle=false, bool loop=false, RocalImageSizeEvaluationPolicy decode_size_policy=ROCAL_USE_MOST_FREQUENT_SIZE, unsigned max_width=0, unsigned max_height=0, RocalDecoderType rocal_decoder_type=RocalDecoderType::ROCAL_DECODER_TJPEG)
	Creates JPEG image reader. It allocates the resources and objects required to read and decode COCO Jpeg images stored on the file systems. It has internal sharding capability to load/decode in parallel is user wants. If images are not Jpeg compressed they will be ignored. More...

RocalTensor ROCAL_API_CALL	rocalJpegCaffeLMDBRecordSource (RocalContext context, const char *source_path, RocalImageColor rocal_color_format, unsigned internal_shard_count, bool is_output, bool shuffle=false, bool loop=false, RocalImageSizeEvaluationPolicy decode_size_policy=ROCAL_USE_MOST_FREQUENT_SIZE, unsigned max_width=0, unsigned max_height=0, RocalDecoderType rocal_decoder_type=RocalDecoderType::ROCAL_DECODER_TJPEG)
	Creates JPEG image reader and decoder for Caffe LMDB records. It allocates the resources and objects required to read and decode Jpeg images stored in Caffe LMDB Records. It has internal sharding capability to load/decode in parallel is user wants. If images are not Jpeg compressed they will be ignored. More...

RocalTensor ROCAL_API_CALL	rocalJpegCaffeLMDBRecordSourceSingleShard (RocalContext p_context, const char *source_path, RocalImageColor rocal_color_format, unsigned shard_id, unsigned shard_count, bool is_output, bool shuffle=false, bool loop=false, RocalImageSizeEvaluationPolicy decode_size_policy=ROCAL_USE_MOST_FREQUENT_SIZE, unsigned max_width=0, unsigned max_height=0, RocalDecoderType rocal_decoder_type=RocalDecoderType::ROCAL_DECODER_TJPEG)
	Creates JPEG image reader and decoder for Caffe LMDB records. It allocates the resources and objects required to read and decode Jpeg images stored in Caffe2 LMDB Records. It has internal sharding capability to load/decode in parallel is user wants. More...

RocalTensor ROCAL_API_CALL	rocalJpegCaffe2LMDBRecordSource (RocalContext context, const char *source_path, RocalImageColor rocal_color_format, unsigned internal_shard_count, bool is_output, bool shuffle=false, bool loop=false, RocalImageSizeEvaluationPolicy decode_size_policy=ROCAL_USE_MOST_FREQUENT_SIZE, unsigned max_width=0, unsigned max_height=0, RocalDecoderType rocal_decoder_type=RocalDecoderType::ROCAL_DECODER_TJPEG)
	Creates JPEG image reader and decoder for Caffe2 LMDB records. It allocates the resources and objects required to read and decode Jpeg images stored in Caffe2 LMDB Records. It has internal sharding capability to load/decode in parallel is user wants. If images are not Jpeg compressed they will be ignored. More...

RocalTensor ROCAL_API_CALL	rocalJpegCaffe2LMDBRecordSourceSingleShard (RocalContext p_context, const char *source_path, RocalImageColor rocal_color_format, unsigned shard_id, unsigned shard_count, bool is_output, bool shuffle=false, bool loop=false, RocalImageSizeEvaluationPolicy decode_size_policy=ROCAL_USE_MOST_FREQUENT_SIZE, unsigned max_width=0, unsigned max_height=0, RocalDecoderType rocal_decoder_type=RocalDecoderType::ROCAL_DECODER_TJPEG)
	Creates JPEG image reader and decoder for Caffe2 LMDB records. It allocates the resources and objects required to read and decode Jpeg images stored on the Caffe2 LMDB Records. It accepts external sharding information to load a singe shard. only. More...

RocalTensor ROCAL_API_CALL	rocalMXNetRecordSource (RocalContext context, const char *source_path, RocalImageColor rocal_color_format, unsigned internal_shard_count, bool is_output, bool shuffle=false, bool loop=false, RocalImageSizeEvaluationPolicy decode_size_policy=ROCAL_USE_MOST_FREQUENT_SIZE, unsigned max_width=0, unsigned max_height=0, RocalDecoderType rocal_decoder_type=RocalDecoderType::ROCAL_DECODER_TJPEG)
	Creates JPEG image reader and decoder for MXNet records. It allocates the resources and objects required to read and decode Jpeg images stored in MXNet Records. It has internal sharding capability to load/decode in parallel is user wants. If images are not Jpeg compressed they will be ignored. More...

RocalTensor ROCAL_API_CALL	rocalMXNetRecordSourceSingleShard (RocalContext p_context, const char *source_path, RocalImageColor rocal_color_format, unsigned shard_id, unsigned shard_count, bool is_output, bool shuffle=false, bool loop=false, RocalImageSizeEvaluationPolicy decode_size_policy=ROCAL_USE_MOST_FREQUENT_SIZE, unsigned max_width=0, unsigned max_height=0, RocalDecoderType rocal_decoder_type=RocalDecoderType::ROCAL_DECODER_TJPEG)
	Creates JPEG image reader and decoder for MXNet records. It allocates the resources and objects required to read and decode Jpeg images stored on the MXNet records. It accepts external sharding information to load a singe shard. only. More...

RocalTensor ROCAL_API_CALL	rocalFusedJpegCrop (RocalContext context, const char *source_path, RocalImageColor rocal_color_format, unsigned num_threads, bool is_output, std::vector< float > &area_factor, std::vector< float > &aspect_ratio, unsigned num_attempts, bool shuffle=false, bool loop=false, RocalImageSizeEvaluationPolicy decode_size_policy=ROCAL_USE_MOST_FREQUENT_SIZE, unsigned max_width=0, unsigned max_height=0)
	Creates JPEG image reader and partial decoder. It allocates the resources and objects required to read and decode Jpeg images stored on the file systems. It has internal sharding capability to load/decode in parallel is user wants. If images are not Jpeg compressed they will be ignored and Crops t. More...

RocalTensor ROCAL_API_CALL	rocalFusedJpegCropSingleShard (RocalContext context, const char *source_path, RocalImageColor color_format, unsigned shard_id, unsigned shard_count, bool is_output, std::vector< float > &area_factor, std::vector< float > &aspect_ratio, unsigned num_attempts, bool shuffle=false, bool loop=false, RocalImageSizeEvaluationPolicy decode_size_policy=ROCAL_USE_MOST_FREQUENT_SIZE, unsigned max_width=0, unsigned max_height=0)
	Creates JPEG image reader and partial decoder. It allocates the resources and objects required to read and decode Jpeg images stored on the file systems. It accepts external sharding information to load a singe shard. only. More...

RocalTensor ROCAL_API_CALL	rocalJpegTFRecordSource (RocalContext context, const char source_path, RocalImageColor rocal_color_format, unsigned internal_shard_count, bool is_output, const char user_key_for_encoded, const char *user_key_for_filename, bool shuffle=false, bool loop=false, RocalImageSizeEvaluationPolicy decode_size_policy=ROCAL_USE_MOST_FREQUENT_SIZE, unsigned max_width=0, unsigned max_height=0, RocalDecoderType rocal_decoder_type=RocalDecoderType::ROCAL_DECODER_TJPEG)
	Creates TensorFlow records JPEG image reader and decoder. It allocates the resources and objects required to read and decode Jpeg images stored on the file systems. It has internal sharding capability to load/decode in parallel is user wants. If images are not Jpeg compressed they will be ignored. More...

RocalTensor ROCAL_API_CALL	rocalJpegTFRecordSourceSingleShard (RocalContext context, const char *source_path, RocalImageColor rocal_color_format, unsigned shard_id, unsigned shard_count, bool is_output, bool shuffle=false, bool loop=false, RocalImageSizeEvaluationPolicy decode_size_policy=ROCAL_USE_MOST_FREQUENT_SIZE, unsigned max_width=0, unsigned max_height=0, RocalDecoderType rocal_decoder_type=RocalDecoderType::ROCAL_DECODER_TJPEG)
	Creates TensorFlow records JPEG image reader and decoder. It allocates the resources and objects required to read and decode Jpeg images stored on the file systems. It accepts external sharding information to load a singe shard. only. More...

RocalTensor ROCAL_API_CALL	rocalRawTFRecordSource (RocalContext p_context, const char source_path, const char user_key_for_raw, const char user_key_for_filename, RocalImageColor rocal_color_format, bool is_output, bool shuffle=false, bool loop=false, unsigned out_width=0, unsigned out_height=0, const char record_name_prefix="")
	Creates Raw image loader. It allocates the resources and objects required to load images stored on the file systems. More...

RocalTensor ROCAL_API_CALL	rocalRawTFRecordSourceSingleShard (RocalContext p_context, const char source_path, RocalImageColor rocal_color_format, unsigned shard_id, unsigned shard_count, bool is_output, bool shuffle=false, bool loop=false, unsigned out_width=0, unsigned out_height=0, const char record_name_prefix="")
	Creates Raw image loader. It allocates the resources and objects required to load images stored on the file systems. More...

RocalTensor ROCAL_API_CALL	rocalVideoFileSource (RocalContext context, const char *source_path, RocalImageColor color_format, RocalDecodeDevice rocal_decode_device, unsigned internal_shard_count, unsigned sequence_length, bool is_output=false, bool shuffle=false, bool loop=false, unsigned step=0, unsigned stride=0, bool file_list_frame_num=true)
	Creates a video reader and decoder as a source. It allocates the resources and objects required to read and decode mp4 videos stored on the file systems. More...

RocalTensor ROCAL_API_CALL	rocalVideoFileSourceSingleShard (RocalContext context, const char *source_path, RocalImageColor color_format, RocalDecodeDevice rocal_decode_device, unsigned shard_id, unsigned shard_count, unsigned sequence_length, bool shuffle=false, bool is_output=false, bool loop=false, unsigned step=0, unsigned stride=0, bool file_list_frame_num=true)
	Creates a video reader and decoder as a source. It allocates the resources and objects required to read and decode mp4 videos stored on the file systems. It accepts external sharding information to load a singe shard only. More...

RocalTensor ROCAL_API_CALL	rocalVideoFileResize (RocalContext context, const char *source_path, RocalImageColor color_format, RocalDecodeDevice rocal_decode_device, unsigned internal_shard_count, unsigned sequence_length, unsigned dest_width, unsigned dest_height, bool shuffle=false, bool is_output=false, bool loop=false, unsigned step=0, unsigned stride=0, bool file_list_frame_num=true, RocalResizeScalingMode scaling_mode=ROCAL_SCALING_MODE_DEFAULT, std::vector< unsigned > max_size={}, unsigned resize_shorter=0, unsigned resize_longer=0, RocalResizeInterpolationType interpolation_type=ROCAL_LINEAR_INTERPOLATION)
	Creates a video reader and decoder as a source. It allocates the resources and objects required to read and decode mp4 videos stored on the file systems. Resizes the decoded frames to the dest width and height. More...

RocalTensor ROCAL_API_CALL	rocalVideoFileResizeSingleShard (RocalContext context, const char *source_path, RocalImageColor color_format, RocalDecodeDevice rocal_decode_device, unsigned shard_id, unsigned shard_count, unsigned sequence_length, unsigned dest_width, unsigned dest_height, bool shuffle=false, bool is_output=false, bool loop=false, unsigned step=0, unsigned stride=0, bool file_list_frame_num=true, RocalResizeScalingMode scaling_mode=ROCAL_SCALING_MODE_DEFAULT, std::vector< unsigned > max_size={}, unsigned resize_shorter=0, unsigned resize_longer=0, RocalResizeInterpolationType interpolation_type=ROCAL_LINEAR_INTERPOLATION)
	Creates a video reader and decoder as a source. It allocates the resources and objects required to read and decode mp4 videos stored on the file systems. Resizes the decoded frames to the dest width and height. It accepts external sharding information to load a singe shard only. More...

RocalTensor ROCAL_API_CALL	rocalRawCIFAR10Source (RocalContext context, const char source_path, RocalImageColor color_format, bool is_output, unsigned out_width, unsigned out_height, const char filename_prefix="", bool loop=false)
	Creates CIFAR10 raw data reader and loader. It allocates the resources and objects required to read raw data stored on the file systems. More...

RocalStatus ROCAL_API_CALL	rocalResetLoaders (RocalContext context)
	reset Loaders More...

RocalTensor ROCAL_API_CALL	rocalJpegCaffeLMDBRecordSourcePartialSingleShard (RocalContext p_context, const char *source_path, RocalImageColor rocal_color_format, unsigned shard_id, unsigned shard_count, bool is_output, std::vector< float > &area_factor, std::vector< float > &aspect_ratio, unsigned num_attempts, bool shuffle=false, bool loop=false, RocalImageSizeEvaluationPolicy decode_size_policy=ROCAL_USE_MOST_FREQUENT_SIZE, unsigned max_width=0, unsigned max_height=0)
	Creates JPEG image reader and partial decoder for Caffe LMDB records. It allocates the resources and objects required to read and decode Jpeg images stored in Caffe2 LMDB Records. It has internal sharding capability to load/decode in parallel is user wants. More...

RocalTensor ROCAL_API_CALL	rocalJpegCaffe2LMDBRecordSourcePartialSingleShard (RocalContext p_context, const char *source_path, RocalImageColor rocal_color_format, unsigned shard_id, unsigned shard_count, bool is_output, std::vector< float > &area_factor, std::vector< float > &aspect_ratio, unsigned num_attempts, bool shuffle=false, bool loop=false, RocalImageSizeEvaluationPolicy decode_size_policy=ROCAL_USE_MOST_FREQUENT_SIZE, unsigned max_width=0, unsigned max_height=0)
	Creates JPEG image reader and partial decoder for Caffe2 LMDB records. It allocates the resources and objects required to read and decode Jpeg images stored in Caffe22 LMDB Records. It has internal sharding capability to load/decode in parallel is user wants. More...

RocalTensor ROCAL_API_CALL	rocalJpegExternalFileSource (RocalContext p_context, RocalImageColor rocal_color_format, bool is_output=false, bool shuffle=false, bool loop=false, RocalImageSizeEvaluationPolicy decode_size_policy=ROCAL_USE_MOST_FREQUENT_SIZE, unsigned max_width=0, unsigned max_height=0, RocalDecoderType rocal_decoder_type=RocalDecoderType::ROCAL_DECODER_TJPEG, RocalExternalSourceMode external_source_mode=RocalExternalSourceMode::ROCAL_EXTSOURCE_FNAME)
	Creates JPEG external source image reader. More...

RocalTensor ROCAL_API_CALL	rocalAudioFileSource (RocalContext context, const char *source_path, unsigned shard_count, bool is_output, bool shuffle=false, bool loop=false, bool downmix=false)

RocalTensor ROCAL_API_CALL	rocalAudioFileSourceSingleShard (RocalContext p_context, const char *source_path, unsigned shard_id, unsigned shard_count, bool is_output, bool shuffle=false, bool loop=false, bool downmix=false)

Detailed Description

The AMD rocAL Library - Data Loaders.

Function Documentation

◆ rocalAudioFileSource()

RocalTensor ROCAL_API_CALL rocalAudioFileSource	(	RocalContext	context,
		const char *	source_path,
		unsigned	shard_count,
		bool	is_output,
		bool	shuffle = `false`,
		bool	loop = `false`,
		bool	downmix = `false`
	)

Creates Audio file reader and decoder. It allocates the resources and objects required to read and decode audio files stored on the file systems. It has internal sharding capability to load/decode in parallel if user wants. If the files are not in standard audio compression formats they will be ignored, Currently wav format is supported

Parameters

[in]	context	Rocal context
[in]	source_path	A NULL terminated char string pointing to the location of files on the disk
[in]	shard_count	Defines the parallelism level by internally sharding the input dataset and load/decode using multiple decoder/loader instances. Using shard counts bigger than 1 improves the load/decode performance if compute resources (CPU cores) are available.
[in]	is_output	Boolean variable to enable the audio to be part of the output.
[in]	shuffle	Boolean variable to shuffle the dataset.
[in]	loop	Boolean variable to indefinitely loop through audio.
[in]	downmix	Boolean variable to downmix all input channels to mono. If downmixing is turned on, the decoder output is 1D. If downmixing is turned off, it produces 2D output with interleaved channels incase of multichannel audio.

Returns: Reference to the output audio

◆ rocalAudioFileSourceSingleShard()

RocalTensor ROCAL_API_CALL rocalAudioFileSourceSingleShard	(	RocalContext	p_context,
		const char *	source_path,
		unsigned	shard_id,
		unsigned	shard_count,
		bool	is_output,
		bool	shuffle = `false`,
		bool	loop = `false`,
		bool	downmix = `false`
	)

Creates Audio file reader and decoder. It allocates the resources and objects required to read and decode audio files stored on the file systems. It has internal sharding capability to load/decode in parallel is user wants. If the files are not in standard audio compression formats they will be ignored.