ROCprofiler-SDK tool library#
The tool library utilizes APIs from rocprofiler-sdk and rocprofiler-register libraries for profiling and tracing HIP applications. This document provides information to help you design a tool by utilizing the rocprofiler-sdk and rocprofiler-register libraries efficiently. The command-line tool rocprofv3 is also built on librocprofiler-sdk-tool.so.X.Y.Z, which uses these libraries.
ROCm runtimes design#
The ROCm runtimes are designed to directly communicate with a helper library named rocprofiler-register during initialization. This library performs cursory checks to find if a tool requires ROCprofiler-SDK services. This detection is based on the presence of one or more instances of rocprofiler_configure in the tool or ROCP_TOOL_LIBRARIES environment variable. This design provides drastic improvement over previous designs, which relied solely on a tool racing to set runtime-specific environment variables like HSA_TOOLS_LIB before the runtime initialization.
Tool library design#
When ROCprofiler-SDK detects rocprofiler_configure in a tool’s symbol table, ROCprofiler-SDK invokes rocprofiler-configure with parameters such as ROCprofiler-SDK version that invokes the function, number of tools already invoked, and a unique identifier for the tool. The tool returns a pointer to a rocprofiler_tool_configure_result_t struct, which, if non-null, provides ROCprofiler-SDK with:
Function to be called for tool initialization, which is also the opportunity for context creation.
Function to be called when ROCprofiler-SDK is finalized.
A pointer to data to be provided to the tool when ROCprofiler-SDK calls the initialization and finalization functions.
ROCprofiler-SDK provides a rocprofiler-sdk/registration.h header file, which forward declares the rocprofiler_configure function with the necessary compiler function attributes to ensure that the rocprofiler-configure symbol is publicly visible.
#include <rocprofiler-sdk/registration.h>
namespace
{
// saves the data provided to rocprofiler_configure
struct ToolData
{
uint32_t version;
const char* runtime_version;
uint32_t priority;
rocprofiler_client_id_t client_id;
};
// tool initialization function
int
tool_init(rocprofiler_client_finalize_t fini_func,
void* tool_data_v);
// tool finalization function
void
tool_fini(void* tool_data_v);
}
extern "C"
{
rocprofiler_tool_configure_result_t*
rocprofiler_configure(uint32_t version,
const char* runtime_version,
uint32_t priority,
rocprofiler_client_id_t* client_id)
{
//If not the first tool to register, indicate that the tool doesn't want to do anything
if(priority > 0) return nullptr;
// (optional) Provide a name for this tool to rocprofiler
client_id->name = "ExampleTool";
// (optional) create configure data
static auto data = ToolData{ version,
runtime_version,
priority,
client_id };
// construct configure result
static auto cfg =
rocprofiler_tool_configure_result_t{ sizeof(rocprofiler_tool_configure_result_t),
&tool_init,
&tool_fini,
static_cast<void*>(&data) };
return &cfg;
}
Tool initialization#
Note
ROCprofiler-SDK does NOT support calls to any runtime function (HSA, HIP, and so on) during tool initialization. Invoking any functions from the runtimes results in a deadlock.
For each tool that contains a rocprofiler_configure function and returns a non-null pointer to a rocprofiler_tool_configure_result_t struct, ROCprofiler-SDK invokes the initialize callback after completing the scan for all rocprofiler_configure symbols. In other words, ROCprofiler-SDK
collects all rocprofiler_tool_configure_result_t instances before invoking the initialize member of any of these instances.
When ROCprofiler-SDK invokes initialize function in a tool, this is the opportunity to create contexts:
#include <rocprofiler-sdk/rocprofiler.h>
namespace
{
int
tool_init(rocprofiler_client_finalize_t fini_func,
void* data_v)
{
// create a context
auto ctx = rocprofiler_context_id_t{0};
rocprofiler_create_context(&ctx);
// ... associate services with context ...
// start the context (optional)
rocprofiler_start_context(ctx);
return 0;
}
}
Although not mandatory, it is recommended that tools store the context handles to control the data collection for the services associated with the context.
Tool finalization#
When the initialize callback is invoked in the tool, ROCprofiler-SDK provides a function pointer of type rocprofiler_client_finalize_t.
The tool can invoke this function pointer to explicitly invoke the finalize callback from the rocprofiler_tool_configure_result_t instance:
#include <rocprofiler-sdk/rocprofiler.h>
namespace
{
int
tool_init(rocprofiler_client_finalize_t fini_func,
void* data_v)
{
// ... see initialization section ...
// function, which finalizes the tool after 10 seconds
auto explicit_finalize = [](rocprofiler_client_finalize_t finalizer,
rocprofiler_client_id_t* client_id)
{
std::this_thread::sleep_for(std::chrono::seconds{ 10 });
finalizer(client_id);
};
// start the context
rocprofiler_start_context(ctx);
// dispatch a background thread to explicitly finalize after 10 seconds
std::thread{ explicit_finalize, fini_func, static_cast<ToolData*>(data_v)->client_id }.detach();
return 0;
}
}
Otherwise, ROCprofiler-SDK invokes the finalize callback via an atexit handler.
Full rocprofiler-configure sample#
All the code snippets from the previous sections are combined here to demonstrate complete ROCProfiler configuration.
#include <rocprofiler-sdk/registration.h>
namespace
{
struct rocp_tool_data
{
uint32_t version;
const char* runtime_version;
uint32_t priority;
rocprofiler_client_id_t client_id;
rocprofiler_client_finalize_t finalizer;
std::vector<rocprofiler_context_id_t> contexts;
};
void
tool_tracing_callback(rocprofiler_callback_tracing_record_t record,
rocprofiler_user_data_t* user_data,
void* callback_data);
int
tool_init(rocprofiler_client_finalize_t fini_func,
void* tool_data_v)
{
rocp_tool_data* tool_data = static_cast<rocp_tool_data*>(tool_data_v);
// Save the finalizer function
tool_data->finalizer = fini_func;
// create a context
auto ctx = rocprofiler_context_id_t{0};
rocprofiler_create_context(&ctx);
// Save your contexts
tool_data->contexts.emplace_back(ctx);
// Associate code object tracing with this context
rocprofiler_configure_callback_tracing_service(
ctx,
ROCPROFILER_CALLBACK_TRACING_CODE_OBJECT,
nullptr,
0,
tool_tracing_callback,
tool_data);
// ... Associate services with contexts ...
return 0;
}
void
tool_fini(void* tool_data);
}
extern "C"
{
rocprofiler_tool_configure_result_t*
rocprofiler_configure(uint32_t version,
const char* runtime_version,
uint32_t priority,
rocprofiler_client_id_t* client_id)
{
// (optional) Provide a name for this tool to rocprofiler
client_id->name = "ExampleTool";
// Info provided back to tool_init and tool_fini
auto* my_tool_data = new rocp_tool_data{ version,
runtime_version,
priority,
client_id,
nullptr };
// Create configure data
static auto cfg =
rocprofiler_tool_configure_result_t{ sizeof(rocprofiler_tool_configure_result_t),
&tool_init,
&tool_fini,
my_tool_data };
return &cfg;
}