Tool library#

The tool library utilizes APIs from rocprofiler-sdk and rocprofiler-register libraries for profiling and tracing HIP applications. This document provides information to help you design a tool by utilizing the rocprofiler-sdk and rocprofiler-register libraries efficiently. The command-line tool rocprofv3 is also built on librocprofiler-sdk-tool.so.0.4.0, which uses these libraries.

ROCm runtimes design#

The ROCm runtimes are designed to directly communicate with a helper library named rocprofiler-register during initialization. This library performs cursory checks to find if a tool requires ROCprofiler-SDK services. This detection is based on the presence of one or more instances of rocprofiler_configure in the tool or ROCP_TOOL_LIBRARIES environment variable. This design provides drastic improvement over previous designs, which relied solely on a tool racing to set runtime-specific environment variables like HSA_TOOLS_LIB before the runtime initialization.

Tool library design#

When ROCprofiler-SDK detects rocprofiler_configure in a tool’s symbol table, ROCprofiler-SDK invokes rocprofiler-configure with parameters such as ROCprofiler-SDK version that invokes the function, number of tools already invoked, and a unique identifier for the tool. The tool returns a pointer to a rocprofiler_tool_configure_result_t struct, which, if non-null, provides ROCprofiler-SDK with:

  • Function to be called for tool initialization, which is also the opportunity for context creation.

  • Function to be called when ROCprofiler-SDK is finalized.

  • A pointer to data to be provided to the tool when ROCprofiler-SDK calls the initialization and finalization functions.

ROCprofiler-SDK provides a rocprofiler-sdk/registration.h header file, which forward declares the rocprofiler_configure function with the necessary compiler function attributes to ensure that the rocprofiler-configure symbol is publicly visible.

#include <rocprofiler-sdk/registration.h>

namespace
{
// saves the data provided to rocprofiler_configure
struct ToolData
{
    uint32_t                              version;
    const char*                           runtime_version;
    uint32_t                              priority;
    rocprofiler_client_id_t               client_id;
};

// tool initialization function
int
tool_init(rocprofiler_client_finalize_t fini_func,
          void* tool_data_v);

// tool finalization function
void
tool_fini(void* tool_data_v);
}

extern "C"
{
rocprofiler_tool_configure_result_t*
rocprofiler_configure(uint32_t                 version,
                      const char*              runtime_version,
                      uint32_t                 priority,
                      rocprofiler_client_id_t* client_id)
{
    //If not the first tool to register, indicate that the tool doesn't want to do anything
    if(priority > 0) return nullptr;

    // (optional) Provide a name for this tool to rocprofiler
    client_id->name = "ExampleTool";

    // (optional) create configure data
    static auto data = ToolData{ version,
                                 runtime_version,
                                 priority,
                                 client_id };

    // construct configure result
    static auto cfg =
        rocprofiler_tool_configure_result_t{ sizeof(rocprofiler_tool_configure_result_t),
                                             &tool_init,
                                             &tool_fini,
                                             static_cast<void*>(&data) };

    return &cfg;
}

Tool initialization#

Note

ROCprofiler-SDK does NOT support calls to any runtime function (HSA, HIP, and so on) during tool initialization. Invoking any functions from the runtimes results in a deadlock.

For each tool that contains a rocprofiler_configure function and returns a non-null pointer to a rocprofiler_tool_configure_result_t struct, ROCprofiler-SDK invokes the initialize callback after completing the scan for all rocprofiler_configure symbols. In other words, ROCprofiler-SDK collects all rocprofiler_tool_configure_result_t instances before invoking the initialize member of any of these instances. When ROCprofiler-SDK invokes initialize function in a tool, this is the opportunity to create contexts:

#include <rocprofiler-sdk/rocprofiler.h>

namespace
{
int
tool_init(rocprofiler_client_finalize_t fini_func,
          void* data_v)
{
    // create a context
    auto ctx = rocprofiler_context_id_t{};
    rocprofiler_create_context(&ctx);

    // ... associate services with context ...

    // start the context (optional)
    rocprofiler_start_context(ctx);

    return 0;
}
}

Although not mandatory, it is recommended that tools store the context handles to control the data collection for the services associated with the context.

Tool finalization#

When the initialize callback is invoked in the tool, ROCprofiler-SDK provides a function pointer of type rocprofiler_client_finalize_t. The tool can invoke this function pointer to explicitly invoke the finalize callback from the rocprofiler_tool_configure_result_t instance:

#include <rocprofiler-sdk/rocprofiler.h>

namespace
{
int
tool_init(rocprofiler_client_finalize_t fini_func,
          void* data_v)
{
    // ... see initialization section ...

    // function, which finalizes the tool after 10 seconds
    auto explicit_finalize = [](rocprofiler_client_finalize_t finalizer,
                                rocprofiler_client_id_t* client_id)
    {
        std::this_thread::sleep_for(std::chrono::seconds{ 10 });
        finalizer(client_id);
    };

    // start the context
    rocprofiler_start_context(ctx);

    // dispatch a background thread to explicitly finalize after 10 seconds
    std::thread{ explicit_finalize, fini_func, static_cast<ToolData*>(data_v)->client_id }.detach();

    return 0;
}
}

Otherwise, ROCprofiler-SDK invokes the finalize callback via an atexit handler.

Full rocprofiler_configure Sample#

All of the snippets from the previous sections have been combined here for convenience.

#include <rocprofiler-sdk/registration.h>

namespace
{
struct rocp_tool_data
{
    uint32_t                              version;
    const char*                           runtime_version;
    uint32_t                              priority;
    rocprofiler_client_id_t               client_id;
    rocprofiler_client_finalize_t         finalizer;
    std::vector<rocprofiler_context_id_t> contexts;
};

void
tool_tracing_callback(rocprofiler_callback_tracing_record_t record,
                      rocprofiler_user_data_t*              user_data,
                      void*                                 callback_data);

int
tool_init(rocprofiler_client_finalize_t fini_func,
          void* tool_data_v)
{
    rocp_tool_data* tool_data = static_cast<rocp_tool_data*>(tool_data_v);

    // Save the finalizer function
    tool_data->finalizer = fini_func;

    // create a context
    auto ctx = rocprofiler_context_id_t{};
    rocprofiler_create_context(&ctx);

    // Save your contexts
    tool_data->contexts.emplace_back(ctx);

    // associate code object tracing with this context
    rocprofiler_configure_callback_tracing_service(
        ctx,
        ROCPROFILER_CALLBACK_TRACING_CODE_OBJECT,
        nullptr,
        0,
        tool_tracing_callback,
        tool_data);

    // ... associate services with contexts ...

    return 0;
}

void
tool_fini(void* tool_data);
}

extern "C"
{
rocprofiler_tool_configure_result_t*
rocprofiler_configure(uint32_t                 version,
                      const char*              runtime_version,
                      uint32_t                 priority,
                      rocprofiler_client_id_t* client_id)
{
    // (optional) Provide a name for this tool to rocprofiler
    client_id->name = "ExampleTool";

    // info provided back to tool_init and tool_fini
    auto* my_tool_data = new rocp_tool_data{ version,
                                             runtime_version,
                                             priority,
                                             client_id,
                                             nullptr };

    // create configure data
    static auto cfg =
        rocprofiler_tool_configure_result_t{ sizeof(rocprofiler_tool_configure_result_t),
                                             &tool_init,
                                             &tool_fini,
                                             my_tool_data };

    return &cfg;
}