Skip to content

Evaluate using Profile-Guided Optimization (PGO) and LLVM BOLT for Perspective C++ data engine #2405

Closed

Description

@zamazan4ik

Hi!

Recently I checked Profile-Guided Optimization (PGO) improvements on many projects. All current results are available here. According to the multiple tests, PGO can help with improving performance in many cases (including databases). Since Perspective cares about the performance, I think trying to optimize Perspective's C++ part with PGO would be a good idea.

I can suggest the following action points:

  • Perform PGO benchmarks on Perspective. And if it shows improvements - add a note to the documentation about possible improvements in Perspective performance with PGO.
  • Providing an easier way (e.g. a build option) to build scripts with PGO can be helpful for the end-users and maintainers since they will be able to optimize Perspective according to their own workloads.
  • Optimize pre-built Perspective binaries with PGO.

Since the Perspective native part is the library, I think the Pydantic-core experience can be reused here. Also, Clang supports PGO for shared libraries.

Maybe testing Post-Link Optimization techniques (like LLVM BOLT) would be interesting too (Clang and Rustc already use BOLT as an addition to PGO) but I recommend starting from the usual PGO.

Here are some examples of how PGO optimization is integrated in other projects:

Activity

  1. locked and limited conversation to collaborators on Oct 29, 2023
  2. converted this issue into a discussion #2406 on Oct 29, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions