Skip to content

link against jemalloc #36

Closed
Closed

Description

@nickdesaulniers

not sure if it would make a difference, but I noticed that in bolt's dockerfile they're linking llvm against jemalloc.
https://github.com/llvm/llvm-project/blob/main/bolt/utils/docker/Dockerfile#L23
That might be interesting to try, perhaps if any of the memory allocations are showing up in profiles of kernel builds. I guess we're using whatever malloc implementation musl has? I wonder if it has its own? "mallocng?" https://git.musl-libc.org/cgit/musl/tree/src/malloc/mallocng/malloc.c

Activity

  1. nickdesaulniers commented on Jun 17, 2022

    @nickdesaulniers
    MemberAuthor
  2. nickdesaulniers commented on Jun 17, 2022

    @nickdesaulniers
    MemberAuthor

    Here's a profile of the kernel build with stage5 pgo:

    Samples: 13M of event 'cycles:ppu', Event count (approx.): 9699067643159                                            
      Overhead  Shared Object              Symbol                                                                       
    +    2.91%  clang                      [.] clang::SourceManager::getFileIDLocal
    +    2.15%  clang                      [.] llvm::StringMapImpl::LookupBucketFor
    +    1.72%  clang                      [.] __libc_malloc_impl
    +    1.61%  clang                      [.] clang::TokenLexer::Lex
    +    1.49%  clang                      [.] clang::Lexer::LexTokenInternal
    +    1.40%  clang                      [.] memcpy
    +    1.00%  clang                      [.] clang::Preprocessor::Lex
    +    0.99%  clang                      [.] clang::ASTContext::getDeclAttrs
    +    0.95%  clang                      [.] clang::Lexer::LexIdentifierContinue
    +    0.95%  clang                      [.] GetFullTypeForDeclarator
    +    0.90%  clang                      [.] (anonymous namespace)::IntExprEvaluator::VisitBinaryOperator
    +    0.88%  clang                      [.] CheckICE
         0.80%  clang                      [.] __libc_free
    +    0.75%  clang                      [.] clang::SourceManager::getFileID
    +    0.74%  clang                      [.] clang::TokenLexer::ExpandFunctionArguments
    +    0.67%  clang                      [.] alloc_slot
    +    0.65%  clang                      [.] get_meta
    +    0.65%  clang                      [.] memset
         0.62%  [unknown]                  [k] 0xffffffffaba01087 (kptr_restrict not disabled)
    +    0.55%  clang                      [.] clang::Preprocessor::getMacroDefinition  (end top 20)
         0.50%  clang                      [.] clang::Parser::ParseDeclarationSpecifiers
         0.50%  clang                      [.] memcmp
         0.49%  clang                      [.] llvm::FoldingSetBase::FindNodeOrInsertPos
         0.47%  clang                      [.] GetDiagInfo
         0.46%  clang                      [.] clang::Sema::LookupName
         0.46%  clang                      [.] strlen
    

    so __libc_malloc_impl, __libc_free, alloc_slot and get_meta all seem to be coming from musl's memory allocator. Seeing memcpy and memset in the top 20 is also with memcmp and strlen not too far behind :(.

  3. nickdesaulniers commented on Jun 17, 2022

    @nickdesaulniers
    MemberAuthor

    @petrhosek mentioned that https://github.com/mjansson/rpmalloc might be interesting to play with; it's used for host tools for fuchsia.

  4. petrhosek commented on Jun 17, 2022

    @petrhosek

    I'd also experiment with https://github.com/microsoft/mimalloc and https://github.com/microsoft/snmalloc, all of these are already supported by LLVM on Windows and we could consider extending that support to other platforms.

  5. nickdesaulniers commented on Jun 18, 2022

    @nickdesaulniers
    MemberAuthor

    heh, all of those are listed: https://github.com/llvm/llvm-project/blob/be6af89f85ebd04646b5704301470f02b70a0447/llvm/lib/Support/CMakeLists.txt#L89-L95 so it seems that we can just set LLVM_INTEGRATED_CRT_ALLOC=path/to/rpmalloc.

  6. nickdesaulniers commented on Jun 21, 2022

    @nickdesaulniers
    MemberAuthor

    Adding the following -D LLVM_INTEGRATED_CRT_ALLOC=/rpmalloc-1.4.4 to my cmake invocation produces:

    CMake Error at CMakeLists.txt:575 (message):
      LLVM_INTEGRATED_CRT_ALLOC is only supported on Windows.
    

    which is lame. I guess that's the purpose of https://reviews.llvm.org/D101427.

  7. added 2 commits that reference this issue on Jul 1, 2022
    caf65d1
    63b584a
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions