You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Upgrading to grpc 1.39.0 makes our cmake-install build too slow #7052
We upgraded grpc from 1.38.1 to 1.39.0 and our cmake-install build became unusably slow, so we rolled it back. To roll this change forward git revert ba41d5a99818922b09eee5da148786c1d68c17f5 (ba41d5a)
The line is slow when checking a module that links with grpc, like google_cloud_cpp_grpc_utils
I suspect there may be some change in this version of grpc to its pkg-config files. Maybe it's also exposing a bug in pkg-config itself, maybe similar to #6846
NOTE: We need to figure this out because we need to be able to upgrade gRPC
NOTE: I do not think that this is a gRPC bug. But it may have exposed a bug, or inefficiency, in the pkg-config implementation where it's just slow on packages with lots of "requires" libraries.
FWIW, here's the gRPC PR that introduced this change: grpc/grpc#25840
AFAICT, gRPC seems to be using pkg-config correctly. I don't see anything wrong that gRPC is doing wrt pkg-config. This just may be a bug or limitation of pkg-config itself.
Is there a "fix" that gRPC could/should make? I don't see it.
Is there a "fix" or workaround that we could make?
Are we using pkg_check_modules correctly? Should we be using that?
At this point, I'm suspecting that pkg-config simply does not scale well.
Summary: Cloud C++ upgraded to grpc 1.39.0 and our builds that use pkg-config got really slow. We had to roll back. The issue is that grpc listed all of its pkg-config deps (which are mostly Abseil deps) in its Requires: line. For some reason, this makes pkg-config (actually pkgconf on fedora) quite slow. I think this is actually a pkg-config performance problem; I do not think grpc nor abseil are necessarily doing anything wrong.
@devjgm Thank you for the investigation! I didn't know there is another pkg-config, pkgconf. From what Fedora said, chances are more Linux distributions would replace pkg-config with pkgconf so fixing this problem will save us in future. In that sense, thank you for filing the issue on pkgconf!
Activity
devjgm commentedon Jul 23, 2021
@veblush FYI
devjgm commentedon Jul 23, 2021
We see this in a docker image w/ the following info:
devjgm commentedon Jul 23, 2021
I just discovered that fedora uses
pkgconfnotpkg-config. The project is https://github.com/pkgconf/pkgconf. Fedora 34 uses pkgconf 1.7.3.devjgm commentedon Jul 23, 2021
I suspect the reason for the slowness is that grpc moved a bunch of linker flags from the pkg-config
Libs:line toRequires:.grpc 1.38.1
grpc 1.39.0
NOTE: I do not think that this is a gRPC bug. But it may have exposed a bug, or inefficiency, in the pkg-config implementation where it's just slow on packages with lots of "requires" libraries.
devjgm commentedon Jul 24, 2021
FWIW, here's the gRPC PR that introduced this change: grpc/grpc#25840
AFAICT, gRPC seems to be using pkg-config correctly. I don't see anything wrong that gRPC is doing wrt pkg-config. This just may be a bug or limitation of pkg-config itself.
pkg_check_modulescorrectly? Should we be using that?At this point, I'm suspecting that pkg-config simply does not scale well.
devjgm commentedon Jul 26, 2021
@derekmauro and @dinor7c JFYI
Summary: Cloud C++ upgraded to grpc 1.39.0 and our builds that use
pkg-configgot really slow. We had to roll back. The issue is that grpc listed all of its pkg-config deps (which are mostly Abseil deps) in itsRequires:line. For some reason, this makes pkg-config (actuallypkgconfon fedora) quite slow. I think this is actually apkg-configperformance problem; I do not think grpc nor abseil are necessarily doing anything wrong.devjgm commentedon Jul 26, 2021
Fedora 34 by default uses
pkgconf1.7.3 from https://github.com/pkgconf/pkgconf. This version shows the problem in this issue.I compiled
pkg-config0.29.0 from https://pkgconfig.freedesktop.org/releases/pkg-config-0.29.tar.gz , and this version does NOT show the problem.ci: avoid fedora's `pkgconf` which is slow
ci: avoid fedora's `pkgconf` which is slow (#7061)
veblush commentedon Jul 28, 2021
@devjgm Thank you for the investigation! I didn't know there is another pkg-config, pkgconf. From what Fedora said, chances are more Linux distributions would replace pkg-config with pkgconf so fixing this problem will save us in future. In that sense, thank you for filing the issue on pkgconf!