Skip to content

GH-33849: [C++] Fix builds with ARROW_BUILD_SHARED=OFF and ARROW_BUILD_EXAMPLES=ON - #34350

Merged
kou merged 2 commits into
apache:mainfrom
abetomo:fix-cpp-build-error
Mar 13, 2023
Merged

kou merged 2 commits into
apache:mainfrom
abetomo:fix-cpp-build-error

Conversation

@abetomo

@abetomo abetomo commented Feb 25, 2023 •

Copy link
Copy Markdown
Contributor

Rationale for this change

Building with ARROW_BUILD_SHARED=OFF and ARROW_BUILD_EXAMPLES=ON causes an error.

What changes are included in this PR?

At examples/arrow/CMakeLists.txt, set arrow_substrait_[shared|static] and gandiva_[shared|static] appropriately according to ARROW_BUILD_SHARED.

Are these changes tested?

In --preset ninja-debug-maximal, ARROW_BUILD_EXAMPLES=ON.

$ cmake .. --preset ninja-debug-maximal -DARROW_BUILD_SHARED=OFF -DARROW_BUILD_STATIC=ON
$ cmake --build .

Build was completed.

$ cmake .. --preset ninja-debug-maximal -DARROW_BUILD_SHARED=ON -DARROW_BUILD_STATIC=ON
$ cmake --build .

Build was completed.

Are there any user-facing changes?

@github-actions

Copy link
Copy Markdown

@kou

kou commented Feb 25, 2023 •

Copy link
Copy Markdown
Member

It seems that we also need to care about Apache Arrow Flight related examples:

diff --git a/cpp/examples/arrow/CMakeLists.txt b/cpp/examples/arrow/CMakeLists.txt
index aa33c18e76..76c176086a 100644
--- a/cpp/examples/arrow/CMakeLists.txt
+++ b/cpp/examples/arrow/CMakeLists.txt
@@ -34,7 +40,7 @@ endif()
 if(ARROW_FLIGHT)
   # Static gRPC means we cannot linked to shared Arrow, since then
   # we'll violate ODR for gRPC symbols
-  if(ARROW_GRPC_USE_SHARED)
+  if(ARROW_BUILD_SHARED AND ARROW_GRPC_USE_SHARED)
     set(FLIGHT_EXAMPLES_LINK_LIBS arrow_flight_shared)
     if(APPLE)
       set(GRPC_REFLECTION_LINK_LIBS gRPC::grpc++_reflection)
@@ -100,7 +106,7 @@ if(ARROW_FLIGHT)
                     "${CMAKE_CURRENT_BINARY_DIR}/helloworld.grpc.pb.cc")
 
   if(ARROW_FLIGHT_SQL)
-    if(ARROW_GRPC_USE_SHARED)
+    if(ARROW_BUILD_SHARED AND ARROW_GRPC_USE_SHARED)
       set(FLIGHT_SQL_EXAMPLES_LINK_LIBS arrow_flight_sql_shared)
     else()
       set(FLIGHT_SQL_EXAMPLES_LINK_LIBS arrow_flight_sql_static)

Could you install gRPC and try cmake .. --preset ninja-debug-maximal -DARROW_BUILD_SHARED=OFF -DARROW_BUILD_STATIC=ON -DgRPC_SOURCE=SYSTEM?

@abetomo

abetomo commented Feb 28, 2023

Copy link
Copy Markdown
Contributor Author

Thanks for your comment.
I will try it.

@abetomo

abetomo commented Mar 4, 2023

Copy link
Copy Markdown
Contributor Author

@kou As for Flight, if debug/flight-grpc-example and debug/flight-sql-example are generated, is it building as expected?

@kou

kou commented Mar 4, 2023

Copy link
Copy Markdown
Member

Yes.
They should be built with both of -DgRPC_SOURCE=SYSTEM configuration and -DgRPC_SOURCE=BUNDLED.
With this pull request, they are built with -DgRPC_SOURCE=BUNDLED but are NOT built (build error) with -DgRPC_SOURCE=SYSTEM.
#34350 (comment) may fix the latter.

…W_BUILD_EXAMPLES=ON

Signed-off-by: abetomo <abe@enzou.tokyo>
@abetomo

abetomo commented Mar 12, 2023

Copy link
Copy Markdown
Contributor Author

I could reproduce the error.

$ cmake --build .
[1/152] Linking CXX executable debug/flight-grpc-example
[2/152] Linking CXX executable debug/compute-register-example
[3/152] Linking CXX executable debug/rapidjson-row-converter
ninja: job failed: : && /usr/bin/c++ -Wno-noexcept-type  -fdiagnostics-color=always  -Wall -Wno-conversion -Wno-sign-conversion -Wunused-result -fno-semantic-interposition -msse4.2  -g -Werror -O0 -ggdb  examples/arrow/CMakeFiles/flight-grpc-example.dir/flight_grpc_example.cc.o examples/arrow/CMakeFiles/flight-grpc-example.dir/helloworld.pb.cc.o examples/arrow/CMakeFiles/flight-grpc-example.dir/helloworld.grpc.pb.cc.o -o debug/flight-grpc-example  debug/libarrow.a  -larrow_flight_shared  -Wl,--no-as-needed  /usr/lib/libgrpc++_reflection.so.1.50.1  -Wl,--as-needed  /usr/lib/libprotobuf.so  gflags_ep-prefix/src/gflags_ep/lib/libgflags_debug.a  /usr/lib/libbrotlienc.so  /usr/lib/libbrotlidec.so  /usr/lib/libbrotlicommon.so  bzip2_ep-install/lib/libbz2.a  orc_ep-install/lib/liborc.a  lz4_ep-install/lib/liblz4.a  snappy_ep/src/snappy_ep-install/lib/libsnappy.a  /usr/lib/libzstd.so.1.5.2  google_cloud_cpp_ep-install/lib/libgoogle_cloud_cpp_storage.a  google_cloud_cpp_ep-install/lib/libgoogle_cloud_cpp_rest_internal.a  google_cloud_cpp_ep-install/lib/libgoogle_cloud_cpp_common.a  crc32c_ep-install/lib/libcrc32c.a  awssdk_ep-install/lib/libaws-cpp-sdk-identity-management.a  awssdk_ep-install/lib/libaws-cpp-sdk-sts.a  awssdk_ep-install/lib/libaws-cpp-sdk-cognito-identity.a  awssdk_ep-install/lib/libaws-cpp-sdk-s3.a  awssdk_ep-install/lib/libaws-cpp-sdk-core.a  /usr/lib/libcurl.so  awssdk_ep-install/lib/libaws-crt-cpp.a  awssdk_ep-install/lib/libaws-c-s3.a  awssdk_ep-install/lib/libaws-c-auth.a  awssdk_ep-install/lib/libaws-c-mqtt.a  awssdk_ep-install/lib/libaws-c-http.a  awssdk_ep-install/lib/libaws-c-compression.a  awssdk_ep-install/lib/libaws-c-sdkutils.a  awssdk_ep-install/lib/libaws-c-event-stream.a  awssdk_ep-install/lib/libaws-c-io.a  awssdk_ep-install/lib/libaws-c-cal.a  awssdk_ep-install/lib/libaws-checksums.a  awssdk_ep-install/lib/libaws-c-common.a  awssdk_ep-install/lib/libs2n.a  utf8proc_ep-install/lib/libutf8proc.a  jemalloc_ep-prefix/src/jemalloc_ep/dist//lib/libjemalloc_pic.a  mimalloc_ep/src/mimalloc_ep/lib/mimalloc-2.0/libmimalloc-debug.a  /usr/lib/libgrpc++.so.1.50.1  /usr/lib/libgrpc.so.28.0.0  /usr/lib/libssl.so  /usr/lib/libcrypto.so  /lib/libz.so  re2_ep-install/lib/libre2.a  /usr/lib/libcares.so.2.5.1  /usr/lib/libaddress_sorting.so.28.0.0  /usr/lib/libupb.so.28.0.0  /usr/lib/libabsl_raw_hash_set.so.2206.0.0  /usr/lib/libabsl_hashtablez_sampler.so.2206.0.0  /usr/lib/libabsl_hash.so.2206.0.0  /usr/lib/libabsl_city.so.2206.0.0  /usr/lib/libabsl_low_level_hash.so.2206.0.0  /usr/lib/libabsl_statusor.so.2206.0.0  /usr/lib/libgpr.so.28.0.0  /usr/lib/libabsl_bad_variant_access.so.2206.0.0  /usr/lib/libabsl_status.so.2206.0.0  /usr/lib/libabsl_str_format_internal.so.2206.0.0  /usr/lib/libabsl_strerror.so.2206.0.0  /usr/lib/libabsl_random_distributions.so.2206.0.0  /usr/lib/libabsl_random_seed_sequences.so.2206.0.0  /usr/lib/libabsl_random_internal_pool_urbg.so.2206.0.0  /usr/lib/libabsl_random_internal_randen.so.2206.0.0  /usr/lib/libabsl_random_internal_randen_hwaes.so.2206.0.0  /usr/lib/libabsl_random_internal_randen_hwaes_impl.so.2206.0.0  /usr/lib/libabsl_random_internal_randen_slow.so.2206.0.0  /usr/lib/libabsl_random_internal_platform.so.2206.0.0  /usr/lib/libabsl_random_internal_seed_material.so.2206.0.0  /usr/lib/libabsl_random_seed_gen_exception.so.2206.0.0  /usr/lib/libabsl_cord.so.2206.0.0  /usr/lib/libabsl_bad_optional_access.so.2206.0.0  /usr/lib/libabsl_cordz_info.so.2206.0.0  /usr/lib/libabsl_cord_internal.so.2206.0.0  /usr/lib/libabsl_cordz_functions.so.2206.0.0  /usr/lib/libabsl_exponential_biased.so.2206.0.0  /usr/lib/libabsl_cordz_handle.so.2206.0.0  /usr/lib/libabsl_synchronization.so.2206.0.0  /usr/lib/libabsl_stacktrace.so.2206.0.0  /usr/lib/libabsl_symbolize.so.2206.0.0  /usr/lib/libabsl_debugging_internal.so.2206.0.0  /usr/lib/libabsl_demangle_internal.so.2206.0.0  /usr/lib/libabsl_graphcycles_internal.so.2206.0.0  /usr/lib/libabsl_time.so.2206.0.0  /usr/lib/libabsl_strings.so.2206.0.0  /usr/lib/libabsl_strings_internal.so.2206.0.0  /usr/lib/libabsl_throw_delegate.so.2206.0.0  /usr/lib/libabsl_int128.so.2206.0.0  /usr/lib/libabsl_civil_time.so.2206.0.0  /usr/lib/libabsl_time_zone.so.2206.0.0  /usr/lib/libabsl_malloc_internal.so.2206.0.0  /usr/lib/libabsl_base.so.2206.0.0  /usr/lib/libabsl_spinlock_wait.so.2206.0.0  /usr/lib/libabsl_raw_logging_internal.so.2206.0.0  /usr/lib/libabsl_log_severity.so.2206.0.0  /usr/lib/libprotobuf.so  -ldl  -lm  -lrt && :
/usr/lib/gcc/x86_64-alpine-linux-musl/12.2.1/../../../../x86_64-alpine-linux-musl/bin/ld: cannot find -larrow_flight_shared: No such file or directory
collect2: error: ld returned 1 exit status
ninja: subcommand failed
$ cmake --build .
[1/70] Linking CXX executable debug/flight-sql-example
[2/70] Linking CXX executable debug/flight-test-server
[3/70] Linking CXX executable debug/arrow-flight-internals-test
ninja: job failed: : && /usr/bin/c++ -Wno-noexcept-type  -fdiagnostics-color=always  -Wall -Wno-conversion -Wno-sign-conversion -Wunused-result -fno-semantic-interposition -msse4.2  -g -Werror -O0 -ggdb  examples/arrow/CMakeFiles/flight-sql-example.dir/flight_sql_example.cc.o -o debug/flight-sql-example  debug/libarrow.a  debug/libarrow_flight.a  -larrow_flight_sql_shared  /usr/lib/libgrpc++.so.1.50.1  /usr/lib/libprotobuf.so  gflags_ep-prefix/src/gflags_ep/lib/libgflags_nothreads_debug.a  debug/libarrow.a  /usr/lib/libbrotlienc.so  /usr/lib/libbrotlidec.so  /usr/lib/libbrotlicommon.so  bzip2_ep-install/lib/libbz2.a  orc_ep-install/lib/liborc.a  lz4_ep-install/lib/liblz4.a  snappy_ep/src/snappy_ep-install/lib/libsnappy.a  /usr/lib/libzstd.so.1.5.2  google_cloud_cpp_ep-install/lib/libgoogle_cloud_cpp_storage.a  google_cloud_cpp_ep-install/lib/libgoogle_cloud_cpp_rest_internal.a  google_cloud_cpp_ep-install/lib/libgoogle_cloud_cpp_common.a  crc32c_ep-install/lib/libcrc32c.a  awssdk_ep-install/lib/libaws-cpp-sdk-identity-management.a  awssdk_ep-install/lib/libaws-cpp-sdk-sts.a  awssdk_ep-install/lib/libaws-cpp-sdk-cognito-identity.a  awssdk_ep-install/lib/libaws-cpp-sdk-s3.a  awssdk_ep-install/lib/libaws-cpp-sdk-core.a  /usr/lib/libcurl.so  awssdk_ep-install/lib/libaws-crt-cpp.a  awssdk_ep-install/lib/libaws-c-s3.a  awssdk_ep-install/lib/libaws-c-auth.a  awssdk_ep-install/lib/libaws-c-mqtt.a  awssdk_ep-install/lib/libaws-c-http.a  awssdk_ep-install/lib/libaws-c-compression.a  awssdk_ep-install/lib/libaws-c-sdkutils.a  awssdk_ep-install/lib/libaws-c-event-stream.a  awssdk_ep-install/lib/libaws-c-io.a  awssdk_ep-install/lib/libaws-c-cal.a  awssdk_ep-install/lib/libaws-checksums.a  awssdk_ep-install/lib/libaws-c-common.a  awssdk_ep-install/lib/libs2n.a  utf8proc_ep-install/lib/libutf8proc.a  jemalloc_ep-prefix/src/jemalloc_ep/dist//lib/libjemalloc_pic.a  mimalloc_ep/src/mimalloc_ep/lib/mimalloc-2.0/libmimalloc-debug.a  /usr/lib/libgrpc.so.28.0.0  /usr/lib/libssl.so  /usr/lib/libcrypto.so  /lib/libz.so  re2_ep-install/lib/libre2.a  /usr/lib/libcares.so.2.5.1  /usr/lib/libaddress_sorting.so.28.0.0  /usr/lib/libupb.so.28.0.0  /usr/lib/libabsl_raw_hash_set.so.2206.0.0  /usr/lib/libabsl_hashtablez_sampler.so.2206.0.0  /usr/lib/libabsl_hash.so.2206.0.0  /usr/lib/libabsl_city.so.2206.0.0  /usr/lib/libabsl_low_level_hash.so.2206.0.0  /usr/lib/libabsl_statusor.so.2206.0.0  /usr/lib/libgpr.so.28.0.0  -ldl  -lm  -lrt  /usr/lib/libabsl_bad_variant_access.so.2206.0.0  /usr/lib/libabsl_status.so.2206.0.0  /usr/lib/libabsl_str_format_internal.so.2206.0.0  /usr/lib/libabsl_strerror.so.2206.0.0  /usr/lib/libabsl_random_distributions.so.2206.0.0  /usr/lib/libabsl_random_seed_sequences.so.2206.0.0  /usr/lib/libabsl_random_internal_pool_urbg.so.2206.0.0  /usr/lib/libabsl_random_internal_randen.so.2206.0.0  /usr/lib/libabsl_random_internal_randen_hwaes.so.2206.0.0  /usr/lib/libabsl_random_internal_randen_hwaes_impl.so.2206.0.0  /usr/lib/libabsl_random_internal_randen_slow.so.2206.0.0  /usr/lib/libabsl_random_internal_platform.so.2206.0.0  /usr/lib/libabsl_random_internal_seed_material.so.2206.0.0  /usr/lib/libabsl_random_seed_gen_exception.so.2206.0.0  /usr/lib/libabsl_cord.so.2206.0.0  /usr/lib/libabsl_bad_optional_access.so.2206.0.0  /usr/lib/libabsl_cordz_info.so.2206.0.0  /usr/lib/libabsl_cord_internal.so.2206.0.0  /usr/lib/libabsl_cordz_functions.so.2206.0.0  /usr/lib/libabsl_exponential_biased.so.2206.0.0  /usr/lib/libabsl_cordz_handle.so.2206.0.0  /usr/lib/libabsl_synchronization.so.2206.0.0  /usr/lib/libabsl_stacktrace.so.2206.0.0  /usr/lib/libabsl_symbolize.so.2206.0.0  /usr/lib/libabsl_debugging_internal.so.2206.0.0  /usr/lib/libabsl_demangle_internal.so.2206.0.0  /usr/lib/libabsl_graphcycles_internal.so.2206.0.0  /usr/lib/libabsl_time.so.2206.0.0  /usr/lib/libabsl_strings.so.2206.0.0  /usr/lib/libabsl_strings_internal.so.2206.0.0  /usr/lib/libabsl_throw_delegate.so.2206.0.0  /usr/lib/libabsl_int128.so.2206.0.0  /usr/lib/libabsl_civil_time.so.2206.0.0  /usr/lib/libabsl_time_zone.so.2206.0.0  /usr/lib/libabsl_malloc_internal.so.2206.0.0  /usr/lib/libabsl_base.so.2206.0.0  /usr/lib/libabsl_spinlock_wait.so.2206.0.0  /usr/lib/libabsl_raw_logging_internal.so.2206.0.0  /usr/lib/libabsl_log_severity.so.2206.0.0  /usr/lib/libprotobuf.so && :
/usr/lib/gcc/x86_64-alpine-linux-musl/12.2.1/../../../../x86_64-alpine-linux-musl/bin/ld: cannot find -larrow_flight_sql_shared: No such file or directory
collect2: error: ld returned 1 exit status
ninja: subcommand failed

If I just added -DgRPC_SOURCE=SYSTEM, it seemed to be using grpc_ep-install generated in the build directory, so I added -DProtobuf_SOURCE=SYSTEM.
Also added -DARROW_FLIGHT_SQL=ON to build debug/flight-sql-example.

…_SHARED=OFF

Signed-off-by: abetomo <abe@enzou.tokyo>
@abetomo
abetomo force-pushed the fix-cpp-build-error branch from 7e66ec5 to d9fa805 Compare March 12, 2023 23:33
@github-actions github-actions Bot added the awaiting review Awaiting review label Mar 12, 2023
@abetomo

abetomo commented Mar 12, 2023

Copy link
Copy Markdown
Contributor Author

The build was successful with the #34350 (comment) patch.

@kou

kou commented Mar 13, 2023

Copy link
Copy Markdown
Member

@github-actions crossbow submit example

@github-actions

Copy link
Copy Markdown

Revision: d9fa805

Submitted crossbow builds: ursacomputing/crossbow @ actions-46b0d97237

Task Status
cpp-tutorial-example Github Actions
example-cpp-minimal-build-static Github Actions
example-cpp-minimal-build-static-system-dependency Github Actions
example-python-minimal-build-fedora-conda Github Actions
example-python-minimal-build-ubuntu-venv Github Actions

@kou kou left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1

@kou
kou merged commit 29ec779 into apache:main Mar 13, 2023
@github-actions github-actions Bot added awaiting merge Awaiting merge and removed awaiting review Awaiting review labels Mar 13, 2023
@ursabot

ursabot commented Mar 13, 2023

Copy link
Copy Markdown

Benchmark runs are scheduled for baseline = 5b2fbad and contender = 29ec779. 29ec779 is a master commit associated with this PR. Results will be available as each benchmark for each run completes.
Conbench compare runs links:
[Finished ⬇️0.0% ⬆️0.0%] ec2-t3-xlarge-us-east-2
[Failed ⬇️0.36% ⬆️0.36%] test-mac-arm
[Finished ⬇️4.08% ⬆️0.0%] ursa-i9-9960x
[Finished ⬇️0.38% ⬆️0.09%] ursa-thinkcentre-m75q
Buildkite builds:
[Finished] 29ec7790 ec2-t3-xlarge-us-east-2
[Finished] 29ec7790 test-mac-arm
[Finished] 29ec7790 ursa-i9-9960x
[Finished] 29ec7790 ursa-thinkcentre-m75q
[Finished] 5b2fbade ec2-t3-xlarge-us-east-2
[Failed] 5b2fbade test-mac-arm
[Finished] 5b2fbade ursa-i9-9960x
[Finished] 5b2fbade ursa-thinkcentre-m75q
Supported benchmarks:
ec2-t3-xlarge-us-east-2: Supported benchmark langs: Python, R. Runs only benchmarks with cloud = True
test-mac-arm: Supported benchmark langs: C++, Python, R
ursa-i9-9960x: Supported benchmark langs: Python, R, JavaScript
ursa-thinkcentre-m75q: Supported benchmark langs: C++, Java

@abetomo
abetomo deleted the fix-cpp-build-error branch March 13, 2023 05:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[C++] ARROW_BUILD_SHARED=OFF and ARROW_BUILD_EXAMPLES=ON causes build error

3 participants