Skip to content

[CI] Refactor CI builders - #8826

Merged
comphead merged 15 commits into
apache:mainfrom
comphead:dev0
Jan 11, 2024
Merged

comphead merged 15 commits into
apache:mainfrom
comphead:dev0

Conversation

@comphead

@comphead comphead commented Jan 10, 2024 •

Copy link
Copy Markdown
Contributor

Which issue does this PR close?

Closes #.

Rationale for this change

rust.yml contains a lot of duplicates and a little bit overcomplicated, it would be nice to simplify the main CI file

What changes are included in this PR?

PR idea is to reduce complexity for rust.yml and factor out common properties like Rust into a single builder. Separately introduced other builders for each os.

Are these changes tested?

Are there any user-facing changes?

@comphead comphead changed the title [CI] Test Mozilla sccache to make compile faster [CI] Refactor CI builders Jan 11, 2024
@comphead
comphead marked this pull request as ready for review January 11, 2024 05:36
@comphead
comphead requested a review from alamb January 11, 2024 05:37
@comphead

Copy link
Copy Markdown
Contributor Author

@Jefffrey please review once you have time

@Jefffrey Jefffrey left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good 👍

Comment thread datafusion/common/src/error.rs Outdated
# Set debuginfo=line-tables-only as debuginfo=0 causes immensely slow build
# See for more details: https://github.com/rust-lang/rust/issues/119560
#
# set RUST_MIN_STACK to avoid rust stack overflows on tpc-ds tests

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe put this as a TODO item, since ideally we'd want the tests to pass without needing this setting?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We still need it as TPC-DS tests spontaneously fail on stack overflow. But we prob need to check why is it happening. That might be a bigger problem with piling up the stackframes.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree it would be great to file a ticket to track the stack overflow

https://github.com/apache/arrow-datafusion/issues?q=is%3Aissue+is%3Aopen+stack+overflow doesn't show any obvious candidate

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Filed #8837

@alamb alamb left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comment thread .github/actions/setup-macos-builder/action.yaml Outdated
# Set debuginfo=line-tables-only as debuginfo=0 causes immensely slow build
# See for more details: https://github.com/rust-lang/rust/issues/119560
#
# set RUST_MIN_STACK to avoid rust stack overflows on tpc-ds tests

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree it would be great to file a ticket to track the stack overflow

https://github.com/apache/arrow-datafusion/issues?q=is%3Aissue+is%3Aopen+stack+overflow doesn't show any obvious candidate

Comment thread .github/actions/setup-windows-builder/action.yaml Outdated
comphead and others added 2 commits January 11, 2024 12:07
Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>
Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>
@comphead
comphead merged commit d9a1d42 into apache:main Jan 11, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants