Skip to content

Keep the first data point of a canonical download - #10004

Closed
econstantin wants to merge 3 commits into
QuantConnect:masterfrom
econstantin:bug-10002-canonical-download-first-data-point
Closed

econstantin wants to merge 3 commits into
QuantConnect:masterfrom
econstantin:bug-10002-canonical-download-first-data-point

Conversation

@econstantin

Copy link
Copy Markdown
Contributor

Description

CanonicalDataDownloaderDecorator.GetContractsData called .Any() on the consuming enumerable of its BlockingCollection. That call removed the first data point from the collection and dropped it.

This change takes the first data point with TryTake, and returns it before the rest of the collection.

It also adds an internal constructor that takes the chain providers, so a test can run the decorator without the global Composer.

Related Issue

Fixes #10002

Motivation and Context

Each download of a canonical symbol through the chain provider loses 1 data point. The download exits with code 0 and logs no error. The contracts download in parallel, so 2 downloads of the same window can lose different data points.

Requires Documentation Change

No.

How Has This Been Tested?

This change adds 4 tests to CanonicalDataDownloaderDecoratorTests:

  • ReturnsEveryBarOfEveryContractForCanonicalFuture runs the decorator on a canonical future with a stub IDataDownloader that returns 3 minute bars for each of 2 contracts, then compares the returned bars with the bars the stub produced.
  • ReturnsTheOnlyBarForCanonicalFuture checks a download that produces 1 bar. The old logic returned 0 bars.
  • ReturnsNullForCanonicalFutureWhenNoContractHasData and ReturnsNullForCanonicalFutureWithNoContract check the path with no data.

Results on master at 6f30bf499, with Windows 11, .NET SDK 10.0.403, and Python 3.11:

  • With the old logic in GetContractsData: 2 failed (ReturnsEveryBarOfEveryContractForCanonicalFuture and ReturnsTheOnlyBarForCanonicalFuture), 30 passed.
  • With this change: 32 passed.
  • The test suite with the filter TestCategory!=TravisExclude&TestCategory!=ResearchRegressionTests: 39344 passed, 0 failed, 100 skipped.

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • Refactor (non-breaking change which improves implementation)
  • Performance (non-breaking change which improves performance. Please add associated performance test and results)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)
  • Non-functional change (xml comments/documentation/etc)

Checklist:

  • My code follows the code style of this project.
  • I have read the CONTRIBUTING document.
  • I have added tests to cover my changes.
  • All new and existing tests passed.
  • My branch follows the naming convention bug-<issue#>-<description> or feature-<issue#>-<description>

econstantin and others added 3 commits October 30, 2026 12:44
CanonicalDataDownloaderDecorator.GetContractsData called Any() on the
consuming enumerable of its BlockingCollection. That call removed 1 data
point from the collection and dropped it. So each download of a canonical
symbol through the chain provider lost the first data point that a contract
added.

Take the first data point with TryTake and return it before the rest.

Add an internal constructor that takes the chain providers, so a test can
run the decorator without the global Composer.
Add ReturnsTheOnlyBarForCanonicalFuture. With the old logic, a download
of 1 bar returned 0 bars.

Remove the wait between contracts from the test of every bar. The old
logic dropped a data point in any thread order, so the test did not need
it.
@Martin-Molinero

Copy link
Copy Markdown
Member

Included by #10096 thank you

@econstantin
econstantin deleted the bug-10002-canonical-download-first-data-point branch November 18, 2026 02:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

CanonicalDataDownloaderDecorator drops the first data point of a canonical download

2 participants