Skip to content
daggerokPublic

About

Themes ETF holdings-to-watchlist: static ./api/themes feed generated by a Bun updater from themesetfs.com catalog, daily holdings CSV, Yahoo Finance history/dividends, SEC N-PORT-P holdings fallback

Topics

Resources

Stars

0 stars

Watchers

0 watching

Forks

Latest commit

 

History

87 Commits

Folders and files

Repository files navigation

Themes ETFs

One of the app's features lets you select Themes ETFs in the Watchlist and aggregate their holdings to see how often each ticker appears across the selected funds. Repeated holdings make overlapping exposure visible: the more selected funds include a ticker, the greater its potential influence on the portfolio; gains in that holding may help, while declines may hurt, and actual impact also depends on each fund's position size. Another feature makes it faster and easier to find funds with stronger growth over different periods, higher dividend yields or distributions, greater Total Return (price performance plus dividends), and other key performance metrics. A client-side tool (Parcel + Tailwind CSS v4, built into dist) reading the generated ./api/themes static feed (official Themes ETFs catalog and daily full-holdings CSV downloads, Yahoo Finance price history/dividend events, and SEC EDGAR N-PORT-P holdings fallback) into a searchable ETF/category catalog with per-fund tabs, watchlist aggregation, ticker copy and CSV/TXT export — the same look, feel, columns and business logic as the sibling applications.

Using Bun

bunx degit daggerok/Themes#main ./themes && cd $_
bun install
bun run serve

bun run serve is the Parcel dev server (it copies api/ into dist/api first); bun run build writes the production site to dist and bun run build-github-pages does the same for the /Themes/ public URL. GitHub Pages is deployed by .github/workflows/github-pages.yml. The app reads only relative ./api/themes/** files generated by scripts/update-data.ts; the committed feed is complete, so the site works without an update run. The repository homepage is intended to be https://daggerok.github.io/Themes/.

Column types and filters

Every column of the ETF catalog and of the Watchlist, Holdings, History and Distributions tabs has a type: text (ABC), number (123), percentage (%), money ($), date (D), date and time (DT) or time of day (T). The type is detected from the texts the column shows (80% of the filled cells must agree, otherwise text) and is written in the badge next to the column title: click it to cycle the type, Shift+click to return to auto-detection. Dates are read as 2024-06-15, 6/15/2024, 15.06.2024, Jun 15, 2024 or 15-Jun-2024, date and time as 2024-06-15T09:30:00Z or 2024-06-15 09:30, time as 09:30, 16:00:00 or 9:30 PM

A row of filter inputs sits under the column headers (the Filters button hides it, Clear filters empties it). Filters of different columns are combined with AND, the search box applies on top, and Copy Tickers and the exports use the filtered rows. Filters and type overrides are remembered in the browser. Sticky # (next to Filters, off by default, remembered in the browser) numbers the rows by their rank in the table sorted by the current column before the column filters, so a filtered fund keeps its rank and the numbers keep gaps; the sort, the search and the category and blacklist choices rank again. The catalog starts sorted by Net Assets, largest first, unavailable values sort last in both directions, and every export starts with the # column. The red Clear button opens a dialog that lists what can be reset (the selection, searches, sort order, open tab, shown columns, column filters, remembered table views and the blacklist), all ticked the first time and afterwards as they were left at the last OK; Enter confirms, Esc or a click outside cancels, the theme is always kept, so the page looks like a first visit (also after a reload)

Inside one filter: a space means AND, a comma means OR, a leading ! means NOT, ? matches an empty or unavailable value and !? a value that is there; a value that is unavailable matches only ? and negated conditions. An unquoted space ends the value, so quote values that contain one (>="2024-06-15 09:30")

Type Examples
Text bank contains, "two words", !bank, =exact, ^starts, ends$, /regex/, tech, health
Number, percentage, money >10, >=10 <50, =22 (matches what rounds to 22), !=22, 10..50, ..50, 10.., >1B and K M B T suffixes, an optional $ or %
Date, date and time >2024-06-01, 2024 (the whole year), 2024-06 (the whole month), 2024-01..2024-06, today, yesterday, -7d.. (the last 7 days), +2w, -3m, -1y
Time >09:30, 09:30..16:00, =12:00 (the whole minute)

The Columns menu next to Filters lists every column of the ETF table from the first to the last, all of them shown by default, with a search box and the All, Clear, Toggle and Reset buttons. Use and Ticker are listed but locked. Hiding a column only removes it from the table: the filters, the sorting, the exports and Copy Tickers still use it. The choice is remembered in the browser (localStorage, never the data) and the menu is shown on the ETF catalog only

The asset classes are one Asset classes multi-select next to the All ETFs pill instead of one tab per class: every class is selected by default (= all ETFs), Only or unchecking narrows the table, and the All ETFs pill is lit only while nothing narrows it (all or none of the classes checked); clicking the pill clears the selection. The choice is remembered in the browser (localStorage, never the data)

Updating the static Themes ETFs data

Run the updater with Bun:

bun test
bun ./scripts/update-data.ts

Run bun ./scripts/update-data.ts -h (or --help) to print the update controls and isolated-run example.

Every control has a default in scripts/update-data.config.json; the updater and the workflow resolve them through the same resolveControls function. Precedence: file defaults < advanced JSON < nonblank named workflow inputs < protected Actions variable (SEC_UA) or environment variable. A blank named input inherits the file value, and a scheduled run uses the file defaults as-is.

The Update Themes ETFs data GitHub Actions workflow (.github/workflows/update-data.yml, scheduled weekly and manually dispatchable) exposes 24 individual inputs plus an advanced JSON object (for example {"SEC_YIELD":"0:5","VERBOSE":true}) that reaches every remaining control, and commits api/themes/** and nothing else - a data run never rewrites app code or CI configuration. All supplied filters use AND logic.

Data sources

Block Source Notes
Fund universe, ticker, fund name, category, catalog NAV, market price, and expense ratio themesetfs.com/etfs The server-rendered catalog's window.productsData first-party entries. The current seed has 13 active Themes ETFs.
Holdings (daily), net assets, and shares outstanding https://themesetfs.com/storage/holdings/Holdings-<TICKER>.csv Direct official daily full-holdings CSV linked from each fund page. The parser reads date, stock_ticker, cusip, security_name, shares, price, market_value, weightings, net_assets, shares_outstanding, country, and sector fields.
Daily price history Yahoo Finance public chart API The feed stores daily Date / Close / adjusted Close / Volume rows. Adjusted close is rounded to cents before writing to prevent last-digit churn.
Dividend history and frequency Yahoo Finance chart dividend events Distribution rows contain ex-date and amount. Frequency is inferred from observed dates; 00 - None is a requested display placeholder for no returned events, not proof that a fund never distributes.
Holdings fallback SEC EDGAR Form N-PORT-P, Themes ETF Trust CIK 0001976322 Used only when the direct official holdings CSV is unavailable (also through the proxy), EDGAR_FALLBACK is on, the filing names the fund's own series (series id from SEC's company_tickers_mf.json; the fund name only when SEC lists no series) and its report period is strictly newer than the published holdings. The newest report period comes first and primary_doc.xml is read from the trust's own EDGAR folder.

The direct Themes paths answered normal browser-style requests during the 2026-09-29 acceptance and initial-seed runs, but themesetfs.com answers some datacenter networks, including GitHub-hosted runners, with HTTP 403. On HTTP 403 / 429 / 5xx (or an unusable body) for the catalog or a holdings CSV, the updater reads the same public URL through the read-only r.jina.ai rendering proxy: the catalog is parsed from the proxied markdown table, and a CSV is parsed if the proxy returns it. The proxy is retried at most once, sits behind one global gate (at least 3.2 seconds between proxy request starts, raised to REQUEST_SLEEP when that is larger) and is skipped after the first rejected CSV, because r.jina.ai cannot render application/octet-stream downloads; after two direct 403 answers the direct attempt is skipped for the rest of the run. Direct requests stay on independent per-worker lanes paced by REQUEST_SLEEP, with bounded retries for network errors and HTTP 408 / 425 / 429 / 5xx.

When the official source is completely unavailable the previously published data is kept instead of aborting: an unreachable catalog reuses the published index.json fund list, and a fund whose holdings cannot be refreshed (CSV, proxy, and SEC fallback all failed) keeps its published holdings, labeled retained in holdingsSource and source.retained. The run fails loudly only when there is nothing published to fall back on (no previous catalog, or every requested ticker has neither fresh nor published holdings). The 403 path cannot be reproduced from a residential network; it is covered by offline tests that force 403 on every direct URL.

The daily holdings CSV

The download is a real daily CSV, not a browser-only export. scripts/update-data.ts includes a small, dependency-free RFC-4180 reader (parseCsv) that tolerates CRLF, quoted commas, and escaped quotes. The shared holdings sheets retain the provider's security name, stock ticker, CUSIP, shares, market value, and weighting; sector is displayed as the shared Asset Category, with the source money-market flag taking precedence where present.

net_assets and shares_outstanding are repeated on each daily CSV row and are lifted to fund-level metadata. The generated JSON is paginated (001.json, 002.json, …), and stale page files are pruned when a fund's sheet shrinks.

Return arithmetic

The Themes site catalog supplies current NAV/market price, while the current source path does not provide a matching historical NAV-performance table. The app's YTD / 1-year / 3-year / 5-year / 10-year fields are therefore derived from Yahoo adjusted market-price closes, not represented as official NAV total returns. A horizon is published only when the history reaches back to it (the oldest row may start at most 5 days after the window start) and the fund is old enough for it: a fund younger than one year (calendar anniversary of Yahoo's first trade date, inclusive) publishes no 1-year return, and no 3, 5 or 10-year one or CAGR before those anniversaries. A younger fund, a HISTORY_RANGE window without published history to join, or a flat placeholder history (every close identical) gives null for that horizon and for its CAGR, never the since-inception return under a 3Y/5Y/10Y label. The returns block travels as one unit: when Yahoo fails, the previous returns, CAGRs, dividends and performanceAsOf are kept together.

Metrics and caveats

Metric Status Reason
YTD / 1 Yr / 3 Yr / 5 Yr / 10 Yr returns derived market-price return or — The current static feed uses Yahoo adjusted closes for history; it does not claim an official Themes NAV performance series.
returnsBasis always present, non-empty text Says how returns are computed: here derived from Yahoo Finance adjusted market-price closes (estimate, not official Themes ETFs NAV total returns). Never empty or -.
performanceAsOf YYYY-MM-DD or null Date the returns are as of: the last Yahoo close in the published history (not the NAV date). Kept from the previous publication when Yahoo is unavailable; null when the fund has no history yet.
Dividend yield / frequency derived from Yahoo events or 00 - None The source path is Yahoo dividend events, not an official Themes distribution-history table. 00 - None only means no usable events were returned for the feed.
30-Day SEC Yield — The current Themes catalog/CSV pipeline does not publish a matching SEC-yield field.
CUSIP / ISIN at fund level, inception date, premium / discount, bid-ask spread, documents — unless a future common schema supplies it They are not claimed from the catalog/CSV source currently used by this feed.
Recent price history short but published as returned, returns null LGCF, LIMI, and SMCF had five flat Yahoo daily rows in the 2026-09-29 initial seed; the rows are published as returned, but their returns, CAGRs and yield are null (a flat placeholder carries no return information).
dividendYieldBasis computed-trailing-12m or null Themes publishes no yield, so every non-null dividendYield is computed by the updater: Yahoo distributions of the last 12 months over the catalog NAV. The code is null exactly when dividendYield is null, and it is kept together with a retained yield. Present (possibly null) on every row, including catalog-only rows.
Dividend yield null without a distribution in the last 12 months A fund with no Yahoo distribution event in the trailing 12 months has an unknown yield (null, text —), not a published 0.00%.
Catalog-only funds dataFile: null, all metrics null A catalog fund whose holdings could never be fetched has an index row without funds/<T>/ data: dataFile is null and metrics carries the full key set with every value null.
Funds leaving the catalog removed from the feed When the official catalog is read successfully and no longer lists a published fund, its row and funds/<T>/ folder are removed (DROPPED FUNDS in the run output), unless the catalog has fewer than half of the published funds (then it is treated as truncated and nothing is removed); a catalog-only row the live catalog no longer lists (for example an old mistyped TICKERS entry) is always dropped. NEW FUNDS: A, B is printed (and added to $GITHUB_STEP_SUMMARY) when a fund is published for the first time, also when its first read failed and a later run succeeds; NEW FUNDS (not published yet): C names new catalog funds whose first read failed.
Retained data previous publication kept, labeled When the catalog or a fund's holdings cannot be refreshed from any source, the published values stay and holdingsSource / source.retained say so; nothing is zeroed. A fund is written only after all of its sources were read: pages first, then meta.json, stale pages last; a fund rejected by a data filter (AUM, yields, return ranges) is left exactly as published. Every request has a 45 s timeout (headers and body), retried like any other failure, and a run stops taking new funds after 25 minutes and still writes the index. The r.jina.ai proxy receives a generic User-Agent, never the SEC contact.
SEC N-PORT-P fallback used only when strictly newer The daily issuer CSV is usually newer than any N-PORT report, so a published CSV sheet is kept (not newer) and a filing replaces only an older sheet or fills a fund with nothing published. N-PORT identifiers are the position's own (a derivative never takes its reference security's CUSIP or ticker), a ticker is set only for a U.S. equity traded in USD and never on two different securities, and a missing weight, value or share count stays empty (never 0).
Dates calendar text CSV, published and Yahoo dates are read as plain calendar text (ISO, U.S. and month-name forms), never through the local time zone, so local runs match CI.
SEDOL / FIGI / coupon / maturity always — The direct Themes holdings CSV fields currently parsed by this feed do not provide these values.

Update controls

Keys of scripts/update-data.config.json; also accepted as environment variables and, for the ones without a dedicated workflow input (SEC_YIELD, HISTORY_RANGE, SEC_UA, VERBOSE, USE_SYSTEM_CA), through advanced. Every control also reads THEMES_<NAME> from the environment (for example THEMES_CONCURRENCY=7, THEMES_TICKERS="BOTT CLOD"), and HISTORICAL_PAGE_SIZE is an alias of HISTORY_PAGE_SIZE. Aliases sit in the environment layer: the plain name wins when both are set, an explicitly empty alias counts as set, and validation is the same.

Control Default Meaning
MAX_FETCHES 0 Funds to process; 0 = full pass. A positive value resumes after the cursor in api/themes/update-state.json and wraps around to the first fund of the filtered set; a TICKERS run never reads, writes or deletes the cursor.
TICKERS "" Space/comma-separated ticker allowlist; it is ANDed with every other filter. A ticker that is neither in the live catalog nor a published fund is an error before any fund request (nothing is written).
CATEGORY "" Case-insensitive substring match on the official Themes catalog category.
AUM : min:max net-assets range; accepts K / M / B / T suffixes and nano / micro / small / mid / large presets.
TER / DIVIDEND_YIELD / SEC_YIELD : Numeric min:max ranges in percent. Dividend data is derived from Yahoo events; SEC Yield is currently absent.
PERFORMANCE_YTD, PERFORMANCE_1Y, PERFORMANCE_3Y, PERFORMANCE_5Y, PERFORMANCE_10Y / TOTAL_RETURN_YTD, TOTAL_RETURN_1Y, TOTAL_RETURN_3Y, TOTAL_RETURN_5Y, TOTAL_RETURN_10Y : min:max filters on the derived adjusted-market-price return metrics.
CONCURRENCY / REQUEST_SLEEP 2 / 1 Parallel workers and seconds between request starts within each worker lane for direct requests; r.jina.ai proxy fallback requests share one global gate of at least 3.2 seconds between starts. Keep requests polite.
HOLDINGS_PAGE_SIZE / HISTORY_PAGE_SIZE 250 / 1000 Rows per generated JSON page.
MAX_RETRIES 3 Integer >= 1: retries after the initial request for network errors and HTTP 408 / 425 / 429 / 5xx.
HISTORY_RANGE max max or Ny (for example 5y): limits the Yahoo history request window (explicit period1/period2); a window keeps the older published history and distributions (rescaled by the overlapping days), and a window that does not join them keeps the published history and returns.
EDGAR_FALLBACK true Try the fund's own SEC N-PORT-P holdings when an official CSV cannot be obtained; used only when strictly newer than the published holdings.
SKIP_YAHOO / SKIP_THEMES false Retain prior history or catalog/holdings data while skipping the corresponding provider stage.
SEC_UA daggerok ETF feed daggerok@gmail.com User-Agent declared to SEC requests; the repository Actions variable SEC_UA overrides it when set.
VERBOSE false Print retry and fallback notices.
USE_SYSTEM_CA auto TLS trust store: auto restarts the updater once with Bun's --use-system-ca when a request fails with an untrusted-certificate error; true always uses the system CA store; false never restarts. Not an individual workflow input: use advanced, the config file or the CLI environment.

TICKERS combines with category/AUM/TER/yield/return filters using AND logic; it does not override them. Funds not selected for a successful update retain their previously published metadata and data files, so a bounded or partly failed run cannot empty the site.

Examples

MAX_FETCHES=10 bun ./scripts/update-data.ts
TICKERS="BOTT CLOD AUMI" bun ./scripts/update-data.ts
AUM="10M:1B" TER=":0.35" bun ./scripts/update-data.ts
CATEGORY="Commodities" bun ./scripts/update-data.ts

TypeScript and verification

The browser app lives in src/: index.html (markup), main.tsx (TypeScript), index.css (Tailwind CSS v4 plus the app styles) and favicon.ico, built by Parcel into dist (bun run build) - no tsconfig.json needed. Bun runs the updater TypeScript out of the box.

Verification before every publish: bun install --frozen-lockfile, bun test, bun build --target=bun scripts/update-data.ts --outfile=/dev/null, and git diff --check.

Brands table

Brand Where to get the data
AAM aamlive.com | AAM
abrdn (Aberdeen) aberdeeninvestments.com | aberdeen
Amplify amplifyetfs.com | Amplify
ARK Invest ark-funds.com | ARK
Capital Group capitalgroup.com | Capital-Group
Fidelity fidelity.com | Fidelity
First Trust ftportfolios.com | First-Trust
Franklin Templeton franklintempleton.com | Franklin
Global X globalxetfs.com/explore | Global-X
Goldman Sachs am.gs.com | Goldman-Sachs
Invesco invesco.com | Invesco
iShares ishares.com | iShares
JPMorgan am.jpmorgan.com | JPMorgan
NEOS neosfunds.com | Neos
Northern Trust etfs.ntam.northerntrust.com | Northern-Trust
Pacer ETFs paceretfs.com | Pacer
Parametric eatonvance.com | Parametric
ProShares proshares.com | ProShares
Schwab schwabassetmanagement.com | Schwab
SP Funds sp-funds.com | SP-Funds
SPDR ssga.com | SPDR
Sprott ETFs sprottetfs.com | Sprott
Tema ETFs temaetfs.com | Tema
Themes ETFs themesetfs.com/etfs | Themes
VanEck vaneck.com | VanEck
Vanguard investor.vanguard.com | Vanguard
VictoryShares vcm.com VictoryShares ETFs | VictoryShares
WisdomTree wisdomtree.com | WisdomTree
Xtrackers etf.dws.com | Xtrackers

Sibling applications

Application Data provider Repository
AAM Official AAM catalog/detail HTML + full holdings XLS + SEC N-PORT holdings fallback + Yahoo market history/dividends AAM
abrdn (Aberdeen) Official Aberdeen gateway + SEC N-PORT holdings fallback + Yahoo history/dividends aberdeen
Amplify Amplify ETFs Firestore data feed + SEC EDGAR N-PORT-P holdings fallback + Yahoo Finance history/dividends Amplify
ARK Invest ark-funds.com fund pages + overview/NAV-history/performance JSON + official daily holdings CSV + SEC EDGAR N-PORT-P holdings fallback + Yahoo Finance distributions/history fallback ARK
Capital Group Official Capital Group fund data + SEC N-PORT holdings fallback + Yahoo history fallback Capital-Group
Fidelity SEC EDGAR N-PORT-P + Yahoo Finance Fidelity
First Trust ftportfolios.com official ETF list + fund summary, holdings, distribution and price-history export pages + SEC EDGAR N-PORT-P holdings fallback + Yahoo Finance history fallback First-Trust
Franklin Templeton franklintempleton.com ETF listings + product pages + SEC EDGAR N-PORT-P Franklin
Global X globalxetfs.com Next.js catalog and fund pages + dated full-holdings CSV Global-X
Goldman Sachs am.gs.com fund data API (catalog, fund details, daily full holdings) + SEC EDGAR N-PORT-P holdings fallback Goldman-Sachs
Invesco invesco.com fund pages and sitemap + official Invesco fund API (monthly returns, NAV, AUM, yields, daily holdings, expense ratio) + SEC EDGAR N-PORT-P holdings fallback + Yahoo Finance history/dividends Invesco
iShares iShares (BlackRock) product workbooks iShares
JPMorgan am.jpmorgan.com fund explorer + product-data JSON JPMorgan
NEOS neosfunds.com lineup table + official fund pages + daily holdings CSV Neos
Northern Trust etfs.ntam.northerntrust.com funds list + per-fund CSV/JSON downloads Northern-Trust
Pacer ETFs paceretfs.com product catalog and fund pages (Cloudflare WAF; r.jina.ai proxy fallback) + SEC EDGAR N-PORT-P (Pacer Funds Trust) + Yahoo Finance history/dividends Pacer
Parametric eatonvance.com ETF catalog and Parametric product pages + SEC EDGAR N-PORT-P holdings + Yahoo Finance history/dividends Parametric
ProShares proshares.com ETF finder + fund pages + official data host ProShares
Schwab schwabassetmanagement.com product pages + CSV exports Schwab
SP Funds sp-funds.com homepage catalog, fund pages and daily holdings CSV + SEC EDGAR N-PORT-P holdings fallback + Yahoo Finance history/dividends SP-Funds
SPDR SSGA / State Street public feeds SPDR
Sprott ETFs sprottetfs.com fund pages + SEC EDGAR N-PORT-P (Sprott Funds Trust) + Yahoo Finance history/dividends Sprott
Tema ETFs Tema official fund pages + dated daily holdings CSV; SEC EDGAR N-PORT-P holdings fallback only + Yahoo Finance price/history/dividend fallback Tema
Themes ETFs themesetfs.com catalog + daily holdings CSV + Yahoo Finance history/dividends + SEC N-PORT-P holdings fallback Themes
VanEck vaneck.com ETF finder + product pages VanEck
Vanguard Vanguard product pages + SEC EDGAR N-PORT-P Vanguard
VictoryShares VCM VictoryShares catalog and product JSON + SEC EDGAR N-PORT-P holdings fallback + Yahoo Finance adjusted-market-price history VictoryShares
WisdomTree WisdomTree product table + SEC EDGAR N-PORT-P + Yahoo Finance WisdomTree
Xtrackers Official DWS catalog/US sitemap + PDP/XLSX + SEC N-PORT-P holdings fallback + Yahoo Finance daily prices/history/dividends Xtrackers

License

MIT - same as all sibling ETF repositories.

Themes ETFs, Themes Management Company LLC, and the fund names/tickers referenced here may be trademarks of their respective owners. This is an independent, unofficial tool; it is not affiliated with, endorsed by, or sponsored by Themes ETFs or Themes Management Company LLC. Public data is reproduced from the issuer's catalog/downloads, public SEC EDGAR filings, and Yahoo Finance for research purposes. All other trademarks, including index names, are the property of their respective owners.

About

Themes ETF holdings-to-watchlist: static ./api/themes feed generated by a Bun updater from themesetfs.com catalog, daily holdings CSV, Yahoo Finance history/dividends, SEC N-PORT-P holdings fallback

Topics

Resources

Stars

0 stars

Watchers

0 watching

Forks

Contributors

Languages