Full audit · regenerated from each repo's main

Dataset registry — Python lecture family

Every dataset referenced by lecture source across the 8 synced repos. The successor to the hand-built 2026-07-15 audit — same taxonomy, now generated.

40distinct static data files referenced
24data files committed across the repos
24committed files no code cell references
0legacy-repo URLs (8 on 2026-07-15)
23lectures fetching live API data
40datasets fully migrated to data-lectures

1Hosting patterns in use

Eight distinct patterns. The 2026-07-15 audit found seven; adding lecture-wasm surfaced the sibling repo URL pattern, and the migration added data-lectures while retiring legacy-repo URL entirely.

PatternWhat it looks likeRefsWhere
data-lecturesreads this repo's published tree — the target state65advanced.myst, intro, programming, python.myst, wasm
%%file embeddedwritten by the lecture itself, then read back12dp, intro, jax, programming, python.myst, wasm
live APIfetched at build time from a third-party service23 lecturesadvanced.myst, intro, programming, python.myst, wasm

2 URL spellings for "raw file on GitHub"

Down from six — the blob/master…?raw=true spelling retired with the legacy-repo refs. One canonical spelling is a styleguide question for QuantEcon.manual#108.

2Registry A — static dataset files (40)

Every distinct file a lecture reads, grouped by the repo that owns the bytes. lecture-wasm mirrors intro's lectures and reads intro's copies by URL, so it appears as a consumer, not an owner.

FileContentsLecture(s)HostingFlags / notes
data-lectures — 40 files
NEWQDATA.csvCogley-Sargent (2005) "Drifts and Volatilities" — quarterly US inflation, unemployment and T-bill ratephillips_drifts_volatilitiesdata-lectures✓ migrated
SCF_plus_mini.csvSCF+ mini — net wealth, income and survey weights, 1950-2016inequality, inequality (wasm)data-lectures✓ migrated
SCF_plus_mini_no_weights.csvSCF+ mini, weight-expanded — net wealth and income, 1950-2016mle, mle (wasm)data-lectures✓ migrated
acs_data_summary.csvAmerican Community Survey — earnings count, mean and dispersion by occupation cellmatch_transportdata-lectures✓ migrated
ames_house_prices.csvAmes, Iowa — residential house sales, 2006-2010bivariate_dist, fitting_distributions, observed_distributionsdata-lectures✓ migrated
assignat.xlsxFrench Revolution — assignat issues, budgets and seigniorage (Sargent-Velde)french_rev, french_rev (wasm)data-lectures✓ migrated
bbh_macro_quarterly.csvBhandari-Borovička-Ho replication — quarterly US macro series for the belief-wedge VAR, 1955Q1-2019Q4subjective_beliefs_business_cyclesdata-lectures✓ migrated
bbh_michigan_monthly.csvMichigan Surveys of Consumers monthly aggregates and the US unemployment rate, 1978-01 to 2020-03 (BBH replication extract)subjective_beliefs_business_cyclesdata-lectures✓ migrated
caron.npyFrench Revolution — monthly specie value of the assignat, 1791-1796french_rev, french_rev (wasm)data-lectures✓ migrated
chapter_3.xlsxThe Ends of Four Big Inflations — appendix tables, transcribedinflation_history, inflation_history (wasm)data-lectures✓ migrated
cities_brazil.csvWorld Population Review — Brazilian city populations, 2023heavy_tails, heavy_tails (wasm)data-lectures✓ migrated
cities_us.csvWorld Population Review — US city populations, 2023heavy_tails, heavy_tails (wasm)data-lectures✓ migrated
countries.csvWorldData.info country reference tablepandas_paneldata-lectures✓ migrated
dataBHS.csvBarillas-Hansen-Sargent "Doubts or variability?" — quarterly US log consumption and real returns, 1948Q1-2006Q4five_preferencesdata-lectures✓ migrated
dette.xlsxFrench Revolution — public debt, military spending and revenues (Sargent-Velde)french_rev, french_rev (wasm)data-lectures✓ migrated
employ.csvEurostat employment in Europe — by age and sex, 2007–2016pandas_paneldata-lectures✓ migrated
epl_match_goals.csvEnglish Premier League — full-time scores, 2015-16 to 2024-25fitting_distributionsdata-lectures✓ migrated
fig_3.xlsxFrench Revolution — figure 3 series (Sargent-Velde)french_rev, french_rev (wasm)data-lectures✓ migrated
forbes-billionaires.csvForbes Billionaires — individual net worthheavy_tails, heavy_tails (wasm)data-lectures✓ migrated
forbes-global2000.csvForbes Global 2000 — firm size measuresheavy_tails, heavy_tails (wasm)data-lectures✓ migrated
fp.dtaTreisman (2016) Russia's Billionaires — country-year panel of billionaire counts and covariatesmledata-lectures✓ migrated
fred_data.csvUS Treasury yields and NBER recessions — GS1/GS5/GS10, DFII5/DFII10 and USREC, monthly 1953-04 to 2024-12risk_aversion_or_mistaken_beliefsdata-lectures✓ migrated
hansen_jagannathan_1991_data.jsonHansen-Jagannathan (1991) replication — US asset returns 1891-1986, with annual consumption (three-table bundle)hansen_jagannathan_1991data-lectures✓ migrated
hansen_singleton_1982_data.csvHansen-Singleton (1982) replication — monthly US gross real market return and consumption growth, 1959-1978hansen_singleton_1982data-lectures✓ migrated
hansen_singleton_1983_data.csvHansen-Singleton (1983) replication — monthly US returns, consumption and inflation, 1959-1978hansen_singleton_1983data-lectures✓ migrated
japan_deaths_by_age.csvJapan — deaths by single year of age, 2023fitting_distributions, observed_distributionsdata-lectures✓ migrated
japan_earthquakes.csvJapan region — earthquakes of magnitude 5 and above, 2000-2024fitting_distributionsdata-lectures✓ migrated
japan_population_by_age.csvJapan — population by single year of age, 2024prob_distdata-lectures✓ migrated
life-expectancy-vs-gdp-per-capita.csvLife expectancy vs GDP per capita — Our World in Data grapher exportsimple_linear_regression, simple_linear_regression (wasm)data-lectures✓ migrated
lingcod_msy_recovery.csvPacific Coast lingcod — biomass and fishing pressure relative to MSYmsy_fisherydata-lectures✓ migrated
longprices.xlsPrice levels in four hard-currency countries, 1600-2000inflation_history, inflation_history (wasm)data-lectures✓ migrated
maketable1.dtaAcemoglu-Johnson-Robinson (2001) colonial origins — table 1 replication dataolsdata-lectures✓ migrated
maketable2.dtaAcemoglu-Johnson-Robinson (2001) colonial origins — table 2 replication dataolsdata-lectures✓ migrated
maketable4.dtaAcemoglu-Johnson-Robinson (2001) colonial origins — table 4 replication dataolsdata-lectures✓ migrated
mpd2020.xlsxMaddison Project Database 2020 — GDP per capita and population, 1 CE to 2018long_run_growth, long_run_growth (wasm)data-lectures✓ migrated
nom_balances.npyFrench Revolution — monthly nominal assignat balances, 1789-1796french_rev, french_rev (wasm)data-lectures✓ migrated
realwage.csvOECD real minimum wages — 32 countries, 2006–2016pandas_paneldata-lectures✓ migrated
test_pwt.csvPenn World Table teaching extract — eight countries, year 2000pandas, polarsdata-lectures✓ migrated
us_adult_heights.csvUnited States — adult standing height by sex, NHANES 2015-2018fitting_distributions, observed_distributions, prob_distdata-lectures✓ migrated
usa-gini-nwealth-tincome-lincome.csvUS Gini coefficients — net wealth, total income and labour income, 1950-2016inequality, inequality (wasm)data-lectures✓ migrated

3Registry B — data embedded in lecture source (8 files)

Written to the working directory by the lecture itself (%%file, %%writefile, or an in-lecture open(…, 'w')), then read back. Self-contained everywhere — including Colab — at the cost of data living inside narrative source.

FileLecture(s)Series
graph.txtshort_pathdp · intro · jax · wasm — same data maintained in 4 repos
newfile.txtpython_essentialsprogramming
numbers.txtdebuggingprogramming
output.txtpython_essentialsprogramming
output2.txtpython_essentialsprogramming
test_table.csvpython_advanced_featuresprogramming
us_cities.txtpython_advanced_features, python_essentialsprogramming
web_graph_data.txtfinite_markovpython.myst

4Registry C — live API data (23 lectures)

Fetched at build time from third-party services — the build-reproducibility surface, and the blocker for the WASM/JupyterLite target (pyodide cannot reach most of these APIs; that is lecture-wasm's core problem, meta#143).

Provider / accessSeries or tickersLectureSeriesWhy live
FRED fredgraph.csv URLPCND, PCESV, DPCERD3Q086SBEA, CNP16OVdoubts_or_variabilityadvanced.mystincidental
FRED fredgraph.csv URLUSRECsubjective_beliefs_business_cyclesadvanced.mystincidental
FRED pandas_datareaderUNRATE, USREC, M0892AUSM156SNBR, UMCSENT, CPILFESL, INDPRObusiness_cycleintromixed
snapshot pipeline already exists (data-lectures business_cycle_data.csv) but is unadopted
FRED fredgraph.csv URLUNRATEpandasprogrammingAPI is the lesson
FRED fredgraph.csv URLUNRATEpolarsprogrammingAPI is the lesson
mirror of pandas' FRED section — the lesson is pl.read_csv from a URL
FRED fredgraph.csv URLCPIAUCSL, UNRATE, TB3MSphillips_drifts_volatilitiespython.mystincidental
splices post-2000Q4 observations onto the Cogley-Sargent sample (NEWQDATA.csv) to test whether the extra quarter-century changes the findings. Snapshot-ready, but a snapshot would need periodic refresh — the section's point is the *latest* vintage
FRED pandas_datareaderPCEPI, GDPC1, GDPPOT, FEDFUNDSphillips_lost_conquestpython.mystincidental
FRED pandas_datareaderCPIAUCNS, LNS14000028phillips_two_storiespython.mystincidental
LNS14000028 is the unemployment rate for white men 20+
FRED pandas_datareaderM2SL, GDPDEF, GDPC1, TB3MSsargent_suricopython.mystincidental
FRED pandas_datareaderUNRATEunemployment_linearpython.mystincidental
FRED pandas_datareaderUNRATEunemployment_shockspython.mystincidental
FRED pandas_datareaderUNRATE, USREC, M0892AUSM156SNBR, UMCSENT, CPILFESL, INDPRObusiness_cyclewasmmixed
mirror of intro's business_cycle; cannot run under pyodide
FRED pandas_datareaderhistorical mirror — intro has since moved off pandas_datareader hereheavy_tailswasmincidental
mirror of an older intro heavy_tails
World Bank wbgapiNY.GDP.MKTP.KD.ZG, SL.UEM.TOTL.NE.ZS, FS.AST.PRVT.GD.ZSbusiness_cycleintromixed
teaches wb.series.info querying and metadata as part of the narrative
World Bank wbgapiNY.GDP.MKTP.CDheavy_tailsintroincidental
World Bank wbgapiSI.POV.GINI, NY.GDP.PCAP.KDinequalityintromixed
demonstrates searching for the Gini series ID with wbgapi
World Bank wbgapiGC.DOD.TOTL.GD.ZSpandasprogrammingAPI is the lesson
the section is literally "Using wbgapi and yfinance to Access Data"
World Bank wbgapiNY.GDP.MKTP.KD.ZG, SL.UEM.TOTL.NE.ZS, FS.AST.PRVT.GD.ZSbusiness_cyclewasmmixed
mirror of intro's business_cycle; cannot run under pyodide
World Bank wbgapiSI.POV.GINI, NY.GDP.PCAP.KDinequalitywasmmixed
mirror of intro's inequality
Yahoo Finance yfinanceAMZN, COSTbivariate_distintroincidental
two downloads in a solved exercise, fitting a bivariate normal to monthly returns; the lecture points at observed_distributions for the same workflow, so the fetch is the illustration's input rather than its lesson
Yahoo Finance yfinanceCT=Fcommod_priceintroincidental
Yahoo Finance yfinanceAMZNfitting_distributionsintroincidental
one download, for the section showing that a normal fit to monthly returns fails in the tails
Yahoo Finance yfinanceAMZN, BTC-USDheavy_tailsintroincidental
Yahoo Finance yfinanceAMZN, COSTobserved_distributionsintroincidental
moved here from prob_dist by lecture-python-intro#811, which split the observed-data half of that lecture into observed_distributions; prob_dist no longer calls yfinance
Yahoo Finance yfinance11-ticker exercise listpandasprogrammingAPI is the lesson
Yahoo Finance yfinance11-ticker exercise listpolarsprogrammingAPI is the lesson
mirror of pandas' yfinance exercise, rewritten with pl.DataFrame
Yahoo Finance yfinance^IXICkesten_processespython.mystincidental
Yahoo Finance yfinanceCT=Fcommod_pricewasmincidental
mirror of intro's commod_price
Yahoo Finance yfinanceAMZN, BTC-USDheavy_tailswasmincidental
mirror of intro's heavy_tails
Yahoo Finance yfinanceAMZN, COSTprob_distwasmincidental
mirror of intro's prob_dist
API is the lesson the fetch workflow is the teaching point — keep live mixed discovery is taught, plots could read a snapshot incidental just needs a series — snapshot-ready

5Cross-series reuse

Files consumed by more than one repo. Before the migration the only true cross-series files were the pandas_panel trio; the wasm mirror now multiplies every intro dataset into a second consumer.

DatasetConsumersHow it's shared today
SCF_plus_mini.csvintro + wasmall consumers read data-lectures — the target state
SCF_plus_mini_no_weights.csvintro + wasmall consumers read data-lectures — the target state
assignat.xlsxintro + wasmall consumers read data-lectures — the target state
caron.npyintro + wasmall consumers read data-lectures — the target state
chapter_3.xlsxintro + wasmall consumers read data-lectures — the target state
cities_brazil.csvintro + wasmall consumers read data-lectures — the target state
cities_us.csvintro + wasmall consumers read data-lectures — the target state
countries.csvprogramming + python.mystall consumers read data-lectures — the target state
dette.xlsxintro + wasmall consumers read data-lectures — the target state
employ.csvprogramming + python.mystall consumers read data-lectures — the target state
fig_3.xlsxintro + wasmall consumers read data-lectures — the target state
forbes-billionaires.csvintro + wasmall consumers read data-lectures — the target state
forbes-global2000.csvintro + wasmall consumers read data-lectures — the target state
life-expectancy-vs-gdp-per-capita.csvintro + wasmall consumers read data-lectures — the target state
longprices.xlsintro + wasmall consumers read data-lectures — the target state
mpd2020.xlsxintro + wasmall consumers read data-lectures — the target state
nom_balances.npyintro + wasmall consumers read data-lectures — the target state
realwage.csvprogramming + python.mystall consumers read data-lectures — the target state
usa-gini-nwealth-tincome-lincome.csvintro + wasmall consumers read data-lectures — the target state
graph.txtdp + intro + jax + wasmduplicated as %%file blocks — 4 copies of the same data to keep in sync

6Committed but unreferenced data files (24)

Files in a repo's tree that no executed code cell reads. Orphan-sweep candidates are tracked in meta#337; shadowed and exercise-download files are functional, just invisible to a URL-level audit.

orphan dead weight — nothing reads it shadowed a %%file cell regenerates it at build time mirror-orphan wasm copy; the wasm lecture reads intro's URL exercise download prose link target — referenced, not by code
FileStateWhy
lecture-python-intro — 4 files
lectures/datasets/GDP_per_capita_world_bank.csvorphanno reference anywhere; the QuantEcon/data copy was dropped at Phase 2
lectures/datasets/Metadata_Country_API_NY.GDP.PCAP.CD_DS2_en_csv_v2_4770417.csvorphanmetadata twin of the World Bank GDP-per-capita orphan
lectures/datasets/fig_3.odsorphansource-format twin of fig_3.xlsx; referenced by nothing
lectures/graph.txtshadowedshort_path regenerates it via %%file before reading it, so the committed bytes are never consumed — the build prints "Overwriting graph.txt". It became an orphan when QuantEcon/lecture-wasm#63 removed the only URL read of it in the organisation; deletable as Track X
lecture-python-programming — 5 files
lectures/_static/lecture_specific/pandas/data/ticker_data.csvorphanno reference anywhere
lectures/_static/lecture_specific/python_advanced_features/numbers.txtshadoweddebugging lecture writes it with %%file
lectures/_static/lecture_specific/python_advanced_features/test_table.csvexercise downloadprose exercise link tells the reader to download this exact URL — keep
lectures/_static/lecture_specific/python_foundations/test_table.csvorphanduplicate of the python_advanced_features copy
lectures/_static/lecture_specific/python_foundations/us_cities.txtshadowedpython_essentials writes it with %%writefile
lecture-python.myst — 2 files
lectures/_static/lecture_specific/finite_markov/web_graph_data.txtshadowedfinite_markov writes it with %%file
lectures/web_graph_data.txtshadowedshadowed duplicate at lectures/ root
lecture-dp — 10 files
lectures/_static/lecture_specific/finite_markov/web_graph_data.txtorphaninherited from python.myst; no finite_markov data reference in this repo
lectures/_static/lecture_specific/match_transport/acs_data_summary.csvorphaninherited copy, no consuming lecture in this repo
lectures/_static/lecture_specific/mle/fp.dtaorphaninherited from python.myst, no consuming lecture
lectures/_static/lecture_specific/ols/maketable1.dtaorphaninherited from python.myst, no consuming lecture
lectures/_static/lecture_specific/ols/maketable2.dtaorphaninherited from python.myst, no consuming lecture
lectures/_static/lecture_specific/ols/maketable4.dtaorphaninherited from python.myst, no consuming lecture
lectures/_static/lecture_specific/pandas_panel/countries.csvorphaninherited copy; the consuming lectures live in other repos and now read data-lectures
lectures/_static/lecture_specific/pandas_panel/employ.csvorphaninherited copy; the consuming lectures live in other repos and now read data-lectures
lectures/_static/lecture_specific/pandas_panel/realwage.csvorphaninherited copy; the consuming lectures live in other repos and now read data-lectures
lectures/graph.txtshadowedshort_path regenerates it via %%file
lecture-wasm — 2 files
lectures/datasets/GDP_per_capita_world_bank.csvmirror-orphanorphan in intro, mirrored into wasm
lectures/datasets/Metadata_Country_API_NY.GDP.PCAP.CD_DS2_en_csv_v2_4770417.csvmirror-orphanorphan in intro, mirrored into wasm
continuous_time_mcs — 1 files
lectures/old_stuff.txtorphanscratch file at lectures/ root; referenced by nothing

7Provenance — how each file came to exist

The axis the manifest convention formalizes (verbatim / constructed / dynamic snapshot, meta#336). For migrated datasets this comes from their manifests; for everything else, from the curated audit annotations.

12verbatim — third-party file as distributed; the citation is the provenance
18constructed · builder committed — built by a script/notebook that is committed
10constructed · pipeline lost — construction documented but never committed
FileProvenanceRecorded in
verbatim — 12
assignat.xlsxFrench Revolution — assignat issues, budgets and seigniorage (Sargent-Velde)manifest
cities_brazil.csvWorld Population Review — Brazilian city populations, 2023manifest
cities_us.csvWorld Population Review — US city populations, 2023manifest
countries.csvWorldData.info country reference tablemanifest
dette.xlsxFrench Revolution — public debt, military spending and revenues (Sargent-Velde)manifest
fig_3.xlsxFrench Revolution — figure 3 series (Sargent-Velde)manifest
fp.dtaTreisman (2016) Russia's Billionaires — country-year panel of billionaire counts and covariatesmanifest
life-expectancy-vs-gdp-per-capita.csvLife expectancy vs GDP per capita — Our World in Data grapher exportmanifest
longprices.xlsPrice levels in four hard-currency countries, 1600-2000manifest
maketable1.dtaAcemoglu-Johnson-Robinson (2001) colonial origins — table 1 replication datamanifest
maketable2.dtaAcemoglu-Johnson-Robinson (2001) colonial origins — table 2 replication datamanifest
maketable4.dtaAcemoglu-Johnson-Robinson (2001) colonial origins — table 4 replication datamanifest
constructed · builder committed — 18
NEWQDATA.csvCogley-Sargent (2005) "Drifts and Volatilities" — quarterly US inflation, unemployment and T-bill ratemanifest
SCF_plus_mini.csvSCF+ mini — net wealth, income and survey weights, 1950-2016manifest
SCF_plus_mini_no_weights.csvSCF+ mini, weight-expanded — net wealth and income, 1950-2016manifest
ames_house_prices.csvAmes, Iowa — residential house sales, 2006-2010manifest
bbh_macro_quarterly.csvBhandari-Borovička-Ho replication — quarterly US macro series for the belief-wedge VAR, 1955Q1-2019Q4manifest
bbh_michigan_monthly.csvMichigan Surveys of Consumers monthly aggregates and the US unemployment rate, 1978-01 to 2020-03 (BBH replication extract)manifest
dataBHS.csvBarillas-Hansen-Sargent "Doubts or variability?" — quarterly US log consumption and real returns, 1948Q1-2006Q4manifest
epl_match_goals.csvEnglish Premier League — full-time scores, 2015-16 to 2024-25manifest
forbes-billionaires.csvForbes Billionaires — individual net worthmanifest
forbes-global2000.csvForbes Global 2000 — firm size measuresmanifest
fred_data.csvUS Treasury yields and NBER recessions — GS1/GS5/GS10, DFII5/DFII10 and USREC, monthly 1953-04 to 2024-12manifest
hansen_singleton_1982_data.csvHansen-Singleton (1982) replication — monthly US gross real market return and consumption growth, 1959-1978manifest
hansen_singleton_1983_data.csvHansen-Singleton (1983) replication — monthly US returns, consumption and inflation, 1959-1978manifest
japan_deaths_by_age.csvJapan — deaths by single year of age, 2023manifest
japan_earthquakes.csvJapan region — earthquakes of magnitude 5 and above, 2000-2024manifest
japan_population_by_age.csvJapan — population by single year of age, 2024manifest
us_adult_heights.csvUnited States — adult standing height by sex, NHANES 2015-2018manifest
usa-gini-nwealth-tincome-lincome.csvUS Gini coefficients — net wealth, total income and labour income, 1950-2016manifest
constructed · pipeline lost — 10
acs_data_summary.csvAmerican Community Survey — earnings count, mean and dispersion by occupation cellmanifest
caron.npyFrench Revolution — monthly specie value of the assignat, 1791-1796manifest
chapter_3.xlsxThe Ends of Four Big Inflations — appendix tables, transcribedmanifest
employ.csvEurostat employment in Europe — by age and sex, 2007–2016manifest
hansen_jagannathan_1991_data.jsonHansen-Jagannathan (1991) replication — US asset returns 1891-1986, with annual consumption (three-table bundle)manifest
lingcod_msy_recovery.csvPacific Coast lingcod — biomass and fishing pressure relative to MSYmanifest
mpd2020.xlsxMaddison Project Database 2020 — GDP per capita and population, 1 CE to 2018manifest
nom_balances.npyFrench Revolution — monthly nominal assignat balances, 1789-1796manifest
realwage.csvOECD real minimum wages — 32 countries, 2006–2016manifest
test_pwt.csvPenn World Table teaching extract — eight countries, year 2000manifest

8Draft rules for styleguide/datasets.md

Distilled from the live-API pedagogy analysis (§4) and the migration pilots; feeding QuantEcon.manual#108.

1. Default to snapshots.

A lecture reads data from a stable snapshot URL (data-lectures). Live API calls are the exception and require a reason recorded in the lecture source.

2. Live APIs are for teaching data access, not for getting data.

A live call is justified when the fetch workflow is itself the lesson (the pandas lecture's wbgapi/yfinance sections) or when currency is the point. “The lecture needs series X” is not a reason — that's what snapshots are for.

3. Every live-API lecture gets a snapshot twin.

A refresh builder in data-lectures producing the snapshot (the business_cycle_data.csv pattern, already prototyped). Breakage becomes a one-line URL switch, and the WASM build always uses the twin — pyodide cannot reach the live APIs at all.

4. Prefer direct CSV endpoints over wrapper libraries.

Where live access stays, fredgraph.csv-style URLs beat pandas_datareader-style wrappers: one less dependency, and the git history shows the wrappers are what broke.

5. Snapshots carry provenance metadata.

Source, series IDs, retrieval date, license and refresh cadence — the manifest schema in this repo is the template; the provenance classes in §7 say which fields are required.