Full audit · regenerated from each repo's main

Dataset registry — Python lecture family

Every dataset referenced by lecture source across the 8 synced repos. The successor to the hand-built 2026-07-15 audit — same taxonomy, now generated.

36distinct static data files referenced
61data files committed across the repos
35committed files no code cell references
0legacy-repo URLs (8 on 2026-07-15)
16lectures fetching live API data
4datasets fully migrated to data-lectures

1Hosting patterns in use

Eight distinct patterns. The 2026-07-15 audit found seven; adding lecture-wasm surfaced the sibling repo URL pattern, and the migration added data-lectures while retiring legacy-repo URL entirely.

PatternWhat it looks likeRefsWhere
data-lecturesreads this repo's published tree — the target state7intro, programming, python.myst
own-repo URLfetches a file committed in its own repo via a GitHub raw URL17advanced.myst, intro, programming, python.myst
local pathpd.read_csv('…') relative path — no URL; breaks in Colab/download8advanced.myst, intro
sibling repo URLfetches another lecture repo's committed copy by URL11wasm
external data repoQuantEcon/high_dim_data via raw and media (LFS) hosts12intro, wasm
%%file embeddedwritten by the lecture itself, then read back11dp, intro, jax, programming, python.myst
live APIfetched at build time from a third-party service16 lecturesadvanced.myst, intro, programming, python.myst, wasm

5 URL spellings for "raw file on GitHub"

Down from six — the blob/master…?raw=true spelling retired with the legacy-repo refs. One canonical spelling is a styleguide question for QuantEcon.manual#108.

2Registry A — static dataset files (36)

Every distinct file a lecture reads, grouped by the repo that owns the bytes. lecture-wasm mirrors intro's lectures and reads intro's copies by URL, so it appears as a consumer, not an owner.

FileContentsLecture(s)HostingFlags / notes
data-lectures — 4 files
countries.csvWorldData.info country reference tablepandas_paneldata-lectures✓ migrated
employ.csvEurostat employment in Europe — by age and sex, 2007–2016pandas_paneldata-lectures✓ migrated
lingcod_msy_recovery.csvPacific Coast lingcod — biomass and fishing pressure relative to MSYmsy_fisherydata-lectures✓ migrated
realwage.csvOECD real minimum wages — 32 countries, 2006–2016pandas_paneldata-lectures✓ migrated
lecture-python-intro — 19 files
SCF_plus_mini.csvSCF+ wealth/income survey extractinequality, inequality (wasm)external data repobuilt by generating_mini.md (executable MyST) in high_dim_data from SCF_plus.dta
SCF_plus_mini_no_weights.csvSCF+ extract, no survey weightsmle, mle (wasm)external data repovariant of the SCF_plus_mini pipeline. Was pinned to feature branch update_scf_noweights (critical flag in the 2026-07-15 audit); reads high_dim_data main since lecture-python-intro#793
assignat.xlsxAssignat issuance / price data, French Revolutionfrench_rev, french_rev (wasm)own-repo URL sibling repo URL
caron.npyFrench Revolution money balances (Caron)french_rev, french_rev (wasm)local path sibling repo URL⚠ breaks in Colab/notebook download hand-prepared NumPy array, no construction record
chapter_3.xlsxHyperinflation tables, Sargent "Ends of Four Big Inflations"inflation_history, inflation_history (wasm)own-repo URL sibling repo URLhand-transcribed by authors
cities_brazil.csvBrazilian city populationsheavy_tails, heavy_tails (wasm)external data repovia media.githubusercontent (LFS) manual download from worldpopulationreview.com, documented in high_dim_data README
cities_us.csvUS city populationsheavy_tails, heavy_tails (wasm)external data repovia media.githubusercontent (LFS) manual download from worldpopulationreview.com, documented in high_dim_data README
dette.xlsxFrench government debt seriesfrench_rev, french_rev (wasm)own-repo URL sibling repo URL
fig_3.xlsxFrench Revolution fiscal datafrench_rev, french_rev (wasm)own-repo URL sibling repo URL
forbes-billionaires.csvForbes billionaires listheavy_tails, heavy_tails (wasm)external data repovia media.githubusercontent (LFS) webscrape_forbes.ipynb in high_dim_data (Forbes API)
forbes-global2000.csvForbes Global 2000 firm sizeheavy_tails, heavy_tails (wasm)external data repovia media.githubusercontent (LFS) webscrape_forbes.ipynb in high_dim_data (Forbes API)
graph.txt15-node weighted digraph for the shortest-path problemshort_path (wasm)sibling repo URLmaintained as %%file blocks in intro, dp and jax; lecture-wasm instead fetches intro's committed copy by URL (requests.get), making that copy load-bearing — it was merely "shadowed" in the 2026-07-15 audit
japan_population_by_age.xlsxJapan population by age group (prob_dist bimodality example)prob_distlocal path⚠ breaks in Colab/notebook download new since the 2026-07-15 audit added by lecture-python-intro#790; upstream source not yet recorded
life-expectancy-vs-gdp-per-capita.csvOur World in Data, life expectancy vs GDP per capitasimple_linear_regression, simple_linear_regression (wasm)own-repo URL sibling repo URLOWID export, as downloaded
longprices.xlsLong-run price levels (Sargent–Velde)inflation_history, inflation_history (wasm)own-repo URL sibling repo URLhand-transcribed by authors
mpd2020.xlsxMaddison Project Database 2020 (GDP per capita, long run)long_run_growth, long_run_growth (wasm)own-repo URL sibling repo URL
nom_balances.npyNominal balances, French Revolutionfrench_rev, french_rev (wasm)local path sibling repo URL⚠ breaks in Colab/notebook download hand-prepared NumPy array, no construction record
us_adult_heights.csvHeights of US adults, from NHANES (prob_dist motivating example)prob_distlocal path⚠ breaks in Colab/notebook download new since the 2026-07-15 audit added by lecture-python-intro#790; NHANES-derived, extraction not scripted
usa-gini-nwealth-tincome-lincome.csvUS Gini coefficients, wealth & income (long series)inequality, inequality (wasm)own-repo URL sibling repo URLbuilt by inequality/data.ipynb (committed beside it) from SCF_plus_mini
lecture-python-programming — 1 files
test_pwt.csvPenn World Table 7.0 extractpandasown-repo URL
lecture-python.myst — 6 files
fp.dtaTreisman (2016) Russia billionaires replication datamleown-repo URL
hansen_singleton_1982_data.csvFRED-derived consumption/returns snapshothansen_singleton_1982own-repo URLmake_data.py + README beside the data (exact series and sample documented)
hansen_singleton_1983_data.csvFRED-derived snapshothansen_singleton_1983own-repo URLmake_data.py + README beside the data
maketable1.dtaAcemoglu–Johnson–Robinson (2001) replication, table 1olsown-repo URL
maketable2.dtaAJR (2001) replication, table 2olsown-repo URL
maketable4.dtaAJR (2001) replication, table 4olsown-repo URL
lecture-python-advanced.myst — 6 files
acs_data_summary.csvACS occupation summarymatch_transportlocal path⚠ breaks in Colab/notebook download construction (ACS filters, grouping, sorting) described in lecture prose only
bbh_macro_quarterly.csvMacro quarterly series (Bhandari et al.)subjective_beliefs_business_cycleslocal path⚠ breaks in Colab/notebook download extracted from the paper's replication package (Zenodo DOI, CC BY 4.0); extraction not scripted
bbh_michigan_monthly.csvMichigan survey monthly series (Bhandari et al.)subjective_beliefs_business_cycleslocal path⚠ breaks in Colab/notebook download same Zenodo replication package; extraction not scripted
dataBHS.matUS consumption/income series, MATLAB replication bundlefive_preferenceslocal path⚠ breaks in Colab/notebook download lives at lectures/ root
fred_data.csvFRED snapshot — GS1, GS5, GS10, DFII5, DFII10, USRECrisk_aversion_or_mistaken_beliefsown-repo URLlecture names the FRED series; no build script anywhere
hansen_jagannathan_1991_data.jsonHansen–Jagannathan (1991) asset-returns bundlehansen_jagannathan_1991own-repo URLlecture documents 3 sources (FRED yields deflated by CPIAUCSL, …); no build script

3Registry B — data embedded in lecture source (8 files)

Written to the working directory by the lecture itself (%%file, %%writefile, or an in-lecture open(…, 'w')), then read back. Self-contained everywhere — including Colab — at the cost of data living inside narrative source.

FileLecture(s)Series
graph.txtshort_pathdp · intro · jax — same data maintained in 3 repos
newfile.txtpython_essentialsprogramming
numbers.txtdebuggingprogramming
output.txtpython_essentialsprogramming
output2.txtpython_essentialsprogramming
test_table.csvpython_advanced_featuresprogramming
us_cities.txtpython_advanced_features, python_essentialsprogramming
web_graph_data.txtfinite_markovpython.myst

4Registry C — live API data (16 lectures)

Fetched at build time from third-party services — the build-reproducibility surface, and the blocker for the WASM/JupyterLite target (pyodide cannot reach most of these APIs; that is lecture-wasm's core problem, meta#143).

Provider / accessSeries or tickersLectureSeriesWhy live
FRED fredgraph.csv URLPCND, PCESV, DPCERD3Q086SBEA, CNP16OVdoubts_or_variabilityadvanced.mystincidental
FRED fredgraph.csv URLUSRECsubjective_beliefs_business_cyclesadvanced.mystincidental
FRED pandas_datareaderUNRATE, USREC, M0892AUSM156SNBR, UMCSENT, CPILFESL, INDPRObusiness_cycleintromixed
snapshot pipeline already exists (data-lectures business_cycle_data.csv) but is unadopted
FRED fredgraph.csv URLUNRATEpandasprogrammingAPI is the lesson
FRED pandas_datareaderUNRATEunemployment_linearpython.mystincidental
FRED pandas_datareaderUNRATEunemployment_shockspython.mystincidental
FRED pandas_datareaderUNRATE, USREC, M0892AUSM156SNBR, UMCSENT, CPILFESL, INDPRObusiness_cyclewasmmixed
mirror of intro's business_cycle; cannot run under pyodide
FRED pandas_datareaderhistorical mirror — intro has since moved off pandas_datareader hereheavy_tailswasmincidental
mirror of an older intro heavy_tails
World Bank wbgapiNY.GDP.MKTP.KD.ZG, SL.UEM.TOTL.NE.ZS, FS.AST.PRVT.GD.ZSbusiness_cycleintromixed
teaches wb.series.info querying and metadata as part of the narrative
World Bank wbgapiNY.GDP.MKTP.CDheavy_tailsintroincidental
World Bank wbgapiSI.POV.GINI, NY.GDP.PCAP.KDinequalityintromixed
demonstrates searching for the Gini series ID with wbgapi
World Bank wbgapiGC.DOD.TOTL.GD.ZSpandasprogrammingAPI is the lesson
the section is literally "Using wbgapi and yfinance to Access Data"
World Bank wbgapiNY.GDP.MKTP.KD.ZG, SL.UEM.TOTL.NE.ZS, FS.AST.PRVT.GD.ZSbusiness_cyclewasmmixed
mirror of intro's business_cycle; cannot run under pyodide
World Bank wbgapiSI.POV.GINI, NY.GDP.PCAP.KDinequalitywasmmixed
mirror of intro's inequality
Yahoo Finance yfinanceCT=Fcommod_priceintroincidental
Yahoo Finance yfinanceAMZN, BTC-USDheavy_tailsintroincidental
Yahoo Finance yfinanceAMZN, COSTprob_distintroincidental
Yahoo Finance yfinance11-ticker exercise listpandasprogrammingAPI is the lesson
Yahoo Finance yfinance^IXICkesten_processespython.mystincidental
Yahoo Finance yfinanceCT=Fcommod_pricewasmincidental
mirror of intro's commod_price
Yahoo Finance yfinanceAMZN, BTC-USDheavy_tailswasmincidental
mirror of intro's heavy_tails
Yahoo Finance yfinanceAMZN, COSTprob_distwasmincidental
mirror of intro's prob_dist
API is the lesson the fetch workflow is the teaching point — keep live mixed discovery is taught, plots could read a snapshot incidental just needs a series — snapshot-ready

5Cross-series reuse

Files consumed by more than one repo. Before the migration the only true cross-series files were the pandas_panel trio; the wasm mirror now multiplies every intro dataset into a second consumer.

DatasetConsumersHow it's shared today
SCF_plus_mini.csvintro + wasmwasm mirrors intro and reads intro's copy by URL
SCF_plus_mini_no_weights.csvintro + wasmwasm mirrors intro and reads intro's copy by URL
assignat.xlsxintro + wasmwasm mirrors intro and reads intro's copy by URL
caron.npyintro + wasmwasm mirrors intro and reads intro's copy by URL
chapter_3.xlsxintro + wasmwasm mirrors intro and reads intro's copy by URL
cities_brazil.csvintro + wasmwasm mirrors intro and reads intro's copy by URL
cities_us.csvintro + wasmwasm mirrors intro and reads intro's copy by URL
countries.csvprogramming + python.mystall consumers read data-lectures — the target state
dette.xlsxintro + wasmwasm mirrors intro and reads intro's copy by URL
employ.csvprogramming + python.mystall consumers read data-lectures — the target state
fig_3.xlsxintro + wasmwasm mirrors intro and reads intro's copy by URL
forbes-billionaires.csvintro + wasmwasm mirrors intro and reads intro's copy by URL
forbes-global2000.csvintro + wasmwasm mirrors intro and reads intro's copy by URL
life-expectancy-vs-gdp-per-capita.csvintro + wasmwasm mirrors intro and reads intro's copy by URL
longprices.xlsintro + wasmwasm mirrors intro and reads intro's copy by URL
mpd2020.xlsxintro + wasmwasm mirrors intro and reads intro's copy by URL
nom_balances.npyintro + wasmwasm mirrors intro and reads intro's copy by URL
realwage.csvprogramming + python.mystall consumers read data-lectures — the target state
usa-gini-nwealth-tincome-lincome.csvintro + wasmwasm mirrors intro and reads intro's copy by URL
graph.txtdp + intro + jaxduplicated as %%file blocks — 3 copies of the same data to keep in sync

6Committed but unreferenced data files (35)

Files in a repo's tree that no executed code cell reads. Orphan-sweep candidates are tracked in meta#337; shadowed and exercise-download files are functional, just invisible to a URL-level audit.

orphan dead weight — nothing reads it shadowed a %%file cell regenerates it at build time mirror-orphan wasm copy; the wasm lecture reads intro's URL exercise download prose link target — referenced, not by code
FileStateWhy
lecture-python-intro — 3 files
lectures/datasets/GDP_per_capita_world_bank.csvorphanno reference anywhere; the QuantEcon/data copy was dropped at Phase 2
lectures/datasets/Metadata_Country_API_NY.GDP.PCAP.CD_DS2_en_csv_v2_4770417.csvorphanmetadata twin of the World Bank GDP-per-capita orphan
lectures/datasets/fig_3.odsorphansource-format twin of fig_3.xlsx; referenced by nothing
lecture-python-programming — 5 files
lectures/_static/lecture_specific/pandas/data/ticker_data.csvorphanno reference anywhere
lectures/_static/lecture_specific/python_advanced_features/numbers.txtshadoweddebugging lecture writes it with %%file
lectures/_static/lecture_specific/python_advanced_features/test_table.csvexercise downloadprose exercise link tells the reader to download this exact URL — keep
lectures/_static/lecture_specific/python_foundations/test_table.csvorphanduplicate of the python_advanced_features copy
lectures/_static/lecture_specific/python_foundations/us_cities.txtshadowedpython_essentials writes it with %%writefile
lecture-python.myst — 2 files
lectures/_static/lecture_specific/finite_markov/web_graph_data.txtshadowedfinite_markov writes it with %%file
lectures/web_graph_data.txtshadowedshadowed duplicate at lectures/ root
lecture-dp — 10 files
lectures/_static/lecture_specific/finite_markov/web_graph_data.txtorphaninherited from python.myst; no finite_markov data reference in this repo
lectures/_static/lecture_specific/match_transport/acs_data_summary.csvorphaninherited copy, no consuming lecture in this repo
lectures/_static/lecture_specific/mle/fp.dtaorphaninherited from python.myst, no consuming lecture
lectures/_static/lecture_specific/ols/maketable1.dtaorphaninherited from python.myst, no consuming lecture
lectures/_static/lecture_specific/ols/maketable2.dtaorphaninherited from python.myst, no consuming lecture
lectures/_static/lecture_specific/ols/maketable4.dtaorphaninherited from python.myst, no consuming lecture
lectures/_static/lecture_specific/pandas_panel/countries.csvorphaninherited copy; the consuming lectures live in other repos and now read data-lectures
lectures/_static/lecture_specific/pandas_panel/employ.csvorphaninherited copy; the consuming lectures live in other repos and now read data-lectures
lectures/_static/lecture_specific/pandas_panel/realwage.csvorphaninherited copy; the consuming lectures live in other repos and now read data-lectures
lectures/graph.txtshadowedshort_path regenerates it via %%file
lecture-wasm — 14 files
lectures/_static/lecture_specific/inequality/usa-gini-nwealth-tincome-lincome.csvmirror-orphanwasm's inequality reads intro's copy by URL
lectures/_static/lecture_specific/simple_linear_regression/life-expectancy-vs-gdp-per-capita.csvmirror-orphanwasm's simple_linear_regression reads intro's copy by URL
lectures/datasets/GDP_per_capita_world_bank.csvmirror-orphanorphan in intro, mirrored into wasm
lectures/datasets/Metadata_Country_API_NY.GDP.PCAP.CD_DS2_en_csv_v2_4770417.csvmirror-orphanorphan in intro, mirrored into wasm
lectures/datasets/assignat.xlsxmirror-orphanwasm's french_rev reads intro's copy by URL
lectures/datasets/caron.npymirror-orphanwasm's french_rev reads intro's copy by URL
lectures/datasets/chapter_3.xlsxmirror-orphanwasm's inflation_history reads intro's copy by URL
lectures/datasets/dette.xlsxmirror-orphanwasm's french_rev reads intro's copy by URL
lectures/datasets/fig_3.odsmirror-orphanorphan in intro, mirrored into wasm
lectures/datasets/fig_3.xlsxmirror-orphanwasm's french_rev reads intro's copy by URL
lectures/datasets/longprices.xlsmirror-orphanwasm's inflation_history reads intro's copy by URL
lectures/datasets/mpd2020.xlsxmirror-orphanwasm's long_run_growth reads intro's copy by URL
lectures/datasets/nom_balances.npymirror-orphanwasm's french_rev reads intro's copy by URL
lectures/graph.txtmirror-orphanwasm's short_path fetches intro's committed graph.txt by URL, not this copy
continuous_time_mcs — 1 files
lectures/old_stuff.txtorphanscratch file at lectures/ root; referenced by nothing

7Provenance — how each file came to exist

The axis the manifest convention formalizes (verbatim / constructed / dynamic snapshot, meta#336). For migrated datasets this comes from their manifests; for everything else, from the curated audit annotations.

8verbatim — third-party file as distributed; the citation is the provenance
7constructed · builder committed — built by a script/notebook that is committed
9constructed · pipeline lost — construction documented but never committed
11author-assembled — hand-built by authors; provenance is lecture prose only
1toy — invented in-lecture teaching data
FileProvenanceRecorded in
verbatim — 8
countries.csvWorldData.info country reference tablemanifest
dataBHS.matUS consumption/income series, MATLAB replication bundleaudit annotation
fp.dtaTreisman (2016) Russia billionaires replication dataaudit annotation
life-expectancy-vs-gdp-per-capita.csvOWID export, as downloadedaudit annotation
maketable1.dtaAcemoglu–Johnson–Robinson (2001) replication, table 1audit annotation
maketable2.dtaAJR (2001) replication, table 2audit annotation
maketable4.dtaAJR (2001) replication, table 4audit annotation
mpd2020.xlsxMaddison Project Database 2020 (GDP per capita, long run)audit annotation
constructed · builder committed — 7
SCF_plus_mini.csvbuilt by generating_mini.md (executable MyST) in high_dim_data from SCF_plus.dtaaudit annotation
SCF_plus_mini_no_weights.csvvariant of the SCF_plus_mini pipeline. Was pinned to feature branch update_scf_noweights (critical flag in the 2026-07-15 audit); reads high_dim_data main since lecture-python-intro#793 audit annotation
forbes-billionaires.csvwebscrape_forbes.ipynb in high_dim_data (Forbes API)audit annotation
forbes-global2000.csvwebscrape_forbes.ipynb in high_dim_data (Forbes API)audit annotation
hansen_singleton_1982_data.csvmake_data.py + README beside the data (exact series and sample documented)audit annotation
hansen_singleton_1983_data.csvmake_data.py + README beside the dataaudit annotation
usa-gini-nwealth-tincome-lincome.csvbuilt by inequality/data.ipynb (committed beside it) from SCF_plus_miniaudit annotation
constructed · pipeline lost — 9
acs_data_summary.csvconstruction (ACS filters, grouping, sorting) described in lecture prose onlyaudit annotation
bbh_macro_quarterly.csvextracted from the paper's replication package (Zenodo DOI, CC BY 4.0); extraction not scriptedaudit annotation
bbh_michigan_monthly.csvsame Zenodo replication package; extraction not scriptedaudit annotation
employ.csvEurostat employment in Europe — by age and sex, 2007–2016manifest
fred_data.csvlecture names the FRED series; no build script anywhereaudit annotation
hansen_jagannathan_1991_data.jsonlecture documents 3 sources (FRED yields deflated by CPIAUCSL, …); no build scriptaudit annotation
lingcod_msy_recovery.csvPacific Coast lingcod — biomass and fishing pressure relative to MSYmanifest
realwage.csvOECD real minimum wages — 32 countries, 2006–2016manifest
us_adult_heights.csvadded by lecture-python-intro#790; NHANES-derived, extraction not scriptedaudit annotation
author-assembled — 11
assignat.xlsxAssignat issuance / price data, French Revolutionaudit annotation
caron.npyhand-prepared NumPy array, no construction recordaudit annotation
chapter_3.xlsxhand-transcribed by authorsaudit annotation
cities_brazil.csvmanual download from worldpopulationreview.com, documented in high_dim_data READMEaudit annotation
cities_us.csvmanual download from worldpopulationreview.com, documented in high_dim_data READMEaudit annotation
dette.xlsxFrench government debt seriesaudit annotation
fig_3.xlsxFrench Revolution fiscal dataaudit annotation
japan_population_by_age.xlsxadded by lecture-python-intro#790; upstream source not yet recordedaudit annotation
longprices.xlshand-transcribed by authorsaudit annotation
nom_balances.npyhand-prepared NumPy array, no construction recordaudit annotation
test_pwt.csvPenn World Table 7.0 extractaudit annotation
toy — 1
graph.txtmaintained as %%file blocks in intro, dp and jax; lecture-wasm instead fetches intro's committed copy by URL (requests.get), making that copy load-bearing — it was merely "shadowed" in the 2026-07-15 audit audit annotation

8Draft rules for styleguide/datasets.md

Distilled from the live-API pedagogy analysis (§4) and the migration pilots; feeding QuantEcon.manual#108.

1. Default to snapshots.

A lecture reads data from a stable snapshot URL (data-lectures). Live API calls are the exception and require a reason recorded in the lecture source.

2. Live APIs are for teaching data access, not for getting data.

A live call is justified when the fetch workflow is itself the lesson (the pandas lecture's wbgapi/yfinance sections) or when currency is the point. “The lecture needs series X” is not a reason — that's what snapshots are for.

3. Every live-API lecture gets a snapshot twin.

A refresh builder in data-lectures producing the snapshot (the business_cycle_data.csv pattern, already prototyped). Breakage becomes a one-line URL switch, and the WASM build always uses the twin — pyodide cannot reach the live APIs at all.

4. Prefer direct CSV endpoints over wrapper libraries.

Where live access stays, fredgraph.csv-style URLs beat pandas_datareader-style wrappers: one less dependency, and the git history shows the wrappers are what broke.

5. Snapshots carry provenance metadata.

Source, series IDs, retrieval date, license and refresh cadence — the manifest schema in this repo is the template; the provenance classes in §7 say which fields are required.