mccall_q#

  • Series: lecture-python.myst

  • File: lectures/mccall_q.md

  • Audit date: 2026-08-26

  • Corpus snapshot: e25fdf2345

  • Categories audited: writing, math, code, figures, references, links (JAX out of scope)

  • Overall score: 7.8 / 10

  • Priority: HIGH

Score breakdown#

Category

Score

One-line note

Writing

4/10

qe-writing-002 ×5; qe-writing-005 ×3; qe-writing-003 ×3, +4 more.

Math

9.5/10

qe-math-009 ×4.

Code

7/10

qe-code-001 ×17.

JAX

out of scope

JAX rules target lecture-jax.

Figures

7/10

qe-fig-005 ×4; qe-fig-008 ×7; qe-fig-001 ×5.

References

9/10

qe-ref-001 ×1.

Links

10/10

no mechanical violations detected.

Admonitions

N/A

no admonitions, exercises or solutions.

Issues#

Critical#

None found.

High severity#

  • [qe-code-001] (reviewer) — Follow PEP8 unless closer to mathematical notation. Count: 17. Lines: 133, 136, 186, 192, 193, 214, 494, 532, 549, 560, …. Example: binary operators and comparisons are written without surrounding spaces throughout, against the same file’s own practice elsewhere (v_next - v at 190, c + β * (v @ q) at 174). flake8 confirms: E225 at flag=0 (186), ))<=eps (192), flag=1 (193), params=[ (494), accept==0 (532), random()<=eps (560) and ))<=δ (574); E231 (no space after a comma) at the five figsize=(10,6) calls (136, 225, 627, 648, 669) and at np.zeros((len(w_new),2)) (666); and unspaced additive operators at n+1 (133, 644), qtable_new-qtable (574) and valfunc-valfunc_VFI (602) - the tight β*np.max(...) (534, 538, 540) and lr*TD (572) are not findings, since PEP8 endorses binding the higher-priority operator tighter inside a mixed expression. Three substantive items a spacing sweep would miss: 214’s for i in range(n) is nested inside for i in range(num_plots) (211) and clobbers the outer index - it works today only because the label f"iterate {i}" is consumed at 212 before the inner loop runs; 549 unpacks c, β, w = self.c, self.β, self.w in run_one_epoch and never uses any of the three (flake8 F841 x3); and the convergence flag that VFI computes and returns (186, 193, 197) is unpacked at 223 and 658 and never tested, so a run that hits max_iter=500 without converging is indistinguishable from one that converged. Line 538 is also 92 characters.

  • [qe-fig-001] — Do not set figure size unless necessary. Count: 5. Lines: 136, 225, 627, 648, 669. Example: figsize=.

  • [qe-fig-008] — Use lw=2 for line charts. Count: 7. Lines: 137, 212, 628, 629, 649, 670, 681. Example: plot() without lw=.

  • [qe-writing-002] (reviewer) — Keep writing clear, concise, and valuable. Count: 5. Lines: 42, 245, 264, 298, 699. Example: lines 42-51 spend five separate paragraphs establishing that what Q-learning learns is a Q-table, including “But it is something that is closely affiliated with it” (47) and “Sometimes a Q-function or Q-table is called a quality-function or quality-table” (51), before the definition arrives at 245. Four sentences are broken as written: “A quality function \(Q\) maps state-action pairs into optimal values. / They are tightly linked to optimal value functions” (245-247, singular subject, plural pronoun); “equals the maximum value of that a previously unemployed worker” (264); “This fact provides us with an / an alternative” (298-299, a doubled word across the line break); and “The above graphs indicates that” (699, subject-verb disagreement, and there is one graph, not several). Line 58 also ends without a full stop.

  • [qe-writing-008] — Remove excessive whitespace between words. Count: 79. Lines: 33, 36, 38, 44, 51, 56, 58, 64, 66, 89, …. Example: 2 spaces.

Medium severity#

  • [qe-fig-005] — Descriptive figure names for cross-referencing. Count: 4. Lines: 128, 221, 625, 639. Example: code-cell figure without mystnb figure metadata.

  • [qe-math-009] (reviewer) — Choose simplicity in mathematical notation. Count: 4. Lines: 258, 349, 385, 461. Example: the same Q estimate wears three different decorations: \hat Q_t in the informal derivation (349, 350, 357, 358, 365, 368), \widetilde{Q} in the algorithm (385, 399, 406, 407, 421-440, 457, 725, 726) and \tilde{Q} in step 6 of the pseudo-code (461) - the last differing from the second only in macro, so the same symbol is typeset two widths apart on the page. \widetilde{TD} puts an accent over a two-letter operator name (406, 411, 457, 483, 725-726), and \textrm{argmax} / \textrm{argmin} (421, 427, 434, 440) hand-set what the series already provides as \argmax (lectures/_config.yml:110, "argmax" : ["\\operatorname*{argmax}", 0]). The state and action spaces are calligraphic, \(\mathcal{W}\) and \(\mathcal{A}\) (258, 260), where plain \(W\) and \(A\) are unused in the file and would do; the sibling lecture inventory_q writes the same two sets as \(\mathsf X\) and \(\mathsf A\).

  • [qe-writing-001] — Use one sentence per paragraph. Count: 1. Lines: 483. Example: 2 sentences in one paragraph.

  • [qe-writing-003] (reviewer) — Maintain logical flow. Count: 3. Lines: 537, 487, 483. Example: (1) the quit_allowed flag means the opposite of its name, and as a result the lecture’s two headline figures are swapped. In temp_diff, the branch if self.quit_allowed == 0 takes β*np.max(qtable[state_next, :]) (538) - a max over both actions at the same state, i.e. the employed worker may switch to reject, which is quitting - while else takes β*qtable[state_next, 1] (540), locking the worker into the accept column (489: “the second represents accepting the wage”), i.e. quitting forbidden. Match that against the math: {eq}`eq:old4` (405-409) carries the \max_{a'} and is the version the text says allows quitting; {eq}`eq:temp-diff` (723-728) replaces it with \(\widetilde{Q}^{old}(w,\text{accept})\) and is the version that forbids it. So plot_epochs(...) at 696, which sits under ‘## Q-learning’ where quitting is allowed, takes plot_epochs’s default quit_allowed=1 (663) and actually forbids it, while plot_epochs(..., quit_allowed=0) at 742, under ‘## Employed worker can’t quit’, actually allows it. Each figure illustrates the other section. The two defaults disagree with each other as well: the class defaults to quit_allowed=0 (509), which is why the earlier run at 607 is the right way round. (2) 487 tells the reader “we let s represent the state index between \(0\) and \(n=50\)”, but \(n\) is 10 at 129 and 30 at 640, and 50 appears nowhere in the lecture. (3) The Remark at 483 says \(\widetilde{TD}\) is “defined in {eq}`eq:old3` automatically above” - \(\widetilde{TD}\) is defined in {eq}`eq:old4` (409), eq:old3 (400) is the update rule, and “automatically above” is a stray word.

  • [qe-writing-004] — Avoid unnecessary capitalization in narrative text. Count: 1. Lines: 344. Example: mid-sentence ‘Law’.

  • [qe-writing-005] (reviewer) — Use bold for definitions, italic for emphasis. Count: 3. Lines: 62, 245, 345. Example: “on average” (345) is bold used for emphasis, which the rule reverses - italic is what that sentence wants. And the same term is bolded three times as if new each time: Q-function at 49, again at 62 alongside quality function, and quality function again at 245 opening its own section. There is no italic anywhere in the file - a grep for single-asterisk spans returns nothing - so bold is carrying both jobs, definitions and emphasis alike (statistical learning 42, value function 44, Q-table 49, temporal difference learning 64, gain 368, temporal difference errors 411, experimentation 417, deep Q-learning 751 are the definitional uses).

  • [qe-writing-007] (reviewer) — Use visual elements to enhance understanding. Count: 2. Lines: 615, 696. Example: the Q-table is the object the whole lecture is about - it is a \((n+1) \times 2\) matrix, described at 53-54 and 489 - and the only time the reader sees one it is print(qtable) at 615, a wall of floats. A heatmap over (state, action) would show at a glance the thing the lecture claims at 701, that “the Q-learning algorithm has trouble learning the Q-table well for wages that are rarely drawn”. That claim also needs the wage distribution next to the value functions: the distribution is plotted at 648 and the value functions at 696, fifty lines apart and never overlaid, so the reader has to hold one figure in mind while reading the other.

Low severity#

  • [qe-ref-001] — Use correct citation style. Count: 1. Lines: 20. Example: {cite} in narrative flow: ‘{cite}’.

Strengths#

  • The lecture earns its Q-learning derivation instead of asserting it: the exact Q-factor system {eq}`eq:impliedq` (270-275), then the scalar reduction \(Q_r\) (295) exploiting IID wages, then the deliberately “illegitimate” step of erasing the integral (329-339), then the temporal-difference errors (355-360) and the adaptive rule (364-366) - and 329 tells the reader in advance that the step is not legitimate, which is more honest than most treatments.

  • Every experiment is measured against a benchmark computed in the same file: valfunc_VFI from value function iteration (223, 658) is carried through and plotted against the learned value function at 628-629 and again at 670, and compute_error (601-602) puts the mean absolute deviation in the legend of every snapshot (682) - so “the quality of approximation … improves for longer epochs” (703) is read off the figure rather than argued.

  • The Remark at 483 states the exact condition under which the algorithm can be expected to work - \(\widetilde{TD} = 0\) at an optimal Q-table, and convergence depends on visiting all state-action pairs often enough - rather than leaving convergence implicit.

  • The two-regime comparison is a genuinely good design: the same plot_epochs function is run twice, once where an employed worker may quit and once where she may not (696, 742), against the same VFI benchmark - so the cost of losing the option to explore is visible rather than argued (see the judgment findings for the flag that inverts which is which).

  • The state-space experiment at 637-660 re-runs the whole comparison with \(n = 30\) instead of \(n = 10\), which is what makes the “rarely drawn wages” point at 701 observable at all.

  • Randomness is threaded explicitly rather than left to global state: one rng = np.random.default_rng(123) at 82 is passed down through run_epochs (585), run_one_epoch (544), temp_diff (525) and draw_offer_index (517), so every draw in the lecture is reproducible from a single seed - and this is a fix relative to the np.random.seed(123) still carried by the lecture-dp copy of the same lecture.