Replicating Qamar et al. 2013 with an AI research agent

What the paper claimed

Observers sorted a grating's orientation into one of two categories: both centred on horizontal, one narrow ($\sigma_1 = 3^\circ$), one wide ($\sigma_2 = 12^\circ$; $15^\circ$ for Monkey L). Contrast changed at random from trial to trial, which changes how noisy the observer's measurement is. The best decision boundary moves outward as noise grows, so an observer who ignores its own uncertainty is measurably worse off. Five models were compared: the optimal observer (Opt), Opt with a free prior (Opt-P), two heuristics whose boundary is a linear or quadratic function of noise (Lin-σ, Quad-σ), and a fixed boundary (Fixed). Fixed always lost, and the winner differed between subjects. A small network with divisive normalization came close to the optimal posterior; linear and quadratic networks did not.

What we found

• The main conclusions hold. No model winner changes after the corrections. Fixed is never the best model and stays outside the "within $\log 30$" set for every main-experiment subject and both monkeys.
• The archived optimiser had a bug in one gradient term (Lin-σ and Quad-σ). It missed much better optima for both monkeys. Monkey A still prefers Quad-σ and Monkey L still prefers Lin-σ.
• Several claims need qualifying. In the control experiment Fixed is within $\log 30$ for one subject. The boundary ANOVA survives the sphericity correction only at p ≈ 0.016. The independent-σ check is strong across contrasts but inconclusive across six subjects.
• The network story is weaker than published. QDN is still near-optimal (0.22%), but LIN and QUAD were undertrained, and divisive normalization did not "emerge automatically" through learning: it was built into the architecture and its initialization.
• New: half the human subjects show strong trial-history effects, but modelling them changes no model comparison. Noise added to the optimal observer does not explain the data; a Bayesian observer that over-weights its own uncertainty matches the best heuristic in Monkey A; in humans it is clearly better for no one, and where recovery shows it could be identified it is rejected. Divisive normalization helps networks generalise across contrast when it divides by roughly uniform total activity (Phase 3).

Sections