# Replicate, audit and extend Qamar et al. 2013 ## 1. Context About 16 years ago I did computational neuroscience research with Wei Ji Ma, published as Qamar et al. 2013 (PNAS). Use this session to replicate that work. This is also an assessment of what a modern AI agent can do, so the process and its timing matter as much as the results. ## 2. Materials Everything is in `archive/`: zipped directories from my old computer, the published paper and its supporting information (SI) PDF. Unzip, triage and process them as you see fit. Work backwards from the paper: find the MATLAB code and the data (human and monkey behaviour). Also check whether any monkey neural recordings are in there. If they are, we can take the research beyond replication. ## 3. How to work - Understand this ask fully and thoroughly. Design your own system: planning, review, execution and the research programme. Explore the problem space fully first, then give me your plan. I will review it before you start. - Keep a research log of what you do at a high level, from a research-breakdown perspective: what, why, and which resources (tools, agents, compute) you used. - Measure time yourself. Set up hooks now for the main agent and the subagents, and log both the wall-clock time you operate in and the aggregate subagent time summed up. Scope the hooks to this directory and do not affect other Claude sessions. - Use local git only. - Think hard and do this well. ## 4. Phase 1: replication - Replicate all of the work end to end, including any parameter sweeps, but do it in a much faster and more efficient way than the original. - Do it twice (agent's proposal, approved). First faithfully, quirks included, as a sanity check that you truly reproduce the paper. Then corrected, to see the magnitude of any change. The corrected analysis is the real result. - If the original implementation has errors, your replication must fix them. Tell me about any mistakes you find in my final code that generated the paper's analyses and figures, and report how much each fix changes the results and whether any conclusion changes. - If there is ambiguity or an open question, ask me. - Where the original optimiser let parameters stray outside the ranges stated in the SI, fit within the published ranges, since the paper's evidence calculation assumes them as the prior (agent's proposal, approved). - If the original analysis used only part of a subject's data, have a subagent find out whether that was intentional or an oversight. It should read all of my code that processes that file, then look at the data itself and judge whether omitting it was justified. I don't remember. ## 5. Phase 2: behavioural extensions - Based on newer research or your own ideas, which other Bayesian or near-Bayesian models could fit the human and monkey data better? Implement them, test them and report. - Trial history (agent's proposal, approved): the paper's models treat trials as independent, but choices may carry over from one trial to the next. Test history-effect hypotheses and report them, with plots on the website. - Before any history analysis, verify that the stored trial order really is the presentation order (agent's proposal, approved); I don't remember it myself. ## 6. Phase 3: neural modelling - What newer knowledge or models could we implement and test for a biologically plausible neural account of the optimality and suboptimality of these decisions? For Phases 2 and 3: flesh them out fully, execute them, and show how they integrate with the paper's findings. ## 7. Phase 4: monkey neural data If monkey neural data is in the archive, propose further exploration. If it would have to be requested from someone else, don't request it for now. I might ask for it later, so do nothing that would block on it. ## 8. Research context: a map of the Ma lab's work - Have a subagent mine all of Wei Ji Ma's lab's related work and present it as a map (an SVG figure): how the modelling of each task relates to the others, and what builds on what. - Make it interactive: selecting a paper lights up all its edges; inbound edges (the work it builds on) and outbound edges (the work it influenced) have different colours; every node opens the paper's link. - Give it a legend that explains the edge colours, and make sure you are happy with it. - On the website, frame the map as placing Qamar et al. 2013 in the broader field. ## 9. Phase 5: a new experiment As a last step of the meta-task, take the paper's main ideas, especially the generative model C → s → x, and design a more modern and more complex task: a game, even a 3D one. - Why: a) if the game is catchy or "viral", people will play it for free and be nudged to keep playing, which gives us far more behavioural data; b) from the game data we can work out the Bayes-optimal model and repeat the psychophysics model fitting against Bayesian and non-Bayesian (suboptimal) models. - Using the map from section 8, identify the modelling opportunities and gaps, choose the Bayesian model to tackle, and design the game to match. - This phase is an interactive session with me. It still counts in the meta-task time log, including the time to land on an idea, build the task, host it on Cloudflare with logging of players' data, and add a page that fits a pool of models to the latest data on the fly. The point is to show that with these tools agentic science goes from idea to execution fast, and the main remaining job is marketing the experiments so that people take part. ## 10. Deliverables 1. **A simple research-preview website.** Build it locally for me to review; if I'm happy, we push it to Cloudflare. It should cover: - the meta-task: how long and how well an October-2026 agent does compared with a year of 2010 research, including the old way vs the new way speed comparison (send me the timing improvements as you get them); - replication with corrections, and whether and how the corrections change the paper's key findings, separately for the behavioural and the neural parts; - the extension work; - the literature map from section 8. 2. **A clean, organised code repository** that runs all of the analyses more correctly and faster. Code quality is paramount: deeply modular code with clean abstractions, and code review focused on correctness. ## 11. Compute My Mac is powerful, so use it, and work in parallel where it helps, but always keep 25% of the cores free for my own use. If we need more compute we can use Modal later, but first see how far the Mac can go. ## 12. Checking in - Show me drafts as local files when they are ready (for example the literature map's HTML), so I can review them early. - When I ask for status, tell me what each running subagent is doing, where things stand in the overall plan, and what remains.
The brief · how this project was set up
The brief, as one prompt
The project lead steered this work through a series of chat messages. This page combines their first 15 messages into the one opening brief they could have written up front: every later correction, clarification and added request is placed where it belongs in the brief, in the lead's own level of direction. The text is reworded, not verbatim: the informal chat messages are rewritten as plain prose. A few items were the agent's proposals, which the lead approved in reply; they stay in the brief and are marked "(agent's proposal, approved)".