Penn is Head of Product for Anthropic's AI Research and Labs teams and was the company's first technical product manager, hired in 2023 when the product org was five engineers. She has shipped every Claude model generation since Claude 2 and incubated Claude Code, MCP, Skills, computer use and tool use. She is the loudest public advocate anywhere at a frontier lab for the idea that evals, not specs, define a product — which makes her the natural internal sponsor for buying evaluation data.
Background
She traded high-yield bonds at JP Morgan Chase, then built AI systems for Amazon's Alexa, then joined Anthropic in 2023 as its first technical PM. That path — markets, then a consumer voice assistant, then a frontier lab — shows in how she talks: she treats model capability as a distribution to be measured rather than a feature to be specified.
Her central public argument is that the PRD is dead and evals replace it: for probabilistic systems, a runnable definition of success is the specification. She pairs that with two ideas that matter commercially. 'The jagged edge' — capability is uneven, and every domain has rough spots you have to find. And 'product overhang' — capability jumps arrive discontinuously and, without evals acting as radar, you do not notice your own model got better. Both are arguments for buying more and better evaluation data.
What they run now
Product for the research organisation and for Anthropic Labs. Labs is where Claude Design was built, which is the one place a frontier lab has made visual and UI design quality an explicit product target; Anthropic has also published a frontend-design grading rubric scoring design quality, originality, craft and functionality. Penn's own framing — eval-driven development, find your domain's jagged edge, watch for overhang — is the methodology layer sitting on top of exactly that. She is the person who decides whether a capability area gets an eval suite, and therefore whether someone needs to be paid to build one.
What they buy
Inference, clearly flagged: she is a sponsor and specifier, not a procurement signatory. What her remit implies she needs is evaluation suites and rubrics for capability areas Labs is pushing into — including design and frontend craft, given Claude Design and Anthropic's published design grader — plus the RL environments Anthropic is reported to be spending heavily on (reporting in Sep 2025 put internal discussion above $1B, with Mercor and reportedly Mechanize named). Anthropic's own Human Data job postings describe owning 'strategic vendor partnerships' for RLHF, safety, tool use and agentic data; Penn would be the research-side customer of that function rather than its operator.
Career
- 2023 – presentHead of Product, AI Research and Labs, Anthropic — Joined as Anthropic's first technical PM; incubated Claude Code, MCP, Skills, computer use, tool use, reasoning
- —Built AI systems, Amazon (Alexa)
- —High-yield bond trading, JP Morgan Chase
On the record
Evals replace specs · 2026-07
Argues evals define what success looks like once you have built it, replacing prose PRDs for probabilistic products. source
Product overhang · 2026-07
Without evals as radar, capability jumps happen and you do not know they happened. source
The jagged edge · 2026-07
Model capability is uneven; every domain has rough edges teams have to go find deliberately. source
Human judgement · 2026-07
As building gets easier, human judgement becomes the scarce skill rather than a redundant one. source
Interviews & talks
Why it matters here
High, and the highest-value Anthropic name available publicly for this purpose. Anthropic is the only frontier lab that has publicly made design taste a graded target (Claude Design, the frontend-design rubric), and Penn owns product for the org that built it while personally evangelising eval-driven development. She is also the person whose public argument Julian's product answers directly: she says you cannot ship what you cannot measure, and design quality is the thing nobody can measure well. She will not sign a contract, but she can create the internal demand that makes one happen.
How to reach
Do not pitch data. Pitch the gap in her own argument: eval-driven development works where a rubric can be written, and design craft is the case where rubrics collapse into 'looks AI-generated' — Anthropic's own frontend-design grader scores originality and craft, which are precisely the dimensions where automated graders drift from expert judgement. Offer a measurement result rather than a service: e.g. a held-out set of design outputs graded by practising designers versus the automated grader, showing where they disagree. Channels: LinkedIn, or a substantive public reply to her podcast arguments — she engages publicly and the eval-vs-PRD thread is an open conversation. Speculation: routing through Anthropic Labs people (Prithvi Rajasekaran wrote the design rubric) may be faster than going at her directly.
What we could not establish
- No X handle found that I could tie to her.
- No education history found.
- Sources disagree on which Claude model generation she has shipped 'through' — different write-ups name different current model codenames, so I have not asserted one.
- Her LinkedIn headline reads 'Member of Technical Staff'; 'Head of Product, AI Research and Labs' comes from podcast billing and a third-party profile.
- No evidence she personally touches human-data vendor selection — that sits with Anthropic's Human Data / Data Operations function, for which no named individual could be found.
Why they are on the map
Penn is Head of Product for Anthropic's AI Research and Labs teams and was the company's first technical product manager, hired in 2023 when the product org was five engineers. She has shipped every Claude model generation since Claude 2 and incubated Claude Code, MCP, Skills, computer use and tool use. She is the loudest public advocate anywhere at a frontier lab for the idea that evals, not specs, define a product — which makes her the natural internal sponsor for buying evaluation data. source — evidence dated 2026-09
Decides whether the problem is funded.
How Anthropic buys data
Open roles (updated Aug 2026) for 'Data Operations Manager, Human Data' (owns data strategy and 'strategic vendor partnerships', $270k-365k) and 'Product Management, Human Data Platform' (labeling interfaces, vendor tooling, crowdworker sessions, data-quality KPIs). The Information (via TechCrunch, Sep 2025) reported Anthropic leaders discussed spending over $1B on RL environments, working with Mercor and reportedly Mechanize. Surge AI publicly marketed that its human data powers Claude 3 (2024). Anthropic Labs' own design grader (Mar 2026) scores design quality, originality, craft and functionality, which is the closest public sign of design-taste evaluation.
Human Data sits alongside research: Data Operations Managers own data strategy and vendors for research teams (RLHF, safety, tool use, agentic), and a Human Data Platform team builds labeling interfaces, environment simulation and vendor tooling. RL environments have their own infra team ('Environments Infrastructure'). Claude Design is built inside Anthropic Labs (Krieger/Mann), and Jenny Wen leads design for Claude.
Known vendors: surge-ai, mercor, mechanize
Sources
- Lenny's Podcast episode page (role, career, views)
- Dianne Penn, Anthropic: Evals, the Jagged Edge, Overhang
- Unsupervised Learning Ep 77 with Dianne Na Penn
- LinkedIn profile
- Anthropic job posting: Data Operations Manager, Human Data (vendor partnerships, $270k-365k)