Owns the human-data budget and vendor list
The index · 862 people · 68 organisations
Who's who
Everyone on both sides of the market for expert human judgement: the labs and model builders who buy it, the companies that sell it, and the researchers who decide what “good” looks like.
Looking for the seventy creative-lane contacts and the network graph? Who to talk to →
The companies that sell expert data to the labs, and the 93 people who run them — each with a full dossier.
New entrants & independents
People worth tracking who are not at one of the indexed companies — new founders, stealth companies, operators between roles.
Mercor
Marketplace that recruits vetted domain experts to create training and evaluation data (and RL environments) for frontier AI labs.
Mercor co-founder who built its technology (AI interviewer, matching, platform) and, per Forbes, was elevated to co-CEO in 2026 to lead new enterprise offerings. Billionaire on paper (~$2.2B, Forbes July 2026).
23-year-old co-founder and CEO of Mercor, the largest expert-data marketplace by gross volume ($2B run rate in June 2026). He is the category's loudest public voice, framing expert AI training as 'the largest new job category in history' and pushing rubrics and RL environments as the product labs buy.
Mercor co-founder who ran operations as COO through the pivot into expert data, then stepped back to board chairman in October 2025. Still a ~22% owner and board chair, but no longer runs day-to-day operations.
Former Uber Chief Product Officer hired as Mercor's President in May 2025 — the senior operator brought in to scale a company run by 22-year-olds. Forbes says he oversees hiring and client reporting.
Mercor's CFO, announced in June 2026, as the company prepared a ~$20B raise. An eight-year Tesla finance veteran — a signal Mercor is building IPO-grade finance.
Human-data operator who led human data operations at OpenAI, then became a Managing Director at Mercor. His LinkedIn now shows Google DeepMind, i.e. he appears to have moved back to the buyer side.
Surge AI
Bootstrapped, high-quality RLHF, evaluation and RL-environment data from vetted experts for roughly a dozen frontier labs.
Founded Surge AI in 2020 with a few million dollars of his own savings and bootstrapped it past $1B of revenue with around 100-250 staff. He reportedly still owns about 75%. He is the most prominent voice arguing that human taste and judgement, not volume, decide which models win.
Surge AI's first engineer (Aug 2020), now described as CTO. He built the platform behind Surge's quality-controlled human labelling.
Handshake AI
AI-data arm of the Handshake campus career network, selling PhD and professional expert data and evaluations to frontier labs.
Co-founder and CEO of Handshake, who 're-founded' the 12-year-old career network around AI training data. It went from zero to $1B+ gross in about 16 months and he was named to the TIME100 AI 2026. He is the main public voice framing AI tutoring as a new part-time profession.
Handshake co-founder, listed on its leadership page and board. His current operating role is not stated publicly.
Handshake co-founder, still listed on the leadership page. His current role and involvement in Handshake AI are not public.
Garrett Lord's long-time number two, who joined Handshake in 2015 and has held product, finance (CFO until May 2022), legal and marketing roles. Now President & COO across the combined career-network and AI business.
Ex-Scale AI general manager who joined Handshake AI in June 2025 to own strategy, positioning and the innovation roadmap. He led the Cleanlab acquisition narrative and is Handshake AI's clearest strategic voice on expert quality.
Listed on Handshake's leadership page as Chief Commercial Officer of Handshake AI, i.e. the lab-facing sales lead. The Org lists prior roles at Palantir and Amgen.
Handshake AI's Chief Business Officer, running business development and operations. He spent four years as Head of Product Deployment and Operations at Scale AI — delivery at the largest incumbent.
Listed as Head of AI Research at Handshake AI. He leads a research team of about 14 (directors, fellows, data scientists), which presumably now includes the Cleanlab team acqui-hired in January 2026.
micro1
Vetted expert network (plus datasets, RL evals and robotics data) supplying training/evaluation data to AI labs and enterprises.
25-year-old solo founder-CEO of micro1, the fastest-growing small-cap in expert data ($500M gross run rate, Aug 2026). He is the sector's most outspoken 'America-first' voice (no sales to Chinese labs) and is pushing beyond hours into resellable datasets and robotics data.
micro1's CFO, also listed as VP Government — relevant as micro1 raises a new round and pitches itself as a China-free American data supplier.
Listed as micro1's CRO since October 2022 — the commercial lead through its growth from ~$7M to a $500M gross run rate. Previously at Turing, a direct competitor.
micro1's senior AI/research leader: a Stanford PhD (Andrew Ng, Dan Jurafsky) and serial founder who gives the company research credibility with labs. Inc. describes him as a former Stanford CS professor and ex-Apple.
Turing
Former remote-developer staffing marketplace now selling coding/STEM expert data, RL environments and evals to frontier labs and enterprises.
Founder-CEO of Turing. He took a remote-developer staffing marketplace and repositioned it as a 'research accelerator' for frontier labs. He is the loudest voice arguing that commodity data labelling is dead and that expert, workflow-grounded data and RL environments are what labs now buy.
Turing's technical co-founder. He is an ML researcher who built the vetting and matching system that turned Turing's developer pool into a data supply chain. With Ece Kamar named CTO in Sep 2026, his current title is unclear.
Turing's new President and Global CRO (Jul 2026). He runs all go-to-market with labs and enterprises. He is a veteran of the first generation of this market: he ran Figure Eight (ex-CrowdFlower), the crowd-labelling platform that Appen acquired in 2019.
Turing's CTO since 1 Sep 2026. She spent 16 years at Microsoft Research and ended as Corporate VP leading the AI Frontiers Lab (AutoGen, Magentic-One agent work). She is the most research-credentialed hire in the expert-data vendors this year.
Invisible Technologies
AI training data (RL Data Lab, expert marketplace) for foundation-model builders plus enterprise AI workflow/agent platform Meridial.
Founded Invisible in 2015 as a human-plus-software 'digital assistant' operation and took it to unicorn status. He stepped down as CEO in a restructuring and is now executive chairman.
Former McKinsey senior partner who ran QuantumBlack Labs (about 1,000 engineers). He became Invisible's CEO in January 2025, has since raised $100M+ at a $2B+ valuation, bought WeCP and repositioned Invisible as an enterprise AI platform on top of its training-data business.
Invisible's Chief Go-To-Market Officer. His leadership-page bio says he supports enterprises training the world's largest LLMs.
Invisible's COO per its leadership page, overseeing operations of a business built on 3,000+ operators in 35+ countries.
Co-leads Invisible's RL Data Lab, which builds the data and evaluation infrastructure model builders use to train and benchmark frontier models.
Co-leads Invisible's RL Data Lab, designing RL training and eval data built around real deployment challenges.
AfterQuery
Expert-written post-training data, long-horizon tasks and RL environments for frontier labs, validated by in-house training runs.
23-year-old co-founder and CEO of AfterQuery, which went from YC W25 to a reported $3.2B valuation in about 18 months. He is the public face of the company and has positioned it as a research-led data vendor that proves data quality to labs with its own training runs and published benchmarks.
22-year-old co-founder and CTO of AfterQuery. Owns the software that validates and filters expert data and the internal training pipeline the company uses to prove quality to labs.
Third, lower-profile co-founder of AfterQuery, named by Mateega as one of the three who left post-grad jobs to start the company. Press coverage (Forbes, TechCrunch) mentions only two founders, so his current role is unclear.
Snorkel AI
Stanford-born 'frontier AI data lab' selling expert-built datasets, rubrics, benchmarks and RL environments to AI labs and enterprises.
Co-founder and CEO of Snorkel AI, which he spun out of the Stanford AI Lab's Snorkel weak-supervision project. He has steered the company from enterprise labeling software to expert data, rubrics and evaluation environments sold to frontier labs, and is one of the most articulate public voices arguing the frontier is now 'expert agentic data'.
Stanford CS professor and MacArthur Fellow whose Hazy Research lab produced Snorkel; academic co-founder of Snorkel AI and serial founder (SambaNova, Together AI, Lattice and Inductiv, both acquired by Apple). Influential as the intellectual root of the 'data-centric AI' school that many data vendors now sell.
Snorkel co-founder focused on technical strategy and engineering (The Org lists him as Head of Engineering). Built the original open-source Snorkel library at Stanford.
Snorkel AI co-founder who leads research; her 2025-26 output is squarely on how expert data and rubrics are evaluated (rubric failure modes, automated benchmark design, RLVR in low-data regimes). She is the research owner behind Snorkel's claim that rubric quality is make-or-break.
Founding team member (described by Fortune as a co-founder) who leads Snorkel's evaluation and expert-feedback systems and directs the $3M Open Benchmarks Grants. He is Snorkel's public face on benchmark integrity, quoted in Fortune (Sep 2026) on OpenAI changing eval metrics around a launch.
Chief Scientist at Snorkel AI and assistant professor of CS at UW-Madison; drives Snorkel's benchmark and evaluation research (OSWorld 2.0, Senior SWE-bench, SlopCode Bench, RIFT) and sits on the Open Benchmarks Grants steering committee.
Labelbox
Sells RL environments, expert data (via its Alignerr network) and an enterprise agent platform to frontier AI labs and enterprises.
Co-founder and CEO of Upcraft (Chicago, AI sales agents) which Labelbox acquired on Feb 10, 2026 to automate recruiting, qualification and engagement of Alignerr experts; now listed as Labelbox VP of Growth. Effectively owns the machinery that grows Labelbox's expert supply.
Co-founder and CEO of Labelbox, which he has moved from labeling software to a 'data factory' for frontier labs built on the Alignerr expert network, and in 2026 to RL environments (Horizon) and an enterprise agent platform (Recursion). He controls one of the largest expert networks in the market and is an active acquirer (Upcraft, Feb 2026).
Labelbox co-founder, variously listed as President (Crunchbase), COO (First Round Review) and Chief Product Officer (Craft). An aerospace engineer who met Manu Sharma at Embry-Riddle.
Toloka
Amsterdam-based AI data company selling expert-built training, evaluation and agent data to labs, with Mindrift as its expert network.
Founded Toloka inside Yandex in 2014 and has run it as CEO since 2020, through its carve-out via Nebius and the 2025 Bezos-led round that gave it independent voting control. She also leads Mindrift, Toloka's expert-sourcing platform, so she directly owns both the lab-facing business and the expert supply side.
Former Yandex CTO and ex-CEO of Microsoft's Advertising and Web Services (Bing), who became Shopify CTO in 2024 and joined Toloka as executive chairman when he co-invested in the May 2025 Bezos-led round. Gives Toloka senior big-tech credibility with lab and enterprise buyers.
CTO of Toloka per The Org (unverified profile), leading engineering for the data platform. Ex-Yandex.
Chief Sales Officer at Toloka per The Org (unverified), leading US sales; previously at Turing, a direct competitor in expert data for labs.
Appen
ASX-listed, 30-year-old crowd data vendor (1M+ contributors) selling annotation, LLM expert data and evals to hyperscalers and model builders.
Appen's CEO since Feb 2024. He took over days after Google's contract termination and has run the turnaround: cost cuts, a pivot to generative-AI and LLM expert data, and strong growth in Appen China. He is the operator of the sector's only audited, public pure-play.
Appen's CFO (interim from Aug 2023, permanent from Feb 2024). He has been at Appen since 2016 and is the keeper of the only audited P&L among the pure-play human-data vendors.
Appen's product and technology chief. He owns CrowdGen (the crowd's work interface), Mercury (the project and crowd backend) and the expert-sourcing platform Elite AI.
Appen Global's sales lead since Dec 2024, recruited from Scale AI and Snorkel. He is the GTM face to US labs and hyperscalers outside China.
Head of Appen's crowd (1M+ contributors, 200+ countries) since Aug 2025, which makes him the supply-side owner. He took over as crowd satisfaction fell (Crowd NPS dropped from 33 to 22 in FY2025).
Runs Appen China, now nearly half of Appen's revenue (US$102.9M in FY2025, +75%) and its profit engine. Appen describes it as the largest AI data company in China.
Deccan AI
India-operated post-training data, RL environments (STARK) and evals (Helix) for frontier labs and enterprises.
Founder of Deccan AI, an India-operated expert-data vendor that raised a $25M Series A in March 2026. A former banker/consultant/strategist who pitches 'accuracy' over superintelligence and runs a very large Indian expert pool at lower cost than US rivals.
Deccan AI's revenue/sales leader, listed on the company leadership page with 20+ years of B2B sales at AWS and Intuit and a Kellogg MBA. Owns the lab and enterprise pipeline.
Heads strategy at Deccan AI per its leadership page; 15+ years across Google, McKinsey and Amex, IIT Madras and IIM Ahmedabad.
Contra Labs (Contra.Work Inc.)
Creative human-data unit of the Contra freelance marketplace: designer preference data, trajectories, benchmarks and a creative arena for AI labs and tools.
Co-founder and CEO of Contra, the commission-free freelance marketplace (about $45M raised; Unusual, NEA, Cowboy). In March 2026 he repointed Contra's creative supply into Contra Labs, the most direct incumbent in creative human data. A creative himself (audio engineer, music composer), he pitches Contra Labs as 'the eval layer for creative AI'.
Co-founder and CTO of Contra; the engineering lead behind the marketplace that supplies Contra Labs. A well-known open-source JavaScript/PostgreSQL engineer (Slonik and other libraries) who blogs on engineering and, since 2026, on AI-assisted development and fine-tuning.
First author of Contra Labs' flagship paper, The Human Creativity Benchmark (Jun 2026), listed with a dual Contra (New York) and MIT affiliation. She is the most credentialed ML researcher publicly attached to Contra Labs, and she may be the unnamed 'Head of Research', which is unconfirmed.
Contra Labs researcher named on both of its public papers: the Human Creativity Benchmark and the TASTE designer-preference dataset with Lica World. She is one of the few people who has run professional-designer annotation studies at commercial scale and published the method.
Taste Labs
Sells design preference data, rubrics and RL/eval environments to frontier labs, plus brand-verification software (Brand API) to app companies.
Founder and CEO of Taste Labs, the best-funded pure-play 'taste' data company ($18.5M seed from CRV and Amplify, Jun 2026). She came from growth, strategy and marketing at Exa, a search-API startup selling to AI labs, so she has sold to labs before. She is the category's main public voice, arguing that AI slop is caused by post-training on averaged preferences.
Research-side Member of Technical Staff at Taste Labs. She wrote the company's public research agenda and runs its Prototype fellowship for outside researchers. Before Taste Labs she worked on the economics and provenance of AI training data, a rare background in this market.
Gray Swan AI
AI security firm: a 15,000-person prize-paid red-teaming crowd (Arena) feeding automated red-teaming (Shade) and runtime guardrails (Cygnal) sold to labs and enterprises.
CMU security and privacy professor who runs Gray Swan, the company that turned a crowd of prize-paid jailbreakers into a frontier-lab red-teaming and guardrail business. He is the operator of one of the few human-data crowds paid by competition rather than by the hour.
AI-safety researcher behind the universal adversarial-suffix jailbreak, representation engineering, circuit breakers and HarmBench, and co-founder of Gray Swan. His research is the technical core of the company's attack and defence products.
Head of CMU's Machine Learning Department, OpenAI board member chairing its Safety and Security Committee, and Gray Swan's chief scientist. Few people sit closer to how frontier labs think about adversarial testing.
Enterprise-security go-to-market executive hired in January 2026 to lead Gray Swan's global market expansion. He is the commercial counterpart to the academic founders.
Listed by Gray Swan as Chief Product Officer, owning the Shade/Cygnal/Arena product line. Little public detail verified.
Centaur Labs (trading as Centaur AI)
Expert medical/scientific data annotation and model evaluation via a gamified expert crowd (DiagnosUs), sold to health and AI developers.
MIT-trained collective-intelligence researcher who turned a thesis finding (crowds of medical students can out-classify individual dermatologists) into Centaur, the best-known gamified expert-annotation company in medicine. His core idea is performance-weighted expert crowds rather than credential-weighted ones.
Centaur co-founder and VP of Engineering with prior experience running mapping and data-labeling teams at Cruise, i.e. hands-on operational knowledge of large labeling programs.
Self-taught engineer and Centaur co-founder, listed as CTO; built the platform behind DiagnosUs and the expert-aggregation pipeline.
Mechanize
Builds RL environments and evaluations (software-engineering tasks with automated graders) for frontier labs' coding agents.
Co-founder of Epoch AI who left to found Mechanize, a small RL-environment shop reportedly in $1.5B+ licence-and-hire talks with Google. He is the most prominent advocate of the view that frontier training data must be expensive, expert-built and sold as artefacts, not hours.
Former Epoch AI researcher and Mechanize co-founder who speaks publicly on evals and training-data quality. Known for long-timeline, economics-first views of AI progress.
Former Epoch AI researcher and Mechanize co-founder, co-author of its essays on RL-task economics and the automation of labour. A prominent, contrarian voice on AI economics and AI-risk debates.
Sepal AI
Former YC S24 vendor of expert-built benchmarks, evals, training data and RL environments for labs; acquired by Mercor.
Co-founder and CEO of Sepal AI, sold to Mercor in February 2026. Before Sepal he helped scale Turing's LLM-trainer business, so he has run expert supply at two of the market's vendors.
Technical co-founder of Sepal AI; after the Mercor acquisition his site says he is working on RL environments and links to Mercor.
Sepal AI co-founder who ran go-to-market and operations; earlier built Turing's foundational LLM-trainer business and managed operations for 500+ AI trainers. Her LinkedIn headline now shows Mercor.
Datacurve
Expert coding data, RL environments and evals for frontier labs, sourced via the gamified Shipd bounty platform for engineers.
Co-founder and CEO of Datacurve, a YC W24 coding-data specialist that treats expert supply as a consumer product (Shipd) rather than a labelling operation. One of the youngest founders in the market; Forbes 30 Under 30 per Waterloo.
Technical co-founder of Datacurve and co-author of its DeepSWE benchmark. Former Google intern and Waterloo CS student focused on multimodal RL and browser-automation agents.
Halluminate
RL environments and expert-built data training frontier models on investment banking, private equity and consulting work.
Ex-Capital One Labs product/research lead who co-founded Halluminate (YC S25), first a computer-use agent benchmark and simulation company, now selling finance RL environments built by ex-bankers and deal professionals to frontier model developers.
Halluminate co-founder/CTO, a Cornell CS graduate and Milstein scholar with data-engineering experience at two NYC startups; co-author of the company's 2026 finance due-diligence benchmark.
Irregular (formerly Pattern Labs)
Frontier AI security lab running pre-release offensive-cyber evaluations (SOLVE, CyScenarioBench, FrontierCyber) for OpenAI, Anthropic, Google DeepMind and Meta.
CEO of Irregular, the Tel Aviv firm that has become the default third-party cyber evaluator named in frontier-lab system cards. He sells expert-built evaluation scenarios, not labour hours, to the most competitive buyers in the market.
CTO of Irregular, responsible for the simulated attack-and-defence environments that frontier labs use for pre-release cyber evaluations.
Vals AI
Independent, private-test-set benchmarks of AI models on real professional work (law, finance, medicine, coding), sold to labs, vendors and enterprises.
25-year-old Stanford CS graduate who co-founded Vals, now the most-cited independent benchmark shop for professional-domain AI (law, finance, medicine), with results in lab model cards. He is the face of the 'independent scorekeeper' thesis that won a16z's $40M Series A in August 2026.
Stanford CS graduate and Vals co-founder/CTO who builds the evaluation infrastructure behind Vals' private benchmarks; background in quant (Hudson River Trading) and big-tech engineering internships.
Design Arena (Intelligence, formerly Arcada Labs)
Free crowdsourced arena where millions vote on AI design outputs; sells the preference data and custom evals to frontier labs.
Co-founder and CEO of Intelligence, the company behind Design Arena, the largest crowdsourced preference arena for AI-generated design. She claims $60M ARR selling arena data and evaluations to frontier labs, which makes her the commercial proof point, and the main counter-thesis, for Julian's lane: consumer-crowd taste instead of expert panels.
Co-founder and CTO of Intelligence / Design Arena. She owns the arena stack: prompt routing to many models, vote collection and the leaderboards across web, image, video, audio, slides and agent categories. That stack produces the data sold to frontier labs.
Arena (formerly LMArena / Chatbot Arena; Arena Intelligence Inc.)
Crowdsourced model leaderboard whose free votes power paid evaluations and preference data sold to labs and enterprises.
Statistician-turned-CEO who runs Arena, the crowdsourced leaderboard that turned into a ~$100M run-rate evaluation business in under a year. He is the person who sets the terms on which free crowd preference competes with paid expert judgement, the central question for any specialist human-data vendor.
Berkeley professor and serial founder (Databricks, Anyscale) who advised the Chatbot Arena students, co-founded the company and chairs it. He gives Arena its enterprise credibility and company-building playbook.
Systems researcher who started Chatbot Arena as a Berkeley PhD 'weekend project' and now runs Arena's technology. He built the infrastructure (FastChat, Vicuna, MT-Bench, Arena-Hard) that made crowdsourced pairwise preference the default public yardstick for models.
Prolific
Self-serve marketplace of ID-verified research participants and AI taskers, sold to academics and AI teams for evals, RLHF and red-teaming.
Psychological scientist who co-founded Prolific after struggling to recruit participants for her own PhD studies. She was the company's early CEO. Current public materials name only Phelim Bradley as CEO/co-founder, and her present role is unclear.
Co-founder and CEO of Prolific. He took an Oxford academic-participant marketplace and turned it into a human-data platform for AI evals, RLHF and red-teaming. He is a public advocate that human data remains 'where the alpha is', and the face of HUMAINE, Prolific's human-preference leaderboard.
Prolific's VP of Data & AI (earlier listed as VP of Data, Research & Analytics). He leads the data science behind participant quality, AI-agent detection and evaluation products such as HUMAINE (attribution to HUMAINE is inferred, not confirmed).
Prolific's VP Product, owning the product that packages 'verified, diverse humans behind an API' for AI teams. She is one of the few voices in this market arguing publicly that models need cultural sensitivity and human judgement, not just more data.
Needs the data for a model; writes the spec
Runs vendor projects; decides if a pilot converts
Contracts, licensing, provenance
Decides whether the problem is funded
A talk to first · B map and watch · C context. Tier C is hidden by default.
OpenAI
Frontier lab behind ChatGPT, GPT-5.x and the GPT Image family (Images 2.0 Apr 2026, 2.5 Sep 2026 with templates/posters/sketch). The biggest single buyer of expert human data, now with image/design use cases shipping in ChatGPT.
Product lead for ChatGPT Images — credited as Product Lead on GPT-Image-1.5 (December 2025) and the public face of the ChatGPT Images 2.0 upgrade shipped in September 2026. She decides what the image surface is for, which in practice means she defines what a 'good' output is at the level users experience it: infographics, slides, maps, manga, multilingual text.
Barr is a programme manager on OpenAI's Human Data team and one of the few people there who talks about the team in public — she recruits for it openly on X and LinkedIn. She shows up in the credits of the products where human data was decisive: a foundational contributor on deep research, a GDPval co-author, and in the Data & Evaluation group for ChatGPT Images. She is a realistic first point of contact into OpenAI's human-data function.
Named Research Lead on ChatGPT Images (GPT-Image-1.5, December 2025) and a long-standing contributor to OpenAI's image line including DALL·E 3 and the multimodal-neurons interpretability work on CLIP. He is the person who decides what the image model is optimised toward, which makes him the technical counterpart to any aesthetic preference dataset.
Credited 'Data Leads' for GPT-4o image generation (Mar 2025); GDPval co-author (Oct 2025)
GPT-4o system card 'Data acquisition leads' (2024); listed in Data & Evaluation for ChatGPT Images (Dec 2025)
Credited 'Data Leads' for GPT-4o image generation (Mar 2025) and in Data & Evaluation for ChatGPT Images (Dec 2025)
Amelia 'Mia' Glaese runs research and safety at OpenAI after the July 2026 reorganisation folded the safety organisation back under research and created her role; safety teams now report to her. Before OpenAI she led DeepMind's Sparrow, the paper that introduced decomposing 'good behaviour' into natural-language rules and asking human raters about each rule separately. That methodology is the intellectual ancestor of every rubric-based human-judgement contract a lab signs, which makes her unusually easy to talk to about taste rubrics.
Thacker runs OpenAI's human-data research programmes globally and is OpenAI's London site lead. She is the most senior operator publicly identifiable on the human-data side at OpenAI: the person whose organisation designs the campaigns, recruits and manages the expert annotator pools, and contracts the vendors that produce OpenAI's training and evaluation data. She was a co-author on GDPval, OpenAI's expert-graded real-work benchmark.
OpenAI's multimodal lead and a Technical Fellow — the research executive over native image, audio and video generation. He is credited as Multimodal Lead on ChatGPT Images (GPT-Image-1.5, December 2025) and founded the 'omni' team that produced GPT-4o. Everything OpenAI ships that has to look good passes through his organisation, which makes him the highest-leverage research-side sponsor for visual-taste data at the company.
Yu is a programme manager on OpenAI's Human Data team who says on her own site that she builds the data-collection pipelines for OpenAI's models and has led data initiatives across multimodal and reasoning models, image generation included. She is credited in the 'Data' group for GPT-4o image generation and in 'Data & Evaluation' for ChatGPT Images. Of the OpenAI human-data staff who can be named publicly, she has the clearest and longest-running attachment to image data.
First author of GDPval, OpenAI's benchmark of model performance on real-world economically valuable work: 44 occupations across the nine largest sectors of US GDP, with tasks built from the actual deliverables of professionals averaging fourteen years of experience and graded against expert standards. She is the person at OpenAI who has industrialised 'pay experienced practitioners to produce and grade real work', which is structurally the same purchase as creative-taste data.
Creator of DALL·E and DALL·E 2, later a Sora lead, and now OpenAI's World Simulation Lead — credited in that title on both Sora 2 (September 2025) and ChatGPT Images (December 2025). He owns the video and world-model line at OpenAI, which was reshaped when the standalone Sora app was discontinued in March 2026.
LinkedIn profile: "Human Data Red Team - Research" at OpenAI, Berlin, Berlin, Germany, in role since 2025-09.
LinkedIn profile: "Data Scientist, Human Data" at OpenAI, San Francisco, California, United States, in role since 2023-08.
Listed under 'Data & Evaluation' in ChatGPT Images (gpt-image-1.5) credits
Listed under 'Data & Evaluation' in ChatGPT Images (gpt-image-1.5) credits
LinkedIn profile: "Data Scientist, Human Data" at OpenAI, San Francisco, California, United States, in role since 2020-01.
LinkedIn profile: "IMAGE AND VIDEO ANNOTATION EXPERT" at OpenAI, Kenya, in role since 2025-11.
LinkedIn profile: "Robotics Data Annotation & QA via Apex Systems" at OpenAI, San Francisco Bay Area, in role since 2026-07.
LinkedIn profile: "Human Data" at OpenAI, San Francisco, California, United States, in role since 2026-01.
Listed under 'Data & Evaluation' in ChatGPT Images (gpt-image-1.5) credits
LinkedIn profile: "Economist | AI Ethics & Training Data Architect | Policy Specialist" at OpenAI, Pakistan, in role since 2025-02.
LinkedIn profile: "Data Scientist, Human Data" at OpenAI, San Francisco Bay Area, in role since 2019-11.
Listed under 'Data & Evaluation' in ChatGPT Images (gpt-image-1.5) credits
Equal-contribution author on GDPval expert-graded eval (Oct 2025)
Listed under 'Data & Evaluation' in ChatGPT Images (gpt-image-1.5) credits
LinkedIn profile: "Data Scientist, Human Data" at OpenAI, United States, in role since 2018-01.
LinkedIn profile: "AI Robotics Operator (Data Acquisition & Quality)" at OpenAI, Los Angeles, California, United States, in role since 2026-06.
LinkedIn profile: "Ai data annotation specialist" at OpenAI, Kenya, in role since 2025-10.
Listed under 'Data & Evaluation' in ChatGPT Images (gpt-image-1.5) credits
Listed under 'Data & Evaluation' in ChatGPT Images (gpt-image-1.5) credits
LinkedIn profile: "Data Operations Lead" at OpenAI, Greater Cleveland, in role since 2024-06.
Listed under 'Data & Evaluation' in ChatGPT Images (gpt-image-1.5) credits
LinkedIn profile: "Data operations" at OpenAI, San Francisco, California, United States, in role since 2026-06.
LinkedIn profile: "ML Data Operations - Robotics" at OpenAI, San Francisco Bay Area, in role since 2026-04.
Credited 'Model Behavior' for 4o image generation (Mar 2025) - taste/behaviour spec for image outputs
LinkedIn profile: "Data Scientist, Human Data" at OpenAI, San Francisco Bay Area, in role since 2021-07.
'Human Data Advisors' on 4o image gen (Mar 2025); InstructGPT/RLHF first author; GPT-4 IF data collection lead
Chief Research Officer, quoted on safety/research restructuring (Jul 2026); in ChatGPT Images leadership credits (Dec 2025)
LinkedIn profile: "Member of Data Program Staff" at OpenAI, San Francisco, California, United States, in role since 2026-07.
Co-author on both HealthBench (262 physician graders, May 2025) and GDPval (Oct 2025) - likely expert-data ops; exact role unverified
LinkedIn profile: "Manager, AI Task Design & Evaluation" at OpenAI, Miami, Florida, United States, in role since 2026-05.
LinkedIn profile: "Human Data" at OpenAI, San Antonio, Texas Metropolitan Area, in role since 2026-03.
LinkedIn profile: "Human Data, Safety" at OpenAI, San Francisco, California, United States, in role since 2026-09.
LinkedIn profile: "Human Data, Research" at OpenAI, United States, in role since 2025-08.
LinkedIn profile: "OpenAI Data Annotation Trainee" at OpenAI, Ife, Osun State, Nigeria, in role since 2025-07.
LinkedIn profile: "Data Scientist, Human Data" at OpenAI, San Francisco, California, United States, in role since 2020-01.
Listed under 'Data & Evaluation' in ChatGPT Images (gpt-image-1.5) credits
Bowman is a programme manager on OpenAI's Human Data team whose visible work is expert-graded evaluation. He is the fourth-listed author on HealthBench, the benchmark built from conversation-specific rubrics written by 262 physicians, and a core Research contributor on GPT-4.5. He is the clearest public example of an OpenAI human-data PM who runs a large, credentialled expert panel end to end.
LinkedIn profile: "Member of Data Program Staff" at OpenAI, San Francisco, California, United States, in role since 2026-09.
LinkedIn profile: "Data Operations Specialist" at OpenAI via Apex System, San Jose, California, United States, in role since 2026-07.
Equal-contribution author on GDPval expert-graded eval (Oct 2025)
LinkedIn profile: "Research Engineer, Pre-training Data Processing " at OpenAI, Alwar, Rajasthan, India, in role since 2021-05.
LinkedIn profile: "AI DATA ANNOTATION SPECIALIST" at OpenAI, Nairobi, Nairobi County, Kenya, in role since 2023-07.
LinkedIn profile: "Data Scientist, Human Data" at OpenAI, United States, in role since 2019-01.
Safety Lead for 4o image gen (Mar 2025); listed in Data & Evaluation for ChatGPT Images (Dec 2025)
LinkedIn profile: "Annotation Specialist" at OpenAI, Mesa, Arizona, United States.
LinkedIn profile: "AI Model Evaluation & Data Annotation" at OpenAI, Nairobi, Nairobi County, Kenya, in role since 2025-12.
Core research on 4o image gen (Mar 2025); Listed under 'Data & Evaluation' in ChatGPT Images (gpt-image-1.5) credits (Dec 2025)
Listed under 'Data & Evaluation' in ChatGPT Images (gpt-image-1.5) credits
Listed under 'Data & Evaluation' in ChatGPT Images (gpt-image-1.5) credits
Meta (Meta Superintelligence Labs)
MSL builds the Muse model family (Muse Spark LLM, Apr 2026; Muse Image and Muse Video, Jul 2026) for Meta AI, Vibes and ads. It is a major buyer of image/video aesthetic and preference data.
LinkedIn profile: "Program Manager, Product Data Operations" at Meta, Austin, Texas, United States, in role since 2026-07.
LinkedIn profile: "Team Manager, AI Product Data Operations - Facebook & Instagram" at Meta, Hastings-On-Hudson, New York, United States, in role since 2025-09.
LinkedIn profile: "Program Manager, Product Data Operations" at Meta, Greater Richmond Region, in role since 2022-05.
One of the most accomplished image-generation researchers working anywhere: MaskGIT, Muse, Imagic, Palette, StyleDrop and MAGVIT at Google, then GPT-4o image generation at OpenAI, now a research leader for multimodal work at Meta's TBD Lab. With Jiahui Yu's departure in August 2026 she is plausibly the most senior image person left at Meta Superintelligence Labs.
LinkedIn profile: "Senior Program Manager – Monetization GenAI, Product Data Operations" at Meta, Greater Seattle Area, in role since 2026-02.
LinkedIn profile: "Program Manager, Product Data Operations" at Meta, New York City Metropolitan Area, in role since 2025-11.
LinkedIn profile: "Senior Project Manager - Product Data Operations (GenAI)" at Meta, Brooklyn, New York, United States, in role since 2023-12.
LinkedIn profile: "Program Manager, Product Data Operations" at Meta, Sunnyvale, California, United States, in role since 2025-04.
LinkedIn profile: "Program Manager, Product Data Operations, Reality Labs " at Meta, San Francisco, California, United States, in role since 2025-01.
Chief Scientist of Meta Superintelligence Labs, announced by Mark Zuckerberg in July 2025. A co-creator of ChatGPT and co-author of GPT-4 at OpenAI, where he also led synthetic data work. He sets MSL's research agenda, which determines what data the lab buys, but his own centre of gravity is reasoning and text rather than visual taste.
LinkedIn profile: "Risk & Fraud Operations Analyst (Data Labelling)" at Meta, Dublin, County Dublin, Ireland, in role since 2025-05.
LinkedIn profile: "AI Training, AI Response Evaluation, Multimodal Data Labeling and Policy Adherence" at Meta, San Francisco Bay Area, in role since 2026-03.
LinkedIn profile: "Staff UX Researcher — AI Evaluation & Research Infrastructure" at Meta, Palo Alto, California, United States, in role since 2026-01.
LinkedIn profile: "Principal PM & Area Lead — AI Platform & Evals" at Meta, New York, New York, United States, in role since 2026-03.
LinkedIn profile: "Sr. Staff Research Scientist - AI evals" at Meta, United States, in role since 2026-06.
LinkedIn profile: "AI Data Operations Lead via TekSystems" at Meta, Edmond, Oklahoma, United States, in role since 2026-05.
Hired as GPT-4o co-creator with post-training leadership. Evidence is from 2025.
Named in SemiAnalysis Jul 2026 as an ex-OpenAI MSL hire.
Ex-Gemini pretraining/reasoning; listed among MSL researchers in Jul 2026.
Named in SemiAnalysis Jul 2026 as an ex-OpenAI MSL hire (RL/reasoning).
Led multimodal at Meta's TBD Lab and shipped Muse Spark, Muse Image, Muse Video and Voice Mode — then announced on 14 August 2026 that he was leaving Meta to start a new company. Previously OpenAI's perception lead and, before that, a Google Brain researcher on PaLI and Parti. He is no longer a Meta buyer; he is a future one, once his company is named.
LinkedIn profile: "Director, Muse Product Post-Training" at Meta, South San Francisco, California, United States, in role since 2026-03.
LinkedIn profile: "Sr.Engineering Manager, Data & AI Evals" at Meta, New York City Metropolitan Area, in role since 2021-09.
LinkedIn profile: "AI Evals lead - Meta Superintelligence Labs" at Meta, New York, New York, United States, in role since 2025-10.
Former GitHub CEO, now head of Products and Applied Research at Meta Superintelligence Labs since June 2025, reporting to Chief AI Officer Alexandr Wang. He owns the consumer-integration side — the surfaces where Meta's Muse image and video models actually reach people — which makes him the nearest thing Meta has to an executive sponsor for output quality that users judge by eye.
LinkedIn profile: "Senior Product Operations Manager: AI Evals" at Meta, Los Angeles, California, United States, in role since 2026-04.
LinkedIn profile: "GenAI Physics SME Lead - Product Data Operations (contracted via Tundra Technical Solutions)" at Meta, Port Townsend, Washington, United States, in role since 2026-07.
LinkedIn profile: "AI Product Data Operations" at Meta, San Francisco Bay Area, in role since 2024-07.
Hired for Gemini post-training, coding, reasoning and perception. Evidence is from 2025.
LinkedIn profile: "Prompt Engineer & Data annotation" at Meta, Cincinnati Metropolitan Area, in role since 2025-03.
Hired as GPT-4o voice co-creator and multimodal post-training lead. Evidence is from 2025.
LinkedIn profile: "ML Product Labeling Data Operations" at Meta, Greater Chicago Area, in role since 2021-11.
LinkedIn profile: "Product Data Operations" at Meta, Seattle, Washington, United States, in role since 2026-02.
LinkedIn profile: "Product Data Quality Operations Lead - AI Data Quality, Trust & Safety, AR/VR, LLMs" at Meta, Dublin, County Dublin, Ireland, in role since 2025-11.
RL on chain of thought; listed among MSL researchers in Jul 2026.
LinkedIn profile: "AI Data Program Management, Product Data Operations" at Meta, Greater Seattle Area, in role since 2025-09.
LinkedIn profile: "Model Quality / Data Annotation Analyst (Human In The Loop / RLHF)" at Meta (Tundra), Carlsbad, California, United States, in role since 2026-03.
LinkedIn profile: "Product Manager, Model Evals" at Meta, San Francisco, California, United States, in role since 2026-02.
LinkedIn profile: "Program Manager, AI/ML Data & Model Evaluation" at Meta, New York, New York, United States, in role since 2022-04.
Named in SemiAnalysis Jul 2026 as an ex-OpenAI MSL hire; prior RLHF/alignment publications.
Google DeepMind
Builds Gemini plus the Imagen, Nano Banana, Veo, Genie and Lyria generative-media models; runs very large human side-by-side evals for subjective visual quality, which makes it a top buyer of image/video taste judgement.
Leads Google DeepMind's video model work in the Veo and Omni lineage. On an August 2026 panel he stated the position that matters most to a data vendor: high-quality data is the constraint, not volume, and he specifically prefers professionally shot content over random web video. He also called video evaluation the hardest open problem in his work.
VP of Product Management at Google DeepMind, the product executive above the generative-media line that includes Veo, Imagen, Lyria and the Gemini image models. He is the altitude at which a multi-year creative-data programme would be approved, and the person who fronts DeepMind's creative-industry positioning at I/O and in partnerships.
Kirkwood leads 'human data engagements' at Google DeepMind out of the UK - on the face of the title, the person who sets up and runs GDM's engagements with the external rater pools and data suppliers that feed Gemini and the generative-media models. She has essentially no public footprint beyond her LinkedIn headline, so everything below her title is inference from how GDM describes its data-enrichment work.
Engineering lead on Nano Banana, and the person who has stated the measurement problem most plainly: faces and aesthetic quality are very hard to quantify, so benchmarks are inadequate and the team falls back on attention to detail and human judgement. She is the technical counterpart to Nicole Brichtova and the person who would have to integrate any purchased evaluation data into the training loop.
Co-author of Gecko (T2I eval with >100K human ratings) and Imagen 3 core contributor
Product lead for Google DeepMind's entire generative-media line — Nano Banana / Gemini Image, Imagen, Veo and Omni. She has said publicly, and with visible discomfort, that modelling teams end up setting the world's default aesthetic and are not obviously the right people to do it, and she has named the data she cannot manufacture internally: real-world professional task trajectories. That is the single most explicit buying signal from any frontier-lab visual-product lead in this batch.
Research lead for Nano Banana and technical lead for Gemini 2.5 Flash Image, with fourteen years at Disney Research Zürich and Adobe Research before Google DeepMind. He is the rare frontier-model research lead who spent a decade building tools for professional creatives, and he has said publicly that human evaluation is central to how they judge the model and that training on AI-generated images is a real risk.
Lead author of Gecko: T2I evaluation, prompts and human ratings (>100K annotations)
Winkler works on the data side of mid-training and RL - her current LinkedIn headline reads 'Data (Mid-Training / RL ...)'. Her older public posts point at Google/DeepMind: she shared a 'Senior Technical Program Manager, Data' opening in late 2023, a Gemini post in December 2023, and a 'Request for services - Expert AI Tutor' post in early 2024. I could not confirm which of the frontier labs she is at today.
Senior author on Gecko human-rating eval paper
LinkedIn profile: "TPM Lead - Post-Training Frontier Model Research" at Google DeepMind, San Francisco Bay Area, in role since 2026-06.
LinkedIn profile: "Human Data, Gemini" at Google DeepMind, San Francisco Bay Area, in role since 2024-10.
Imagen 3 core contributor; co-author PASTA (preference-adaptive T2I with paid rater data, ICML 2025)
LinkedIn profile: "Gemini Post-Training Lead - Safety & Multi-Turn verticals" at Google DeepMind, Zürich Metropolitan Area, in role since 2024-05.
LinkedIn profile: "Gemini Post-training Modeling & Evals TPM Lead" at Google DeepMind, San Francisco Bay Area, in role since 2023-07.
Gecko co-author; Imagen 3 contributor
Co-author of Genie 3 world-model announcement
Imagen 3 core contributor (report details 366K ratings from 3,225 raters via external platform)
LinkedIn profile: "Product Manager, Gemini Post-Training" at Google DeepMind, New York, New York, United States, in role since 2025-01.
Took over Gemini app from Sissie Hsiao (Apr 2025); Labs owns Flow/creative tools
LinkedIn profile: "Senior Staff Technical Program Manager, Agentic Post-Training" at Google DeepMind, San Francisco Bay Area, in role since 2026-04.
LinkedIn profile: "Senior Staff Software Engineer / Manager, GenAI Evaluation Infrastructure" at Google DeepMind, San Francisco Bay Area, in role since 2025-05.
LinkedIn profile: "Senior Staff Engineer - Human Data Labeling " at Google DeepMind, San Francisco, California, United States, in role since 2024-06.
Named Nano Banana team member
Named Nano Banana core team member; Imagen 3 core contributor
LinkedIn profile: "Senior Staff Technical Program Manager, Gemini Audio Frontier Models & Evaluation" at Google DeepMind, Sunnyvale, California, United States, in role since 2026-07.
LinkedIn profile: "Senior Staff Research Engineer/Manager - Post-training, GenAI" at Google DeepMind, San Francisco Bay Area, in role since 2023-01.
Named as working on Gemini RL and Omni Thinking on AI Engineer leads panel with Veo/Nano Banana leads
Co-author of Genie 3 announcement
LinkedIn profile: "Program Manager (Research and Evaluation) - Gemini " at Google DeepMind, London, England, United Kingdom, in role since 2021-10.
LinkedIn profile: "Gemini Evals & Data (Director)" at Google DeepMind, New York, New York, United States, in role since 2023-07.
LinkedIn profile: "Head of Post-Training Eval" at Google DeepMind, Los Altos, California, United States, in role since 2024-06.
Imagen 3 advisor; believed to lead Gemini model product (not verified this pass)
LinkedIn profile: "AI TPM Lead | Gemini Post-training Evals" at Google DeepMind, Los Altos, California, United States, in role since 2026-07.
Gecko co-author; Imagen 3 core contributor
LinkedIn profile: "Senior Technical Program Manager | Gemini Post-Training - Model Quality" at Google DeepMind, Zurich, Zurich, Switzerland, in role since 2026-05.
Anthropic
Builds Claude. Runs an in-house Human Data team plus vendor pipelines; launched Claude Design (Apr 2026), which makes UI/visual design taste an explicit training and eval target.
Penn is Head of Product for Anthropic's AI Research and Labs teams and was the company's first technical product manager, hired in 2023 when the product org was five engineers. She has shipped every Claude model generation since Claude 2 and incubated Claude Code, MCP, Skills, computer use and tool use. She is the loudest public advocate anywhere at a frontier lab for the idea that evals, not specs, define a product — which makes her the natural internal sponsor for buying evaluation data.
Leads design for Claude and Cowork at Anthropic, after being Director of Design at Figma where she owned FigJam, Slides and community products. She is the most articulate public voice inside a frontier lab on the question of whether AI can hold taste, and she sets the internal standard that Anthropic's design-quality work is calibrated against. For a creative-data vendor she is the person who defines what 'good' looks like, even if she does not hold the budget.
LinkedIn profile: "Strategic Product & Data Partnerships" at Anthropic, New York, New York, United States, in role since 2026-03.
LinkedIn profile: "Data Operations Manager" at Anthropic, New York City Metropolitan Area, in role since 2026-03.
LinkedIn profile: "Strategic Data Partnerships" at Anthropic, New York City Metropolitan Area, in role since 2026-04.
Instagram co-founder who joined Anthropic as Chief Product Officer in May 2024 and moved in January 2026 to co-lead Anthropic Labs with Ben Mann. Labs is the unit that shipped Claude Code, Skills, Cowork and — in April 2026 — Claude Design, which makes visual/UI taste an explicit product surface. He is the executive who can authorise a spend on design-taste data, and he is personally close enough to the work to be acknowledged in Anthropic's own frontend-design grader write-up.
The Anthropic Labs engineer who wrote the company's public design grader — a four-dimension rubric (design quality, originality, craft, functionality) used to score the output of long-running app-building agents. He also co-wrote the November 2025 piece diagnosing why model-generated UI looks generic. He is the closest thing in the industry to a named owner of 'how we score whether a model has taste', and his own write-up concedes the grader is calibrated on few-shot examples rather than on collected human preference data.
Leads character/personality fine-tuning and wrote Claude's Jan 2026 constitution; defines what 'good' outputs look like.
Named as leading Labs with Krieger, Jan 2026.
LinkedIn profile: "Human Data Operations" at Anthropic, Los Angeles Metropolitan Area, in role since 2026-08.
LinkedIn profile: "AI Data Quality & Annotation Specialist" at Anthropic, San Francisco, California, United States, in role since 2023-10.
Started and leads Societal Impacts (human studies, values, red-teaming datasets).
LinkedIn profile: "Data Operations" at Anthropic, United States, in role since 2026-06.
Co-author of 'Teaching Claude Why' on safety training data (May 2026).
Listed as Chief Science Officer; research leadership over training.
Co-author of the Jan 2026 agent-evals guide (human graders used to calibrate LLM graders).
LinkedIn profile: "Human Data Project" at Anthropic, Lincoln, Rhode Island, United States, in role since 2026-05.
LinkedIn profile: "Data Operations " at Anthropic, New York City Metropolitan Area, in role since 2025-04.
Co-author of 'Demystifying evals for AI agents', which covers SME review and crowdsourced human grading (Jan 2026).
LinkedIn profile: "Generalist | Human Data Operations" at Anthropic, Portland, Oregon, United States, in role since 2026-01.
LinkedIn profile: "Member of Technical Staff (Research Manager, Post-Training)" at Anthropic, London, England, United Kingdom, in role since 2022-10.
Adobe (Firefly / Generative AI)
Firefly image/video/audio/vector models marketed as 'commercially safe' — trained on licensed Adobe Stock data. Largest licensed-data buyer in creative AI; pays contributors annually and hires teams for dataset acquisition and annotation.
Runs Adobe's generative-AI model organisation — the Firefly image, video, audio and vector models and the Firefly app that now also hosts partner models. He is the senior technical requester above the Applied Research group that owns Firefly's training-data lifecycle (acquisition, curation, annotation), so a creative-data vendor selling into Adobe ends up specified by his org.
LinkedIn profile: "Data Operations Manager" at Adobe, Chicago, Illinois, United States, in role since 2023-03.
LinkedIn profile: "Sr. Engineering Program Manager, AI Research Data Programs" at Adobe, San Francisco Bay Area, in role since 2018-02.
CTO of Adobe's Digital Media business — Creative Cloud and Document Cloud — and the executive who also oversees Adobe Research. He launched Firefly and is the named voice on Adobe's model strategy. He is an exec sponsor, not a buyer: he sets whether Adobe pays for human judgement at all, and his org contains both the Firefly applied-research data team and Adobe Research.
LinkedIn profile: "Senior Data Operations Manager" at Adobe, Greater Pittsburgh Region, in role since 2023-03.
Founded and runs Adobe's Firefly Enterprise and Firefly Foundry businesses — custom Firefly-family models fine-tuned on a customer's own brand IP, sold usage-based rather than per seat. Crucially, the Sept 2026 Adobe req for a Sr Manager of ML Engineering leading 'a distributed team focused on training data for Generative AI' sits inside Firefly Foundry. That makes her business unit the one place at Adobe with an explicit, budgeted, per-customer data pipeline.
LinkedIn profile: "AI/ML Product Manager, Agent Evaluation & Grounding" at Adobe, San Francisco, California, United States, in role since 2025-04.
LinkedIn profile: "AI Annotation Specialist, Adobe Express" at Adobe, Bucharest, Romania, in role since 2026-03.
LinkedIn profile: "Senior Manager of Engineering, Generative AI Evaluation & Data Platforms" at Adobe, Stow, Massachusetts, United States, in role since 2015-03.
LinkedIn profile: "Manager, Product manager - Content evaluation" at Adobe, France, in role since 2024-09.
LinkedIn profile: "Group Product Manager- (Gen AI), Scientific Evaluation Lead" at Adobe, San Francisco, California, United States, in role since 2026-03.
LinkedIn profile: "Senior Director - AI Engineering and Evaluation" at Adobe, Rocklin, California, United States, in role since 2023-11.
LinkedIn profile: "Engineer - Data operations" at Adobe, Seattle, Washington, United States, in role since 2024-01.
LinkedIn profile: "Engineering Manager — Agentic AI / Evals" at Adobe, San Francisco, California, United States, in role since 2026-07.
LinkedIn profile: "Senior Group Engineering Manager - Post-training, Inference (Adobe Firefly)" at Adobe, Seattle, Washington, United States, in role since 2024-10.
Luma AI
Video/world-model lab (Ray3, Ray3.2, Uni-1.1 image model) backed by HUMAIN (Series C, Nov 2025). Publishes human-rated evaluation reports built with filmmakers and artists, a direct fit for expert creative-judgement data.
Jain co-founded Luma and runs it, and his stated philosophy of post-training is almost a sales script for curated human data: show the model what you want it to learn, and show it good examples. Luma took Series C backing from HUMAIN (Saudi Arabia) in November 2025 and is pushing into world models and international markets, with offices in London and Riyadh and Forward Deployed Creative roles posted in Germany, the UK, the US and KSA.
LinkedIn profile: "AI Data Annotation Manager" at Luma AI, Rushden, England, United Kingdom, in role since 2025-07.
Day is one of two named authors of Luma's Ray3 Evaluation Report (October 2025), which is the most detailed public description by any video lab of a human-rated creative evaluation framework. She and Simran Motwani define the seven dimensions Luma grades its video models on and state that outputs were rated by trained evaluators with diverse domain expertise. Almost nothing else about her is public, so she is high-value and low-visibility at the same time.
Motwani is the second named author of Luma's Ray3 Evaluation Report and is listed at Luma as a software engineer on a user-maintained org chart. The combination suggests she is the engineering half of Luma's evaluation function — the person who builds the harness, the comparison tooling and the aggregation, while the rubric is a joint product with Sabrina Day. She is the technically-minded contact for anyone proposing to plug an external rater panel into Luma's evaluation pipeline.
LinkedIn profile: "Data Annotation Manager" at Luma AI, Greater Kolkata Area, in role since 2025-07.
Co-founder/CTO per The Org (listed as 'Alex Y.'; unverified).
Song has been Luma's Chief Scientist, leading research from 3D through video to multimodal models (Ray, Uni-1), and is the author of DDIM — one of the foundational diffusion sampling methods. Important caveat: as of September 2026 his own website describes him as former Chief Scientist at Luma AI, while third-party listings still show him in post. If he has left, Luma's research leadership and its data philosophy are both in flux, which matters for anyone planning an approach.
Lead author of Terminal Velocity Matching (Luma, 2025-11) and Inductive Moment Matching.
Runs Luma's creator studio in LA, which is a source of expert creative feedback.
Black Forest Labs
Makers of FLUX (image, editing, and since July 2026 FLUX 3 joint image/video/audio). Stable Diffusion's original authors. ~$3.25B valuation, Meta/Adobe/Canva customers. Runs pairwise human-preference evals and is building a data procurement function. A top taste-data target near Julian.
Blattmann is one of BFL's three founding researchers and the one most associated with its video line — he co-authored Stable Video Diffusion before founding the company. He speaks for BFL's strategy publicly (the open-weights-as-funnel argument) and is a plausible internal owner of video data quality, which is the modality where BFL's evaluation claims are weakest.
Sauer's own site lists him as Co-Founder & Research Scientist at Black Forest Labs, which upgrades the 'research staff' label in the index. He is the distillation and fast-inference specialist — StyleGAN-T, Adversarial Diffusion Distillation, Latent ADD — and a co-author on the SD3 rectified-flow paper that won a best-paper award at ICML 2024. Distillation work lives or dies on preference evaluation, because the standard question is whether the fast student still looks as good as the teacher.
Esser is the third founding researcher, co-author of VQGAN ('Taming Transformers') and of the Stable Diffusion and SD3 papers. Of the three founders he is the one who has spoken most directly about taste: he has publicly framed curation of samples as something the model cannot do for you. That is a rhetorical opening a creative-data vendor can build on.
Rombach is the first author of the Latent Diffusion / Stable Diffusion line and now runs Black Forest Labs from Freiburg im Breisgau, roughly a two-hour drive from Julian in Lübeck's sense of 'domestic'. He is the final signature on what BFL spends on data and evaluation, and he personally frames data pre-processing and filtering — not architecture — as the place a visual lab distinguishes itself. FLUX 3 shipped in July 2026 with headline human-preference win rates and no disclosed rater methodology, which is precisely the gap a credible creative-eval vendor fills.
Co-author of FLUX.1 Kontext paper (June 2025); original latent diffusion co-author.
Co-author of FLUX.1 Kontext paper (June 2025); SDXL lead author.
Kulal is listed as a co-founder of Black Forest Labs on his LinkedIn and across third-party trackers, having come out of Stanford's vision group and Stability AI. He is a research-side name rather than a budget holder, and there is little public material on what he owns inside BFL.
Co-author of FLUX.1 Kontext paper (June 2025); senior research staff.
Canva (incl. Leonardo.ai)
Design platform that built its own layered 'design foundation model' (Oct 2025, Canva AI 2.0) and owns Leonardo.ai (Phoenix, Lucid Origin image models). Buyer for design/layout quality judgements and image aesthetics.
Canva co-founder and CPO, and the person who has publicly described Canva's design foundation model in the most data-revealing terms of anyone in this batch: trained on structured data, millions of designs, and the actual sequence of edits used to build them, with the team deliberately breaking designs so the model learns to recognise spacing and hierarchy errors. That is a description of an internal human-judgement data programme, said out loud.
LinkedIn profile: "Data Labelling PH Operations Manager" at Canva, Binãn, Calabarzon, Philippines, in role since 2025-01.
Owns Canva's AI product surface — Canva AI, Magic Studio and the Canva AI 2.0 release built on Canva's own design foundation model. He is the person who publicly described what that model is trained on: millions of professionally created templates already separated into editable components. He decides what counts as a good generated design in product terms, and is the natural requester for design-quality judgement data.
Co-founded Leonardo.Ai in Dec 2022 and led it through in-house foundation models (Phoenix, 2024; Lucid Origin, 2025) and the 2024 acquisition by Canva. He runs the image-model research and product group that sits beside Canva's design model, with a team that was 120 researchers, engineers and designers at acquisition and 200+ staff overall.
CTO; Head of AI Research reports to him.
Leonardo co-founder; current role not verified.
Leonardo co-founder; current role not verified.
LinkedIn profile: "Data Tagging Lead (Content Metadata, Labeling & Annotation)" at Canva, Metro Manila, in role since 2025-01.
Named in Canva's Head of AI Research job posting as one of two leaders the role reports to — the Gen AI lead, alongside CTO Brendan Humphreys. That makes him the research-side counterpart to Danny Wu's product side, and a plausible owner of what Canva's generative models are trained and evaluated on. Almost nothing else about him is public.
LinkedIn profile: "Staff Research Engineer (Post-Training & Inference)" at Canva, Greater Sydney Area, in role since 2024-09.
Krea
Creative-tool platform turned model lab (FLUX.1 Krea, Krea 2 open weights, Krea Realtime video). Openly 'opinionated' about aesthetics and anti-'AI look'. Already commissioned a Contra Labs creative-expert benchmark. One of the most taste-focused buyers.
CTO and co-founder per Contrary (June 2025).
Millon is a founding engineer at Krea and co-author with Sangwu Lee of the FLUX.1 Krea post-training write-up, plus sole author of the Krea Realtime 14B release. He is the most publicly reachable person on Krea's model team — active GitHub, active X, LinkedIn — and the practical entry point to Krea's research side.
Lee is the lead author of both the FLUX.1 Krea post-training write-up and the Krea 2 technical report, which together are the most explicit public account anywhere of how a lab builds an opinionated aesthetic into a model. He personally documented rejecting open-source preference datasets because they bias toward symmetric, simple composition and the 'AI look', and building instead on in-house preference labels from people who know the model's specific weaknesses. He is the most articulate taste-engineer in this batch — and, for that reason, the hardest sell.
Perez co-founded Krea with Diego Rodriguez and runs it as CEO. Krea's whole positioning is taste: an AI creative platform that amplifies a user's own aesthetic rather than averaging everyone's, with stated plans for hyper-personalisation so creators get models that understand their individual preferences. That is the same thesis as Julian's business, stated by a CEO — which makes Krea either a natural partner or a competitor for the same idea.
Author on Krea 2 technical report (June 2026).
Ideogram
Text-to-image lab (ex-Google Imagen founders) specialised in typography and graphic design; Ideogram 4.0 (June 2026) open-weights. Explicitly benchmarks against graphic-designer preference, so a natural buyer of designer pairwise judgements.
Ideogram co-founder and one of the lead authors of Google's Imagen work — the research half of the founding team. If Ideogram runs post-training on human preference data, he is one of a handful of people who would write the spec for it. Public record on his current day-to-day remit is thin.
Founder and CEO of Ideogram, the text-to-image lab built around typography and graphic design. Ideogram 4 (3 Jun 2026) shipped open-weights under Apache 2.0 with a model card whose headline evaluation is a blind study run by Contra Labs with ten professional designers — meaning Ideogram already pays for, or at least publishes, third-party designer-judgement evaluation. That makes Norouzi the most directly qualified buyer of creative-taste data in this batch.
Ideogram's founding designer on the AI side — at a company with no dedicated human-data function and a flat technical-staff structure, the in-house designer is the person most likely to be writing the internal taste rubrics and judging output quality. Quoted publicly in the How&How case study on Ideogram's brand work.
ML researcher at Ideogram per ZoomInfo directory.
ML team MTS per ZoomInfo directory.
ML MTS per ZoomInfo directory.
Co-founder, diffusion-model pioneer (DDPM); current day-to-day role not verified.
Creative lead — probable internal judge of aesthetic output quality.
Recraft
Design-focused image/vector generation model (Recraft V3, V4 Feb 2026) trained on its own ~20B-param models; markets 'design taste'. Core buyer profile for designer aesthetic judgements.
Founded and runs Recraft, the London image/vector lab that markets its models explicitly on 'design taste'. She personally authored the V4 launch post (17 Feb 2026) stating the model was developed in close collaboration with designers and tuned around design aesthetics and professional expectations — a taste claim with no published methodology behind it. At a company this size she is the economic buyer for any data spend.
LinkedIn profile: "Data Annotation Manager" at Recraft, London, England, United Kingdom, in role since 2025-10.
Runs Recraft's AI team — around eleven people per The Org — and is the named technical voice on how Recraft V4 was trained. He is the most actionable working-level buyer in this batch: a small, London-based model team that owns training, evaluation and, by implication, the annotation programme Recraft advertises for in the same Engineering org.
LinkedIn profile: "Data Annotation Architect/ Data Analyst" at Recraft, London, England, United Kingdom, in role since 2025-08.
LinkedIn profile: "Data Annotation Architect" at Recraft, London, England, United Kingdom, in role since 2026-02.
Heads design at Recraft with a very small team — two reports per The Org. In a company that markets its model on 'design taste', the head of design is the most likely internal owner of what taste means and the most likely coordinator of the designer collaboration Recraft says shaped V4. Public record on him is almost empty.
LinkedIn profile: "Data Annotation Architect | Analyst-Developer" at Recraft, London Area, United Kingdom, in role since 2025-12.
Runway
AI video/world-model lab (Gen-4.5, robotics/world models) serving filmmakers and enterprises, ~$5.3B valuation. Film-industry DNA and its own studio make it a strong buyer of expert cinematic/aesthetic judgement.
Germanidis co-founded Runway and is now co-CEO alongside Cristóbal Valenzuela, having previously been CTO. He is the intellectual driver of Runway's pivot from video generation to general world models, and as of September 2026 he is publicly arguing that world models are the RL environments for everything, including robot policy evaluation. That pivot is the key fact for Julian: Runway's centre of gravity is moving away from filmmaker taste and toward physical-world simulation.
Yang is Runway's Research Director and one of the few people there with a long product history of shipping human-motion and character tools — at Adobe Research he built the body tracking in Character Animator and the motion capture behind After Effects' Project GoFigure. At Runway he works on video generative models and controllability. He is a technical requester: the person who would specify what a video-quality dataset actually has to contain.
Grumman is a Research Lead at Runway and one of only two research people (with Jimei Yang) listed on the 2026 Runway AI Summit programme, where she appears on the Research Q&A panel described as covering research decisions few outside the company see. There is very little else public about her, so this is a thin but real dossier.
Co-founder with a design background; CIO per TechCrunch May 2026.
CPO per Runway AI Summit 2026 speaker list.
General Counsel; would own data licensing and vendor contracts.
Co-CEO; design/art background; drives creative-industry positioning.
Umpherson is Chief Creative Officer of Runway and Runway Studios, which makes her the most explicit taste authority in this entire batch — her job title is literally judging creative quality. She judges at The One Show, speaks at VIEW Conference, and owns the relationship between Runway's models and working filmmakers, agencies and brands. She is a gatekeeper rather than a technical buyer, which changes the pitch but not the value.
Named CTO in Feb 2026 leadership announcement.
ElevenLabs
Leading AI audio lab (TTS, dubbing, Scribe STT, Eleven Music; $11B valuation Feb 2026). Active, explicit buyer of voice data and runs data ops with external labelling vendors; audio not visual, but a template for creative-taste data buying.
Co-founder and CTO of ElevenLabs, running research and research engineering across TTS, dubbing, Scribe speech-to-text and Eleven Music. A Data Ops team sits beside his research org and exists to answer what data is needed to train the models and how to source it at scale — including managing external vendors and third-party labelling teams. He is the demand side of that loop.
LinkedIn profile: "Data Operations" at ElevenLabs, Georgia, in role since 2025-05.
LinkedIn profile: "Language Data Annotation & AI Data Evaluation" at ElevenLabs, Berlin, Berlin, Germany, in role since 2025-10.
Leads AI safety at ElevenLabs — a team of roughly seventeen — covering misuse prevention for voice and audio models, including consent and verification for voice data. She is a gatekeeper rather than a buyer: any human-data programme touching voices, likeness or performance has to satisfy her function.
Leads key partnerships; possible counterpart for data partnerships.
CEO; Series D announcement Feb 2026.
VP Ops managing ~64 people; possible home of data ops (surname not shown).
Midjourney
Leading aesthetics-first image and video model (V8.1 in 2026), small self-funded team. Gets huge in-house taste signal from user image rankings and personalization. Also has a research group studying creative taste and preference data, a direct fit for Julian.
Chung is a research scientist at Midjourney working on AI for creative activity, and the first author of LiteraryTaste — a preference dataset built by having 60 people annotate 100 pairs of creative texts each, then modelling individual versus collective taste. He is, as far as public evidence goes, the person inside a major image lab who has most explicitly treated taste itself as a dataset problem. That makes him the single best-matched researcher in this batch to what Julian sells.
Holz founded and runs Midjourney, which has been profitable since 2022, has never raised institutional venture capital, and is run as a small self-funded research lab. He is the final word on everything including data, but Midjourney's whole culture is to build in-house and disclose nothing, and it is now defending copyright suits from Disney, Universal and Warner Bros. Discovery. That combination makes him a hard buyer and a legally cautious one.
Kreminski ran Midjourney's Storytelling Lab from 2023 to 2025 and co-authored LiteraryTaste, but his own CV now lists him as Assistant Professor of Design Tech at Cornell Tech since July 2026 — he has left Midjourney. He remains one of the sharpest public voices on creative homogenisation (his best-paper work showing LLMs narrow human ideation) and on why preserving diversity of creative output is a data problem.
Midjourney-affiliated co-author of LiteraryTaste (Nov 2025), a creative-writing taste preference dataset gathered from paid Upwork annotators.
Midjourney-affiliated co-author of LiteraryTaste (Nov 2025), a creative-writing taste preference dataset gathered from paid Upwork annotators.
Stability AI
Stable Diffusion maker, now pivoted to enterprise and professional creative workflows (image, audio, 3D) with music-label and EA partners. Smaller research bench after departures. Relevant for licensed audio/visual data and brand-adherence evals.
LinkedIn profile: "Director of Data Strategy and Operations" at Stability AI, San Francisco Bay Area, in role since 2026-06.
Named CTO per Stability AI announcement.
Senior author of ARC post-training (2025) and Stable Audio Open; LinkedIn lists him as Research Scientist at Stability AI.
Akkaraju has run Stability AI since June 2024 and has rebuilt it around licensed data and professional creative workflows. In August 2026 he closed a $76M Series B whose investors are Electronic Arts, Sony Music Group, Universal Music Group, Warner Music Group and AMD Ventures — the rights holders who used to sue companies like his. That makes licensing provenance, not model benchmarks, Stability's central strategic asset, and it makes any data supplier's chain of title the first question in the room.
Lead author of Stable Audio Open and ARC co-author; LinkedIn shows Stability AI.
Pika
Consumer AI video lab (Pika 2.5, Pikaformance, social app, 'AI Selves') valued at $470M. It is hiring its own post-training researcher to run video human-preference studies and build reward models, which is an explicit buyer signal.
Meng is Pika's co-founder and CTO and the stronger research name of the pair — her Stanford work with Stefano Ermon includes SDEdit, a widely used image-editing method, and contributions to consistency and distillation models. She is the person at Pika who would specify data if Pika decided to compete on model quality; there is no public evidence it has.
Guo co-founded Pika with Chenlin Meng out of Stanford and runs it as CEO. Pika raised around $80M and has been valued near $470M, but has repositioned as a consumer-first, social video product rather than a professional model lab — closer to a creative app than a frontier research shop. That makes it a weak fit for expert-judgement data: its quality signal is engagement, not craft.
Founding Creative Director; likely internal judge of output quality and style.
xAI (SpaceXAI)
Builds Grok and Grok Imagine (image and video, ranked #1 text-to-video on Artificial Analysis in Jan 2026). SpaceX acquired it in Feb 2026 and it became SpaceX's AI division in May 2026. It runs a large in-house 'AI tutor' workforce and hires expert tutors directly.
LinkedIn profile: "Manager, Human Data Operations" at SpaceXAI, Tokyo, Japan, in role since 2026-08.
LinkedIn profile: "Manager, Human Data Operations" at SpaceXAI, United States, in role since 2026-09.
Pasini, 20, runs xAI's data annotation organisation - the AI-tutor workforce that produces Grok's human data. He took it over in 2025 and in September that year executed the cut of roughly 500 generalist annotators, shrinking the group from about 1,500 to around 900 and re-pointing it at specialist tutors. In xAI's February 2026 lineup he is named as leader of 'Expert Mentors & Grokopedia'. Whatever his age, he is the closest thing xAI has to an economic buyer of human data.
LinkedIn profile: "Global Data Partnerships & BD - X API" at SpaceXAI, London, England, United Kingdom, in role since 2024-12.
LinkedIn profile: "Manager, Human Data Operations" at SpaceXAI, Herndon, Virginia, United States, in role since 2026-09.
LinkedIn profile: "Manager, Human Data Operations" at SpaceXAI, New York, New York, United States, in role since 2026-09.
LinkedIn profile: "Human Data" at xAI, San Francisco Bay Area, in role since 2024-07.
LinkedIn profile: "Human Data Team Lead" at xAI, Mexico, in role since 2024-09.
LinkedIn profile: "Human data team lead (AI)" at xAI, Riyadh, Saudi Arabia, in role since 2024-12.
LinkedIn profile: "Human Data Expert Team Lead" at SpaceXAI, San Francisco, California, United States, in role since 2025-06.
LinkedIn profile: "Human Data Expert Team Lead" at SpaceXAI, Delson, Quebec, Canada, in role since 2025-07.
LinkedIn profile: "Human Data Engineer II" at SpaceXAI, Cleveland, Ohio, United States, in role since 2026-08.
LinkedIn profile: "Human Data Team Lead | Coding" at SpaceXAI, South Africa, in role since 2026-04.
LinkedIn profile: "Human Data" at SpaceXAI, Miami Beach, Florida, United States, in role since 2025-11.
Danielle Salvaggio is a human data team lead at xAI - one of roughly thirty team leads listed in its Human Data Operations org, each running a pod of contract AI tutors. Her LinkedIn headline reads 'Human Data Team Lead - xAI'. There is no public material beyond the title: no talks, writing, papers or press. This is a quality gatekeeper inside an in-house workforce, not a buyer.
Announced 14 Mar 2026 as joining xAI/SpaceX to lead underlying Grok model work in the rebuild
LinkedIn profile: "Human Data Team Lead" at SpaceXAI, Nagoya, Aichi, Japan, in role since 2024-09.
LinkedIn profile: "Human Data " at SpaceXAI, Boston, Massachusetts, United States, in role since 2025-07.
LinkedIn profile: "Human Data Expert Team Lead" at xAI, Greater Winnipeg Metropolitan Area, in role since 2025-04.
LinkedIn profile: "Human Data Team Lead" at xAI, Houston, Texas, United States, in role since 2024-08.
LinkedIn profile: "Human Data Team Lead" at SpaceXAI, Greater Cleveland, in role since 2026-02.
LinkedIn profile: "Human Data Team Lead" at SpaceXAI, Sydney, New South Wales, Australia, in role since 2025-07.
LinkedIn profile: "Human Data" at xAI, United States, in role since 2024-08.
LinkedIn profile: "AI Human Data Team Specialist" at SpaceXAI, Dubai, United Arab Emirates, in role since 2025-07.
LinkedIn profile: "Team Lead, Human Data (Voice AI)" at SpaceXAI, Marina del Rey, California, United States, in role since 2024-10.
LinkedIn profile: "Human Data Team Lead" at SpaceXAI, Panama City, Florida, United States, in role since 2025-01.
Nikita Paskevic is a human data team lead at xAI - one of roughly thirty team leads listed in its Human Data Operations org, each running a pod of contract AI tutors. Listed by name on xAI's Human Data Operations team page. There is no public material beyond the title: no talks, writing, papers or press. This is a quality gatekeeper inside an in-house workforce, not a buyer.
Nikkole Blackwood is a human data lead at xAI - one of roughly thirty team leads listed in its Human Data Operations org, each running a pod of contract AI tutors. Her LinkedIn headline reads 'Human Data Lead @xAI'; a third-party business directory also lists her as Team Lead, Human Data at x.ai. There is no public material beyond the title: no talks, writing, papers or press. This is a quality gatekeeper inside an in-house workforce, not a buyer.
LinkedIn profile: "Human Data Team" at xAI, Portugal, in role since 2025-04.
Holland leads a 16-person human data team at xAI covering the X platform domain; some listings also tag him 'Post Training'. He is a career musician and music teacher - Eastman-trained trumpet player, Virginia Symphony, Air Force band, 20+ years teaching high school in Texas - who moved into AI data work, and he markets that pedigree explicitly as the source of his judgement and patience with training relationships.
LinkedIn profile: "Human Data - Capital Markets" at SpaceXAI, New York City Metropolitan Area, in role since 2026-04.
LinkedIn profile: "Human Data Team Lead" at SpaceXAI, Canada, in role since 2025-01.
LinkedIn profile: "Human Data | Lead" at xAI, Greater Toronto Area, Canada, in role since 2025-04.
LinkedIn profile: "Human Data Wizard" at xAI, Greater Houston, in role since 2024-12.
Duerr has led a human data team at xAI since October 2024. She is a Duke religion PhD who came in through the door xAI actually uses - freelance evaluation work, then AI Tutor, then team lead - and she argues publicly for the value of humanities-trained people in data work. She is a quality gatekeeper: she sets and enforces what good looks like for a tutor pod, not a budget holder.
LinkedIn profile: "Human Data Expert Team Lead" at SpaceXAI, Brooklyn, New York, United States, in role since 2026-04.
LinkedIn profile: "Human Data Team" at SpaceXAI, Mumbai, Maharashtra, India, in role since 2025-01.
Microsoft AI (MAI)
Suleyman's in-house model lab (MAI-Image-2.5, MAI-Voice-2, MAI-Thinking-1) powering Copilot, PowerPoint and Foundry; explicitly courts creatives for image quality, so a plausible taste-data buyer.
LinkedIn profile: "Senior Program Manager - AI Evaluation" at Microsoft, Greater Seattle Area, in role since 2025-11.
LinkedIn profile: "Director - Continuous Evaluation Program" at Microsoft, County Dublin, Ireland, in role since 2024-04.
LinkedIn profile: "Global Evaluation Strategy Lead | TeleSMB EMEA & Japan SAS Lead (via Accenture)" at Microsoft, Dublin, County Dublin, Ireland, in role since 2022-07.
LinkedIn profile: "Product Manager – AI Agents & Evaluation (Power Apps)" at Microsoft, Seattle, Washington, United States, in role since 2024-08.
LinkedIn profile: "AI Data Quality Analyst (Google • Microsoft • Amazon • Etc)" at Microsoft, Boston, Massachusetts, United States, in role since 2014-05.
LinkedIn profile: "Senior Continuous Evaluation Program Manager" at Microsoft, Doha Metropolitan Area, in role since 2024-10.
LinkedIn profile: "Senior Security Site Evaluation Manager EMEA" at Microsoft, Glasgow, Scotland, United Kingdom, in role since 2022-10.
LinkedIn profile: "Principal AI Security Research Manager | Head of AI Evaluation Systems" at Microsoft, Raleigh-Durham-Chapel Hill Area, in role since 2026-07.
LinkedIn profile: "Continuous Evaluation Program Manager" at Microsoft, Waterville, Washington, United States, in role since 2025-12.
Russell works on generative-AI content partnerships - her LinkedIn headline begins 'Gen AI Content Partnerships and ...' and the index carries her as Head of AI Data Partnerships. That function, licensing and sourcing content and data for model training, is the closest thing on the buyer side to a procurement counterpart for a data vendor. I could not confirm which company she is at: no press, company page or directory entry tied her name to an employer.
LinkedIn profile: "Director, Security Site Evaluation - AMERS" at Microsoft, Greater Seattle Area, in role since 2025-03.
LinkedIn profile: "Senior Product Manager, Copilot AI Quality Evals" at Microsoft, Prague, Czechia, in role since 2026-01.
LinkedIn profile: "Continuous Evaluation Program - Program Manager" at Microsoft, Atlanta Metropolitan Area, in role since 2024-07.
LinkedIn profile: "Principal Product Manager – Viva Engage (Enterprise AI Agents & Evals)" at Microsoft, San Francisco, California, United States, in role since 2022-12.
Listed as MAI chief scientist
LinkedIn profile: "Principal Product Manager - M365 Copilot Platform | Evaluation & Experimentation" at Microsoft, Redmond, Washington, United States, in role since 2024-11.
LinkedIn profile: "Director, M365 Data & Security Organization - AI for Security & Security for AI, Agent Evals" at Microsoft, Sammamish, Washington, United States, in role since 2022-03.
LinkedIn profile: "Continous Evaluation Program Manager" at Microsoft, Campinas, São Paulo, Brazil, in role since 2024-07.
LinkedIn profile: "Senior Program Manager, Continuous Evaluation" at Microsoft, Phoenix, Arizona, United States, in role since 2026-05.
Authored Jun 2026 launch of seven MAI models (Image-2.5, Voice-2, Thinking-1) citing blind human side-by-sides
LinkedIn profile: "Director Continuous Evaluation Program" at Microsoft, Everett, Washington, United States.
LinkedIn profile: "AI Evaluation And Data Annotation Specialist" at Microsoft, Derby, England, United Kingdom.
LinkedIn profile: "Principal PM Lead - Agent Store, RAI, Agent virality, Evals, Developer tools" at Microsoft, Bengaluru, Karnataka, India, in role since 2018-07.
LinkedIn profile: "Principal Applied Scientist | Frontier Tuning Evaluation" at Microsoft, Brazil, in role since 2026-09.
LinkedIn profile: "Director, Datacentre Physical Security Site Evaluation - EMEA" at Microsoft, Dublin, County Dublin, Ireland, in role since 2024-09.
LinkedIn profile: "Principal PM Manager - M365 Copilot Connectors, Response Quality & Evals" at Microsoft, Bengaluru, Karnataka, India, in role since 2023-06.
LinkedIn profile: "Senior Product Manager - AI governance, Agent evals, DevEx, Growth" at Microsoft, Redmond, Washington, United States, in role since 2025-06.
LinkedIn profile: "Senior Product Manager, AI Evaluation Infrastructure & Synthetic Data" at Microsoft, Irvine, California, United States, in role since 2025-05.
LinkedIn profile: "Program Manager - Continuous Evaluation Program" at Microsoft, Des Moines, Iowa, United States, in role since 2024-06.
Figma
Design platform shipping Figma Make (prompt-to-app/UI), First Draft and Figma Weave (ex-Weavy image/video canvas, acquired Oct 2025). Runs human 'taste and judgement' evals with contractors — direct fit for UI/design-quality data.
Kossnick runs all AI products at Figma (Figma Make, First Draft, AI search, Weave) and personally designed the evaluation process behind Figma Make. That process contains the single most explicit public statement by a design company that it pays human contractors to run nightly 'taste and judgement' evals against internal brand guidelines. He is the clearest named buyer of design-taste data in the whole index.
CTO; owns AI engineering.
CDO — design craft owner; likely sponsor of design-quality eval standards.
CPO; named in Figma AI eval article.
Gamma
AI presentation/website/social design generator ($100M ARR, ~50 staff, $2.1B valuation). Its core quality problem is visual layout and design taste; fine-tunes open models and evaluates frontier models.
Heads AI product at Gamma, where she is responsible for the AI-native generation experience across a user base Gamma describes as 50M+. She is the person at Gamma who publicly compares image models on prompt adherence, text rendering, photorealism and constraint-following — i.e. the one already thinking in evaluation rubrics for visual quality.
CEO; SaaStr AI 2026 talk on $100M ARR with ~50 people.
Runs Gamma's AI engineering team (~18 people, reporting to CEO Grant Lee) — the group that owns Gamma's model selection, prompting, fine-tuning of open models and eval frameworks for generated decks, sites and images. He is the technical requester for any design-quality data Gamma would buy, but he has almost no public footprint.
Head of Design — likely taste arbiter for generated layouts.
Framer
Website builder with AI site generation and Framer Agents (June 2026) working on the design canvas; founders frame 'taste' as the hard part of AI design. Potential buyer of web-design quality evals.
Lovable
Prompt-to-app builder (Stockholm) running on frontier models (Claude Opus, GPT-5.5, etc.). Buys/evaluates model quality for generated UI; a candidate buyer of UI/design-quality evals rather than training data.
Co-founder; manages ~124 per The Org.
Wrote 'We Gave Our Agent a Vent Tool' on agent self-improvement from production friction.
Owns data function; possible owner of eval data.
Head of Product (~10 reports); likely owns output-quality priorities.
Vercel (v0)
v0 generates UI/React apps via a composite pipeline (frontier LLM + RAG + custom RFT 'autofixer' models). Trains small models with RL; relevant buyer for UI-quality preference data.
NVIDIA (Nemotron / Cosmos)
Open-weights Nemotron LLMs and Cosmos world/video models; publishes its human preference datasets (HelpSteer) openly, making it a transparent, repeat human-data buyer - mostly text today, video via Cosmos.
The data executive behind NVIDIA's open models: she appears in the Nemotron 3 report under Leadership, Product, and Evaluation/Safety/Release, and her LinkedIn presents her as building the data that fuels NVIDIA's models. Her whole career is large-scale human data collection — she led Mozilla's Common Voice, one of the biggest crowdsourced speech datasets ever built — which makes her the rare buyer who understands vendor economics and data ops from the inside.
Runs post-training for NVIDIA's Nemotron models and is listed under Leadership and Post-training in the Nemotron 3 report. He is the senior author on HelpSteer3-Preference, the dataset NVIDIA built with more than 6,400 annotators across 77 countries sourced through Scale AI and Translated — which makes him one of the few frontier-scale buyers in this index whose human-data supply chain is publicly documented.
The person who actually designs NVIDIA's human preference datasets. He is lead author of HelpSteer, HelpSteer2 and HelpSteer3-Preference and co-author of SteerLM, and he writes the public documentation for them — the annotation scales, the qualification bars per domain, the vendor split between Scale AI and Translated. If Julian wants to know what a frontier buyer's spec looks like, or wants to be measured against one, this is the operational counterpart.
LinkedIn profile: "Head of Data Operations, Autonomous Vehicles" at NVIDIA, Los Angeles, California, United States, in role since 2020-02.
LinkedIn profile: "Technical Program Manager, Data Operations" at NVIDIA, Tomball, Texas, United States, in role since 2023-11.
Nemotron 3 Leadership list
Listed under Leadership and Data (and as Eileen Peters Long in Evaluation) in Nemotron 3
HelpSteer3 co-author and Nemotron 3 'Data' contributor
HelpSteer3 co-author; Nemotron 3 data and post-training
LinkedIn profile: "Data Operations Manager" at NVIDIA, United States, in role since 2020-01.
HelpSteer3 co-author; Nemotron 3 post-training
Nemotron 3 Leadership list
Group builds text2image/text2video/text2-3D foundation models for NVIDIA AI Foundry
LinkedIn profile: "AI Data Annotation Manager" at NVIDIA, Redmond, Washington, United States, in role since 2017-11.
HelpSteer3 co-author; Nemotron 3 post-training
LinkedIn profile: "Head of Data Operations - Nemotron, Robotics, AV" at NVIDIA, San Francisco, California, United States, in role since 2025-08.
Nemotron 3 Leadership and Data lists
Apple Foundation Models (AFM)
Builds on-device/server AFM models plus ADM 3 image model for Image Playground, Genmoji and Photos editing (WWDC 2026). Apple's design culture and UI-generation research make it a natural buyer of design/UI taste data.
A career researcher in automatic UI generation — his CMU thesis was literally about automatically generating high-quality user interfaces — now a research scientist in Apple's AI/ML group and a co-author on both of Apple's CHI 2026 UI papers. He also co-created RICO, the mobile UI dataset that much of the field's UI-understanding work was built on, which makes him one of the few people in this index with a track record of shipping the datasets others train on.
The senior author on Apple's designer-feedback UI-generation paper and, by the look of his publication record, the person who anchors Apple's Seattle-area HCI group where designer-in-the-loop model work happens. He is also a CHI 2026 subcommittee chair and an ICSE 2026 programme committee member, which makes him a well-connected node in the design-and-code research community rather than a procurement contact.
Co-author on designer-feedback UI-generation paper
Swearngin is the researcher who, with colleagues at Apple, collected about 1,500 design annotations from 21 professional designers — comments, sketches and direct edits rather than rankings — and used them to fine-tune UI-generation models that beat ranking-trained baselines and GPT-5 in human evaluation. That paper is the strongest published proof that Julian's product category works. She has since moved to Microsoft Research AI Frontiers, so she is no longer an Apple buyer.
Leads Apple Foundation Models, ML research and AI safety after Giannandrea retirement
Co-author on designer-feedback UI-generation paper
LinkedIn profile: "ML Data Program Manager [via Magnit]" at Apple, Fremont, California, United States, in role since 2026-02.
LinkedIn profile: "Head of Data Partnerships at Apple Music" at Apple, Los Angeles Metropolitan Area, in role since 2021-03.
LinkedIn profile: "Global Maps Data Partnerships" at Apple, London, England, United Kingdom, in role since 2022-03.
LinkedIn profile: "Engineering Program Manager, Machine Learning & Data Operations" at Apple, San Francisco Bay Area, in role since 2024-04.
LinkedIn profile: "AIML Data Ops - Site Project/Program Lead" at Apple, Australia, in role since 2026-03.
LinkedIn profile: "AIML Safety and Music Annotation Manager" at Apple, Austin, Texas Metropolitan Area, in role since 2025-05.
LinkedIn profile: "Senior Director, AIML Data Operations" at Apple, Cupertino, California, United States, in role since 2024-09.
LinkedIn profile: "AIML Annotations Account & Program Manager, Data Operations" at Apple, Greater Seattle Area, in role since 2024-01.
Named head of AFM team (successor to Ruoming Pang)
Mistral AI
Europe's frontier lab (open-weight LLMs, Pixtral vision, Voxtral speech, Le Chat, Forge). Raised €3B Series D (Sep 2026, >€21B valuation, Samsung-led). EU anchor buyer; Le Chat image generation is outsourced to Black Forest Labs FLUX.
Lample runs Mistral's entire Science organisation, which is where the company's in-house Human Data Annotation team and its data-annotation TPMs sit. He is therefore the executive owner of every human-data campaign at Europe's only frontier lab — the single most important EU-based sponsor for a German vendor, even though his own research output is text and multimodal rather than design.
A multimodal research scientist at Mistral and a core contributor on Magistral (reasoning/post-training) and Voxtral (audio), with a research history in grounded multimodal generation and in how models are evaluated. She is the kind of person who writes the spec for what human annotators are asked to judge, rather than the person who signs the contract.
First-listed core contributor on Magistral RL reasoning paper (Jun 2025)
Core contributor on Magistral (Jun 2025)
First core contributor on Ministral 3 report (Jan 2026) and first author of Voxtral (Jul 2025)
CEO; controls budget after €3B Series D (Sep 2026)
LinkedIn profile: "AI Data Partnerships" at Mistral, San Francisco, California, United States, in role since 2026-06.
LinkedIn profile: "Head of Data Strategy & Partnerships" at Mistral, France, in role since 2026-06.
Core contributor on Ministral 3 (Jan 2026) and Magistral (Jun 2025), author on Pixtral
LinkedIn post advertising Mistral's 'Data Annotation Technical Program Manager' role (~May 2024); current status unverified
Core contributor on Magistral (Jun 2025); named in Ministral 3 contributors (Jan 2026)
Core contributor on Magistral (Jun 2025); computer-vision background
Core contributor on Voxtral speech model (Jul 2025)
Core contributor on Ministral 3 report (Jan 2026)
Cohere
Enterprise/sovereign LLM company (Command A+, Command A Vision, Aya Vision, North). Acquired Germany's Aleph Alpha (Apr 2026) — a European foothold. Multimodal (document/vision) but enterprise-first; taste data demand mainly UI/document/visual-reasoning.
First author of Aya Vision, Cohere's multilingual multimodal model, and the person who built its evaluation apparatus — AyaVisionBench and m-WildVision, both of which score model outputs on open-ended visual prompts by comparison. He is Cohere's most identifiable technical requester for image-and-text human judgement, though the demand is multilingual/document understanding rather than design taste.
Senior author on Aya Vision (May 2025); Aya multilingual data programmes
Author on Aya Vision and Command A (2025); RLHF research
LinkedIn profile: "Data Partnerships Manager" at Cohere, Toronto, Ontario, Canada, in role since 2023-11.
Joined Aug 2025 as first Chief AI Officer leading research and product development; TIME100 AI 2026
LinkedIn profile: "Data Partnerships Manager" at Cohere, New York, New York, United States, in role since 2024-10.
Promoted to head Cohere Labs Sep 2025 after Sara Hooker left; multilingual data/eval research
Command A author (Apr 2025), cited in its human-annotation/adversarial data section (Bartolo et al.)
Chief Scientist per Wikipedia; author on Aya Vision (May 2025)
Second author of Aya Vision (May 2025)
Thinking Machines Lab
Mira Murati's lab (~200 staff): Tinker fine-tuning platform plus Inkling, a natively multimodal (text/image/audio/video) open-weight model (Jul 2026). Pitch is human-in-the-loop customisation — a natural buyer of expert preference data.
The inventor of PPO and one of the architects of RLHF and ChatGPT, now the last founding scientist running research at Thinking Machines Lab after a year of co-founder departures. He does not buy datasets, but he sets the lab's post-training direction — and in September 2026 he publicly framed the path to self-improving models as human feedback that lets models absorb researcher taste, which is the most quotable endorsement of taste data by anyone at this level.
Lead author of 'On-Policy Distillation' post-training blog (Oct 2025)
CEO; public 'human collaboration' thesis ('The Future Worth Building Is Human', Jul 2026)
Named CTO 14 Jan 2026 after Zoph's exit (ex-Meta FAIR/PyTorch)
Amazon AGI (Nova)
Amazon's foundation-model org (Nova family incl. Canvas image and Reel video). Jul 2026 pivot: Canvas/Reel/Omni to keep-the-lights-on, focus on a frontier model from Pieter Abbeel's FMR - image/video taste demand likely reduced.
LinkedIn profile: "Machine Learning Data Operations Manager - Information Data Operations, Amazon | Alexa " at Amazon, Santa Barbara, California, United States, in role since 2020-08.
LinkedIn profile: "Manager; Data Program, Information, Speech and Multimodal Services" at Amazon, San Francisco Bay Area, in role since 2025-08.
LinkedIn profile: "Senior Business Intelligence Engineer & Data Architect | Data Program Lead" at Amazon, Austin, Texas Metropolitan Area, in role since 2022-09.
Leads agent-model training research on AGI team (computer-use/UI agents)
LinkedIn profile: "OPMIII (Delivery & Program Leader) - AI Data Operations" at Amazon, India, in role since 2022-10.
LinkedIn profile: "Head of AI/ML Strategy, Generative AI & Data Strategy" at Amazon, France, in role since 2025-09.
LinkedIn profile: "Program Manager II, Global Data Operations - Worldwide Grocery Stores" at Amazon, Austin, Texas, United States, in role since 2026-09.
LinkedIn profile: "Head of Physical AI Data Collection & Operations - Frontier AI & Robotics (FAR)" at Amazon, San Francisco Bay Area, in role since 2026-09.
Named by Jassy to lead AI models + AGI org, reporting to CEO
Heads FMR, now Amazon's top AI priority; new frontier model planned for re:Invent 2026
LinkedIn profile: "Sr Technical Program Manager | Alexa Annotation Center of Excellence" at Amazon, Greater Seattle Area, in role since 2024-09.
LinkedIn profile: "Data Operations Program Manager" at Amazon, United States, in role since 2026-01.
LinkedIn profile: "Sr. Program Manager, AGI Data Quality" at Amazon, Boston, Massachusetts, United States, in role since 2025-10.
LinkedIn profile: "Manager - ML Data Operations" at Amazon, Chennai, Tamil Nadu, India, in role since 2021-06.
LinkedIn profile: "Manager II ML Data Operations" at Amazon, Hyderabad, Telangana, India, in role since 2022-12.
Reflection AI
Ex-DeepMind founders building 'America's open frontier lab' — open-weight frontier LLM/agents (unreleased as of Jun 2026), $25B valuation (Apr 2026), SpaceX compute deal. Text/agentic focus; image/design data demand not evident.
Safe Superintelligence Inc.
Ilya Sutskever's secretive lab (~50 staff mid-2025) pursuing 'straight-shot' superintelligence; no product or model released. Deep-pocketed (Nvidia $5B, Jul 2026) but opaque — low near-term fit for creative/taste data.
Abridge
Clinical ambient-documentation AI (270+ health systems) now training its own models on clinician edits. Expert (clinician) feedback buyer; no visual/design relevance.
No one here matches the filters.
Cognition (Devin)
Devin agent + Devin Desktop (ex-Windsurf); post-trains SWE-series coding models (SWE-2 on Kimi K3, Sep 2026). RL-environment buyer more than human-preference buyer.
Cursor (Anysphere)
AI code editor (>$1B ARR, Nov 2025) training its own Composer models via RL on Kimi K2.5. UI/front-end generation quality is a latent design-taste need, but data comes mostly from real user sessions.
Harvey
Legal AI platform now post-training its own models (Harvey Tenet, Aug 2026). Proven expert-data buyer (Mercor) — but legal text, not creative/design.
Perplexity
AI answer engine/browser (Comet) that post-trains Sonar models on open weights. Answer-quality evals are the data need; limited taste/design relevance.
Sierra
Customer-service agent platform (Bret Taylor, Clay Bavor). Research arm builds τ-bench agent evals; data needs are conversational/agentic, not visual.
ByteDance Seed
ByteDance's foundation-model org: Doubao LLM, Seedream (image), Seedance 2.0 (video, Apr 2026). Among the world's largest image/video-generation efforts — highly relevant demand for aesthetic/video preference data, but access is hard.
DeepSeek
High-Flyer-backed open-weight LLM lab (V3/R1, V4 Flash/Pro 2026). Text/reasoning-first; minimal external human-data buying evident. Context-only for taste data.
No one here matches the filters.
Moonshot AI (Kimi)
Kimi K2/K2.5 (multimodal, MoonViT, Jan 2026) and K3 (2.8T open weights, Jul 2026); ~300 staff, $35B valuation. Its models are used by US firms (Cursor, Cognition, TML bootstrapped Inkling post-training data from K2.5).
No one here matches the filters.
Qwen (Alibaba Tongyi Lab)
Alibaba's LLM/VLM/image-gen family (Qwen3, Qwen3-VL, Qwen-Image 2.0). Qwen-Image RL work (Jun 2026) makes it the most taste-relevant Chinese buyer for image preference data.
Zhipu AI (Z.ai)
Tsinghua spin-out, HK-listed Jan 2026; GLM-5.x LLMs, GLM-4.5V vision, CogView/CogVideo generation. Context-only buyer; multimodal RL work is design-adjacent.
No one here matches the filters.
Computational UI design & models of visual/aesthetic perception of layouts; ERC Advanced Grant 2024–29; CHI Academy 2025
ReproHum project & HEDS 3.0 Human Evaluation Datasheet (2024–25): reproducibility standards for human evaluation
'The Problem of Human Label Variation' (EMNLP 2022) — founding paper on treating annotator disagreement as signal, not noise
Design2Code (NAACL 2025): screenshot-to-webpage benchmark with human evaluation of visual fidelity
LAION-Aesthetics & improved CLIP+MLP aesthetic predictor (2022) — de facto aesthetic filter for Stable Diffusion-era data; audited in 'The Algorithmic Gaze' (2026)
First author GenAI-Arena (NeurIPS 2024) — arena-style human preference collection for generative vision models
PRISM Alignment Dataset (NeurIPS 2024 D&B best paper): participatory, individualised human feedback from 1.5K raters in 75 countries — subjective/pluralistic preference methodology
Senior author on HPSv2 (2023) and HPSv3 (ICCV 2025) human-preference benchmarks for text-to-image
Davidsonian Scene Graph (ICLR 2024, with Google): reliable fine-grained T2I evaluation validated against human judgements
UIClip (UIST 2024) UI design-quality model; 'Improving UI Generation Models from Designer Feedback' (Apple, 2025); DesignPref (Nov 2025)
Senior author UIClip (2024) and DesignPref (2025); long-running crowdsourcing & accessibility research
ImageReward (NeurIPS 2023) and VisionReward (Dec 2024): fine-grained multi-dimensional human preference reward models for image & video generation; also on CogVideoX
Co-lead of RichHF-18K (CVPR 2024 Best Paper) — fine-grained human feedback for T2I; Google team also works on perception/aesthetics modelling
Graphic design generation & evaluation: 'Can GPTs Evaluate Graphic Design Based on Design Principles?' (SIGGRAPH Asia 2024), OpenCOLE (CVPRW 2024), GDUG workshop organiser
CrowdTruth ('Truth is a Lie', 2015) and DICES (NeurIPS 2023 D&B): rater-diversity & 'beyond gold standard' evaluation; also Adversarial Nibbler (T2I safety crowdsourcing)
Toward Verifiable and Reproducible Human Evaluation for Text-to-Image Generation (CVPR 2023): standardised T2I human-eval protocol; showed automatic metrics diverge from humans
HEIM — Holistic Evaluation of Text-to-Image Models (NeurIPS 2023) with crowd human ratings on aesthetics, originality etc.; HELM
TASTE (arXiv 2605.20731, May 2026): designer-annotated 9-dimension preference dataset for AI-generated graphic design, built with Contra (supply side) — closest academic analogue to Miju's thesis
Senior author on RichHF (CVPR 2024 Best Paper); long track record on human visual attention/saliency on UIs and images
Senior author Q-Bench / Q-Align (ICML 2024): LMM visual scoring for quality & aesthetics (IQA/IAA/VQA)
GenAI-Arena (NeurIPS 2024 D&B): open arena collecting human votes on image/video generation & editing; ImagenHub, VideoScore
Human Preference Score v1/v2 (2023) and co-author of HPSv3 (ICCV 2025, with Mizzen AI): wide-spectrum T2I human preference dataset HPDv3
First author DesignPref (Nov 2025): 12K UI design comparisons by professional designers; shows personalised > aggregated preference models
Rich Human Feedback for Text-to-Image Generation (RichHF-18K, CVPR 2024 Best Paper): region-level implausibility/misalignment annotations beyond pairwise prefs
Pick-a-Pic + PickScore (NeurIPS 2023): open dataset of >1M real user preferences on text-to-image outputs
VQAScore & GenAI-Bench (2024): compositional text-to-visual benchmark with large human rating set
First author VBench (CVPR 2024), co-first author VBench-2.0 (2025)
VBench (CVPR 2024) and VBench-2.0 (2025): video-generation benchmark suites with human-preference alignment checks
Writes on art, perception and aesthetics of generative imagery (e.g. 'Can computers create art?', 2018; visual-indeterminacy papers) — theory side of taste
Diffusion-DPO (CVPR 2024): aligning diffusion models directly on Pick-a-Pic human preference pairs — key consumer of T2I preference data
Annotator disagreement / perspectivism & learning from disagreement (2021 survey); MilaNLP hosted Kirk
Senior author Design2Code (2025); broad human-centred NLP and human-eval work
First author Q-Align (ICML 2024) — aesthetic & quality scoring with LMMs; now inside a Chinese frontier lab (overlap with china cluster)
LAION-5B (NeurIPS 2022 D&B) co-author and open-data/scaling-law lead; German open-dataset voice
Co-author of RichHF (CVPR 2024 Best Paper); works on human-AI thought partnership and eliciting richer human feedback
Data Workers' Inquiry (2024–25); TIME100 AI 2025 — labour conditions & power in data annotation
RewardBench (2024), Tülu 3 post-training recipes, RLHF Book — public reference on preference-data pipelines & costs
Visual Genome crowdsourcing; human-centred multimodal data & evaluation (e.g. DOCCI / dense annotation work) at UW/AI2
Rico (UIST 2017): 66K-screen mobile UI dataset underlying most UI-understanding/generation work
Senior author 'Artificial Artificial Artificial Intelligence' (2023) — crowd workers widely use LLMs; data-quality threat to human data
Consent in Crisis (NeurIPS 2024) and Data Provenance audits — licensing/supply shock for training data, a driver of paid human data
AlpacaFarm (2023) & AlpacaEval: simulating/replacing human preference feedback — the 'cheap synthetic vs. expensive human' question
First author 'Artificial Artificial Artificial Intelligence' (2023) and 'Prevalence and Prevention of LLM Use in Crowd Work' (CACM 2025)
'En attendant les robots' (2019) and DiPLab studies of micro-work and AI data supply chains (Madagascar, Latin America)
Data Shapley (ICML 2019) — data valuation, i.e. pricing individual training examples
Fairwork cloudwork ratings of annotation platforms; 'Feeding the Machine' (2024)
'Ghost Work' (2019, with Siddharth Suri) — canonical account of on-demand data labour
Open Problems and Fundamental Limitations of RLHF (2023) — catalogue of human-feedback data-quality failure modes
Public professional sources, every entry linked
Model technical reports and their credits, papers, job posts, company blogs, vendor case studies, talks. Roles are checked against 2025–26 evidence because this market reshuffles fast; the evidence date sits on every line. The structure is in BUYERS.md.
Almost no lab names its head of human data
Budget owners are the hardest role to find in public — most rosters stop at research leads and open job posts. Closing that gap is the first job of pass 2.