Miju Labs

← All buyers · Apple Foundation Models (AFM)

A

Amanda Swearngin

Principal Researcher, Microsoft Research (ex-Apple) · Apple Foundation Models (AFM)left

high confidenceTier B · Map and watchSecond-wave labsUI

Swearngin is the researcher who, with colleagues at Apple, collected about 1,500 design annotations from 21 professional designers — comments, sketches and direct edits rather than rankings — and used them to fine-tune UI-generation models that beat ranking-trained baselines and GPT-5 in human evaluation. That paper is the strongest published proof that Julian's product category works. She has since moved to Microsoft Research AI Frontiers, so she is no longer an Apple buyer.

Background

Three years as a software engineer in Microsoft's Dynamics group, then a PhD at the University of Washington advised by Amy Ko and James Fogarty on data-driven design, program analysis and constraint solving for UI design software, on an NSF Graduate Research Fellowship, with internships at Adobe Research, Google and Microsoft Research. She joined Apple in 2020 as a research engineer/scientist, where she worked on computational UI understanding — the Screen Recognition work behind VoiceOver (a CHI best paper), the Xcode Accessibility Inspector, and a UI-understanding framework used by more than twenty internal teams.

Her own site lists her as a Principal Researcher at Microsoft Research AI Frontiers from 2025, working on agentic systems and human-AI experiences; her Microsoft profile confirms the title and lab. The designer-feedback paper was Apple work, published at CHI 2026 after her move.

What they run now

Now at Microsoft: AI agents and human-AI interaction in the AI Frontiers lab, including recruiting research interns for agent work. The directly relevant work is the CHI 2026 paper with Jason Wu, Arun Krishna Vajjala, Alan Leung, Jeffrey Nichols and Titus Barik: rather than asking designers to rank generated UIs, they let designers critique them the way designers actually work — commenting, sketching over, and directly manipulating elements — and captured the rationale behind each critique. About 1,500 annotations from 21 designers were then used to fine-tune several LLMs, and human judges preferred the designer-aligned models over ranking-feedback baselines and over GPT-5.

What they buy

Historically she was a collector rather than a buyer: the 21 designers in the study were, per the roster's earlier evidence, internal Apple designers, and the annotation platform was built for the study. At Microsoft Research her budget will be small and her needs project-shaped — recruiting design professionals for studies, or collecting critique data for agent evaluation. Inference: any purchase would be a research-scale panel of design-literate annotators, not an industrial data contract.

Career

  1. 2025 – presentPrincipal Researcher, Microsoft Research (AI Frontiers) — Agentic systems and human-AI experiences.
  2. 2020 – 2025Research Engineer / Research Scientist, Apple — Computational UI understanding; Screen Recognition, Accessibility Inspector.
  3. 2015 – 2019Graduate Student Researcher, University of Washington
  4. 2010 – 2015SDE II / SDET, Dynamics, Microsoft
  5. 2015–2019University of Washington, Ph.D., Computer Science (Amy Ko, James Fogarty)

On the record

Why RLHF ranking fails for UI · 2026-02

Most LLMs cannot reliably generate well-designed UIs, and ranking-based feedback does not match how designers work. source

Feedback modality · 2026-02

Comments, sketches and direct manipulation capture design rationale that a preference label discards. source

Interviews & talks

Why it matters here

High as evidence, low as a sale. Her paper is the citation Julian should put in front of every other buyer in this index: it quantifies that designer critique beats preference ranking for UI quality, using a dataset a vendor could supply. She herself has moved out of the Apple foundation-models orbit and into agent research, so she is a source of methodology, a potential design partner and a reference — not a customer. Her move also means the Apple roster entry needs correcting.

How to reach

Approach as a peer on methodology. Speculation: what she would actually value is scale — she ran 21 designers and 1,500 annotations; a vendor offering a calibrated panel of several hundred designers with the same three feedback modalities and measured inter-rater agreement is a natural collaboration, and a co-authored or co-published follow-up is worth more to Julian than a small contract. Bluesky and her personal site are the reachable channels; CHI is the natural venue.

What we could not establish
  • Exact month she left Apple in 2025 and whether she retains any Apple affiliation on follow-up work.
  • Whether the 21 designers were all Apple employees, and whether Apple has since scaled the collection with external designers.
  • Whether her Microsoft work touches design/UI generation at all, or only agents.

Why they are on the map

Co-author: 'Improving UI Generation Models from Designer Feedback' (CHI 2026) - 21 designers, 1,500 design annotations source — evidence dated 2026-02

Shapes what 'good data' means.

How Apple Foundation Models (AFM) buys data

AFM 2025 report (Jul 2025): collected large text-image preference data to train a reward model; human graders disagree on 20-30% of pairs; 'internal and external' human eval, vendors unnamed. AFM v3 (Jun 2026) uses in-house human graders. Apple researchers (CHI 2026) collected 1,500 designer annotations from 21 internal designers to fine-tune UI-generation models - direct design-feedback data work.

Dec 2025: Giannandrea retired; Amar Subramanya (VP AI) under Craig Federighi runs AFM, ML research and safety; Zhifeng Chen leads the foundation-models team. Human grading appears in-house; HCI/UI research group in Seattle (Nichols, Barik, Swearngin).

Sources

  1. Personal site — Amanda Swearngin (Microsoft Research 2025–present; Apple 2020–2025)
  2. Microsoft Research — Amanda Swearngin, Principal Researcher, AI Frontiers
  3. Improving User Interface Generation Models from Designer Feedback (arXiv 2509.16779, CHI 2026)
  4. Apple ML Research — Designer feedback paper page