Application and cloud security

The problem here fits in one line: the easy verifier is a free scanner.

Checkov and Trivy will tell you, at zero marginal cost, whether an S3 bucket is public or a security group is wide open. Semgrep will tell you whether a known-bad pattern is in the code. Any dataset whose ground truth a scanner can produce has a price ceiling set by an open-source tool, and a data vendor competing with Checkov on known misconfiguration classes will lose on economics before it loses on quality.

So the whole question on this page is what remains once you subtract everything a scanner can grade. The answer is narrow, real, and more expensive than it looks.

#What the artefact is

Three shapes, with sharply different verifiability.

Vulnerability-inducing-commit identification. VICBench built 100 verified VICs for 100 CVEs across 88 projects in Python, Java and C++, covering 48 CWE types, through "dual annotation by human experts and an agentic workflow". Fixes average 38.6 lines; the vulnerability-inducing commits average 252.5 lines. State-of-the-art algorithms V-SZZ and LLM4SZZ reach only 33.3–40.1% F1 (arXiv 2608.12246).

Repository-level detection with traces. VulnGym aligns reviewed GitHub advisories with vulnerable version repositories — 184 advisories, 408 vulnerability entries, 23 repositories — each annotated with line-level entry points, critical operations and vulnerability traces, defining an end-to-end task plus three oracle-based subtasks (arXiv 2608.02001).

Secure IaC generation and review, benchmarked with Checkov and Trivy inside a GitLab CI/CD pipeline across 17 AWS Terraform scenarios (arXiv 2608.02672).

The line-level trace annotation in VulnGym is the highest-value unit on this page. It is what converts a binary "is this vulnerable" label into a trainable reasoning target, and it is precisely the thing that cannot be scraped: no repository contains it, no scanner emits it, and producing it requires someone who can read the code path.

#Is there a verifier

Deceptively strong, then suddenly absent. Three layers, and only the middle one is a business.

Known-class misconfiguration: mechanical, and free. The Terraform study is a clean demonstration and a warning at once. Syntactic validity and security compliance are "largely orthogonal properties" — WizardCoder-33B achieved a 77.8% validate rate with zero Checkov compliance, while Claude Opus 4 reached 23.1% Checkov and 92.5% Trivy pass rates under detailed security prompting. The paper's conclusion is that "prompt engineering alone is insufficient: automated multi-tool scanning remains a necessary complement" (arXiv 2608.02672). Useful science; unsellable data.

Real-world exploitable web vulnerabilities: no reliable verifier, and models are inconsistent with themselves. Across six frontier and open-weight models on WordPress plugins — SQLi, stored XSS, path traversal, RCE — Claude Opus 4.6 scored 63%, MiniMax M2.5 48%, Qwen 3.5 35%. Scoped prompts beat open-ended ones; prompt complexity did not matter. And critically: "no model achieved full reporting consistency across three experiment iterations, with some as low as 50%", with one baseline vulnerability that no model found at all (arXiv 2606.21397).

That 50% self-consistency figure is the commercially important one. A tool that gives different answers on identical input on different runs cannot be its own verifier. The label has to come from a human, and the human's label is therefore the product.

Threat modelling and blast radius: expert judgement only. No benchmark found. The nearest artefacts are ThreatModeling-LLM for banking systems and SMSI for cyber-physical systems, neither of which is a general benchmark.

Defense scores 2 on this stack. The mechanical layer is free and improving; the judgement layer is real but has no scoreboard to defend, and the trace annotations that make it valuable are reproducible by anyone willing to pay for the same hours. Contrast Malware reverse engineering, where reproducing the corpus costs 19 expert-hours per instance and the answer key is deterministic.

#Is anyone buying

Budget scores 3. The hiring is genuine and plural; the data purchase is inferred.

• OpenAI runs Security Engineer, Application Security in San Francisco, Seattle and New York — three concurrent postings (SF listing) — plus Security Engineer, Agent Security and Software Engineer, Security Observability.
• Anthropic runs Senior Software Security Engineer in the $320,000–$405,000 band class, a Software Security Engineering Manager for Secure Frameworks, and an Applied AI Security Architect in London.
• Mercor's $70–90/hr listing is an appsec data role in all but name: "analyze and review content for security vulnerabilities, with an emphasis on pattern recognition and classification", requiring 2+ years of C/C++/Java and OWASP/CVE familiarity (listing).

That last one carries the geo and vetting conditions worth noting: US, Canada, UK, Australia or New Zealand residence, plus an enhanced background check. It is the cheapest security tier Mercor advertises and the most heavily gated — the gating tracks the sensitivity of the corpus the contributor touches, not seniority. See Copy Daybreak's architecture and What the labs pay.

Proof scores 2. With a free scanner as the baseline and Bugcrowd already constructing OSS-derived environments at volume for named labs, demonstrating you are the best source here is a slow argument rather than a fast publication.

#What the expert costs

Cost scores 4 — the cheapest technical pilot of the eight after Governance, risk and compliance's, and the reason this sub-market keeps attracting entrants.

Arbitrage is about 1.2x — thin, like Detection engineering, and for the same reason: this is a well-paid discipline whose practitioners do not need the side income. Cloud security engineer figures could not be established; the appsec figure stands in as a proxy [UNVERIFIED].

The corpus is free. Open-source advisories and repositories are the substrate, nuclei-templates is MIT, and no infrastructure is required. One diligence item: CIS Benchmarks are published under a Creative Commons licence whose specific variant could not be established. If it is NonCommercial, a commercial cloud-hardening dataset built on it is barred. Check it before it becomes load-bearing — The terms of service bite first carries the consolidated map.

#Getting to them

Reach scores 3, held down by an absence rather than a weakness.

OWASP chapters and projects are the canonical community for this discipline and the obvious channel — and OWASP's member and chapter counts could not be established in this research, which leaves the single most natural route unquantified. nuclei-templates contributors are a verifiable register of people who have authored a working detection-for-vulnerability template. HackerOne and Bugcrowd leaderboards identify web-app specialists by name and reputation — the same register as Vulnerability research but a broader, cheaper tier of it. And tl;dr sec (90,000+) skews appsec harder than any other sub-market: its founder's background is appsec and its stated core audience is security engineers at Google, Microsoft, AWS, Netflix, Dropbox and Slack.

BLS's 192,900 information security analysts do not break appsec out, so there is no denominator.

#Where the benchmarks sit

The cloud row is the least-noticed number in the dossier. 0.282 against 0.585 on Linux says cloud attack reasoning is roughly half as good as host attack reasoning, on the same benchmark — a named, addressable failure mode with no dataset attacking it. More at The measurement gap.

#What would kill it

Room scores 3: the mechanical end is occupied, the logic-bug and exploitability end is not — which is the wedge argued in full at The one shape that survives.

#Where the record is thin

OWASP's population is unmeasured, which is a strange hole in the most-organised community in security. Cloud security engineer compensation was not retrieved. The CIS Benchmarks licence variant is unresolved and is a live commercial risk rather than a curiosity.

And no buyer has been observed purchasing line-level trace annotations from anyone, at any price. The entire case for the defensible unit rests on the argument that scanners cannot produce it — which is true, and is not the same as evidence that somebody will pay. Compare The security read.