Validation bench — onboarding
Join the on-demand expert validation bench
Policy Window's validation bench is an on-demand roster of subject-matter experts who review AGI Social Scientist research-module outputs for validity-critical claims. This page describes how the bench works, what we ask of validators, and how to apply.
1 · What a validator does
For each commissioned research module, a validator reviews the AGI Social Scientist output for validity-critical issues that the AI cannot reliably check on its own:
- Claim-extraction fidelity — does the AI output accurately represent the underlying primary source? Cite-check sample claims; flag misrepresentations.
- Source-context check — was the regulator / court / institution quoted in context, or was a partial quote spliced misleadingly?
- Methodological review — does the AGI SS evidence-map inference chain make sense? Are alternative interpretations adequately considered?
- Expert interview — for modules with the Interview Add-On, conduct the consent-gated interview per charter §7.7 protocol (informed consent, declared retention, anonymisation by default).
- Qualitative validation — for case studies, validate the lived-experience framing against the source institution.
2 · What a validator does NOT do
- Author article prose. Catalog rows + research-module outputs are AI-drafted; the validator reviews and approves (or disputes) — never writes original prose under a Policy Window byline.
- Provide legal advice on behalf of Policy Window (charter §7.4). Validators with legal expertise are welcome but disclaim the legal-advice frame in their validation rationale.
- Lobby on behalf of the sponsor. Validators are sourced independently of the commissioning sponsor; they may not accept side commissions tied to their PW validation work for 6 months after the engagement (charter §4.5).
- Take findings positions.The validator's role is to flag what's wrong with an AGI SS output, not to advocate for what the output should conclude.
3 · What we ask
- Conflict-of-interest disclosure. Before an engagement starts, declare any funding, employment, board, or advisory relationship with the sponsor, the subject organisation, or competing providers. We rotate validators when COI risk is high.
- Written protocol acceptance. Sign the per-engagement protocol (consent + retention + anonymisation defaults per charter §7.7).
- Turnaround commitment. Typical engagement window is 2-3 weeks; validators commit to a fixed timeline at engagement start.
- Written rationale. Every validation decision is a ValidationRecord row with a written rationale visible to readers on the published output.
4 · Compensation
Validators are paid per engagement at a fixed rate scaled to the module size + validator credentials. Indicative rates (subject to commission scope):
- Evidence-pack review (3-5 hours): paid as a fixed honorarium per pack
- Expert interview + write-up (4-6 hours): paid as a fixed honorarium per interview + transcription cost
- Implementation case-study validation (8-12 hours): paid as a fixed honorarium per case study
- Research-sprint validation lead (20-40 hours over 3-4 weeks): paid as fixed-scope contract
Specific rates set at contract time, transparent to the validator and the sponsor. We do not negotiate rates downward based on validator capacity to negotiate; we publish the rate card to the validator before contract signing.
5 · How to apply
Email hello@policywindow.org with subject [validation-bench-application] and the following:
- Your domain (e.g. EU AI Act implementation, US algorithmic accountability, frontier AI safety evaluation, AI procurement compliance, etc.)
- Recent peer-reviewed publication or equivalent evidence of subject-matter expertise (URL is fine)
- Conflict-of-interest baseline — current employer + any commercial relationships with AI labs, regulators, or trade associations
- Availability — typical hours per month you could commit + timezone
- One module-type preference— which of the four engagement types (evidence-pack review, expert interview, case-study validation, sprint lead) you'd prefer to start with
We acknowledge within 5 business days; first engagement offer comes when a paying customer in your domain commits. You can withdraw from the shortlist at any time.