{"$schema":"https://policywindow.org/critique/api/schema","critique_id":"CRIT-000010","slug":"refusal-as-silence-gendered-disparities-in-vision","url":"https://policywindow.org/critique/c/refusal-as-silence-gendered-disparities-in-vision","doi":null,"status":"published","critique_type":"editorially_approved_ai_native_critique","publication_date":"2026-06-15","current_version":"1.0","target_paper":{"title":"Refusal as silence: Gendered disparities in Vision-Language Model responses","authors":["Sha Luo","S Kim","Zening Duan","Kaiping Chen"],"journal":"New Media & Society","doi":"10.1177/14614448261441886","url":"https://doi.org/10.1177/14614448261441886","publicationDate":"2026-05-04","paperType":"empirical","accessBasis":"abstract_only","fullTextUsed":false,"fictional":false,"doi_url":"https://doi.org/10.1177/14614448261441886"},"source_journal":{"tier":"A","rankingSources":["https://doi.org/10.1177/14614448261441886","https://openalex.org/W7160318704"],"rankingNote":"New Media & Society is a top-tier communication and media-studies journal. Tier A."},"selection":{"aiAgiCentralityScore":5,"societalRelevanceScore":4,"aiAgiCategories":["inequality_bias_fairness","human_AI_interaction"],"selectionReason":"A counterfactual audit of how a vision-language model's refusals vary by user gender identity speaks directly to algorithmic-fairness debates, making the robustness and scope of the disparity claim worth scrutiny."},"scores":{"aiAgiContribution":4,"evidentiarySupport":4,"methodologicalRisk":2,"overclaiming":2,"reproducibilityOrAuditability":2,"societalImpactRelevance":4,"severity":"low","confidence":"medium"},"severity_cap_for_access_basis":"moderate","plain_language_summary":"This study asks whether an AI image-and-text model (GPT-4V) refuses requests more often depending on the gender identity the user presents. Using a clever 'counterfactual persona' design — same image, same task, only the stated identity changes — the authors find that transgender and non-binary personas hit significantly higher refusal rates, even for harmless requests. It is a careful identity-audit design and a genuinely useful warning about using AI tools to code content. Our cautions, from the abstract, are about scope and reproducibility: it is one model on one task (binary gender classification), so the leap to AI-refusal behaviour in general is partial; and because large models are updated over time and can give different answers to the same prompt, refusal-rate findings like these can be hard to reproduce unless the run protocol (model version, repetitions) is fixed — something the abstract does not describe.","claims":[{"id":"C1","text":"A vision-language model refuses harmless requests more often for transgender and non-binary personas.","type":"causal","evidenceOffered":"A counterfactual persona design holding the task and image constant finds that \"transgender and non-binary personas experience significantly higher refusal rates, even in non-harmful contexts\".","support":"moderate","overclaiming":"minor","assessment":"The counterfactual design (varying only stated identity) is well suited to isolating an identity effect, and the harmless-context finding is striking. The main uncertainty the abstract leaves open is reproducibility, since model outputs are non-deterministic and version-dependent.","mainWeakness":"Refusal rates from a single proprietary model at one point in time may not reproduce across model versions or repeated runs; the abstract does not state the run protocol.","confidence":"medium"},{"id":"C2","text":"The result generalises to AI refusal behaviour and algorithmic fairness broadly.","type":"descriptive","evidenceOffered":"The study is scoped to one model and task — \"Focusing on a Vision-Language Model (GPT-4V)\" — yet frames a broad caution to \"caution against uncritical use of artificial intelligence systems for content coding\".","support":"moderate","overclaiming":"minor","assessment":"The methodological caution about equity audits is well taken and transferable. The specific disparity finding, though, is one model and one binary-classification task; other models, tasks and safety configurations may refuse differently.","mainWeakness":"Single-model, single-task evidence limits how far the specific disparity magnitude generalises.","confidence":"medium"}],"sections":[{"id":"what","title":"What the paper does","body":"Using a counterfactual persona design on GPT-4V (same image and task, varying only the stated gender identity), the study finds transgender and non-binary personas face significantly higher refusal rates even in non-harmful contexts, and draws methodological implications for equity audits."},{"id":"repro","title":"Reproducibility and scope","body":"Two abstract-level cautions. Large models are non-deterministic and frequently updated, so a refusal-rate disparity measured at one time on one model version can be hard to reproduce unless repetitions and the exact model build are pinned — the abstract does not describe this. And the finding is one model on one binary-classification task, so the specific magnitude may not transfer to other systems or tasks, even as the methodological warning does."}],"strongest_critique":"The disparity finding rests on a single proprietary, non-deterministic, version-dependent model on one classification task, so both its reproducibility and its generalisation beyond GPT-4V are uncertain on the evidence the abstract presents — even though the counterfactual design and the audit-methodology warning are sound.","strongest_fair_defence":"The counterfactual persona design is exactly the right tool for isolating an identity effect, holding image and task constant; the transgender/non-binary refusal disparity in explicitly non-harmful contexts is a clear, falsifiable result with real fairness stakes, and the authors frame it as a caution about audit practice rather than a universal law.","final_judgment":"A well-designed identity audit with a striking, policy-relevant finding; the cautions, visible from the abstract, are reproducibility (a non-deterministic, version-dependent model with no stated run protocol) and single-model/single-task scope. Severity low.","review_process":{"aiAgentsUsed":["claim_extraction","ai_agi_relevance","overclaiming","adversarial","author_defence","citation_integrity","legal_risk","plain_language","meta_review"],"reviewRounds":1,"humanEditor":{"name":"Founding editorial review (Policy Window)","role":"Editor-in-chief (founding)","approvalDate":"2026-06-15","declaredConflict":"none"},"expertCertification":{"used":false}},"author_response":{"notified":false,"status":"not_yet_invited","editorialActionAfterResponse":"Founding pilot: authors will be invited to reply once the standing board is ratified; this critique addresses claims, framing and generalisation only, never the authors."},"versions":[{"version":"1.0","date":"2026-06-15","note":"Initial publication.","changeType":"initial"}],"transparency":{"modelCardUrl":"/critique/model-card","publicAuditSummary":"Abstract-only critique: the target's abstract was reconstructed from the OpenAlex record and every verbatim span the critique relies on was checked to be an exact substring of it. The bibliographic record (DOI) was independently confirmed via Crossref. Severity is capped to the abstract-only access basis; the critique engages the paper's framing and stated claims only. Characterization was drafted under the journal's faithfulness discipline (represent the paper accurately; no manufactured flaws).","privateAuditRecordExists":true,"citationVerification":{"status":"complete","checkedSources":[{"label":"DOI 10.1177/14614448261441886","url":"https://doi.org/10.1177/14614448261441886","verified":true},{"label":"OpenAlex work record (abstract source)","url":"https://openalex.org/W7160318704","verified":true}],"fabricatedCitations":0},"riskReview":{"copyright":"completed","defamation":"completed","note":"Abstract quoted sparingly under criticism/review. Critique targets the paper's claims, framing and generalisation only — never the authors."}}}