{"$schema":"https://policywindow.org/critique/api/schema","critique_id":"CRIT-000046","slug":"perceived-human-vs-ai-empathy","url":"https://policywindow.org/critique/c/perceived-human-vs-ai-empathy","doi":null,"status":"published","critique_type":"editorially_approved_ai_native_critique","publication_date":"2026-07-05","current_version":"1.0","target_paper":{"title":"Comparing the value of perceived human versus AI-generated empathy","authors":["Matan Rubin","Joanna Z. Li","Federico Zimmerman","Desmond C. Ong","Amit Goldenberg","Anat Perry"],"journal":"Nature Human Behaviour","doi":"10.1038/s41562-025-02247-w","url":"https://doi.org/10.1038/s41562-025-02247-w","publicationDate":"2025","paperType":"empirical","accessBasis":"licensed_access","fullTextUsed":true,"fictional":false,"doi_url":"https://doi.org/10.1038/s41562-025-02247-w"},"source_journal":{"tier":"A","rankingSources":["resolved from the monitored-venue determination"],"rankingNote":"A-tier per the monitored-venue determination; Nature Human Behaviour is elite interdisciplinary (Nature family), WoS JCR Q1. Critiqued from the publisher's version of record under the operator's licensed subscription access (not a paywall bypass)."},"selection_provenance":{"id":"perceived-human-vs-ai-empathy","venue":"Nature Human Behaviour","inMonitoredSet":true,"determinedTier":"A","recordedTier":"A","effectiveTier":"A","kind":"monitored","disclosed":true,"offListPeerReviewed":false},"selection":{"aiAgiCentralityScore":4,"societalRelevanceScore":4,"aiAgiCategories":["human_AI_interaction"],"selectionReason":"Autonomous production cycle (licensed inbox, A-tier priority); full-text critique via two-stage produce+sharpen + 3-lens convergence gate (3 survives, 0 weakened).","domain":"psychology"},"scores":{"aiAgiContribution":4,"evidentiarySupport":4,"methodologicalRisk":2,"overclaiming":3,"reproducibilityOrAuditability":3,"societalImpactRelevance":4,"severity":"moderate","confidence":"high"},"severity_cap_for_access_basis":"high","plain_language_summary":"Across nine preregistered studies (n = 6,282), this paper investigates whether empathic responses are perceived differently when attributed to AI versus humans. All participants received AI-generated empathic text, but were randomly told it came from either an AI or a human partner. Human-attributed responses were rated as more empathic, more supportive, and elicited more positive emotions. The paper's main theoretical advance is that this gap is driven specifically by the affective ('feeling with') and motivational ('caring') dimensions of empathy, while cognitive empathy ('understanding') shows little difference. Participants also preferred to wait for a human response rather than receive an immediate AI one. The central critique is that the paper's tripartite distinction — the claim that affective and motivational empathy drive the gap while cognitive empathy does not — rests on a novel self-constructed 12-item questionnaire whose confirmatory factor analysis yields an RMSEA of 0.11, well above conventional thresholds for acceptable fit (0.06–0.08). This psychometric weakness undermines confidence that the instrument validly separates the three empathy constructs, leaving the differential finding insufficiently grounded. A secondary concern is that the paper's conclusions move from descriptive labelling effects to an unsupported ontological claim that the differences 'stem from humans' unique potential to feel and care, which AI currently does not possess' — a philosophical position the study's between-subjects labelling design cannot establish.","claims":[{"id":"CLAIM-001","text":"The paper's central theoretical contribution — that the human-AI empathy gap is driven specifically by affective and motivational rather than cognitive empathy — rests on a novel self-constructed 12-item questionnaire whose CFA yields RMSEA = 0.11, well above conventional thresholds for acceptable fit.","type":"methodological","evidenceOffered":"showed an acceptable fit over a single-factor model (root mean square error of approximation (RMSEA), 0.11","support":"moderate","overclaiming":"moderate","assessment":"The paper describes the three-factor CFA as showing 'acceptable fit' with RMSEA = 0.11 and CFI = 0.91. By standard criteria (Hu & Bentler, 1999: RMSEA at most 0.08 for adequate, at most 0.06 for good; CFI at least 0.95 for good fit), the model fit is poor. The CFI of 0.91 is at the floor of lenient acceptability and below the 0.95 threshold. The paper's main advance over prior work is the differential-dimension claim, which depends on the instrument cleanly separating cognitive, affective, and motivational empathy. With poor discriminant validity evidence, observed differences across subscales may reflect measurement artifact rather than genuine construct separation.","mainWeakness":"The three-factor measurement model does not fit the data well enough to support confident separation of the three empathy constructs, undermining the differential findings.","confidence":"high","anticipatedRebuttal":"The authors could point to Study 3's experimental prompt-manipulation as convergent validity: when the AI was prompted to emphasise a specific empathy component, participants rated that component highest, confirming discriminant sensitivity. Additionally, the CFI of 0.91 meets some lenient thresholds, and the three-factor model fits better than a single-factor alternative.","response":"Study 3's prompt-manipulation is a construct-response correspondence test — it shows the items react to prompts designed to foreground each dimension, but it does not resolve whether the items reliably separate the latent constructs in the primary between-subjects design where the same holistic prompt is used. The CFA fit indices (RMSEA 0.11, CFI 0.91) are the direct test of whether the measurement model adequately captures the tripartite structure, and they fall outside standard bounds. Relative fit over a single-factor model shows only that three factors are better than one, not that the fit is adequate in absolute terms.","falsificationCondition":"An independent psychometric validation of the 12-item questionnaire — CFA with RMSEA below 0.08 and CFI above 0.95 in an independent sample — would retire this objection and confirm the instrument's construct separation."},{"id":"CLAIM-002","text":"The paper's conclusions move from a descriptive labelling effect to the ontological claim that these differences 'stem from humans' unique potential to feel and care, which AI currently does not possess,' which the study's between-subjects labelling design cannot establish.","type":"causal","evidenceOffered":"unique potential to feel and care, which AI currently does not possess","support":"weak","overclaiming":"major","assessment":"The experimental design manipulates labels on identical AI-generated text. All observed rating differences therefore reflect how humans respond to labels, not anything about AI's actual emotional capacities. The conclusion that differences 'stem from' AI's inability to feel conflates what the data show (a perception/labelling effect) with what the authors believe philosophically (an ontological claim about AI phenomenology). The data are equally consistent with in-group favouritism, social-desirability effects, or mere prejudice against AI sources — none of which require the premise that AI 'does not possess' the capacity to feel.","mainWeakness":"An inferential leap from self-report ratings under a label manipulation to claims about AI's inner states, which the design cannot adjudicate.","confidence":"high","anticipatedRebuttal":"The authors could argue that current LLMs are widely agreed to lack subjective experience (they cite Refs 48–54), and that their framing reflects the scientific consensus rather than an empirical claim their design must support.","response":"Whether AI truly feels is an open philosophical question on which reasonable experts disagree. Even granting the authors' position, the study's data cannot be adduced as evidence for it — the data show a labelling effect on human perception, not a window into AI phenomenology. The sentence 'these differences stem from humans' unique potential to feel and care' asserts a causal origin for the observed gap that the design was not built to test. The hedge 'currently' partially mitigates but does not eliminate the gap between what was measured and what is concluded.","falsificationCondition":"A study that manipulates participants' beliefs about whether AI can genuinely feel (rather than just the source label) and shows this belief mediates the empathy-gap effect would provide the missing link between perception and the ontological claim."},{"id":"CLAIM-003","text":"Across the nine studies, attention-check failure rates are remarkably high (23–29%), yet no analysis examines whether exclusion rates differed by experimental condition or whether excluded participants differed systematically from retained ones.","type":"descriptive","evidenceOffered":"failed attention checks (n = 263) or reported technical issues (n = 10),","support":"moderate","overclaiming":"minor","assessment":"Study 1a loses 263 of 998 participants (~26%) to attention checks. Similar rates appear across studies (1b: ~23%, 1c: ~25%, 1d: ~28%, 2a: ~29%, 2b: ~27%, 3: ~23%). If AI-condition participants were more likely to disengage and fail attention checks — a plausible differential-attrition mechanism given the nature of the manipulation — the retained sample would be differentially motivated across conditions, potentially inflating the labelling effect. The paper transparently reports exclusion counts but does not analyse whether attrition was condition-dependent.","mainWeakness":"High exclusion rates with no analysis of differential attrition by condition could bias estimated labelling effects.","confidence":"medium","anticipatedRebuttal":"The authors preregistered the attention-check exclusion criteria, and all studies use random assignment, so baseline characteristics should be balanced. High exclusion rates in online studies are common and do not by themselves indicate bias.","response":"Random assignment balances baseline characteristics, but post-randomisation differential attrition can reintroduce systematic differences. The concern is not that exclusions are high per se, but that they may be condition-dependent — something a simple condition-by-attrition chi-square test could check. Its absence leaves the question open.","falsificationCondition":"Reporting that attention-check failure rates did not differ significantly between human-label and AI-label conditions across the studies would directly address this concern."}],"sections":[],"strongest_critique":"The paper's central theoretical contribution — that the human-AI empathy gap is driven specifically by affective and motivational rather than cognitive empathy — rests on a novel self-constructed 12-item questionnaire whose confirmatory factor analysis yields an RMSEA of 0.11, well above conventional thresholds for acceptable model fit (at most 0.08 for adequate, at most 0.06 for good). The CFI of 0.91 is at the floor of lenient acceptability and below the 0.95 standard. This poor fit raises serious doubts about whether the instrument validly separates the three empathy constructs — without that separation, the differential findings that constitute the paper's main advance over prior human-preference research are insufficiently grounded.","strongest_fair_defence":"The paper is methodologically ambitious and admirably transparent. Nine preregistered studies with a combined n of 6,282 provide substantial statistical power. The authors systematically address alternative explanations (halo effects via studies 2a/2b, model dependence via study 1c with Llama 3.1, single-exchange limitation via study 1d with iterative exchanges). They share data and analysis code on OSF and document deviations from preregistration. The limitations section is forthright about the short-interaction scope, the absence of actual human-response comparisons, and the online-sample constraint. Study 3's prompt manipulation provides convergent validity for the three-construct distinction, partially offsetting the CFA concern. The basic descriptive finding — that labelling identical empathic responses as human versus AI shifts ratings — is robust and well-replicated. Effect sizes are reported throughout with confidence intervals.","final_judgment":"This is a well-powered, preregistered, multi-study investigation of an important and timely question. Its core descriptive finding — that labelling identical empathic responses as human versus AI shifts self-reported empathy ratings — is robust and well-replicated internally. However, the paper's main theoretical advance (the differential role of affective/motivational versus cognitive empathy) depends on a measurement instrument with poor CFA fit, and the conclusions section makes an unsupported inferential leap from perceptual labelling effects to claims about AI's capacity to feel and care. These are bounded overclaims on an otherwise sound body of work.","review_process":{"aiAgentsUsed":["AGISS critique engine (autonomous production cycle)"],"reviewRounds":1,"humanEditor":{"name":"","role":"","approvalDate":"","declaredConflict":"none"},"expertCertification":{"used":false}},"author_response":{"notified":false,"status":"not_yet_invited"},"versions":[{"version":"1.0","date":"2026-07-05","note":"","changeType":"initial"}],"transparency":{"modelCardUrl":"/critique/model-card","publicAuditSummary":"Critique produced by the autonomous production cycle (two-stage produce+sharpen + 3-lens convergence gate, 3 survives / 0 weakened) and auto-published under the operator's auto-publish + post-audit model; the Mon/Thu audit is the post-hoc gate.","privateAuditRecordExists":true,"citationVerification":{"status":"complete","checkedSources":[],"fabricatedCitations":0},"riskReview":{"copyright":"completed","defamation":"completed","note":"Nature Human Behaviour (Springer Nature) quoted sparingly under criticism/review; critique targets claims, methods and inference only. Full text accessed under the operator's licensed subscription (never a paywall bypass)."}}}