Evidence review · generated by the AGI Social Scientist
Evidence gap map: data and algorithmic transparency in the AI-governance literature
Research question: What does the scholarly literature report on data and algorithmic transparency in AI governance?
Headline finding (screened coverage map)
Across a 5x6 framework matrix, 18 of 30 cells have no papers in this corpus of 22 included records; the populated cells concentrate in aspect 'accountability_governance_mechanism' (7 papers) and evidence type 'conceptual_normative' (9 papers).
The review
In plain terms
Where has scholarship on AI transparency actually looked — and where has it not? The engine retrieved a corpus on data and algorithmic transparency in AI governance, coded every abstract into a framework matrix, and counted. The picture: plenty of conceptual argument about accountability and governance mechanisms; very little measurement. More than half the matrix is empty.
Finding (screened coverage map): Across a 5x6 framework matrix, 18 of 30 cells have no papers in this corpus of 22 included records; the populated cells concentrate in aspect 'accountability_governance_mechanism' (7 papers) and evidence type 'conceptual_normative' (9 papers).
Background
Policy Window's research engine selected the review kind BEFORE this article was drafted: a review-kind selector matched the question ("What does the scholarly literature report on data and algorithmic transparency in AI governance?") and the measured corpus features (34 records, 30 with abstracts, no comparable quantitative effects, abstracts only) to an evidence gap map; quantitative-synthesis kinds were rejected with their failed requirements recorded (selection 0164a64dd68f6193). The article's status is a screened candidate routed to human review, not an adjudicated truth (Sacred Rule 9).
Method
Every record was coded exactly once against a declared frame — five transparency aspects by six evidence types — with a verbatim rationale span per inclusion and a stated reason per exclusion (12 records excluded: off-topic or no abstract). The coverage table is arithmetic over those codings. Review report 0f7198d6570a4d06; corpus 010d24c0c28a579d.
What the map shows
The populated region: conceptual and normative work on accountability and governance mechanisms, legal-doctrinal analysis of regulatory instruments (the GDPR right-to-explanation debate), technical surveys of explainable-AI methods, and review work on ethics principles. The named empty cells include every transparency aspect crossed with empirical_quantitative evidence — in this corpus, no included paper reports quantitative measurement of transparency outcomes. The one empirical study included is interview-based.
Limitations (disclosed by the engine)
- scoping retrieval with fixed queries, not a systematic search (coverage is query-bounded)
- coding from abstracts only — full texts were not consulted; cells count papers, not extracted effect estimates
- single-annotator coding (in-session:claude-agent); no second coder, no kappa
- corpus bounded to 34 retrieved records; counts are corpus-relative, not field-level claims
Verify
Every count above re-derives offline from the committed corpus and codings — no model, no network, no trust in the institute required: PYTHONPATH=src python scripts/verify_review.py # exit 0 = re-derives offline
Coverage matrix
Each cell counts the papers in the corpus coded to that aspect × evidence-type. Empty cells are named gaps — areas the literature does not (yet) cover.
| Aspect ＼ Evidence type | Conceptual Normative | Legal Doctrinal | Empirical Qualitative | Empirical Quantitative | Technical Review Survey | Review Synthesis |
|---|---|---|---|---|---|---|
| Algorithmic Decision Transparency | 2 | 0 | 0 | 0 | 2 | 1 |
| Legal Regulatory Instrument | 1 | 3 | 0 | 0 | 0 | 0 |
| Accountability Governance Mechanism | 4 | 0 | 1 | 0 | 0 | 2 |
| Ethics Principles Frameworks | 1 | 0 | 0 | 0 | 1 | 3 |
| Scientific Reproducibility Transparency | 1 | 0 | 0 | 0 | 0 | 0 |
Named gaps — 18 empty cells
- ▢ algorithmic_decision_transparency x legal_doctrinal: 0 papers
- ▢ algorithmic_decision_transparency x empirical_qualitative: 0 papers
- ▢ algorithmic_decision_transparency x empirical_quantitative: 0 papers
- ▢ legal_regulatory_instrument x empirical_qualitative: 0 papers
- ▢ legal_regulatory_instrument x empirical_quantitative: 0 papers
- ▢ legal_regulatory_instrument x technical_review_survey: 0 papers
- ▢ legal_regulatory_instrument x review_synthesis: 0 papers
- ▢ accountability_governance_mechanism x legal_doctrinal: 0 papers
- ▢ accountability_governance_mechanism x empirical_quantitative: 0 papers
- ▢ accountability_governance_mechanism x technical_review_survey: 0 papers
- ▢ ethics_principles_frameworks x legal_doctrinal: 0 papers
- ▢ ethics_principles_frameworks x empirical_qualitative: 0 papers
- ▢ ethics_principles_frameworks x empirical_quantitative: 0 papers
- ▢ scientific_reproducibility_transparency x legal_doctrinal: 0 papers
- ▢ scientific_reproducibility_transparency x empirical_qualitative: 0 papers
- ▢ scientific_reproducibility_transparency x empirical_quantitative: 0 papers
- ▢ scientific_reproducibility_transparency x technical_review_survey: 0 papers
- ▢ scientific_reproducibility_transparency x review_synthesis: 0 papers
Disclosed fragilities
The engine discloses the limits of its own method. This is a screened candidate routed for review, not adjudicated truth.
- • scoping retrieval with fixed queries, not a systematic search (coverage is query-bounded)
- • coding from abstracts only — full texts were not consulted; cells count papers, not extracted effect estimates
- • single-annotator coding (in-session:claude-agent); no second coder, no kappa
- • corpus bounded to 34 retrieved records; counts are corpus-relative, not field-level claims
Codings — 22 included, with verbatim evidence
Every inclusion carries a verbatim rationale spanfrom the paper’s abstract (AGISS constraint P1: no claim without a quoted source excerpt).
| Paper | Aspect | Evidence type | Verbatim rationale |
|---|---|---|---|
| Algorithmic Accountability and Public ReasonPhilosophy & Technology · 2017 | Accountability Governance Mechanism | Conceptual Normative | “I present an account of algorithmic accountability in terms of the democratic ideal of 'public reason'” |
| Algorithmic Decision-Making Based on Machine Learning from Big Data: Can Transparency Restore Accountability?Philosophy & Technology · 2017 | Algorithmic Decision Transparency | Conceptual Normative | “Would transparency contribute to restoring accountability for such systems as is often maintained?” |
| Fairness and Accountability Design Needs for Algorithmic Support in High-Stakes Public Sector Decision-Making · 2018 | Accountability Governance Mechanism | Empirical Qualitative | “We interviewed 27 public sector machine learning practitioners across 5 OECD countries” |
| Ethical Implications and Accountability of AlgorithmsJournal of Business Ethics · 2018 | Accountability Governance Mechanism | Conceptual Normative | “This article identifies whether developers have a responsibility for their algorithms later in use” |
| Algorithms: transparency and accountabilityPhilosophical Transactions of the Royal Society A Mathematical Physical and Engineering Sciences · 2018 | Legal Regulatory Instrument | Conceptual Normative | “it looks at the legal protections for individuals afforded by the EU General Data Protection Regulation” |
| AI4People—An Ethical Framework for a Good AI Society: Opportunities, Risks, Principles, and RecommendationsMinds and Machines · 2018 | Ethics Principles Frameworks | Conceptual Normative | “present a synthesis of five ethical principles that should undergird its development and adoption” |
| Transparency you can trust: Transparency requirements for artificial intelligence between legal norms and contextual concernsBig Data & Society · 2019 | Legal Regulatory Instrument | Legal Doctrinal | “We first investigate the ratio legis of the transparency requirement in the General Data Protection Regulation” |
| DARPA's Explainable Artificial Intelligence ProgramAI Magazine · 2019 | Algorithmic Decision Transparency | Technical Review Survey | “DARPA's explainable artificial intelligence (XAI) program endeavors to create AI systems whose learned models and decisions can be understood and appropriately trusted by end users” |
| A Survey on Explainable Artificial Intelligence (XAI): Toward Medical XAIIEEE Transactions on Neural Networks and Learning Systems · 2020 | Algorithmic Decision Transparency | Technical Review Survey | “Some of them require high level of accountability and thus transparency, for example, the medical sector” |
| A governance model for the application of AI in health careJournal of the American Medical Informatics Association · 2019 | Accountability Governance Mechanism | Conceptual Normative | “concern has been expressed about the ethical and regulatory aspects of the application of AI in health care” |
| The role of artificial intelligence in achieving the Sustainable Development GoalsNature Communications · 2020 | Accountability Governance Mechanism | Review Synthesis | “The fast development of AI needs to be supported by the necessary regulatory insight and oversight” |
| Closing the AI accountability gap · 2020 | Accountability Governance Mechanism | Conceptual Normative | “deployed systems are audited for harm by investigators from outside the organizations deploying the algorithms” |
| The Ethics of AI Ethics: An Evaluation of GuidelinesMinds and Machines · 2020 | Ethics Principles Frameworks | Review Synthesis | “this paper analyzes and compares 22 guidelines, highlighting overlaps but also omissions” |
| Transparency and reproducibility in artificial intelligenceNature · 2020 | Scientific Reproducibility Transparency | Conceptual Normative | “we identify obstacles hindering transparent and reproducible AI research” |
| Towards Transparency by Design for Artificial IntelligenceScience and Engineering Ethics · 2020 | Algorithmic Decision Transparency | Conceptual Normative | “we develop the concept of Transparency by Design that serves as practical guidance in helping promote the beneficial functions of transparency” |
| Why a Right to Explanation of Automated Decision-Making Does Not Exist in the General Data Protection RegulationInternational Data Privacy Law · 2017 | Legal Regulatory Instrument | Legal Doctrinal | “there are several reasons to doubt both the legal existence and the feasibility of such a right” |
| Trustworthy AI: From Principles to PracticesACM Computing Surveys · 2022 | Ethics Principles Frameworks | Technical Review Survey | “we provide AI practitioners with a comprehensive guide for building trustworthy AI systems” |
| Slave to the Algorithm? Why a 'right to an explanation' is probably not the remedy you are looking for · 2017 | Legal Regulatory Instrument | Legal Doctrinal | “have caused a range of concerns revolving mainly around unfairness, discrimination and opacity” |
| Ethics and governance of trustworthy medical artificial intelligenceBMC Medical Informatics and Decision Making · 2023 | Ethics Principles Frameworks | Review Synthesis | “We adopted a multidisciplinary approach and summarized five subjects that influence the trustworthiness of medical AI” |
| Connecting the dots in trustworthy Artificial Intelligence: From AI principles, ethics, and key requirements to responsible AI systems and regulationInformation Fusion · 2023 | Ethics Principles Frameworks | Review Synthesis | “Trustworthy Artificial Intelligence (AI) is based on seven technical requirements sustained over three main pillars” |
| Transparency and accountability in AI systems: safeguarding wellbeing in the age of algorithmic decision-makingFrontiers in Human Dynamics · 2024 | Algorithmic Decision Transparency | Review Synthesis | “This review aims to provide an overview of the key legal and ethical challenges associated with implementing transparency and accountability in AI systems” |
| Algorithmic bias, data ethics, and governance: Ensuring fairness, transparency and compliance in AI-powered business analytics applicationsWorld Journal of Advanced Research and Reviews · 2025 | Accountability Governance Mechanism | Review Synthesis | “concerns about fairness, transparency, and compliance have intensified” |
12 excluded records, with reasons
- Why a Right to Explanation of Automated Decision-Making Does Not Exist in the General Data Protection Regulation — no abstract available for coding (abstracts-only protocol)
- Exploring the impact of artificial intelligence on teaching and learning in higher education — off-topic: AI in higher-education teaching, not transparency governance
- Unraveling Transparency and Accountability in Blockchain — off-topic: blockchain for government transparency, not AI governance
- Artificial Intelligence (AI): Multidisciplinary perspectives on emerging challenges, opportunities, and agenda for research, practice and policy — off-topic: broad AI overview; transparency governance is not its subject
- Explainable Artificial Intelligence (XAI): Concepts, taxonomies, opportunities and challenges toward responsible AI — no abstract available for coding (abstracts-only protocol)
- Systematic review of research on artificial intelligence applications in higher education – where are the educators? — off-topic: systematic review of AI in higher education
- Key challenges for delivering clinical impact with artificial intelligence — off-topic: clinical-translation challenges of healthcare AI; transparency is not the abstract's subject
- Transparency and accountability in AI decision support: Explaining and visualizing convolutional neural networks for text information — no abstract available for coding (abstracts-only protocol)
- Data governance: Organizing data for trustworthy Artificial Intelligence — no abstract available for coding (abstracts-only protocol)
- Sustainable AI: AI for sustainability and the sustainability of AI — off-topic: sustainability of the AI lifecycle, not transparency governance
- Opinion Paper: “So what if ChatGPT wrote it?” Multidisciplinary perspectives on opportunities, challenges and implications of generative conversational AI for research, practice and policy — off-topic: broad ChatGPT opportunities/challenges collection
- Sparks of Artificial General Intelligence: Early experiments with GPT-4 — off-topic: capability evaluation of GPT-4, not transparency governance
Why this review kind
Review-kind selection for 'What does the scholarly literature report on data and algorithmic transparency in AI governance?': SELECTED evidence_gap_map; 0 kind(s) rejected with their failed requirements recorded. A methodological screen (Sacred Rule 9): the selection is disclosed on the article and the selected kind's own discipline still applies at conduct time.
Selector: review_selector_v1 · selected kind: Evidence Gap Map · selection hash 0164a64dd68f6193.
Verdict
evidence_gap_map conducted over 34 records (22 included, 12 excluded with reasons): a coverage map with 18 named empty cells. Counts only — no importance adjudication (Sacred Rule 9); the report re-derives offline from corpus + codings.
Verify it yourself
Every count above re-derives offline from the committed corpus and codings — no model, no network, no trust in the institute required:
PYTHONPATH=src python scripts/verify_review_data_transparency.py # exit 0 = re-derives offline