# Open Clinical AI documentation

> Setup guides, component architecture, validation methods and cited research.


---

# Component documentation and contracts

OpenClinAI brings together independently maintained applications and a validation
runner. Start with each component's native documentation for setup, supported
interfaces and current application behavior. The public website does not redefine
those contracts. The OpenMRS links below identify the reviewed assembled revisions;
they do not imply an upstream merge or deployed release.

- **ChartSearchAI:** [project documentation](https://github.com/pmanko/openmrs-module-chartsearchai/tree/ca136063556be2d67dc3b638dc06d12a516f5ff9/docs) and [source/setup](https://github.com/pmanko/openmrs-module-chartsearchai/tree/ca136063556be2d67dc3b638dc06d12a516f5ff9).
- **ChartSearchAI frontend:** [source/setup](https://github.com/pmanko/openmrs-esm-chartsearchai/tree/475f6eb60ea14011f719c861b02d02daf2156390).
- **QueryStore:** [REST API](https://github.com/pmanko/openmrs-module-querystore/blob/286993cc094499ed29f97d4574775cd2c96f5676/docs/rest-api.md) and [architecture decisions](https://github.com/pmanko/openmrs-module-querystore/blob/286993cc094499ed29f97d4574775cd2c96f5676/docs/adr.md).
- **Med Agent Hub:** [source/setup](https://github.com/pmanko/med-agent-hub).
- **Catalyst:** [application specification](https://github.com/DIGI-UW/catalyst-ai/blob/main/docs/specification.md), [Hub integration](https://github.com/DIGI-UW/catalyst-ai/blob/main/docs/med-agent-hub.md) and [cross-project delivery](../catalyst-delivery.md).
- **Validation harness:** [setup and experiment commands](https://github.com/pmanko/clinical-ai-validation-harness#readme) and [current validation contract](https://github.com/pmanko/clinical-ai-validation-harness/blob/main/specs/006-validation-harness-mvp/spec.md).

The [workspace roadmap](https://github.com/pmanko/openclinai.org/blob/main/specs/roadmap.md)
coordinates delivery. [Published reports](https://reports.openclinai.org/) and
[recorded demonstrations](https://openclinai.org/catalyst/) describe particular
runs, not current deployment or clinical acceptance. Review provenance, methods,
limitations and human review before using their findings.


---

# Why local-first clinical AI

Local-first clinical AI addresses unreliable connectivity, limited computing resources
and the need to keep health data under local control. The research below motivates
experiments with locally hosted open models, deployment-specific clinical context
and independent evaluation. Findings are qualified by their study setting; local
hosting and clinical reliability need verification for each deployment.

> Sources, exact figures, and confidence notes are collected in the project's
> research file: [Background & evidence (research)](#/spec/specs/artifacts/planning/global-health-ai-background-research-2026-06-14).

## Where most primary care actually happens

The setting, not the algorithm, is what makes this hard.

- **Power.** Close to **1 billion people** in low- and lower-middle-income countries are served by health facilities with unreliable electricity or none at all; in sub-Saharan Africa only about **40%** of facilities have reliable electricity ([WHO, 2023](https://www.who.int/news-room/fact-sheets/detail/electricity-in-health-care-facilities)).
- **Connectivity.** **78%** of people in low-income countries are offline, and in rural areas only about **1 in 6** use the internet ([ITU, Facts and Figures 2024](https://www.itu.int/itu-d/reports/statistics/2024/11/10/ff24-internet-use-in-urban-and-rural-areas/)). An always-online, cloud-only design cannot reach them.
- **People.** WHO projects a shortfall of roughly **11 million health workers by 2030**, concentrated in low- and lower-middle-income countries ([WHO health workforce](https://www.who.int/health-topics/health-workforce)). (The exact figure has moved across WHO documents; treat it as an order of magnitude.)
- **The consensus fix.** A 2026 scoping review of clinical-AI deployment in these settings found fragile infrastructure, hardware limits, and fragmented records to be the dominant barriers, and recommended **offline-capable AI with local data caching** ([Al-Ganad et al., *Frontiers in Digital Health*, 2026](https://www.frontiersin.org/journals/digital-health/articles/10.3389/fdgth.2026.1743634/full)).

## Why high-income-trained AI underperforms here

- WHO's own guidance warns that AI "systems trained primarily on data collected from individuals in **high-income countries may not perform well** for individuals in low- and middle-income settings" ([WHO, *Ethics and governance of AI for health*, 2021](https://www.who.int/news/item/28-06-2021-who-issues-first-global-report-on-ai-in-health-and-six-guiding-principles-for-its-design-and-use)).
- That is borne out in practice: a 2025 review reports LLMs make **up to three times more errors** on information about low-income countries, and accuracy can fall sharply in lower-resourced languages — while over **87%** of healthcare-LLM research is led by high-income-country institutions ([Chen et al., *Lancet Regional Health – Western Pacific*, 2025](https://pmc.ncbi.nlm.nih.gov/articles/PMC12556221/)). (Some underlying figures are single-source within that review.)

The AI is least reliable exactly where it is needed most — which is the case for contextualizing it to each deployment rather than shipping one global model.

## Why patient data should stay local

- WHO cautions that data disclosed to a model provider "can usually not be retrieved as future iterations of the model may be trained on this data" ([WHO, *Guidance on large multi-modal models*, 2024](https://www.who.int/news/item/18-01-2024-who-releases-ai-ethics-and-governance-guidance-for-large-multi-modal-models)).
- A review of **464** healthcare-LLM studies found six distinct privacy risks from external/cloud models, that **38%** reported no patient-data protection at all, and recommended that "**priority should be given to deploying the LLM locally**" ([Zhong et al., *JMIR*, 2025](https://www.jmir.org/2025/1/e76571)).
- The governance literature points the same way: the Health Data Governance Principles call for keeping data "close to their point of generation," and at WHO's 2026 digital-health debate, low- and middle-income countries argued health data should be "a national asset under local control" ([Health Data Governance Principles](https://healthdatagovernance.org/principles/); [Health Policy Watch, 2026](https://healthpolicy-watch.news/who-debates-global-ai-rules/)).

Local processing supports this guidance, but privacy is deployment-specific. Review configured providers, evaluation services and publication/redaction before sharing evidence.

## Why small, right-sized open models — with a strong judge

- **Small can run offline.** A 3.8-billion-parameter open model runs fully offline on a phone at roughly the quality of a previous-generation cloud model ([Microsoft Research, Phi-3 technical report, 2024](https://arxiv.org/abs/2404.14219)).
- **Open is closing the gap.** On 1,933 real radiology cases, the leading closed model scored 79.6% and the best open model 73.2% — the authors concluded "open-source LLMs are quickly closing the gap to proprietary LLMs" ([Kim et al., *npj Digital Medicine*, 2025](https://pmc.ncbi.nlm.nih.gov/articles/PMC11814077/)). Open medical models such as MedGemma are explicitly built to run locally or offline ([Sellergren et al., 2025](https://arxiv.org/abs/2507.05201); its top score uses extra test-time computation).
- **Right-size the model to the job.** Matching model size to task difficulty — small models for routine work, escalating only when needed — captures most of the quality at a fraction of the cost (survey: [Moslem & Kelleher, 2026](https://arxiv.org/abs/2603.04445); direction is robust, exact savings are setup-dependent).
- **But a small model can't check itself.** In a 2026 study, AI self-verification wrongly accepted incorrect medical answers **more than 60%** of the time, and smaller models were no better — the authors warn it "cannot serve as a universal safety layer" ([Jin et al., 2026](https://arxiv.org/abs/2605.10850), a recent preprint). This supports evaluating answers against independent evidence and human review, with separately configured model evaluators where appropriate.

This motivates experiments with right-sized open models, record-grounded answers and independent review. It does not establish correctness or safety for a particular model team.

## Why guideline-concordant and testable

WHO's **SMART Guidelines** exist to turn paper recommendations into a standards-based, computable form so digital systems deliver guideline-concordant care faithfully and faster — organized as a ladder of knowledge layers from **L1 narrative** through machine-readable and executable forms ([WHO SMART Guidelines](https://www.who.int/teams/digital-health-and-innovation/smart-guidelines); [WHO, 2021](https://www.who.int/news/item/18-02-2021-from-paper-to-digital-pathway-who-launches-first-smart-guidelines)). The first such guidelines covered antenatal care, family planning, STIs, and HIV — HIV being **one** concrete example of a computable WHO guideline, not the limit of what a validation harness covers.

Software that delivers clinical guidance should be **standards-based, verifiable, and testable**. Harness experiments can evaluate claim support against captured source evidence using the selected adapter, scenarios and evaluation methods. WHO's AI guidance reinforces the bar, warning that large models carry "documented risks of producing false, inaccurate, biased, or incomplete statements" and can encourage "automation bias" where errors go unnoticed ([WHO, 2024](https://www.who.int/news/item/18-01-2024-who-releases-ai-ethics-and-governance-guidance-for-large-multi-modal-models)).

## The through-line

| The reality | The project's answer |
|---|---|
| Care runs offline, on modest hardware | Configurable locally hosted open models; disconnected operation depends on the selected services and hardware |
| High-income-trained AI underperforms for these patients | Application-supplied clinical context and evaluation on the intended records, terminology and population |
| Sending data to clouds is a privacy and sovereignty risk | Prefer local processing; verify configured providers and disclosure boundaries |
| Models hallucinate; small models can't self-verify | Evaluate claim support against recorded evidence with independent review |
| Clinical software should be guideline-concordant and testable | Configured experiments that capture responses, available evidence and review findings |

The research supports **local processing, context appropriate to the deployment, and independent evaluation against recorded evidence** as priorities for clinical AI experiments.


---

# OpenClinAI

OpenClinAI brings together tools for clinical AI, patient-chart search, health-data
analysis, and reproducible evaluation. This repository connects the projects and
provides a shared development workspace.

**[Project website](https://openclinai.org)** ·
**[Explore the projects](#explore-the-projects)** ·
**[Contribute](#contribute)**

## Projects

| Project | What it does | Setup and source |
| --- | --- | --- |
| **ChartSearchAI** | Lets clinicians ask questions about an OpenMRS patient chart and inspect answers with source citations. | [Backend](https://github.com/pmanko/openmrs-module-chartsearchai) · [OpenMRS frontend](https://github.com/pmanko/openmrs-esm-chartsearchai) |
| **Med Agent Hub** | A shared service for applications to configure and call language models, including clinical-answer workflows. | [Repository](https://github.com/pmanko/med-agent-hub) |
| **Catalyst** | A supervised SQL workbench and dashboard builder for exploring health data, reviewing queries and publishing results to Apache Superset. | [Repository](https://github.com/DIGI-UW/catalyst-ai) |
| **QueryStore** | Retrieves and organizes OpenMRS clinical records for chart search and AI applications. | [Repository](https://github.com/pmanko/openmrs-module-querystore) |
| **Clinical AI Validation Harness** | Runs clinical-AI experiments, collects outputs and evidence, and produces evaluation and review reports. | [Repository](https://github.com/pmanko/clinical-ai-validation-harness) |

ChartSearchAI integrates with [OpenMRS](https://openmrs.org/). Catalyst supports
SQL-connected data sources, including reporting workflows for
[OpenELIS Global](https://openelis-global.org/). Each project provides installation
and usage documentation in its repository.

For ChartSearchAI background and community discussion, see the
[OpenMRS project page](https://openmrs.atlassian.net/wiki/spaces/projects/pages/373325839/Chart+Search+aka+ChartSearchAI).
The OpenMRS repository links above point to the development forks used by this workspace.

## Explore the projects

- [ChartSearchAI overview](https://openclinai.org/chartsearchai/): patient-chart
  search and clinical-answer workflows.
- [Catalyst overview](https://openclinai.org/catalyst/) and
  [screenshot walkthrough](https://openclinai.org/catalyst/hiv-gallery/): query
  workbench and reporting workflows.
- [Evaluation reports](https://reports.openclinai.org/): published experiments
  and supporting evidence.

To install a project, follow its setup guide in the repositories listed above.

## Contribute

For product development, use that project's repository and contributor instructions.
For component integration and shared development tooling, use this workspace.
The [architecture](specs/architecture.md) explains how the projects fit together;
the [roadmap index](specs/roadmap.md) lists current priorities and links each
delivery stream's focused spec. Read the [repository instructions](AGENTS.md) before editing.

### Workspace checkout

Cloning requires Git and authenticated GitHub access to this private repository.
The checkout uses the component revisions recorded by the workspace.

```sh
git clone --recurse-submodules https://github.com/pmanko/openclinai.org.git
cd openclinai.org
```

For an existing clone, check for local changes before updating its submodules:

```sh
git submodule update --init --recursive
```

### Workspace checks

Requires Python 3.10+:

```sh
python3 -m unittest discover -s tests -v
python3 scripts/check_workspace.py
```

These commands check documentation links and component checkout consistency.
Each product supplies its own build and test instructions.

### Shared development

For a fresh OpenMRS instance with the HIV dataset, ChartSearchAI and research
accounts, follow the [research setup guide](environments/README.md) and run
`make chartsearch-research-setup`. This imports the baseline; it is not an upgrade.
After setup, `make local-stack-up` starts the whole environment with the saved
data and local settings. Stop it with `make local-stack-down`.

The root `Makefile` provides the shared environment and product build commands:

| Command | Purpose |
| --- | --- |
| `make openmrs-source-pair-build` | Build QueryStore, then ChartSearchAI, and stage their modules. |
| `make chartsearch-esm-build` | Build and stage the OpenMRS frontend. |
| `make local-stack-up` | Start the prepared OpenMRS, Hub and model-router environment. |
| `make local-stack-down` | Stop the local environment. |
| `make chartsearch-research-setup` | Fresh HIV OpenMRS setup and seven research accounts. |
| `make catalyst-mvp-up` | Start the configured Catalyst environment, retaining existing data. |
| `make validate-run SET=demo` | Run the selected validation experiment against prepared services. |

Read the component's setup instructions before building or starting it. Product
builds require the component-specific Java, Node, Docker or model-service tools.
Connection settings and credentials belong in ignored environment files.

### Website

Website pages and assets are in `landing/`; documentation and interactive previews
are in `site/`. The [website guide](site/README.md) covers local builds, checks and
publication. Static hosting configuration lives in `compose/website/`, and the
GitHub Pages workflow is configured in this repository.


---

# Validation research

Interactive canvas — full HTML at /openclinai.org/canvas/specs/artifacts/canvases/validation-research.html

---

# Demo-data profile & cohorts

Interactive canvas — full HTML at /openclinai.org/canvas/specs/artifacts/canvases/demo-data-profile.html

---

# Catalyst demos

Interactive canvas — full HTML at /openclinai.org/canvas/specs/artifacts/canvases/catalyst-demos.html

---

# OpenClinAI Architecture

OpenClinAI combines clinical applications, model execution, clinical-record
retrieval and validation. This document describes the target architecture and
interfaces. The [roadmap](roadmap.md) tracks implementation and delivery.

## 1. Components

| Component | Responsibilities |
| --- | --- |
| OpenClinAI umbrella | Component repositories and version pins; workspace configuration; builds, deployment and release orchestration; project documentation and website |
| Validation harness | Run configured experiments, collect outputs and traces, record provenance, evaluate results and produce evidence and review reports |
| ChartSearchAI and ESM | OpenMRS patient-chart questions, conversations, authorization, provider selection, streaming answers and clinical-evidence presentation |
| QueryStore | OpenMRS clinical-record projections, synchronization, retrieval, context selection and freshness/completeness metadata |
| Med Agent Hub | Model profiles, prompts, role mappings, model access and clinical-answer workflows |
| Catalyst | SQL workbench, source/schema context, query execution, Datasets, Widgets, Dashboards and publication |

ChartSearchAI uses QueryStore for clinical context and supports bundled inference
and configured Med Agent Hub workflows. Catalyst supplies schema context and
orchestrates query generation through model roles configured in Med Agent Hub.

Model servers execute inference requests. FHIR Data Pipes provides ingestion for
the reference analytics deployment; Apache Superset renders published dashboards.
OpenELIS also provides native reporting.

## 2. Development Workspace

The component layout is:

```text
openclinai.org/
  specs/
  scripts/
  tests/
  landing/
  site/
  compose/
    website/
  targets/
    validation-harness/
    med-agent-hub/
    catalyst/
    chartsearchai/
    chartsearchai-esm/
    querystore/
```

Each component has one umbrella gitlink. `.gitmodules` records repository locations,
and Git records the selected revisions.

The umbrella resolves component checkouts, configures shared environments, orders
builds, orchestrates deployment and verifies releases. Products expose native
build, test and runtime commands. Workspace tooling invokes those commands with
explicit component paths and environment settings.

## 3. Validation Experiments

The validation harness is an independently installable experiment runner. Its
modules provide target adapters, scenario execution, trace collection, evaluation
and report generation.

### Inputs

| Input | Purpose |
| --- | --- |
| Target adapter and connection settings | Select the target interface and connect to a local or remote service |
| Scenarios, fixtures and parameters | Define experiment inputs and execution conditions |
| Evaluation and review settings | Select evaluators, criteria and review workflows |
| Target and run metadata | Identify the tested system, models, data and experiment configuration |
| Output location | Store responses, traces, evaluation results and reports |

Adapters invoke product interfaces and capture their responses. Experiment
configuration selects the adapter and target. Readiness checks verify the
capabilities and connectivity needed by the experiment.

An experiment runs with the harness package, its selected adapter/evaluator
dependencies, input data and access to its target. The same interface supports
services launched by the umbrella, independently deployed services and externally
provided targets.

### Outputs and Provenance

Run artifacts contain experiment inputs, target responses, traces, evaluation
results and review records. Target identity can include a source revision, image
digest, model/profile identifier or deployment receipt supplied by the caller or
reported by the target. Unavailable metadata is explicitly identified.

Evaluators read captured inputs, outputs and evidence according to the experiment's
evaluation settings. Evaluation may invoke configured model services, recording
model, prompt and configuration provenance. Human review records the reviewer,
findings and rationale.

Report generation is an offline transformation of captured evidence, evaluation
results and review records. A report can be regenerated with the target and
evaluator services unavailable.

## 4. Interfaces

| Interaction | Contract |
| --- | --- |
| Umbrella → product | Component revision, build/runtime configuration and native command interface |
| Umbrella → harness | Experiment configuration, target connection settings, provenance and output location |
| Harness → product | Adapter-specific requests through the product's API or execution interface |
| Harness → evaluator | Captured inputs, outputs and evidence plus evaluation settings |
| Evaluator → report | Results, findings and review records linked to their source evidence |
| ChartSearchAI → QueryStore | Clinical-record retrieval and selected context with provenance and completeness metadata |
| ChartSearchAI → Med Agent Hub | Configured clinical profile execution with streamed answer and evidence events |
| Catalyst → Med Agent Hub | Configured model roles for Catalyst's application-controlled SQL workflow |

Product contracts define clinical and application behavior. Experiment contracts
define execution, evidence, scoring, sampling and review semantics. The umbrella
selects component versions and connects their interfaces.

## 5. Website and Publication

The `openclinai.org` repository contains the public website's page content, assets,
navigation, documentation/status interfaces, sitemap and site configuration.
It also contains the associated build tools, publication scripts, deployment
workflows and web-server configuration. Website CI and release operations run
from this repository.

The validation harness produces experiment artifacts and rendered reports.
OpenClinAI publication tooling consumes selected artifacts, maintains public
catalogs and publishes them to the configured web destinations. Publication uses
the recorded evidence and evaluation results as its inputs.

## 6. Contract References

These links identify contracts at the component revisions recorded by the workspace.

- [OpenClinAI provider integration requirements/reference](../docs/openmrs-provider-interface.md)
- [Native OpenMRS provider contract](https://github.com/pmanko/openmrs-module-chartsearchai/blob/ca136063556be2d67dc3b638dc06d12a516f5ff9/README.md#provider-integration-contract)
- [QueryStore API](https://github.com/pmanko/openmrs-module-querystore/blob/286993cc094499ed29f97d4574775cd2c96f5676/docs/rest-api.md)
- [Catalyst–Med Agent Hub contract](https://github.com/DIGI-UW/catalyst-ai/blob/94742f1af6d634f03b47b9e97f3512829ba265b1/docs/med-agent-hub.md)


---

# Background & evidence — clinical AI for low-resource health settings

Research on infrastructure, health-data governance and open models informs
OpenClinAI's clinical AI experiments. Sources include WHO guidance, peer-reviewed
literature and recent preprints. Confidence notes distinguish reported findings
from qualified estimates; some guidance and paywalled studies are represented by
official summaries or abstracts rather than full texts.

---

## 1. WHO SMART Guidelines — computable, guideline-concordant care

- **SMART = Standards-based, Machine-readable, Adaptive, Requirements-based, Testable** — WHO's approach to getting its recommendations into countries' digital systems faithfully and fast. (WHO, "From paper to digital pathway," 18 Feb 2021 — who.int/news; confirmed across 3 sources incl. JMIR 2025. *high*)
- They are "a comprehensive set of reusable digital health components … that transform the guideline adaptation and implementation process to **preserve fidelity and accelerate uptake**." (WHO Digital Health & Innovation — who.int/teams/digital-health-and-innovation/smart-guidelines. *high*)
- Organized into **five knowledge layers (L1–L5)** — "a systematic, transparent and testable structure": **L1 Narrative** (human-readable recommendations) → **L2 Operational** (software-neutral requirements, delivered as Digital Adaptation Kits) → **L3 Machine-readable** (structured specs with coding/terminology/interoperability standards) → **L4 Executable** (software running static algorithms) → **L5 Dynamic** (algorithms optimized with analytics). (WHO smart.who.int DAK pages; L1–L4 *high*, L5 one-line definition *medium* — WHO confirms the name "Dynamic" but L5 content is largely "not yet available".)
- Motivation (peer-reviewed): converting narrative guidelines to digital systems has been "laborious, prone to error, and lacks accompanying technical documentation appropriate for digital use." (Muliokela et al., JMIR Medical Informatics, 7 Feb 2025 — medinform.jmir.org/2025/1/e58858. *high*) Foundational paper: Mehl et al., "WHO SMART guidelines…," Lancet Digital Health, 2021 (PMID 33610488 — thesis confirmed, full text paywalled).
- **Project tie-in:** the harness asks for the same property SMART Guidelines demand of software — answers that are **standards-based, verifiable, and traceable**, not opaque.


## 2. WHO ethics & governance of AI / LMMs in health

- WHO's first global guidance, **"Ethics and governance of artificial intelligence for health"** (28 Jun 2021, ISBN 9789240029200), sets six principles: human autonomy; well-being/safety/public interest; transparency & explainability; responsibility & accountability; inclusiveness & equity; responsive & sustainable AI. (who.int/news 28-06-2021. *high*)
- **Equity warning:** AI "systems trained primarily on data collected from individuals in **high-income countries may not perform well for individuals in low- and middle-income settings**." (WHO, 28 Jun 2021. *high*)
- Second, dedicated guidance, **"…Guidance on large multi-modal models (LMMs)"** (18 Jan 2024, ISBN 9789240084759), 40+ recommendations. (who.int/news 18-01-2024. *high*)
- **Hallucination/harm:** LMMs carry "documented risks of producing **false, inaccurate, biased, or incomplete statements, which could harm people** using such information in making health decisions." (WHO, 18 Jan 2024. *high*)
- **Automation bias:** LMMs "can also encourage 'automation bias' … whereby errors are overlooked that would otherwise have been identified." (WHO, 18 Jan 2024. *high*)
- Data risk: data disclosed to LMM developers "can usually not be retrieved as future iterations of the model may be trained on this data." (WHO, 18 Jan 2024. *high*)
- Safeguards WHO recommends: inclusive design with clinicians/patients from the start; mandatory independent post-release auditing & impact assessment; assigning a regulator to approve health LMMs; models designed "to perform well-defined tasks with the necessary accuracy and reliability." (WHO, 18 Jan 2024. *high*)


## 3. Why low-resource settings need a different design

- **Electricity:** "Close to **1 billion people** in low- and lower-middle-income countries are … served by health-care facilities **without reliable electricity or with no electricity** at all." In sub-Saharan Africa only **40%** of facilities have reliable electricity, **15%** none. (WHO fact sheet, 31 Aug 2023 — who.int/news-room/fact-sheets/detail/electricity-in-health-care-facilities. *high*)
- **Connectivity:** **78%** of people in low-income countries are offline (vs 7% in high-income); in rural low-income areas only **1 in 6 (16%)** use the internet. (ITU Facts and Figures 2024. *high*)
- **Workforce:** WHO projects a shortfall of **~11 million health workers by 2030**, mostly in low/lower-middle-income countries. (WHO health-workforce page. *high* for current figure; the number has moved 10–18M across WHO docs/years — treat as version-dependent.)
- **Research/data concentration:** >87% of healthcare-LLM research is led by high-income-country institutions; Africa ≈0.31% despite ~20% of world population. (Chen et al., Lancet Regional Health – Western Pacific, Oct 2025 — PMC12556221. *high*)
- **Performance gap:** "LLMs make up to **three times more errors** when retrieving information related to low-income countries"; accuracy can fall "from around 80% in English to just 50% in Thai." (Chen et al., 2025. *high* for citation; underlying primary numbers *medium*.)
- **Cloud economics:** a 70B model needs ~8 A100 GPUs (~US$300k/yr cloud); serving one per 100k people "could consume up to 15% of national healthcare expenditure"; even a $20/mo fee "exceeds the financial means of nearly half the global population." (Chen et al., 2025. *high* as reported.)
- **Independent confirmation + recommendation:** 2026 scoping review (44 studies) — fragile infrastructure a barrier in 77.3%, hardware limits 50%, literacy/staffing gaps 61.4%, fragmented/paper records 81.8%; recommends "**offline-capable AI models with local data caching**." (Al-Ganad et al., Frontiers in Digital Health, 2026. *high*)
- **Project tie-in:** unreliable infrastructure motivates experiments with locally hosted models and application-supplied clinical context. Disconnected operation depends on the complete service configuration and available hardware.

## 4. Data privacy, sovereignty & local ownership

- **Sovereignty:** at WHO's May 2026 digital-health/AI debate, LMICs warned AI "risks accelerating data extraction"; Cameroon (African Region) feared corporations would "harvest data from the Global South to train AI models"; Barbados argued health data should be "**a national asset under local control**." (Health Policy Watch, 2 May 2026. *high*)
- **Governance principle:** the Health Data Governance Principles support federated approaches that keep data near its point of generation, informing the case for local processing. ([Transform Health, 2022](https://healthdatagovernance.org/principles/). *Paraphrase of the published principles.*)
- **Cloud-LLM privacy risk:** a review of **464** healthcare-LLM studies found six privacy risks from external/cloud LLMs and that **38.4%** reported *no* PHI-protection measures; its top recommendation: "**Priority should be given to deploying the LLM locally.**" (Zhong et al., JMIR, 21 Nov 2025 — jmir.org/2025/1/e76571. *high*)
- **Data colonialism:** research on decolonizing global health examines extractive practices that move data away from the communities that generate it. ([Scoping review, 2025](https://pmc.ncbi.nlm.nih.gov/articles/PMC12560380/). *Concept summarized, not a verbatim definition.*)

- **Project tie-in:** local processing can reduce disclosure risk. Verify configured providers, evaluation services and evidence redaction for each deployment; this is not a blanket privacy guarantee.


## 5. Open-weight models & right-sizing per task

- **Small can run offline:** a **3.8B** open model (Phi-3-mini) runs fully offline on an iPhone in ~1.8 GB at >12 tok/s, at roughly GPT-3.5 quality. (Microsoft Research, Phi-3 Technical Report, Apr 2024 — arxiv.org/abs/2404.14219. *high*)
- **Open is closing the gap:** on 1,933 real radiology cases, closed GPT-4o scored **79.6%** and open Llama-3-70B **73.2%**; authors: "open-source LLMs are quickly closing the gap to proprietary LLMs." (Kim et al., npj Digital Medicine, 12 Feb 2025 — PMC11814077. *high*)
- **Medical open models:** MedGemma (open, Gemma-3-based, 4B/27B) scores 64.4% (4B) / 87.7% (27B w/ test-time scaling) on MedQA vs 50.7%/74.9% base; authors recommend it when a use case needs "ability to run locally or offline." (Sellergren et al., Google, Jul 2025 — arxiv.org/abs/2507.05201. *high*; the 87.7% includes test-time scaling.)
- **Right-sizing / routing:** matching model size to task difficulty (small for easy, escalate to large only for hard) yields large savings (e.g. a router hitting 97% of GPT-4 quality at ~24% cost). (Moslem & Kelleher survey, ADAPT/TCD, Apr 2026 — arxiv.org/abs/2603.04445. Direction *high*; exact percentages *medium*, setup-dependent.)
- **Capable size range on modest hardware:** Llama 3.2 1B, Qwen2.5 1.5B, Gemma 2 2B/9B, Phi-3-mini 3.8B/Phi-4 14B, Mistral 7B — 1–9B built for on-device. (model cards; anchored by Phi-3. *high*.)
- **Safety limit — small models can't self-verify:** AI self-verification wrongly accepted incorrect medical answers **>60%** of the time, and smaller models were no better — "cannot serve as a universal safety layer." (Jin et al., "Verification Mirage," UBC/Vector, May 2026 — arxiv.org/abs/2605.10850. *medium*, recent preprint, consistent w/ peer-reviewed LLM-as-judge literature.) This motivates independent evidence-based evaluation and human review; a larger model's judgment also needs validation for the task.
- **Project tie-in:** applications can use configured open models, while harness experiments use separately selected evaluation methods and reviewers. Model evaluation may use local or remote services; provider configuration determines connectivity and disclosure requirements.

**Study limitations:** the self-verification finding is reported in a recent preprint, not established deployment performance. MedGemma's 27B headline result includes test-time scaling.

---

## How the external evidence maps to the project (the through-line)

| The reality (cited above) | The project's answer (in-repo) |
|---|---|
| Care runs offline, on modest hardware (§3) | Configurable local model services; disconnected operation depends on the selected services and hardware |
| HIC-trained AI underperforms for LMIC patients (§2, §3) | Application-supplied clinical context and evaluation on intended records, terminology and populations |
| Sending PHI to clouds is a privacy/sovereignty risk (§4) | Prefer local processing; verify providers and publication/redaction boundaries |
| LMMs hallucinate; small models can't self-verify (§2, §5) | Evaluate claim support against captured source evidence using selected evaluators and human review |
| Software should be guideline-concordant & testable (§1) | Configured experiments that capture responses, available evidence and review findings |

The research supports **local processing, context appropriate to the deployment,
and independent evaluation against recorded evidence** as priorities for clinical AI experiments.


---

# Guardrails and control-flow methodology — research

This public research summary informs clinical-AI experiments. It does not prescribe
Med Agent Hub's current implementation or replace product safety contracts.

## Findings

- **Defense in depth:** every guardrail has failure modes. Combine prompt guidance,
  deterministic checks and review rather than trusting one system prompt. The
  Swiss Cheese model and agentic-defense research support layering, not a claim
  that any particular clinical system is safe.
- **Mix enforcement types:** NeMo Guardrails separates input, retrieval, dialog,
  execution and output checks. LlamaFirewall combines deterministic detectors and
  a model-based auditor. Its reported AgentDojo attack-success reduction
  (17.6% to 1.75%) motivates layering; it does not predict clinical-QA performance.
- **Prompt guidance:** explicit scoped instructions, diverse examples, quote-first
  grounding and a non-contradictory instruction hierarchy are useful methods.
  Vendor guidance for frontier models does not establish equivalent reliability
  for small local models.
- **Well-formed is not correct:** structured decoding can constrain an output's
  shape, but cannot guarantee groundedness, correct clinical interpretation or
  appropriate abstention. Measure those outcomes separately.
- **Reliability is separate from accuracy:** small-model studies report consistent
  but incorrect answers and sensitivity to paraphrases. Evaluate diverse cases,
  paraphrases and adversarial content, not only one successful response.
- **Verification needs independent evidence:** a model's self-check is not a
  universal safety layer. Citation resolution can be checked deterministically;
  claim support and clinical interpretation still require suitable evaluation
  and human review.

## Applying the research to experiments

Capture the exact inputs, output, model and prompt provenance when available,
retrieved evidence, deterministic findings and reviewer rationale. Keep optional
model judgments separate from deterministic safety checks. Study unsupported
claims, fabricated citations, absent evidence, abstention and disclosure limits.
A score or schema-valid response is not product or clinical acceptance.

Whether a workflow needs deterministic control flow or additional output checks
is a product design decision informed by measured failures. Consult the
[product-native contracts](#/spec/specs/background/component-contracts) and the
[current harness experiment contract](https://github.com/pmanko/clinical-ai-validation-harness/blob/main/specs/006-validation-harness-mvp/spec.md).

## Caveats

- Here, prompt-level guidance and code/constrained-decoding enforcement are
  different mechanisms. Do not equate them with academic uses of “soft” and
  “hard” ethical guardrails.
- Published measurements do not establish exact performance for a deployment's
  4–8B models, prompts, corpus or clinical questions.
- NeMo and LlamaFirewall are examples, not a complete framework comparison or a
  recommendation to add either dependency.
- Security attack-success rates and clinical evidence-grounding measures are
  distinct outcomes.

## Sources

- [Swiss Cheese AI safety](https://arxiv.org/html/2408.02205v3)
- [Agentic defense taxonomy](https://arxiv.org/html/2603.11088v1)
- [NeMo Guardrails process](https://docs.nvidia.com/nemo/guardrails/latest/user-guides/guardrails-process.html)
- [LlamaFirewall](https://arxiv.org/pdf/2505.03574)
- [Anthropic prompting guidance](https://platform.claude.com/docs/en/build-with-claude/prompt-engineering/claude-prompting-best-practices)
- [OpenAI GPT-5 prompting guide](https://developers.openai.com/cookbook/examples/gpt-5/gpt-5_prompting_guide)
- [Small-model clinical reliability](https://arxiv.org/abs/2603.00917)
- [Reliability versus accuracy](https://arxiv.org/html/2512.14754v1)