coreSR.AIcoreSR
Usage Guide

QUIPS

How to assess risk of bias for prognostic factor studies

1

What QUIPS is and when to use it

  1. 1QUIPS (Quality In Prognosis Studies) — Hayden et al., Ann Intern Med 2013
  2. 2Use for studies of prognostic factors: observational research that asks whether and how strongly a factor predicts a future health outcome
  3. 3NOT for treatment-effect studies (use RoB 2.0 / ROBINS-I), diagnostic accuracy studies (use QUADAS-2), or prediction model development/validation (use PROBAST)
  4. 4Unit of assessment: one QUIPS assessment per (publication × prognostic factor × outcome). The bias profile can differ across PF–outcome pairs in the same paper, so the same paper may produce N×M assessments.
Tip: QUIPS is deliberately judgment-driven. The CoreSR rollup gives you a deterministic SUGGESTION; reviewer override is expected as the primary path, not an exception.
2

The 6 domains

  1. 1Domain 1 — Study Participation: bias from non-representativeness of the study sample
  2. 2Domain 2 — Study Attrition: bias from differential loss to follow-up by PF status or outcome
  3. 3Domain 3 — Prognostic Factor Measurement: bias from misclassification of the PF
  4. 4Domain 4 — Outcome Measurement: bias from differential or invalid outcome measurement
  5. 5Domain 5 — Study Confounding: the most consequential domain in practice; bias from unmeasured or unadjusted confounders
  6. 6Domain 6 — Statistical Analysis and Reporting: bias from inadequate model, inappropriate analytic strategy, or selective reporting
3

Hinge items per domain

  1. 1Each domain has 1-2 "hinge" items that the rollup treats specially — when the hinge is rated N (or sometimes PN), the domain rating jumps to High regardless of other items
  2. 2D1 hinge: 1.5 (participation rate)
  3. 3D2 hinges: 2.1 (extent of loss to follow-up) + 2.4 (differential attrition between completers and lost)
  4. 4D3 hinges: 3.2 (PF measurement validity) + 3.5 (PF missingness)
  5. 5D4 hinges: 4.2 (outcome measurement validity) + 4.3 (consistency of ascertainment / blinding)
  6. 6D5 hinges: 5.1 (critical confounders measured) + 5.7 (critical confounders adjusted for)
  7. 7D6 hinges: 6.3 (model adequacy for the design) + 6.4 (selective reporting)
Tip: In the review panel, hinge items are tagged with a small "hinge" badge and a tooltip. The reviewer can still override the auto-suggested domain rating.
4

Setting up Step 1 (review question)

  1. 1Open Risk of bias → QUIPS and click "Create new QUIPS project"
  2. 2Add at least one review question with five fields: Population, Prognostic factor, Outcome, Confounding context, and Moment of prediction
  3. 3The first review question becomes the project default; you can add more for projects covering multiple PFs or outcomes
  4. 4Optionally upload a protocol PDF and click "Extract from PDF" to pre-fill the five fields
5

Step 1b — Locking expected confounders (Domain 5 prerequisite)

  1. 1Click "Generate for outcome" and supply an outcome label (e.g. "5-year mortality after MI"); add the PF and clinical context for better AI gap analysis
  2. 2CoreSR looks the outcome up in shared/confounder-knowledge-base.json (the same KB that powers ROBINS-I) and seeds the list with critical + important confounders
  3. 3AI gap analysis suggests context-specific additions the static KB might miss
  4. 4Edit the list — add, remove, or change importance (critical / important / optional)
  5. 5Click "Lock" to freeze the list. Domain 5 AI scoring will refuse to run until the list for an outcome is locked.
  6. 6When the reviewer unlocks + edits later, a NEW VERSION is created and Domain 5 is flagged stale on existing assessments using that outcome — re-run AI on Domain 5 only or acknowledge that the new list does not change your judgment
Tip: The lock uses a race-safe compare-and-set on lockedAt — if two reviewers try to edit at once, the second save fails cleanly with a 409 instead of clobbering.
6

Step 2 — Per-assessment metadata

  1. 1Each assessment carries four metadata fields: study design, prognostic factor type, outcome type, analysis type
  2. 2These fields gate Domain 6 model-adequacy logic (e.g. logistic regression on time-to-event data is PN; Cox on time-to-event is Y) and Item 3.3 NA logic (cutpoint handling is NA when PF type is binary)
  3. 3Set these on the project page or on the assessment detail page. Changing them re-evaluates which items are NA and updates the item-answer stubs accordingly.
7

How AI scoring works (Phase A → B → C)

  1. 1Phase A: 6 parallel per-domain evidence extractions. Each domain has its own Zod schema; failed slices stay marked failed and the reviewer fills them manually.
  2. 2Phase B: per-item structured AI scoring. Domain 5 prompts include the locked expected-confounders list inline. Domain 6 prompts include the metadata so model-adequacy gating works correctly.
  3. 3Phase C: deterministic rollup in server/quips/rollup.ts — six per-domain heuristics with hinge-item weighting; computeOverallSummary produces the CoreSR convention output.
  4. 4Idempotency: every AI run is keyed on (metadata, doc text, expectedConfounders version, calibration, decision-rules version). Re-running with no upstream change short-circuits.
8

Numeric thresholds — per-project calibration

  1. 1Several items use numeric thresholds that QUIPS itself does not specify: participation rate (1.5), loss-to-follow-up (2.1), PF missingness (3.5), confounder missingness (5.5)
  2. 2CoreSR ships defaults from epidemiological convention (≥80% / 60-80% / 40-60% / <40% participation, etc.) but allows per-project overrides
  3. 3Open the project settings to tune thresholds for context-specific reviews (e.g. emergency-department studies have lower acceptable participation rates)
  4. 4The chosen calibration is recorded in the AI prompt + audit log so historical assessments stay reproducible
9

Domain rollup — auto-applied with reviewer override

  1. 1When the heuristic produces a suggestion, it becomes the algorithmRating immediately and finalRating mirrors it (PROBAST-style auto-apply)
  2. 2Reviewer userRating overrides the algorithm. Both are persisted; the dashboard shows the algorithm rating in italics when an override is in place.
  3. 3NA items are skipped in the rollup — the autoSetReason is shown inline so reviewers know why
  4. 4Reviewer can override any item answer or domain rating one-click; rationale is encouraged and audit-logged
10

Domain 5 stale-confounders flow

  1. 1When you supersede a locked expected-confounders list, every Domain 5 row for assessments using that outcome is marked stale
  2. 2Stale Domain 5 rows show a yellow banner with two options: re-run AI on Domain 5 (preferred) or acknowledge that the new list does not change your judgment for this study
  3. 3Acknowledge requires a free-text rationale and is audit-logged
  4. 4Sign-off REFUSES (409) when any Domain 5 is still stale — the platform forces you to deal with it before locking the assessment in
11

CoreSR overall summary (collapsed by default)

  1. 1QUIPS does NOT mandate an overall risk-of-bias rating — domain ratings are the canonical output
  2. 2CoreSR offers a computed convention (any High → High; any Moderate → Moderate; else Low) for downstream consumers (dashboards, GRADE)
  3. 3The tile is COLLAPSED BY DEFAULT (codex audit recommendation) so domain ratings dominate the page
  4. 4CSV exports use the column name synthero_overall_convention (not overall_risk) so it can't be confused with a QUIPS-mandated rating
  5. 5Reviewer override on the overall is supported and logged
12

Sign-off and audit

  1. 1Click "Sign off" on the overall tile to mark the assessment complete
  2. 2Sign-off records signedOffBy + signedOffAt and is audit-logged
  3. 3Domain 5 must not be stale; assessment must have a final rating per domain
  4. 4The append-only audit log captures every metadata change, override, lock, supersede, AI run, and sign-off — useful for methods appendix and reviewer accountability
13

Bulk operations and project export

  1. 1"Run AI on all" enumerates which assessments are ready (have files + locked confounders + outcome label) and which are skipped (and why)
  2. 2"Export CSV" produces RFC 4180 CSV with study metadata, six per-domain ratings, the CoreSR overall convention, sign-off status, and which expected-confounders version Domain 5 was scored against
  3. 3Per-domain "Re-extract" button clears that slice from descriptionFields so the next Run AI re-extracts only that domain
14

Pulling from screening

  1. 1In a full-text screening batch, "Send Included to Risk of Bias → QUIPS" creates a QUIPS project keyed on (batch × tool), auto-creates a default review question, and one assessment per included reference
  2. 2Idempotent: re-pushing the same batch only adds NEW references
  3. 3After import: fill in the five review-question fields, generate-and-lock expected confounders for each unique outcome, set per-assessment metadata, then run AI
15

Reviewer best practice

  1. 1Read the AI rationale and evidence quote BEFORE accepting an answer — the QUIPS items are dense and AI can cite the wrong field
  2. 2When AI confidence is low or evidence is sparse, spend extra time on that row
  3. 3For Domain 5: even if CoreSR auto-marks an item Y, manually inspect the per-confounder coverage subpanel — semantic equivalence (e.g. "diabetes" vs "type-2 diabetes") may not be detected by the string-match heuristic
  4. 4Hinge items deserve disproportionate attention — they drive the domain rating
  5. 5For Domain 6, the model-adequacy answer (6.3) depends on the metadata you set in Step 2. If Step 2 is wrong, 6.3 will be wrong. Re-check.
  6. 6QUIPS does not mandate an overall — the domain ratings are what reviewers and downstream consumers should care about. Treat the CoreSR summary as a convenience, not a verdict.
coreSR.AICoreSR

AI-powered evidence synthesis