Artificial intelligence (AI) can already score parts of a clinical exam. Medical schools and colleges now have to decide what it should be allowed to do in their assessments, and what stays with their examiners.
A decision about whether a learner passes, progresses or is ready to work unsupervised should stay with a person, because that decision has to be owned, explained and open to appeal. Examiners vary, and the evidence below shows by how much. Their decisions can still be questioned, reviewed and appealed, and someone answers for each one.
AI is most useful before the exam. It can give learners far more practice and feedback than examiners have time for, so they come to the decision better prepared.
Can AI mark an OSCE?
Partly. An Objective Structured Clinical Examination (OSCE) station has defined elements, such as whether a history covered the right ground. AI can score those the same way every time.
Consistent isn't the same as correct. In a 2025 study, four large language models scored medical students' OSCE interviews across 10 cases, against 174 scores agreed by experts. The models matched the experts' score exactly between 27% and 44% of the time, and came within one point between 67% and 87% of the time. The model tested for consistency gave almost the same score on every run (Geathers et al., 2025, arXiv).
A tool can give the same answer every time and still give the wrong one. For practice, being close and explaining why is useful: it points a learner at what to work on. For a decision about their career, close isn't good enough. We've written before about why sounding right and being right are different things for a language model.
What an examiner sees that a checklist misses
The parts of a station that are easiest to score consistently are the defined elements: the checklist. Expertise doesn't always show up there.
In one study, 14 clinical clerks, 14 family practice residents and 14 experienced family physicians worked through the same OSCE stations. On examiners' global ratings, the experienced physicians scored significantly better than the clerks and residents. On the checklists, they scored significantly worse. The authors concluded that "binary checklists may not be valid measures of increasing clinical competence" (Hodges et al., 1999, Academic Medicine).
That is the human edge in assessment. An experienced examiner can see whether a consultation hung together, not just whether each item was ticked.
Examiners vary too
It would be easy to argue that people should decide because they get it right. The evidence doesn't support that.
In the United Kingdom's postgraduate clinical examination for physicians, taken by 10,145 candidates, 12% of the variation in marks came from differences between examiners in how lenient or strict they were. Adjusting for those differences would have changed the result for about 4% of candidates (McManus et al., 2006, BMC Medical Education).
So the case for a person making the decision can't rest on accuracy. It rests on something else.
A decision about a learner has to be owned
When a candidate fails, they can ask why. An examiner can explain the judgement, a panel can review it, and the candidate can appeal. Someone is answerable for it. A score from a model that nobody can fully explain doesn't offer any of that.
Regulators have reached the same place. England's qualifications regulator, Ofqual, has told awarding organisations that "AI may not be used as a sole marker in any assessments that are part of regulated qualifications" (Ofqual, 2026). Writing for Ofqual in January 2026, two of its staff put it plainly: AI is "nowhere near ready to take over high stakes marking" (Ofqual, 2026).
In medicine the stakes are higher still. The person passed as ready will go on to see patients unsupervised.
Formative marks and summative decisions
Medical education already separates formative and summative assessment, and that separation shows where AI fits.
| Formative | Summative | |
|---|---|---|
| Purpose | Feedback to help the learner improve | A decision on whether the learner passes or progresses |
| How often | After every practice attempt | A few times in a programme |
| Cost of an error | One attempt, corrected on the next | A delayed career, or a doctor progressing too early |
| Explanation | Tells the learner what to work on | Must be justified and open to appeal |
| Who acts on it | The learner | The institution |
| Role for AI | Marks and feedback on every attempt | The decision stays with an examiner |
AI can do the formative work at a volume no faculty could staff. The AMEE guide on AI in health professions assessment suggests that AI's most appropriate current role may lie in low-stakes, formative assessment (Masters et al., 2025, Medical Teacher).
Where AI helps most: feedback
Feedback is where AI adds most, because it's the part of assessment examiners have least time for. A systematic review of OSCE feedback starts from the observation that "many examiners do not provide helpful or insightful feedback in the text space provided after OSCE stations" (Alsahafi et al., 2023, MedEdPublish).
That isn't a failing of examiners. Stations run to time, and the written comment is often the first thing squeezed. AI can give every learner specific feedback on every practice attempt, as often as they want to practise.
That's the pattern we described in what pro-human AI means in medical education. The AI does the part that doesn't need an expert present each time. The expert's time goes on the judgement only they can make, and the learner arrives at the exam better prepared.
Where Gestalt fits
Gestalt builds consultation practice with formative feedback. A learner works through a realistic consultation and gets specific feedback on how they reasoned and communicated.
That puts our work on the formative side of the line. A learner can get a formative mark and feedback as often as they want. Their practice stays private unless they choose to share it, and an institution sees a cohort-level picture of where its learners are strong and where they need work. Every summative decision, about whether a learner passes or progresses, stays with a human examiner.
The examiner's role doesn't shrink when learners practise more. It becomes the part of assessment that matters most.