A conversational diagnostic artificial intelligence built by Google interviewed 98 patients from home before their urgent primary care appointments at Beth Israel Deaconess Medical Center (BIDMC) in Boston, and no conversation had to be stopped for safety, according to a feasibility study published Oct. 8 in The Lancet. Physicians who read the AI’s summary before the visit said it helped them prepare in three quarters of cases.

The peer-reviewed paper, “Conversational diagnostic artificial intelligence in ambulatory primary care: a prospective feasibility study,” was led by BIDMC physicians working with researchers at Google, and BIDMC announced the results on Oct. 9 in a release carried by Medical Xpress. Google said in a company blog post that it is the company’s first publication in The Lancet’s main title. The study was funded by Alphabet, Google’s parent company, and one of the senior authors, Adam Rodman, was a visiting researcher at Google during part of the study, R&D World reported.

How the study worked

The AI, called the Articulate Medical Intelligence Explorer (AMIE), is a Gemini-based system that Google describes on its research site as its flagship project for AI in clinical reasoning and conversations. Between April and November 2025, 114 English-speaking adults who had booked an urgent visit at BIDMC’s ambulatory primary care clinic were enrolled; 98 completed both the AMIE conversation and the clinician appointment. The design was single-center and single-arm, with no control group.

Every conversation was monitored in real time by a board-certified internal medicine physician who could intervene. Across the 98 encounters, the supervisors noted one hallucination and added clinical clarification in five interactions, BIDMC said. In one case, a provider described the interaction as somewhat harmful because the patient may have become anxious after lymphoma appeared on the list of possible diagnoses.

On diagnosis, AMIE’s list of possibilities included the final diagnosis in 90 percent of cases, a figure given in Google’s blog post; its single top suggestion matched in 56 percent, according to R&D World’s account of the paper. Clinical evaluators rated the AI favorably in 87 to 100 percent of cases across 17 criteria, and patients rated it favorably in 48 to 96 percent of cases across 16 criteria, according to Google’s research summary. Patients gave high marks for listening, explaining information and helping them feel at ease, BIDMC said, but concerns remained about the confidentiality of what they shared and about the chatbot’s honesty and trustworthiness.

Primary care physicians completed post-visit surveys in 60 of the 98 cases, including 44 in which they had reviewed the AMIE transcript or summary beforehand. In 33 of those 44 cases (75 percent) they said it helped them prepare, and in 25 (57 percent) they said it might have changed how they approached the visit.

“AI tools must first be shown to operate safely and acceptably in real care workflows,” said Adam Rodman, director of AI programs at BIDMC’s Carl J. Shapiro Institute for Research and Education and a co-senior author, in the BIDMC release.

Marc L. Cohen, clinical chief of primary care in BIDMC’s Division of General Medicine and the other co-senior author, said the study “serves as an exemplar of how the safety of AI may be assessed prior to clinical implementation.” Co-first author Peter Brodeur, a clinical fellow in cardiovascular medicine at BIDMC, said: “This study helps establish the baseline characteristics of such real-world conversations.”

Why it matters

Much of the evidence for medical chatbots has come from simulated patients or written case vignettes. This is a prospective study with real patients, real appointments and a documented supervision protocol, which gives clinics a template for how to test such tools before placing them in front of patients. The supervision findings, one hallucination and five clarifications in 98 conversations, are a reminder that oversight was needed.

For clinicians, the most practical result may be the pre-visit summary: three quarters of physicians who read one found it useful, and more than half said it might have changed their approach. For patients, attitudes toward AI in health care improved after the conversation and stayed higher after the clinician visit, BIDMC said, though their worries about confidentiality did not go away. Rodman noted in the release that research suggests patient trust in medical AI flows from relationships with their human providers.

The caveats

The authors and Google both describe the work as a feasibility study rather than proof of benefit. It took place at a single clinic, fewer than 100 patients completed the full protocol, there was no control arm, and it measured safety and acceptability rather than health outcomes. A physician was watching every exchange, so the results say nothing about unsupervised use. The study was funded by AMIE’s developer, and one senior author had a research role at Google during the study. Google’s blog post says larger clinical trials are needed before patient-facing AI can be assessed at scale.

This article is for general information and does not replace advice from a qualified clinician. Corrections: email the editor using the address on the Contact page.