This project evaluates the use of Microsoft 365 Copilot for structured data extraction in the context of systematic literature review. Following a team workshop on AI prompting techniques, we developed standardised prompts targeting demographic, methodological, and clinical variables — including study design, participant characteristics, and treatment details — and pilot-tested them on 19 randomly selected articles, comparing AI outputs against manually extracted data.
Initial testing showed that citation- or title-based prompting produced unreliable results, including fabricated responses. Through iterative refinement — increasing prompt specificity, clarifying context, adding fallback instructions (e.g., "if not stated, report as NA"), and providing Copilot direct access to full-text articles via in-browser viewing or PDF upload — extraction accuracy improved substantially, with final prompts achieving less than 10% discrepancy versus manual extraction.
The project also documents key limitations of AI-assisted extraction, including false positives, false negatives (particularly from tables and supplementary material), and hallucinations, offering practical guidance for research teams considering AI tools in evidence synthesis.
Project supervised by — Dr Mohammed Moinuddin
Lecturer in Biostatistics
School of Medicine and Dentistry
Ongoing project
AI-Assisted Data Extraction for Systematic Reviews: Evaluating Microsoft Copilot for Literature Review Synthesis
A pilot project testing and refining AI prompting strategies (zero-shot, few-shot, chain-of-thought) with Microsoft 365 Copilot to automate structured data extraction from full-text research articles, achieving less than 10% discrepancy against manual extraction.
