RecoverCMS

TER076

Exploring and Evaluating Clinician-Generated Prompts for Language Models in Whiplash Care

Edit Back
Project Information
Data collection commenced
Recover Code
TER076
Title
Exploring and Evaluating Clinician-Generated Prompts for Language Models in Whiplash Care
Recover Lead
Joshua Simmich
Co-investigators
Papic C, Ross MHR, Sterling M, Russell T
Status
Data collection commenced
Stream
Technology-enabled rehabilitation + Improving health outcomes after musculoskeletal injury
CTP Relevance
Optimising service models
STARS
Non-STARS
Ethical Clearance Number
--
Plans
No plans

Synopsis

This study has two aims. First, it will explore how musculoskeletal physiotherapists use—or would like to use—chat-based language models to support assessment, diagnosis, and treatment of whiplash. A single 90-minute in-person focus group with a stratified sample of five to six clinicians will surface real-world prompts, scenarios where models help or fall short, patient specificity in queries, and strategies for handling unreliable responses. Draft prompts derived from this discussion will then be emailed back to participants for feedback and refinement. Second, the finalized prompts (approximately 25–30 items ranging from simple yes/no checks to complex case scenarios) will be submitted to leading language models. Two independent physiotherapists will score each response as either “guideline-concordant or not contradictory” versus “contradictory,” with a third rater resolving any disagreements. Expected Outcomes: The project will yield a validated library of clinician-authored prompts organized by assessment, diagnosis, and treatment tasks; qualitative insights into factors driving physiotherapist engagement with language models; and quantitative measures of model alignment with 2024 whiplash guidelines. Findings will guide best practices for safe, effective integration of conversational language models into musculoskeletal care.

Milestones
Name Start Date Due Date Status Date Completed Notes Actions
Semi-structured interviews with physiotherapists 17/12/2025 27/02/2026 In progress
Transcription of interviews 05/01/2026 27/02/2026 In progress
Qualitative data analysis 27/02/2026 13/03/2026 Planned / Scheduled
Development of whiplash prompts 31/03/2026 In progress
Gathering feedback on prompts 01/04/2026 15/04/2026 Not started
Testing prompts against LLMs 16/04/2026 16/04/2026 Not started
Evaluating LLM responses 17/04/2026 30/04/2026 Not started
Quantitative data analysis 17/04/2026 01/05/2026 Not started
Manuscript preparation 15/05/2026 Not started
KT Plan

No KT plan activities recorded yet.

Project Progress
05 Feb 2026 11:13 AM

Currently recruiting and interviewing physiotherapy participants

Linked Grants

No grants linked to this project.

Linked Publications

No publications linked to this project.

Linked Presentations

No presentations linked to this project.

Linked KT Activities

No KT activities linked to this project.

Linked CCI Activities

No CCI activities linked to this project.