AzTrials
AzTrials

Clinical Decision-MakingADAM's APPLENCT05657002

A Study to the Impact of Accuracy Problem Lists in Electronic Health Records on Correctness and Speed of Clinical Decision-making Performed by Dutch Healthcare Providers

A study testing patient A with accurate problem list, patient B with inaccurate problem list and patient A with inaccurate problem list in people with Clinical Decision-Making and Decision Making, Computer-assisted, run by Eva Klappe at 1 site in Netherlands.

Completed

Finished. The study has ended and is not enrolling. Results may be posted on the official record.

Official record
Search all trials

Trial summary

The primary objective of this study is to determine whether patient records with complete, structured and up-to-date problem lists ('accurate problem lists'), result in better clinical decision-making, compared to patient records that convey the same information in a less structured way where the problem list has… Read more about this study

Who is it for?
Any ageHealthy volunteers can take part
What is being tested?
patient A with accurate problem list, patient B with inaccurate problem list and patient A with inaccurate problem list
Could I receive a placebo?
No, everyone receives an active treatment
How many people
160 participants
Where
1 siteNetherlands
Study dates
1 Dec 2022to 21 Dec 2022
Phase
Interventional
Are there results?
None posted in the registry

Key requirements

Who can take part, as listed by the sponsor. Only the study team can confirm whether you are eligible, so take this list to your doctor.

  • Ages Any age
  • All sexes
  • Healthy volunteers accepted
  • Child, Adult, Older adult

Who may be able to join

  • Healthcare professionals who are allowed to prescribe medication, thus hold a position as: medical specialist, medical resident, nurse specialist or physician assistant, research-specialists
  • Healthcare professionals must have followed at least the 'basic EHR Epic course'. This electronic health record course lasts for three days and includes how to send letters, register diagnoses in a record, request testing, all in the software system EPIC, which concludes with an exam on the theory.

Who cannot join

  • Non-Dutch speaking employees as the patient cases and the exercises are described in Dutch

Trial locations and contacts

1 site in 1 country.

Netherlands1 site
  • Amsterdam UMC, Location AMC

    Amsterdam, North Holland, 1105AZ

Other trials in Netherlands

About this study

Official title: a Randomized Controlled Trial Study to Determine the Impact of Accuracy of Problem Lists in Electronic Health Records on Clinical Decision-making

The primary objective of this study is to determine whether patient records with complete, structured and up-to-date problem lists ('accurate problem lists'), result in better clinical decision-making, compared to patient records that convey the same information in a less structured way where the problem list has missing and/or duplicate diagnoses ('inaccurate problem lists'). The secondary objective is to determine whether the time required to make a correct decision is less for patient records with accurate problem lists compared to patient records with inaccurate problem lists.

Read the detailed description

A problem list in Electronic Health Records (EHRs) is considered an essential feature in the collection of structured data. The problem list provides a centralized summary of each patient's medical problems and these problems or diagnoses are selected from the terminology underlying the problem list, such as SNOMED CT. If well maintained and structured, the problem list is a valuable tool for reviewing records of (unfamiliar) patients as it quickly shows the required information when needed. While studies have shown that the use of structured formats can serve as prompt for extra details, greater consistency of information and clinical decision-making there is little evidence whether a patient record with complete and structured problem lists results in more accurate and faster clinical decision making.

In the study (ADAM's APPLE: Adequate Data registration And Monitoring, subproject: Accurate Presentation of Problem List Elements), the investigators will perform a crossover randomized controlled trial in which a laboratory experiment will be performed among individual healthcare professionals to assess the impact of patient records with accurate and inaccurate problem lists on clinical decision-making. The participants will be presented with two records of two different patients in a training environment of the software system EPIC, one of them with an accurate problem list and the other that conveys the complete information in the patient record (in free text notes) but with an inaccurate problem list with missing diagnoses and duplicate information. The participants do not know which of the two records includes the accurate problem list and which record includes the inaccurate problem list. Participants are asked to decide whether or not to prescribe two medications for those two patients. One medication is not allowed per patient because the patient is allergic to that medication, which is documented on the allergy list. For the first patient record, the other medication is not allowed, because of a contraindicated diagnosis and for the second patient record the other medication is not allowed, because of a side effect that has occurred using that medication in the medical history. Based on the correctness of the motivation for correct answers and the time to the right answers, the research question if accurate problem lists in patient records lead to better and faster decision-making is answered.

Prior to this study, two healthcare professionals in the research team determined suitable use cases and questions for this study. These use cases were based on real-world unstructured versions of patient records. Two optimized accurate problem lists were also created for both patient records, which was defined according to the problem list policy at our institution (i.e. all current active problems and relevant medical history should be documented on the problem list).

Being tested
patient A with accurate problem list, patient B with inaccurate problem list, patient A with inaccurate problem list, patient B with accurate problem list

What participants receive

What participants receive. In a randomised study each person is assigned to one group by chance.

  • patient A with accurate problem listOther

    A problem list that contains a diagnosis code that is contraindicated with a type of medication (Y). Also, all other relevant diagnoses and medical history for the patient are up-to-date on the problem list, which was defined according to the problem list policy at our institution (i.e. all current active problems and relevant medical history should be documented on the problem list). Additionally, the problem list is included in eight out of thirteen notes using so-called smart phrases that can automatically import a (part of a) problem list. One note includes the problem list with the diagnosis relevant for the question asked.

  • patient B with inaccurate problem listOther

    A problem list that does not contain the diagnosis code and corresponding details explaining medical history of this diagnosis caused by a type of medication (Y). Additionally, the problem list is included in three out of thirteen notes using so-called smart phrases that can automatically import a (part of a) problem list. The relevant diagnosis is not documented on the problem list and hence is not included in the imported problem list in the notes.

    The expert panel provided and anonymized two real-world representative examples of hematology patient records that included inaccurate problem lists and that had many free-text notes. An 'inaccurate problem list' is defined as a problem list where diagnoses are missing resulting in missed trigger medication or order-alerts, where diagnoses are 'active' although they should be closed or removed and/or where the problem list contained duplicated diagnoses.

  • patient A with inaccurate problem listOther

    A problem list that does not contain the diagnosis code that is contraindicated with the type of medication (Y). Additionally, the problem list is included in eight out of thirteen notes using so-called smart phrases that can automatically import a (part of a) problem list. The relevant diagnosis is not documented on the problem list and hence is not included in the imported problem list in the notes.

    The expert panel provided and anonymized two real-world representative examples of hematology patient records that included inaccurate problem lists and that had many free-text notes. An 'inaccurate problem list' is defined as a problem list where diagnoses are missing resulting in missed trigger medication or order-alerts, where diagnoses are 'active' although they should be closed or removed and/or where the problem list contained duplicated diagnoses.

  • patient B with accurate problem listOther

    A problem list that contains the diagnosis code and corresponding details explaining medical history of this diagnosis caused by a type of medication (Y). Also, all other relevant diagnoses and medical history for the patient are up-to-date on the problem list, which was defined according to the problem list policy at our institution (i.e. all current active problems and relevant medical history should be documented on the problem list). Additionally, the problem list is included in three out of thirteen notes using so-called smart phrases that can automatically import a (part of a) problem list. One note includes the problem list with the diagnosis and details relevant for the question asked.

Study groups in NCT05657002
Study groupRoleWhat they receive
accurate problem list, then inaccurate problem listActive comparatorin round 1, the participants will use the patient record of patient A, with an accurate problem list and answer the question: can the patient be prescribed Medication X and Y where medication X is a control question and medication Y is related to a contraindicated diagnosis (on problem list) In round 2, the participants will use the patient record of patient B, with an inaccurate problem list and answer the question: can the patient be prescribed Medication X and Y where medication X is a control question and medication Y is related to medical history (not on problem list)
Receives: Other: patient A with accurate problem list; Other: patient B with inaccurate problem list
inaccurate problem list, then accurate problem listActive comparatorin round 1, the participants will use the patient record of patient A, with an inaccurate problem list and answer the question: can the patient be prescribed Medication X and Y where medication X is a control question and medication Y is related to a contraindicated diagnosis (not on problem list) In round 2, the participants will use the patient record of patient B, with an accurate problem list and answer the question: can the patient be prescribed Medication X and Y where medication X is a control question and medication Y is related to medical history (on problem list)
Receives: Other: patient A with inaccurate problem list; Other: patient B with accurate problem list

What is being measured

The primary outcome is the main question the study is designed to answer.

  1. the correctness of the answer of medication B including the right motivation

    Measured using a questionnaire showing the two question(s) per patient record on a separate tablet during the experiment. Two Yes/No questions per patient record (so a total of four questions per participant) are answered. Answers are stored using an automatically assigned unique anonymized identifier. The question for both patient records is: "can the patient be prescribed medication A and/or B?". Medications A and B can both not be prescribed, but medication B is related to the problem list diagnoses and medication A is related to the allergy list which is the same for both versions of the patient records. A motivation is required per Yes/No answer to determine the correctness of the answer and prevent from a chance of gambling. An independent researcher from the research team who did not perform the experiments will categorize the motivation of the answers for medication B.

    Measured: during the experiment/questionnaire

Secondary outcomes (4)
  1. the total time to answer the two questions correctly, where the answer of medication B also includes the right motivation

    The time to enter the motivation was not measured. However, the participants might give the right answer but the wrong motivation. For this secondary outcome measure, only the answers that have the right motivation are considered as 'right answer'.

    Measured: during the experiment/questionnaire

  2. the total time to answer the two questions correctly including the right motivations for medication A and B

    A time stamp is registered when the participant opens the question, and a time stamp is registered when the participant confirms the answers. A time stamp is therefore registered for the total of two Yes/No answers per patient record.

    Measured: during the experiment/questionnaire

  3. the correctness of both the answer for medication A and B including the right motivations for A and B

    Measured using a questionnaire showing the two question(s) per patient record on a separate tablet during the experiment. Two Yes/No questions per patient record (so a total of four questions per participant) are answered. Answers are stored using an automatically assigned unique anonymized identifier. The question for both patient records is: "can the patient be prescribed medication A and/or B?". Medications A and B can both not be prescribed, but medication B is related to the problem list diagnoses and medication A is related to the allergy list which is the same for both versions of the patient records. A motivation is required per Yes/No answer to determine the correctness of the answer and prevent from a chance of gambling. An independent researcher from the research team who did not perform the experiments will categorize the motivation of the answers for medication A and B.

    Measured: during the experiment/questionnaire

  4. the correctness of the answer for medication A including the right explanation

    Measured using a questionnaire showing the two question(s) per patient record on a separate tablet during the experiment. Two Yes/No questions per patient record (so a total of four questions per participant) are answered. Answers are stored using an automatically assigned unique anonymized identifier. The question for both patient records is: "can the patient be prescribed medication A and/or B?". Medications A and B can both not be prescribed, but medication B is related to the problem list diagnoses and medication A is related to the allergy list which is the same for both versions of the patient records. A motivation is required per Yes/No answer to determine the correctness of the answer and prevent from a chance of gambling. An independent researcher from the research team who did not perform the experiments will categorize the motivation of the answers for medication A.

    Measured: during the experiment/questionnaire

Timeline

From the start of the study to its completion; the mark is today. Estimated dates are the sponsor’s plan.

  1. First posted20 Dec 2022
  2. Study start1 Dec 2022
  3. Primary completion21 Dec 2022
  4. Study completion21 Dec 2022

Resources

More details

For doctors and researchers: how the study is designed, who runs it, and its registry IDs.

Study design
Study type
Interventional
Phase
Phase N/A
Primary purpose
Health services research
Allocation
Randomised
Intervention model
Crossover
Model description
Participants will be presented with 2 patient records in EPIC, one with an accurate problem list and the other that conveys the complete information in the patient record (in free text notes) but with an inaccurate problem list with missing diagnoses and duplicates. Each participant answers a total of 4 questions for 2 separate patient cases (2 questions per case), one of those cases having the inaccurate and the other having the accurate problem list. Using 2 separate patient cases rather than one was decided to prevent a memory effect, i.e., participants would have had time to understand the patient's conditions described in the record which could impact the time required to answer questions in round 2. Additionally, the order in which patients cases are provided is not randomized, since in practice it also happens that a new patient is followed up by another new patient which requires professionals to systematically go through two separate records with little time in between.
Masking
Single blind
Who is masked
Participant
Enrolment
160 participants (actual)
IDs, dates and publications
ClinicalTrials.gov ID
NCT05657002
Sponsor's protocol ID
Amsterdam UMC 2019-AMC-JK-7
Other IDs
2019-AMC-JK-7
Acronym
ADAM's APPLE
Status last verified
Dec 2022
First submitted
22 Nov 2022
Last updated
23 Dec 2022
Results posted
No
Data monitoring committee
No
FDA-regulated drug
No
FDA-regulated device
No
Individual data sharing
No

Questions about this trial

What is this clinical trial studying?

A study testing patient A with accurate problem list, patient B with inaccurate problem list and patient A with inaccurate problem list in people with Clinical Decision-Making and Decision Making, Computer-assisted, run by Eva Klappe at 1 site in Netherlands.

The full registered description is in the About this study section and the official record.

Who may be eligible to take part?

The sponsor lists 2 inclusion and 1 exclusion criteria. Eligibility is decided by the study team after screening, so read the full criteria and contact a listed site.

Is this clinical trial currently recruiting?

The record says: finished, as of the sponsor's last update on 23 Dec 2022. Status can change between updates, so confirm with the study contact before making plans.

Where is the study taking place?

The record lists 1 site in Netherlands. Whether a site is taking part right now should be confirmed with that site. See every site.

What are the possible benefits and risks?

They depend on the study and the treatment, and no study can promise a benefit. The study team explains the known risks during informed consent, before anyone joins; the record itself does not list them.

Are results available for this study?

The study is completed, but results are not available in the source record. That does not mean the treatment did or did not work.

How can a doctor refer a patient or contact the research team?

The record lists no contact. The official record names the sponsor, who can be asked directly. AzTrials does not enrol participants or take referrals.

What does the primary outcome measure?

the correctness of the answer of medication B including the right motivation (time frame: during the experiment/questionnaire). It is what the study was designed to measure, not a finding.

Where can I read the original study information?
Does a listing mean the treatment is approved or proven to work?

No. A listing means the study is registered. It does not show that the treatment is safe, works, or is approved by a regulator.

How often is this information updated?

AzTrials copies changed records from ClinicalTrials.gov every night and reloads the whole registry every week. The sponsor last updated this record on 23 Dec 2022.

Free account

Log in to keep searching

You’ve used your 3 free searches. AzTrials is free: log in or create a free account to keep searching clinical trials and save the ones you want to follow.

Create a free accountLog in