Peer ReviewNCT03119376
Web-based Tool to Improve the Assessment of Reporting (COBPeer)
A study testing COBPEER in people with Peer Review, run by Assistance Publique - Hôpitaux de Paris at 1 site in France.
Trial summary
The peer review process is a cornerstone of biomedical research publication. Despite being essential, the assessment of the completeness of the reporting and the identification of switched outcomes are not appropriately performed. Read more about this study
- Who is it for?
- Ages 18 years and olderHealthy volunteers can take part
- Conditions the study is for
- Peer Review
- What is being tested?
- COBPEER
- Could I receive a placebo?
- No, everyone receives an active treatment
- How many people
- 120 participants
- Where
- 1 siteFrance
- Study dates
- 10 Dec 2017to 20 Jan 2018
- Phase
- Interventional
- Are there results?
- None posted in the registry
Key requirements
Who can take part, as listed by the sponsor. Only the study team can confirm whether you are eligible, so take this list to your doctor.
- Ages 18 years and older
- All sexes
- Healthy volunteers accepted
- Adult, Older adult
Who may be able to join
- master students,
- PhD students,
- residents involved in clinical research during their study,
- clinicians who have never reviewed a manuscript
Who cannot join
- none
Trial locations and contacts
1 site in 1 country.
About this study
Official title: Evaluation of an Online Tool (COBPeer) Dedicated to Junior Peer Reviewers for Assessing the Reporting of Randomized Controlled Trial Reports
The peer review process is a cornerstone of biomedical research publication. Despite being essential, the assessment of the completeness of the reporting and the identification of switched outcomes are not appropriately performed. Furthermore, these tasks do not suppose a high level of expertise and could be performed by junior researchers.
To assess the completeness of reporting and identified switched outcome(s), junior peer reviewers could use a simple online tool (COBPeer) based on the CONSORT (Consolidated Standards of Reporting Trials) 2010 checklist and Elaboration and Explanation publication for reporting parallel-group RCTs. The tool would feature bullet points eliciting the meaning of each checklist item. The aim of this study that will compare the accuracy of junior peer reviewers using the tool to that of usual peer reviewers when evaluating the completeness of reporting and switched of primary outcome(s) in reports of RCTs
Read the detailed description
BACKGROUND: Context Inadequate reporting is a frequent cause of waste of research. To overcome this issue, the CONSORT statement, an evidence-based, minimum set of recommendations for reporting RCTs was developed in 1996. These guidelines have since been updated in 2001 and more recently in 2010. Many journals endorse the CONSORT statement. Some journals provide recommendations to authors to follow the CONSORT guidelines and some editors enforce the use of the CONSORT guidelines by requesting authors to submit a checklist in either the submission or acceptance stage. Nevertheless, inadequate reporting remains.
HYPOTHESE: To assess the completeness of reporting, junior peer reviewers could use a simple online tool based on the CONSORT 2010 checklist and Elaboration and Explanation publication for reporting parallel-group RCT
OBJECTIVE: 1) Develop an online tool and training module dedicated to junior peer reviewers for a) assessing the completeness of reporting of key items and b) identifying switched primary outcome(s) in reports of RCTs. 2) Compare the performance of junior peer reviewers using this tool with the usual peer-review process in identifying inadequate reporting and switched primary outcome(s) in reports of RCTs.
STUDY DESIGN: randomized ratio 1:1 controlled trial with two parallel arms. Participants will be invited to participate by invitation.
ELIGIBILITY: eligible participants are juniors peer reviewers. Junior peer reviewers are defined as early stage researchers: master students, PhD students, residents involved in clinical research during their study, and clinicians who have never reviewed a manuscript
INTERVENTION: the peer-review tool will be based on 10 items of the CONSORT Statement. This tool reminds the peer reviewer of the CONSORT item which must be reported by explicating it with a series of questions. The training module will explain each item to be evaluated with examples followed by a series of exercises from published randomized trial extracts.
MANUSCRIPT SELECTION: sample of randomized controlled trials published by Annals of Emergency Medicine, British Medical Journal, British Medical Journal Open and BioMed Central series medical.
EVALUATION BY THE JUNIOR PEER REVIEWERS: the junior peer reviewer who has accepted to participate and who has passed the training with success, will have to evaluate 1 article of the selected randomized sample.
EVALUATION OF THE USUAL PEER REVIEWERS: the evaluation by the usual peer-reviewers will be obtained by an analysis of the comments of the peer-reviewers which are available on line or obtained from the publisher.
GOLD STANDARD: peers of researchers who are experts in conducting systematic reviews will evaluate each article independently from other evaluations. Disagreements will be resolved by consensus.
PRIMARY OUTCOME: the mean number of items accurately classified per manuscript
SECONDARY OUTCOMES: the mean number of items accurately classified per manuscript for the 10 CONSORT items; the sensitivity, specificity and likelihood ratio to detect the item as adequately reported and to identify switch in primary outcome(s).
SAMPLE SIZE: 120 manuscripts and 120 peer reviewers juniors.
- Conditions
- Peer Review
- Being tested
- COBPEER
- Keywords
- peer review, online tool, Randomized Controlled Trial
What participants receive
What participants receive. In a randomised study each person is assigned to one group by chance.
COBPEEROther
At the end of the online training, they can participate in the study and so peer review 1 manuscript randomly selected from our sample with our tool (COBPeer).
| Study group | Role | What they receive |
|---|---|---|
| USUAL PEER REVIEWER | No intervention | Two researchers will read all the peer-review reports and editors' comments for the first round. The researchers will determine whether the peer-reviewers and/or editors raised some concern on the completeness of reporting of the 10 CONSORT items considered and identified a switch primary outcome(s) between the manuscript and the register. The assessment of all peer-review reports and editors' comments for each manuscript will be combined (i.e., the item will be rated as incompletely reported if at least one peer reviewer rated it as such). The 2 researchers will be blinded to the gold standard assessment. |
| COBPeer | Experimental | Junior peer reviewers will be invited to participate in an online training course on peer review (COBPeer). Receives: Other: COBPEER |
| GOLD STANDARD | No intervention | Pairs of systematic reviewers will independently extract data from eligible reports. Reviewers involved in the data extraction will have expertise in the conduct of systematic reviews and will assess the completeness of reporting from the systematic reviewer perspective. They will not have access to the tool to avoid being influenced by the tool. The systematic reviewers will also systematically compare the primary outcome(s) reported in the manuscript and the primary outcome(s) reported in the registry and will document any discrepancies. |
What is being measured
The primary outcome is the main question the study is designed to answer.
the mean number of items accurately classified per manuscript
Each item will be classified as"adequately reported" (yes/no)
Measured: through study completion, an average of 6 month
Secondary outcomes (2)
the mean number of items accurately classified per manuscript for the 10 CONSORT items
Each item will be classified as"adequately reported" (yes/no)
Measured: through study completion, an average of 6 month
the sensitivity, specificity and likelihood ratio to detect the item as adequately reported and to identify switch in primary outcome(s).
test for pair proportion
Measured: through study completion, an average of 6 month
Timeline
From the start of the study to its completion; the mark is today. Estimated dates are the sponsor’s plan.
- First posted18 Apr 2017
- Study start10 Dec 2017
- Primary completion20 Jan 2018
- Study completion20 Jan 2018
Resources
- Official record on ClinicalTrials.govThe registry entry this page is built from, with its full change history.
- Publications (2)Papers the sponsor has linked to this study.
- Other trials for Peer ReviewStudies of the same condition, open and under way.
- More trials run by Assistance Publique - Hôpitaux de ParisThe sponsor's other registered studies.
- What the trial phases meanHow a treatment moves from first tests in people to approval.
More details
For doctors and researchers: how the study is designed, who runs it, and its registry IDs.
Study design
- Study type
- Interventional
- Phase
- Phase N/A
- Primary purpose
- Other
- Allocation
- Non-randomised
- Intervention model
- Parallel groups
- Masking
- Single blind
- Who is masked
- Participant
- Enrolment
- 120 participants (actual)
Sponsor and investigators
- Lead sponsor
- Assistance Publique - Hôpitaux de Paris
- Sponsor type
- University, hospital or other
- Collaborators
- University of Oxford, University of Ottawa, University of California, Los Angeles
- Responsible party
- Principal investigator, Isabelle BOUTRON, Assistance Publique - Hôpitaux de Paris
IDs, dates and publications
- ClinicalTrials.gov ID
- NCT03119376
- Sponsor's protocol ID
- AC001
- Status last verified
- Sep 2018
- First submitted
- 3 Apr 2017
- Last updated
- 11 Sep 2018
- Results posted
- No
- Data monitoring committee
- No
- FDA-regulated drug
- No
- FDA-regulated device
- No
- Individual data sharing
- No
Publications
Chauvin A, Ravaud P, Moher D, Schriger D, Hopewell S, Shanahan D, Alam S, Baron G, Regnaux JP, Crequit P, Martinez V, Riveros C, Le Cleach L, Recchioni A, Altman DG, Boutron I. Accuracy in detecting inadequate research reporting by early career peer reviewers using an online CONSORT-based peer-review tool (COBPeer) versus the usual peer-review process: a cross-sectional diagnostic study. BMC Med. 2019 Nov 19;17(1):205. doi: 10.1186/s12916-019-1436-0.
PubMed 31744489Chauvin A, Moher D, Altman D, Schriger DL, Alam S, Hopewell S, Shanahan DR, Recchioni A, Ravaud P, Boutron I. A protocol of a cross-sectional study evaluating an online tool for early career peer reviewers assessing reports of randomised controlled trials. BMJ Open. 2017 Sep 15;7(9):e017462. doi: 10.1136/bmjopen-2017-017462.
PubMed 28918414
Study at a glance
From the registry record, last updated by the sponsor on 11 Sep 2018. Recruitment status is as the sponsor reported it: confirm with the study team.
- Official title
- Evaluation of an Online Tool (COBPeer) Dedicated to Junior Peer Reviewers for Assessing the Reporting of Randomized Controlled Trial Reports
- ClinicalTrials.gov ID
- NCT03119376
- Acronym / sponsor ID
- AC001
- Conditions
- Peer Review
- Keywords
- peer review, online tool, Randomized Controlled Trial
- Study type
- Interventional
- Recruitment status
- Completed
- Enrolment
- 120 participants (actual)
- Lead sponsor
- Assistance Publique - Hôpitaux de ParisUniversity, hospital or other
- Collaborators
- University of Oxford, University of Ottawa, University of California, Los Angeles
- Study start
- 10 Dec 2017 (actual)
- Primary completion
- 20 Jan 2018 (actual)
- Study completion
- 20 Jan 2018 (actual)
- First posted
- 18 Apr 2017 (actual)
- Results
- Not posted in the source record
Clinical summary
The research question as the sponsor registered it. This describes what the study plans to test, not what it found.
The peer review process is a cornerstone of biomedical research publication. Despite being essential, the assessment of the completeness of the reporting and the identification of switched outcomes are not appropriately performed. Furthermore, these tasks do not suppose a high level of expertise and could be performed by junior researchers.
To assess the completeness of reporting and identified switched outcome(s), junior peer reviewers could use a simple online tool (COBPeer) based on the CONSORT (Consolidated Standards of Reporting Trials) 2010 checklist and Elaboration and Explanation publication for reporting parallel-group RCTs. The tool would feature bullet points eliciting the meaning of each checklist item. The aim of this study that will compare the accuracy of junior peer reviewers using the tool to that of usual peer reviewers when evaluating the completeness of reporting and switched of primary outcome(s) in reports of RCTs
Detailed description
BACKGROUND: Context Inadequate reporting is a frequent cause of waste of research. To overcome this issue, the CONSORT statement, an evidence-based, minimum set of recommendations for reporting RCTs was developed in 1996. These guidelines have since been updated in 2001 and more recently in 2010. Many journals endorse the CONSORT statement. Some journals provide recommendations to authors to follow the CONSORT guidelines and some editors enforce the use of the CONSORT guidelines by requesting authors to submit a checklist in either the submission or acceptance stage. Nevertheless, inadequate reporting remains.
HYPOTHESE: To assess the completeness of reporting, junior peer reviewers could use a simple online tool based on the CONSORT 2010 checklist and Elaboration and Explanation publication for reporting parallel-group RCT
OBJECTIVE: 1) Develop an online tool and training module dedicated to junior peer reviewers for a) assessing the completeness of reporting of key items and b) identifying switched primary outcome(s) in reports of RCTs. 2) Compare the performance of junior peer reviewers using this tool with the usual peer-review process in identifying inadequate reporting and switched primary outcome(s) in reports of RCTs.
STUDY DESIGN: randomized ratio 1:1 controlled trial with two parallel arms. Participants will be invited to participate by invitation.
ELIGIBILITY: eligible participants are juniors peer reviewers. Junior peer reviewers are defined as early stage researchers: master students, PhD students, residents involved in clinical research during their study, and clinicians who have never reviewed a manuscript
INTERVENTION: the peer-review tool will be based on 10 items of the CONSORT Statement. This tool reminds the peer reviewer of the CONSORT item which must be reported by explicating it with a series of questions. The training module will explain each item to be evaluated with examples followed by a series of exercises from published randomized trial extracts.
MANUSCRIPT SELECTION: sample of randomized controlled trials published by Annals of Emergency Medicine, British Medical Journal, British Medical Journal Open and BioMed Central series medical.
EVALUATION BY THE JUNIOR PEER REVIEWERS: the junior peer reviewer who has accepted to participate and who has passed the training with success, will have to evaluate 1 article of the selected randomized sample.
EVALUATION OF THE USUAL PEER REVIEWERS: the evaluation by the usual peer-reviewers will be obtained by an analysis of the comments of the peer-reviewers which are available on line or obtained from the publisher.
GOLD STANDARD: peers of researchers who are experts in conducting systematic reviews will evaluate each article independently from other evaluations. Disagreements will be resolved by consensus.
PRIMARY OUTCOME: the mean number of items accurately classified per manuscript
SECONDARY OUTCOMES: the mean number of items accurately classified per manuscript for the 10 CONSORT items; the sensitivity, specificity and likelihood ratio to detect the item as adequately reported and to identify switch in primary outcome(s).
SAMPLE SIZE: 120 manuscripts and 120 peer reviewers juniors.
Study design
- Primary purpose
- Other
- Allocation
- Non-randomised
- Intervention model
- Parallel groups
- Masking
- Single blind (participant)
- Number of arms
- 3
| Arm | Type | Receives |
|---|---|---|
| USUAL PEER REVIEWER | No intervention | –Two researchers will read all the peer-review reports and editors' comments for the first round. The researchers will determine whether the peer-reviewers and/or editors raised some concern on the completeness of reporting of the 10 CONSORT items considered and identified a switch primary outcome(s) between the manuscript and the register. The assessment of all peer-review reports and editors' comments for each manuscript will be combined (i.e., the item will be rated as incompletely reported if at least one peer reviewer rated it as such). The 2 researchers will be blinded to the gold standard assessment. |
| COBPeer | Experimental | Other: COBPEERJunior peer reviewers will be invited to participate in an online training course on peer review (COBPeer). |
| GOLD STANDARD | No intervention | –Pairs of systematic reviewers will independently extract data from eligible reports. Reviewers involved in the data extraction will have expertise in the conduct of systematic reviews and will assess the completeness of reporting from the systematic reviewer perspective. They will not have access to the tool to avoid being influenced by the tool. The systematic reviewers will also systematically compare the primary outcome(s) reported in the manuscript and the primary outcome(s) reported in the registry and will document any discrepancies. |
Who may qualify
- Ages
- 18 years and older
- Sex
- All
- Healthy volunteers
- Accepted
- Age groups
- Adult, Older adult
Key inclusion criteria (4 listed)
- master students,
- PhD students,
- residents involved in clinical research during their study,
- clinicians who have never reviewed a manuscript
Key exclusion criteria (1 listed)
- none
Eligibility is decided by the study team after screening; this overview does not establish that a patient qualifies. Full inclusion and exclusion criteria.
Interventions
| Intervention | Type | Arms |
|---|---|---|
| COBPEERAt the end of the online training, they can participate in the study and so peer review 1 manuscript randomly selected from our sample with our tool (COBPeer). | Other | COBPeer |
Primary outcome
the mean number of items accurately classified per manuscript
Time frame: through study completion, an average of 6 month
Each item will be classified as"adequately reported" (yes/no)
2 secondary outcomes
the mean number of items accurately classified per manuscript for the 10 CONSORT items
Time frame: through study completion, an average of 6 month
the sensitivity, specificity and likelihood ratio to detect the item as adequately reported and to identify switch in primary outcome(s).
Time frame: through study completion, an average of 6 month
Results and safety
The study is completed, but results are not available in the source record. That does not mean the treatment did or did not work.
Research links
- Official record on ClinicalTrials.govLast updated by the sponsor 11 Sep 2018.
- Derived from this study: PubMed 31744489Chauvin A, Ravaud P, Moher D, Schriger D, Hopewell S, Shanahan D, Alam S, Baron G, Regnaux JP, Crequit P, Martinez V, Riveros C, Le Cleach L, Recchioni A, Altman DG, Boutron I. Accuracy in detecting inadequate research reporting by early career peer reviewers using an online CONSORT-based peer-review tool (COBPeer) versus the usual peer-review process: a cross-sectional diagnostic study. BMC Med. 2019 Nov 19;17(1):205. doi: 10.1186/s12916-019-1436-0.
- Derived from this study: PubMed 28918414Chauvin A, Moher D, Altman D, Schriger DL, Alam S, Hopewell S, Shanahan DR, Recchioni A, Ravaud P, Boutron I. A protocol of a cross-sectional study evaluating an online tool for early career peer reviewers assessing reports of randomised controlled trials. BMJ Open. 2017 Sep 15;7(9):e017462. doi: 10.1136/bmjopen-2017-017462.
- Individual participant dataSharing planned: No
Related trials for Peer Review
Recruiting
Artificial Intelligence (AI)-Enhanced Pretreatment Peer-review Process to Improve Patient Safety in Radiation Oncology
Cancer, Prostate Cancer
Not yet recruitingObservational study
Delphi Process to Achieve Global Consensus on Gender Equity in Critical Care
Professional Role, Research Peer Review
Referral and contact
AzTrials does not enrol participants or take referrals. Contact the study team or a site listed below; they decide who can take part. This study is not recruiting at the moment.
The record lists no contact. Use the official record to reach the sponsor.
1 site in 1 country: see every site and its status.
Questions about this trial
What is this clinical trial studying?
A study testing COBPEER in people with Peer Review, run by Assistance Publique - Hôpitaux de Paris at 1 site in France.
The full registered description is in the About this study section and the official record.
Who may be eligible to take part?
The sponsor lists 4 inclusion and 1 exclusion criteria. Eligibility is decided by the study team after screening, so read the full criteria and contact a listed site.
Is this clinical trial currently recruiting?
The record says: finished, as of the sponsor's last update on 11 Sep 2018. Status can change between updates, so confirm with the study contact before making plans.
Where is the study taking place?
The record lists 1 site in France. Whether a site is taking part right now should be confirmed with that site. See every site.
What are the possible benefits and risks?
They depend on the study and the treatment, and no study can promise a benefit. The study team explains the known risks during informed consent, before anyone joins; the record itself does not list them.
Are results available for this study?
The study is completed, but results are not available in the source record. That does not mean the treatment did or did not work.
How can a doctor refer a patient or contact the research team?
The record lists no contact. The official record names the sponsor, who can be asked directly. AzTrials does not enrol participants or take referrals.
What does the primary outcome measure?
the mean number of items accurately classified per manuscript (time frame: through study completion, an average of 6 month). It is what the study was designed to measure, not a finding.
Where can I read the original study information?
On the ClinicalTrials.gov record for NCT03119376, which also lists 2 linked publications.
Does a listing mean the treatment is approved or proven to work?
No. A listing means the study is registered. It does not show that the treatment is safe, works, or is approved by a regulator.
How often is this information updated?
AzTrials copies changed records from ClinicalTrials.gov every night and reloads the whole registry every week. The sponsor last updated this record on 11 Sep 2018.