Burnout, ProfessionalNCT03746574
Achieving the Quadruple Aim by Reducing Burnout
A study testing Compassion Curriculum in people with Burnout, Professional, run by Providence Health & Services.
Trial summary
To demonstrate that a multimodal intervention can reduce burnout in a multidisciplinary primary care practice while improving other components of the Quadruple Aim including staff engagement, patient experience, productivity and quality. Read more about this study
- Who is it for?
- Any ageHealthy volunteers can take part
- Conditions the study is for
- Burnout, Professional
- What is being tested?
- Compassion Curriculum
- Type of treatment
- Lifestyle or therapy programme
- Could I receive a placebo?
- No, everyone receives an active treatment
- How many people
- 341 participants
- Where
- No sites listed yet
- Study dates
- 1 Jan 2016to 31 Dec 2016
- Phase
- Interventional
- Are there results?
- None posted in the registry
Key requirements
Who can take part, as listed by the sponsor. Only the study team can confirm whether you are eligible, so take this list to your doctor.
- Ages Any age
- All sexes
- Healthy volunteers accepted
- Child, Adult, Older adult
Who may be able to join
- All employees working in selected clinics
Who cannot join
- none
About this study
Official title: Achieving the Quadruple Aim by Reducing Burnout: A Randomized Trial of a Compassion Based Curriculum for Outpatient Teams
To demonstrate that a multimodal intervention can reduce burnout in a multidisciplinary primary care practice while improving other components of the Quadruple Aim including staff engagement, patient experience, productivity and quality.
Read the detailed description
Recruitment
This was a randomized trial of eleven internal medicine and family medicine clinics of Providence Medical Group (PMG) in Portland Oregon. All of the clinics were non-residency clinics. The study was conducted between January 2015 and January 2016. Introductory meetings were held with leadership dyads (medical director and manager) of all 23 clinics in the Portland area to overview the curriculum and outline conditions of study participation. Expectations included: holding the curriculum sessions during usual work hours, endorsement of participation by clinic staff, and that specific individuals would be identified to facilitate sessions for the clinic. Leadership of each clinic, subsequently, discussed the curriculum with the staffs of the clinics and eleven committed to participate and were then randomized. The staffs of the intervention clinics then received the twelve session Strengthening Compassion curriculum. Four of the intervention clinics held the sessions at the beginning of the day and opened clinic 80 minutes late. One clinic held the sessions during an extended lunch hour.
Randomization and Follow-up
Eleven clinics volunteered to participate. (47.8% of eligible clinics) Five clinics were randomly selected to participate in the Strengthening Compassion curriculum and six were selected as controls.
341 individuals participated in the study, including providers, clinical, and administrative staff. The baseline survey was completed by 153 participants in the intervention clinics and 188 participants in the control clinics. 244 of the initial participants completed a survey at the end of the intervention period and 191 at 6 month follow up Participants in the intervention clinics engaged in a 12 session curriculum offered every other week for six months. Each session lasted 80 minutes and all staff at the intervention clinics were expected to participate. A total of 16 hours of experiences were provided.
Each session was facilitated by internal non-physician staff (e.g. social workers, medical assistants, nurses) recruited by clinic leadership. The facilitators received two hours of on-line training in group facilitation, mindfulness meditation, content/exercises for each session, and were provided with a detailed facilitators guide. The facilitators also received one half hour of additional training before each session via networked phone calls and were supported by on-line materials.
Study Outcomes
Outcomes were measured utilizing a fifty one question survey administered at baseline, at the end of the 6-month curriculum and at 6 months follow-up. This survey was a composite of validated surveys that measure compassion, mindfulness, burnout, clinic relations, job satisfaction, and clinical satisfaction. The Compassion component of the survey used the Santa Clara Brief Compassion Scale (SCBCS). Mindfulness was measured with the Mindful Attention Awareness Scale (MAAS). Burnout was measured with the Maslach Burnout Inventory-Human Services Survey for Medical Personnel (MBI-HSS (MP)) comprised of three sub-scales that measure emotional exhaustion, depersonalization, and personal accomplishment. Questions about caregiver roles in the clinic, clinic relations, job satisfaction, clinical satisfaction, time working in the clinic, and time working in healthcare, were derived from a previously utilized survey provided by the Center for Outcomes Research and Education of the Providence Health and Services. Data on caregiver engagement was collected in October 2015 and October 2016 via organization wide survey through Willis-Tower-Watson. Engagement and disengagement are pre-defined categories on the Willis-Tower Watson survey. Patient experience data was collected by Press Ganey as part of their standard survey process. Impact on productivity and quality were assessed from data, routinely collected, in the Providence St. Joseph Health administrative data base.
- Conditions
- Burnout, Professional
- Being tested
- Compassion Curriculum
- Medical subject headings
- Burnout, Professional
What participants receive
What participants receive. In a randomised study each person is assigned to one group by chance.
Compassion CurriculumBehavioural
12 session curriculum offered every other week for six months. Each session lasted 80 minutes and all staff at the intervention clinics were expected to participate. A total of 16 hours of experiences were provided.
| Study group | Role | What they receive |
|---|---|---|
| Intervention Clinics | Experimental | Participants in the intervention clinics experienced a 12 session compassion curriculum intevention offered every other week for six months. Each session lasted 80 minutes and all staff at the intervention clinics were expected to participate. A total of 16 hours of experiences were provided. Receives: Behavioral: Compassion Curriculum |
| Control Clinics | No intervention | Completed baseline, end of curriculum, and 6 month follow up survey. Otherwise no intervention |
What is being measured
The primary outcome is the main question the study is designed to answer.
Change in Burnout Score on Maslach Burnout Inventory
Score on Maslach Burnout Inventory: Maslach Burnout Inventory - Human Services Survey (MBI-HSS) is a 22-item survey that covers 3 areas: Emotional exhaustion (EE), Depersonalization (DP), and Low sense of personal accomplishment (PA). There are multiple questions for each of these subscales and responses are in the form of a frequency rating scale (never, a few times a year or less, once a month or less, a few times a month, once a week, a few times a week, every day).
Measured: Baseline, 6 Months and 12 Months
Change in Compassion: Score on Santa Clara Brief Compassion Scale
Score on Santa Clara Brief Compassion Scale: The Santa Clare Brief Compassion Scale is a brief index that assesses compassion and its link to prosocial behaviors. An example from a 'compassion for humanity' item include ""When I hear about someone (a stranger) going through a difficult time, I feel a great deal of compassion for him or her." Responses are given on a 7-point scale from "Not at all true for me" to "Very true for me."
Measured: Baseline, 6 Months and 12 Months
Change in Mindfulness: Mindful Attention Awareness Score
Mindful Attention Awareness Score: The trait MAAS is a 15-item scale designed to assess a core characteristic of mindfulness, namely, a receptive state of mind in which attention, informed by a sensitive awareness of what is occurring in the present, simply observes what is taking place. Scored on a scale 1=almost never--> 6=almost always
Measured: Baseline, 6 Months and 12 Months
Change in Intent to stay on Caregiver survey
Intent to stay on Caregiver survey
Measured: Baseline and 12 months
Secondary outcomes (7)
Change inCaregiver Engagement measured on annual Willis Tower Watson Survey
Caregiver Engagement measured on annual Willis Tower Watson Survey
Measured: Baseline and 12 months
Change in Patient Experience on routine Press Ganey Survey
Patient Experience on routine Press Ganey Survey
Measured: Baseline, 6 Months and 12 Months
Change in Visit per month
Average number of visits
Measured: Baseline, 6 Months and 12 Months
Change in Panel Size
Panel Size per FTE
Measured: Baseline, 6 Months and 12 Months
Change in Diabetes Control
HbA1c
Measured: Baseline, 6 Months and 12 Months
Change in BP control
% people with blood pressure <140/80
Measured: Baseline, 6 Months and 12 Months
Change in Cancer Screening Rate
Average of rates of screening for breast, colon, and cervical cancer
Measured: Baseline, 6 Months and 12 Months
Timeline
From the start of the study to its completion; the mark is today. Estimated dates are the sponsor’s plan.
- First posted19 Nov 2018
- Study start1 Jan 2016
- Primary completion1 Jul 2016
- Study completion31 Dec 2016
Resources
- Official record on ClinicalTrials.govThe registry entry this page is built from, with its full change history.
- Publications (7)Papers the sponsor has linked to this study.
- Other trials for Burnout, ProfessionalStudies of the same condition, open and under way.
- What the trial phases meanHow a treatment moves from first tests in people to approval.
More details
For doctors and researchers: how the study is designed, who runs it, and its registry IDs.
Study design
- Study type
- Interventional
- Phase
- Phase N/A
- Primary purpose
- Prevention
- Allocation
- Randomised
- Intervention model
- Parallel groups
- Model description
- Randomization to intervention or control occuring at the level of clinic. All participants within a clinic are assigned to same group
- Masking
- None (open label)
- Enrolment
- 341 participants (actual)
Sponsor and investigators
- Lead sponsor
- Providence Health & Services
- Sponsor type
- University, hospital or other
- Responsible party
- Principal investigator, Mark Rosenberg, MD, Providence Health & Services
- Investigators
- Mark Rosenberg, MD, Principal investigator, Providence St. Joseph
IDs, dates and publications
- ClinicalTrials.gov ID
- NCT03746574
- Sponsor's protocol ID
- 15-132
- Status last verified
- Nov 2018
- First submitted
- 14 Nov 2018
- Last updated
- 19 Nov 2018
- Results posted
- No
- Data monitoring committee
- No
- FDA-regulated drug
- No
- FDA-regulated device
- No
- Individual data sharing
- No
Publications
Bodenheimer T, Sinsky C. From triple to quadruple aim: care of the patient requires care of the provider. Ann Fam Med. 2014 Nov-Dec;12(6):573-6. doi: 10.1370/afm.1713.
PubMed 25384822Hall LH, Johnson J, Watt I, Tsipa A, O'Connor DB. Healthcare Staff Wellbeing, Burnout, and Patient Safety: A Systematic Review. PLoS One. 2016 Jul 8;11(7):e0159015. doi: 10.1371/journal.pone.0159015. eCollection 2016.
PubMed 27391946Salyers MP, Bonfils KA, Luther L, Firmin RL, White DA, Adams EL, Rollins AL. The Relationship Between Professional Burnout and Quality and Safety in Healthcare: A Meta-Analysis. J Gen Intern Med. 2017 Apr;32(4):475-482. doi: 10.1007/s11606-016-3886-9. Epub 2016 Oct 26.
PubMed 27785668Panagioti M, Geraghty K, Johnson J, Zhou A, Panagopoulou E, Chew-Graham C, Peters D, Hodkinson A, Riley R, Esmail A. Association Between Physician Burnout and Patient Safety, Professionalism, and Patient Satisfaction: A Systematic Review and Meta-analysis. JAMA Intern Med. 2018 Oct 1;178(10):1317-1331. doi: 10.1001/jamainternmed.2018.3713.
PubMed 30193239West CP, Dyrbye LN, Rabatin JT, Call TG, Davidson JH, Multari A, Romanski SA, Hellyer JM, Sloan JA, Shanafelt TD. Intervention to promote physician well-being, job satisfaction, and professionalism: a randomized clinical trial. JAMA Intern Med. 2014 Apr;174(4):527-33. doi: 10.1001/jamainternmed.2013.14387.
PubMed 24515493West CP, Dyrbye LN, Erwin PJ, Shanafelt TD. Interventions to prevent and reduce physician burnout: a systematic review and meta-analysis. Lancet. 2016 Nov 5;388(10057):2272-2281. doi: 10.1016/S0140-6736(16)31279-X. Epub 2016 Sep 28.
PubMed 27692469Panagioti M, Panagopoulou E, Bower P, Lewith G, Kontopantelis E, Chew-Graham C, Dawson S, van Marwijk H, Geraghty K, Esmail A. Controlled Interventions to Reduce Burnout in Physicians: A Systematic Review and Meta-analysis. JAMA Intern Med. 2017 Feb 1;177(2):195-205. doi: 10.1001/jamainternmed.2016.7674.
PubMed 27918798
Study at a glance
From the registry record, last updated by the sponsor on 19 Nov 2018. Recruitment status is as the sponsor reported it: confirm with the study team.
- Official title
- Achieving the Quadruple Aim by Reducing Burnout: A Randomized Trial of a Compassion Based Curriculum for Outpatient Teams
- ClinicalTrials.gov ID
- NCT03746574
- Acronym / sponsor ID
- 15-132
- Conditions
- Burnout, Professional
- Study type
- Interventional
- Recruitment status
- Completed
- Enrolment
- 341 participants (actual)
- Lead sponsor
- Providence Health & ServicesUniversity, hospital or other
- Study start
- 1 Jan 2016 (actual)
- Primary completion
- 1 Jul 2016 (actual)
- Study completion
- 31 Dec 2016 (actual)
- First posted
- 19 Nov 2018 (actual)
- Results
- Not posted in the source record
Clinical summary
The research question as the sponsor registered it. This describes what the study plans to test, not what it found.
To demonstrate that a multimodal intervention can reduce burnout in a multidisciplinary primary care practice while improving other components of the Quadruple Aim including staff engagement, patient experience, productivity and quality.
Detailed description
Recruitment
This was a randomized trial of eleven internal medicine and family medicine clinics of Providence Medical Group (PMG) in Portland Oregon. All of the clinics were non-residency clinics. The study was conducted between January 2015 and January 2016. Introductory meetings were held with leadership dyads (medical director and manager) of all 23 clinics in the Portland area to overview the curriculum and outline conditions of study participation. Expectations included: holding the curriculum sessions during usual work hours, endorsement of participation by clinic staff, and that specific individuals would be identified to facilitate sessions for the clinic. Leadership of each clinic, subsequently, discussed the curriculum with the staffs of the clinics and eleven committed to participate and were then randomized. The staffs of the intervention clinics then received the twelve session Strengthening Compassion curriculum. Four of the intervention clinics held the sessions at the beginning of the day and opened clinic 80 minutes late. One clinic held the sessions during an extended lunch hour.
Randomization and Follow-up
Eleven clinics volunteered to participate. (47.8% of eligible clinics) Five clinics were randomly selected to participate in the Strengthening Compassion curriculum and six were selected as controls.
341 individuals participated in the study, including providers, clinical, and administrative staff. The baseline survey was completed by 153 participants in the intervention clinics and 188 participants in the control clinics. 244 of the initial participants completed a survey at the end of the intervention period and 191 at 6 month follow up Participants in the intervention clinics engaged in a 12 session curriculum offered every other week for six months. Each session lasted 80 minutes and all staff at the intervention clinics were expected to participate. A total of 16 hours of experiences were provided.
Each session was facilitated by internal non-physician staff (e.g. social workers, medical assistants, nurses) recruited by clinic leadership. The facilitators received two hours of on-line training in group facilitation, mindfulness meditation, content/exercises for each session, and were provided with a detailed facilitators guide. The facilitators also received one half hour of additional training before each session via networked phone calls and were supported by on-line materials.
Study Outcomes
Outcomes were measured utilizing a fifty one question survey administered at baseline, at the end of the 6-month curriculum and at 6 months follow-up. This survey was a composite of validated surveys that measure compassion, mindfulness, burnout, clinic relations, job satisfaction, and clinical satisfaction. The Compassion component of the survey used the Santa Clara Brief Compassion Scale (SCBCS). Mindfulness was measured with the Mindful Attention Awareness Scale (MAAS). Burnout was measured with the Maslach Burnout Inventory-Human Services Survey for Medical Personnel (MBI-HSS (MP)) comprised of three sub-scales that measure emotional exhaustion, depersonalization, and personal accomplishment. Questions about caregiver roles in the clinic, clinic relations, job satisfaction, clinical satisfaction, time working in the clinic, and time working in healthcare, were derived from a previously utilized survey provided by the Center for Outcomes Research and Education of the Providence Health and Services. Data on caregiver engagement was collected in October 2015 and October 2016 via organization wide survey through Willis-Tower-Watson. Engagement and disengagement are pre-defined categories on the Willis-Tower Watson survey. Patient experience data was collected by Press Ganey as part of their standard survey process. Impact on productivity and quality were assessed from data, routinely collected, in the Providence St. Joseph Health administrative data base.
Study design
- Primary purpose
- Prevention
- Allocation
- Randomised
- Intervention model
- Parallel groups
- Masking
- None (open label)
- Number of arms
- 2
- Model description
- Randomization to intervention or control occuring at the level of clinic. All participants within a clinic are assigned to same group
| Arm | Type | Receives |
|---|---|---|
| Intervention Clinics | Experimental | Behavioral: Compassion CurriculumParticipants in the intervention clinics experienced a 12 session compassion curriculum intevention offered every other week for six months. Each session lasted 80 minutes and all staff at the intervention clinics were expected to participate. A total of 16 hours of experiences were provided. |
| Control Clinics | No intervention | –Completed baseline, end of curriculum, and 6 month follow up survey. Otherwise no intervention |
Who may qualify
- Sex
- All
- Healthy volunteers
- Accepted
- Age groups
- Child, Adult, Older adult
Key inclusion criteria (1 listed)
- All employees working in selected clinics
Key exclusion criteria (1 listed)
- none
Eligibility is decided by the study team after screening; this overview does not establish that a patient qualifies. Full inclusion and exclusion criteria.
Interventions
| Intervention | Type | Arms |
|---|---|---|
| Compassion Curriculum12 session curriculum offered every other week for six months. Each session lasted 80 minutes and all staff at the intervention clinics were expected to participate. A total of 16 hours of experiences were provided. | Behavioural | Intervention Clinics |
Primary outcome
Change in Burnout Score on Maslach Burnout Inventory
Time frame: Baseline, 6 Months and 12 Months
Score on Maslach Burnout Inventory: Maslach Burnout Inventory - Human Services Survey (MBI-HSS) is a 22-item survey that covers 3 areas: Emotional exhaustion (EE), Depersonalization (DP), and Low sense of personal accomplishment (PA). There are multiple questions for each of these subscales and responses are in the form of a frequency rating scale (never, a few times a year or less, once a month or less, a few times a month, once a week, a few times a week, every day).
Change in Compassion: Score on Santa Clara Brief Compassion Scale
Time frame: Baseline, 6 Months and 12 Months
Score on Santa Clara Brief Compassion Scale: The Santa Clare Brief Compassion Scale is a brief index that assesses compassion and its link to prosocial behaviors. An example from a 'compassion for humanity' item include ""When I hear about someone (a stranger) going through a difficult time, I feel a great deal of compassion for him or her." Responses are given on a 7-point scale from "Not at all true for me" to "Very true for me."
Change in Mindfulness: Mindful Attention Awareness Score
Time frame: Baseline, 6 Months and 12 Months
Mindful Attention Awareness Score: The trait MAAS is a 15-item scale designed to assess a core characteristic of mindfulness, namely, a receptive state of mind in which attention, informed by a sensitive awareness of what is occurring in the present, simply observes what is taking place. Scored on a scale 1=almost never--> 6=almost always
Change in Intent to stay on Caregiver survey
Time frame: Baseline and 12 months
Intent to stay on Caregiver survey
7 secondary outcomes
Change inCaregiver Engagement measured on annual Willis Tower Watson Survey
Time frame: Baseline and 12 months
Change in Patient Experience on routine Press Ganey Survey
Time frame: Baseline, 6 Months and 12 Months
Change in Visit per month
Time frame: Baseline, 6 Months and 12 Months
Change in Panel Size
Time frame: Baseline, 6 Months and 12 Months
Change in Diabetes Control
Time frame: Baseline, 6 Months and 12 Months
Change in BP control
Time frame: Baseline, 6 Months and 12 Months
Change in Cancer Screening Rate
Time frame: Baseline, 6 Months and 12 Months
Results and safety
The study is completed, but results are not available in the source record. That does not mean the treatment did or did not work.
Research links
- Official record on ClinicalTrials.govLast updated by the sponsor 19 Nov 2018.
- Background: PubMed 25384822Bodenheimer T, Sinsky C. From triple to quadruple aim: care of the patient requires care of the provider. Ann Fam Med. 2014 Nov-Dec;12(6):573-6. doi: 10.1370/afm.1713.
- Background: PubMed 27391946Hall LH, Johnson J, Watt I, Tsipa A, O'Connor DB. Healthcare Staff Wellbeing, Burnout, and Patient Safety: A Systematic Review. PLoS One. 2016 Jul 8;11(7):e0159015. doi: 10.1371/journal.pone.0159015. eCollection 2016.
- Background: PubMed 27785668Salyers MP, Bonfils KA, Luther L, Firmin RL, White DA, Adams EL, Rollins AL. The Relationship Between Professional Burnout and Quality and Safety in Healthcare: A Meta-Analysis. J Gen Intern Med. 2017 Apr;32(4):475-482. doi: 10.1007/s11606-016-3886-9. Epub 2016 Oct 26.
- Background: PubMed 30193239Panagioti M, Geraghty K, Johnson J, Zhou A, Panagopoulou E, Chew-Graham C, Peters D, Hodkinson A, Riley R, Esmail A. Association Between Physician Burnout and Patient Safety, Professionalism, and Patient Satisfaction: A Systematic Review and Meta-analysis. JAMA Intern Med. 2018 Oct 1;178(10):1317-1331. doi: 10.1001/jamainternmed.2018.3713.
- Background: PubMed 24515493West CP, Dyrbye LN, Rabatin JT, Call TG, Davidson JH, Multari A, Romanski SA, Hellyer JM, Sloan JA, Shanafelt TD. Intervention to promote physician well-being, job satisfaction, and professionalism: a randomized clinical trial. JAMA Intern Med. 2014 Apr;174(4):527-33. doi: 10.1001/jamainternmed.2013.14387.
- Background: PubMed 27692469West CP, Dyrbye LN, Erwin PJ, Shanafelt TD. Interventions to prevent and reduce physician burnout: a systematic review and meta-analysis. Lancet. 2016 Nov 5;388(10057):2272-2281. doi: 10.1016/S0140-6736(16)31279-X. Epub 2016 Sep 28.
- Background: PubMed 27918798Panagioti M, Panagopoulou E, Bower P, Lewith G, Kontopantelis E, Chew-Graham C, Dawson S, van Marwijk H, Geraghty K, Esmail A. Controlled Interventions to Reduce Burnout in Physicians: A Systematic Review and Meta-analysis. JAMA Intern Med. 2017 Feb 1;177(2):195-205. doi: 10.1001/jamainternmed.2016.7674.
- Individual participant dataSharing planned: No
Related trials for Burnout, Professional
Recruiting
Professional Readiness Through Emotional Preparedness (PREP)
Burnout
Enrolling by invitationObservational study
Floatation Experience in Nurses and Physicians
Burnout, Professional, Negative Affect
RecruitingObservational study
Burnout Syndrome in the Intensive Care Unit Before and During a Palliative Care Trial
Burnout Syndrome
Recruiting
Ambient AI Scribe (Voa Health) in Outpatient Clinics: Draft Notes, Documentation Burden, and Well-Being
Burnout, Professionals, Medical Records Systems, Computerized
Not yet recruitingObservational study
Physical Activity in Lawyers: Factors Affecting Musculoskeletal Pain
Musculoskeletal Pain, Physical Activity
Referral and contact
AzTrials does not enrol participants or take referrals. Contact the study team or a site listed below; they decide who can take part. This study is not recruiting at the moment.
Mark Rosenberg, MD principal investigator
Providence St. Joseph
Questions about this trial
What is this clinical trial studying?
A study testing Compassion Curriculum in people with Burnout, Professional, run by Providence Health & Services.
The full registered description is in the About this study section and the official record.
Who may be eligible to take part?
The sponsor lists 1 inclusion and 1 exclusion criteria. Eligibility is decided by the study team after screening, so read the full criteria and contact a listed site.
Is this clinical trial currently recruiting?
The record says: finished, as of the sponsor's last update on 19 Nov 2018. Status can change between updates, so confirm with the study contact before making plans.
What are the possible benefits and risks?
They depend on the study and the treatment, and no study can promise a benefit. The study team explains the known risks during informed consent, before anyone joins; the record itself does not list them.
Are results available for this study?
The study is completed, but results are not available in the source record. That does not mean the treatment did or did not work.
How can a doctor refer a patient or contact the research team?
The record lists no contact. The official record names the sponsor, who can be asked directly. AzTrials does not enrol participants or take referrals.
What does the primary outcome measure?
Change in Burnout Score on Maslach Burnout Inventory (time frame: Baseline, 6 Months and 12 Months). The study lists 4 primary outcomes in all. It is what the study was designed to measure, not a finding.
Where can I read the original study information?
On the ClinicalTrials.gov record for NCT03746574, which also lists 7 linked publications.
Does a listing mean the treatment is approved or proven to work?
No. A listing means the study is registered. It does not show that the treatment is safe, works, or is approved by a regulator.
How often is this information updated?
AzTrials copies changed records from ClinicalTrials.gov every night and reloads the whole registry every week. The sponsor last updated this record on 19 Nov 2018.