Prime Global logo
IsMpp EU 24
posters Showcase
video Augmented reality
poster commenting Augmented reality
poster avatar
Home

Assessment of reading age of published plain language summary abstracts

Valerie Mossa, Jon Hoggardb and Brian Normana
aPrime, London, UK; bPrime, Auckland, New Zealand

Table of contents

Conclusions

  • Contrary to current guidance, most plain language summary abstracts available in PubMed Central had reading age >12 years.
  • Inclusion of a patient author reduced the average reading age of the plain language summaries.
  • Publication professionals have a key role in supporting authors to develop accessible plain language summaries.
Top

Introduction

  • Plain language summaries (PLS) are important to support accessibility of clinical data to as wide an audience as possible.
  • A recent survey of format preferences of PLS in patients with chronic diseases (psoriasis, multiple sclerosis, or rheumatoid arthritis) suggests a preference for graphical summaries or mediumcomplexity text.1
  • There are also various guidelines for preparation of PLS. For greatest impact and equity for patients, the language and writing style of a PLS needs to be clear and understandable across a range of ages and education levels.
    • EU regulations advocate aiming for a reading age of 12 years and upwards.2
    • ClinicalTrials.gov suggests aiming for a reading level of 6th–8th grade (approximate age range 11-14 years). 3
  • We assessed PLS abstracts available on PubMed Central (PMC) to determine reading age and whether they followed these guidelines.
Top

Methods

  • We searched PMC for publications tagged for data-mining and identified as having a PLS or plain English summary abstract.
    • The search was performed in July 2023 and restricted to a date range of 1 February 2019 to 31 December 2022.
  • To measure readability, we used the Flesch-Kincaid calculator to estimate Reading Ease and Grade Level scores for each abstract.
    • Simple Measure of Gobbledygook (SMOG) and Automated Readability were used to validate results.
    • The Python library was used to calculate each of these statistics from the abstract text (https://pypi.org/project/textstat).
  • We assessed whether journal choice (where there were ≥3 abstracts per journal) had an impact on estimated readability score.
  • We also assessed whether inclusion of a patient author affected estimated readability score.
Top

Results

  • In total, 1053 PMC records met the search criteria between February 2019 and December 2022.
  • A recent survey of format preferences of PLS in patients with chronic diseases (psoriasis, multiple sclerosis, or rheumatoid arthritis) suggests a preference for graphical summaries or mediumcomplexity text.1
  • There are also various guidelines for preparation of PLS. For greatest impact and equity for patients, the language and writing style of a PLS needs to be clear and understandable across a range of ages and education levels.
    • EU regulations advocate aiming for a reading age of 12 years and upwards.2
    • ClinicalTrials.gov suggests aiming for a reading level of 6th–8th grade (approximate age range 11-14 years). 3
  • We assessed PLS abstracts available on PubMed Central (PMC) to determine reading age and whether they followed these guidelines.
Figure 1: Flesch-Kincaid Reading Ease and Grade Level scores for the 1053 abstracts
This chart shows a mean Flesch-Kincaid reading ease score of 35.4, a minimum of -8.1 and maximum of 72.4. It also shows a mean Flesch-Kincaid grade level score of 13.7, a minimum of 6.7 and maximum of 25.6.
  • In total, 1042 (99%) of the PLS abstracts were estimated to have a reading age of 8th grade or higher (≥13 years) and 1021 (97%) were estimated to have a reading age of 9th grade or higher (≥14 years).
  • SMOG score (mean 15.0; range 0–22.9) and Automated Readability score (mean 17.7; range 6.0–117.4) gave similar results.
  • In total, 1042 (99%) of the PLS abstracts were estimated to have a reading age of 8th grade or higher (≥13 years) and 1021 (97%) were estimated to have a reading age of 9th grade or higher (≥14 years).
  • SMOG score (mean 15.0; range 0–22.9) and Automated Readability score (mean 17.7; range 6.0–117.4) gave similar results.
Figure 2: Flesch-Kincaid reading Grade Level score for those journals with three or more PLS abstracts
This chart shows the mean Flesch-Kincaid reading Grade Level across a range of journals with a wide range of mean values with the best having a mean reading Grade Level of 10.
Figure 3: Impact of patient authors on Grade Level
This chart shows that having a patient author is likely to lower the Flesch-Kincaid reading Grade Level significantly.
Top

Limitations

  • Consistent with standard practice for research of this type, only articles that were tagged for data-mining within PMC were assessed. It is possible that different results might have been obtained with wider inclusion criteria.
  • We restricted the search to a 3-year period finishing in December 2022. As education increases on ensuring appropriate readability of PLS, accessibility for a broad audience may improve.
  • Given PLS have multiple target audiences, it is possible that some journals recommend higher reading age for their PLS.
Top
References
  1. Martínez Silvagnoli L et al. J Med Internet Res. 2022;24:e22122
  2. European Medicines Agency. Summaries of clinical trial results for laypersons. Available at https://health.ec.europa.eu/system/files/2020-02/2017_01_26_summaries_of_ct_results_for_laypersons_0.pdf. Accessed 2 January 2024.
  3. Plain Language Checklist for Lay Brief Summaries. Available at https://prsinfo.clinicaltrials.gov/Plain_Language_Checklist_for_Lay_Brief_Summaries.pdf. Accessed 2 January 2024
Acknowledgements

We would like to thank the Prime Production and Editor teams for their support with developing this poster.

Disclosures

Brian Norman and Valerie Moss are employees of Prime, London, UK. Jon Hoggard is an employee of Prime, Auckland, New Zealand.

Top
Supplementary materials
Top