Writing and reviewing abstracts for academic meetings are essential skills for any academic pediatrician. An accurate and reliable peer review process is critical for fair determination of which abstracts are accepted for presentation at these meetings. Unfortunately, studies of abstract review practices across many fields in medicine have found that there is often poor concordance in scores among abstract reviewers,1–5 with some scores being so divergent that they are no different from what would be expected if assigned randomly.6,7 However, several studies have examined the development of abstract and journal article rating criteria and have found that the use of a standardized scoring process is feasible and can lead to better interrater agreement.8–15 The highest levels of interrater reliability occur with the more objective criteria such as those focusing on methods and analysis.3,11,15
The objectives of this commentary are to provide a guide on how to write an abstract for an academic meeting and an example of a structured abstract scoring system with relative weights of importance for the criteria, with the highest weight placed on methods and analysis. We will first discuss the key components that should be present in individual sections of the abstract (title, Background and Objectives, Methods and Analysis, Results, and Conclusions); the components that are part of our abstract scoring system are summarized in Table 1. We will then elaborate on additional criteria that can be incorporated into an abstract scoring system by using examples from the guidelines that have been used by the Academic Pediatric Association (APA) in their mentored abstract review process with fellows for the Pediatric Academic Societies (PAS) meetings since the fall of 2011. The 7-point scale and some of the guidelines were designed based on the scoring system outlined in the PAS Abstract Review Instructions.16
Assessing Individual Components of the Abstract
Although not used in our scoring system to rate an abstract, the title of an abstract is important to capture the attention and the interest of the reader. Effective titles often use the active voice or ask a question that the abstract will answer. The title should use the fewest words possible to describe the study accurately and emphasize the strengths of the study (eg, multicenter or randomized controlled trial). Titles should also avoid use of nonstandard abbreviations.
Background and Objectives
The Background and Objectives sections should familiarize the reader with the importance of the topic, the gap in knowledge, and the research question. For example, an abstract on inpatient asthma exacerbations would provide background on the prevalence of asthma followed by what makes this study different from past studies. The Background section, which should be no more than a few sentences long, should justify the need for the new study and lay the groundwork for the objectives.
The Objectives section should follow the Background section and clearly state a specific research question in concise, specific language. The objectives for a quantitative study should include predictor and outcome variables (eg, “the effect of resident work hour changes on handoff errors”) or precise content of a survey. The objectives for a qualitative study should clearly specify the phenomena the study describes. This section should also prepare the reader to evaluate the Methods section because the research design will be based on the research question.
Most abstracts submitted to national meetings have a word limit of ∼250 words. Therefore, the background and objectives should be succinct to leave room for the majority of the abstract to focus on research design, methods, analysis, and results.
Methods, Analysis, and Results
Methods and Analysis
The Methods section is the most important section of the abstract and should provide enough details to clearly describe the setting, study design, research participants, sampling procedures, interventions, data collection, and analysis. The section should be as succinct as possible and highlight the key parts of the methods. The authors must convince the reader that the study design is most appropriate for the research question being asked. For example, a randomized controlled trial is often the most appropriate type of study to assess the effect of an intervention. Observational studies are best for assessing harm or risk (eg, it would not be ethical to randomize someone to smoke cigarettes). Qualitative studies are suitable to generate hypotheses or identify the points of view of people with certain experiences. In addition to choosing the most appropriate type of study, it is important to describe the research participants and the sampling methods.
The abstract reviewer should scrutinize the methods and analysis to ensure the study’s reliability and validity (credibility and trustworthiness for qualitative studies). For example, most surveys should have validity evidence; all should be designed to answer the research question. Predictor and outcome variables must be operationalized (and therefore measured in a way as to avoid information bias) and make sense in the context of the objectives. Quantitative studies must measure and account for all relevant confounding variables. Subjects should be chosen in such a way as to avoid selection bias. The reviewer also must assess the appropriate use of data analyses (eg, the use of a regression when multiple confounders are present, strategies for trustworthiness for qualitative analysis). In addition, different categories of studies have specific criteria that should be assessed to determine whether the most rigorous methods were used for that study design. For example, clinical trials should be designed so patients are randomly assigned and the study is controlled and blinded. See Table 2 for a list of rigorous and appropriate methods specific to common study types. Specific texts17,18 offer an in-depth look at different types of study designs. Checklists exist for the evaluation of full articles for multiple study designs, but only a few have been designed specifically for the appraisal of conference abstracts (eg, STROBE19 for observational studies, CONSORT20 for randomized controlled trials, PRISMA21 for systematic reviews). These checklists may be helpful in both writing and reviewing abstracts with relevant study designs.
The Results section should have a succinct summary of important results and analyses presented in a logical fashion (eg, from descriptive to bivariate to multivariate). Key statistical measures, such as P values and confidence intervals, should be presented. In general, confidence intervals are preferred because they provide a clearer picture of what statistical significance means in practical terms. Themes, relationships between themes, and key interpretations should describe the results of a qualitative study. The results should address all objectives. A figure or table could also clearly summarize findings.
The Conclusions section should concisely summarize the results of the study without stretching the conclusions beyond the actual findings. This section may end with a sentence that considers potential broader implications of the research, such as possible effects on practice, policy, or education.
Additional Components Used for Scoring Abstracts
After assessing the individual components of an abstract, a reviewer must evaluate the importance of the topic addressed in the abstract. Topics that would be particularly appealing and of interest to the conference attendees should receive a higher rating. The potential impact of the research should be evident, including whether it will influence the delivery of health services, clinical care, education of trainees, research endeavors, or public policy. For example, a study of a new serum marker that is highly sensitive and specific for early detection of neonatal sepsis and that is also inexpensive would get a high score for importance given the potential widespread use and clinical impact.
Abstracts should also be judged on the originality of the question, design, or results. Asking and answering questions that have been presented year after year makes abstracts unexciting to reviewers and conference attendees. For topics that have been researched extensively, studies should present new insights. Research questions should differ enough from previous studies that they add new information to the field. Studies of common conditions could receive points for originality if they examine the condition in a different population, study a much larger sample than previous studies, or challenge existing practice.
Quality of Presentation
Abstracts must be assessed for overall quality, because even the most novel studies with the most rigorous methods can appear flawed if the reader cannot make sense of the writing or abstract structure. Sentences should be written in a clear and easy-to-understand manner. The author should adhere to all submission guidelines and follow standard guidelines for presenting numerical or qualitative results and statistical or qualitative analyses. All content must be in the correct section of the abstract (eg, no results in the Methods section or methods in the Results section). The abstract should be consistent when referring to variables and other terms. Finally the author should avoid vague words (eg, “substantially”) and instead present statistical results or clear themes and meaningful interpretations.
The scoring system discussed in this section has been used by the APA’s mentored review process for abstracts submitted for PAS meetings. Table 1 presents key components to be assessed for each scoring component. Table 3 summarizes the relative weights for the scoring system. The Methods, Analysis, and Results sections receive the most weight in our scoring system, with 3 points out of a possible 7, because the study’s design, implementation, and data analysis procedures indicate whether a study is valid or credible and can be replicated. Previous studies of abstract scoring have shown that emphasis on the Methods section contributes more to selection of clinical abstracts for presentation than other characteristics,15 and higher interrater agreement occurs with scoring based on review of the methods rather than more subjective criteria.3,11,15 Methods have also been weighted higher in other abstract scoring systems.13 In comparison, the background and objectives are allotted just 0.5 points because they should be straightforward and have fewer implications for study validity or credibility and importance of results. The importance of a study has been assigned up to 2 points because even a well-designed study may not merit presentation at a conference, especially a platform or plenary session, if it is of little interest to conference attendees or has few implications for practice. Finally we have assigned ≤1 point for originality and 0.5 points for quality of presentation. No points are assigned for the title and conclusions because their content is generally reflected in other parts of the total score.
The total points are then added together and subtracted from 8 to give the actual score with a range of 1 to 7, with 1 being the best. These scores generally relate to the disposition of the abstract (eg, plenary, platform, poster, or not presented). Final scores can be modified slightly at the discretion of the reviewer to reflect his or her overall judgment or major imbalances among the scoring components. For example, topics of very high interest to conference attendees may be given slightly higher scores even with somewhat imperfect methods. On the other hand, a reviewer may lower the score of an abstract on a novel topic with potential high interest if it is poorly written and there is concern that the poor presentation represents less than rigorous performance of the study. In general, these adjustments should be minor, and the individual components of the score should reflect these concerns. Finally, as stated in the PAS Abstract Review Instructions, it is important to use the entire scale of scores from 1 to 7 when reviewing a pool of abstracts.16 Assigning too many to the same score (usually in the middle of the range, such as mostly scores of “3” or “4”) without using the extremes of the scale can make it difficult to differentiate the quality of the abstracts and make reliable decisions on disposition.
In this commentary we have presented the key components of an abstract and an example of guidelines for a structured abstract scoring system. These scoring guidelines have been used in the APA mentored abstract review process for fellows in training for the last 4 years, with positive feedback from fellows and faculty. Use of standardized abstract and journal article rating guidelines can help to reduce some of the subjectivity associated with the abstract review process and can lead to improved concordance among abstract reviewers.8–15 These guidelines should provide a useful framework for junior faculty and trainees participating in the abstract review process and also for investigators writing their first abstracts.
Dr Glick drafted the initial manuscript and edited drafts of this manuscript; Drs Szilagyi, Freed, Hanson, and Dreyer designed the initial version of the abstract review guidelines and edited drafts of this manuscript; and all authors approved the final manuscript as submitted.
FINANCIAL DISCLOSURE: The authors have indicated they have no financial relationships relevant to this article to disclose.
FUNDING: No external funding.
POTENTIAL CONFLICT OF INTEREST: The authors have indicated they have no potential conflicts of interest to disclose.
- Academic Pediatric Association
- Pediatric Academic Societies
- Bhandari M,
- Templeman D,
- Tornetta P
- Rothwell PM,
- Martyn CN
- Hasbahceci M,
- Basak F,
- Uysal O
- ↵Pediatric Academic Societies. 2014 Pediatric Academic Societies & Asian Society for Pediatric Research Joint Meeting Abstract Review Instructions. 2014;(281). Available at: www.pas-meeting.org/reviewers/reviewerguidelines_2014.pdf. Accessed December 6, 2015
- Hulley SB,
- Cummings SR,
- Browner WS,
- Grady DG,
- Newman TB
- Patton MQ
- ↵STROBE. Draft STROBE checklist for abstracts. 2011. Available at: http://strobe-statement.org/fileadmin/Strobe/uploads/checklists/STROBE_checklist_conference_abstract_DRAFT.pdf. Accessed December 6, 2015
- Copyright © 2016 by the American Academy of Pediatrics