by Raphael Yaakov, MS
September 2024
Keywords: Human Cells, Tissues, and Cellular and Tissue-based Products (HCT/P), Cellular and tissue-based products (CTPs), GRADE, Randomized controlled trial (RCT), Evidence-based Medicine, Artificial Intelligence (AI), Bayesian, Data Modeling, Decision Support System, Policy and Reimbursement
The framework offers a structured approach for assessing the quality of the evidence. The framework lends itself to be applied broadly across research and practice, from evaluation of systemic review, clinical guidelines, to public health policies.1 It has served as a cornerstone for guiding clinicians, researchers, and policymakers in evaluating the quality of evidence and formulating recommendations.1-3 The framework also increases transparency, allowing stakeholders to examine criteria and rationale behind decisions; this can also foster trust and facilitates the acceptance of guidelines and policies.
The GRADE framework considers various factors such as study design, risk of bias, inconsistency, imprecision, and publication bias. GRADE can be applied to various types of studies, including randomized controlled trials (RCTs), observational studies, and systematic reviews. The quality of evidence is rated on a four-level scale: high, moderate, low, and very low. Recommendations are classified as either strong or weak, based on the quality of evidence. The simplified classification helps healthcare decision-makers make informed choices swiftly about patient care, balancing benefits, risks, and patient preferences. The overarching goal, of course, is to improve patient care and outcomes.
While the GRADE frame is a valuable tool in various contexts for literature appraisal, it is not without limitations.2-4 Some aspects of the evaluation process, such as judging the risk of bias or the importance of outcomes, can be subjective. Different reviewers may arrive at different conclusions, potentially affecting the consistency of assessments. Even experienced reviewers have low interrater reliability when evaluating complex evidence with varying study design.5 A study evaluating clinical practice guideline development process of American Association of Blood Banking for the use of prophylactic vs. therapeutic platelet transfusion in patients with thrombocytopenia observed only fair interrater agreement on the strength of recommendations.6
Moreover, its application can be complex and time-consuming, requiring a thorough understanding of the methodology and significant effort to assess evidence accurately. Another challenge is the availability and accessibility of evidence and data for evaluation. Often limited information or resources can also make it difficult to assess evidence and reduce generalizability. Last but not least, patient-centered outcomes are not fully captured or quantified, limiting relevance to clinical practice.
Assessing certainty of evidence from diverse sources is essential for relevance and applicability. Consideration of real-world evidence (RWE), observational data along with results from randomized controlled clinical trials (RCTs), for example may provide a bigger picture of a policy related question. Given the implication of recommendations from a policy and reimbursement perspective and impact on patient care and outcomes, gaps and limitations in the GRADE framework need to be critically evaluated. Furthermore, policy decision-makers need to explore new approaches aimed at enhancing literature appraisal and interpretation to standardize and scale evidence-based medicine.
As artificial intelligence (AI) matures and becomes more accessible, it provides a wide bandwidth of tools to extract key findings from scientific literature and enhance data models from multiple sources for a comprehensive evidence base that is continuously updated. Bayesian models, for instance, can integrate new information in a framework and update recommendations, giving policy makers advanced tools to evaluate data in real-time, run scenario analysis and visualize decision pathways. Moreover, uncertainty in evidence may be explicitly quantified, providing confidence in findings. Applying AI for policy decisions represents a transformative approach to increase efficiency and reduce subjectivity and variability in evidence appraisal.
To learn more about exciting developments in AI and current industry insights and trends, follow eKare. eKare is a physician-led digital health and AI company committed to providing unparalleled solutions for advanced wound imaging and data analytics. With combined 50+ years of experience in clinical research, the eKare team has supported over 150+ clinical studies across the globe. As a trusted industry partner, eKare continues to advance wound care technology to improve care delivery and patient outcomes.
Disclaimer
The views and opinions expressed in this article are those of the authors and do not necessarily reflect the views or positions of any entities they represent
About the Author
Raphael Yaakov serves as the VP of Clinical Development at eKare, Inc.. His experience spans across public health, clinical research, and technology. He has helped support key initiatives in patient education programs, global epidemiological surveillance to managing early to late phase drug and device studies. He completed his undergraduate course work in life sciences at the Pennsylvania State University and holds a MS with a concentration in pharmacoeconomics and outcomes research from University of the Sciences in Philadelphia.
References
- Brozek JL, Canelo-Aybar C, Akl EA, et al. GRADE Guidelines 30: the GRADE approach to assessing the certainty of modeled evidence-An overview in the context of health decision-making. J Clin Epidemiol. 2021;129:138-150. doi:10.1016/j.jclinepi.2020.09.018
- Fischer A.J., Threlfall A., Meah S., Cookson R., Rutter H., Kelly M.P. The appraisal of public health interventions: an overview. J Public Health. 2013;35(4):488–494
- Hultcrantz M., Rind D., Akl E.A., Treweek S., Mustafa R.A., Iorio A. The GRADE Working Group clarifies the construct of certainty of evidence. J Clin Epidemiol. 2017;87:4–13.
- Hilton Boon M, Thomson H, Shaw B, et al. Challenges in applying the GRADE approach in public health guidelines and systematic reviews: a concept article from the GRADE Public Health Group. J Clin Epidemiol. 2021;135:42-53. doi:10.1016/j.jclinepi.2021.01.001.
- Susan L. Norris, Lisa Bero. GRADE Methods for Guideline Development: Time to Evolve?. Ann Intern Med.2016;165:810-811. [Epub 20 September 2016]. doi:10.7326/M16-1254.
- Kumar A, Miladinovic B, Guyatt GH, Schünemann HJ, Djulbegovic B. GRADE guidelines system is reproducible when instructions are clearly operationalized even among the guidelines panel members with limited experience with GRADE. J Clin Epidemiol. 2016;75:115-118. doi:10.1016/j.jclinepi.2015.11.020.
