This is an in progress post. More updates will be coming (soon).

Artificial Intelligence / Machine learning application to other disciplines is an evolving area, including medicine. Many researchers are rushing to use machine learning without a comprehensive understanding. There are many pitfalls and errors in machine learning that even AI practitioners sometimes are not invulnerable. Many reporting guidelines and checklists were developed in an attempt to fix those problems, yet I expect many more to come as more issues appear. Some of them are for general AI research and some of them are specific to a particular field. In this blog post, I compile general information, reporting guidelines and checklists for AI-in-medicine researchers. They might not directly solve the problems, but they can help us keep the issues in concern and give more crucial information to readers. Besides researchers, healthcare stakeholders can use these resources to evaluate AI-in-medicine research. If you find any errors or missing resources, please kindly contact me.

General

State of AI Report 2021

State of AI Report 2022

2021 AI Index Report

2022 AI Index Report

2023 AI Index Report

Guidelines for Human-AI Interaction

Generative AI in scientific paper authoring

Community

Ethics & Society at Hugging Face

Ethical guidelines for developing the Diffusers library

Research

Model Cards for Model Reporting

Datasheets for Datasets

AI in Peer Review

The Use of Generative Artificial Intelligence Technologies is Prohibited for the NIH Peer Review Process

Policy on Use of Generative Artificial Intelligence in the ARCs grants programs 2023

Science funding agencies say no to using AI for peer review

Funders agree on the use of AI tools in funding applications

Government

Europe

EU AI Act

The State of State AI Policy (2021-22 Legislative Session)

Ethics guidelines for trustworthy AI

US

National Security Commission on Artificial Intelligence’s (NSCAI)

NIST AI Risk Management Framework (AI RMF)

An Accountability Framework for Federal Agencies and Other Entities

Thailand

2021 Thailand AI Ethics Guidelines by MDES

2022 AI Ethics Guidelines by NSTDA

2022 AI Ethics Guidelines by NSTDA (Book)

2023 Thailand Artificial Intelligence Guidelines 1.0 (TAIG 1.0) by Faculty of Law, Chulalongkorn University

Reproducibility

Leakage and the Reproducibility Crisis in ML-based Science

Medicine

Minimum information about clinical artificial intelligence modeling: the MI-CLAIM checklist

Protocol for development of a reporting guideline (TRIPOD-AI) and risk of bias tool (PROBAST-AI) for diagnostic and prognostic prediction model studies based on artificial intelligence

Developing a reporting guideline for artificial intelligence-centred diagnostic test accuracy studies: the STARD-AI protocol

Reporting guideline for the early-stage clinical evaluation of decision support systems driven by artificial intelligence: DECIDE-AI

Publishers’ and journals’ instructions to authors on use of generative artificial intelligence in academic and scientific publishing: bibliometric analysis

Specialty

Proposed Requirements for Cardiovascular Imaging-Related Machine Learning Evaluation (PRIME): A Checklist: Reviewed by the American College of Cardiology Healthcare Innovation Council

Standardized Reporting of Machine Learning Applications in Urology: The STREAM-URO Framework

Recommendations on compiling test datasets for evaluating artificial intelligence solutions in pathology

Five critical quality criteria for artificial intelligence-based prediction models

Government

Artificial Intelligence and Machine Learning (AI/ML)-Enabled Medical Devices

FDA-approved A.I.-based algorithms

European Health Data Space

Technical Performance Assessment of Quantitative Imaging in Radiological Device Premarket Submissions

Clinical trials

Reporting guidelines for clinical trial reports for interventions involving artificial intelligence: the CONSORT-AI extension

Guidelines for clinical trial protocols for interventions involving artificial intelligence: the SPIRIT-AI extension

Peer review

Think Again Before Using Generative AI During Peer Review or As You Prepare an Application