Technical Evaluations of AI Algorithms
Volledige vakbeschrijving
This module is the second module within the learning Data science in Healthcare, and builds on module 1 ‘Data and technology in healthcare’. The module is organised around a real-life clinical example/ problem. Students are trained to obtain and process "rich data" based on relevant (medical) data and data sources in healthcare: to choose and implement machine learning algorithms: to solve clinical or health problems and challenges. Given a specific question, students will first look at what types of data are needed and what requirements must be set to assure good data quality. Based on the fundamentals from Module 1, AI algorithms are discussed and their specific advantages, disadvantages and most appropriate use cases are presented. The focus will be on the thorough understanding of methods as opposed to an exhaustive list of all available algorithms.
Students learn how to select an appropriate artificial intelligence algorithm. Particularly for healthcare settings, limitations and usefulness must be carefully weighted to ensure the best possible outcome while preserving trust by clinicians and patients. Students learn details of various principles of data processing and various tests to determine data quality (such as dealing with missing data and various forms of bias). Concepts such as repeatability, generalisability, transferability, accuracy, reliability, sensitivity and specificity and the difference between training, testing and validating algorithms will become part of the students’ vocabulary. Students learn various validation methods to determine the internal and external validity of AI algorithms and determine the performance on the basis of international reporting standards.
Doelstellingen van dit vak
The specific course objectives are:
Expert:
The student is able to:
- Identify a type of learning problem for realistic clinical application Propose a good first strategy to develop an AI approach for a clinical application
- Judge the reliability of the benchmarks performed for an AI healthcare solution
- Evaluate the trustworthiness of AI claims.
- See how choice for algorithms and training data might produce biases
- Propose solutions to bridge AI ideas to clinical relevance
Investigator
The student is able to:
- Know when advanced algorithms are likely to bring added benefit.
- Interpret differences in various metric for their realistic impact
- Ask critical questions about evaluation strategy and hyperparameter choices