Machine Learning for Healthcare - MIT 6.S897

MIT 6.S897 Machine Learning for Healthcare, Spring 2019 - YouTube

This course provides a comprehensive, end-to-end exploration of the application of machine learning to solve critical challenges in modern healthcare. The curriculum is designed to move beyond theoretical algorithms to address the unique complexities and high-stakes nature of the medical domain. The course begins by establishing a strong foundation in the practice of medicine, the structure of clinical data, and the history of AI in healthcare. It then delves into core predictive tasks, teaching students how to build models for risk stratification, survival analysis, and diagnosis using a wide array of data modalities, including structured electronic health records, physiological time series, clinical notes (NLP), and medical imaging like echocardiography and digital pathology.
Moving from prediction to intervention, the course introduces advanced frameworks for causal inference and reinforcement learning, equipping students to ask "what if?" and learn optimal sequential treatment policies from observational data. A significant portion of the curriculum is dedicated to unsupervised learning for scientific discovery, focusing on data-driven methods for disease progression modeling and the identification of novel patient subtypes, with a look toward the future of precision medicine and genomics.
Finally, the course confronts the critical real-world challenges of deployment and ethics. It provides a pragmatic view on navigating the regulatory landscape (FDA, IRB), integrating models into complex clinical workflows, and ensuring algorithmic robustness against dataset shift. The curriculum concludes with a deep dive into the essential topics of fairness, to ensure models do not perpetuate societal biases, and interpretability, to build the trust necessary for clinical adoption. Students emerge with not only the technical skills to build sophisticated models but also the contextual and ethical understanding required to translate these powerful technologies into safe, effective, and equitable clinical practice.

L1: Introduction

This lecture serves as an introduction to the Spring 2019 "Machine Learning for Healthcare" course, delivered by Professor David Sontag from MIT's Computer Science and Institute for Medical Engineering and Science. The lecture outlines the fundamental problems in healthcare, the historical context of AI in medicine, the reasons why the present is an opportune time for transformation, and the unique challenges and goals of applying machine learning in this domain.
The Problem: Transforming Healthcare
Professor Sontag begins by highlighting the critical issues plaguing the U.S. healthcare system: exorbitant costs, late diagnosis and poor management of chronic diseases, and frequent medical errors. He personalizes these systemic failures with poignant stories from his own family. His grandfather's late diagnosis of Alzheimer's disease caused preventable family distress. More tragically, his mother passed away from multiple myeloma, a blood cancer, without ever starting treatment. Her condition was misjudged as low-risk based on standard blood markers, but the cancer silently caused fatal heart failure. These anecdotes underscore the urgent need for better tools to transform healthcare, with machine learning and AI being a crucial piece of the puzzle.
A Brief History of AI in Medicine
The lecture revisits the long history of AI in medicine, which dates back to the 1960s and 70s. Key early examples include:

• MYCIN (1970s): Developed at Stanford, this system aimed to identify infectious bacteria and recommend appropriate therapies. It famously outperformed infectious disease experts in proposing effective treatments in 69% of cases and even featured a rudimentary dialog interface.
• INTERNIST-1 (1980s): Created in Pittsburgh for primary care diagnosis, this system used a large, manually-created knowledge base (taking 15 person-years to build) with over 40,000 connections between diseases and symptoms to determine a differential diagnosis.
• Data-Driven Discovery (1980s): At Stanford, a data-driven approach using a rheumatoid arthritis disease registry led to the novel discovery that the steroid prednisone elevates cholesterol levels.
• Neural Networks (1990s): The 90s saw a rise in studies using neural networks for various medical problems like breast cancer and heart attack prediction.

Despite their early promise and impressive capabilities, these systems failed to be integrated into clinical practice. The primary reasons for this failure were their inability to fit into existing clinical workflows, the difficulty and cost of maintaining the manually-curated knowledge bases, and the poor generalizability of models when transferred to different institutions.
The Opportunity Now: A Confluence of Factors
Professor Sontag argues that, after decades of limited success, the time is finally right for a machine learning revolution in healthcare. This optimism is driven by three major developments:

1. Data Availability: The widespread adoption of Electronic Medical Records (EMRs) in the U.S., largely spurred by a $30 billion investment from the 2009 economic stimulus package, has created a massive repository of digital health data. This includes publicly available research datasets like the MIMIC database from MIT (containing rich, de-identified ICU data) and large-scale insurance claims databases like Truven MarketScan. National initiatives like the "All of Us" program are further expanding these data resources, encompassing everything from clinical notes and imaging to genomics and data from wearable devices.
2. Data Standardization: Decades of work have led to the creation of standardized vocabularies and data models. These include ICD-10 for diagnoses, LOINC for lab tests, and ontologies like the Unified Medical Language System (UMLS). Modern standards like FHIR (Fast Healthcare Interoperability Resources) and common data models like OMOP are making it easier to exchange and aggregate data from different systems, enabling the development of scalable, portable machine learning algorithms.
3. Breakthroughs in Machine Learning: Recent advances in the field provide the necessary tools to analyze this complex data. Key developments include algorithms for learning with high-dimensional features, progress in unsupervised and semi-supervised learning (critical for dealing with limited labeled data), and the power of modern deep learning architectures like Convolutional Neural Networks (CNNs) and Recurrent Neural Networks (RNNs).

This convergence has sparked massive industry interest, with billions of dollars being invested by tech giants like Google and IBM, as well as hundreds of startups, all aiming to leverage AI in healthcare.
How Machine Learning Will Transform Healthcare
The lecture outlines the healthcare ecosystem—comprising patients, providers (doctors, hospitals), and payers (insurance companies)—and provides concrete examples of how machine learning can intervene at different points.

• In the Provider Space (e.g., the Emergency Department):
  • Clinical Decision Support: Algorithms can analyze patient data in real-time to suggest the most relevant clinical guidelines or "pathways" for conditions like cellulitis, helping to standardize and improve care.
  • Anticipating Needs: By predicting a patient's condition (e.g., chest pain), the system can proactively surface relevant "order sets" for tests and interventions, streamlining the clinician's workflow.
  • Automated Diagnosis: Using deep learning on imaging data (like chest X-rays to detect pneumonia) or physiological signals (like EKGs to find arrhythmias) can reduce the need for specialist consults and democratize access to high-quality diagnostics.
  • Improving Data Quality: A system deployed at Beth Israel Deaconess uses supervised learning to predict a patient's "chief complaint" from triage notes, offering a standardized list that improves the quality of collected data for future research and quality control.
• Managing Chronic Disease:
  • Disease Progression Modeling: Machine learning can create more accurate models to predict the long-term trajectory of chronic diseases like kidney disease or cancer.
  • Precision Medicine: This is a key application where models could predict how a specific patient will respond to different treatment strategies. Using the example of multiple myeloma, an algorithm could forecast a patient's disease burden under Treatment A versus Treatment B, guiding clinicians toward the optimal choice. This is framed as a causal inference problem.
  • Better Tracking and Measurement: Technologies like MIT's "Emerald" system use wireless signals to monitor elderly patients at home and detect falls. In diabetes management, ML can better predict future glucose levels based on diet and activity, leading to a more effective "artificial pancreas."
• Scientific Discovery: The same data can be used to discover new subtypes of diseases (e.g., using clustering to find different types of asthma), identify new drug targets, and design new medicines.

What's Unique and Challenging About ML in Healthcare
The lecture concludes by emphasizing that healthcare is not just another domain for applying standard machine learning techniques. It presents a unique set of challenges:

• High Stakes: Decisions can be a matter of life or death, demanding algorithms that are robust, safe, fair, and accountable.
• Data Challenges: The field is characterized by a scarcity of labeled data, making unsupervised and semi-supervised learning crucial. Many key questions are causal in nature (e.g., "what treatment works best?"). Data is often missing or "censored" (e.g., a patient leaves a study before the outcome is observed), requiring specialized methods.
• Logistical Hurdles: Strict patient privacy regulations make data access difficult and time-consuming. Furthermore, deploying new algorithms is a major challenge due to the closed nature of commercial EMR systems, creating a significant gap between research and clinical practice.

The course aims to equip students with the intuition and technical skills to navigate these challenges, formalize healthcare problems as machine learning tasks, and understand the subtleties of safely deploying these powerful technologies.

L2: Overview of Clinical Care

Introduction: The Goals and Practice of Medicine
Peter Szolovits, a long-time researcher in medical AI, begins by explaining that the lecture's purpose is to provide context for the course by exploring the fundamental goals of healthcare and the day-to-day practice of medicine. This understanding is crucial for identifying the right problems for machine learning to solve. He contrasts historical and modern medicine, from a tribal shaman emphasizing the healer-patient relationship to a high-tech Intensive Care Unit (ICU), noting that the human element of trust and the placebo effect remain vital.
Part 1: The Goals of Healthcare - Defining and Measuring Health
The lecture first tackles the broad goal of achieving "health," defined by the World Health Organization as "a state of complete physical, mental, and social well-being, not merely the absence of disease or infirmity."

• Longevity and Life Expectancy:
  • The most straightforward measure of health is how long people live. Szolovits presents striking data showing a dramatic increase in global life expectancy, from an average of 25-40 years in 1800 to over 70 in many countries today.
  • He highlights that these gains are not just in wealthy nations; even the lowest-performing countries have seen significant improvements. However, he also points to acute crises that can cause sharp drops, such as the genocide in Rwanda and the government's initial failure to address the HIV crisis in South Africa.
  • An analysis of death rates by age reveals a typical pattern: a small peak in mortality at birth, a dip in early childhood, a rise during the teenage years due to risk-taking behavior, a stable period in young adulthood, and an inexorable rise from middle age onward.
• Health Disparities: A critical issue is the persistent gap in life expectancy based on socioeconomic and racial factors. In the U.S., Black individuals consistently have a lower life expectancy than their White and Hispanic counterparts, a fact that drives many public health and policy initiatives.
• Causes of Death and Morbidity:
  • The leading causes of death in the modern era are chronic, non-infectious diseases like heart disease and cancer. This marks a major shift from previous centuries when infectious diseases were dominant.
  • Beyond mortality, the lecture addresses morbidity—the prevalence of chronic conditions that diminish quality of life. As people age, they accumulate ailments like arthritis, hypertension, and hearing impairment.
• Quality of Life and "Compression of Morbidity":
  • Szolovits introduces the concept of evaluating life not just by its length but by its quality, often quantified through "Activities of Daily Living" (ADLs) like bathing, dressing, and eating.
  • He presents a crucial forward-looking question: What is the ideal outcome of medical progress? He contrasts three scenarios:
    1. Extending Life without Improving Health: This leads to a larger population of people living longer but in a state of disability.
    2. Compression of Morbidity: The ideal scenario where people live a long, healthy life and then experience only a short period of illness before death.
    3. Lifespan Extension: Simply stretching out the current health trajectory over a longer period.

Part 2: The Practice of Medicine - A Cyclic Process
This section delves into how clinicians work, generating the data that machine learning models will use.

• Traditional Tasks: Medicine is traditionally broken down into three tasks:
  • Diagnosis: Figuring out what is wrong with the patient.
  • Prognosis: Predicting the likely course of the disease.
  • Therapy: Deciding on a course of action to treat the patient.
• The Cyclic Model of Care: Szolovits argues that medicine is not a one-shot process but a continuous cycle:
  • A patient presents with a complaint.
  • The doctor examines the patient, gathering data and forming an initial hypothesis about the diagnosis.
  • Based on this hypothesis, a therapy is planned and administered.
  • The doctor observes the patient's response.
  • If the outcome differs from what was expected, it provides a strong clue to revise the diagnosis, and the cycle repeats.
    This process is analogous to the scientific method: form a hypothesis, run an experiment, observe the results, and refine the theory.
• The Learning Healthcare System:
  • The concept of a "learning healthcare system" is introduced, where new knowledge is generated as a natural byproduct of patient care.
  • This is contrasted with the traditional gold standard for medical evidence: the Randomized Controlled Trial (RCT). While powerful, RCTs have significant limitations: they often use "pure" patient cases that don't reflect the complex reality of patients with multiple conditions (comorbidities), and they are designed to be as short and small as possible for financial reasons.
  • Learning directly from routine medical data is a promising alternative, but it faces a fundamental challenge: the lack of randomization. Doctors make treatment decisions based on what they believe is best for the patient, introducing significant bias that complicates causal analysis.

Part 3: Public Health and Epidemiology
The focus shifts from individual patient care to the health of entire populations.

• History of Public Health Data: The collection of public health data dates back to the 17th-century "London Bills of Mortality." This evolved into the systematic classification of diseases, which today is managed through the International Classification of Diseases (ICD) system.
• The Story of John Snow and Cholera: A classic example of epidemiological investigation. In the 1850s, Dr. John Snow mapped cholera deaths in London and traced the outbreak to a single contaminated water pump on Broad Street. By simply removing the pump handle, he stopped the epidemic, demonstrating the power of data-driven public health intervention.
• Quarantine: A more controversial public health tool, quarantine has been used throughout history, from Ellis Island to the Ebola outbreak. It is now considered a last resort due to the significant risks to human rights.

Part 4: Paying for Healthcare - The Economic Context
The final section addresses the economic realities that shape healthcare and create incentives for using AI.

• Insatiable Demand and High Costs: Healthcare costs are driven by a near-insatiable demand (everyone wants to live longer, healthier lives) and the continuous development of expensive new technologies and drugs.
• The U.S. as an Outlier: The United States spends a significantly higher percentage of its GDP on healthcare than any other developed nation, yet it achieves worse outcomes (e.g., lower life expectancy) than many countries that spend far less.
• Attempts at Cost Control:
  • Managed Care: Practices like requiring pre-approval for procedures were introduced in the 1980s and 90s to control costs. They largely failed to curb overall spending and were deeply unpopular with doctors.
  • Capitation: Paying providers a flat fee per patient per year to manage their care. This shifts financial risk to the provider, creating an incentive to keep patients healthy but also a dangerous incentive to under-treat.
  • Modern Reforms (e.g., ObamaCare): These efforts try to leverage better IT and payment models like Accountable Care Organizations and penalties for hospital readmissions to improve efficiency and control costs.

Szolovits concludes by noting that money determines much of what is possible in healthcare, and that historically low IT spending in the field has hindered the kind of data-driven progress this course aims to foster.

L3: Deep Dive into Clinical Care

Introduction: The Treachery of Data and the MIMIC Database
The lecture begins with a powerful cautionary tale using the MIMIC-III database, a rich dataset of ICU admissions from the Beth Israel Deaconess Medical Center. The core lesson is to know your data before you analyze it, as it is filled with artifacts and hidden complexities.
Szolovits presents a histogram of heart rates from the database, which surprisingly shows a bimodal (two-peaked) distribution. This is highly unusual for physiological data. The investigation reveals the two peaks correspond to two different data collection systems used over the 12-year period:

• CareVue (Old System): Shows the bimodal distribution.
• MetaVision (New System): Shows a standard, single-peaked distribution.

The mystery is solved by examining the age of the patients. The CareVue system was used in the Neonatal Intensive Care Unit (NICU), while MetaVision was not. The second, higher heart rate peak was simply the data from babies, who have naturally faster heart rates than adults.
A second artifact is revealed: a cluster of patients with a recorded age of 300 years. This is not an error but a result of the HIPAA privacy rule, which prohibits recording the specific age of any patient 90 years or older. To de-identify the data, all such patients are assigned an age of 300. This illustrates how external rules and data collection practices can create bizarre and misleading patterns in the data.
A Catalog of Medical Data Types
The lecture provides a comprehensive overview of the different kinds of data available in a typical Electronic Health Record (EHR), many of which are present in MIMIC.

• Demographics: Basic patient information like age, sex, socioeconomic status, insurance type, language, and ethnicity.
• Vital Signs: Standard measurements taken upon admission, such as weight, height, pulse, respiration rate, and temperature.
• Medications: A complex category that includes prescriptions, over-the-counter drugs, and even illegal substances (if admitted). A major challenge is medication reconciliation—figuring out what drugs a patient is actually taking, as official records are often inaccurate.
• Lab Tests: Measurements from samples of blood, urine, spinal fluid, bone marrow, etc. These can be quantitative (e.g., potassium level) or qualitative (e.g., presence of "atypical lymphocytes").
• Pathology: Examination of tissue samples, such as from biopsies or surgeries, to identify disease characteristics.
• Microbiology: Identifying infectious organisms in the body and testing their sensitivity to different antibiotics.
• Input/Output: Tracking the balance of fluids going into and coming out of a patient to manage hydration.
• Notes: A vast and rich source of unstructured text data, including:
  • Discharge Summaries: Long, comprehensive reports written at the end of a hospital stay.
  • Notes from doctors, nurses, and specialists.
  • Emergency department notes and referral letters.
• Billing Data: Highly structured data, often using codes like ICD-9/10, designed primarily for billing insurance companies but also used for research.
• Imaging Data: X-rays, CT scans, MRIs, and photographs, which are increasingly being analyzed by machine learning models.
• Quantified Self: Data from personal devices like activity trackers and glucose monitors, reflecting a growing trend of patient-generated data.

Deep Dive into MIMIC Data: Examples and Challenges
Szolovits uses MIMIC data to illustrate the properties and pitfalls of these data types.

• Demographics and Simple Prediction: An analysis of MIMIC demographics shows interesting correlations. For instance, patients with private insurance are much younger on average, as Medicare covers those over 65. A simple logistic regression model to predict in-hospital mortality using only demographics finds that age is the most significant predictor, underscoring its importance in risk assessment.
• The Chaos of Standards: The lecture highlights the extreme difficulty of data harmonization due to a lack of universal standards.
  • Medications: A single drug can be identified by a hospital's internal formulary code, a commercial code (GSN), the FDA's National Drug Code (NDC), and others. Even the NDC system is complicated by formatting changes over time. The same medication might appear multiple times with different codes (e.g., D5W with and without an NDC code). Some "medications" in the database are actually billing items like a "bag" or "vial."
  • Procedures: Different systems are used to code procedures. ICD-9 is used for billing and epidemiology, while CPT (Current Procedural Terminology) is a highly detailed, copyrighted system owned by the American Medical Association.
  • Lab Tests: The LOINC (Logical Observation Identifiers Names and Codes) system is a successful example of a standard that provides unique, hierarchical codes for virtually every lab test, greatly aiding in data aggregation.
• Chart Events and Data Volume: Chart events are bedside observations, mostly entered by nurses. There are millions of these in MIMIC (e.g., over 5 million heart rate measurements). The sheer volume and frequency of data collection in the ICU are immense, with thousands of chart entries per admission.

Case Study: Diurnal Variation in Lab Tests
Szolovits attempts to replicate a study on whether lab test results become more abnormal at night.

• The Confounding Variable: The initial hypothesis is that there's a physiological (diurnal) reason for this. However, the obvious confounder is that patients who need their blood drawn at 3 a.m. are likely sicker than those who have it done during routine daytime rounds.
• Analysis in MIMIC: An analysis of white blood cell counts in MIMIC does show a higher fraction of abnormal results at night. However, when trying to build a predictive model for mortality based on the time of the test, the results are noisy and inconclusive. The signal is weak, and there isn't enough data in MIMIC for this specific test to draw a strong conclusion, demonstrating the difficulty of performing robust research even with a large dataset.

The Power of Clinical Notes
The lecture concludes by focusing on the immense value of unstructured text notes.

• Human Interpretation: Notes contain a level of human understanding, context, and synthesis that is absent from raw lab data or vital signs. In an exercise where researchers tried to predict patient mortality, they found the notes to be the most useful source of information.
• Types and Lengths of Notes: Discharge summaries are the longest and most comprehensive, often running to tens of thousands of characters. Nursing and doctor's notes are typically shorter, providing snapshots of the patient's condition at a point in time.
• Example of a Discharge Summary: A multi-page discharge summary is shown to illustrate its structure and richness. It includes:
  • Chief complaint and history of present illness.
  • Surgical procedures performed.
  • Past medical, social, and family history.
  • Physical exam findings and pertinent lab results.
  • A detailed summary of the hospital course.
  • Discharge medications, disposition (where the patient is going), and follow-up instructions.

Final Takeaways and Future Standards
The lecture wraps up with a discussion of emerging standards and the overarching lessons learned.

• New Standards: FHIR (Fast Health Interoperability Resources) is a modern, simplified standard designed to make it easier to exchange essential health information between different systems. It is being widely adopted, partly due to pressure from the U.S. Congress.
• Key Lessons:
  1. Know Your Data: You must understand how and why data was collected to avoid being misled by artifacts.
  2. Harmonization is Difficult: Merging data from different sources is a massive, time-consuming challenge.
  3. Data Cleaning is a Major Task: A significant portion of any data science project in healthcare (often 50% or more) is spent on cleaning and preparing the data before any modeling can begin.

L4: Risk Stratification 1

The lecture begins by defining risk stratification as the process of separating a patient population into categories of risk (e.g., high, low, medium). Unlike diagnosis, which has strict performance criteria, risk stratification is "fuzzier" and its primary goal is to couple predictions with interventions, often to prevent a negative outcome and reduce healthcare costs.
Key Differences from Diagnosis:

• Goal: To identify groups for intervention, not to assign a definitive label.
• Performance Metrics: Often focuses on metrics like Positive Predictive Value (PPV)—of the patients flagged as high-risk, what percentage actually are?
• Data Used: Tends to be more diverse, incorporating demographics and even socioeconomic information that might be excluded from a purely clinical diagnosis.

Examples of Risk Stratification:

1. Infant Morbidity: The traditional method is the Apgar score, a simple scoring system used at birth. Modern machine learning approaches, like one from Johns Hopkins, aim to provide more accurate predictions.
2. Coronary Care Admission (A 1984 Case Study): An early example of using logistic regression to predict which ER patients with heart-related issues truly needed expensive admission, demonstrating that the core idea is not new.
3. Hospital Readmission: A major focus in modern U.S. healthcare, driven by government penalties for hospitals with high 30-day readmission rates. The goal is to identify at-risk patients before discharge so that interventions (e.g., better discharge instructions, follow-up nurse visits) can be implemented.

Part 2: Old vs. New Approaches

• Traditional Approach: Relies on manually calculated scoring systems (like Apgar or the FINDRISC diabetes score) based on a small number of variables. These are well-researched but often underutilized because they are not integrated into clinical workflows.
• Modern Machine Learning Approach: Leverages high-dimensional data automatically extracted from EHRs.
  • Advantages: Can be integrated into workflows, potentially offers higher accuracy by using more features, and is much faster to develop and deploy for new, specific outcomes.
  • Dangers: Introduces new risks and biases that must be carefully managed.

Part 3: Case Study - Early Detection of Type 2 Diabetes
This section details a real-world application of ML-based risk stratification.

• The Problem: An estimated 25% of type 2 diabetes cases in the U.S. are undiagnosed. The goal is to find this at-risk population using existing data so that preventative interventions can be offered.
• The Data: The model uses administrative data from a health insurance company (a "payer"), which includes:
  • Medical claims (diagnoses, procedures).
  • Pharmacy records (medications).
  • Lab test results (a crucial and not always available data source for payers).
• ML Formulation: The problem is framed as a binary classification task.
  • Setup: The model uses a patient's historical data (e.g., from 2007-2009) to predict if they will be newly diagnosed with type 2 diabetes in a future window (e.g., 2010-2012).
  • The Time "Gap": A gap is intentionally left between the feature data and the prediction window to avoid label leakage. A clinician might already suspect a patient is diabetic, making a short-term prediction trivial and not actionable. The model aims to find unexpected, longer-term risk.
  • Handling Censoring: The model must account for patients entering or leaving the insurance plan. Left-censoring (missing past data) is handled by using whatever data is available. Right-censoring (missing future outcomes) is handled in this study by excluding patients, a simplification that Sontag notes could introduce bias.
• Algorithm and Features:
  • Model: L1-regularized logistic regression is used because it induces sparsity—it automatically selects a smaller subset of the most predictive features from a very high-dimensional space. This aids in interpretability and makes the model easier to deploy in new settings that might not have all the original features.
  • Feature Engineering: Features are cleverly designed to handle missing data. Instead of using lab values directly, the model uses binary indicators like "was the test ever administered?", "was the result ever high/low?", and "was the value increasing/decreasing?". These features are computed over different time windows (last 6 months, 24 months, etc.) and combined into a 42,000-dimensional feature vector.
• Evaluation and Results:
  • Interpretability: The model identified known predictors like "impaired fasting glucose" but also less obvious ones like "obstructive sleep apnea," which likely acts as a surrogate for obesity, a condition that is often not explicitly coded in claims data.
  • Performance Metric: The model is evaluated using Positive Predictive Value (PPV) at K. This measures the precision of the top K predictions, which directly corresponds to the efficiency of an intervention with a limited capacity.
  • Outcome: The ML model significantly outperformed a traditional, simpler model, identifying a much higher percentage of true future diabetics in both its top 100 and top 10,000 predictions.

Part 4: Guest Discussion on Real-World Implementation (Leonard "Len" D'Avolio)
Len D'Avolio, CEO of the risk stratification company Sift, provides an industry perspective on applying these models in practice.

• What is Risk Stratification? To him, it's a generic "tool" (a hammer) whose meaning is defined by the specific problem being solved, whether it's predicting patient falls or managing operating room supplies.
• The Business of Risk Stratification:
  • His company focuses on performance improvement where there is a clear financial incentive, primarily in preventative care to keep people out of the hospital.
  • Problem Identification is Key: The process starts not with machine learning, but with the C-suite (CEO, CFO, CMO). They first identify a problem where a solution could generate or save millions of dollars. Without a strong financial case, a project will not succeed.
• A Deeper Dive: Predicting Psychiatric Admissions:
  • The Goal: To identify patients at rising risk of an unplanned inpatient psychiatric admission within a 90-day window (a timeframe determined by clinicians).
  • Valuable Data: Beyond claims and labs, the most powerful data for this problem comes from care management assessments and unstructured clinician notes.
• Deployment and Building Trust:
  • The Biggest Challenge is Cultural, Not Technical: Getting clinicians with decades of experience to trust an algorithm over their own intuition is the hardest part. It is not easy to plug these models into a workflow.
  • Involve Clinicians from Day One: They must be part of the team, helping to define the problem and validate the model's outputs along the way. Showing them examples ("We identified Mr. Smith, does that make sense to you?") is crucial for building trust.
  • The "GPS" Analogy: D'Avolio compares the risk score to a GPS. It has more data and can do the math faster, but the clinician is still the driver and can always choose a different route based on their own judgment.
  • Technical Integration: Be flexible. The output could be a simple Excel file delivered via SFTP or a real-time API call. The key is to accommodate the client's existing systems and not force them into a new interface. Modern standards like FHIR are promising but not yet a practical reality for most organizations.
• Evaluation in Practice:
  • Clients Don't Pay for PPV: They pay for a return on investment (ROI). The final evaluation is always in dollars and cents. The PPV is used during modeling to estimate the potential financial impact and build the business case.
  • Interpretability vs. Performance: The goal is to build enough trust during the development process that clinicians don't need a per-patient explanation for every prediction. The focus is on solving the problem (building the house), not on being dogmatic about the tool (the hammer).

L5: Risk Stratification 2

Part 1: Continuing the Theme of Risk Stratification
The lecture continues the topic from the previous week, using the case study of predicting type 2 diabetes onset to explore the nuances of building and evaluating risk models.
1. Deriving Labels: The Challenge of "Electronic Phenotyping"
A crucial, non-trivial step in supervised learning is creating the ground-truth labels. The central question is: how do we definitively know from retrospective data that a patient was newly diagnosed with diabetes in a specific time window?

• Simple Rules are Insufficient:
  1. Medications: Looking for diabetes medications (like insulin) is a good start, but it's flawed. Some drugs have multiple uses (e.g., metformin), patients may pay cash (leaving no record in insurance claims), or a patient might be diagnosed but not yet on medication.
  2. Lab Results: Looking for abnormal lab values (like a high HbA1c) is another strong signal, but lab results alone can be misleading without clinical context (e.g., gestational diabetes).
• The Formal Phenotyping Process: This process, often a research project in itself, typically involves two steps:
  1. Manual Chart Review: A clinical expert manually reviews a few hundred patient records to create a "gold standard" set of labels. This is essential for understanding the data's complexities.
  2. Generalization to the Full Population:
     • Rule-Based Approach: The expert knowledge is codified into a complex decision tree or flowchart (a "phenotype algorithm"). These rules (e.g., "patient has a diagnosis code for type 2 diabetes AND is not on type 1 medication AND has an abnormal lab value") are applied to the entire dataset. A public repository for such algorithms is PheKB. However, these rules are slow to build and often have low recall (they miss many true cases).
     • "ML for ML" Approach: A more modern technique is to use the manually-labeled "gold standard" data to train a separate machine learning model. This model's sole purpose is to predict the label (phenotype) for the entire population, using all available data (including future data). These machine-generated labels are then used to train the primary risk stratification model that predicts future onset.

2. Evaluating Risk Stratification Models
Beyond simple accuracy, several metrics are used to evaluate risk models.

• Receiver-Operator Characteristic (ROC) Curve: A plot of the True Positive Rate vs. the False Positive Rate at various decision thresholds. The ideal curve is in the top-left corner.
• Area Under the Curve (AUC): A single-number summary of the ROC curve, ranging from 0.5 (random guessing) to 1.0 (perfect). AUC is equivalent to the probability that the model will correctly rank a random positive case higher than a random negative case. A key property is that AUC is invariant to class imbalance.
• Calibration: This measures whether a model's predicted probabilities are meaningful. A well-calibrated model that predicts a 70% risk for a group of patients will be correct for approximately 70% of those patients. This is evaluated using a calibration plot, which graphs the predicted probability against the observed frequency of the outcome. Calibration is crucial when the model's output probability is used directly in decision-making.

3. What Can Go Wrong? Key Subtleties in Healthcare ML
The lecture highlights several critical pitfalls that can invalidate machine learning models.

• Subtlety 1: Non-Stationarity (Dataset Shift): The underlying data distribution changes over time, meaning a model trained on past data may fail in the future.
  • Changing Disease Definitions: The diagnostic criteria for diseases like diabetes change over time, altering the prevalence of diagnosis codes.
  • Changing Clinical Practice: The types of lab tests ordered can change dramatically due to new technology, updated clinical guidelines, or shifts in insurance reimbursement.
  • Changing Data Systems: The nationwide switch from the ICD-9 to ICD-10 coding system means that features based on older codes disappear from new data, rendering old models useless.
  • Evaluation Flaw: The standard method of randomly splitting data from one time period into training and testing sets will fail to detect this problem. A more robust evaluation splits data chronologically: train on the past, test on the future. The performance drop between the validation and test sets can quantify the effect of non-stationarity.
• Subtlety 2: Intervention-Tainted Outcomes: Models can learn paradoxical and dangerous correlations because they fail to account for clinical interventions.
  • The Classic Example: A model learns that patients with asthma have a lower risk of dying from pneumonia. This is factually true in the data but dangerously misleading.
  • The Reason: Asthmatic patients with pneumonia are recognized as high-risk and are treated more aggressively (e.g., admitted to the ICU). The intervention causes the better outcome.
  • The Danger: Naively applying this model would lead to classifying new asthmatic patients as "low-risk" and sending them home, potentially with fatal consequences.
  • Solutions: This is a causal inference problem. "Hacks" include redefining the outcome to be a pre-treatment surrogate (e.g., a high lactate level) or treating the intervention as a censoring event. The true solution involves explicitly modeling causality.
• Subtlety 3: The Hype of Deep Learning: While deep learning has transformed fields like medical imaging, its advantage over simpler models on structured EHR data (labs, codes, vitals) is often minimal.
  • The Google EHR Paper: A widely-cited paper showed that their sophisticated deep learning model performed no better than a well-tuned L1-regularized logistic regression model for tasks like predicting mortality and readmission.
  • Reasons: Simpler models tend to be more robust to non-stationarity. Furthermore, EHR features like lab tests are already highly informative and were designed by humans for decision-making, reducing the need for the complex, automated feature learning that deep models excel at.

Part 2: Introduction to Survival Modeling
The lecture shifts from binary classification ("will an event occur?") to predicting time-to-event ("when will an event occur?").

• The Problem with Censoring: The key challenge is right-censored data. We observe a patient for a certain amount of time, but the event (e.g., death, disease onset) has not occurred by the end of the observation period. We only know the event time is greater than the censoring time.
• Why Not Use Standard Regression? Simply ignoring censored patients would severely bias the model, as it would be trained only on individuals who experienced the event relatively early.
• Formalizing the Problem: The data for survival analysis is a triplet: the features (X), the observed time (T), and a binary indicator (B) that notes whether the time T represents a true event or a censoring event.
• The Survival Function (S(t)): The core concept is the survival function, S(t) = P(T > t), which gives the probability that the true event time is later than some time t.
• The Kaplan-Meier Estimator: This is a non-parametric method to estimate the survival function directly from data that includes censored observations. The resulting Kaplan-Meier curve is a step function that shows the probability of survival over time. It is a fundamental tool for visualizing and comparing survival rates between different groups of patients (e.g., a treatment group vs. a control group).

L6: Physiological Time-Series

Part 1: Conclusion of Risk Stratification and Survival Modeling
The lecture begins by finishing the previous week's topic on survival modeling, a method for formalizing risk stratification as a time-to-event regression problem.
Recap and Formalization

• The Goal: To predict when an event (e.g., death, disease onset) will occur, rather than just if it will occur in a fixed window.
• The Challenge: The model must handle right-censored data, where the event has not occurred by the end of the observation period for a given patient.
• Key Concepts:
  • Probability Density Function (f(t)): The probability of the event occurring at a specific time t.
  • Survival Function (S(t)): The probability of the event occurring after time t (i.e., "surviving" past t). This is equal to 1 minus the cumulative density function.

Parametric Survival Modeling and Learning

• Parametric Forms: The density function f(t) is assumed to follow a specific mathematical form, such as an Exponential or Weibull distribution. These are chosen because they are defined only for positive time. The parameters of this distribution (e.g., λ) can be predicted by a machine learning model, such as a neural network, based on a patient's features.
• Maximum Likelihood Estimation (MLE): The model is trained by maximizing the likelihood of the observed data. The key insight is how the likelihood function treats censored vs. uncensored data:
  • For an uncensored data point where the event occurred at time t, its contribution to the likelihood is the density f(t).
  • For a censored data point where the observation ended at time t, its contribution is the survival probability S(t), as this represents the probability that the true event happens sometime after t.
    This allows the model to learn from all data points, avoiding the bias that would result from ignoring censored individuals.

Evaluation of Survival Models

• Concordance Index (C-statistic): This is the standard performance metric. It measures the model's ability to correctly rank pairs of patients. Specifically, it calculates the fraction of comparable pairs for which the patient who experienced the event earlier was correctly assigned a higher risk (i.e., a lower survival probability) by the model. The C-statistic is equivalent to the Area Under the ROC Curve (AUC) in the case of a binary outcome with no censoring.
• Partial Likelihood Estimators (e.g., Cox Proportional Hazards): An alternative learning framework that directly optimizes for correct ranking (like the C-statistic) rather than predicting the exact event time. This can be more robust when the absolute timing is less important than identifying the highest-risk individuals.

Recap on Non-Stationarity
The lecture revisits the critical problem of non-stationarity (dataset shift over time) and proposes several research directions for addressing it:

1. Imputation/Transformation: Use statistical methods to fill in missing data or learn a new data representation that is more stable over time.
2. Re-weighting: Give more weight during training to historical data points that are more similar to the current data distribution.
3. Online Learning: Use algorithms designed to adapt to changes in the data stream over time, often by detecting "change points."

Part 2: Physiological Time Series Modeling
The main topic of the lecture is modeling physiological data, such as vital signs from an ICU or EKG signals. The approach taken depends heavily on the amount of available data and domain knowledge.
Case Study 1: ICU Monitoring (Low Data, High Domain Knowledge)

• The Problem: ICU monitors generate a constant stream of noisy data (e.g., heart rate, blood pressure). The goal is to distinguish true physiological events from artifacts caused by interventions like probe recalibration or temporary sensor disconnects. Solving this can reduce alarm fatigue, a major safety issue where clinicians begin to ignore frequent false alarms.
• The Model: Switching Linear Dynamical Systems (SLDS):
  1. This is a type of probabilistic graphical model (or Bayesian network) that models the patient's state over time.
  2. It uses latent (unobserved) variables to represent the patient's true state (e.g., xt = true blood pressure) and the presence of an artifact (e.g., st = "probe disconnected").
  3. It models the relationship between the latent true state, the artifact state, and the observed, noisy signal (e.g., yt = measured blood pressure).
• Learning with Domain Knowledge: Since the true states (x and s) are never observed, this is an unsupervised learning problem. To succeed, the model relies heavily on injected domain knowledge to constrain the learning process:
  1. Statistical Knowledge: Normal heart rate dynamics are modeled with a sophisticated autoregressive process that separates the signal into a slow-moving baseline and fast, random fluctuations.
  2. Mechanistic Knowledge: Artifacts are modeled based on physical principles. For example, a disconnected temperature probe is modeled as cooling down with an exponential decay based on the room's ambient temperature.
• Outcome: By combining this statistical model with deep domain knowledge, the system can successfully identify and filter out artifacts from the noisy signal, even with very little labeled training data.

Case Study 2: Detecting Atrial Fibrillation from EKGs (High Data, Low Domain Knowledge)

• The Problem: Atrial Fibrillation (AF) is an irregular heartbeat (arrhythmia) that can lead to serious complications like stroke. The goal is to automatically classify short EKG signals into one of 14 different heart rhythm categories.
• Traditional Approach (Expert Features): For decades, EKG analysis was based on cardiologists identifying specific points in the signal (P, QRS, T waves) and hand-crafting features based on the intervals and shapes between them (e.g., the RR interval). These expert features were then fed into traditional classifiers.
• Modern "Black Box" Approach (Deep Learning): A 2019 Nature Medicine paper from Stanford demonstrated a new state-of-the-art approach. The key factors that enabled its success were:
  • A Better Sensor: The Zio patch, a wearable EKG, provides higher quality data.
  • Massive Data: The model was trained on over 90,000 records, an order of magnitude more than previous competitions.
  • A More Complex Task: The model learned to distinguish between 14 classes, not just 4.
• The Model: 1D Convolutional Neural Network (CNN):
  • The model is a very deep (34-layer) CNN that takes the raw EKG signal as input.
  • It uses 1D convolutions, where a filter slides along the time series to detect local patterns.
  • By stacking many layers, the network automatically learns a hierarchy of relevant features, completely bypassing the need for manual feature engineering.
• Outcome: This data-driven, end-to-end approach performed as well as, or better than, a panel of board-certified cardiologists. This represents a paradigm shift where massive data allows a "black box" model to outperform systems built on decades of human-derived domain knowledge.

L7: NLP 1

Part 1: Why We Care About Clinical Text (Peter Szolovits)
The lecture begins by establishing the immense value of unstructured clinical text, such as doctor's notes and discharge summaries. This is motivated by a real-world research project on the genetics of Rheumatoid Arthritis (RA).

• The Problem with Codified Data: The team started by identifying patients using structured billing codes (ICD-9 codes) for RA. The results were shocking:
  • Using one RA billing code had a Positive Predictive Value (PPV) of only 19%. This means only 19% of the patients identified actually had the disease.
  • Requiring three RA billing codes only increased the PPV to 27%.
  • Reason: Billing codes are designed for reimbursement, not for accurate diagnosis. A code for "myocardial infarction" (heart attack) is used whether the patient is having one or if they are just being evaluated for one, because the work performed by the hospital is the same.
• The Power of NLP: To achieve the 95% PPV needed for their genetics study, the team turned to NLP.
  • A model trained on codified data alone (labs, prescriptions, codes) achieved a PPV of 88%.
  • A model trained on narrative text alone achieved a PPV of 89%.
  • Combining both codified and NLP data resulted in a final PPV of 94%, successfully identifying a clean cohort for the study.

This compelling case study demonstrates that clinical notes contain crucial information that is often missing from or misrepresented in structured data.
Part 2: Traditional NLP Methods (Pre-Neural Networks)
This section covers the foundational, non-neural network approaches to clinical NLP.

• The Challenge of Clinical Language: Szolovits presents a nursing note that is nearly incomprehensible to an outsider, filled with abbreviations and jargon (e.g., "SOB c DOE" for "shortness of breath with dyspnea on exertion"). This highlights the specialized and difficult nature of the language.
• Early Linguistic Approaches: The idea of using formal grammar to parse and understand language dates back to the 1960s. The goal was to map syntactic structures to semantic meanings. However, these systems were brittle and often failed on real-world, messy text. A radical experiment in the 1980s even tried to force doctors to write notes in a custom-designed, computer-readable language, which failed spectacularly.
• Term Spotting: The Bread and Butter: The most common and practical approach is "term spotting."
  • Process: Identify keywords and phrases related to a concept of interest (e.g., a disease or symptom).
  • Negation Detection: A critical component is identifying negation. A simple but effective algorithm called NegEx works by looking for negation phrases (e.g., "no sign of," "denies") within a small window (e.g., five words) of a medical term.
  • The Unified Medical Language System (UMLS): A massive meta-thesaurus created by the National Library of Medicine. It integrates over 180 different medical terminologies, mapping millions of different terms (e.g., "heart attack," "myocardial infarction") to a single, unique concept identifier. This is a foundational resource for normalizing text and expanding the list of terms to search for.
• Portability of Models: In a follow-up study, the RA phenotyping model developed at Partners HealthCare in Boston was tested at Vanderbilt and Northwestern. Surprisingly, the model ported reasonably well, demonstrating that despite local differences in language, the core clinical concepts were generalizable.

Part 3: Guest Discussion with Dr. Katherine Liao
Dr. Katherine "Kat" Liao, a rheumatologist and researcher, provides an inside look at the evolution and real-world application of these NLP techniques.

• Evolution of Phenotyping:
  • The original pipeline was incredibly manual and clinician-dependent. The biggest bottleneck was getting doctors to create term lists and manually review hundreds of charts to create gold-standard labels.
  • Modern Automation: The process is now much more automated.
    1. Term Generation: Instead of asking clinicians, the system now automatically extracts relevant medical terms by processing publicly available sources like Wikipedia and the Merck Manual.
    2. Silver-Standard Labels: The model is first trained on "silver-standard" labels (e.g., the main ICD code for the disease) to pre-filter features before using the more expensive gold-standard labels.
  • This automated pipeline is now being used for large-scale biobanks like the Partners Biobank and the Million Veterans Program.
• From Research to Clinical Practice:
  • The High Bar for Clinical Use: While misclassifying a patient in a research study is acceptable (it just reduces statistical power), misclassifying a patient in a clinical setting is a huge deal. The bar for accuracy and safety is much higher.
  • The Goal: Clinical Decision Support: The goal is not to have an algorithm make a definitive diagnosis, but to provide a probability of disease. For a patient in a remote area without access to a specialist, a high probability of RA could trigger an alert for the primary care physician to consider a referral.
  • Barriers to Implementation:
    1. Alarm Fatigue: Clinicians are already overwhelmed with alerts from current EHR systems. Any new system must be carefully designed to avoid adding to this noise. The rollout of the Epic EHR system has, in some cases, reversed progress by replacing carefully curated alert lists with noisy, generic ones.
    2. User-Unfriendly EHRs: Modern EHRs are often seen as billing tools, not clinical care tools. They are complex and difficult to use, making the integration of new technologies a major challenge.
• Dealing with Data Shifts (ICD-9 to ICD-10):
  • The transition to the more granular ICD-10 system in 2015 was a major disruption.
  • The increased granularity did not necessarily improve accuracy for RA phenotyping; in fact, it may have made it worse initially.
  • The biggest challenge is mapping between the two systems to harmonize historical and current data, as there is no perfect one-to-one translation. This is an active area of research.
• The Importance of Collaboration:
  • Successful projects require deep, sustained collaboration between clinicians, computer scientists, and biostatisticians. Dr. Liao credits the i2B2 program for "locking them in a room" for hours each week, forcing them to learn each other's languages and perspectives.
  • Finding the right people—those who are open-minded and willing to work across disciplines—is critical. As one of her mentors said, "You have to kiss a lot of frogs."

L8: NLP 2

Part 1: Sophisticated Term Spotting and Feature Expansion
The lecture begins by building on the previous session's topic of "term spotting" (identifying key phrases in text) and introduces more advanced, automated methods for creating the feature sets used in phenotyping algorithms.

• The "Anchors and Correlated Features" Method: This approach, from David Sontag's lab, automates the process of finding relevant terms beyond a small, expert-provided list.
  • Start with Anchors: An expert provides a few highly specific, high-predictive-value terms for a disease (e.g., a specific ICD code or a uniquely indicative medication).
  • Create a "Silver-Standard": Any patient record containing an anchor term is temporarily labeled as positive for the phenotype.
  • Learn Correlated Features: A machine learning model (L2-regularized logistic regression) is trained to predict the presence of these anchor terms using all other words and codes in the patient's record as features.
  • Expand the Vocabulary: The features that the model learns are most predictive of the anchors (e.g., symptoms, related lab tests) are added to the original term list.
  • Build the Final Model: A final phenotyping model is built using this expanded, automatically generated feature set.
  • Result: This method proved highly effective, achieving AUCs between 0.83 and 0.95 across eight different phenotypes and demonstrating the ability to identify patient cohorts earlier in their disease course.

Part 2: The Importance of Context and De-identification
This section highlights the challenge of ambiguity in clinical text and describes a machine learning approach to solve a related problem: de-identification.

• The Context Problem: A single word, like "Huntington," can have multiple meanings depending on context: a patient's name, a disease, a hospital, or a street name. A robust NLP system must be able to distinguish these.
• A "Kitchen Sink" Approach to De-identification: To build a system that could remove Personally Identifiable Health Information (PHI), researchers created a massive feature set for every word (token) in the text.
  • Features: The model used a "kitchen sink" of features, including the word itself, its capitalization, part-of-speech, document section, UMLS concepts, and, crucially, its syntactic relationships to other words derived from a parser.
  • Model: A Support Vector Machine (SVM) was used to handle the millions of resulting features.
  • Result: The system, named "Stat De-ID," achieved extremely high performance (~99% precision and recall), far surpassing older rule-based methods.
  • The HIPAA Caveat: The lecture notes a critical legal point: the HIPAA privacy law requires the removal of 100% of PHI, not 99%. Since no system is perfect, this creates a legal gray area for sharing de-identified data.

Part 3: Unsupervised Methods for Clinical Text
The lecture explores unsupervised learning techniques that can discover patterns in clinical notes without needing pre-defined labels.

• Topic Modeling (LDA) for Psychiatric Readmission:
  • Problem: Predicting 30-day readmission for psychiatric patients, a notoriously difficult task.
  • Method: Latent Dirichlet Allocation (LDA), a topic modeling algorithm, was applied to the bag-of-words representation of patient notes. LDA unsupervisedly clusters co-occurring words into "topics."
  • Result: The model identified clinically meaningful topics (e.g., an "alcohol abuse" topic, a "psychosis" topic). Adding these 75 learned topics as features to a baseline model significantly improved its ability to predict readmission risk (AUC ~0.7).
• Tensor Factorization for Lymphoma Classification:
  • Problem: To see if the different subtypes of lymphoma could be discovered directly from pathology reports in an unsupervised manner.
  • Method: Instead of a 2D patient-by-word matrix, the data was represented as a 3D tensor: Patients x Words x Language Subgraphs. The subgraphs, derived from parsing, captured syntactic relationships. Tensor factorization was then used to find latent patterns in this 3D structure.
  • Result: This tri-clustering approach significantly outperformed standard 2D matrix factorization, demonstrating that the three-way relationships between patients, words, and their syntactic roles contained valuable information for identifying disease subtypes.

Part 4: The Modern Era of Language Modeling (Neural Networks)
This final, major section covers the revolutionary shift in NLP driven by neural networks, particularly word embeddings and the Transformer architecture.

• The Core Idea of Language Modeling: To predict the next word in a sequence. The quality of a model is often measured by perplexity (lower is better).
• From N-grams to Word2Vec:
  • N-gram models were the classic statistical approach, predicting a word based on the previous N-1 words. They are surprisingly effective at generating text that is locally coherent (e.g., Shakespearean quadrigrams sound much better than unigrams).
  • Word2Vec was a breakthrough that represents words as dense numerical vectors ("embeddings"). Words used in similar contexts are mapped to nearby points in a high-dimensional space. Famously, these embeddings capture semantic relationships through vector arithmetic (e.g., vector('king') - vector('man') + vector('woman') ≈ vector('queen')).
• The Rise of Contextual Embeddings and Transformers (2018-Present):
  • ELMo: A major advance over Word2Vec. Instead of a single, static vector for each word, ELMo generates a context-dependent embedding for each word's appearance in a sentence. This allows it to distinguish between different meanings of the same word (e.g., a "play" in a theater vs. "play" in a sport).
  • The Transformer Architecture ("Attention Is All You Need"): A novel architecture that completely replaced recurrent networks (like LSTMs) with a mechanism called self-attention. This allows the model to weigh the influence of all other words in the input when processing a given word, enabling better capture of long-range dependencies and massive parallelization during training.
  • BERT (from Google): A model based on the Transformer architecture. Its key innovation is masked language modeling, where it is trained to predict randomly hidden words in a sentence by looking at context from both the left and the right. BERT achieved massive state-of-the-art improvements on numerous NLP benchmarks.
  • GPT-2 (from OpenAI): An enormous (1.5 billion parameters) Transformer model trained on a massive web text corpus. It demonstrated an astounding ability to generate long, coherent, and stylistically appropriate text from a short prompt. The results were so realistic that OpenAI initially withheld the full model due to concerns about its potential for malicious use (e.g., generating "fake news").

The lecture concludes by reflecting that while these new models represent a phenomenal engineering success, their "black box" nature leaves a scientific conundrum. They have little resemblance to traditional linguistic theories or human cognition, raising the question of what we have truly learned about intelligence itself from their success.

L9: Tech into Clinic

Part 1: The Challenge of Deployment and the Hype Cycle (Peter Szolovits)
The lecture begins with a pragmatic and cautionary perspective on translating AI and machine learning from research into real-world clinical practice.

• The Hype Cycle: Szolovits introduces the "Gartner hype cycle," a model describing the typical trajectory of new technologies.
  • Innovation Trigger: A new idea emerges from R&D.
  • Peak of Inflated Expectations: Venture capitalists and the media create massive hype, leading to over-promising and high expectations.
  • Trough of Disillusionment: The technology fails to meet the initial lofty expectations, leading to failures, consolidation, and a perception that the idea was a bust (e.g., the "AI Winter" of the late 1980s).
  • Slope of Enlightenment: Practical, valuable applications begin to emerge as the technology matures.
  • Plateau of Productivity: The technology becomes a routine, accepted part of the industry, often without being labeled as "AI" anymore (e.g., expert systems embedded in Microsoft Excel).
• A Cautionary Tale: IBM Watson Health:
  • Following its Jeopardy! victory, IBM aggressively marketed Watson as a revolutionary force in medicine, particularly in oncology. The marketing promised a system that would read all medical literature and patient records to recommend optimal cancer treatments.
  • The Reality: The system failed to deliver on its promise of "creating new knowledge." Instead of learning from data, it largely became a sophisticated rule-based system trained by manually entering treatment guidelines from oncologists at Memorial Sloan Kettering. It struggled with real-world data and failed to demonstrate its value, leading to high-profile contract cancellations (e.g., a $60 million project with M.D. Anderson).
• A More Successful Example: Computerized Physician Order Entry (CPOE):
  • CPOE systems provide real-time decision support to clinicians as they order medications or tests. This approach is successful because it is integrated directly into the workflow at a critical decision point.
  • Proven Benefits: CPOE has been shown to significantly reduce medication errors, lower pharmacy costs (by suggesting cheaper, equally effective drugs), and improve adherence to evidence-based guidelines.
  • Slow Adoption: Despite its proven benefits, adoption of CPOE has been slow, highlighting the immense inertia and complexity of changing clinical practice.
• Challenges in Quality Assurance and Evaluation:
  • The Reproducibility Crisis: Szolovits cites the work of John Ioannidis, who famously argued that "most published research findings are false." Many biomedical studies cannot be replicated, often due to publication bias (only positive results are published) and statistical artifacts from mining large datasets.
  • Staged Evaluation: A rigorous, staged approach is needed to evaluate new clinical AI systems:
    1. Regression Testing: Automatically re-running old cases to ensure new changes don't break existing functionality.
    2. Retrospective Review: Testing the model on historical data.
    3. Prospective "Silent" Trial: Running the model in real-time but not showing its output to clinicians, then retrospectively evaluating its performance.
    4. Prospective Controlled Trial: The gold standard, where the system's impact on actual patient outcomes is measured.
  • FDA Regulation: The FDA is now treating AI software as a "medical device." While they have been cautious, they have begun approving a number of applications, almost all of which are in medical imaging (e.g., detecting wrist fractures, diabetic retinopathy, or tumors).

Part 2: A View from the Trenches - Interview with Dr. Adam Wright
Dr. Adam Wright, who oversees clinical decision support for the Epic EHR system at Partners HealthCare, provides a real-world perspective on implementing these systems.

• Current State of Decision Support:
  • The foundation of their system is still largely rule-based, especially for medication-related alerts (drug interactions, allergies, dosing).
  • They have begun to deploy machine learning models for specific problems, such as predicting which hospitalized patients are at high risk of falling. Interventions for high-risk patients can range from raising bed rails to assigning a human "sitter."
  • Setting Thresholds: Deciding the risk threshold for an intervention is complex. While ideally a full cost-utility analysis would be done, in practice, it's often "played by ear," with lower thresholds for low-cost suggestions and higher thresholds for expensive, automated actions.
• Barriers to Implementation:
  • Alert Fatigue: Clinicians are bombarded with alerts, many of which have a low PPV (10-20%). This leads them to ignore or override alerts, even telling junior residents to "just override all of those alerts" and not page them. This is a major safety concern.
  • Data Availability and Quality: Dr. Wright argues that the biggest barrier is the lack of good, reliably documented, structured data. The EHR often lacks coded data for critical context, such as a patient being at the end of life or in the middle of a "code blue" (cardiac arrest). Without this context, the system generates inappropriate alerts (e.g., suggesting a flu shot for a dying patient).
  • Workflow Integration: Figuring out who on a large care team should receive an alert and how to handle edge cases (like test results that come back after a patient is discharged) remains a significant challenge.
• The Promise of Interoperability and APIs:
  • The Shift: The biggest reason for optimism is the move away from closed, proprietary EHR systems. Modern EHRs like Epic are opening up through APIs, allowing third-party applications to be integrated.
  • SMART on FHIR: An open API standard that allows developers to build apps that can be embedded in any compliant EHR. Currently, it's better for reading and visualizing data than for writing data back or taking action.
  • Proprietary App Stores (e.g., Epic's App Orchard): These offer more powerful read/write capabilities but lock developers into a specific vendor's ecosystem.
  • New Regulations: Recent federal regulations limiting the fees vendors can charge for API access may dramatically accelerate the development of a more open app ecosystem.
• Generalizability and Model Maintenance:
  • Transferability: Models trained on one hospital's data often transfer reasonably well to another, especially if they are based on core human physiology. Models tied to a specific workflow transfer poorly. The standard practice is to validate an external model on local data before deploying it.
  • "Set and Forget" Problem: Models can become stale over time. Dr. Wright's team tries to re-evaluate or retrain models and knowledge bases on a regular schedule (e.g., every 6-12 months) to ensure they remain accurate and up-to-date with changing medical practice and data systems.

L10: Cardiac Imaging

Part 1: Introduction to Cardiac Medicine and Diagnostics
The lecture begins with a primer on cardiovascular medicine to provide the necessary clinical context for understanding how machine learning can be applied.

• The Importance of Cardiovascular Disease: It remains the leading cause of death worldwide, underscoring the urgency of developing better diagnostic and treatment strategies.
• Cardiac Anatomy and Physiology:
  • The Heart as a Pump: The main function of the heart is to pump oxygenated blood throughout the body. Blood flows from the body into the right atrium, to the right ventricle, to the lungs (to pick up oxygen), then to the left atrium, to the left ventricle (the main workhorse), and finally out to the body.
  • The Electrical System: The heart's rhythmic beating is controlled by an electrical conduction system, which is what is measured by an electrocardiogram (ECG or EKG). The electrical and mechanical functions are intimately linked.
• Major Cardiac Diseases: Physicians classify cardiac diseases based on the physiological function that has failed:
  • Pump Failure (Heart Failure): The heart cannot pump blood effectively.
  • Blood Supply Failure (Coronary Artery Disease/Heart Attack): Blockages in the arteries supplying the heart muscle.
  • One-Way Flow Failure (Valvular Disease): Heart valves are either too tight (stenosis) or too leaky (regurgitation).
  • Rhythm Failure (Arrhythmia): The heart beats too fast, too slow, or irregularly (e.g., atrial fibrillation).
• Major Diagnostic Modalities: Cardiology is a very imaging-centric and expensive field. Key diagnostic tools include:
  • Electrocardiogram (ECG/EKG): The cheapest and most common test, used for rhythm and heart attack diagnosis.
  • Echocardiogram (Echo): An ultrasound of the heart, used to assess structure and pump function. This is the main focus of the lecture.
  • Magnetic Resonance Imaging (MRI): Produces beautiful, high-resolution images but is very expensive and not widely used for routine cardiac care.
  • Angiography (CT or X-ray): Visualizes the coronary arteries to look for blockages.
• The Data Problem: A central challenge is data availability. The data that exists is what insurers are willing to pay for, which is tied to established clinical practices. It is very difficult to get novel data (like serial echoes on asymptomatic people) to build new predictive models, creating a chicken-and-egg problem for innovation.

Part 2: Computer Vision in Cardiac Imaging
This section covers the core computer vision tasks relevant to interpreting cardiac images and the challenges of applying them in a clinical context.

• Image Classification: Assigning a label to an entire image (e.g., "pneumonia present" on a chest X-ray).
  • The Clinical Reality: While deep learning has excelled at this, its practical utility can be limited. A radiologist can read an X-ray in 1-2 minutes and is legally responsible for the interpretation. They are unlikely to cede this task to a machine, so the main benefit may be in triage (prioritizing which studies the radiologist should look at first) or in resource-poor settings.
  • The "Black Box" Problem: Medicine demands descriptive accuracy (explainability) as well as predictive accuracy. A simple "yes/no" classification is often insufficient. A clinician needs to be able to point to the finding on the image to explain the diagnosis to the patient and plan treatment. Techniques like saliency maps are a step toward explainability but are often not yet satisfyingly precise.
• Semantic Segmentation: Assigning a class label to every pixel in an image.
  • The Clinical Task: This directly mimics a tedious but essential task for cardiologists: manually tracing the borders of heart chambers (e.g., the left ventricle) on an image to calculate volumes and assess pump function.
  • The U-Net Architecture: This specific deep learning architecture, with its contracting and expanding paths and skip connections, has become the de facto standard for medical image segmentation.
• Image Registration: Aligning different images into a common coordinate system. This is crucial for fusing information from different modalities (e.g., PET and CT scans) or for correcting motion artifacts in modalities with low temporal resolution.

Part 3: An Automated Pipeline for Echocardiogram Interpretation
Professor Deo details his group's ambitious project to create a fully automated pipeline for interpreting echocardiograms, moving from raw data to clinical measurements and diagnoses.

• The Motivation: The Asymptomatic Gap: There is a long period (10-20 years) where risk factors like high blood pressure cause silent, irreversible damage to the heart. Symptoms of heart failure often appear only after significant damage has occurred, at which point the prognosis is poor (50% mortality in 5 years). The goal is to develop low-cost, serial metrics that can track this subclinical disease progression in asymptomatic individuals.
• The Project: The pipeline aims to automate all the major steps of echo interpretation:
  • View Classification: Automatically identifying the dozens of different standard views in a typical echo study.
  • Quality Assessment: Scoring the quality of each view.
  • Semantic Segmentation: Using a U-Net to segment the key cardiac structures in the five primary views.
  • Measurement Calculation: Automatically computing standard clinical measures like ventricular volume and ejection fraction from the segmentations.
  • Disease Detection: Directly classifying diseases like hypertrophic cardiomyopathy and cardiac amyloidosis from the videos.
• Challenges and Results:
  • Data Acquisition: Getting bulk access to echo data was nearly impossible, requiring the creation of a "keystroke encoder" to automate the manual download process.
  • Labeling: Professor Deo himself did much of the tedious manual tracing of images needed to create the training data.
  • Performance: The automated measurements were highly correlated with manual measurements. Interestingly, when there were large discrepancies, a manual review often revealed that the human measurement was wrong (e.g., due to a typo).
  • The Gold Standard Problem: A major challenge is that the "gold standard" (a human tracing two frames) is itself noisy and highly variable. The automated method, which analyzes every frame of every video, may in fact be more robust.
• Potential Impact: An automated pipeline could enable low-cost, point-of-care screening in primary care settings, facilitate more frequent monitoring of patients (e.g., those on chemotherapy), and create massive datasets for discovering new disease subtypes.

Part 4: The Future: Moving Beyond Imaging to Biology
The lecture concludes with Professor Deo's current work at the "One Brave Idea" initiative, which aims to bridge the gap between imaging, clinical data, and the underlying biology of heart disease.

• The Limitations of Current Data: Coronary artery disease is a disease of the blood vessels of the heart, which are inaccessible for direct biological study. Existing large-scale studies are rich in clinical and imaging data but poor in deep biological data (e.g., proteomics, single-cell genomics) because these assays are too expensive.
• A New Approach: Computer Vision on Blood Cells:
  • The Idea: To use circulating blood cells (which are known to be involved in the disease process) as a low-cost, accessible window into a patient's biology.
  • The Method: Instead of expensive 'omic' technologies, the project uses high-throughput microscopy and computer vision to analyze the morphology of tens of thousands of cells per individual. By using different fluorescent dyes and perturbations (e.g., adding a drug to the cells), they can create a vast, high-dimensional phenotypic space.
  • The Goal: This cheap, scalable assay can be performed on tens of thousands of patients, creating a dataset large enough for deep learning. The goal is to link these novel cellular phenotypes to traditional clinical data, imaging data, and patient outcomes to discover new biological pathways and markers of early-stage disease.

L11: Differential Diagnosis

Part 1: Introduction to Differential Diagnosis and Early Models
The lecture begins by defining diagnosis as identifying the cause of a phenomenon and differential diagnosis as the process of distinguishing a specific disease from others with similar features.

• The Complexity of Biology: Szolovits shows a complex physiological model of the human circulatory system to illustrate that while we have deep mechanistic models of some body systems, they are too complex and require too many parameters to be practical for diagnosing an individual patient. This motivates the need for simpler, more abstract models for diagnostic reasoning.
• Model 1: Flowcharts (1970s):
  • One of the earliest approaches was to codify medical logic into a series of flowcharts. An example from the MIT Health Center in the 1970s shows a color-coded sheet for diagnosing urinary tract infections, guiding a nurse through a series of questions to a conclusion.
  • Limitations: These models are very rigid, fragile, and unable to handle unusual cases. However, they are still used in some contexts, such as for doctors who need a "cheat sheet" for diseases they rarely encounter (e.g., tropical diseases in Boston).
• Model 2: Physical Association Models (1960s):
  • An early, almost comical, model used a physical card catalog system. Each card represented a disease, and holes around the edge represented symptoms. To diagnose, a clinician would stick a needle through the hole corresponding to a patient's symptom and shake the deck; the cards that fell out were the potential diseases.
  • Limitations: This method fundamentally fails if a patient has more than one symptom or disease.

Part 2: Probabilistic Models for Diagnosis
This section covers the shift to more mathematically principled probabilistic models.

• Model 3: Naive Bayes (1960s-Present):
  • This model assumes a single underlying disease causes a set of manifestations (symptoms, signs, lab results).
  • Key Assumption: The manifestations are conditionally independent given the disease. This is a strong, often incorrect assumption, but it makes the math tractable.
  • Using Bayes' rule, the model updates the probability of each possible disease as new evidence (manifestations) is observed. In medicine, this is often done using log-odds to turn multiplication into simpler addition.
  • ROC Curves: The performance of such diagnostic tests is often evaluated using a Receiver Operator Characteristic (ROC) curve, which plots the trade-off between sensitivity (true positive rate) and specificity (1 - false positive rate). The Area Under the Curve (AUC) is a common summary metric, where 0.5 is random chance and 1.0 is a perfect test.
• Model 4: Decision-Theoretic Models (1970s):
  • These models go beyond diagnosis to recommend actions by incorporating the principles of rationality (acting to maximize expected utility).
  • Decision Trees: A problem is framed as a tree of decisions and chance events. An example of a patient with a gangrenous foot illustrates the trade-offs: amputate the foot now (high chance of survival but lower quality of life) versus trying medical treatment (chance of full recovery, but also a risk of death or a worse amputation later).
  • Utility: The "goodness" of each outcome is assigned a numerical utility score. Crucially, these utilities should be elicited from the patient, reflecting their personal values (e.g., a professional basketball player would have a very different utility for an amputated foot than a professor).
  • Solving the Tree: The model is solved by "folding back" from the end, calculating the expected utility at each chance node and choosing the action with the highest utility at each decision node.
• Model 5: Information-Theoretic Models (e.g., Acute Renal Failure Program, 1973):
  • This model combines probabilistic reasoning with a heuristic for managing complexity.
  • The Problem: Diagnosing the cause of acute kidney failure from a list of 14 possible causes and 27 possible tests. Building a full decision tree would be computationally infeasible.
  • The Heuristic: The program uses an information-theoretic approach. At each step, it recommends asking the question that will most reduce the entropy (uncertainty) of the disease probability distribution.
  • Two-Phase Process: The program continues asking these information-maximizing questions until the probability of a single disease crosses a high threshold (e.g., 95%). At that point, it switches to a more detailed, utility-based decision analysis model to consider expensive or risky tests and treatments.

Part 3: Handling Multiple Diseases (Heuristic and Bayesian Approaches)
The lecture moves to the more realistic and complex scenario where a patient can have multiple simultaneous diseases.

• Model 6: The QMR/Internist System (1970s-1990s):
  • A massive, manually-curated knowledge-based system developed at the University of Pittsburgh over 15+ person-years. It contained ~500 diseases and ~3,500 manifestations.
  • Knowledge Representation: For each disease-manifestation pair, it stored two heuristic numbers: evoking strength (how strongly the symptom suggests the disease) and frequency (how often the disease causes the symptom).
  • Reasoning Heuristic: Its key innovation was a heuristic to handle multiple diseases. It would form a differential diagnosis around the highest-scoring disease and then partition the patient's other symptoms and potential diseases into competing and complementary sets. It would solve the competing set as a single-disease problem before moving on to the complementary set, breaking the complex problem into smaller, more manageable pieces.
  • Evaluation: While initially lauded, a rigorous 1994 study showed that QMR and similar systems were not very accurate. The correct diagnosis was often ranked low (e.g., 10th or 12th) on the differential list.
• Model 7: Bayesian Network Reformulation of QMR:
  • A group at Stanford took the QMR knowledge base and reformulated it as a proper Bayesian network, a model that can represent multiple diseases and their relationships to manifestations.
  • Model: It used a "leaky noisy-OR" model to represent the idea that a symptom can be caused by multiple diseases.
  • Result: This more principled probabilistic model performed significantly better than the original heuristic-based QMR, correctly ranking the diagnosis as #1 in many more cases.
• Modern Symptom Checkers: Today's commercial symptom checkers (like the one demonstrated from Buoy Health) are modern descendants of these early diagnostic systems, using similar principles of sequential questioning and probabilistic updates.

Part 4: Meta-Reasoning and Modern Reinforcement Learning
The final section explores advanced concepts of reasoning under constraints and the latest approaches using reinforcement learning.

• Model 8: Bounded Rationality and Meta-Reasoning:
  • This concept, pioneered by researchers like Eric Horvitz and Nobel laureate Daniel Kahneman, acknowledges that real-world decision-making happens under constraints of time and computational resources.
  • Meta-Reasoning: A system should reason not only about the patient but also about its own reasoning process. It should calculate the expected value of computation—is it worth spending more time thinking to get a slightly better answer, or should it act now with the best information it has (a "just-in-time" algorithm)? This is critical in time-sensitive situations like a patient in cardiac arrest.
• Model 9: Diagnosis as a Reinforcement Learning Problem (REFUEL):
  • A very recent (NeurIPS 2018) approach that frames the entire diagnostic process as a reinforcement learning (RL) problem.
  • Formulation:
    • State: The set of known patient findings (positive and negative).
    • Actions: Either ask about another finding (a test) or declare a final diagnosis.
    • Reward: A positive reward for a correct diagnosis, a negative reward for an incorrect one or for exceeding the maximum number of questions.
  • Method: A deep Q-learning network is trained to learn an optimal policy for asking questions and making a diagnosis. The model includes clever techniques like reward shaping (to encourage asking questions likely to have positive answers) and feature rebuilding (to learn the correlations between symptoms).
  • Result: The RL agent significantly outperforms a baseline on a large, simulated dataset. The next step is to validate this promising approach on real clinical data.

L12: Pathology

Part 1: Introduction to Pathology and its Challenges
The lecture begins with an overview of the field of pathology and why it is a prime area for the application of machine learning.

• What is Pathology? Pathology is the medical specialty that diagnoses disease by examining tissues and cells. In a typical patient journey, if a radiologist finds a suspicious mass, a tissue biopsy is performed. The pathologist's analysis of that tissue (e.g., determining if it's cancer) is often the definitive diagnosis that guides major treatment decisions like surgery or chemotherapy.
• A Visual and Data-Rich Field: Pathology is highly visual, involving the interpretation of an incredible diversity of patterns in stained tissue samples viewed under a microscope. It is more data-rich than radiology because it allows for the examination of individual cells and their interactions.
• Early Computational Efforts: The idea of applying computation to pathology is not new, with early attempts dating back 40 years. These initial efforts used simple methods and tiny datasets to measure basic features like the size of cancer cells and correlate them with patient outcomes.
• The Modern ML Approach: Dr. Beck's own early work represented a shift towards a more modern machine learning approach. Instead of hand-picking one or two features, his group used image processing to extract hundreds of quantitative features from images and then used regression-based methods to let the data reveal which features were most predictive of patient survival. This data-driven approach highlighted the importance of the tumor microenvironment (how cancer cells interact with surrounding normal cells), a concept that is now central to modern cancer therapy (e.g., immunotherapy).
• The Problem of Variability: Despite being a cornerstone of medicine, pathology suffers from significant inter- and intra-observer variability.
  • Inter-observer: Different pathologists looking at the same case often come to different conclusions. A landmark study on breast biopsies showed that for cases classified as "atypia" (a borderline category between normal and cancer) by experts, community pathologists only agreed 48% of the time, with many calling the same case normal or even outright cancer.
  • Intra-observer: Pathologists often disagree with themselves. When shown the same case again after an eight-month washout period, they change their diagnosis about one-third of the time.
  • Reason: This variability is largely due to the overwhelming amount of data. A single slide can be a gigapixel-sized image containing millions of cells. A human pathologist simply does not have time to exhaustively examine every cell and must rely on intuition and incomplete sampling.

Part 2: Deep Learning for Computational Pathology
This section details how modern deep learning, particularly convolutional neural networks (CNNs), is being applied to solve key problems in pathology.

• Case Study: The CAMELYON16 Challenge (Breast Cancer Metastasis Detection):
  • The Clinical Task: A critical and tedious task for pathologists is to examine lymph node biopsies from breast cancer patients to check for metastasis (the spread of cancer). Finding tiny clusters of cancer cells is like finding a needle in a haystack and is a major source of diagnostic errors (mostly false negatives).
  • The ML Approach: The challenge was to build an algorithm to automatically detect cancer in massive, gigapixel-sized whole-slide images.
    • The Challenge of Scale: The images are too large to be fed directly into a CNN.
    • Patch-Based Method: The solution is to break the large image into millions of small patches. A CNN is trained to classify each individual patch as "tumor" or "normal."
    • Intelligent Sampling: A key insight is that the strategy for sampling patches during training is crucial. Moving from random sampling to more intelligent strategies (e.g., focusing on hard-to-classify examples) significantly improves model performance.
    • Inference: At test time, the entire slide is tiled into patches, each is classified, and the results are visualized as a heat map, highlighting the areas most likely to contain cancer.
  • Results: The deep learning models were incredibly successful.
    • Human + AI: Combining the predictions of a pathologist and the AI system reduced the human error rate by 85%. This is because the human and the AI make uncorrelated errors (the human misses small tumors, the AI gets confused by staining artifacts).
    • AI Alone: By the end of the competition, the best fully-automated system achieved an error rate of less than 1%, significantly outperforming human pathologists.

Part 3: Real-World Applications and the Future of Pathology
Dr. Beck discusses how these technologies are being developed and validated at his company, PathAI, and how they will change the field.

• Augmenting the Pathologist: The goal is not to replace pathologists but to empower them. The AI acts as an assistant, exhaustively pre-screening the slide and providing a heat map that directs the pathologist's attention to the most critical areas. This makes the process faster, more accurate, and more standardized.
• Precision Immunotherapy:
  • The Problem: Immunotherapy is a revolutionary cancer treatment, but it only works for a subset of patients. A key biomarker used to predict response is the expression of a protein called PD-L1. Pathologists are notoriously bad at quantifying PD-L1, with very low agreement rates.
  • The AI Solution: PathAI has built a system that uses deep learning for both tissue segmentation (identifying cancer vs. stroma) and cell classification (identifying cancer cells, immune cells, etc., and whether they are positive for PD-L1). This allows for a precise, quantitative, and reproducible measurement of the biomarker.
• Discovering New Biomarkers:
  • The same platform can be used for discovery. By extracting hundreds of novel quantitative features from standard H&E stained slides (which are not currently used for this purpose), the system can identify new image-based features that are predictive of response to therapy.
  • Integrating 'Omics Data: This image data can be integrated with other data types, like gene expression (RNA) data. An analysis on the TCGA dataset showed how a quantitative immune cell feature measured from an image was strongly correlated with a specific set of immune-related genes, bridging the gap from clinical outcome to tissue morphology to the underlying molecular biology.
• The PathAI Platform: To enable this work at scale, PathAI has built a platform that includes a network of over 500 pathologists who can provide high-quality labels for training data. This allows them to rapidly build and validate new models.

Part 4: Conclusion and Q&A

• Why Now? The recent sea change in the field is due to the convergence of three factors:
  1. Availability of Digital Data: Widespread adoption of whole-slide imaging.
  2. Large-Scale Computing: Access to cloud computing makes training massive models feasible.
  3. Algorithmic Advances: The breakthrough performance of deep CNNs for vision tasks.
• The Future: AI-powered pathology is already central to clinical research and will likely enter routine clinical practice in the early 2020s. The role of the pathologist will evolve from a myopic diagnostician to a more integrative consultant who synthesizes vast amounts of data to guide patient care.
• Causal Inference: In the Q&A, the challenge of using observational data versus randomized controlled trial (RCT) data is discussed. While PathAI currently focuses on RCT data because it is well-curated and allows for clean causal inferences about treatment effects, there is great interest in applying causal inference methods to larger, messier observational datasets.

L13: Mammography

Part 1: Cancer Detection and Triage in Mammography
The lecture begins by addressing the low incidence of cancer in routine breast cancer screening and proposing a "triage" system to improve radiologist efficiency.

• The Screening Funnel: Out of 1,000 screened patients, typically only about 5 or 6 are diagnosed with cancer. This means over 99% of mammograms are cancer-free.
• The Triage Idea: The core idea is to train a cancer detection model to automatically identify a large subset of the population as "healthy" with high confidence. This would allow radiologists to skip reading these low-risk mammograms and focus their attention on a smaller, higher-incidence population, potentially improving accuracy and efficiency.
• Dataset and Ground Truth:
  • The model was trained on a large, real-world dataset of consecutive mammograms from Mass General Hospital (MGH) from 2009-2016. Using consecutive scans is crucial to avoid the selection biases often found in curated public datasets.
  • The ground truth for cancer was determined not just by what the radiologist found on the mammogram, but by linking to the EHR and a multi-hospital cancer registry. A case was considered positive if cancer was diagnosed by any means within one year. This mimics the real-world goal of catching any and all cancers.

Part 2: Technical Approach - Modeling Challenges and Solutions
This section delves into the specific technical challenges of applying deep learning to mammograms and the solutions the team developed.

• Why Mammograms are Different from ImageNet:
  • Scale and Signal-to-Noise Ratio: Mammograms are enormous (e.g., 3200x2600 pixels), but the cancer itself might be a tiny 50x50 pixel region. In contrast, in a typical ImageNet photo, the object of interest (like a dog) takes up a large portion of a much smaller image.
  • Context Dependency: The visual pattern of cancer is highly context-dependent. The same pattern in a different part of the breast might be benign. A dog, on the other hand, is a dog regardless of its background.
  • Data Scarcity and Size: Despite having hundreds of thousands of mammograms, the number of cancer cases is still small (<2,000) compared to object classification datasets. At the same time, the images are so large that they are computationally expensive to process, limiting the batch size that can fit on a GPU.
• Key Modeling Decisions for Stable Training:
  • Initialization: ImageNet pre-training is essential. Models initialized with ImageNet weights learn immediately, while models with random initialization struggle for a very long time to converge, if at all. This is likely because the pre-trained network has already learned stable activation dynamics and useful low-level features.
  • Training Strategy (One-Stage vs. Two-Stage):
    • Common Approach (Two-Stage): Many researchers first pre-train a model on small, annotated patches of mammograms (e.g., a patch containing a "benign mass") and then fine-tune this model on the full-resolution images.
    • Yala's Finding (One-Stage is Enough): This complex two-stage process is unnecessary. The instability it's meant to solve is primarily due to using a very small batch size. By simply increasing the effective batch size (e.g., to 24 or more by accumulating gradients over multiple steps), a one-stage model trained directly on full images performs just as well and is much simpler.
  • Architecture: Any standard, fully convolutional architecture like ResNet works well. The choice is more a matter of a speed/performance trade-off.
• Implementing the Triage Model:
  • The model is trained independently on each of the four views of a mammogram. The final score for the exam is the maximum score across all views.
  • Setting the Triage Threshold: To ensure no cancers are missed, the threshold is set on the development set as the minimum score assigned to any cancer case that was correctly identified by the original radiologists. Any exam scoring below this threshold is automatically marked as "healthy."
  • Calibration: The model's raw outputs are calibrated using Platt's Method to ensure the predicted probabilities align with the true incidence of cancer in the population.

Part 3: Triage Model Analysis and Results

• Overall Performance: The model achieved a strong AUC of 0.82.
• Subgroup Analysis (Fairness): The model's performance was consistent across different age groups, races, and breast density categories. This is a critical validation step to ensure the model works for all populations it is intended to serve.
• Comparison to Radiologists:
  • The model's risk scores correlated well with the radiologists' true positive findings.
  • Crucially, the model and radiologists made uncorrelated errors. Many of the radiologists' false positives were assigned a low risk score by the model and fell below the triage threshold. This is the source of the system's benefit.
• Simulated Impact: A retrospective simulation on the test set showed that the triage system could:
  • Allow radiologists to skip reading 19% of all mammograms.
  • Maintain the same cancer detection rate (sensitivity was not significantly different).
  • Significantly improve specificity from 93.0% to 93.7% by reducing false positives.

Part 4: Assessing Breast Cancer Risk
The lecture shifts from detecting current cancer to predicting a woman's future risk of developing cancer over a five-year period.

• The Problem: Traditional risk models (like the Tyrer-Cuzick model) rely on clinical risk factors (age, family history, etc.) and have limited accuracy. Can the mammogram itself contain information about future risk?
• Modeling and Analysis: The approach is very similar to the detection model, using the same one-stage training on full images.
  • Models Tested: A model using only the image ("ImageOnly"), and a hybrid model combining the image features with traditional risk factors ("Image+RF").
• Results:
  • Superior Performance: Both the ImageOnly (AUC 0.68) and Hybrid (AUC 0.70) models significantly outperformed the state-of-the-art Tyrer-Cuzick model (AUC 0.62). This allows for the creation of much more accurate high-risk and low-risk cohorts.
  • Addressing Health Disparities: The Tyrer-Cuzick model, developed on a white UK population, performed worse than random for African-American women. In contrast, the deep learning models maintained their strong performance across all racial subgroups, highlighting the importance of training on diverse data and performing subgroup analyses.
  • Disentangling from Density: The model's predictions were shown to be a more powerful indicator of risk than traditional breast density. Women with non-dense breasts whom the model flagged as high-risk had a much higher cancer incidence than women with dense breasts whom the model flagged as low-risk.

Part 5: How to Mess Up (Pitfalls in Medical ML)
Yala concludes with a frank discussion of the many ways projects like this can go wrong.

• Dataset Collection: This is the most common source of error.
  • Enriched/Case-Control Datasets: Public datasets are often balanced (50% cancer, 50% healthy) and may have hidden collection biases (e.g., positive cases from one year, negative cases from another). A model trained on such data can learn spurious correlations (like the year of the scan) and will not generalize to a real-world, low-incidence screening population.
• Modeling:
  • Hidden Confounders: The model can learn unexpected patterns. Yala's team discovered their model's predictions were correlated with the specific mammography machine used to take the image, a bias they are now working to correct using adversarial training.
• Analysis:
  • Excluding "Inconvenient" Cases: A common mistake is to exclude certain types of cases for modeling convenience (e.g., a recent paper excluded women with "too small" breasts), which can introduce serious biases and harm specific subpopulations.
  • Subgroup Analysis is Essential: Always report performance not just as an average, but broken down by key demographic groups to ensure the model is equitable.
• Clinical Deployment: Real-world deployment is hard. The in-house IT infrastructure may be old and lack support for standard tools (like HTTP requests). A successful deployment requires working closely with the local IT team and being flexible enough to adapt to their environment (e.g., providing a Docker container and helping them write C# code to interact with it).

L14: Causal Inference 1

Part 1: Why Causality Matters in Healthcare
The lecture begins by shifting the focus from purely predictive modeling to causal questions, arguing that the most important questions in healthcare are not just about prediction but about intervention.

• The mantra: Everyone knows that "correlation does not imply causation," but for many predictive tasks, correlation is sufficient. However, when the goal is to intervene and improve outcomes, understanding causality is essential.
• Example 1: Preventing Type 2 Diabetes: A risk stratification model might predict who is likely to develop diabetes. A purely predictive model might find that "gastric bypass surgery" has a strong negative weight, meaning it's associated with a lower risk. The critical causal question is: Does performing this surgery on an obese person prevent them from developing diabetes? A simple correlation cannot answer this.
• Example 2: Predicting Breast Cancer Survival: A deep learning model might be trained to predict a patient's five-year survival from their diagnostic images. A new patient is predicted to have a long survival time. A naive conclusion would be to withhold treatment.
  • The Danger: This is a potentially fatal error. The patients in the training data who survived a long time likely did so because they received treatment. The predictive model, by ignoring the treatment (the intervention), learns a misleading correlation. Answering "what would happen if we don't treat this patient?" is a causal, counterfactual question that the simple predictive model cannot answer.
• Example 3: Guiding Treatment Decisions: To move beyond current clinical practice, we need to understand how different patients respond to different treatments (treatment effect heterogeneity). A model trained to simply mimic past treatment decisions can, at best, replicate the current standard of care. It cannot identify subgroups of patients who might benefit from a non-standard treatment.
• Example 4: Smoking and Lung Cancer: This is a classic causal question where a randomized controlled trial (RCT) is impossible and unethical. We must rely on observational data, but this data is biased because smokers and non-smokers differ in many ways (confounding factors).

Part 2: A Formal Language for Causal Inference
To address these questions, the lecture introduces a formal mathematical framework for causality.

• The Simplified Causal Graph: The focus is not on learning the structure of a causal graph from scratch, but on estimating the strength of relationships in a known, simple graph: X → T → Y.
  • X: The baseline covariates or features of a patient (e.g., age, gender, medical history, images).
  • T: The treatment or intervention (for today, assumed to be a binary choice, e.g., drug A vs. drug B, made once at baseline).
  • Y: The outcome of interest (e.g., survival time, blood sugar level).
• The Rubin-Neyman Causal Model (Potential Outcomes): This framework provides the language to ask counterfactual questions.
  • Y(1): The potential outcome for a person if they were to receive treatment 1.
  • Y(0): The potential outcome for that same person if they were to receive treatment 0.
• Treatment Effects:
  • Conditional Average Treatment Effect (CATE): The effect of the treatment for a specific individual (or subgroup) with features X. It is the difference in the expected potential outcomes: E[Y(1) | X] - E[Y(0) | X]. This captures treatment effect heterogeneity.
  • Average Treatment Effect (ATE): The average effect of the treatment across the entire population. It is the average of the CATE over all individuals: E[Y(1)] - E[Y(0)].
• The Fundamental Problem of Causal Inference: For any given individual in our dataset, we only get to observe one of their potential outcomes. If they received treatment 1, we see Y(1). We never get to see Y(0), their counterfactual outcome. We must find a way to estimate it.
• A Toy Example: Naive vs. True ATE: A simple numerical example with eight patients demonstrates that a naive comparison of outcomes between the group that received treatment A and the group that received treatment B can lead to the exact opposite conclusion of the true Average Treatment Effect calculated using the unobserved potential outcomes. This is due to confounding—the groups receiving A and B were systematically different from each other.

Part 3: Assumptions for Causal Inference from Observational Data
To overcome the fundamental problem of causal inference, we must make strong, untestable assumptions.

1. Ignorability (No Unobserved Confounding): This is the most critical assumption. It states that, conditional on the observed covariates X, the treatment assignment T is independent of the potential outcomes Y(0) and Y(1).
   • What it means: There are no hidden variables that influence both who gets the treatment and what their outcome would be.
   • How it's violated: If a doctor decides to give a treatment based on a factor (e.g., the patient has diabetes) that is not recorded in our dataset X, and that factor also affects the outcome, then ignorability is violated, and our estimates will be biased. Validating this assumption requires deep domain expertise.
2. Overlap (Common Support): For every type of patient (i.e., for every value of X), there is a non-zero probability of them receiving either treatment.
   • What it means: There is no group of patients that always gets one treatment and never gets the other. There must be some stochasticity or variation in treatment assignment.
   • How to check it: Unlike ignorability, overlap can be checked empirically in the data by looking for subgroups that only ever receive one treatment.

Part 4: A Method for Causal Inference - Covariate Adjustment
Assuming ignorability and overlap hold, we can estimate causal effects. The lecture introduces the first of several methods.

• The Adjustment Formula (G-Formula): A key mathematical result that allows us to estimate the expected potential outcomes from observational data. It shows that E[Y(1)] = Eₓ[E[Y | X, T=1]]. In words, the average outcome under treatment 1 can be calculated by, for every type of person X, finding the average outcome for those who actually got treatment 1, and then averaging this value over the entire population.
• Covariate Adjustment (Response Surface Modeling): This is a practical method for implementing the adjustment formula, framed as a reduction to a standard machine learning problem.
  • Train a Model: Train a machine learning model, f(X, T), to predict the outcome Y using both the patient features X and the treatment T as inputs.
  • Estimate the CATE: For any individual i, the CATE is estimated as f(Xᵢ, 1) - f(Xᵢ, 0). This is done by feeding the same patient's features into the model twice, once with T=1 and once with T=0, to get the predicted outcomes under both scenarios.
  • Estimate the ATE: The ATE is simply the average of the estimated CATEs over all individuals in the dataset.
• Where it Can Go Wrong:
  • The Goal is Different: Standard machine learning optimizes for predictive accuracy on Y. It may learn to ignore the treatment variable T if it is only a weak predictor, even though T is the variable we are most interested in for causal questions. This is a major area of active research.
  • Violation of Overlap: If there is a region of the feature space X where we only observe one treatment, the model f will be forced to extrapolate when asked to predict the outcome for the other treatment. This extrapolation can be wildly inaccurate, leading to "garbage" estimates. This is precisely where the overlap assumption is crucial for ensuring the model can learn from relevant data for both treatment arms across the entire feature space.

L15: Causal Inference 2

Part 1: Recap and Motivation for Causal Inference
The lecture begins by recapping the causal inference framework introduced in the previous session and motivating its importance with real-world examples, including the COVID-19 pandemic.

• The Causal Setting: The focus is on a simple causal graph: X → T → Y, where X represents baseline patient features, T is a binary intervention (treatment 0 or 1), and Y is the outcome. The goal is to estimate the effect of the intervention T on the outcome Y using observational data.
• Potential Outcomes: The framework uses the language of potential outcomes to define causal effects:
  • Y(1): The outcome for a patient if they were to receive treatment 1.
  • Y(0): The outcome for the same patient if they were to receive treatment 0.
• Treatment Effects:
  • Conditional Average Treatment Effect (CATE): The effect for a specific patient (or subgroup) i: E[Yᵢ(1) | Xᵢ] - E[Yᵢ(0) | Xᵢ].
  • Average Treatment Effect (ATE): The average effect across the entire population: E[Y(1)] - E[Y(0)].
• Motivation: Causal Questions in the COVID-19 Pandemic: The lecture frames several urgent pandemic-related questions in causal terms (X, T, Y):
  • X: Population demographics, T: Social distancing policy, Y: Number of new cases. (Question: What is the effect of social distancing?)
  • X: Patient comorbidities, T: Use of a ventilator, Y: Patient survival. (Question: For which patients are ventilators beneficial?)
  • Simpson's Paradox Example: The lecture uses the apparent contradiction in COVID-19 case fatality rates between Italy and China to illustrate the importance of accounting for confounding variables. Italy had a higher overall fatality rate, but a lower rate within every single age group. This paradox is resolved by recognizing that Italy's patient population was much older on average. This is a classic example of confounding, where age (X) is related to both the "treatment" (country, T) and the outcome (fatality, Y).

Part 2: Method 1 - Covariate Adjustment (Recap and Deep Dive)
This section revisits the first method for causal inference, covariate adjustment, and explores the critical role of modeling assumptions.

• The Method:
  • Train a machine learning model, f(X, T), to predict the outcome Y using both patient features X and the treatment T as inputs.
  • Use this learned model to estimate the potential outcomes for each individual by plugging in both T=1 and T=0. The CATE for individual i is estimated as f(Xᵢ, 1) - f(Xᵢ, 0).
• The Danger of Misspecification: A detailed example demonstrates the peril of using a misspecified model.
  • Scenario 1 (Correct Model): If the true relationship between X, T, and Y is linear, and we fit a linear model βX + γT, the coefficient γ directly and correctly estimates the ATE. This explains why statisticians often interpret model coefficients causally.
  • Scenario 2 (Incorrect Model): If the true relationship has a non-linear term (e.g., δX²), but we still fit a simple linear model, the estimated treatment effect coefficient (γ̂) will be biased. The estimated effect will be a function of the true effect (γ) plus a bias term that depends on the unmodeled non-linearity (δ). This bias can be so severe that the model might conclude a treatment is harmful when it is actually beneficial.
• Solution: Flexible, Non-Parametric Models: To avoid this misspecification bias, it is crucial to use flexible, non-parametric models like Gaussian Processes or Deep Neural Networks that can capture complex, non-linear relationships. The lecture highlights a specific neural network architecture with a shared representation for X and separate "heads" for T=0 and T=1, which has been shown to be effective.

Part 3: A Special Case of Covariate Adjustment - Matching
The lecture introduces matching as an intuitive, non-parametric approach and shows that it is a special case of covariate adjustment.

• The Intuition: To estimate the counterfactual outcome for a patient, find their "twin" in the dataset—another patient who is as similar as possible (has the nearest features X) but who received the opposite treatment. The twin's observed outcome is used as an estimate of the first patient's unobserved counterfactual outcome.
• The Method (1-Nearest Neighbor Matching): For each patient i, find their nearest neighbor j who received the opposite treatment. The CATE is estimated based on the difference between patient i's observed outcome and patient j's observed outcome.
• The Connection to Covariate Adjustment: Matching is mathematically equivalent to performing covariate adjustment where the predictive model f(X, T) is a k-nearest neighbor regressor. The predicted outcome for a new point is simply the outcome of its nearest neighbor in the training data for that treatment group.
• Pros and Cons:
  • Pro: It's intuitive, interpretable (you can point to the specific "twin" used for the estimate), and non-parametric.
  • Con: It performs poorly in high dimensions (the "curse of dimensionality" makes it hard to find true nearest neighbors) and is very sensitive to the choice of distance metric.

Part 4: Method 2 - Propensity Score Methods
This section introduces a completely different approach to causal inference that focuses on the treatment assignment mechanism rather than the outcome model.

• The Intuition: The core idea is to make an observational study "look like" a randomized controlled trial (RCT). In an RCT, the treatment groups are, on average, identical. In an observational study, they are not. Propensity score methods re-weight the data points to balance the covariate distributions between the treatment groups, creating a "pseudo-population" where the groups are comparable.
• The Propensity Score:
  • The propensity score, e(X), is the probability that a patient with features X receives treatment 1: P(T=1 | X).
  • This is a standard machine learning problem: we train a probabilistic classifier (like logistic regression) to predict the treatment T from the features X, ignoring the outcome Y.
• The Estimator (Inverse Propensity Weighting - IPW):
  • The ATE is estimated by taking a weighted average of the outcomes in each treatment group.
  • Each patient's outcome Yᵢ is weighted by the inverse of the probability of receiving the treatment they actually received.
  • For a patient i in the treatment group (Tᵢ=1), their weight is 1 / e(Xᵢ).
  • For a patient i in the control group (Tᵢ=0), their weight is 1 / (1 - e(Xᵢ)).
  • Intuition: A patient who was very unlikely to receive treatment 1 but got it anyway (low propensity score) is considered "surprising" and represents a large part of the unobserved population. Their outcome is therefore up-weighted to account for this.
• Derivation: A formal proof is provided to show that this IPW estimator is an unbiased estimator of the ATE.
• Pros and Cons:
  • Pro: It avoids the need to model the complex outcome relationship between Y and (X, T), which was the source of bias in covariate adjustment.
  • Con: The estimator can have very high variance, especially if the overlap assumption is weak. If a patient has a propensity score very close to 0 or 1, their inverse propensity weight will be enormous, making the final estimate highly sensitive to that one individual's outcome. This is often mitigated with a heuristic called "clipping."

The lecture concludes by mentioning that even more advanced methods, like doubly robust estimators, exist that combine the strengths of both covariate adjustment and propensity score weighting.

L16: RL 1

Part 1: From Causal Effects to Optimal Policies
The lecture begins by bridging the previous week's topic of causal inference with the new topic of sequential decision-making.

• Recap of Causal Effects: The lecture revisits the core concepts from the previous week: potential outcomes (Y(0), Y(1)), covariates (X), a one-time treatment (T), and the goal of estimating the Conditional Average Treatment Effect (CATE), which is the effect of the treatment on the outcome for a specific individual.
• From Effects to Policies: The focus shifts from merely estimating an effect to using that estimate to make decisions. An optimal policy, denoted π(X), is a rule that maps patient information (X) to a treatment action. A natural policy derived from the CATE is: if the CATE is positive (treatment is beneficial), administer the treatment; otherwise, don't.
• Beyond One-Time Decisions: However, much of medicine involves a sequence of decisions made over time. The rest of the lecture focuses on this more complex sequential setting.

Part 2: The Sequential Setting - Sepsis Management as an Example
Sepsis, a life-threatening complication of infection, is used as a motivating example for sequential decision-making.

• A Cascade of Decisions: Managing a septic patient in the ICU involves a chain of related decisions:
  1. A patient has breathing difficulties, so a doctor decides whether to put them on mechanical ventilation.
  2. If ventilated, the patient may need sedation for comfort. This decision is dependent on the first one.
  3. Sedation can lower blood pressure, which may require vasopressors or fluids to manage.
• The Problem of Scale: The number of possible treatment trajectories (sequences of actions) is astronomically large, making it impossible to test them all in a traditional randomized controlled trial (RCT). This complexity is the primary reason for turning to the framework of reinforcement learning (RL).

Part 3: Reinforcement Learning Fundamentals
The lecture introduces the core concepts and terminology of RL, framing it as a general method for learning optimal sequential policies.

• RL Success Stories: The successes of RL in complex games (Chess, Go, StarCraft) are highlighted as examples of what the framework can achieve.
• Core Components of a Decision Process:
  • State (S): Represents the current state of the environment (e.g., the patient's vitals, lab results, ventilator settings).
  • Action (A): An intervention or treatment chosen by the agent (e.g., the doctor).
  • Reward (R): A numerical value indicating the outcome of taking an action in a state (e.g., a positive reward if vitals improve, a negative reward for death).
• The RL Loop: The process is a continuous loop: an agent (doctor) observes the current state (S), takes an action (A), receives a reward (R), and transitions to a new state (S'). The goal is to learn a policy that maximizes the cumulative reward over time.
• Markov Decision Processes (MDPs): This is the dominant mathematical framework used in RL. It makes a key simplifying assumption:
  • The Markov Assumption: The future is conditionally independent of the past, given the present. In other words, the next state and reward depend only on the current state and action, not on the entire history of what came before.
  • The "State" is Key: This assumption can be made more plausible by defining the state (S) to include relevant historical information (e.g., a summary of past treatments or lab values). However, this creates a trade-off, as a very large state can be statistically difficult to handle.

Part 4: Learning Optimal Policies
The lecture describes how to find the best policy within the RL framework.

• Defining "Good" - The Value of a Policy:
  • Return (G): The sum of all future rewards from a given point in time.
  • Value (V): The expected return of following a particular policy. V is the expected return if you start in state s and follow policy π. The ultimate goal of RL is to find the policy π* that maximizes this value.
• A Toy Example: The Robot in a Room: A grid-world example is used to build intuition. A robot must navigate a grid to reach a high-reward state (+1) while avoiding a low-reward state (-1) and incurring a small negative reward for each step taken. The audience works through the problem, correctly intuiting that the optimal policy involves taking the longer path to avoid the risk of stochastically slipping into the negative terminal state.
• Paradigms of Reinforcement Learning:
  • Model-Based RL: Learns a model of the environment's dynamics (i.e., the state transition probabilities) and then uses that model to plan the best policy.
  • Policy-Based RL: Directly learns and optimizes the policy itself, without explicitly modeling values or transitions.
  • Value-Based RL: Learns a value function that estimates the expected return of taking actions in states. The policy is then derived from this value function. This is the focus of the lecture.
• Q-Learning: A Dominant Value-Based Method:
  • The Q-function (Q(s, a)): This is the central object in Q-learning. It represents the value (expected return) of being in state s, taking action a, and then following the optimal policy thereafter.
  • The Bellman Optimality Equation: The Q-function for the optimal policy (Q*) must satisfy a recursive relationship: the value of taking action a in state s is equal to the immediate reward plus the discounted maximum Q-value of the next state.
  • Learning Algorithm: Q-learning is a fixed-point iteration algorithm that learns the Q-function. It starts with an arbitrary Q-table (for discrete states) and repeatedly updates the values based on the Bellman equation and observed transitions (s, a, r, s').
  • Function Approximation: In real-world problems with large or continuous state spaces, the Q-function cannot be represented as a table. Instead, it is approximated by a function, such as a neural network (Deep Q-Learning). Learning becomes a regression problem where the model tries to minimize the difference between its current prediction and the Bellman target (the "temporal difference error").

Part 5: The Challenge of Off-Policy Learning in Healthcare
The final section discusses the specific challenges of applying RL in a healthcare context, where data is observational and not generated by the learning agent itself.

• On-Policy vs. Off-Policy:
  1. On-Policy: The agent learns from data it generates itself by actively exploring the environment (e.g., playing a game). This is rarely possible in healthcare.
  2. Off-Policy: The agent learns from a fixed dataset of past experiences generated by some other policy (e.g., doctors' past decisions recorded in an EHR). This is the standard setting for RL in healthcare.
• The Confounding Problem Revisited: Off-policy learning is fundamentally a causal inference problem. The observed data is "confounded" because the decisions made by doctors were not random; they were based on the patients' states.
• Stronger Assumptions Needed: To learn an optimal policy from observational data, the assumptions from the one-step causal setting must be extended:
  1. Sequential Ignorability: At every time step, the action taken was independent of the future potential outcomes, given the entire observed history up to that point. This means we must have recorded all the information the doctor used to make their decision at every step.
  2. Sequential Positivity (Overlap): At every point in every possible trajectory, every action must have a non-zero probability of being taken. This is an extremely strong and often violated assumption in practice, as doctors would never try certain actions in certain states. This limits our ability to evaluate policies that differ significantly from what was done in the past.
• Why Healthcare is Not (Just) a Game: The lecture concludes by contrasting the idealized world of games with the messy reality of healthcare. Games have full observability, perfect Markov dynamics, and allow for limitless exploration via simulation. Healthcare has partial observability, complex non-Markovian dynamics, and is severely data-limited. These differences represent the major obstacles that must be overcome to successfully apply RL to improve patient care.

L17: RL 2

Part 1: Reinforcement Learning - Beyond Average Reward (David Sontag)
The lecture begins by revisiting the core goal of reinforcement learning (RL) and questioning its standard formulation.

• The Goal of RL: To find an optimal policy (π*) that maximizes the value (V^π), which is defined as the expected sum of future rewards across an entire population.
• Is Average Reward the Right Goal? Sontag challenges this standard objective.
  • The Problem: Maximizing the average reward can mask poor performance on certain subgroups. A policy might achieve a high average by producing extremely good outcomes for some patients while producing very poor outcomes (like death) for others.
  • Alternative Goals: In a high-stakes setting like healthcare, we might prefer a policy that optimizes for the worst-case outcome or ensures a certain level of performance across different quartiles of the population. This highlights a critical tension between the standard RL framework and the ethical requirements of medicine.

Part 2: From Causal Inference to Optimal Policies and Back
This section connects the frameworks of one-step causal inference (from the previous week) and multi-step reinforcement learning, showing how they can inform each other.

• Estimating the Value of a Policy:
  • In RL (via Q-learning): The value of the learned optimal policy (V^π*) can be read directly from the learned Q-function. It's the maximum Q-value at the initial state.
  • In Causal Inference (via Covariate Adjustment):
    • First, we learn a model f(X, T) to predict the potential outcomes.
    • From this, we can define a policy, for example: π(X) = 1 if CATE(X) > 0, and 0 otherwise.
    • The value of this policy for an individual X is the outcome under the action dictated by the policy, which can be visualized as the upper envelope (the maximum) of the two potential outcome curves.
    • The overall value of the policy is the average of this upper envelope across the entire population.
  • Key Insight: The value of an optimal policy is a different and often more clinically relevant quantity than the Average Treatment Effect (ATE).
• A New Approach: Directly Optimizing the Policy: The lecture introduces a method that bypasses the intermediate step of estimating potential outcomes and instead learns the optimal policy directly.
  • The Estimator (Inverse Propensity Weighting for Policy Value): A new estimator for the value of a policy π is presented. It is an Inverse Propensity Weighted (IPW) average of the observed outcomes.
    • Formula: The estimator only sums over patients for whom the treatment they actually received in the data (Tᵢ) matches the treatment the policy π would have recommended (π(Xᵢ)).
    • Weighting: Each of these "on-policy" outcomes is weighted by the inverse of the probability of having received that treatment (the inverse propensity score).
  • The Power of this Approach: This turns the problem of finding the optimal policy into a weighted classification problem. We can search for the policy π that directly maximizes this IPW value estimate. This avoids the need to model the complex and potentially misspecified outcome model f(X, T).
  • Caveats: This method is highly dependent on accurately knowing or estimating the propensity scores and is very sensitive to violations of the overlap assumption (which leads to extremely high variance).
• Connecting to RL: This IPW approach can be extended to the sequential RL setting. The value of a policy can be estimated by weighting the reward at each timestep by the product of the inverse propensity scores for the entire sequence of actions leading up to that point. This is a foundational method for off-policy evaluation.

Part 3: Evaluating Dynamic Treatment Strategies in Practice (Barbra Dickerman)
Guest lecturer Barbra Dickerman presents her work on using causal inference methods to evaluate dynamic treatment strategies in cancer research.

• Dynamic Treatment Strategies: These are clinically realistic strategies where the intervention over time depends on evolving patient characteristics (e.g., "continue treatment until a side effect occurs, then stop").
• The Challenge: Treatment-Confounder Feedback: When evaluating these strategies with observational data, a major problem arises.
  • A past treatment (A₀) can affect a future health state (L₁, e.g., blood pressure).
  • This future health state (L₁) is a confounder that influences both the next treatment decision (A₁) and the final outcome (Y).
  • Standard statistical methods (like regression) that adjust for L₁ will fail. Adjusting for L₁ makes it a "collider," which opens up a spurious, non-causal pathway between the initial treatment and the outcome, leading to biased estimates.
• The Solution: G-Methods: A special class of causal methods designed to handle treatment-confounder feedback. The lecture focuses on one such method: the parametric g-formula.
  • How it Works (Model-Based RL): The g-formula is a model-based approach.
    1. Model the World: First, it uses the observational data to fit a series of parametric regression models that describe the "rules of the world": how confounders evolve over time, how treatment decisions are made, and how the outcome occurs. This is equivalent to learning a model of the Markov Decision Process.
    2. Simulate Interventions: It then uses this learned model of the world to run a Monte Carlo simulation. It simulates a large, hypothetical population where every single person perfectly adheres to a pre-specified dynamic treatment strategy.
    3. Estimate the Risk: By running this simulation for each strategy of interest, it can estimate the cumulative risk (or value) under each strategy and compare them.
• Application: Physical Activity and Prostate Cancer Survival:
  • The Goal: To estimate the effect of adhering to guideline-based physical activity levels on survival in men with prostate cancer, a question not feasible for a long-term RCT.
  • The Process: The team followed a rigorous three-step process:
    1. Specify the Target Trial: They explicitly defined the hypothetical trial they were trying to emulate, including the dynamic strategies (e.g., "maintain X level of activity unless you develop a condition that limits activity").
    2. Measure Confounders: They used a high-quality cohort study with rich data on potential confounders.
    3. Apply the G-formula: They used the g-formula to simulate adherence to different physical activity levels and estimate the 10-year mortality risk for each.
  • Sensitivity Analyses: Recognizing the potential for unmeasured confounding (e.g., by underlying disease severity), they conducted several sensitivity analyses, including using a negative outcome control. They showed that physical activity had no effect on "questionnaire non-response" (an outcome that should be similarly confounded but not causally affected by exercise), providing evidence that their main finding was not likely due to confounding.
• Conclusion: G-methods are a powerful tool for evaluating pre-specified dynamic strategies from observational data. However, their limitation is that they can only evaluate the strategies you specify in advance; they cannot discover the optimal strategy from the infinite space of possibilities. This limitation is precisely what the model-free RL methods (like Q-learning) are designed to address.

L18: Disease Progression Modeling and Subtyping 1

Part 1: Introduction to Disease Progression Modeling
The lecture introduces the field of disease progression modeling, which aims to understand and predict how a patient's disease evolves over time.

• The Goal: To model a patient's disease trajectory, represented as a curve where the x-axis is time and the y-axis is a measure of "disease burden" (e.g., symptom severity, lab values).
• Three Key Questions:
  1. Prognosis (Where is the patient?): Can we "stage" a patient at the time of diagnosis to understand how far along they are in their disease and predict long-term outcomes like survival? (This is a supervised learning problem).
  2. Description (What is a typical trajectory?): Can we create a population-level map of how a disease like Parkinson's typically progresses, showing which symptoms appear at which stage? (This is an unsupervised learning/discovery problem).
  3. Subtyping (How do patients differ?): Can we identify distinct subtypes of a disease based on how patients progress or respond to treatment? For example, can we find clusters of Parkinson's patients with "fast" vs. "slow" motor progression? (This is also an unsupervised learning/discovery problem).
• Example: Multiple Myeloma Staging: The lecture shows two real-world staging systems for the blood cancer multiple myeloma. These systems take baseline patient data (lab results, genetic markers) and categorize patients into stages (e.g., Stage 1, 2, 3), which are strongly correlated with survival and are used to guide treatment decisions.

Part 2: Prognosis as a Supervised Learning Problem
This section frames the problem of predicting a patient's future disease status as a supervised machine learning task.

• Case Study: Alzheimer's Disease:
  • The Goal: Given a patient's baseline data (X) at time zero (e.g., MRI scans, genetic info, cognitive scores), predict their future disease status (Y) at multiple future time points (e.g., 6 months, 12 months, 18 months).
  • The Outcome (Y): The disease status is quantified using a score like the Mini-Mental State Examination (MMSE), a 30-point test of cognitive function.
• The Challenge and the Naive Approach: One could train a separate, independent regression model for each future time point. However, this approach is suboptimal for several reasons:
  • Data Sparsity and Imbalance: The number of patients with available data decreases over time due to dropouts (censoring). For later time points, there may be very little data to learn from.
  • Noisy Labels: The outcome scores at any single time point can be noisy.
  • Ignoring Correlations: It fails to leverage the fact that disease status at 6 months is highly correlated with the status at 12 months.
• Solution: Multi-Task Learning (MTL):
  • The Core Idea: Instead of learning independent models for each time point (each "task"), MTL learns them jointly, sharing information between them.
  • The Intuition: We believe that the different prediction tasks share some underlying structure. For example, the same set of baseline features that predict disease status at 6 months are also likely to be predictive at 12 months.
  • The Method (for Linear Models): The optimization objective is modified to include a regularization term that penalizes differences between the weight vectors of the different tasks. A common approach is to penalize the squared L2-norm of the difference between the weight vectors of adjacent time points (e.g., ||W₆ - W₁₂||²). This encourages the models for nearby time points to be similar, effectively sharing statistical strength and making the learning process more robust to noise and data sparsity.
• Results from Alzheimer's Study: A paper applying this MTL approach (specifically, an algorithm called "convex fused sparse group lasso") to the Alzheimer's problem showed that the multi-task model significantly outperformed independent models, especially as the regularization encouraging similarity between tasks was increased. This demonstrates the value of MTL in a data-poor regime.

Part 3: Disease Subtyping as an Unsupervised Learning Problem
This section shifts from supervised prediction to unsupervised discovery, focusing on finding meaningful patient subgroups.

• The Goal: To use unsupervised learning to discover novel disease subtypes and to get more value from limited labeled data.
• The Method: K-Means Clustering: The lecture provides a brief tutorial on the K-means algorithm, a simple and widely-used clustering method that partitions data points into K clusters by iteratively assigning points to their nearest cluster center and then updating the center to be the mean of its assigned points.
• Case Study: Discovering Asthma Subtypes: A landmark study used K-means clustering to analyze baseline data from asthma patients.
  • The Data: The study used three different cohorts of asthma patients with about 20-30 baseline features (demographics, BMI, lung function, biomarkers).
  • The Clustering Process: K-means was first applied to a large "primary care" cohort. It identified three distinct clusters:
    1. Cluster 1 ("Early Onset Atopic"): Very young patients with severe, poorly controlled asthma.
    2. Cluster 2 ("Obese, Non-eosinophilic"): Predominantly older, overweight females.
    3. Cluster 3 ("Benign"): Patients with mild, well-controlled asthma.
  • Validation:
    1. Replication: The algorithm was re-run from scratch on a second, independent "secondary care" cohort of more severe patients. It rediscovered the exact same first two clusters, providing strong evidence that these subtypes were real and not just artifacts of the data.
    2. Clinical Utility (Heterogeneous Treatment Effect): The discovered clusters were then applied to a third, small (n=68) randomized controlled trial dataset. The original trial had found no average benefit for an experimental therapy. However, when the patients were stratified by their cluster membership, a dramatic heterogeneous treatment effect was revealed: the experimental therapy was highly beneficial for patients in Cluster 2 but harmful for patients in Cluster 3. These opposing effects had canceled each other out in the original analysis.
• The Power of the Unsupervised-First Approach: This discovery would have been impossible with a purely supervised approach. Trying to find subgroups that predict treatment response directly on a tiny dataset of 68 patients would have led to massive overfitting. By discovering the subtypes in a large, unsupervised dataset first and then evaluating them on the small trial dataset, the researchers were able to make a robust and clinically significant discovery.

Part 4: Conclusion and Limitations
The lecture concludes by summarizing the methods discussed and highlighting their limitations, which will be addressed in the next session.

• Supervised and Unsupervised approaches are powerful tools for prognosis and subtyping.
• Limitations of these Simple Methods:
  1. They Conflate Stage and Subtype: The clustering methods don't distinguish between a patient's disease stage (early vs. late) and their inherent subtype (fast vs. slow progressor).
  2. They Only Use Baseline Data: They don't make use of the rich longitudinal data available for patients over time.
  3. They Assume a Single Factor: They assume that "subtype" is the only factor explaining patient variation, ignoring other patient-specific factors.
  4. The Unsupervised/Supervised Dichotomy: The lecture poses the question: must it be one or the other, or can we combine these approaches?

The next lecture will introduce more sophisticated probabilistic modeling approaches to overcome these limitations.

L19: Disease Progression Modeling and Subtyping 2

Part 1: Staging from Cross-Sectional Data - Building Intuition
The lecture begins by building intuition for how one might infer a patient's disease stage from a single, cross-sectional snapshot of their data, assuming for now that there is only one disease subtype.

• The 1D Case (One Biomarker):
  • The Data: Imagine a single biomarker is measured for many different patients, each at a single point in time. The data is a set of points on a line.
  • The Goal: To order these patients from "early stage" to "late stage."
  • Possible Conjectures (Assumptions):
    • Monotonic Progression (Left to Right): Early-stage patients have low biomarker values, and late-stage patients have high values.
    • Monotonic Progression (Right to Left): The opposite is true.
    • Non-Monotonic Progression: Early-stage patients are in the middle, and the disease progresses outwards in both directions as the biomarker becomes abnormal.
  • Disambiguation: Without additional information, it's impossible to know which direction represents progression. One could use side information, such as patient age or time-to-death, to resolve this. For example, if the average age is higher on the right side of the line, one might infer that "right" corresponds to a later disease stage.
• The 2D Case and Beyond (Multiple Biomarkers):
  • The Data: Now each patient is a point in a 2D (or higher) space.
  • The Manifold Assumption: The core intuition is that the patients' data points lie on some low-dimensional manifold or trajectory through this high-dimensional space. The task is to discover this underlying manifold.
  • The Role of Data Density and Longitudinal Samples:
    • If we have a very large amount of cross-sectional data, the points may be dense enough to visually trace out the underlying progression curve.
    • If we have only a small amount of data, but for each patient, we have a few longitudinal samples (e.g., measurements at two or three time points), we can use the "arrows" provided by these short trajectories to stitch together the overall, long-term progression manifold.

Part 2: A Probabilistic Model for Disease Progression (COPD Case Study)
This section introduces a formal, probabilistic model for learning disease progression from sparse, irregular, cross-sectional data.

• Case Study: Chronic Obstructive Pulmonary Disease (COPD):
  • Why COPD? It is a well-understood disease with an established "gold standard" staging system (the GOLD criteria), based on lung function tests (spirometry). This allows for validation of the data-driven model's results.
  • The Data: The model uses only diagnosis codes from patients' electronic medical records over a 1-3 year period. The gold standard spirometry data is intentionally excluded.
  • The Goal: To take these short, sparse, irregular patient histories and stitch them together to learn a 20-year model of COPD progression.
• The Model Architecture (A Factored Hidden Markov Model):
  • Latent (Hidden) Variables: The model's core is a set of hidden variables that are never directly observed in the data:
    • Disease Stage (S): A discrete variable (e.g., from 1 to 10) representing the patient's overall disease stage at a given time.
    • Comorbidities (X): A set of binary variables representing the presence or absence of specific conditions related to the disease (e.g., diabetes, depression, lung cancer).
  • Observed Variables (O): The actual clinical findings observed in the EMR (e.g., diagnosis codes).
• Modeling Components:
  • Disease Stage Progression (Top Layer):
    • The transitions between disease stages (e.g., from S₁ to S₂) are modeled using a continuous-time Markov process.
    • This is crucial because the time intervals (Δt) between patient observations are irregular. The model's transition probability matrix is a function of Δt, allowing longer time gaps to correspond to potentially larger jumps in disease stage.
  • The Relationship between Stage and Comorbidities:
    • A key assumption is made: the different comorbidities (the X variables) are conditionally independent of each other given the disease stage (S).
    • This forces all correlations between comorbidities to be mediated through the overall disease stage, making the latent stage variable identifiable and preventing the model from simply ignoring it.
    • Another assumption is that comorbidities are monotonically more likely to appear in later disease stages.
  • The Relationship between Latent State and Observations (Bottom Layer):
    • The relationship between the hidden comorbidities (X) and the observed diagnosis codes (O) is modeled as a noisy-OR network. A patient with a comorbidity (e.g., diabetes) has a certain probability of having the corresponding diagnosis code recorded at any given visit.
    • Anchoring: To give meaning to the hidden comorbidity variables (e.g., to ensure X₁ actually represents diabetes), a technique called anchoring is used. For each comorbidity, a domain expert specifies one or more "anchor" observations (e.g., the specific diagnosis code for diabetes) that are assumed to only be caused by that specific comorbidity. This grounds the latent variables to real-world concepts.
• Learning and Results:
  • The model is trained using Expectation-Maximization (EM) to find the parameters that maximize the likelihood of the observed data, marginalizing over all the hidden variables.
  • Output: The trained model can be used to:
    • Infer an individual's trajectory: Given a patient's observed data, the model can infer their most likely sequence of hidden disease stages and comorbidity onsets.
    • Simulate population-level progression: The model can be used as a simulator to generate long-term (e.g., 10-year) disease trajectories. The simulation for COPD correctly recovered known clinical facts, such as cardiovascular disease being a major comorbidity that appears very early in the disease course, validating the model's plausibility.

Part 3: Pseudo-Time Methods for Trajectory Inference
The lecture introduces a different family of algorithms for ordering cross-sectional data, known as pseudo-time or trajectory inference methods, which are very popular in single-cell genomics.

• The Analogy: The problem of ordering individual cells along a differentiation pathway is analogous to ordering patients along a disease progression pathway. Each cell is a patient, and its gene expression profile is its set of clinical features.
• The Goal: To take a collection of unordered, cross-sectional snapshots (cells or patients) and arrange them along a "pseudo-time" axis that represents the underlying biological or disease process.
• Method 1: Monocle (Minimum Spanning Tree Approach):
  • Dimensionality Reduction: The high-dimensional data is first projected into a low-dimensional space (e.g., using PCA or ICA).
  • Build a Minimum Spanning Tree (MST): An MST is constructed on the low-dimensional points, connecting them in a way that captures the principal structure of the data.
  • Find the Longest Path: The algorithm finds the longest path through the MST, which is assumed to be the main axis of progression.
  • Order the Points: The pseudo-time for each point is its position along this path. The method can naturally handle bifurcations or branching trajectories.
• Method 2: Gaussian Process Latent Variable Model (GP-LVM):
  • This is a probabilistic approach. It assumes that the observed high-dimensional data for each patient was generated from a low-dimensional latent variable, which is their unknown pseudo-time (t).
  • A Gaussian Process (GP) is used to define a flexible, non-linear mapping from the latent pseudo-time back to the observed feature space.
  • Learning: The model is trained to find both the latent pseudo-time for each patient and the parameters of the GP mapping that best explain the observed data. This jointly infers the patient ordering and the underlying progression trajectory.

L20: Precision Medicine

Part 1: The Vision of Precision Medicine
The lecture begins by defining "precision medicine" and outlining the vision for its future, as laid out in a 2017 National Research Council report.

• What is Precision Medicine? It is the idea of moving beyond one-size-fits-all medicine to tailor treatments and prevention strategies to an individual's unique characteristics, particularly their genetics. It is closely related to the concept of disease subtyping.
• Why Now? The report identifies several key drivers making precision medicine feasible today:
  • Massive Molecular Data: The cost of genome sequencing has plummeted from billions of dollars to under $1,000, making it possible to collect genetic data at an unprecedented scale.
  • Advances in IT: We now have the computational power ("big data" capabilities) to analyze this vast amount of information.
  • A "Perfect Storm" of Receptivity: Rising healthcare costs and stagnant quality have made stakeholders (patients, providers, payers) desperate for new, more effective approaches.
• The "Information Commons" Vision: The report envisions a future where medicine is organized like Google Maps. Just as Maps layers diverse information (roads, buildings, census tracts) onto a common coordinate system (latitude/longitude), a medical "information commons" would layer diverse data types (genome, microbiome, clinical signs, environmental exposures) onto a common framework of individual patients.
• Large-Scale Data Collection Efforts: This vision is being realized through massive data collection initiatives:
  • The "All of Us" Research Program: An NIH project aiming to collect comprehensive genetic, clinical, and lifestyle data from a representative sample of one million Americans.
  • The UK Biobank: A similar project in the UK with about half a million participants, whose data is already being used for large-scale research.

Part 2: Re-Taxonomizing Disease
A core goal of precision medicine is to move beyond historical, symptom-based disease definitions to a new, molecularly-based taxonomy.

• The Problem with Old Taxonomies: The lecture uses the historical term "dropsy" as an example. "Dropsy" (swelling/edema) was once considered a disease but is now understood to be a symptom of many different underlying conditions (heart failure, kidney disease, etc.). The modern disease "asthma" may be similarly heterogeneous, representing a collection of different underlying biological problems that all manifest with similar symptoms.
• The "Cloud of Points" Approach: The lecture introduces a conceptual model for a new taxonomy. In a high-dimensional "modality space" containing all patient data, individuals are not uniformly scattered but form clusters.
  • A patient in the center of a cluster is "typical."
  • A patient at the edge of a cluster is an "outlier" and may represent a rare disease subtype or a patient who is about to have a change in their disease course.
• A Clinical Case Study: The story of a young boy with ulcerative colitis who suddenly stopped responding to standard therapies illustrates this. He moved from being in the "center" of the typical patient cluster to becoming an "outlier." Identifying what makes him different is the key to finding an effective treatment.

Part 3: A Shallow Dive into Genetics
To understand the molecular basis of precision medicine, the lecture provides a rapid review of fundamental molecular biology.

• From Mendel to the Double Helix: A brief history covers Mendel's discovery of discrete units of inheritance ("genes"), the identification of DNA as the carrier of genetic information, and Watson and Crick's discovery of its double helix structure.
• The Central Dogma: The flow of information from DNA → RNA → Protein.
• Complexity and Exceptions: Biology is the "science of exceptions." The simple central dogma is complicated by many factors:
  • Gene Regulation: Promoters, repressors, and enhancers control the rate of transcription.
  • "Junk" DNA: Only about 1.5% of the human genome (exons) directly codes for proteins. The function of the other 98.5% is a major area of research.
  • Alternative Splicing: A single gene can produce multiple different proteins by splicing its RNA in different ways.
  • Other RNA types (e.g., long non-coding RNA, RNAi) and post-translational modifications add further layers of regulation.

Part 4: Methods for Genetic Association Studies
This section covers the primary computational methods used to link genetic variation to disease.

• The Data Revolution: The cost of sequencing DNA (genotyping) and RNA (expression profiling) has dropped faster than Moore's Law, making massive datasets available.
• Microarray Analysis and Bi-Clustering:
  • The Method: An early and influential technique for measuring the expression levels of thousands of genes simultaneously.
  • The Analysis (Bi-Clustering): The resulting data (a matrix of patients x genes) is analyzed using clustering techniques. A key insight was to cluster both the patients and the genes simultaneously. This groups patients with similar expression profiles and, at the same time, groups genes that are co-expressed across those patients.
  • A Landmark Study (Perou et al., 2001): This approach was famously used to analyze breast cancer samples. The unsupervised clustering revealed five distinct molecular subtypes of breast cancer that had significantly different clinical outcomes (survival rates). This was a foundational result demonstrating that a purely molecular, data-driven approach could discover clinically meaningful disease subtypes.
• Genome-Wide Association Studies (GWAS):
  • The Goal: To identify which specific genetic variants (often Single Nucleotide Polymorphisms, or SNPs) are associated with a particular disease or trait (phenotype).
  • The Method: For a given phenotype, a GWAS scans hundreds of thousands or millions of SNPs across the genome, performing a statistical test at each one to see if the variant is more common in people with the disease than in controls.
  • The Manhattan Plot: The results are typically visualized in a "Manhattan plot," which shows the statistical significance of each SNP across all chromosomes.
  • The "Small Effects" Problem: A major finding from GWAS is that for most common, complex diseases, the associated genetic variants have very small effect sizes (e.g., an odds ratio of 1.1 or 1.2). This is in stark contrast to environmental factors like smoking, which has an odds ratio of 8 for lung cancer.
• Phenome-Wide Association Studies (PheWAS):
  • The Goal: The reverse of a GWAS. It starts with a specific genetic variant and scans across a wide range of diseases and traits (the "phenome") to see what that variant is associated with.
  • The Method: The first PheWAS study used Vanderbilt's biobank, manually grouping thousands of billing codes into ~700 disease categories. For a given SNP, it tested for an association with each of these categories.
  • Result: PheWAS can confirm known associations (e.g., a known MS-related SNP was strongly associated with MS) and also generate new hypotheses by revealing unexpected associations (e.g., the same SNP showed a weak but intriguing association with certain cancers).
• Expression Quantitative Trait Loci (eQTL) Studies: This approach uses gene expression levels themselves as the traits of interest, looking for genetic variants (QTLs) that are associated with differences in the expression of specific genes.
• The Current State of ML in Genomics: The lecture concludes with a surprising observation. Despite the field's sophistication, most of the successful methods to date (clustering, matrix factorization, GWAS/PheWAS, gene set enrichment analysis) are based on relatively "old-school" statistical and machine learning techniques. There has not yet been a "killer paper" demonstrating that advanced deep learning methods (e.g., 83-layer neural networks) can significantly outperform these more established approaches in this domain, though the speaker suspects that this is likely to change in the near future.

L21: Automating Clinical Workflows

Part 1: Improving Medical Care - Reduce Variance vs. Improve the Average
The lecture begins by posing a fundamental question: what is the best way to improve medical care? Two competing philosophies are presented.

• Philosophy 1: Improve the Average (The "Expert System" Idea): The initial idea in medical AI was to capture the knowledge of world-class experts in a computer system and make it available to all doctors, thereby raising the average level of performance across the entire profession.
• Philosophy 2: Reduce the Variance (The "Protocol" Idea): A later idea recognized that the biggest harm comes from sub-average performance. The goal should be to reduce the variance in care, ensuring that all patients receive a consistent, evidence-based standard of care, even if that means limiting the creativity of top performers.
• Why Variance Reduction Wins: Using a hypothetical cost model, the lecture demonstrates that if the "cost" of poor performance is disproportionately high (which it is in medicine), then narrowing the distribution of performance (reducing variance) can have a much larger impact on overall outcomes than simply shifting the average performance slightly higher. This is the rationale behind the modern emphasis on clinical guidelines and protocols.

Part 2: Top-Down Workflow - Clinical Practice Guidelines
This section describes the primary "top-down" mechanism for reducing variance: the development and dissemination of clinical practice guidelines.

• How Guidelines are Made: Learned societies (like the American Heart Association) convene panels of experts who review the available evidence and formulate explicit recommendations for how to manage specific conditions.
• Evidence Levels: These recommendations are graded based on the strength of the evidence supporting them, from Level A (high-quality evidence from multiple randomized controlled trials) down to Level C (consensus expert opinion).
• The Cholesterol Guideline Example: The lecture walks through the 2018 guideline on managing blood cholesterol. It shows detailed flowcharts and recommendations for different patient populations based on age and risk factors.
• The Problem of Adherence: A major challenge is that adherence to these guidelines is often poor, and it can take many years for new recommendations to be widely adopted in clinical practice.

Part 3: Bottom-Up Workflow - Care Plans and Clinical Pathways
This section explores a "bottom-up" approach to standardizing care that originates from the day-to-day practice of nursing.

• Nursing Care Plans: Nurses, who provide the majority of hands-on patient care, use structured "care plans" to manage individual patients. These plans document assessments, nursing diagnoses, patient outcome goals, specific interventions, and the rationale behind them.
• From Care Plans to Clinical Pathways (The "Cow Paths" Idea):
  • The Goal: To learn the typical patterns of care ("clinical pathways") by analyzing the actual treatment histories of many patients.
  • A Machine Learning Approach (Zhang et al.): The lecture describes a study that attempted to do this for chronic kidney disease patients.
    1. Data Abstraction: Individual medical events (diagnoses, meds, procedures) from each patient visit were grouped into abstract "supernodes." A patient's history became a sequence of these supernodes.
    2. Clustering: Patients were clustered based on the similarity of their visit sequences.
    3. Mining Patterns: Within each cluster, the researchers mined for the most common treatment trajectories or pathways.
  • Limitations: While a promising idea, the study was limited by a small dataset, resulting in small and heterogeneous patient clusters from which it was difficult to extract robust, generalizable patterns.

Part 4: The "Wisdom of the Crowd" and Its Pitfalls
This section discusses another bottom-up approach that learns from the collective behavior of clinicians.

• The "Amazon Recommendation System" for Medicine: A study by Jeff Klann applied collaborative filtering techniques to medical orders. The system would recommend (or question) an order based on what other doctors typically did for similar patients (e.g., "other doctors who ordered tests A and B also ordered test C").
• Methodology: The system learned a Bayesian network representing the probabilistic relationships between diagnoses and orders for specific clinical scenarios (e.g., back pain in the ER).
• The Pitfall: Reinforcing Bad Habits: The obvious danger of this "wisdom of the crowd" approach is that if the collective practice is wrong or based on outdated tradition rather than evidence, the system will simply learn and reinforce those bad habits. The story of Ignaz Semmelweis, who discovered the importance of hand-washing but was ignored by his colleagues, serves as a powerful cautionary tale.

Part 5: Monitoring and Maintaining Workflow Systems
This section addresses the practical challenges of ensuring that these complex clinical IT systems work correctly over time.

• Silent Failures: The lecture details a real-world case study from Dr. Adam Wright's work. A critical drug-monitoring alert (for the drug amiodarone) silently stopped working for years because a background change in the drug's internal code was not propagated to the alert's logic.
• Change Point Detection: To prevent such silent failures, Dr. Wright's team developed a monitoring system that uses change point detection, a machine learning technique for time series analysis.
  • The system continuously monitors the firing rate of every alert in the hospital.
  • It builds a dynamic linear model that accounts for normal seasonality (e.g., fewer alerts on weekends).
  • It calculates the probability that the current data is being generated by a "stable" model, a "temporary outlier" model, or a "level shift" model.
  • When the probability of a level shift becomes high, it triggers a meta-alert, notifying the IT team that a rule may be broken.

Part 6: The Central Role of Communication
The lecture shifts to a topic that is often overlooked in technical discussions: the critical importance of communication in healthcare.

• Healthcare as a Team Sport: Researcher Enrico Coiera argued that the most important aspect of healthcare is not individual decision-making but communication among the care team.
• The Data: Numerous studies show that clinicians spend a huge portion of their time (often over 50%) talking to each other. The hospital is a highly interrupt-driven environment, with communication events happening every few minutes.
• The Need for Better Systems: The lecture argues that we need workflow engines that explicitly manage communication and follow-up. A proposed idea is a system that, when a task is assigned, automatically schedules a future check to see if the task was completed and escalates if it was not.

Part 7: The Failed Dream of the Personal Health Record
The lecture concludes with the story of Google Health, a high-profile attempt to create a "guardian angel"—a lifelong, patient-controlled personal health record.

• The Vision: To empower patients by giving them a single, unified repository for all their health data, which they could manage and share with providers.
• Why It Failed:
  • Data Fragmentation: It was impossible to automatically pull data from all of the different, non-interoperable provider systems.
  • Manual Entry Burden: Patients were unwilling to manually enter their entire medical history.
  • No Provider Buy-in: Even if a patient did manage to populate their record, their doctors had no incentive or workflow to actually look at it.
  • The Vicious Cycle: Because the record was incomplete, it wasn't useful, which meant no one used it, which meant there was no incentive to make it more complete.

The story serves as a final, powerful reminder that even the most compelling technical vision will fail if it does not account for the complex realities of clinical workflow, data infrastructure, and human incentives.

L22: Regulation

Part 1: Introduction and Frameworks for AI in Healthcare (Andy Coravos)
The lecture begins by setting the stage for the complex world of regulating algorithmically-driven healthcare products.

• A Framework for AI Products: A useful way to categorize these products is by their function:
  1. Measure: Tools that collect data, like digital biomarkers or clinical decision support systems.
  2. Diagnose: Tools that use measurements to determine if a patient has a condition.
  3. Treat: Tools that act as interventions, known as "digital therapeutics."
• Everything is an Algorithm: Even a seemingly simple "step count" from a wearable device is the output of an algorithm that takes sensor data as input and makes a prediction. These algorithms can have biases (e.g., being less accurate for older people who walk differently).
• The Complexity of the "Stack": Regulating these products is difficult because they are built on a complex stack of hardware and software. For a product that detects atrial fibrillation (AFib) from a watch, where should the regulatory line be drawn? Is it the sensor, the signal processing, the diagnostic algorithm, or the user interface? The FDA is exploring how to decouple hardware and software components for evaluation.
• Digital Therapeutics: The lecture highlights Akili Interactive, a company developing a video game to treat pediatric ADHD. By structuring themselves like a traditional drug company and running rigorous clinical trials, they are on track to become the first FDA-approved "prescription video game," demonstrating that software can be a legitimate therapeutic intervention.
• Decentralized Clinical Trials: A major innovation enabled by these technologies. Instead of requiring patients to travel to a clinical site, data can be collected remotely via wearables and apps. This has the potential to make trials faster, cheaper, and more accessible to a broader population.

Part 2: The Regulatory Landscape for Healthcare Software
This section provides an overview of the key government agencies involved in overseeing healthcare software in the US.

• The Key Players:
  • FDA (Food and Drug Administration): The primary regulator, responsible for ensuring the safety and effectiveness of medical products.
  • ONC (Office of the National Coordinator for Health IT): Focuses on the standards and interoperability of health information technology like EHRs.
  • FTC (Federal Trade Commission): Focuses on consumer protection and preventing deceptive marketing claims. The FTC may step in when a company makes unproven health claims about a "wellness" product that is not regulated by the FDA.
  • FCC (Federal Communications Commission): Regulates the communication aspects of connected devices.
• What is a "Medical Device"? The FDA regulates "medical devices," a broad category that can include software. The key determinant is the intended use and the claims made by the manufacturer. The exact same product can be a regulated medical device or an unregulated wellness tool, depending entirely on the marketing language used.
• Software as a Medical Device (SaMD): The FDA has created a specific category for software that functions as a medical device on its own, without being part of a hardware device (e.g., an app that analyzes an image to diagnose a condition).
• Pathways to Market (Cleared vs. Approved):
  • 510(k) Clearance: For low- to moderate-risk devices that are "substantially equivalent" to a legally marketed device that is already on the market (a "predicate").
  • De Novo Classification: For novel low- to moderate-risk devices that do not have a predicate. This pathway creates a new classification, which subsequent devices can then use as a predicate.
  • Premarket Approval (PMA): The most stringent pathway, for high-risk devices (e.g., life-sustaining devices like pacemakers). It requires extensive clinical trial data to prove safety and effectiveness.
  • The Pre-Cert Program: A pilot program by the FDA to address the challenge of regulating software that is updated frequently. Instead of reviewing every product update, the FDA is exploring "pre-certifying" an entire company based on its culture of quality and organizational excellence, allowing them to ship updates more rapidly.
• The Regulatory White Paper: The lecture refers to the discussion paper on regulating AI/ML software that the students were assigned to read. The speakers emphasize that the students' feedback is highly valuable to the FDA, as they represent a technically knowledgeable audience that the agency wants to hear from.

Part 3: The Role of the Institutional Review Board (IRB) (Mark Shervey)
Mark Shervey shifts the focus from federal regulation to the local, institutional level of oversight provided by the IRB.

• What is the IRB? An IRB (at MIT, called COUHES) is a committee that reviews and approves research involving human subjects to ensure it is ethical and that participants are protected.
• Why is the IRB Necessary? The IRB was established in response to historical abuses in human research, such as the Nuremberg trials and the Tuskegee Syphilis Study. Its work is guided by the ethical principles outlined in the Belmont Report:
  1. Respect for Persons: Ensuring voluntary, informed consent.
  2. Beneficence: Maximizing benefits while minimizing harm ("do no harm").
  3. Justice: Ensuring the fair distribution of the risks and benefits of research.
• When is IRB Approval Needed? Approval is required for any "systematic investigation" intended to produce "generalizable knowledge" that involves "human subjects." In practice, any research involving identifiable patient data requires IRB review.
• The Challenge of Digital Research: The rise of digital research (e.g., online consent, data from wearables) presents new challenges. Shervey describes a framework for separating "project activities" (e.g., general outreach, sharing content) from "study activities" (e.g., consent, data collection under a specific protocol). Study activities are 100% under IRB oversight, while some project activities may not be. This allows for more agile development on the platform side while maintaining strict, waterfall-like control over the research protocol itself.
• Key Takeaways for Researchers:
  1. When in doubt, ask the IRB.
  2. Limit the collection of Protected Health Information (PHI) to the absolute minimum necessary.
  3. Be aware that agile software development practices are often incompatible with the strict, pre-approved protocols required for human subjects research.

Part 4: Discussion and Public Participation
The session concludes with a discussion and a call to action for the students to participate in the public rulemaking process.

• Students' Feedback on the FDA White Paper: The students provide feedback on the AI/ML regulation paper, suggesting areas for improvement:
  • Clarify who sets and owns "Good Machine Learning Practices."
  • Provide more specific guidance on transparency.
  • Explicitly address and require analysis of algorithmic bias across different demographic subgroups.
  • Incorporate requirements for privacy and security protections.
• A Call to Action: The speakers strongly encourage the students to submit their comments to the public docket for the FDA's white paper. They emphasize that every comment is read and considered by law, and that the technical perspective of students and engineers is a crucial and currently underrepresented voice in the regulatory process.

L23: Fairness

Part 1: Introduction - The Legal and Societal Context of Algorithmic Fairness
The lecture begins by framing the issue of algorithmic fairness not just as a technical problem, but as a critical societal and legal issue.

• A High-Level Meeting: The speaker recounts attending a meeting at the National Academy of Science, Engineering, and Medicine (NASEM) on AI and decision-making. The discussion involved high-level figures from law and science, including a federal judge and a Nobel laureate.
• The Judge's Perspective: A key takeaway was a federal judge's reluctance to hire technically-trained clerks. The judge argued that his role is not to be a technical expert, but an impartial arbiter who weighs arguments from both sides. He feared a technical expert would bring their own biases, conflicting with the legal system's adversarial process for finding truth.
• AI in the Justice System: The lecture highlights the growing use of algorithms in the legal system for high-stakes decisions:
  • Bail Decisions: In California, algorithms are now used to recommend whether a defendant should be granted bail.
  • Sentencing: Algorithms are used to predict a defendant's risk of recidivism (re-offending), which can influence the length of their prison sentence.
• The Controversy and Bias: These systems are highly controversial for several reasons:
  • Racial Bias: The algorithms often reflect and amplify existing biases in the justice system. For example, they may be less likely to grant bail to Black defendants than to White defendants with identical profiles, because they are trained on historical data where judges exhibited this bias.
  • Lack of Transparency: Many of these algorithms are proprietary "black boxes" developed by private companies, making it impossible to audit how they work or what data they were trained on.
  • Legal Challenges: Despite these issues, courts have upheld the use of these algorithms, a decision many consider outrageous.

Part 2: Defining Fairness - A Multifaceted and Conflicting Problem
The lecture moves from the context to the core technical and philosophical challenge of defining what "fairness" actually means.

• Three Intuitive Notions of Fairness:
  1. Similar Error Rates: The algorithm should have similar accuracy across different demographic groups.
  2. Alignment with Societal Goals: The algorithm's outcomes should reflect the world we want to live in, not just the biased world reflected in the data.
  3. Similar People, Similar Treatment: Individuals who are similar on relevant characteristics should receive similar decisions.
• Sources of Bias in Data: Bias is not just about prejudice; it can arise from many sources. A key example is from medical genetics:
  1. Ancestry Bias: A genetic variant was identified in European populations as a risk factor for a serious heart condition (hypertrophic cardiomyopathy). However, this same variant is common and benign in African populations. Because the original model was trained on a biased sample, it led to misinforming countless African-American patients that they were at high risk of early death. This problem persists today.
• Technical Formalizations of Fairness: The lecture introduces three formal, mathematical definitions of fairness from the computer science literature:
  1. Independence (Demographic Parity): The model's prediction (R) must be statistically independent of the protected attribute (A, e.g., race). This means the rate of positive predictions (e.g., being hired) must be the same for all groups. This is related to the legal concept of disparate impact.
  2. Separation (Equalized Odds): The prediction (R) must be independent of the protected attribute (A) conditional on the true outcome (Y). This means the True Positive Rate and False Positive Rate must be the same for all groups. It allows the model to have different prediction rates for different groups, as long as the error rates are balanced.
  3. Sufficiency (Calibration): The true outcome (Y) must be independent of the protected attribute (A) conditional on the prediction (R). This means that for a given prediction score (e.g., a 70% risk score), the actual rate of the outcome should be 70% for all groups.
• The Impossibility Theorem: A crucial and sobering finding is that these three formal definitions of fairness are mutually incompatible. It is mathematically impossible to satisfy any two of them simultaneously, except in trivial cases. This means that achieving "fairness" always involves making a trade-off between competing, reasonable definitions of what it means to be fair.

Part 3: Case Studies in Healthcare Fairness
The lecture presents two case studies that explore bias and fairness in real-world medical datasets.

• Case Study 1: Disparities in Medical and Mental Health (from Irene Chen's work):
  • The Goal: To examine whether machine learning models trained on clinical notes exhibit biases based on race, gender, or insurance type.
  • Method: Topic modeling (LDA) was used to extract topics from notes in two datasets: the MIMIC ICU dataset and a psychiatric dataset from MGH/McLean.
  • Findings (Stereotypes in Data): The topics that were most prevalent in different groups reflected known societal stereotypes and health disparities. For example, in the psychiatric data, male patients had more substance abuse topics, while female patients had more depression topics. In the ICU data, Black patients had more topics related to kidney problems, while White patients had more on atrial fibrillation.
  • Findings (Disparate Error Rates): More importantly, a model trained to predict outcomes (ICU mortality or psychiatric readmission) had different error rates for different groups. For example, the model was significantly more accurate at predicting mortality for men than for women, and for patients with private insurance versus public insurance. This provides concrete evidence of bias in the models.
• Case Study 2: Modeling Mistrust in End-of-Life Care (from Willie Boag's work):
  • The Observation: Analysis of the MIMIC dataset revealed that Black patients spend significantly more time on mechanical ventilation than White patients at the end of life.
  • The Hypothesis: Could this disparity be explained not by physiology, but by a difference in trust between patients/families and the medical system?
  • The Method (Remote Supervision):
    1. Identify Clear Cases of Mistrust: The researchers identified a small set of "red flag" cases from the notes where mistrust was explicitly stated (e.g., "patient leaves against medical advice," "family refuses autopsy").
    2. Learn a "Mistrust" Model: They trained a logistic regression model to predict these explicit labels using thousands of binary features extracted from the structured EHR data (e.g., was the patient in restraints? was family communication documented?).
    3. Apply the Model: This model was then used to assign a "mistrust score" to the entire patient population.
  • The Result: The analysis showed that the disparity in ventilation time between high-trust and low-trust patients was even greater than the disparity between Black and White patients. This suggests that mistrust, which is correlated with race due to historical injustices like the Tuskegee experiment, is a powerful explanatory factor for the observed racial disparity in end-of-life care.

The lecture concludes by emphasizing that fairness is a rich and critical area for future work, requiring both technical innovation and a deep understanding of the societal context and desired goals.

L24: Robustness to Dataset Shift

Part 1: Introduction to Dataset Shift
The lecture introduces the critical problem of dataset shift, which violates the core machine learning assumption that training and test data are drawn from the same distribution. This is a common and dangerous problem when deploying models in the real world.

• The Scenario: A data scientist builds a model at one hospital (e.g., MGH) that works perfectly on held-out test data and even in a local prospective deployment. The central question is: will this model still work a year later, or at a different hospital (e.g., UCSF or a rural clinic)?
• Two Types of Dataset Shift:
  • P(y|x) changes: The relationship between features and labels changes. An example is when the definition of a disease changes between institutions (e.g., one hospital codes for "Type 2 Diabetes" while another just uses a general "Diabetes" code that includes Type 1). The model will likely fail. This type of shift is not the focus of today's lecture.
  • P(x) changes (Covariate Shift): The distribution of the input features changes, but the underlying relationship between features and labels, P(y|x), remains the same. This is the focus of the lecture.
• Two Flavors of Covariate Shift:
  • Adversarial Perturbations: A malicious actor intentionally adds a small, imperceptible amount of noise to an input to cause a misclassification. The famous example is a clear image of a dog that, with a tiny perturbation, is confidently classified as an ostrich. This has serious implications for healthcare, where it could be used for insurance fraud or other malicious purposes.
  • Natural Perturbations: The data distribution changes for natural reasons over time (non-stationarity) or between locations. Examples include changes in lab testing practices, EHR system updates (e.g., the switch from ICD-9 to ICD-10), or differences in patient demographics and clinical practices between hospitals.

Part 2: Identifying and Describing Dataset Shift
Before developing technical solutions, the first step is to identify and characterize potential shifts between populations.

• "Table 1": The Standard of Clinical Research: The lecture emphasizes the importance of "Table 1" from clinical research papers. This table provides detailed descriptive statistics of the study population (e.g., distributions of age, gender, race, common medications, lab values). When sharing a model, one must also share its "Table 1" so that a new institution can compare its own population and make an informed judgment about whether the model is likely to generalize.
• "Datasheets for Datasets": A more recent and formalized proposal for standardizing this process. A datasheet is a questionnaire that forces the dataset creator to document crucial details about the data's motivation, composition, collection process, and pre-processing steps. This helps downstream users understand the potential biases and limitations of the data.

Part 3: Transfer Learning - Adapting to New Distributions
The bulk of the lecture focuses on transfer learning, a set of techniques for adapting a model trained on a source distribution to perform well on a new, target distribution.
Linear Models

• Approach 1: Fine-tuning (Regularization):
  • The Idea: Start with the weights of the model trained on the large source dataset (w_old). Then, train a new model on the small amount of labeled data from the target dataset, but add a regularization term that penalizes the new weights (w) for straying too far from w_old.
  • The Formula: The objective is to min Loss(w; D_target) + λ * ||w - w_old||². This allows the model to "fine-tune" to the new data while retaining the knowledge from the original model.
• Approach 2: Feature Engineering (Manual Representation Mapping):
  • The Idea: When the feature spaces are different between the source and target (e.g., different languages, different EHR coding systems), manually create a common, shared feature space.
  • The Method: The lecture describes a study that transferred a model between two EHR systems (CareVue and MetaVision) within the same hospital. The raw event codes were different, but each had a text description. The researchers mapped the text descriptions from both systems to a common vocabulary (the UMLS) to create a shared feature representation.
  • The Result: The model trained on the original, rich feature space performed best on its own data but generalized poorly. The model trained on the "translated," shared feature space performed slightly worse on its own data but generalized much better to the other system.

Deep Learning Models

• Approach 3: Feature Extraction and Fine-tuning:
  • The Idea: The initial layers of a deep network are thought to learn general, reusable feature representations.
  • The Method: Take a network pre-trained on a large, general dataset (like ImageNet). Chop off the final classification layer(s) and use the outputs of the earlier layers as a fixed "feature extractor." Then, train a new, simple classifier on top of these features using the small target dataset. A more advanced version involves "fine-tuning" the entire network, allowing all weights to be updated, but with a very small learning rate.
• Approach 4: Learning Transferable Embeddings (for Temporal/Sequence Data):
  • The Idea: For sequential data like text or EHR event streams, the goal is to learn embeddings (low-dimensional vector representations) for the input tokens (words, medical codes) that capture their meaning and are transferable.
  • BERT in Healthcare: The lecture highlights the recent, dramatic success of using large, pre-trained language models like BERT in the clinical domain. Embeddings trained on a massive corpus of clinical notes (from MIMIC) and then used as features for downstream tasks (like concept extraction or readmission prediction) have achieved new state-of-the-art results, significantly outperforming models trained from scratch or using non-clinical embeddings. This demonstrates the power of transferring knowledge from a large, unlabeled source corpus to a small, labeled target task.
• Approach 5: Unsupervised Domain Adaptation (Adversarial Training):
  • The Setting: You have a large amount of labeled data from the source and a large amount of unlabeled data from the target.
  • The Idea: Train the network to perform two tasks simultaneously. The main task is to predict the labels for the source data. The secondary, adversarial task is to predict which domain (source or target) a given sample came from.
  • The Method: The network is trained to be good at the main prediction task but bad at the adversarial domain-classification task. This forces the network to learn a feature representation that is not only predictive of the outcome but also indistinguishable between the two domains, making it more likely to generalize.

Part 4: Defenses Against Adversarial Attacks
The lecture concludes by returning to adversarial examples and outlining a principled defense mechanism.

• Finding Adversarial Examples: An adversarial example is found by starting with a correct input and then using gradient ascent to find a small perturbation (δ) that maximizes the classification loss for the true label.
• A Robust Learning Objective: The defense involves changing the training objective. Instead of minimizing the standard empirical loss, we minimize a robust loss.
• The Formula: The new objective is to min_θ E [max_δ Loss(f_θ(x+δ), y)]. In words, we want to find model parameters θ that perform well even on the worst-case adversarial perturbation (max_δ) for each training example.
• Solving the Min-Max Problem: This creates a difficult min-max optimization problem. Recent research has developed methods to solve or approximate this, often by using convex relaxations of the neural network's activation functions. The result is a decision boundary that is explicitly trained to be robust to small perturbations around each training point.

L25: Interpretability

Part 1: The Need for Interpretability
The lecture begins by establishing why interpretability is a critical issue for machine learning, especially in high-stakes domains like healthcare.

• The Problem of Complexity: Modern ML models, particularly deep learning models like GPT-2 with its 1.5 billion parameters, are "black boxes." It is impossible to understand their internal logic by simply inspecting their parameters.
• Cognitive Limits ("The Magical Number 7 ± 2"): The lecture cites George Miller's classic 1956 psychology paper, which found that humans can only hold about seven "chunks" of information in their working memory at a time. This suggests that for an explanation to be useful to a human, it must be simple and sparse; an explanation with 20 different factors is likely to be incomprehensible.
• Why are Models Complex?
  1. Overfitting: Models can fit to noise in the data, creating complex decision boundaries that don't reflect the true underlying signal.
  2. Inherent Complexity: The world itself is complex. Human biology is a product of evolution, not clean engineering, and contains "bizarre stuff." Accurately modeling this reality may simply require a complex model.

Part 2: Two Philosophies of Interpretability
The lecture outlines two main approaches to making complex models understandable.

1. Post-Hoc Explanations ("Just-So Stories"): This is the dominant approach. First, you train a high-performance, complex "black box" model. Then, you use a separate technique to generate a simplified, local explanation for a specific prediction. This doesn't explain the entire model, but it provides a plausible, human-understandable rationale for a single decision.
2. Inherently Interpretable Models: This approach sacrifices some potential performance for simplicity. You deliberately choose to use a simpler, more transparent model from the start, such as a decision tree or a sparse linear model, whose logic can be directly inspected and understood by a human.

Part 3: Post-Hoc Explanations in Detail
This section dives into the history and methodology of generating explanations for existing models.

• Early Expert Systems (MYCIN): The idea of explanation is not new. Expert systems from the 1970s, like MYCIN, were based on human-written rules. Their explanation capability was simply to "play back" the chain of rules that led to a conclusion in response to a user's "Why?" questions. This was a rudimentary but powerful form of transparency.
• LIME (Local Interpretable Model-agnostic Explanations): The foundational modern technique for post-hoc explanations.
  • The Core Idea: To explain why a complex model f made a particular prediction on an input x, LIME learns a simple, interpretable, local model g (like a sparse linear model) that approximates the behavior of the complex model f only in the neighborhood of that specific input x.
  • The Process:
    1. Take the instance x you want to explain.
    2. Generate a new dataset by creating many small, random perturbations of x.
    3. Get predictions from the original complex model f for each of these perturbed instances.
    4. Fit a simple, interpretable model g (e.g., a sparse linear regression) to this new dataset of perturbations and their corresponding predictions from f.
    5. The simple model g is now the "explanation" for the original prediction. For example, the non-zero weights of the linear model highlight the features that were most important for the decision in that local region.
  • Applications: LIME can be applied to text (where it identifies the most important words for a classification) and images (where it identifies "superpixels" or regions of an image that contributed most to the prediction).
• Using Explanations for Model Trust and Debugging: LIME can be used to build trust in a model and identify when it's learning spurious correlations. The lecture gives a famous example of a classifier distinguishing between wolves and huskies that achieved high accuracy by learning to simply detect snow in the background of the husky photos, a flaw that would be revealed by an explanation method like LIME.
• Critiques of LIME: Later work has shown that LIME's explanations can be unstable and highly sensitive to the arbitrary choice of the "neighborhood" size. Alternative methods like LEAFAGE have been proposed to address these issues by sampling from real training data instead of generating synthetic perturbations.

Part 4: A Different Kind of Explanation - Attention Mechanisms
The lecture discusses another popular technique for interpretability in deep learning: attention.

• Attention as Explanation: In models that use an attention mechanism (common in NLP and vision), it's often assumed that the parts of the input that receive the highest "attention" weights are the ones most responsible for the model's output.
• Example: Generating Radiology Reports: The speaker's students built a model that generates a text-based radiology report from a chest X-ray. By visualizing the attention weights, the model can highlight the region of the image it was "looking at" when it generated a specific phrase (e.g., highlighting the heart when writing "moderate cardiomegaly").
• The Controversy ("Attention is Not Explanation"): A recent line of research has challenged this assumption. These studies have shown that the correlation between attention weights and other measures of feature importance (like input gradients) can be weak. It's possible to create models with very different attention patterns that still produce the exact same final prediction. This suggests that while attention is a powerful modeling tool, it should not be blindly trusted as a faithful explanation of the model's reasoning.

Part 5: Inherently Interpretable Models in Detail
The final section explores the alternative philosophy: building models that are simple and transparent by design.

• Falling Rule Lists (Cynthia Rudin's work): This is a powerful example of an inherently interpretable model.
  • The Model: A falling rule list is an ordered if-then-else statement. For example:
    • IF (irregular_shape AND age > 60) THEN risk = 85%
    • ELSE IF (spiculated_margin AND age > 45) THEN risk = 78%
    • ELSE risk = 10%
      This structure is highly readable and mimics a form of human reasoning.
  • The Learning Process: The process of learning an optimal falling rule list is extremely complex, involving a sophisticated Bayesian framework with frequent itemset mining and simulated annealing. The complexity is in the training, not in the final model itself.
  • The Result: Despite their simplicity, falling rule lists have been shown to be surprisingly accurate, achieving performance very close to that of complex black-box models like SVMs and random forests on tasks like predicting hospital readmissions. This demonstrates that there is often not a significant trade-off between interpretability and performance. The primary argument for this approach is that if a simple, interpretable model performs nearly as well as a complex one, we should prefer the simple one.

The lecture concludes with a summary of the topics covered throughout the entire course, from the basics of healthcare data to advanced topics like causality, fairness, and interpretability, and a thank you to the teaching assistants and students.