Results from the paper
Interactive versions of the main results from Joint Machine Learning Model for Longitudinal Functional Decline and Time-to-Intervention Prediction: An Application to Amyotrophic Lateral Sclerosis (under review). Hover over any point for exact values, or use the ⋯ menu on a chart to save it as an image.
- Held-out database
- Models were trained on all databases but one and tested on the remaining one (ArQ, IDPP or PROACT), so each result reflects patients the model never saw during training.
- Concordance index (C-index)
- Discrimination. How well a model ranks patients by who reaches gastrostomy first, weighted to account for censoring (IPCW). 1 is a perfect ranking; 0.5 is no better than chance. Higher is better.
- Margin median absolute error
- The median difference, in days, between the predicted time to gastrostomy and the margin time (see Figure 2). Lower is better.
- Feature sets
- Baseline: characteristics at diagnosis such as age, sex, site of onset, diagnostic delay, phenotype and genetics. Onset Slope: rates of functional decline from symptom onset to the first assessment.
- Models
- Cox Linear and Cox XGBoost are proportional-hazards survival models. MAE-margin XGBoost is the proposed model, trained to directly minimise the absolute error against margin times. The Naive model is a simple reference rule based on 10% weight loss.
Figure 1
Progression classes and time to gastrostomy
Patients were grouped into latent classes by how quickly each functional measure declined. For every class, the model estimates the average trajectory of the measure together with the probability of having had a gastrostomy over the first three years.
How to read it
- The figure has four blocks, one per measure: ALSFRS-R total, bulbar score, swallowing (Q3) and respiratory function (% predicted).
- In each block, the left panel shows the average trajectory of the measure and the right panel shows the cumulative probability of gastrostomy, both over days since the first assessment.
- Colour identifies the progression class, from very slow (dark blue) to fast (red). Shaded bands show confidence intervals.
- ALSFRS-R total and bulbar score have four classes; swallowing and respiratory function have three.
What it shows
- Faster decline goes with earlier gastrostomy for every measure. For ALSFRS-R total, the probability of gastrostomy by 3 years rises from 23% in the very slow class to 80% in the fast class.
- Swallowing separates patients most sharply: by 2 years, 85% of the fast class have had a gastrostomy, compared with 5% of the slow class.
- For bulbar score, the probability by 3 years ranges from 15% (very slow) to 88% (fast); for respiratory function, from 21% (slow) to 70% (fast).
- The late rises in some fast-class trajectories likely reflect the few fast progressors still followed up at that point, so they should be read with caution.
Figure 2
From censored follow-up to margin times
Most patients do not reach gastrostomy while they are followed up, so for them we only know that it had not happened yet. Instead of discarding these censored patients, the model is trained on a margin time that adds their expected remaining time to gastrostomy.
How to read it
- The horizontal axis is each patient's observed follow-up time in years: either the time to gastrostomy or the time at which they were censored.
- The coloured lines (left axis) show the margin time used as the training target. For patients who had a gastrostomy (orange, dashed) it simply equals the observed time.
- The grey line (right axis) shows the percentage of the cohort still at risk at each point in time.
- The dotted vertical line marks the 1,080-day (≈3-year) horizon over which the expected remaining time is estimated.
What it shows
- The cohort includes 7,388 patients from IDPP and PROACT, of whom 1,749 (24%) had a gastrostomy during follow-up and 5,539 were censored.
- For censored patients, the expected remaining time is estimated from the Aalen–Johansen cumulative incidence of gastrostomy, treating death as a competing risk.
- The added time shrinks as follow-up gets longer: a median of about 1.9 years for patients censored in their first year, 1.2 years in the second and 0.5 years in the third. Beyond the horizon, no time is added.
- By the 3-year horizon, 34% of the cohort is still at risk; by 8 years, fewer than 2% are.
Figure 3
Model performance on held-out databases
The primary benchmark, comparing the four approaches and two feature sets. Each row is one held-out database; dots are point estimates and bars show confidence intervals.
How to read it
- Left: concordance index — higher means patients are ranked more accurately by time to gastrostomy.
- Right: margin median absolute error in days — lower means the predicted timing is more accurate.
- Colour identifies the model; the position within each row identifies the feature set.
- Each panel has its own axis range, so compare values within a panel.
What it shows
- Onset-slope features improve the C-index in 8 of 9 model–database combinations; for example, MAE-margin XGBoost in ArQ rises from 0.74 to 0.85.
- MAE-margin XGBoost gives the most accurate timing in ArQ (381 days, onset slope) and IDPP (303 days, baseline), and is second in PROACT (348 days versus 312 for Cox XGBoost).
- Cox Linear ranks patients well (C-index up to 0.85), but its timing errors are roughly two to four times larger (744–1,282 days): ranking patients and predicting when they will need an intervention are different tasks.
- The naive 10% weight-loss rule is no better than chance at ranking (C-index 0.36–0.50) and is off by 496–591 days.
Figure 4
Predictive value of individual features
Each row is a separate model trained on a single feature, showing how much information that feature carries on its own.
How to read it
- Left: concordance index. Right: margin median absolute error in days.
- Bars show the confidence interval and the black tick marks the mean.
- Features are listed in the same order in both panels, from the most to the least informative overall.
What it shows
- How quickly function declines early on is the most informative: the bulbar-score slope from onset to first visit (C-index 0.84) and the swallowing (Q3) slope (C-index 0.82, with the lowest error at 338 days).
- ALSFRS-R total slope (0.74), site of onset (spinal 0.71, bulbar 0.70) and diagnostic delay (0.70) carry moderate information.
- Genetic markers (SOD1, C9orf72, TARDBP, FUS) and clinical phenotype are close to chance on their own (C-index 0.43–0.54), likely in part because each applies to relatively few patients.
- These results show each feature's signal in isolation; features can still add value in combination.
Figure 5
Sensitivity analysis: MICE imputation
The benchmark from Figure 3, repeated with missing covariates filled in using multiple imputation by chained equations (MICE).
How to read it
- The layout is the same as Figure 3: C-index on the left (higher is better), margin median absolute error in days on the right (lower is better).
- Compare each point with its counterpart in Figure 3 to see the effect of imputation.
What it shows
- Cox Linear is essentially unchanged (for example, a C-index of 0.85 in ArQ in both analyses).
- Both XGBoost models get worse with imputation: the error of MAE-margin XGBoost (onset slope) rises from 381 to 808 days in ArQ and from 348 to 815 days in PROACT, and its C-index in PROACT drops from 0.81 to 0.66.
- Onset-slope features still improve the C-index of the Cox models in five of six model–database combinations.
- The naive model does not use these covariates, so its results are identical.