Track: Activity (pEC50 prediction). Metric: RAE, lower is better, with a train-mean predictor scoring about 1.0.
The test set is an analog expansion of the training set: its pEC50 values sit about 0.34 log units above the training mean and it is enriched for actives, so a model that fits the training distribution can degrade on the shift. Most of our effort went into a leak-control protocol that keeps validation trustworthy under that shift rather than into any single model.
The deployable is a three-member blend with one leak-safe calibration gate. On a leak-safe cross-fit estimate it scores an RAE of about 0.560, roughly 45 percent below the 1.0 baseline and ahead of our strongest single model, the graph network at about 0.62 (see Result). Analog Set 2, which decides the ranking, is blind and not directly measurable for us; 0.55 to 0.56 is our honest two-sided estimate.
- All-task exclusion. We train on three readouts at once (below). Any compound used for evaluation is removed from every task, including the auxiliary screens, and compounds are matched across assays on InChIKey rather than SMILES so one molecule cannot pass as two records. Without this, a shared representation can memorize an activity proxy of a compound we are about to score.
- Discover out-of-fold, confirm on a locked set. Blend weights and similar choices are fit on out-of-fold predictions, then confirmed on a held-out set carved once from the training clusters and never used in selection. A benefit that does not reappear on the locked set is not believed.
- No score the leaderboard could compute is optimized against. The 253 unblinded compounds (Analog Set 1) served only as a deployment check, never tuned on, and we treat their RAE as an upper bound on the disjoint Analog Set 2 because the shift erodes in-domain gains.
Folds use Butina clustering at Tanimoto 0.50. The locked-holdout boundary additionally uses single-linkage connected components, with maximum held-out-to-train similarity 0.49, so no near-analog crosses it.
- Dose-response pEC50, about 4,100 compounds: the target.
- Counter-assay pEC50, about 2,600 compounds after deduplication: a PXR-null selectivity readout, used as an auxiliary task.
- Single-concentration primary screen, about 10,900 unique compounds: an activity proxy, used as an auxiliary task and a pretraining signal. It shares no exact compounds with the test set, though the analog expansion gives the test set near-neighbors in it.
The blend combines three families chosen for diverse molecular representation, then applies one calibration gate.
- Multi-view gradient-boosted trees (LightGBM, L1). Late-fused views: RDKit physicochemical descriptors, Morgan fingerprints, and out-of-fold stacked predictions from two auxiliary models (structure to single-concentration activity, and structure to counter-assay selectivity).
- Multitask message-passing network (Chemprop, initialized from CheMeleon), fine-tuned jointly on the three readouts with a masked multitask loss and a partial-freeze schedule. A learned 2D graph representation.
- Fine-tuned 3D foundation model (Uni-Mol), trained to predict pEC50 from a single fixed-seed conformer. A learned 3D-geometry axis distinct from the trees and the graph network.
- Blend: regularized non-negative least squares on out-of-fold predictions, weights about 0.24 / 0.39 / 0.35 (trees / graph / 3D). Non-negativity zeroes a redundant member; all three earn weight, each confirmed on the locked set.
- Inactive-detector gate. A classifier trained on the single-concentration screen predicts in-screen inactivity; its probability drives a one-sided offset that corrects the systematic over-prediction of inactives under the shift. It uses no test data and the offset is fit leak-safely.
- Representation (ships). Adding the 3D model improves the deployment-check RAE by about 0.02, and its advantage grows under the shift: near the in-domain noise floor on the locked set but contributing about 0.02 on the more-shifted deployment check, concentrated in the low-similarity tail. A learned 3D representation transfers to unfamiliar chemotypes more robustly than fingerprints or the 2D graph.
- Task (does not ship). Cross-receptor multitask heads (FXR, LXRα, LXRβ EC50 from public data, leak-gated against test analogs) give a real in-domain gain, with a paired bootstrap of the locked-set contrast excluding zero, but it washes out on the deployment check. The signal is real and shift-fragile, the mirror of the representation axis, which is why we judge ship or no-ship on the shifted tier.
Once Analog Set 1 was unblinded, those 253 labels became legitimate training data, since the ranking is on the disjoint Set 2. The deployed model trains on training plus holdout plus the 253. This forfeits Set 1 as a measurable proxy, so the gate can no longer be calibrated on Set 1 residuals in-sample, and we re-estimate it by a two-fold, cluster-atomic cross-fit over Set 1 (offsets fit on one cluster-half and applied to the other, members retrained per half). The cross-fit gate improves the leak-safe Set-1 estimate from about 0.586 to about 0.556. The offsets are then shrunk by a single global factor, set where harder shrinkage would begin to erode the in-domain gain (a paired-difference test), buying conservative robustness on the structurally distinct Set 2 at a cost within cross-fit noise, about one standard error.
On the 253 deployment check the graph network alone scores about 0.62 and the trees about 0.70; the three-member blend with the gate reaches about 0.564, and the leak-safe cross-fit estimate of the fold-in model is about 0.556 before the shrink. After the shrink, the shipped model's leak-safe cross-fit estimate is about 0.560, which we deploy.
We have no held-out estimate for Set 2: the 253 are now training data and Set 2 is blind, so our blind estimate is about 0.55 to 0.56, two-sided. Two effects oppose: Set 2 is slightly less shifted than Set 1 (a lighter low-similarity tail), lowering absolute error, but it is more active, which can compress the RAE denominator. We do not claim a point improvement over the in-domain number.
Two honesty caveats. The deployed network was the best of four candidates on the locked read, so the estimate carries mild selection optimism. And the submission file uses the revealed labels for the 253 and model predictions for the 260; only the 260 are ranked, and the estimate reflects model performance on those.
We ran a wide battery of further levers after fixing the deployable; none beat the leak-safe estimate beyond noise. A fourth in-family network as a stack member added nothing. An empirical-Bayes per-series level correction failed because the per-series offset, though real, is not predictable from series-level covariates, the standard hierarchical result that a new disjoint series receives the pooled mean. Within-series monotone rank-sharpening failed because the within-series ordering is not clean enough for a label-free correction. TabPFN as a base model was weak and highly correlated with existing members. Detector-partitioned blend weights were redundant with the gate. The crude and semi-pure dose-response tiers as source-weighted training data were measurably worse, those compounds being structurally far from the test series. The 3D model is conformer-specific: fine-tuned on one fixed conformer, it is degraded by averaging over multiple conformers at inference, so a conformer-ensemble gain would require augmentation at training time.
As a direct test of the representation limit behind the activity cliffs, we computed semi-empirical quantum-chemical descriptors for the cliff-bearing compounds (GFN2-xTB: frontier-orbital energies, conceptual-DFT reactivity indices, Mulliken charges, and condensed Fukui functions). Orthogonalized against the electronic descriptors already in the model, they separate within-series cliffs no better than chance and below a generic non-electronic structural control, so they carry no cliff-relevant signal beyond what the model already has. The cliff floor is therefore representation-bound across all five representations we examined: 2D fingerprints, a 2D learned graph network, hand-crafted 3D geometry, a learned 3D model, and quantum-chemical electronic structure. The residual error is dominated by measurement noise and by information and representation limits, not an unexploited modeling lever.
- CheMeleon: Descriptor-based Foundation Models for Molecular Property Prediction (arXiv:2506.15792).
- Zhou G, Gao Z, Ding Q, et al. Uni-Mol: A Universal 3D Molecular Representation Learning Framework. ChemRxiv, 6 March 2023. DOI: 10.26434/chemrxiv-2022-jjm0j-v4.
- Bannwarth C, Ehlert S, Grimme S. GFN2-xTB: An Accurate and Broadly Parametrized Self-Consistent Tight-Binding Quantum Chemical Method with Multipole Electrostatics and Density-Dependent Dispersion Contributions. J. Chem. Theory Comput. 2019, 15 (3), 1652-1671. DOI: 10.1021/acs.jctc.8b01176.