Calibrating Splicing Predictors Into PP3/BP4: Likelihood Ratios Exist, Validation Does Not

Question: How are computational splicing predictors being calibrated into ACMG/AMP PP3 and BP4 evidence strengths, and what likelihood-ratio or Bayesian frameworks have been proposed for non-canonical intronic variants?

Answer

Across these papers, the field has split into two calibration practices that are not yet reconciled. The dominant practice is threshold-on-score: a SpliceAI (or Pangolin, or SpliceTransformer) cut-off is chosen, and PP3 or BP4 is attached at a strength chosen by expert convention — a practice visible in cohort work that simply applies "SpliceAI >0.5" as a filter and then confirms with RNA [18] or applies SpliceAI Δ≥0.2 to redefine missense variant interpretation, while explicitly noting that splice prediction in classification pipelines "is not a standardised process" [4]. The minority, and technically correct, practice converts scores to likelihood ratios against a labelled reference set and derives thresholds from the LR, and exactly one paper in this corpus does so specifically for the non-canonical intronic space, proposing PP3 at MetaSplice score ≥0.544 and BP4 at ≤0.0017, both at Supporting strength, derived from ClinVar-calibrated likelihood ratios [11]. Taken together, these papers support a clear position: LR-based calibration is the right architecture and is being demonstrated for non-canonical intronic variants, but no independently validated Bayesian framework yet exists for that space, and the evidence base used to fit such frameworks — ClinVar — is itself confounded in ways that the retrieved benchmarks make measurable.

Likelihood-ratio calibration has displaced score cut-offs conceptually, but only one rule covers non-canonical intronic variants

MetaSplice is built explicitly on the premise that "clinical classification requires evidence about pathogenicity rather than predicted splicing alone," and derives its PP3/BP4 cut-offs from likelihood ratios calibrated against ClinVar rather than from score distribution [11]. Its reported practical effect is a specificity gain the score-threshold approach cannot deliver: PP3 was assigned to 0.20% of 37,263 benign variants versus 4.44% under the ClinGen-recommended SpliceAI thresholds, and to 1.0% versus 31.2% of formerly uncertain variants since reclassified benign [11]. The author's own framing is that the rule is a candidate warranting independent evaluation, not a specification [11]. Against this, the only other explicit ACMG/AMP calibration framework in the corpus was developed for missense, not splicing, and reaches a cautionary conclusion: strengths of evidence assigned by expert opinion "are not universally applicable and require empirical validation," because concordant predictor and assay results often fail the additive assumptions of the guidelines [7].

Joint and multi-tool calibration is better behaved than any single-tool threshold, but its behaviour in intronic sequence is untested

P-KNN demonstrates that the existing per-tool calibration process "is only valid when pre-committing to a specific tool," which blocks clinicians from combining complementary predictors, and that jointly calibrating 13 tools yielded better-calibrated probabilities and stronger evidence than the best single tool, with mean log likelihood ratios of 2.56 versus 2.24 for pathogenic and 2.29 versus 1.56 for benign variants [15]. The same study reports that guidelines-based evidence summation systematically overestimated pathogenicity, whereas joint calibration correctly integrated computational predictions with experimental measurements [15]. Independent evidence that naive combination is not a fix comes from the MFASS benchmark, where a calibrated consensus of three deep-learning sequence-window predictors did not meaningfully improve over Pangolin alone on an exon-grouped held-out split [2].

Predictor error is structured by distance to the splice site, so any global PP3 threshold is miscalibrated by construction in the deep intronic space

Stratifying MFASS by distance exposes a shared blind spot: all five tools detect disruptions within a few bases of the splice site well, recall declines sharply in the exon interior, and 19% of disrupting variants are missed by every tool, with the shared misses enriched away from splice sites [2]. That MMSplice — the one model built for modular exonic and intronic effects rather than splice-site recognition — shows the same decline indicates the blind spot is not an artifact of splice-site-centric architectures [2]. A separate evaluation of state-of-the-art models reports that near-perfect aggregate performance masks substantial gaps, including splice sites with low usage rates and tissue-specific splice sites [13]. At the level of the deployed annotation layer, outdated gene annotations and liftover errors affect 8.34% of theoretical SNVs in the precomputed SpliceAI scores and the accuracy of predictions in 34.98% of disease-associated panel genes, and re-running SpliceAI increased predicted splice-altering variants by 18.2% in 7,221 participants [6].

The calibration reference set is the binding constraint, and it is circular and prevalence-confounded

Deriving LRs from ClinVar, as MetaSplice does, means the calibration labels are themselves partly downstream of the predictors under calibration — a concern MetaSplice addresses only by training with each variant scored by a model trained without its gene [11]. More generally, reported accuracy of variant pathogenicity prediction is inflated by differences in the prevalence of pathogenic variants across genomic contexts, so genomic heterogeneity rather than model quality can drive apparent performance [16]. The alternative reference standard — experimental measurement — is being built but is not yet aligned with the intronic space that matters: OpenSplice quantifies >590,000 variants but covers exonic and proximal intronic substitutions in >600 exons [8], and COMPASS covers 87,546 variants across five cell lines while reporting that important gaps remain against state-of-the-art predictive models [5].

Predicted splicing effect is not the same quantity as pathogenic probability, and intermediate-effect alleles break the binary LR premise

GT>GC (+2T>C) 5' splice-site variants show that in silico predictors "cannot reliably discriminate between complete and partial splice disruption," and that experimental systems differ in their ability to detect subtle splicing outcomes [3]. The authors therefore argue for classification frameworks that incorporate quantitative functional data and capture the continuum of variant effects rather than binary loss-of-function calls [3]. Splice-site distance itself governs the switch from canonical to alternative/cryptic site usage, with mutations between two nearby splice sites acting with pronounced directionality [19]. Attempts to make prediction quantitative rather than binary are underway: SpliceAI2 reconstructs complete transcripts and quantifies effect magnitude, finding that predicted splice-altering variants account for 15% of excess genetic burden in a rare disease cohort with approximately half in deep intronic regions [9].

Genuine score-to-outcome calibration exists, but it has been applied to synthetic and assay readouts, not to clinical classification

A unified calibration mapping eleven predictors' scores onto the measured fraction of spliced reads now exists, ranked SpliceTransformer above AlphaGenome and SpliceAI on heavily recoded synthetic constructs, and allows scores to be interpreted as splicing outcomes and compared between tools [10]. On the RNA-quantification side, BRAID attaches conformal intervals to ΔPSI calibrated from RNA-seq-to-RT-PCR residuals, achieving 0.971 RT-PCR coverage where MAJIQ, betAS and rMATS intervals reached 0.518, 0.734 and 0.633 [1]. For functional data, ExCALIBR argues that gene-specific score thresholds "lack the rigor of calibration" and replaces them with variant-specific probabilities of pathogenicity [17]. None of these three is wired to PP3/BP4.

Conflicts and gaps

The central conflict is over where calibration should live. MetaSplice positions calibration at the variant-class level with ClinVar as truth and reports large specificity gains over ClinGen-recommended SpliceAI thresholds [11]; a foundation-model probe instead finds that classification accuracy "benefitted from locus-specific thresholding rather than mutation-class calibration" [12]. These are directly opposed prescriptions for the same problem. A second gap: no retrieved paper supplies an LR-calibrated PP3/BP4 rule for the polyrimidine tract or for deep intronic positions separately — MetaSplice explicitly reports its advantage was "not clear in the polypyrimidine tract" even as it was largest at deep intronic positions [11]. A third: clinical specifications remain gene-by-gene expert products, illustrated by a VCEP that modified eight of 28 evidence codes with in silico data folded into a gene-specific specification [14], and by bespoke guidance for snRNA genes where existing guidance "does not adequately capture the unique features" [20] — i.e. the operational calibration is still curator-mediated rather than LR-derived. Fourth, the frequency of the target class is uncertain: in FBN1, 70% of candidate splice variants lay beyond the ±8 splice regions and only 14/20 were RNA-confirmed [18], which sets an upper bound on how much any predictor-only rule can contribute.

What would change the answer

A blinded, prospective test of the MetaSplice PP3/BP4 rule inside a VCEP curation stream, with variant-level rather than class-level labels held out by gene, would settle whether ClinVar-calibrated LRs transfer or merely re-encode ClinVar. Calibration restricted to deep intronic and polypyrimidine-tract strata, fitted on OpenSplice-style saturation mutagenesis extended past the proximal intron [8] and on COMPASS-scale cell-type-resolved measurements [5], would test whether the distance-dependent blind spot [2] is a calibration problem or an irrecoverable information problem. Re-deriving all thresholds from MANE-corrected, re-run SpliceAI outputs rather than precomputed scores would remove a known 8.34% annotation error floor [6]. Finally, applying conformal interval logic to clinical evidence — reporting a PP3 interval given ΔPSI uncertainty rather than a point strength — would make intermediate-effect alleles like GT>GC representable within the Bayesian framework instead of being forced into it [3] [1].

References

[1] Park, J., Kang, K.. "BRAID: RT-PCR-calibrated conformal intervals for splicing ΔPSI". bioRxiv. 2026-07-07T00:00:00.000Z. URL: https://doi.org/10.64898/2026.07.02.736001

[2] Znabu, B. F., Atif, Z., Devkota, P., Alemu, R. H., K. C., J.. "A Reproducible MFASS Benchmark of Splice-Disruption Predictors Reveals a Shared Exon-Interior Blind Spot". bioRxiv. 2026-07-26T00:00:00.000Z. URL: https://doi.org/10.64898/2026.07.21.739871

[3] Lin, J.-H., Wu, H., Tang, X.-Y., Masson, E., Cooper, D. N., Ferec, C., Liao, Z., Zou, W.-B., Chen, J.-M.. "When splicing is not all or none: GT>GC 5' splice-site variants as a model for intermediate effects and challenges in variant classification". bioRxiv. 2026-01-22T00:00:00.000Z. URL: https://doi.org/10.1101/2025.10.30.685113

[4] Rintari, M. R., Chundru, V. K., Oquendo, C. J., Leslie, J., Pagnamenta, A. T., Baralle, D., Wright, C. F.. "Investigating the pathogenicity of missense spliceogenic variants using whole-genome sequencing, RNA-sequencing, and proteomics data from population and clinical research cohorts". medRxiv. 2026-10-13T00:00:00.000Z. URL: https://doi.org/10.64898/2026.10.09.26365070

[5] Koplik, S. E., Yu, A. M., Shelby, M. R., Fonseca, G. C., Roco, C. M., Zhang, Y., Bogard, N., Sabo, A. K., Rosenberg, A. B., Linder, J., Seelig, G.. "Massively parallel assay of human splice variants reveals cis-regulatory drivers of disease-associated and cell type-specific splicing regulation". bioRxiv. 2025-10-13T00:00:00.000Z. URL: https://doi.org/10.1101/2025.10.12.681955

[6] Martin Geary, A. C., Lecoquierre, F., Walker, S., Whiffin, N., Dawes, R.. "Using SpliceAI to triage splice-altering variants in 7,220 individuals with rare conditions highlights limitations of the precomputed scores". medRxiv. 2025-08-28T00:00:00.000Z. URL: https://doi.org/10.1101/2025.08.27.25334471

[7] Pew, S. T., Wiffler, M. B., Thomas, A., Boyle, J. L., Cline, M. S., Camp, N. J., Goldgar, D. E., Tavtigian, S. V.. "When Two plus Four Does Not Equal Six: Combining Computational and Functional Evidence to Classify BRCA1 Key Domain Missense Substitutions.". medRxiv. 2025-05-01T00:00:00.000Z. URL: https://doi.org/10.1101/2024.10.09.24315186

[8] Quarantani, G., Clarke, J., Thompson, M., Sang, F., Valcarcel, J., Lehner, B.. "OpenSplice: the impact of half a million mutations on the alternative splicing of 600 human exons". bioRxiv. 2026-05-23T00:00:00.000Z. URL: https://doi.org/10.64898/2026.05.22.727141

[9] Jaganathan, K., Chen, J., Liu, X., Zhang, Y., Kim, Y., Banerjee, A., Fiziev, P. P., Wilcox, N., Taskiran, I. I., Cai, Y., Wigdor, E. M., Chew, G.-L., Png, G., McRae, J. F., Reddy, A. J., Chao, K.-H., Mehio, R., Sanders, S. J., Farh, K. K.-H.. "A unified framework for quantitative splicing and transcript prediction". bioRxiv. 2026-10-11T00:00:00.000Z. URL: https://doi.org/10.64898/2026.10.08.757673

[10] Bellido Molias, F., Kudla, G.. "Evaluating the performance of splicing predictors on thousands of synthetic gene variants". bioRxiv. 2026-08-27T00:00:00.000Z. URL: https://doi.org/10.64898/2026.08.24.746734

[11] Liu, X.. "MetaSplice: pathogenicity prediction and ACMG/AMP evidence for intronic variants outside the canonical splice sites". bioRxiv. 2026-10-19T00:00:00.000Z. URL: https://doi.org/10.64898/2026.07.23.740393

[12] Alavi, M. V.. "Probing Genomic Foundation Models with Splice-Variant Perturbations". bioRxiv. 2026-10-09T00:00:00.000Z. URL: https://doi.org/10.64898/2026.10.02.756289

[13] Runyan, M., Gupta, S., Leshaem, Y., Geller-McGrath, D., Liu, C., Saferali, A., Dy, J., Radivojac, P., Tesfaigzi, Y., Castaldi, P., Paul, A.. "Improving splice site usage prediction with SPLAIRE". bioRxiv. 2026-06-11T00:00:00.000Z. URL: https://doi.org/10.64898/2026.06.08.731019

[14] Ritter, D. I., Badduke, C., Doonanco, K., Kang, H. C., Pesaran, T., Ridd, S., Sheen, C., Farncombe, K. M., Giles, R. H., Luo, M., Pipko, N., Tsoi, C. T., McGoldrick, K., Mighton, C., Abu Khashabeh, R. H., Sanabria-Salas, M. C., Talab, Y., Deka, K. B., Jacobs, M. F., Tuzlali, E., Gallinger, B., Griffith, M., Krysiak, K., Machado, J., Maher, E. R., Tirosh, A., Kim, R. H.. "Multi-Platform Curation in the Development of ACMG/AMP Specifications for Von Hippel Lindau (VHL) Disease". medRxiv. 2025-08-27T00:00:00.000Z. URL: https://doi.org/10.1101/2025.08.25.25334371

[15] Lin, P.-Y., Brandes, N.. "P-KNN: Maximizing variant classification evidence through joint calibration of multiple pathogenicity prediction tools". bioRxiv. 2025-09-27T00:00:00.000Z. URL: https://doi.org/10.1101/2025.09.24.678417

[16] Lu, B., Liu, X., Lin, P.-Y., Brandes, N.. "Genomic heterogeneity inflates the performance of variant pathogenicity predictions". bioRxiv. 2025-09-10T00:00:00.000Z. URL: https://doi.org/10.1101/2025.09.05.674459

[17] Zeiberg, D., Stewart, R. C., Jain, S., Tejura, M., McEwen, A. E., Fayer, S., Sverchkov, Y., Craven, M., Pejaver, V., Rubin, A. F., Starita, L. M., Fowler, D. M., O'Donnell-Luria, A., Radivojac, P.. "Gene-based calibration of high-throughput functional assays for clinical variant classification". bioRxiv. 2026-02-16T00:00:00.000Z. URL: https://doi.org/10.1101/2025.04.29.651326

[18] Walker, S., Bunyan, D. J., Thomas, H. B., Kesim, Y., Kershaw, C. J., Holloway, J., Wai, H., Day, M., Smith, C. L., Hawkes, G., Wood, A. R., Weedon, M. N., Blair, E., Curtis, S. L., Fielden, C., Evans, J., Whittington, R., Smithson, S. F., Cox, H., Clift, P., McEntagart, M., Prapa, M., Alsters, S., Morris-Rosendahl, D., Dean, J., Morrison, P. J., Dixit, A., Sarkar, A., Prescott, K., Amel Riazat Kesh, L., Tharakan, R., Turner, C., Ellard, S., Shaw-Smith, C., Fasham, J., Clowes, V., Holden, S., Somarathi, S., Mercer, C., Berry, I., O'Keefe, R. T., Banka, S., Baralle, D., Thomas, N. S., Baple, E.. "Utility of genome sequencing and group-enrichment to support splice variant interpretation in Marfan syndrome". medRxiv. 2025-02-23T00:00:00.000Z. URL: https://doi.org/10.1101/2025.02.18.25321172

[19] Li, C., Unic, S., Kuban, M., Brueggemann, M., Koenig, J., Zarnack, K., Legewie, S.. "Context-dependent effects of mutations on complex splicing decisions". bioRxiv. 2025-03-11T00:00:00.000Z. URL: https://doi.org/10.1101/2025.03.06.641829

[20] DSouza, E. N., Blakes, A. J., Paluch, R., Banka, S., Chopra, M., Coffey, A. J., Depienne, C., Galej, W. P., Mazoyer, S., Nava, C., O'Donnell Luria, A., O'Toole, J., Riestra Crespo, P., Rivolta, C., Sanders, S. J., Whiffin, N.. "Guidance for clinical variant classification in genes for spliceosomal small nuclear RNAs". medRxiv. 2026-08-04T00:00:00.000Z. URL: https://doi.org/10.64898/2026.08.03.26359558

How this survey was made

This survey answers one question: “How are computational splicing predictors being calibrated into ACMG/AMP PP3 and BP4 evidence strengths, and what likelihood-ratio or Bayesian frameworks have been proposed for non-canonical intronic variants?”

We keep our own collection of recent preprints and refresh it every day from bioRxiv, medRxiv, arXiv, PsyArXiv, SocArXiv and ChemRxiv. For each survey, we search that collection for recent work and keep the strongest matches: papers that use the same terms as the question, and papers on the same topic in different words.

The text was written from the titles and abstracts of those papers; the full texts were not read. Statements that draw on a particular paper are marked with a number in brackets. The reference list just above is the 20 papers whose work this text draws on, chosen from the 120 papers those searches returned; the rest of the shortlist is not printed. It is not a complete review of everything published on the question.

Treat this as a starting point and a reading list, not a peer-reviewed review. The searches behind this survey ran on October 21, 2026 and covered work from the previous two years.