Embeddings interpretability

PhenoMe ships a correlation engine that bridges abstract deep-learning embeddings and interpretable morphological features. When PCA, t-SNE, or UMAP shows interesting clusters, the engine helps answer:

“What morphological property drives this separation?”

Two correlation modes

Component correlation

Correlates principal components (or t-SNE / UMAP coordinates) with classical properties.

Use when you want to explain the main axes of variation.

Example: “PC1 correlates with eccentricity (r = 0.88)” → the main separation in PCA is driven by cell elongation.

pheno.compute_component_correlation(method="pca")

Embedding-property correlation

Correlates raw embedding dimensions with classical properties.

Use when you want fine-grained insight into which embedding dimensions encode specific traits.

Example: “Embedding dim 142 correlates with nuclear intensity.”

corrs = pheno.compute_embedding_property_correlations()
summary_df = pheno.summarize_embedding_property_correlations(corrs)

For most users, component correlation is the most intuitive starting point.

Multivariate interpretability

Univariate correlation (Pearson / Spearman) looks at one classical property at a time. Deep learning models often rely on combinations of features. Multivariate interpretability trains a secondary model (regression or random forest) to predict the embedding axis from all classical features simultaneously.

Why it matters

Seeing two conditions form distinct clusters only tells you half the story; the critical question is “what geometric or morphological rules is the black-box model using to separate them?”

Multivariate forward probing lets you translate an abstract deep learning axis (for example t-SNE axis 1) into a concrete, readable formula of classical biology.

Reading the output

You get two critical pieces of information per axis:

On-axis fit (R²)

How well do classical properties track this axis on the same dataset?

• High R² (e.g. 0.85) — variation along the axis aligns strongly with combinations of standard features (size, shape, texture). Useful for hypothesis generation about what the layout may reflect.
• Low R² (e.g. 0.15) — classical features weakly track the axis; the embedding may encode patterns outside the current property vocabulary.

Drivers (weights / importances)

Unlike univariate metrics, this shows the combination of features acting together.

Example: if t-SNE axis 1 separates healthy and diseased cells, the model might return [+0.45] Eccentricity + [-0.30] Intensity Entropy - proving both elongation and nuclear texture matter.

Caution

Dimensionality reduction (t-SNE, UMAP, and global PCA) is computed on the full filtered set before LASSO. For transductive layouts, axis coordinates already depend on all samples, so R² is a descriptive on-axis fit, not an unbiased out-of-sample generalization metric. Treat drivers (coefficients / importances) as the primary exploratory output; do not cite R² as validated predictive performance unless you hold out batches or use nested DR on train folds only (feasible for PCA, not standard t-SNE/UMAP).

LASSO regression (sparse linear)

LASSO (L1-regularised regression) identifies a small subset of properties that linearly combine to explain the axis. It shrinks irrelevant properties to exactly zero weight.

• Best for: primary drivers in a human-readable formula.
• Output: descriptive on-axis R² and a list of weights.
• Example: “t-SNE axis 1 is associated (on-axis R² = 0.85) with [+0.65] Area, [-0.42] Circularity, [+0.15] Texture Entropy.”

results = pheno.compute_multivariate_interpretability(model_type="lasso")
# Optional: balance LASSO folds by treatment or batch (when each class has >= cv samples)
# results = pheno.compute_multivariate_interpretability(model_type="lasso", stratify_by="Treatment")
# Default: shows an interactive driver bar chart. Use plot=False for a logger text summary only.

Tip

Set seed for reproducibility of dimensionality reduction and the fitted model. If omitted, PhenoMe falls back to the pipeline’s PhenoMe(seed=...) when set.

Random forest (non-linear)

Random forest captures non-linear relationships and interactions linear models miss.

• Best for: when LASSO gives a low on-axis R² but you still suspect classical properties hold the answer.
• Output: OOB R² (same global-DR caveat for the target axis) plus non-negative feature importances.

results = pheno.compute_multivariate_interpretability(model_type="random_forest")

Reading univariate correlation coefficients

For a quick reference on how to interpret correlation coefficients ($r$), see the Glossary — Correlation coefficient (r).

• r = 0.88 between PC1 and eccentricity → moving along PC1 (left-to-right) corresponds to increasingly elongated cells.
• r = -0.75 between PC2 and area → PC2 encodes cell size.

Strong correlations (|r| > 0.7) suggest a clear biological story. Weaker correlations (|r| < 0.5) may indicate noise or multi-factor effects - that is often when multivariate interpretability becomes useful.

From numbers to hypotheses

1. Compute properties and a dimensionality reduction.
2. Run component correlation for each component (PC1, PC2, …); the top_k properties with highest |r| are the main drivers.
3. Formulate a hypothesis: “PC1 represents cell elongation; clusters along PC1 likely differ in aspect ratio.”
4. Validate: colour the same space by the top property to confirm - pheno.plot_pca(color_by="eccentricity") (if eccentricity is a column) or overlay property values.
5. Report: include correlations in the HTML report via pheno.generate_report(include_correlations=True, top_k_features=...).

Choosing a correlation method

Method	Use when
Pearson (default)	Roughly linear relationships (size, intensity).
Spearman	Monotonic but non-linear; robust to outliers.
Distance correlation	Suspected non-linear dependency; Pearson is low.
Mutual information	Any dependency, including non-monotonic.

For most morphological traits, Pearson is sufficient and GPU-accelerated. Switch to Spearman for outliers or dose-response curves; switch to distance correlation or mutual information when linear metrics miss obvious structure.

Pass the correlation type via correlation_method (not method, which selects the dimensionality reduction: pca, tsne, or umap):

pheno.compute_component_correlation(
    method="pca",
    correlation_method="spearman",
    plot=False,
)

Valid values: "pearson" (default), "spearman", "distance_correlation", "mutual_info". Distance correlation requires the optional dcor package (pip install dcor). See the compute_component_correlation API for the full parameter list.

• Tutorial: 04 - Exploratory analysis and explainability.

See also

Topic	Document
Built-in property names	Property interpretation
Select properties	Select properties
Report with correlations	Report API
Workflow examples	Common workflows