PhenoMe API Reference

Tier: Public API

Auto-generated

This page is rebuilt from docstrings in phenome.pipeline, the mixins under phenome.mixins, and phenome.core.pipeline_results.

See also: Getting started · Concepts · Visualization · Distances · Model wrappers

Class PhenoMe

Main class for phenotyping analysis using deep learning embeddings.

Provides: compute_properties, property_stats_by_group; compute_clustering, detect_outliers, find_prototypes; compute_reference_distances; correct_batches; plot_pca, plot_tsne, plot_umap, and related methods.

Embedding lifecycle:

• Eager: When no checkpoint is used, embeddings live in results.embeddings (np.ndarray).
• Lazy: When a checkpoint is active (self._db), results.embeddings is None; embeddings are read on demand via get_embeddings() from the HDF5 file.
• Temporal: Rows from process_temporal_images() use self._temporal_embeddings while _db is open; get_embeddings() merges them with checkpoint rows by index. After checkpoint_context() exits with temporal data present, the merged array is copied to results.embeddings and the temporal buffer is cleared.

Args:

• device: Optional torch.device for GPU-accelerated analysis operations. Recommended for faster computation. If None, operations run on CPU.
• seed: Optional random seed for reproducibility (default: None). When set, used for all random operations (determinism, clustering, t-SNE, UMAP, mutual information, random sampling in reports). If None, no seeds are set anywhere.
• use_gpu_for_dr: If True and device is CUDA, uses TorchDR for GPU-accelerated dimensionality reduction (PCA, t-SNE, UMAP). If False (default), always use sklearn/umap-learn on CPU.

PhenoMe(
    device: torch.device | str | None = None,
    seed: int | None = None,
    use_gpu_for_dr: bool = False
)

Data ingestion and embedding extraction

Method

find_files

PhenoMe.find_files(
    self,
    image_dir: str,
    mask_dir: str | None = None,
    extensions: list[str] | None = None,
    metadata_fn: collections.abc.Callable[[str], dict[str, typing.Any]] | phenome.metadata.base.MetadataBase | None = None,
    on_missing_metadata: str = 'drop',
    mask_filename_column: str | None = None,
    mask_extensions: list[str] | None = None,
    channel_names: list[str] | None = None
) -> DataFrame

Discover image and mask files from directories, extract metadata, and cache internally.

Recursively scans image_dir for image files and optionally aligns them with masks from mask_dir. Extracts per-file metadata using metadata_fn and stores the result as the internal file DataFrame used by process_images(), compute_properties(), and inspect_data().

Args:

• image_dir (str): Root directory or glob pattern for image files. Scanned recursively. Example: “path/to/images/” or “path/to/**/*.tif”.
• mask_dir (str or None): Root directory for corresponding mask files. If provided, adds a ‘mask_path’ column to the output DataFrame. Default: None (no masks).
• extensions (list of str or None): File extensions to include (e.g., [“.tif”, “.png”, “.jpg”]). If None, defaults to common image formats. Default: None.
• metadata_fn (callable, MetadataBase, or None): Function or object to extract metadata from file paths. Signature: fn(file_path: str) -> dict[str, Any]. Examples:
  • DefaultMetadata(): Simple metadata.
  • PathTemplateMetadata(template="..."): Extract from path pattern.
  • lambda p: {'batch': p.split('/')[-2]}: Custom function. If None, no metadata extracted. Default: None.
• on_missing_metadata (str): How to handle files with missing metadata. One of:
  • ‘drop’: Remove rows with incomplete metadata.
  • ‘keep’: Include rows with NaN metadata. Default: ‘drop’.
• mask_filename_column (str or None): Metadata column name used to locate mask files. If set, masks are located using this column value. Useful for custom mask naming. Default: None (standard alignment logic).
• mask_extensions (list of str or None): Extensions to try when exact mask filename fails (e.g., [“.png”, “.tif”]). Default: None (exact filename required).
• channel_names (list of str or None): Optional labels for image channels, in channel-axis order (same order as list entries in file_path for multi-file samples). When set, multi-channel property columns use these labels as suffixes (e.g. intensity_entropy_DAPI instead of intensity_entropy_ch0). Names must be non-empty, unique (case-insensitive), and contain no whitespace. Default: None (integer _ch{idx} suffixes).

Returns:

pd.DataFrame: File discovery DataFrame with columns:

• ‘file_path’: (str or list of str) Image file path(s).
• ‘mask_path’: (str, if mask_dir set) Corresponding mask path.
• Metadata columns: Custom columns from metadata_fn if provided. Shape: (N, 2+M) where N is number of images, M is metadata columns.

Raises:

• FileNotFoundError: If image_dir does not exist.
• ValueError: If all files are dropped due to on_missing_metadata='drop', or if channel_names is invalid.
• RuntimeError: If metadata extraction fails or masks cannot be resolved.

Example:

Discover images with metadata:

>>> pm = PhenoMe()
>>> df = pm.find_files(
...     "data/images/",
...     metadata_fn=PathTemplateMetadata(template="batch_{batch}/sample_{id}"),
...     extensions=[".tif", ".png"]
... )
>>> print(df.shape)
(150, 4)  # 150 images, 4 columns: file_path, batch, id, [others]

With masks:

>>> df = pm.find_files(
...     image_dir="data/images/",
...     mask_dir="data/masks/",
...     extensions=[".tif"]
... )
>>> print(df.columns)
Index(['file_path', 'mask_path'], dtype='object')

Named channels for property suffixes:

>>> df = pm.find_files(
...     "data/images/",
...     channel_names=["DAPI", "GFP"],
... )

See Also:

• set_file_df: Manually provide a file DataFrame.
• process_images: Process discovered images.

Method

set_file_df

PhenoMe.set_file_df(
    self,
    file_df: pandas.DataFrame,
    channel_names: list[str] | None = None
) -> DataFrame

Set the internal file DataFrame used for processing.

Validates that file_df has a ‘file_path’ column and optionally ‘mask_path’. Stores a normalized copy for use by process_images, compute_properties, inspect_data, and related methods.

Args:

• file_df: DataFrame with ‘file_path’ column (str or list of str per row). May include ‘mask_path’ and metadata columns.
• channel_names: Optional labels for image channels, in channel-axis order (same order as list entries in file_path for multi-file samples). When set, multi-channel property columns use these labels as suffixes (e.g. intensity_entropy_DAPI instead of intensity_entropy_ch0). Default: None (integer _ch{idx} suffixes).

Returns:

The normalized DataFrame that was stored.

Method

process_images

PhenoMe.process_images(
    self,
    model_wrapper: phenome.utils.model_wrapper.ModelWrapper,
    batch_size: int = 32,
    num_workers: int = 4,
    filters: dict[str, list[typing.Any]] | None = None,
    exclude: dict[str, list[typing.Any]] | None = None,
    channel_mode: Literal['split', 'combined'] = 'split',
    channels: list[int] | None = None,
    preprocessing_fn: collections.abc.Callable[[numpy.ndarray], numpy.ndarray] | None = None,
    custom_transformations: typing.Any | None = None,
    preprocessing_description: str | None = None,
    transformations_description: str | None = None,
    append: bool = False,
    resize_size: int | None = 224,
    pad_size: int | None = None,
    checkpoint_path: str | None = None,
    force_rgb: bool = True,
    l2_normalize_channels: bool = True,
    save_every: int = 5,
    lazy_checkpoint: bool = True,
    force_reprocess: bool = False,
    progress_callback: collections.abc.Callable[[int, int, str], None] | None = None
) -> None

Extract embeddings from images using a pretrained or custom model.

Reads images from the file DataFrame (set via find_files() or set_file_df()), processes them through the model in batches, and stores embeddings and metadata in self.results. Supports filtering, channel selection, custom preprocessing, and checkpointing for resumable processing.

Embedding storage:

• Eager (default): Embeddings stored in results.embeddings (ndarray).
• Lazy (with checkpoint): Embeddings stored in HDF5; results.embeddings is None and embeddings are loaded on-demand via get_embeddings().
• Temporal (incremental): New images added to existing checkpoint via process_temporal_images().

Default processing order (when custom_transformations is None):

1. Load image (native dtype)
2. Optional preprocessing_fn
3. Channel selection / channel_mode / force_rgb
4. TypeMaxNorm intensity scaling (identity if max <= 1)
5. Optional pad, then resize (default 224)
6. ImageNet mean/std normalize
7. Vision model -> embedding
8. Split mode: L2-normalize each channel embedding (default True)

A concise processing pipeline is logged at INFO. Inspect get_processing_pipeline() or print_processing_pipeline(). Custom custom_transformations replace steps 4-6.

Args:

• model_wrapper (ModelWrapper): Model instance (e.g., from load_dinov2_model()). Must implement _get_embeddings(tensor: torch.Tensor) -> torch.Tensor. For non-PyTorch models (Keras, TensorFlow, etc.), see the Custom Model Wrapper guide.
• batch_size (int): Batch size for processing. Default: 32. Larger batches are faster but use more GPU memory.
• num_workers (int): Number of loader processes. In Jupyter, automatically reduced to 0 to avoid multiprocessing issues. Default: 4.
• filters (dict or None): Include only rows matching criteria. Format: {column: [value1, value2, ...]}. Default: None (no filtering).
• exclude (dict or None): Exclude rows matching any criterion (OR across fields). Same value format as filters. Default: None.
• channel_mode (str): How to handle multi-channel images. One of:
  • ‘split’: Process each channel separately (default).
  • ‘combined’: Process all channels as RGB or grayscale.
• channels (list of int or None): Specific channel indices to use. If None, uses all available. Default: None.
• preprocessing_fn (callable or None): Custom preprocessing function applied to each image before the model. Signature: fn(image: ndarray) -> ndarray. Example: normalization, contrast adjustment, etc. Default: None.
• custom_transformations (transforms object or None): PyTorch transforms to apply before the model (e.g., resize, normalize). If None, the default order is TypeMaxNorm, optional pad, optional resize, then ImageNet Normalize. Default: None.
• preprocessing_description (str or None): Optional plain-language note for preprocessing_fn. Shown in the INFO processing pipeline log and on get_processing_pipeline(). Internals are never inspected. Recommended when using custom preprocessing so the log describes what the function does. Default: None.
• transformations_description (str or None): Optional plain-language note for custom_transformations. Shown in the INFO processing pipeline log and on get_processing_pipeline(). Internals are never inspected. Default: None.
• append (bool): If True, append to existing embeddings in results.embeddings. If False (default), replace. Default: False.
• resize_size (int or None): Image resize dimension (square). Default: 224. Set to None to skip resizing.
• pad_size (int or None): Padding size (square). Default: None (no padding).
• checkpoint_path (str or None): HDF5 file path for lazy storage and resumable processing. If file exists, processing resumes from last checkpoint. If None, embeddings stored in-memory. Default: None.
• force_rgb (bool): If True, convert grayscale to RGB before model. Default: True.
• l2_normalize_channels (bool): In channel_mode='split', L2-normalize each channel embedding before concatenation so channels contribute equally. Set False to preserve raw channel magnitudes (one channel may dominate). Ignored in combined mode. Default: True.
• save_every (int): Save checkpoint every N batches (when using checkpoint_path). Default: 5.
• lazy_checkpoint (bool): If True and checkpoint_path is set, enable lazy loading (embeddings not kept in-memory). If False, load all embeddings after processing. Default: True.
• force_reprocess (bool): If True and checkpoint_path already exists, delete that file and process all requested images from scratch (no resume). Default: False.
• progress_callback (callable or None): Optional (current, total, desc) -> None hook for GUI/notebook progress bars. Invoked after each batch. When set, the default tqdm bar is disabled. Default: None.

Returns:

None. Modifies self.results in-place with embeddings, metadata, and paths.

Raises:

• FileNotFoundError: If file_df not set (call find_files() or set_file_df() first).
• ValueError: If filters/exclude reference non-existent columns or produce no data.
• RuntimeError: If model inference fails or checkpoint is corrupted.
• IOError: If checkpoint file cannot be created or read.

Example:

Process images with DINOv2 model:

>>> from phenome import PhenoMe, load_dinov2_model
>>> pm = PhenoMe(device="cuda")
>>> pm.find_files("images/")
>>> model = load_dinov2_model("dinov2_vitb14")
>>> pm.process_images(model, batch_size=64, resize_size=224)
>>> print(pm.results.embeddings.shape)
(1000, 768)

Optional description for custom preprocessing (appears in the INFO
processing pipeline log even though the callable itself is opaque):

>>> def enhance_contrast(image):
...     p1, p99 = np.percentile(image, (1, 99))
...     return np.clip(image, p1, p99)
>>> pm.process_images(
...     model,
...     preprocessing_fn=enhance_contrast,
...     preprocessing_description="clip 1st/99th percentiles",
... )

With filtering and checkpoint:

>>> pm.process_images(
...     model,
...     filters={"batch": ["batch_1", "batch_2"]},
...     checkpoint_path="embeddings.h5",
...     lazy_checkpoint=True
... )

With a Jupyter progress bar:

>>> from ipywidgets import IntProgress, Label, VBox
>>> from IPython.display import display
>>> bar, lab = IntProgress(max=1), Label()
>>> display(VBox([lab, bar]))
>>> def on_progress(current, total, desc):
...     bar.max = max(total, 1)
...     bar.value = current
...     lab.value = f"{desc}: {current}/{total}"
>>> pm.process_images(model, progress_callback=on_progress)

See Also:

• find_files: Discover and organize image files.
• process_temporal_images: Incrementally add images to existing checkpoint.
• get_embeddings: Retrieve embeddings (handles lazy loading).
• compute_properties: Extract morphological properties from embeddings.
• get_processing_pipeline: Structured ordered processing pipeline for this run.
• print_processing_pipeline: Print the same text as the INFO log.

Method

process_temporal_images

PhenoMe.process_temporal_images(
    self,
    model_wrapper: phenome.utils.model_wrapper.ModelWrapper,
    files: str | list[str] | pandas.DataFrame,
    batch_size: int = 32,
    num_workers: int = 4,
    channel_mode: Optional[Literal['split', 'combined']] = None,
    channels: list[int] | None = None,
    preprocessing_fn: collections.abc.Callable[[numpy.ndarray], numpy.ndarray] | None = None,
    custom_transformations: typing.Any | None = None,
    preprocessing_description: str | None = None,
    transformations_description: str | None = None,
    resize_size: int | None = None,
    pad_size: int | None = None,
    force_rgb: bool | None = None,
    l2_normalize_channels: bool | None = None,
    extensions: list[str] | None = None
) -> None

Process new images in-memory (temporary) and append to the current session.

Use this to explore additional images (e.g., from a new condition or replicate) without re-running process_images. Temporal images are not persisted to checkpoint; visualize or export before calling clear_temporal_data() or reset().

Requires process_images() to have been run first. Processing parameters (channel_mode, channels, resize_size, etc.) are inherited from the base run when not specified. Temporal rows get metadata[‘source’] = ‘NEW’. Optional preprocessing_description / transformations_description are recorded on the session-only temporal pipeline (same optional notes as process_images; they appear in the INFO processing pipeline log).

Default processing order matches process_images (load, optional preprocess, channels/RGB, TypeMaxNorm, pad/resize, ImageNet, model, split L2). A concise processing pipeline is logged at INFO.

Method

inspect_data

PhenoMe.inspect_data(
    self
) -> DataFrame

Inspect image and mask dimensions, shapes, dtypes, and data ranges.

Delegates to FileDiscovery. Uses stored dtypes (not the float32 cast from read_image). Requires file_df from set_file_df or find_files.

Method

clear_temporal_data

PhenoMe.clear_temporal_data(
    self
) -> int

Remove all temporal images (metadata[‘source’] == ‘NEW’) from the session.

Returns:

• int: Number of temporal rows removed.

Accessors and inspection

Method

get_embeddings

PhenoMe.get_embeddings(
    self,
    indices: int | list[int] | numpy.ndarray | None = None
) -> numpy.ndarray | None

Return embeddings for the given row indices.

When a checkpoint / HDF5 database is active (self._db is not None), only the requested rows are read from disk — the core of the lazy-loading strategy. When temporal embeddings exist (from process_temporal_images), rows with index >= self._temporal_start_idx are read from self._temporal_embeddings; earlier rows come from the checkpoint.

When no database is active, returns results.embeddings (optionally sliced). If that array holds only checkpoint rows (length _temporal_start_idx) and temporal embeddings are still in memory, they are appended to form a full (n_images, D) array. If results.embeddings already has n_images rows (e.g. materialized on checkpoint_context exit), temporal data is not appended again.

Args:

• indices: Optional row index or sequence of row indices (0-based). A single int returns shape (1, D). If None, returns all embeddings.

Returns:

np.ndarray shape (len(indices), D) float32, or None if no embeddings are available.

Method

get_image_info

PhenoMe.get_image_info(
    self,
    idx: int,
    distance_results: pandas.DataFrame | None = None
) -> dict

Return metadata, properties, and optional distance for image idx.

Args:

• idx: Image index (0 to n_images-1).
• distance_results: Optional DataFrame from compute_reference_distances.

Returns:

• dict: Keys image_index, image_name, image_path, metadata keys, property keys, distance (if distance_results provided), is_reference (if applicable).

Method

get_available_metadata_keys

PhenoMe.get_available_metadata_keys(
    self
) -> list

Return sorted metadata keys.

Method

get_available_property_keys

PhenoMeProperties.get_available_property_keys(
    self
) -> list

Return sorted list of property keys stored in results.

Property

has_embeddings

Property on PhenoMe

Return True if embedding data is available (lazy or eager).

A properties-only checkpoint has n_committed > 0 but no embedding dataset; we additionally require embedding_dim to be a positive int.

Property

embedding_dim

Property on PhenoMe

Return the embedding dimensionality, or 0 if unavailable.

Method

get_processing_pipeline

PhenoMe.get_processing_pipeline(
    self
) -> dict[str, Any] | None

Return the session processing pipeline, or None if none was recorded.

The dict has version, base, and optional temporal and properties. After load_results, embedding and property steps come from /processing_pipeline; temporal exists only in the current session.

See Also:

• print_processing_pipeline: Print the concise pipeline text.

Method

print_processing_pipeline

PhenoMe.print_processing_pipeline(
    self
) -> None

Print the ordered processing pipeline for this session.

Uses the same text as the INFO log during process_images and compute_properties. After load_results, prints the steps stored in /processing_pipeline (embeddings and properties).

See Also:

• get_processing_pipeline: Structured dict form of the same record.

Data lifecycle and export

Method

reset

PhenoMe.reset(
    self,
    verbose: bool = False,
    clear_file_df: bool = False
) -> None

Reset all stored data to a clean state.

Closes any open HDF5 database handle before clearing results. By default preserves _file_df so compute_properties and inspect_data continue to work after process_images. Set clear_file_df=True for a full reset (e.g. when switching to a completely new dataset).

Method

reset_properties

PhenoMeProperties.reset_properties(
    self,
    verbose: bool = False
) -> None

Reset only the computed properties.

Args:

• verbose: If True, logs a confirmation message.

Method

transfer_metadata_to_properties

PhenoMeProperties.transfer_metadata_to_properties(
    self,
    columns: str | list[str]
) -> None

Transfer specified metadata columns to properties.

Args:

• columns: A single metadata column name or a list of names to transfer.

Method

save_results

PhenoMe.save_results(
    self,
    path: str | None = None,
    compression: str = 'gzip'
) -> None

Save results to HDF5 (atomic write or in-place flush).

When a checkpoint database is already active (self._db is set), the data is already on disk; this method flushes any uncommitted buffers and logs the existing path. When no database is active, writes self.results to a new HDF5 file using a temporary-file + atomic rename for crash safety.

For full reproducibility, also call export_experiment_config to save seed, use_gpu_for_dr, reference_filters, and model name (not stored in the checkpoint).

Args:

• path: Explicit output path.
• compression: HDF5 compression algorithm (used only for new files).

Method

load_results

PhenoMe.load_results(
    self,
    path: str | None = None,
    lazy_checkpoint: bool = True
) -> None

Load and use an existing results/checkpoint file.

Metadata and properties are loaded immediately. Embeddings are either loaded into RAM (lazy_checkpoint=False) or accessed on-demand from disk (lazy_checkpoint=True).

Uses the internally stored file_df (from find_files or set_file_df) to resolve paths so the checkpoint works on this machine. Call find_files or set_file_df first.

For separate embeddings and properties checkpoints, prefer load_embeddings then load_properties.

Args:

• path: Path to .h5 or .hdf5 file.
• lazy_checkpoint: If True (default), keep the checkpoint file open. If False, load all data into RAM and close the file.

Raises:

• FileNotFoundError: If file does not exist.
• ValueError: If file format is not recognised as HDF5.
• RuntimeError: If file_df is not available (call find_files or set_file_df first).
• ConcurrentCheckpointAccessError: If the checkpoint is already open in another notebook or process (only one instance can access it at a time).

Method

checkpoint_context

PhenoMe.checkpoint_context(
    self,
    path: str | None = None,
    lazy_checkpoint: bool = True
) -> Generator

Context manager that loads a checkpoint and guarantees it is closed on exit.

More reliable than relying on del for cleanup. Use when you need to ensure the HDF5 file handle is released (e.g. before moving or deleting the file, or when opening multiple checkpoints in sequence).

Embeddings: While the context is open, lazy embeddings are read from the HDF5 file via get_embeddings(). On exit, if temporal embeddings were loaded in memory (process_temporal_images while the checkpoint was open), the merged full embedding matrix is copied to results.embeddings and self._temporal_embeddings is cleared. Otherwise results.embeddings may stay None; call load_results(..., lazy_checkpoint=True) or enter another checkpoint_context before using get_embeddings() again.

Args:

• path: Checkpoint HDF5 path. If None, uses the path from the last load_results / process_images session when available.
• lazy_checkpoint: If True, keep the HDF5 file open for on-demand reads inside the context. Default: True.

Example:

pheno.find_files("path/to/images")
with pheno.checkpoint_context("results.h5") as p:
    p.plot_pca(color_by="condition")
# Checkpoint closed here

Method

export_experiment_config

PhenoMe.export_experiment_config(
    self,
    path: str,
    reference_filters: dict[str, typing.Any] | None = None,
    model_name: str | None = None,
    checkpoint_path: str | None = None,
    **extra: Any
) -> None

Export experiment configuration for reproducibility.

Writes a JSON file with init parameters, environment, processing params, and silently recorded method steps. Call after the analysis you want to document. Optional model_name / reference_filters override values already captured from process_images / compute_reference_distances.

Args:

• path: Output path for config.json.
• reference_filters: Optional dict used for compute_reference_distances (e.g. {‘condition’: ‘Control’}). Include for full traceability.
• model_name: Optional model identifier (e.g. ‘dinov2_vitb14_reg’).
• checkpoint_path: Optional checkpoint path used during processing (if not provided and a checkpoint is open, uses self._db.path). **extra: Additional key-value pairs to include in the config.

Method

export_methods_markdown

PhenoMe.export_methods_markdown(
    self,
    path: str
) -> None

Write a markdown methods summary from the silent run log.

Opt-in companion to export_experiment_config for paper supplements. Does nothing unless this method is called.

Method

export_dataset_table

PhenoMe.export_dataset_table(
    self,
    output_path: str | None = None,
    dist_results: pandas.DataFrame | None = None,
    include_embeddings: Union[bool, Literal['separate']] = False,
    export_format: Literal['csv', 'parquet', 'excel'] = 'csv'
) -> DataFrame

Export the dataset as a table (CSV, Parquet, or Excel).

Args:

• output_path: Path to save the export. If None, only returns the DataFrame.
• dist_results: Optional DataFrame from compute_reference_distances.
• include_embeddings: If True, adds embedding columns; if ‘separate’, saves embeddings to a companion .npy file.
• export_format: Output format: ‘csv’, ‘parquet’, or ‘excel’.

Returns:

DataFrame with image_index, image_path, metadata, properties, distances, and embeddings (if requested).

Properties

Method

compute_properties

PhenoMeProperties.compute_properties(
    self,
    metadata_config: typing.Any | None = None,
    property_preset: str | None = None,
    additional_property_functions: dict[str, collections.abc.Callable | list[collections.abc.Callable]] | None = None,
    checkpoint_path: str | None = None,
    save_every: int = 50,
    n_jobs: int = 1,
    lazy_checkpoint: bool = True,
    force_update: bool = False,
    include_metadata: bool = False,
    progress_callback: collections.abc.Callable[[int, int, str], None] | None = None
) -> DataFrame

Compute per-image properties using presets and/or custom functions.

Each function receives a 2D image slice and/or mask slice and returns a dict[str, float]. When multiple channels exist, properties are suffixed with _ch{idx}, or with labels from find_files / set_file_df channel_names when provided (e.g. _DAPI).

Use property_preset for built-in property sets and additional_property_functions to add custom functions on top (optional).

This method now relies solely on file_df (typically produced by PhenoMe.find_files) for resolving image and mask paths. Legacy image_dir / mask_dir parameters are no longer supported.

Images used for properties are loaded, laid out as HxWxC, then scaled with normalize_by_dtype_max (same 0-1 rule as TypeMaxNorm). Masks are loaded and laid out the same way, then binarized (foreground = pixels > 0.5; no intensity scaling). A concise processing pipeline of those applied steps is logged at INFO.

Args:

• metadata_config: Optional MetadataBase instance (e.g. from phenome.metadata). When it has mask_dir and mask_filename_column, uses explicit mask lookup.
• property_preset: Preset name. Use "none" or None for no preset. Valid: "none", "basic", "shape", "intensity", "standard", "complete".
• additional_property_functions: Extra property functions to add on top of preset. Dict of {requirement: fn_or_list} where requirement is "image", "mask", "both", or "any". Merged with preset when both are used; used alone when property_preset is "none" or None.
• checkpoint_path: HDF5 checkpoint for incremental persistence.
• save_every: Commit frequency (images) when checkpointing.
• n_jobs: Number of parallel jobs for property computation. 1 (default) = sequential. Use -1 to use all available CPU cores.
• lazy_checkpoint: If True (default), keep the checkpoint file open. If False, load all data into RAM and close the file.
• force_update: If True and checkpoint_path points to an existing file, clears stored properties in that file, writes current _processing_params into the checkpoint config, and recomputes properties for every image (no resume). Use this after changing property_preset or custom functions, or after log warnings about missing checkpoint property keys. If checkpoint_path is set but the file does not exist yet, only an info log is emitted; computation proceeds as for a new checkpoint.
• include_metadata: If True, add metadata columns to the returned DataFrame.
• progress_callback: Optional (current, total, desc) -> None hook for GUI/notebook progress bars. Invoked after each image (sequential) or after each parallel batch. When set, the default tqdm bar is disabled. Default: None.

Returns:

DataFrame with image_index, image_path, image_name, property values, and optional metadata columns. Also populates results.properties as List[dict] (one dict per image).

Raises:

• ValueError: If no images processed or invalid preset/requirement keys are used.
• TypeError: If additional_property_functions values are not callable.

Example:

Drive a napari/Qt progress bar from a worker thread:

>>> def on_progress(current, total, desc):
...     # update Qt/napari progress on the main thread
...     progress.setValue(int(100 * current / total) if total else 0)
...     progress.setLabelText(desc)
>>> pm.compute_properties(
...     property_preset="basic",
...     progress_callback=on_progress,
... )

Method

property_stats_by_group

PhenoMeProperties.property_stats_by_group(
    self,
    group_by: list[str] | None = None,
    properties: list[str] | None = None,
    print_table: bool = True,
    print_properties: list[str] | None = None,
    group_column_width_max: int = 25,
    content_col_width_max: int = 25
) -> DataFrame

Group properties, compute per-group statistics, and optionally print a table.

Uses _build_properties_dataframe from computed results.properties and metadata (same source as compute_properties).

Args:

• group_by: Metadata columns to group by. If None, auto-selects first 2.
• properties: Property columns to include in aggregation. If None, auto-detects numeric. Matching is case-insensitive; names must be unique ignoring case.
• print_table: If True, log a formatted mean±std table.
• print_properties: Subset of properties to show in the table. If None, shows all. Matching is case-insensitive; names must be unique ignoring case.
• group_column_width_max: Maximum width for each grouping column when printing.
• content_col_width_max: Maximum width for each property statistic column when printing.

Returns:

Aggregated DataFrame with mean/std/min/max per property per group.

Raises:

• TypeError: If group_by/properties are invalid types.
• ValueError: If group_by keys are not in DataFrame columns.

Method

top_properties_different_from_reference

PhenoMeProperties.top_properties_different_from_reference(
    self,
    df: pandas.DataFrame,
    reference_group: dict[str, typing.Any],
    k: int = 5,
    properties: list[str] | None = None,
    metric: Literal['cohens_d', 'mean_diff'] = 'cohens_d',
    print_output: bool = True,
    group_column_width_max: int = 25,
    top_property_col_width_max: int = 25,
    top_effect_col_width_min: int = 12
) -> DataFrame

For each non-reference group, return the top k properties that most differentiate it from the reference.

Uses Cohen’s d (effect size) by default to rank properties by how different each group is from the reference. Both higher and lower values count as “different” (uses |d|).

Cohen’s d formula (pooled): d = (mean_group - mean_ref) / s_pooled, where s_pooled = sqrt([(n_ref-1)*std_ref² + (n_other-1)*std_other²] / (n_ref + n_other - 2)). Requires sample std (ddof=1) from property_stats_by_group.

Args:

• df: Aggregated DataFrame from property_stats_by_group().
• reference_group: Dict mapping grouping column names to values (e.g. {“drug”: “Control”, “time”: “60_min”}).
• k: Number of top properties per group.
• properties: Property names to consider. If None, uses all in DataFrame. Matching is case-insensitive; names must be unique ignoring case.
• metric: ‘cohens_d’ (effect size) or ‘mean_diff’ (absolute mean difference).
• print_output: If True, pretty-print the results.
• group_column_width_max: Maximum width for each grouping column in the printed table.
• top_property_col_width_max: Max width for the property name column in the printed table.
• top_effect_col_width_min: Min width for Cohen’s d and mean diff columns when printing.

Returns:

DataFrame with columns: grouping cols, property, effect_size (Cohen’s d when metric='cohens_d'), mean_diff, ref_mean, group_mean, rank. One row per (group, property) for top-k only.

Advanced analysis

Method

correct_batches

PhenoMeBatchCorrection.correct_batches(
    self,
    batch_metadata_key: 'str',
    method: 'MethodName' = 'sphering',
    source: 'SourceName' = 'embeddings',
    property_keys: 'list[str] | None' = None,
    control_filters: 'dict[str, Any] | None' = None,
    inplace: 'bool' = True,
    checkpoint_path: 'str | None' = None,
    chunk_size: 'int' = 4096,
    force: 'bool' = False,
    ridge_multiplier: 'float' = 0.001,
    min_controls: 'int' = 2
) -> dict[str, Any]

Correct plate-to-plate variation using control wells per batch.

Call after :meth:process_images (and optionally after compute_properties when source='properties'). For lazy checkpoints, embeddings are read and rewritten in chunks of chunk_size rows.

Args:

• batch_metadata_key: Metadata column for batch / plate id (case-insensitive).
• method: "sphering" (default) or "zscore".
• source: "embeddings" or "properties".
• property_keys: Property names to correct when source='properties'. If None, uses all keys from :meth:get_available_property_keys.
• control_filters: Same semantics as filter_indices filters (required).
• inplace: If True, updates results.embeddings or results.properties in place.
• checkpoint_path: If set, persist batch-correction metadata and/or export results.
  • If None (default): Corrections still apply (RAM or an already-open lazy checkpoint), but nothing new is written for persistence—no /batch_correction stats on disk and no new HDF5 via :meth:save_results.
  • If str: With an open checkpoint, it is cloned to this path first; stats are saved under /batch_correction when a DB is active; in eager mode (no DB), results are saved to this path after correction.
• chunk_size: Rows per chunk for embedding I/O and property scatter.
• force: If True, allow re-applying correction after a previous run in this session.
• ridge_multiplier: Covariance ridge strength for sphering (see :func:compute_batch_stats).
• min_controls: Minimum control wells required per batch.

Returns:

Dict with batch stats summary (batches, controls_per_batch, method, …).

Method

compute_clustering

PhenoMeAnalysis.compute_clustering(
    self,
    source: Literal['embeddings', 'properties', 'combined'] = 'embeddings',
    n_clusters: int = 5,
    clustering_method: Literal['kmeans', 'dbscan', 'gmm'] = 'kmeans',
    property_keys: list[str] | None = None,
    filters: dict[str, typing.Any | list[typing.Any]] | None = None,
    exclude: dict[str, typing.Any | list[typing.Any]] | None = None,
    random_state: int | None = None,
    normalize: bool = True,
    reduce_dim: int | None = 100,
    reduce_method: Literal['pca', 'tsne', 'umap'] = 'pca',
    return_silhouette: bool = False,
    dbscan_eps: float | None = None,
    dbscan_min_samples: int | None = None,
    include_metadata: bool = False
) -> pandas.DataFrame | tuple[pandas.DataFrame, float | None]

Perform clustering and store labels in metadata[i]['cluster'] for each image.

Args:

• source: 'embeddings', 'properties', or 'combined'.
• n_clusters: Number of clusters (k-means, GMM). Ignored for clustering_method='dbscan'.
• clustering_method: 'kmeans' (default), 'dbscan', or 'gmm'.
• property_keys: Property subset when source=‘properties’.
• filters: Optional metadata filters.
• exclude: Optional metadata exclusions (same structure as filters).
• random_state: Seed for k-means/GMM. If None, uses pipeline seed when available.
• normalize: Whether to normalize data before clustering (default: True). For embeddings, uses L2 normalization. For properties, uses StandardScaler.
• reduce_dim: If set, apply dimensionality reduction before clustering (default: 100). Only applied when data has more features than reduce_dim. Set to None to disable dimensionality reduction.
• reduce_method: Method for dimensionality reduction when reduce_dim is set: 'pca' (fast, linear), 'tsne' (non-linear, slower), 'umap' (non-linear, preserves structure). Default: 'pca'.
• return_silhouette: If True, compute and return the silhouette score (default: False). Returns (df, score); score is None if it could not be computed. Silhouette requires at least 2 clusters and 2 samples per cluster.
• dbscan_eps: Maximum distance between two samples for DBSCAN (only when clustering_method=‘dbscan’). If None, uses 0.5.
• dbscan_min_samples: Minimum samples in a neighborhood for DBSCAN (only when clustering_method=‘dbscan’). If None, uses max(2, n_samples // 20).
• include_metadata: If True, add metadata columns to the returned DataFrame.

Returns:

If return_silhouette=False: Tier-A DataFrame with image_index, image_path, image_name, cluster, and optional metadata. If return_silhouette=True: Tuple of (df, silhouette_score). Score is None if it could not be computed. Cluster labels are 0..K-1; None for excluded/noise.

Method

detect_outliers

PhenoMeAnalysis.detect_outliers(
    self,
    method: Literal['z-score', 'iqr'] = 'z-score',
    threshold: float = 3.0,
    group_by: str | list[str] | None = None,
    filters: dict[str, typing.Any] | None = None,
    exclude: dict[str, typing.Any] | None = None,
    source: Literal['embeddings', 'properties', 'combined'] = 'embeddings',
    property_keys: list[str] | None = None,
    normalize: bool = True,
    drop_outliers: bool = False,
    plot: bool = True
) -> DataFrame

Detect outliers based on distance to centroid.

When source is 'embeddings' and normalize is True (default), embeddings are L2-normalized and group centroids use the arithmetic mean (not necessarily a unit vector). Euclidean distance to that centroid is not the same as angular distance to the mean direction; consider property-based or combined sources when a different geometry is needed.

Args:

• method: 'z-score' or 'iqr'.
• threshold: Threshold multiplier.
• group_by: Property/metadata key(s) for grouping. Can be a single string or a list of strings. If a list is provided, groups are formed by combining values from all specified columns (e.g., “drug1-10uM”). If None, all images are treated as one group.
• filters: Optional metadata filters.
• exclude: Optional metadata exclusions (same structure as filters).
• source: Feature space for outlier detection:
  • 'embeddings': model embeddings
  • 'properties': scalar phenotypic properties
  • 'combined': normalized concatenation of embeddings and properties.
• property_keys: Property subset when source is 'properties' or 'combined'.
• normalize: Whether to normalize data before outlier detection (default: True). For embeddings, uses L2 normalization. For properties, uses StandardScaler.
• drop_outliers: If True, remove detected outliers from self.results (default: False).
• plot: If True (default), shows one matplotlib figure per group: subplots for detected outliers and a figure title naming the group (Group: …). Images are shown without pipeline image_transforms (same as apply_transforms=False for display).

Returns:

pd.DataFrame: Tier-A DataFrame containing detected outliers. Columns include:

• image_index: Global image index.
• image_path: Path to the image file.
• image_name: Basename of the image file.
• Additional columns for each grouping key (if group_by was set).
• threshold: The threshold value used for the image’s group.
• distance_to_centroid: Distance to the group centroid. If no outliers are detected, returns an empty DataFrame. The index is a range index from 0 to N-1.

Method

find_prototypes

PhenoMeAnalysis.find_prototypes(
    self,
    cluster_col: str | list[str] | None = 'cluster',
    n_prototypes: int = 5,
    source: Literal['embeddings', 'properties', 'combined'] = 'embeddings',
    property_keys: list[str] | None = None,
    filters: dict[str, typing.Any] | None = None,
    exclude: dict[str, typing.Any] | None = None,
    metric: str = 'euclidean',
    normalize: bool = True,
    plot: bool = True
) -> DataFrame

Find images closest to each group centroid.

Args:

• cluster_col: Property/metadata key(s) for grouping. Can be a single string or a list of strings. If a list is provided, groups are formed by combining values from all specified columns (e.g., “drug1-10uM”). If None, all images are treated as one group.
• n_prototypes: How many prototypes per group.
• source: 'embeddings', 'properties', or 'combined'.
• property_keys: Property subset when source=‘properties’.
• filters: Optional metadata filters.
• exclude: Optional metadata exclusions (same structure as filters).
• metric: 'euclidean' or 'cosine'. With metric='euclidean', the group centroid is the arithmetic mean (even when embeddings are L2-normalized). With metric='cosine', the centroid direction is L2-normalized before ranking by similarity.
• normalize: Whether to normalize data before finding prototypes (default: True). For embeddings, uses L2 normalization. For properties, uses StandardScaler.
• plot: If True (default), shows one matplotlib figure per group: subplots for that group’s prototypes and a figure title naming the group (Group: …). Images are shown without pipeline image_transforms (same as apply_transforms=False for display).

Returns:

pd.DataFrame: DataFrame with one row per prototype. Columns include grouping keys (split into individual columns if multiple), image_index, image_path, image_name, and rank_in_group.

Method

analyze_group_enrichment

PhenoMeAnalysis.analyze_group_enrichment(
    self,
    group_by: str | list[str] = 'cluster',
    property_keys: list[str] | None = None,
    filters: dict[str, typing.Any] | None = None,
    exclude: dict[str, typing.Any] | None = None,
    plot: bool = True,
    return_fig: bool = False,
    top_k: int | None = None,
    figsize: tuple[int, int] = (10, 6),
    title: str | None = None,
    correct_multiple_testing: bool = True
) -> pandas.DataFrame | tuple[pandas.DataFrame, Any]

Compute z-score enrichment of properties per group (e.g. cluster labels).

For each group, z-scores are computed against a leave-group-out population (all samples except the current group). When correct_multiple_testing is True (default), Welch two-sample t-tests (group vs leave-one-out rest) are run per property and Benjamini-Hochberg FDR correction is applied across all (group, property) pairs; a significant column is added to the output DataFrame.

Args:

• group_by: Property/metadata key(s) holding group labels (e.g. "cluster"). Can be a single string or a list of strings for composite grouping.
• property_keys: Properties to analyse (all numeric if None).
• filters: Optional metadata filters.
• exclude: Optional metadata exclusions (same structure as filters).
• plot: If True (default), show an interactive Plotly faceted bar chart (one row per group). If False, log a plain-text summary via the package logger instead (unless return_fig requests a figure).
• return_fig: If True, return a tuple (df, fig) where fig is the Plotly figure. When return_fig is True, fig.show() is not called; use plot=True with return_fig=False for the default interactive display.
• top_k: Max properties per group in the figure and in the text summary (None = all).
• figsize: Figure size (width, height) in inches for the Plotly layout.
• title: Optional figure title.
• correct_multiple_testing (bool): If True (default), run Welch t-tests per (group, property), apply Benjamini-Hochberg FDR, and add p_value / significant columns (alpha = 0.05).

Returns:

pd.DataFrame or tuple[pd.DataFrame, Any]:

• If return_fig is False (default): pd.DataFrame with columns: [group_by columns], property, score, mean_group, mean_pop, p_value, abs_score, and optionally significant when correct_multiple_testing is True.
• If return_fig is True: A tuple (enrichment_df, fig).

Method

compute_component_correlation

PhenoMeAnalysis.compute_component_correlation(
    self,
    method: Literal['pca', 'tsne', 'umap'] = 'pca',
    n_components: int = 2,
    source: Literal['embeddings', 'properties', 'combined'] = 'embeddings',
    property_keys: list[str] | None = None,
    filters: dict[str, typing.Any] | None = None,
    exclude: dict[str, typing.Any] | None = None,
    top_k: int | None = None,
    normalize: bool = True,
    correlation_method: Literal['pearson', 'spearman', 'distance_correlation', 'mutual_info'] = 'pearson',
    plot: bool = True,
    return_fig: bool = False,
    figsize: tuple[int, int] = (10, 6)
) -> pandas.DataFrame | tuple[pandas.DataFrame, Any]

Correlate dim-reduction components with phenotypic properties.

Args:

• method: Dimensionality reduction method.
• n_components: Number of components to compute.
• source: 'embeddings', 'properties', or 'combined'.
• property_keys: Property subset when source is 'properties' or 'combined'.
• filters: Optional metadata filters.
• exclude: Optional metadata exclusions (same structure as filters).
• top_k: Number of top correlations to return in summary and per facet in the plot.
• normalize: Whether to normalize data before computing correlations (default: True).
  • Source data (embeddings/properties): Normalized before dimensionality reduction (L2 normalization for embeddings, StandardScaler for properties)
  • Properties: Normalized using StandardScaler before correlation computation
• correlation_method: Correlation method to use. Options:
  • 'pearson': Pearson correlation coefficient (default, linear relationships)
  • 'spearman': Spearman rank correlation (monotonic relationships, robust to outliers)
  • 'distance_correlation': Distance correlation (detects non-linear relationships, requires dcor library: pip install dcor)
  • 'mutual_info': MI-derived correlation coefficient (detects any dependency, mapped to [0, 1] via the Gaussian bivariate transform, not standard NMI)
• plot: If True (default), show an interactive Plotly faceted bar chart (one row per component). If False, log a plain-text summary via the package logger instead (unless return_fig requests a figure).
• return_fig: If True, return a tuple of (correlation_df, figure). When return_fig is True, fig.show() is not called.
• figsize: Figure size (width, height) in inches for the Plotly layout.

Returns:

pd.DataFrame | tuple[pd.DataFrame, Any]:

• If return_fig is False (default): Correlation DataFrame with a property column and component_1, component_2, … columns.
• If return_fig is True: Returns a tuple of (correlation_df, figure).

Raises:

• ValueError: If dimensionality reduction fails, no components are found, no numeric properties are available, or correlation computation fails.

Examples:

Get correlations directly::

    corr = pheno.compute_component_correlation(plot=False)

Get correlations and save the figure::

    corr, fig = pheno.compute_component_correlation(return_fig=True)
    fig.write_html("component_corr.html")

Method

compute_embedding_property_correlations

PhenoMeAnalysis.compute_embedding_property_correlations(
    self,
    property_keys: list[str] | None = None,
    normalize: bool = True,
    method: Literal['pearson', 'spearman', 'distance_correlation', 'mutual_info'] = 'pearson',
    n_jobs: int = 1
) -> dict

Compute correlations between embedding dimensions and phenotypic scalar properties.

This is the time-consuming step that computes raw correlations for each property.

Args:

• property_keys: Property names (all if None).
• normalize: Whether to normalize data before computing correlations (default: True).
  • Embeddings: L2 normalization (unit norm)
  • Properties: StandardScaler normalization (zero mean, unit variance)
• method: Correlation method to use. Options:
  • 'pearson': Pearson correlation coefficient (default, linear relationships)
  • 'spearman': Spearman rank correlation (monotonic relationships, robust to outliers)
  • 'distance_correlation': Distance correlation (detects non-linear relationships, requires dcor library: pip install dcor)
  • 'mutual_info': MI-derived correlation coefficient (detects any dependency, mapped to [0, 1] via the Gaussian bivariate transform, not standard NMI)

Returns:

Dict mapping property names to correlation arrays (n_dims,)

Method

summarize_embedding_property_correlations

PhenoMeAnalysis.summarize_embedding_property_correlations(
    self,
    correlations: dict[str, numpy.ndarray],
    order_by: str = 'mean_abs',
    top_k: int | None = 20,
    plot: bool = True,
    return_fig: bool = False,
    figsize: tuple[int, int] = (10, 8),
    title: str = 'Property Correlations with Embeddings',
    embedding_shape: tuple[int, int] | None = None,
    n_properties: int | None = None,
    correlation_method: str = 'pearson'
) -> pandas.DataFrame | tuple[pandas.DataFrame, Any | None]

Summarize embedding-property correlations, optionally plot, and/or return a Plotly figure.

Computes per-property mean_abs, std (across dimensions), max_abs, min_abs, and mean, orders rows by order_by, and either shows a horizontal violin plot of the distribution of |r| across dimensions per property or logs a plain-text table when plot is False.

Args:

• correlations: Output from :meth:compute_embedding_property_correlations (dict mapping property names to correlation arrays).
• order_by: Metric used to sort properties (descending): mean_abs | max_abs | mean | std.
• top_k: Number of top properties shown in the plot and listed in top_properties; full sorted table is always in summary (None = all).
• plot: If True (default), show a Plotly violin plot of |r| per dimension for the top top_k properties by order_by (None = all). Highest metric at the top of the y-axis.
• return_fig: If True, return both the summary DataFrame and the Plotly figure.
• figsize: Figure size in inches, converted to pixels for Plotly layout.
• title: Chart title.
• embedding_shape: Shape of embeddings (n_samples, n_dims) for metadata display.
• n_properties: Number of properties processed for metadata display.
• correlation_method: Method used for correlation computation for metadata display.

Returns:

Sorted DataFrame with property, mean_abs, std, max_abs, min_abs, and mean. If return_fig is True, returns (summary, figure).

Method

compute_multivariate_interpretability

PhenoMeAnalysis.compute_multivariate_interpretability(
    self,
    method: Literal['pca', 'tsne', 'umap'] = 'tsne',
    component: int = 1,
    model_type: Literal['lasso', 'random_forest'] = 'lasso',
    property_keys: list[str] | None = None,
    filters: dict[str, typing.Any] | None = None,
    exclude: dict[str, typing.Any] | None = None,
    normalize: bool = True,
    cv: int = 5,
    stratify_by: str | None = None,
    rf_n_estimators: int = 100,
    seed: int | None = None,
    plot: bool = True,
    return_fig: bool = False,
    return_meta: bool = False,
    top_k: int = 10,
    figsize: tuple[int, int] = (10, 8)
) -> pandas.DataFrame | tuple[pandas.DataFrame, dict[str, Any]] | tuple[pandas.DataFrame, Any] | tuple[pandas.DataFrame, dict[str, Any], Any]

Explain a dimensionality reduction component using LASSO or Random Forest.

Calculates which phenotypic properties (features) best explain the variability seen in a deep learning embedding dimension (the target, usually t-SNE 1 or 2).

Workflow: dimensionality reduction is run on the full filtered embedding set first; regression then relates classical properties to the resulting axis coordinates. For transductive methods (t-SNE, UMAP), axis coordinates depend on all samples, so the returned r2 (LASSO) describes descriptive on-axis alignment on the same dataset—not independent generalization to new plates or refit embeddings. Prefer drivers for hypothesis generation; interpret r2 as an effect-size-style summary unless DR and evaluation are nested with proper holdouts.

Args:

• method: Dimensionality reduction method (‘pca’, ‘tsne’, or ‘umap’).
• component: Which component to explain (1, 2, …).
• model_type: The regression model to use (‘lasso’ or ‘random_forest’). ‘lasso’ uses L1 regularization for linear, sparse explanations. ‘random_forest’ captures non-linear relationships.
• property_keys: Subset of properties to use as features.
• filters: Optional metadata filters.
• exclude: Optional metadata exclusions.
• normalize: Whether to normalize embeddings before dimensionality reduction (default: True). Property features are scaled fold-wise inside the LASSO pipeline (not globally) to avoid cross-validation leakage.
• cv: Number of cross-validation folds (only for ‘lasso’).
• stratify_by: Optional metadata key (e.g. "Treatment", "Batch") to stratify LASSO folds by class balance. Requires at least cv samples per class; otherwise falls back to unstratified KFold. Balances folds for lambda selection but does not correct global t-SNE/UMAP target leakage.
• rf_n_estimators: Number of trees (only for ‘random_forest’).
• seed: Random seed for reproducibility. If None, uses the pipeline’s seed when set.
• plot: If True (default), show an interactive Plotly bar chart of top drivers. If False, log a plain-text summary via the package logger instead.
• return_fig: If True, include the Plotly figure in the return value (see Returns). When return_fig is True, fig.show() is not called; use plot=True with return_fig=False for interactive display.
• return_meta: If True, include a run-parameters dict in the return value.
• top_k: Number of top drivers to show in the plot or text summary.
• figsize: Figure size (width, height) passed through to Plotly layout; each value is multiplied by 100 to set width and height in layout pixels.

Returns:

pd.DataFrame or tuple, depending on return_meta and return_fig:

• Default: drivers_df with columns feature, weight.
• return_meta=True: (drivers_df, meta); meta includes r2 (descriptive on-axis fit for globally fit DR targets), n_samples, n_features, method, model_type, target_component.
• return_fig=True: (drivers_df, fig) or (drivers_df, meta, fig) when return_meta is also True.

Reporting

Method

generate_report

PhenoMe.generate_report(
    self,
    output_path: str = 'pheno_report.html',
    title: str = 'PhenoMe Analysis Report',
    config: typing.Any | None = None,
    **overrides: Any
) -> str

Generate a comprehensive standalone HTML report from phenotyping results.

Uses ReportConfig as the primary source of options. Pass config= for full control, or use **overrides to tweak individual settings (e.g. include_plots=False).

Args:

• output_path: Path to save the HTML file.
• title: Report title displayed at the top.
• config: Optional ReportConfig for defaults. If None, uses ReportConfig(). **overrides: Any ReportConfig field to override (e.g. include_plots=False, outlier_threshold=2.5, n_clusters=10).

Returns:

Path to the generated HTML file.

Other

Method

get_run_settings

PhenoMe.get_run_settings(
    self
) -> dict

Return the silent run log as a JSON-serializable dict.

Includes init parameters, environment, recorded method steps, processing_params, and the processing pipeline when available. Does not print or write files.

Method

image_preview_png_bytes

_ImageDisplayMixin.image_preview_png_bytes(
    self,
    idx: int,
    distance_results: pandas.DataFrame | None = None,
    channels: int | list | None = None,
    title_fields: list[str] | None = None,
    show_extra_info: bool = False,
    apply_transforms: bool = True,
    downsample: int | None = 720
) -> tuple

Rasterize the same view as plot_image_by_index to PNG bytes.

Used by the interactive explorer with ipywidgets.Image because matplotlib display() / plt.show() from Plotly click callbacks does not reliably target a nested Output widget (output goes to the cell or nowhere).

When show_extra_info is True, also emits the same details as plot_image_by_index via logging and returns their text for UI display.

Returns:

(png_bytes, details_text_or_none) — details_text_or_none is the formatted extra-info block when show_extra_info is True and details exist.

Method

load_embeddings

PhenoMe.load_embeddings(
    self,
    path: str | None = None,
    lazy_checkpoint: bool = True
) -> None

Load an embeddings checkpoint into the pipeline.

Same path/file_df rebase flow as load_results, but requires the file to contain embeddings. Sets self._db for lazy get_embeddings() access when lazy_checkpoint=True.

Typical two-file workflow::

pheno.find_files(data_dir)
pheno.load_embeddings("run_embeddings.h5")
pheno.load_properties("run_properties.h5")

Args:

• path: Path to .h5 or .hdf5 embeddings checkpoint.
• lazy_checkpoint: If True (default), keep the checkpoint file open. If False, load all data into RAM and close the file.

Raises:

• FileNotFoundError: If file does not exist.
• ValueError: If file format is not recognised as HDF5, or the checkpoint has no embeddings.
• RuntimeError: If file_df is not available (call find_files or set_file_df first).
• ConcurrentCheckpointAccessError: If the checkpoint is already open in another notebook or process (only one instance can access it at a time).

Method

load_properties

PhenoMe.load_properties(
    self,
    path: str | None = None,
    lazy_checkpoint: bool = True
) -> None

Load a properties checkpoint into the pipeline.

When results already contain images (e.g. after load_embeddings or process_images), properties are aligned by metadata key then path and overlaid onto the existing rows without replacing an embeddings self._db from a different file.

When no results are loaded yet, performs a full checkpoint setup like load_results. For a properties-only file, the HDF5 handle is closed after load (properties are eager); self._db is kept only if that file also has embeddings.

Args:

• path: Path to .h5 or .hdf5 properties checkpoint.
• lazy_checkpoint: Used on cold start only. If True (default), keep the file open when it also stores embeddings. Ignored when overlaying onto existing results.

Raises:

• FileNotFoundError: If file does not exist.
• ValueError: If file format is not recognised as HDF5, or the checkpoint has no property content, or overlay matches no rows.
• RuntimeError: On cold start, if file_df is not available (call find_files or set_file_df first).
• ConcurrentCheckpointAccessError: If the checkpoint is already open in another notebook or process (only one instance can access it at a time).

Method

plot_centroids

_DRPlotsMixin.plot_centroids(
    self,
    group_by: str | list[str],
    method: Literal['pca', 'tsne', 'umap'] = 'pca',
    n_components: int = 2,
    filters: dict[str, typing.Any | list[typing.Any]] | None = None,
    exclude: dict[str, typing.Any | list[typing.Any]] | None = None,
    source: Literal['embeddings', 'properties', 'combined'] = 'embeddings',
    property_keys: list[str] | None = None,
    show_points: bool = True,
    show_centroids: bool = True,
    figsize: tuple[int, int] = (10, 8),
    return_fig: bool = False,
    normalize: bool = True,
    **dr_kwargs: Any
) -> Any

Plot centroids of groups in reduced embedding space.

Method

plot_image_by_index

_ImageDisplayMixin.plot_image_by_index(
    self,
    idx: int,
    distance_results: pandas.DataFrame | None = None,
    channels: int | list | None = None,
    figsize: tuple = (6, 6),
    title_fields: list[str] | None = None,
    show_extra_info: bool = True,
    apply_transforms: bool = False,
    downsample: int | None = 720,
    ax: Any = None,
    return_fig: bool = False,
    show_mask_overlay: bool = False
) -> Any

Plot a specific image by its index.

Args:

• idx: Image index in results
• distance_results: Optional distance computation results DataFrame
• channels: specific channels to plot
• figsize: Figure size
• title_fields: Optional list of field names to display in title
• show_extra_info: If True, prints detailed information after plotting
• apply_transforms: If True, apply pipeline image_transforms
• downsample: If not None, approximate desired size (in pixels) for the longest image edge when downsampling. The final image size may differ slightly due to integer stepping. If None, no downsampling.
• ax: Optional matplotlib axes to plot on. If provided, a new figure is not created.
• return_fig: If True, return the displayed image as a NumPy array and do not open a matplotlib figure. Mask overlay, when requested, is composited onto that array.
• show_mask_overlay: If True, draws the segmentation mask as a semi-transparent yellow overlay with a crisp contour. Requires masks to have been discovered via mask_dir in find_files. Useful for verifying that masks load correctly and spatially align with their images.

Returns:

The image as a NumPy array when return_fig is True; otherwise None.

Method

plot_pca

_DRPlotsMixin.plot_pca(
    self,
    n_components: int = 2,
    color_by: str | None = None,
    figsize: tuple[int, int] = (10, 8),
    source: Literal['embeddings', 'properties', 'combined'] = 'embeddings',
    property_keys: list[str] | None = None,
    filters: dict[str, typing.Any | list[typing.Any]] | None = None,
    exclude: dict[str, typing.Any | list[typing.Any]] | None = None,
    hover_features: list[str] | None = None,
    return_fig: bool = False,
    render_mode: Literal['auto', 'svg', 'webgl'] = 'webgl',
    normalize: bool = True,
    sample_size: int | None = None,
    **dr_kwargs: Any
) -> Any

Plot PCA of embeddings, properties, or combined features (Plotly, WebGL by default).

Method

plot_tsne

_DRPlotsMixin.plot_tsne(
    self,
    n_components: int = 2,
    perplexity: float = 30.0,
    color_by: str | None = None,
    figsize: tuple[int, int] = (10, 8),
    source: Literal['embeddings', 'properties', 'combined'] = 'embeddings',
    property_keys: list[str] | None = None,
    filters: dict[str, typing.Any | list[typing.Any]] | None = None,
    exclude: dict[str, typing.Any | list[typing.Any]] | None = None,
    hover_features: list[str] | None = None,
    return_fig: bool = False,
    render_mode: Literal['auto', 'svg', 'webgl'] = 'webgl',
    normalize: bool = True,
    sample_size: int | None = None,
    **dr_kwargs: Any
) -> Any

Plot t-SNE of embeddings, properties, or combined features (Plotly, WebGL by default).

Method

plot_umap

_DRPlotsMixin.plot_umap(
    self,
    n_components: int = 2,
    n_neighbors: int = 15,
    min_dist: float = 0.1,
    color_by: str | None = None,
    figsize: tuple[int, int] = (10, 8),
    source: Literal['embeddings', 'properties', 'combined'] = 'embeddings',
    property_keys: list[str] | None = None,
    filters: dict[str, typing.Any | list[typing.Any]] | None = None,
    exclude: dict[str, typing.Any | list[typing.Any]] | None = None,
    hover_features: list[str] | None = None,
    return_fig: bool = False,
    render_mode: Literal['auto', 'svg', 'webgl'] = 'webgl',
    normalize: bool = True,
    sample_size: int | None = None,
    **dr_kwargs: Any
) -> Any

Plot UMAP of embeddings, properties, or combined features (Plotly, WebGL by default).

Method

print_distance_summary

_DistancePlotsMixin.print_distance_summary(
    self,
    distance_results: pandas.DataFrame,
    group_by: str | None = None,
    dist_range: tuple = (0, 100)
) -> None

Print distance summary statistics without plotting. Safe to use when enable_plots=False.

Args:

• distance_results: DataFrame containing ‘distance’ column
• group_by: Metadata key to group by
• dist_range: Tuple of (min, max) distances to include

Class PhenoMeResults

Typed container for per-image phenotyping data (PhenoMe.results).

Access fields via attributes: results.img_path, results.metadata, results.properties, and results.embeddings.

Embeddings lifecycle embeddings tracks the eager copy held by the pipeline:

• Not yet computed — embeddings is None, pipeline._db is None
• Lazy-backed (HDF5) — embeddings is None, pipeline._db is not None (rows are fetched from disk on demand)
• Eagerly computed — embeddings is an np.ndarray of shape (N, D)

The former []-sentinel that mixed “uninitialised” with “lazy” with “being filled during extraction” has been removed. In-progress embeddings during extraction live in pipeline._emb_buffer (a plain List[np.ndarray]) and never touch this container until they are finalised.

Parameters:

• img_path (list of str or list of list[str]): Per-image file path(s). Single-channel images use a plain str; multi-channel images use a List[str] with one path per channel.
• metadata (list of dict): Per-image metadata dicts (e.g. {"drug": "DMSO", "time": "24h"}).
• properties (list of dict): Per-image computed scalar properties (e.g. {"intensity_mean": 0.42, "area": 1024}).
• embeddings (np.ndarray or None): Shape (N, D) float32 array, or None when not available / when embeddings are stored lazily in an HDF5 file.

Accessors and inspection

Property

has_embeddings

Property on PhenoMeResults

True if embeddings are stored eagerly (not lazy or absent).

Property

embedding_dim

Property on PhenoMeResults

Embedding dimensionality, or 0 if no embeddings are stored.

Other

Method

clear

PhenoMeResults.clear(
    self
) -> None

Reset all fields to empty state.

Property

has_properties

Property on PhenoMeResults

True if at least one image has computed properties.

Method

image_name

PhenoMeResults.image_name(
    self,
    idx: 'int'
) -> str

Return the basename of the primary path for image idx.

Property

metadata_keys

Property on PhenoMeResults

Sorted unique metadata keys across all images (case-preserving).

Method

metadata_value

PhenoMeResults.metadata_value(
    self,
    idx: 'int',
    key: 'str'
) -> Any

Case-insensitive metadata lookup for image idx.

Property

n_images

Property on PhenoMeResults

Number of images stored.

Method

primary_path

PhenoMeResults.primary_path(
    self,
    idx: 'int'
) -> str

Return the primary (or only) file path for image idx.

Property

property_keys

Property on PhenoMeResults

Sorted list of property names from the first non-empty properties dict.

Method

property_value

PhenoMeResults.property_value(
    self,
    idx: 'int',
    key: 'str'
) -> float | None

Return a scalar property for image idx, or None if absent.

Method

rebase_paths

PhenoMeResults.rebase_paths(
    self,
    data_dir: 'str',
    old_data_dir: 'str | None' = None
) -> None

Update all image paths by replacing old_data_dir with data_dir.

This is useful when moving a checkpoint and its dataset to a different machine or directory.

Parameters:

• data_dir (str): The new base directory where the images are located.
• old_data_dir (str, optional): The old base directory to be replaced. If not provided, it is automatically detected by finding the common prefix of all stored paths. Special value “relative” indicates that stored paths are already relative and just need to be joined with data_dir.