Embeddings & Models
An embedding is a digital “fingerprint” of an image: a fixed-size vector of numbers that captures the most important visual features—textures, shapes, patterns—without requiring a human to define them.
Image -> Vision model (the "brain") -> Visual fingerprint (the numbers)Supported models
Section titled “Supported models”Common choices:
- DINOv2 (default): self-supervised vision transformer by Meta AI.
- CLIP: vision-language models.
- Custom models: any model can be used with the Vision model wrappers.
DINOv2 transfers well to diverse microscopy images without training or fine-tuning, which is why it is the default.
Embedding dimensions
Section titled “Embedding dimensions”| Model | Embedding dimension |
|---|---|
| ViT-S/14 (Small) | 384 |
| ViT-B/14 (Base) | 768 |
| ViT-L/14 (Large) | 1024 |
| ViT-g/14 (Giant) | 1536 |
For custom models see Vision model wrappers.
Embedding lifecycle: eager vs lazy
Section titled “Embedding lifecycle: eager vs lazy”| Mode | When | Where | Access |
|---|---|---|---|
| Eager | No checkpoint used during process_images() |
results.embeddings (in-memory np.ndarray) |
Direct slice |
| Lazy | Checkpoint path provided or load_results() used |
On disk (HDF5); results.embeddings is None |
get_embeddings() loads rows on demand |
Use get_embeddings() for lazy-safe access. For full HDF5 storage details, see the Database protocol.
Distances and centroids
Section titled “Distances and centroids”Analysis methods (compute_reference_distances, detect_outliers, find_prototypes) L2-normalize embeddings by default. Centroid distances use the arithmetic mean of vectors in the group; that mean is not generally a unit vector, so Euclidean distance to the centroid is not the same as angular (cosine) distance to the mean direction. For angular comparisons on normalized embeddings, use distance_type='cosine' in reference-distance calls or metric='cosine' in prototype search.
Temporal images (in-memory)
Section titled “Temporal images (in-memory)”Use process_temporal_images() to add new images temporarily to an existing analysis without re-running process_images. Temporal images are kept in memory only (not persisted to checkpoint) and are tagged with metadata['source'] = 'NEW'. Call clear_temporal_data() to remove them. See Temporal images.