🦾 Automated Colony Picking Robot Selection Accuracy

Dual-Illumination Plate Imaging — Capturing Colony Phenotype Before Any Pick Decision

Every colony-picking run begins with a high-resolution image of the agar surface. Modern colony pickers such as the Molecular Devices QPix 420 and Singer Instruments PIXL combine episcopic (reflected, top-mounted) and diascopic (transmitted, bottom-mounted) LED illumination across multiple wavelengths to capture not just colony position but translucency, pigmentation, and fluorescent reporter expression — all of which feed downstream selection logic before a single pin ever touches the plate.

• 12 MP: Imaging sensor (CCD/CMOS, telecentric lens)
• 1.2 s: Exposure per plate (6-wavelength LED sequence)
• 8.5 µm/px: Spatial resolution (sub-colony morphology detail)
• OmniTray, 100mm, 150mm: Plate formats (auto-detected by stage)

Illumination geometry and image acquisition pipeline

Colony imaging stations combine several optical subsystems to maximize downstream segmentation accuracy:

Illumination modes: • Episcopic (top) white LED: reveals surface topology, colony height/dome shape, mucoid vs. dry texture • Diascopic (bottom) white LED: reveals translucency — critical for distinguishing satellite colonies (thin, translucent) from primary colonies (opaque, raised) • Episcopic blue (470nm) + emission filter: GFP fluorescence for reporter-linked selection (e.g., directed evolution libraries with a GFP-fusion fitness reporter) • Episcopic green (530nm) + emission filter: RFP/mCherry channel for dual-reporter or co-transformation screens • Dark-field oblique illumination: enhances edge contrast for colonies with low agar contrast (e.g., pale E. coli on LB)

Stage and optics: • Motorized XY stage: ±5µm positioning repeatability (Aerotech or Parker linear stages typical in QPix/PIXL systems) • Telecentric lens: eliminates parallax error across the field of view — critical because colony diameter measurements must be accurate regardless of position on a 150mm plate • Autofocus: contrast-based Z autofocus per plate (or per-quadrant for warped agar), typical focus range ±2mm

Throughput: • Full-plate image capture (all 6 channels): 1.2–2.5s depending on channel count • 96-plate carousel/hotel imaging: <45 minutes for full batch scan-and-queue • Image file output: 16-bit TIFF per channel, ~24MB per plate at full resolution — archived for GAMP 5 audit trail and re-analysis

Pre-processing before segmentation: • Flat-field correction: removes vignetting and uneven illumination using a blank-agar calibration image captured at instrument qualification (IQ/OQ) • Agar surface reflection removal: polarizing filter or software subtraction of specular highlights from condensation • Plate edge and label-region masking: excludes barcode area and rim artifacts from the analysis ROI

Watershed Segmentation — Separating Touching Colonies into Countable, Trackable Objects

Raw plate images contain colonies at every stage of confluence — isolated, touching, and fully merged. Before any machine-learning classifier can assess colony quality, a classical image-segmentation pipeline must first convert pixels into discrete, addressable objects. Watershed transforms on the Euclidean distance map remain the workhorse algorithm because they reliably split touching colonies without requiring per-run manual tuning.

• <80 ms: Segmentation latency (per plate, embedded GPU (Jetson-class))
• Adaptive Otsu: Threshold method (per-quadrant local threshold)
• 88–94%: Split accuracy (touching pairs) (watershed vs. ground truth)
• 12.4%: False-merge rate (dense plates, >300 colonies)

Classical CV pipeline: thresholding, distance transform, watershed

Segmentation converts the calibrated grayscale image into a labeled map of candidate colony regions:

Step 1 — Background subtraction and normalization: • Rolling-ball background subtraction (radius ≈ 50px) removes uneven agar illumination gradient • Contrast-limited adaptive histogram equalization (CLAHE) boosts local contrast for faint colonies

Step 2 — Binarization: • Adaptive Otsu thresholding computed per 256×256px tile to handle illumination gradients across a 150mm plate • Morphological opening (3×3 kernel) removes single-pixel noise; closing fills small holes in colony mask

Step 3 — Distance transform and watershed: • Euclidean distance transform (EDT) computed on the binary mask: each foreground pixel gets distance-to-nearest-background value • Local maxima of the EDT become watershed seed markers (one per presumed colony center) • Watershed flood-fill from markers assigns each pixel to its nearest seed, creating boundary lines at ridges between touching colonies • Marker-controlled watershed (vs. naive watershed) prevents over-segmentation from noise

Step 4 — Region property extraction (per labeled object): • Centroid (x,y) in stage coordinates (mm), computed via image-to-stage affine calibration • Area (px² → mm² via pixel pitch), equivalent diameter • Circularity = 4π·Area/Perimeter² (1.0 = perfect circle; satellite colonies often <0.75 due to irregular budding) • Mean/median pixel intensity per channel (brightfield + fluorescence) • Nearest-neighbor distance: distance to closest other centroid — used for merge-risk and satellite flagging

Failure modes: • Dense plates (>300 CFU on 100mm dish, effectively >38 CFU/cm²): touching-colony rate rises sharply, watershed over-splits or under-splits ~12% of clusters • Faint/small colonies (<0.3mm, early growth timepoint): may fall below threshold, causing false negatives • Agar bubbles, condensation droplets: occasional false-positive ROIs, filtered by size/circularity gating downstream

CNN Morphology Scoring — Teaching a Network to Recognize a "Pickable" Colony

Segmentation alone cannot distinguish a healthy, isolated colony from a satellite artifact, a contaminant, or two colonies that watershed failed to split cleanly. A convolutional neural network trained on millions of labeled colony crops provides the final quality gate, outputting a pick-confidence score that determines whether the robot proceeds automatically, routes to a human-review queue, or discards the candidate outright.

• >2,000,000: Training set size (annotated colony image crops)
• ResNet-18 derivative: Backbone architecture (64×64px input crop, 4-class head)
• 91.2%: Classification accuracy (true-colony vs. satellite/merge/debris)
• ~4 ms/colony: Inference speed (batched, on-device GPU)

Feature space, satellite exclusion logic, and confidence routing

The classification stage operates on each segmented ROI independently, using both learned CNN features and hand-engineered morphology features as auxiliary inputs:

CNN input and architecture: • Each ROI cropped to 64×64px (or 96×96 for larger colony morphotypes), centered on centroid, normalized brightfield + fluorescence channels stacked • ResNet-18-derived backbone (reduced channel width for edge deployment), 4-way softmax output: {isolated-pickable, satellite, merged-cluster, debris/artifact} • Trained via transfer learning from ImageNet initialization, fine-tuned on lab-specific colony datasets (species, media, and camera-specific retraining recommended per SOP)

Auxiliary morphology features (concatenated before final FC layer): • Circularity, area percentile within plate, edge gradient sharpness (Laplacian variance) • Nearest-neighbor distance / own-diameter ratio — the single strongest satellite predictor • Local colony density (colonies within 5mm radius) — informs whether the plate region is over-confluent

Satellite and merge exclusion rules: • Satellite definition: secondary colony budding from a primary colony's edge, typically <40% of primary diameter, appearing within 1 colony-diameter of a larger neighbor • Merge-risk flag: two ROIs whose combined watershed boundary has low curvature contrast (<15° boundary angle) — signals imperfect split, high risk of picking a mixed-clone well • Debris/artifact filter: circularity <0.5 AND area <0.02mm² — typically agar debris or condensation, not viable colonies

Confidence thresholding and routing: • Softmax confidence ≥ threshold (operator-set, typically 0.70–0.90): autopick approved • Confidence below threshold: candidate flagged for manual image review via operator dashboard, or simply skipped (conservative mode) • Raising threshold from 0.70→0.90 typically cuts false-positive picks by ~60% but reduces total picks/plate by ~15–20% — a precision/recall trade-off tuned per application (e.g., rare-variant enrichment favors high recall; clonal purity work favors high precision)

Continuous learning loop: • Sequencing QC results (Stage 5) fed back as labels to retrain the classifier quarterly, closing the loop between picked-well genotype and CNN morphology score

Pin and Needle Mechanics — The Physical Act of Touching, Lifting, and Depositing a Colony

Once a colony clears computer-vision screening, a robotic picking head must physically contact biomass on the agar surface and transfer a viable inoculum to a destination well without cross-contamination. Pin geometry, Z-axis force control, and wash-cycle chemistry determine whether the mechanical pick preserves the genotype/phenotype fidelity that the vision system worked so hard to identify.

• 1, 96, 384-pin heads: Pin array formats (QPix, PIXL, TAP CP7200)
• ±15 µm: Z-height repeatability (autofocus-corrected per colony)
• ~2.8 s: Pick-to-transfer cycle (single-pin mode, incl. wash)
• UV + 80% EtOH: Pin wash protocol (3-stage bath, <1 CFU carryover)

Pick head engineering and contamination control between transfers

Physical colony transfer combines precision motion control with a rigorous decontamination cycle between every pick:

Pin/needle geometry: • Solid stainless-steel or tungsten pins, tip diameter 0.5–1.0mm, flat or slightly domed end for consistent biomass pickup • Single-quill mode (Singer PIXL): one pin picks and transfers sequentially — higher precision, lower throughput • Multi-pin array (96-pin QPix head): simultaneous pick from a pre-mapped grid — used for arrayed (non-selective) colony replication rather than individual CV-guided picks • Pin spring-loading: 5–15g controlled contact force prevents agar gouging while ensuring adequate biomass transfer

Z-axis approach and autofocus: • Agar surface height varies plate-to-plate (±1–2mm) due to pour volume variation; per-colony or per-quadrant Z-autofocus (capacitive or optical) compensates • Touch depth: 0.1–0.3mm into colony dome — enough for biomass transfer without full agar puncture • Approach velocity ramped down near contact (soft-landing profile) to avoid colony smearing across the plate

Destination transfer modes: • Touch-inoculation: pin directly touches destination well containing agar or dried media — simplest, lowest carryover risk • Aspirate-dispense: pin/pipette tip picks colony then is resuspended and dispensed as 5–20µL into liquid culture media (LB, TB, or selective media) in a 96/384-well block • Direct-to-PCR: pin touches a PCR-ready master mix well for immediate colony-PCR screening, bypassing outgrowth entirely

Inter-pick decontamination: • 3-stage wash: bleach/detergent bath → sterile water rinse → 80% ethanol bath, followed by UV-C exposure (2–4s) and hot-air or IR flash-dry • Validated carryover: <1 CFU transferred between consecutive picks in >99.5% of cycles (fluorescent-marker spike-in validation, Stage 5 methodology) • Cycle time cost: wash adds ~1.8s per pick in single-quill mode — the dominant throughput bottleneck at high picking density

Validating What Was Actually Picked — Throughput Metrics and Sequence-Confirmed Accuracy

A colony-picking run is only as valuable as its downstream genotypic fidelity. Vision-based confidence scores and mechanical wash validation are necessary but not sufficient — the gold-standard check is sequencing a statistically meaningful sample of picked wells to confirm that what the camera called a clean, isolated colony actually corresponds to a single, correctly-identified clone in the destination well.

• up to 3,072/hr: Full-system throughput (384-pin array mode, QPix 420)
• ~1,200/hr: Single-quill throughput (CV-gated individual picks, PIXL)
• 98.6%: Identity concordance (Sanger-confirmed clonal purity)
• 5% of wells: QC sampling rate (random stratified per plate)

Throughput benchmarking, sequencing QC protocol, and screening-scale economics

End-to-end system performance is reported as a combination of raw mechanical throughput and validated identity accuracy — the two numbers marketing materials often separate but that matter equally for library screening economics:

Throughput benchmarks by system class: • Molecular Devices QPix 420 (multi-pin array, 96/384-format): up to 3,072 colonies/hour in bulk-array mode without CV gating; ~1,800/hour with full CV-guided single-colony selection • Singer Instruments PIXL (single-quill, CV-native): ~1,000–1,200 colonies/hour, optimized for precision over raw speed • TAP Biosystems (Sartorius) CP7200: comparable multi-pin throughput, integrated into automated colony-to-fermentation workflows • Legacy manual picking baseline: ~200–400 colonies/hour per trained technician, with substantially higher inter-operator variability

Sequencing-based identity validation protocol: • Stratified random sample: 5% of wells per plate (minimum n=20) re-arrayed into a confirmation plate • Colony-PCR with vector- or insert-specific primers, followed by Sanger sequencing (or pooled amplicon NGS for higher-throughput confirmation runs, e.g., Illumina iSeq 100 with 8bp well-barcoding) • Concordance metric: fraction of sequenced wells matching expected genotype (correct insert, correct orientation, no chimeric/mixed reads) • Reported concordance of 98.6% reflects mature, well-tuned CV+mechanical pipelines; early-deployment or poorly-focused systems can fall to 85–90%

Economics and error-cost trade-offs: • Cost per confirmed clone: ~$0.03–0.08 including consumables (pin wash reagents, destination plates) at full automation, vs. ~$0.40–0.60 fully manual (technician time-costed) • False-positive pick (wrong/mixed clone in well) costs downstream: wasted sequencing reagent, wasted incubation time, and — in directed-evolution library screens — potential loss of a rare high-fitness variant if it was mis-picked as a satellite • Data integrity: all image, coordinate, and pick-decision logs retained per ALCOA+ principles and FDA 21 CFR Part 11 electronic-record requirements for GMP-adjacent cell-line development workflows

In a 2023 head-to-head benchmark across three synthetic-biology core facilities, CV-guided single-colony picking reduced downstream sequencing waste (wells requiring re-pick due to mixed or absent clones) from 14.2% (manual picking) to 1.8% (QPix 420 + CNN classifier), while cutting technician hands-on time per 10,000-colony directed-evolution library screen from roughly 48 hours to 3.5 hours of supervised instrument runtime.