Your data#

CHART starts from single-cell image feature tables (e.g. generated by CellProfiler or scallops): one row per segmented cell, with its guide assignment and its image-derived features. Images are needed only by two quality-control components and by the archetype crops.

Tables#

Each well contributes an objects table and a features table, both parquet. Both hold one row per cell and three identifier columns: the cell’s label, unique within the well; its guide; and the perturbation the guide targets. The two tables are joined on these, and cells missing from the features table are dropped with a warning. The objects table carries the geometry that quality control and the crops read (areas, centroids, bounding boxes); the features table carries the measurements. A screen that writes one table per well holding both sets premerged: true under preprocessing_bywell.

Cells without a guide or perturbation assignment are kept unless preprocessing_bywell.drop_unassigned is true.

File names and wells#

The tables are read from preprocessing_bywell.merged_dir, by default preprocessing/bywell/ under data_output_dir; an absolute path or a URL points elsewhere.

preprocessing_bywell:
  merged_dir: /data/screen1/objects
  objects_pattern: 'plate1_{well}.objects.parquet'
  features_pattern: 'plate1_{well}.features.parquet'

Without patterns the files are {well}-objects.parquet and {well}-features.parquet, and a premerged table is {well}.parquet unless merged_pattern names it. {well} is the only placeholder.

The wells are the values {well} takes among the objects files, or the premerged ones. That name identifies the well in every output and must be the key of its images. Once wells are listed under preprocessing_bywell.wells, which per-well image settings require, only those are processed.

Columns#

The schema block says which columns hold each role. The defaults are the published screen’s names.

Key	Role	Default
`cell_column`	cell label, unique within a well	`label`
`guide_column`	guide	`sgRNA`
`gene_column`	perturbation the guide targets	`gene_symbol`
`feature_patterns`	regular expressions selecting the feature columns	`[cell_, nuclei_, cytosol_]`
`control_gene`	perturbation marking control cells	`NTC`
`control_prefix`	prefix marking further control perturbations; `null` for none	`OR`
`centroid_columns`	cell coordinates, y then x, for the quality-control and outlier position plots	`[nuclei_centroid-0, nuclei_centroid-1]`
`bbox_columns`	bounding box for the crops: first row, first column, then one past the last of each	`[cell_bbox-0, cell_bbox-1, cell_bbox-2, cell_bbox-3]`
`nuclei_area_column`	nuclear area, for the `plots` and `segmentation` quality-control components	`nuclei_area`
`cell_area_column`	cell area, for `plots`	`cell_area`
`cytosol_area_column`	cytoplasmic area, for `segmentation`	`cytosol_area`
`nuclei_bbox_columns`	nuclear bounding box, ordered as `bbox_columns`, for `pheno`	`[nuclei_bbox-0, nuclei_bbox-1, nuclei_bbox-2, nuclei_bbox-3]`

A quality-control component whose columns are missing stops the run, naming the key; --components leaves it out.

For a CellProfiler screen with its own identifiers:

schema:
  cell_column: object_number
  guide_column: sgrna_id
  gene_column: target_gene
  feature_patterns: ['^Nuclei_', '^Cells_', '^Cytoplasm_']
  control_gene: safe_harbour
  control_prefix: null
  centroid_columns: [Nuclei_Location_Center_Y, Nuclei_Location_Center_X]
  nuclei_area_column: Nuclei_AreaShape_Area
  cell_area_column: Cells_AreaShape_Area
  cytosol_area_column: Cytoplasm_AreaShape_Area

Features#

feature_patterns match anywhere in a column name: nuclei_ matches cell_nuclei_ratio, ^nuclei_ only names that start with it. Of the matching columns, those matching preprocessing_bywell.exclude_patterns are dropped. These are case-insensitive and by default bbox, centroid, location, centermass, to_origin, _x, _y and orientation, which mark position and orientation rather than measurements. Every remaining column must be numeric; a text column stops the run at the normalize step.

Channels#

The archetype analysis works in one feature space per channel, so feature names must contain their channel, as in the published screen’s nuclei_mean_TOM20. archetypes.all_channels lists every channel in the table. A channel’s space is every feature naming no other channel, less shape measures such as eccentricity, solidity and perimeter. Consequently:

colocalisation features, such as cell_corr_TOM20_CALR, belong to neither channel;
features naming no channel, such as cell_area, enter every channel’s space;
a channel missing from all_channels stays in every other channel’s space;
no channel name may contain another, as DAPI is contained in DAPI1.

Controls#

Control cells are those whose perturbation equals control_gene or starts with control_prefix: in the published screen, non-targeting guides (NTC) and guides against olfactory receptors (OR…), which these cells do not express. They are the reference for the guide filter, the origin of each channel’s PCA space and the comparison group of every test in chart effects. A step that finds none stops.

Images#

Images are read through scallops. Their locations go under preprocessing_bywell, in global for every well or under wells per well; a relative path is taken under scallops_dir.

preprocessing_bywell:
  global:
    cell_labels_dir: /data/screen1/cell-seg.zarr
    nuclei_labels_dir: /data/screen1/nuclei-seg.zarr
  wells:
    A1:
      pheno_dir: /data/screen1/pheno/A1
    A2:
      pheno_dir: /data/screen1/pheno/A2

The crops read each well’s phenotype image, images/<well> in pheno_dir, and its segmentation, labels/<well>-cell and labels/<well>-nuclei in the two label zarrs. archetypes.image_channels gives each stain’s position among the image’s channels. A cell is outlined where the label image holds its label, so cell_column must hold the segmentation’s label values. The pheno and pheno_sbs components read further keys, described under Preprocessing.