Alignment and registration

Automatically align, scale, tanslate and rotate or apply full perspective correction.

All constants described below are contained in the object constants, imported as:

from shinestacker.config.constants import constants

Alignment actions can be added to a job as follows:

job.add_action(Actions("align", [AlignFrames(*options)])

Arguments for the constructor AlignFrames of are:

  • feature_config (optional, default: None): a dictionary specifying the following parameters, with the corresponding default values:

feature_config = {
    'detector': constants.DETECTOR_SIFT,
    'descriptor': constants.DESCRIPTOR_SIFT
}

matching_config (optional, default; None): a dictionary specifying the following parameters, with the corresponding default values:

matching_config= {
    'match_method': constants.MATCHING_KNN,
    'flann_idx_kdtree': 2,
    'flann_trees': 5,
    'flann_checks': 50,
    'threshold': 0.75
}

  • match_method (optional, default: MATCHING_KNN): the method used to find matches. See Feature Matching for more details. Possible values are:

  • flann_idx_kdtree (optional, default: 2): parameter used by the FLANN matching algorithm.

  • flann_tree (optional, default: 5): parameter used by the FLANN matching algorithm.

  • flann_checks (optional, default: 50): parameter used by the FLANN matching algorithm.

  • threshold (optional, default: 0.75): parameter used to select good matches. See Feature Matching for more details.

  • alignment_config (optional, default; None): a dictionary specifying the following parameters, with the corresponding default values:

alignment_config = {
    'transform': constants.ALIGN_RIGID,
    'align_methid': constants.RANSAC,
    'rans_threshold': 5.0,
    'refine_iters': 100,
    'align_confidence': 99.9,
    'max_iters': 2000,
    'border_mode': constants.BORDER_REPLICATE_BLUR,
    'border_value': (0, 0, 0, 0),
    'border_blur': 50,
    'subsample': 0,
    'fast_subsampling': False,
    'min_good_matches': 100,
    'phase_corr_fallback': False,
    'abort_abnormal': False
}

  • transform (optional, default: ALIGN_RIGID): the transformation applied to register images. Possible values are:

    • ALIGN_RIGID (default): allow scale, tanslation and rotation correction. This should be used for image acquired with tripode or microscope.

    • ALIGN_HOMOGRAPHY: allow full perspective correction. This should be used for images taken with hand camera.

  • align_method (optional, default: RANSAC): the method used to find matches. Valid options are:

    • RANSAC (Random Sample Consensus, default)

    • LMEDS (Least Medians of Squares)

  • rans_threshold (optional, default: 5.0): parameter used if ALIGN_HOMOGRAPHY is choosen as tansformation, see Feature Matching + Homography to find Objects for more details.

  • refine_iters (optional, default: 100): refinement iterations. Used only if transform=ALIGN_RIGID.

  • align_confidence (optional, default: 99.9): alignment algorithm confidence (%). Used only if transform=ALIGN_RIGID.

  • max_iters (optional, default: 2000): maximum number of iterations. Used only if transform=ALIGN_HOMOGRAPHY.

  • subsample (optional, default: 0=automatic): subsample image for faster alignment. Faster, but alignment could be less accurate if only a small portion of the image is focused. It can save time, in particular for large images.

  • fast_subsampling (optional, default: False): perform fast image subsampling without interpolation. Used if subsample is set to True.

  • min_good_matches (optional, default: 100): if subsample>1 and the number of good matches is below min_good_matches, the alignment is retried without subsampling. This improbes robustness in case a too large subsampling factor is specified.

  • phase_corr_fallback (optional, default: False): use phase correlation alignment in case the number of matches is too low to allow to estimate the transformation. This algorithm is not very precise, and may help only in case of blurred images.

  • abort_abnormal (optional, default: False): abort in case of anormal tansformation. By default, frames where abnormal transformation is found are skipped, and processing continunes to next frame.

  • border_mode (optional, default: BORDER_REPLICATE_BLUR): border mode. See Adding borders to your images for more details. Possible values are:

    • BORDER_CONSTANT: pad the image with a constant value. The border value is specified with the parameter border_value.

    • BORDER_REPLICATE: the rows and columns at the very edge of the original are replicated to the extra border.

    • BORDER_REPLICATE_BLUR (default): same as above, but the border is blurred. The amount of blurring is specified by the parameter border_blur.

  • border_value (optional, default: (0, 0, 0, 0)): border value. See Adding borders to your images for more details.

  • border_blur (optional, default: 50): amount of border blurring, in pixels. Only applied if border_mode is set to BORDER_REPLICATE_BLUR, which is the default option.

  • plot_summary (optional, default: False): if True, plot a summary histogram with number of matches in each frame. May be useful for inspection and debugging.

  • plot_matches (optional, default: False): if True, for each image matches with reference frame are drawn. May be useful for inspection and debugging.

  • enabled (optional, default: True): allows to switch on and off this module.

Parallel processing

A class AlignFramesParallel implements alignment using parallel processing. This class has extra parameters, in addition to the above ones:

  • max_threads (optional, default: 2): number of parallel processes allowed. The number of actual threads will not be greater than the number of available CPU cores.

  • chunk_submit (optional, default: True): submit at most max_threads parallel processes. If chunk_submit is greater than max_threads a moderate performance gain is achieved at the cost of a possibly large memory occupancy.

  • bw_matching (optional, default: False): perform matches on black and white version of the images in order to save memory. Preliminary tests indicate that the gain with this option is marginal, and this option may be dropped in the future.

  • delta_max (optional, defaut: 2): maximum consecutive frames skipped in case of too few matches found.

Automatic selection of processing strategy

A class AlignFramesAuto implements alignment with either sequential or parallel processing, and automatically tunes parallel processing parameters. This class has extra parameters, in addition to the above ones:

  • mode (optional, default: auto): can be auto, sequential or parallel.

  • memory_limit (optional, default: 8×10243sup>): memory limit to determine optimal running parameters

Allowed configurations

⚠️ Not all combinations of detector, descriptor and match methods are allowed. Combinations that are not allowed give raise to an exception. This is automatically prevented if one works with the GUI, but may occur when using python scripting. Below the table of the allowed combination with a comparison of CPU performances.

CPU performances

⏳ Below the time performances for alignment of two of the sample images with 2000×1300 resolution for allowed configuration combinations, from fastest to slowest. Note that slower may be more precise.

Time (s)

Detector

Descriptor

Match method

0.0250

SURF

ORB

NORM_HAMMING

0.0347

SURF

BRISK

NORM_HAMMING

0.0469

ORB

ORB

NORM_HAMMING

0.0471

ORB

BRISK

NORM_HAMMING

0.1001

BRISK

BRISK

NORM_HAMMING

0.1199

BRISK

ORB

NORM_HAMMING

0.1604

SURF

SIFT

KNN

0.1966

BRISK

SIFT

KNN

0.2121

ORB

SIFT

KNN

0.2738

AKAZE

AKAZE

NORM_HAMMING

0.2863

AKAZE

ORB

NORM_HAMMING

0.2887

AKAZE

BRISK

NORM_HAMMING

0.4075

AKAZE

SIFT

KNN

0.4397

SIFT

SIFT

KNN

References

For a detailed review of the various image registration methods, see the publication below: