BigWarp is a tool for manual, interactive, landmark-based deformable image alignment. It uses the BigDataViewer for visualization and navigation, and uses a Thin Plate Spline implemented in Java to build a deformation from point correspondences.

The interface enables landmark pair placement and displays the effects of the warp on-the-fly.

Installation

BigWarp comes with Fiji. You can access it via Plugins › BigDataViewer › Big Warp Command for the newer startup dialog, or Plugins › BigDataViewer › Big Warp for the legacy dialog. If this is not visible in your installation, try updating Fiji with Help › Update….

Starting BigWarp

Big Warp Command

BigWarp project or landmarks : Select a saved BigWarp project file (json) or landmarks file (csv). Add open image: Select an image open in ImageJ to add to BigWarp. Click the + button to add the selected image to the table. Add image file/folder: Select an file or folder. Click the + button to add the selected image to the table. You can Browse your file system, or explore H5/N5/Zarr containers. Add transformation: Select a transformation. Click the + to add the selected transformation to the currrently selected row of the table. You can Browse your file system, or explore H5/N5/Zarr containers.

Selected images are displayed in a table at the bottom of the dialog. Indicate an image as moving by checking the box in the “Moving” column. Only moving images will be transformed by BigWarp. Any number of images can be set as moving. Clicking remove will remove a row from the table.

BigWarp can also apply “external” transformations to images. These are shown in the Transform column of the table. They can be added with the “Add transformation” option above, or by writing a path or N5URL directly into the table.

Big Warp

moving image : Select an image open in ImageJ to be transformed. target image: Select an image open in ImageJ as a reference image. Will not be transformed. N5/Zarr/HDF5/BDV-XML: The four options below let you selected chunked storage containers for moving and target images Moving: The root of of the storage container for the moving image (ususally ends with ".h5", ".n5", or ".zarr") Moving dataset: The path to the moving image relative to the root ("/" refers to the container root) Target: The root of of the storage container for the target image (ususally ends with ".h5", ".n5", or ".zarr") Target dataset: The path to the target image relative to the root ("/" refers to the container root) Landmarks file: A csv file containing landmarks Apply transform from landmarks: if selected and landmarks are provided, BigWarp will transform the moving image(s) on startup.

Note: The moving or target images open in ImageJ may be multi-channel. Chunked file formats let you open images that are too large for “traditional” ImageJ, or stream image data on the fly from cloud storage.

Usage

Open two images in ImageJ, one moving and the other target and navigate to Plugins › BigDataViewer › Big Warp. A dialog will appear prompting selection of the moving and target images.

Alternatively, save your images with N5, Zarr, or as a bigdataviewer-xml, and specify the paths to those images in the BigWarp dialog. See the section on working with large images below.

Once the two image windows and one table window open, press ␣ Space to enter “landmark mode”. Next, click on a point in the moving image, then click on the corresponding point in the target image. After you have a few moving-target point pairs, press T to transform the moving image (you may need to re-navigate if the two image are very far apart: see the Q and W hotkeys below).

Getting Help

Press F1 at any time to open a help page with a listing of navigation and editing commands. For detailed help, create a post on the image.sc forum with a bigwarp tag. Report bugs on github.

Landmark point placement and display in the viewer

Landmark placements is done in Landmark mode which you enter by pressing ␣ Space. Users place pairs of corresponding points on the moving and target images.

The following table shows the available commands and keystrokes for landmark placement, warping.

Landmark selection and editing in the table

Some changes to landmarks can be done by interacting with the landmark table.

Notes on point addition and landmark pair selection

• Adding a new landmark pair selects that pair (row) in the table.
• If a row is “missing” a moving (target) landmark point, then it must be selected in order to add that missing point by clicking in the moving (target) viewer. Then BigWarp will find the “next” row that is missing a moving (target) landmark, and select that row automatically.
• If the selected row is not missing a landmark, the next click will add a new landmark pair.
• If multiple rows are selected when a viewer is clicked, the result will be as though only the first row was selected.

Selecting transformation types

Press F2 to bring up a transformation type selection window, or U to open the BigWarp options dialog (versions 4.0.0 or later support multiple options for transformations).

Example of a thin plate spline transformation	Example of an affine transformation

Table of transformation types

Masked transformations

It is sometimes useful to limit the transformation to a certain area, for example, when fine-tuning another transformation. Options for masked transformations can be found in the BigWarp options dialog Settings › BigWarp Options or shortcut U. (Available in version 9.0.0 or later)

Turn on a transformation mask by setting the “Mask interpolation option to one of LINEAR, ROTATION, or SIMILARITY.

Details

• Mask interpolation
  • NONE : no masking
  • LINEAR : mask with “linear” interpolation
  • ROTATION : mask “rotation” interpolation
  • SIMILARITY : mask “similarity” interpolation

We generally recommend the SIMILARITY option when using a transform mask, though all options should give similar results in most cases.

• Mask falloff: The shape of the falloff curve of the spherical mask from the center (where it equals one) to the outside where it equals zero. Not relevant for imported masks.
  • COSINE
  • GAUSSIAN
  • LINEAR

• Auto-estimate mask: The size and position of the spherical mask will be automatically estimated as the smallest sphere that contains the clicked landmarks. Any manual edtis to the mask will turn this option off. Not relevant for imported masks.

• Show mask overlay: Overlays two circles on the fixed image window indicating mask size and position. Transformations are fully applied inside the inner circle, the transform is partly applied in between the two circles, and not at all applied outside the outer circle. Not relevant for imported masks.

It is also possible to import a mask from a file with File › Import Mask, or remove an imported mask with File › Remove Mask. Masks should take values between zero and one, where a value of one indicates the transform will be fully applied, a value of zero indicates the transform will not be applied, and partially applied for values in between. For masks with an intensity range that is not \([0,1]\), use the Imported mask intensity range (see below) to map its intensities as desired.

• Imported mask intensity range: Linearly map the intensity of an imported mask to \([0,1]\). The left (right) intensity will be mapped to zero (one). The Only relevant for imported masks.

Navigation and Visualization

BigWarp inherits many image navigation, visualization, and grouping features with BigDataViewer, the details of which can be found on the BigDataViewer page or on the help page. BigWarp specific features are documented below.

The following table shows the available navigation commands using the mouse:

Commands shared with BigDataViewer

• Displaying multiple stacks (“sources”)
• Grouping sources
• Adjusting brightness and color
• Bookmarking views (locations and orientations)

Mouse navigation

Keyboard navigation

For all navigation commands you can hold ⇧ Shift to rotate and browse 10x faster, or hold ⌃ Ctrl to rotate and browse 10x slower. For example, ← Left rotates by 1° clockwise, while ⇧ Shift + ← Left rotates by 10°, and ⌃ Ctrl + ← Left rotates by 0.1°.

Save and load landmarks

Landmarks can be exported and imported from plain text files using the drop down menu in the landmark table panel (File › Export (Import) landmarks).

Export warped images

Export the warped moving image by clicking File › Export as ImagePlus or using the ⌃ Ctrl + E keyboard shortcut.

The default parameters will result in the exported image having the same dimensions as the target image.

Take care when exporting very large data sets as they can cause out-of-memory exceptions.

• Resolution
  • Target: The output will have the same resolution as the target image
  • Moving: The output will have the same resolution as the moving image
  • Specified: The output will have the resolution given in the fields below (in the same units as the moving and target images).

• Field of view
  • Target: The output will have the same field of view as the target image
  • Moving: The output field of view will be the smallest bounding box that contains the warped moving images (approximated)
  • Specified (pixel): The output field of view will be that given by the Offset and Field of View parameter fields where both are in units of pixels
  • Specified (physical): The output field of view will be that given by the Offset and Field of View parameter fields where both are in the physical units of the moving and target images

The warped moving image can be exported as an in-memory or virtual ImagePlus. A virtual ImagePlus is generally faster to generate but slower to browse, whereas an in-memory ImagePlus will be slower to generate but faster to browse.

Apply transforms

Often, it is important to apply transforms estimated with one image to other images in the same space.

If you have moving and target images open in Fiji this script to transform the moving image into the space of the target image. You will need to provide a file containing the saved landmark point pairs.

To manually specify the field-of-view (FOV) of the target space, use this script

To make the scripts above appear in your Fiji Plugins menu, simply copy them into the /plugins/Scripts folder in your Fiji installation.

Export transforms

Export a transformation with File › Export transformation. BigWarp can export non-linear transformations as displacement fields that can either be displayed as a window in ImageJ or written to a file. BigWarp uses the OME-NGFF format to store transformations.

Type Flat: Compose all transformatations BigWarp knows about into one single displacement field. Sequence: If a transformation was imported on startup, this option saves a reference to it, and newly created displacement field contains only the part of the transformation editable by BigWarp. Direction FORWARD: The "forward" transform needed to transform image data. INVERSE: The "inverse" transform needed to transform coordinate data. BOTH: Exports both the forward and inverse transform when possible. Split affine: If selected, separates the affine from the deformable parts when exporting a thin-plate-spline. Threads: The number of parallel threads to use for computation. Format NGFF: The recommended option uses OME-NGFF. TPS: A temporary option that will hopefully one day join the NGFF standard. Virtual: If exporting to ImageJ, uses a virtual stack, i.e., the transformation will be computed on-the-fly, so the transform will initially appear faster, but navigation will be slower when new parts of the images need to be computed. N5 options: The parameters for writing to chunked file formats. Read about them in the N5 documentation. Field of view : Reference Moving (warped): BigWarp will set the spacing option to match the moving image, and will estimate the physical size to the smallest bounding box that fits the entire warped moving image. Target: BigWarp will set the field of view options to values matching the target image. units: the physical unit for the field-of-view of the image. min(unit): the physical coordinates of the top-left pixel. size(unit): the physical extent in each dimension. spacing(unit/px) : the physical spacing between adjacent pixels along each dimension. size(px): the number of pixels in each dimension.

Warp 2d ImageJ ROIs

Download and install this script.

Usage:

1. Run the script downloadable above.
2. Indicate which open images correspond to the moving and fixed images in bigwarp.
3. Select the csv file that stores bigwarp landmarks.
4. Indicate whether you want to transform an ROI from moving to target space or vice versa.
5. Click OK.

Note that the resulting ROI will be densely sampled. This is because an ROI with a particular shape may not have the same shape after being warped (i.e., a circle may not still be circular after being warped).

Example: In this example, we’ve chosen the boats sample as the moving image, the bridge sample as the target image, and have added some landmarks that transform the boats image.

To warp an ROI from the moving boats image to the fixed bridge image, we run the script with the options:

• Fixed image: boats
• Moving image: bridge
• Landmark file:
• Moving to target

The results are shown in the “Forward example” below.

To warp an ROI from the moving boats image to the fixed bridge image, we run the script with the options:

• Fixed image: boats
• Moving image: bridge
• Landmark file:
• Target to moving

The results are shown in the “Inverse example” below.

Forward example	Inverse example

Note, at this time ImageJ ROIs are 2D objects. We recommend using another approach for 3D regions of interest.

Warp points in a csv file

Download and install this script.

Usage:

1. Run the script download-able above.
2. Select the csv file that stores bigwarp landmarks.
3. Select the csv file storing the points you want to transform.
4. Select the location for csv file you want the results to be stored in.
5. Indicate whether you want to transform the points from moving to target space or vice versa.
6. Click OK.

Note the input csv must be formatted:

x1,y1,z1
x2,y2,z2
...
xN,yN,zN

without quotation marks, spaces, or any other characters. Csv files storing bigwarp landmarks will not work as input to this script.

Working with large images

For very large images, we recommend first converting the volume to bigdataviewer’s xml/hdf5 format, or to N5 using the N5 plugin for Fiji. Bigwarp now supports both bdv/xml and n5 formats (versions 7.0.0 and later).

Older versions of BigWarp supported these scripts:

• Script for bdv/xml (see also this forum post)
• Script for N5

Tutorials

• A vitual tutorial given by John Bogovic in April 2022.

• An example of a 2d warping by Nicolas Chiaruttini in response to this forum post.

Publication

doi:10.1109/ISBI.2016.7493463

Publications using BigWarp

1. Russell et al. “3D correlative light and electron microscopy of cultured cells using serial blockface scanning electron microscopy” J Cell Sci 130: 278-291 2017.
2. Collinson et al. “Correlating 3D light to 3D electron microscopy for systems biology” Current Opinion in Biomedical Engineering 2017 3:49-55
3. Lerner et al. “Mycobacterium tuberculosis replicates within necrotic human macrophages” J Cell Biol 2017
4. Hildebrand et al. “Whole-brain serial-section electron microscopy in larval zebrafish” Nature 545:345–349 2017.
5. Zhang et al. “A Complete Electron Microscopy Volume Of The Brain Of Adult Drosophila melanogaster” Cell 174:3 P730-743.e22, 2018.
6. Gao et al. “Cortical column and whole-brain imaging with molecular contrast and nanoscale resolution.” Science 363 (6424) 2019.
7. Wan et al. “Single-Cell Reconstruction of Emerging Population Activity in an Entire Developing Circuit.” Cell 179(2) 2019.
8. Hoffman, Shtengel et al. “Correlative three-dimensional super-resolution and block-face electron microscopy of whole vitreously frozen cells.” Science 367 (6475) 2020.
9. Weigel et al. “ER-to-Golgi protein delivery through an interwoven, tubular network extending from ER” Cell 184:9 2021.
10. Chiaruttini et al. “An Open-Source Whole Slide Image Registration Workflow at Cellular Precision Using Fiji, QuPath and Elastix” Frontiers In Computer Science 2021.
11. Ritter et al. “ESCRT-mediated membrane repair protects tumor-derived cells against T cell attack” Science 376 (6591) 2022.