ROIs


In image processing, two of the most often needed yet complex operations are segmentation and registration. Regions of interest (ROIs) are an effective way of expressing and visualizing the results of a segmentation. But ROIs are a deceptively complex topic, difficult to implement well in a software package.

Discussion

There is clearly is a division between ROIs as a data structure, and ROIs for display. For clarity, we propose the following terminology:

Type Region Annotation
Sampling Discrete (i.e., mask over the pixels)1 Continuous
Shapes Rectangles, closed polygons, segmentation results Lines, points, point markers, floating text
Usage Identify areas for algorithmic processing Visualization
Overlap Can be a problem for some data structures2 Typically fine
  1. But there could be value in allowing regions with non-integer or non-binary boundaries. For example, a region with alpha channel would allow for a "fuzzy" boundary representing the confidence level of inclusion within the region, that can be later adjusted with a threshold.
  2. For example, CellProfiler represents segmentation results using an 8-bit mask, with 0 as background, 1 as the first region, 2 as the second region, etc.

Shortcomings

Johannes Schindelin summarizes the shortcomings of ROIs in ImageJ 1.x:

  • only 2D (extends identically to n dimensions)—in Fiji, we only have several 3D extensions to ImageJ's ROIs, but they are in effect stacks of 2D ROIs, while it would be pretty interesting to have a proper surface model for 3D ROIs, at least as an option
  • no distinction between region of interest and lines/points of interest
  • tries to fit in a geometrical model, where the geometry of pixels is a highly ambiguous thing (just think about it: what does a ROI containing exactly one pixel look like?)
  • is way too tied in with the rest of the GUI (some ROIs even know which ImagePlus they belong to!)
  • no common interface, rather a base class that describes rectangular (about the most unnatural shape of regions of interest in biological data) regions of interest
  • no notion of disconnected regions of interest (it is possible to have compound regions of interest, but some algorithms consider them as separate particles while others treat the union as one region of interest)
  • inherent limitation to integer coordinates
  • no clear-cut distinction between the necessary interfaces, such as visualization, geometrical measures, determining whether a pixel is considered inside/outside
  • (added by Gabriel Landini) inconsistent behavior regarding boundary pixels of ROIs, illustrated by the Draw command (try maximizing a rectangular ROI and applying it with the Draw command; only two borders are drawn because the other two are outside the image)

Relevant Java code

Use of the JAI classes would not imply use of JAI generally. ROIShape is independent  of other JAI classes; ROI is dependent on an iterator.