# Difference between revisions of "Trainable Weka Segmentation"

Author Ignacio Arganda-Carreras, Albert Cardona, Verena Kaynig
Maintainer Ignacio Arganda-Carreras
Source on gitweb
Initial release May 19th, 2011
Development status BETA, active
Category Segmentation, Machine Learning

Advanced: this plugin has many more features and options than the basic and easier to use Trainable Segmentation plugin.

Weka: it makes use of all the powerful tools and classifiers from the latest version of Weka.

Segmentation: it provides a binary result based on the training of a chosen classifier.

## Introduction

The Advanced Weka Segmentation is a Fiji plugin that combines a collection of machine learning algorithms with a set of selected image features to produce pixel-based segmentations. Weka (Waikato Environment for Knowledge Analysis) can itself be called from the plugin. It contains a collection of visualization tools and algorithms for data analysis and predictive modeling, together with graphical user interfaces for easy access to this functionality. As described on their wikipedia site, the advantages of Weka include:

• portability, since it is fully implemented in the Java programming language and thus runs on almost any modern computing platform
• a comprehensive collection of data preprocessing and modeling techniques
• ease of use due to its graphical user interfaces

Weka supports several standard data mining tasks, more specifically, data preprocessing, clustering, classification, regression, visualization, and feature selection.

The main goal of this plugin is to work as a bridge between the Machine Learning and the Image Processing fields. It provides the framework to use and, more important, compare any available classifier to perform image segmentation based on pixel classification.

## The Graphical User Interface

Example of the first look of the plugin window when using it on a TEM image
Advanced Weka Segmentation runs on any 2D image or 2D stack of images. If the image or stack is not 8-bit grayscale, it will be converted.

By default, the plugin starts with two classes, i.e. it will produce binary pixel classification. The user can add traces to both classes using the whole set of tools for ROI (region of interest) drawing available in Fiji. That includes rectangular, round rectangular, oval, elliptical, brush polygon and freehand selections. By default, the freehand selection tool (of 1 pixel width) is automatically selected.

The user can pan, zoom in and out, or scroll between slices (if the input image is a stack) in the main canvas as if it were any other Fiji window. On the left side of the canvas there are two panels of buttons, one for the training and one for the general options. On the right side of the image canvas we have a panel with the list of traces for each class and a button to add the current ROI to that specific class. All buttons contain a short explanation of their functionality that is displayed when the cursor lingers over the buttons.

### Training panel

Example of the aspect of the plugin window after training on a TEM image

#### Train classifier

This button activates the training process. One trace of two classes is the minimum required to start training. The first time this button is pressed, the features of the input image will be extracted and converted to a set of vectors of float values, which is the format the Weka classifiers are expecting. This step can take some time depending on the size of the images, the number of features and the number of cores of the machine where Fiji is running. The feature calculation is done in a completely multi-thread fashion. The features will be only calculated the first time we train after starting the plugin or after changing any of the feature options.

If the training ends correctly, then the displayed image will be completely segmented and the result will be overlayed with the corresponding class colors. Notice that all buttons are now enabled, since all their functionalities are possible after training.

While training, this button will show the label "STOP". By clicking on it, the whole training process will be interrupted and the plugin reseted to the state previous to the training.

#### Toggle overlay

Example of resulting probability map displayed as a hyperstack

This button activates and deactivates the overlay of the result image. The transparency of the overlay image can be adjusted in the Settings dialog.

#### Create result

It creates and displays the resulting image. This image is equivalent to the current overlay but grayscale. By default, the result image will be binary (black and white).

#### Get probability

Based on the current classifier, the probability that each pixel belongs to each class is displayed on a 32-bit hyperstack.

#### Plot result

This button calls the Weka code to generate the model performance chart, i.e. the ROC, precision/recall, etc. curves based on the training data.

These curves allow to visualize the performance of the classifier based on the different thresholds that can be applied to the probability maps.

Weka model performance chart. Displayed after clicking on "Plot result"

### Options panel

#### Apply classifier

By clicking on this button we can apply the current classifier to any 2D image or stack of images we have in our file system. Two dialogs will pop up to, first, ask the user for the input image or stack and, second, ask if the result should be displayed as a probability map or a segmentation (final classes). Then the plugin will perform the image segmentation based on the current classifier and ---consequently--- selected features. This may take a while depending on the number and size of the input images and the number of cores of the machine. After finishing, the input image (or stack) and its corresponding segmentation will be displayed.

Here we can load any previously saved classifier. The plugin will check and adjust the selected features with the attributes of this new classifier. The classifier file format is the one used in Weka (.model).

#### Save classifier

It saves the current classifier into a file, under the standard Weka format (.model). This allows us to store classifiers and apply them later on different sessions (option not available in the simple Trainable Segmentation plugin).

Here we can load the data (in Weka format) from previous traces on the same or other image or stack. Again, the plugin will check and force the consistency between the loaded data and the current image, features and classes. The input file format is the standard Weka format: ARFF. Notice the traces (regions of interests selected by the user) are not saved but their corresponding feature vectors.

#### Save data

With this button we can save the current trace information into a data file that we can handle later with the plugin or the Weka Explorer itself. The plugin will save the feature vectors derived from the pixels belonging to each trace into an ARFF file at a location chosen by the user.

#### Create new class

The default number of classes of the plugin is two, but through this button we can increase up to 5 the classes. The name of the new classes can be changed on the Settings dialog.

#### Settings

Settings dialog of the Advanced Segmentation Plugin

The rest of tunable parameters of the plugin can be changed on the Settings dialog, which is displayed when clicking on this button.

##### Training features

Here we can select and deselect the training features, which are the key of the learning procedure. The plugin creates a stack of images ---one image for each feature. For instance, if only Gaussian blur is selected as a feature, the classifier will be trained on the original image and some blurred versions to it with different $\sigma$ parameters for the Gaussian.

The different available features are:

• Gaussian blur: performs $n$ individual convolutions with Gaussian

kernels with $\sigma$ equal to $1, 2, 4... 2^{n}$. The larger the radius the more blurred the image becomes until the pixels are homogeneous (by default $n=16$).

• Sobel filter: calculates the gradient at each pixel. Gaussian blurs with $\sigma=1, 2, 4... 2^{n}$ are performed prior to the filter.
• Hessian: Calculates a Hessian matrix $H$ at each pixel. Prior to the application of any filters, a Gaussian blur with $\sigma=1, 2, 4... 2^{n}$ is performed. The final features used for pixel classification, given the Hessian matrix $\left(\begin{array}{cc} a & b\\ c & d\end{array}\right)$ are calculated thus:
• Module: $\sqrt{a^{2}+bc+d^{2}}$.
• Trace: $a+d$.
• Determinant: $ad-cb$.
• First eigenvalue: $\frac{a+d}{2}+\sqrt{\tfrac{4b^{2}+(a-d)^{2}}{2}}$.
• Second eigenvalue: $\frac{a+d}{2}-\sqrt{\tfrac{4b^{2}+(a-d)^{2}}{2}}$.
• Orientation: $\frac{1}{2}\arccos\left(4b^{2}+(a-d)^{2}\right)$ This operation returns the orientation for which the second derivative is maximal. It is an angle returned in radians in the range $\left[-\frac{\pi}{2},\frac{\pi}{2}\right]$ and corresponds to an orientation without direction. The orientation for the minimal second derivative can be obtained by adding (or subtracting) $\frac{\pi}{2}$.
• Gamma-normalized square eigenvalue difference: $t^{4}(a-d)^{2}\left((a-d)^{2}+4b^{2}\right)$, where $t=1^{3/4}$.
• Square of Gamma-normalized eigenvalue difference: $t^{2}\left((a-d)^{2}+4b^{2}\right)$, where $t=1^{3/4}$.
• Difference of gaussians: calculates two Gaussian blur images from the original image and subtracts one from the other. $\sigma$ values are again set to $1, 2, 4... 2^{n}$ , so $(n-1)!$ feature images are added to the stack.
• Membrane projections: the initial kernel for this operation is hardcoded as a $19x19$ zero matrix with the middle column entries set to 1. Multiple kernels are created by rotating the original kernel by 6 degrees up to a total rotation of 180 degrees, giving 30 kernels. Each kernel is convolved with the image and then the set of 30 images are Z-projected into a single image via 6 methods:
• sum of the pixels in each image
• mean of the pixels in each image
• standard deviation of the pixels in each image
• median of the pixels in each image
• maximum of the pixels in each image
• minimum of the pixels in each image

Each of the 6 resulting images is a feature. Hence pixels in lines of similarly valued pixels in the image that are different from the average image intensity will stand out in the Z-projections.

• Mean, Variance, Median, Minimum, Maximum: the pixels within a radius of $1, 2, 4... 2^{n}$ pixels from the target pixel are subjected to the pertinent operation (mean/min etc.) and the target pixel is set to that value.
• Anisotropic diffusion: the anisotropic diffusion filtering from Fiji with 20 iterations, $1, 2, 4... 2^{n}$ smoothing per iterations, $a_{1}=0.10, 0.35$, $a_{2}=0.9$, and an edge threshold set to the membrane size.
• Bilateral filter: is very similar to the Mean filter but better preserves edges while averaging/blurring other parts of the image. The filter accomplishes this task by only averaging the values around the current pixel that are close in color value to the current pixel. The 'closeness' of other neighborhood pixels to the current pixels is determined by the specified threshold. I.e. for a value of 10 each pixel that contributes to the current mean have to be within 10 values of the current pixel. In our case, we combine spatial radius of 5, 10 and 20, with a range radius of 50 and 100.
• Lipschitz filter: from Mikulas Stencel plugin. This plugin implements Lipschitz cover of an image that is equivalent to a grayscale opening by a cone. The Lipschitz cover can be applied for the elimination of a slowly varying image background by subtraction of the lower Lipschitz cover (a top-hat procedure). Sequential double scan algorithm (according Rosenfeld) was used. We use down and top hats, with slope = 5, 10, 15, 20, 25.
• Kuwahara filter: another noise-reduction filter that preserves edges. This is a version of the Kuwahara filter that uses linear kernels rather than square ones. We use the membrane patch size as kernel size, 30 angles and 0, 1 and 2.
• Gabor filter: at the moment this option takes quite a lot of time and memory because it generates a very diverse range of Gabor filters (85). This may undergo changes in the future. The implementation details are included in this script.
##### Feature options
• Membrane thickness: expected value of the membrane thickness, 1 pixel by default. The more accurate, the more precise the filter will be.
• Membrane patch size: this represents the size NxN of the field of view for the membrane projection filters.
• Minimum sigma: minimum radius of the filters used to create the features. By default 1 pixel.
• Maximum sigma: maximum radius of the filters used to create the features. By default 16 pixels.
##### Classifier options
Classifier selection in the Advanced Weka Segmentation Settings dialog.

The default classifier is a multi-threaded version of random forest with 200 trees and 2 random features per node. However the user can select any available classifier in the Weka by clicking on "Choose" button. By left-clicking on the classifier text we can also edit the classifier options.

##### Class names

The classes can be renamed using these text boxes.

##### Homogenize classes

The classsifier uses by the default all the user traces to train. By clicking on this option, we filter first the classes in order to provide a homogeneous distribution of the samples. This implies that the less numerous classes will duplicate some of their samples and the more populated classes will lose some of their samples for the sake of even distribution. This option is hardly recommended if we want to give the same importance to all classes. An alternative is to use the Weka CostSensitiveClassifier and set a corresponding cost matrix.

##### Save feature stack

We can save the features as a stack of images by clicking on this button. It will use the last feature configuration that is available.

##### Result overlay opacity

This slider sets the opacity of the resulting overlay image. Depending on the image contrast of our input images, we might be interested on adjusting this value.

#### WEKA

The Weka button launches the Weka chooser, where we can start all the applications available in Weka.

### Macro language compatibility

Advanced Weka Segmentation is completely compatible with the popular ImageJ macro language. Each of the buttons in the GUI are macro-recordable and their commands can be reproduced later from a simple macro file.

Example of macro recording of the Advanced Weka Segmentation tools.

## Library use

The plugin GUI is independent from the plugin methods. The methods are implemented in a separate file in a library-style fashion, so they can be called from any other Fiji plugin without having to interact with the GUI. This facilitates its integration with other plugins and allows easy scripting.

This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation (http://www.gnu.org/licenses/gpl.txt).

This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.