Difference between revisions of "Segmentation of neuronal structures in EM stacks challenge - ISBI 2012"

m (Evaluation metrics)
(resolved the mystery of who is the expert human neuroanatomist)
Line 2: Line 2:
  
  
The images are representative of actual images in the real-world, containing some noise and small image alignment errors. None of these problems led to any difficulties in the manual labeling of each element in the image stack by an expert human neuroanatomist. The aim of the challenge is to compare and rank the different competing methods based on their pixel and object classification accuracy.
+
The images are representative of actual images in the real-world, containing some noise and small image alignment errors. None of these problems led to any difficulties in the manual labeling of each element in the image stack by an [http://albert.rierol.net expert human neuroanatomist]. The aim of the challenge is to compare and rank the different competing methods based on their pixel and object classification accuracy.
  
  

Revision as of 11:37, 24 October 2011

Example of ssTEM image and its corresponding segmentation
In this challenge, a full stack of EM slices will be used to train machine learning algorithms for the purpose of automatic segmentation of neural structures.


The images are representative of actual images in the real-world, containing some noise and small image alignment errors. None of these problems led to any difficulties in the manual labeling of each element in the image stack by an expert human neuroanatomist. The aim of the challenge is to compare and rank the different competing methods based on their pixel and object classification accuracy.


Relevant dates

  • Deadline for submitting results: February 1st, 2012
  • Notification of the evaluation: February 21st, 2012
  • Deadline for submitting abstracts: March 1st, 2012
  • Notification of acceptance/presentation type: March 15th, 2012

How to participate

Everybody can participate in the challenge. The only requirement consists of filling up the registration form here to get a user name and password to download the data and upload the results.

The best ranked methods will be presented a the workshop previous to the IEEE International Symposium on Biomedical Imaging (ISBI) 2012

If you have any doubt regarding the challenge, please, do not hesitate to contact the organizers.

Training data

Input training data and corresponding labels

The training data is a set of 30 sections from a serial section Transmission Electron Microscopy (ssTEM) data set of the Drosophila first instar larva ventral nerve cord (VNC). The microcube measures 2 x 2 x 1.5 microns approx., with a resolution of 4x4x50 nm/pixel.

The corresponding binary labels are provided in an in-out fashion, i.e. white for the pixels of segmented objects and black for the rest of pixels (which correspond mostly to membranes).

To get the training data, please, register in the challenge server, log in and go to the "Downloads" section.

Test data

The contesting segmentation methods will be ranked by their performance on a test dataset, also available in the challenge server, after registration.

The test data is another volume from the same Drosophila first instar larva VNC.

Evaluation metrics

In order to evaluate and rank the performances of the participant methods, we will use topology-based segmentation metrics, together with the pixel error. Each metric will have an updated leader-board.

The metrics are:

  • Minimum Splits and Mergers Warping error, a segmentation metric that penalizes topological disagreements, in this case, the object splits and mergers.
  • Maximal F-score of Rand index, a measure of similarity between two clusters or segmentations.
  • Pixel error, or squared Euclidean distance between the original and the result labels.


If you want to apply these metrics yourself to your own results, you can do it within Fiji using this script.

Organizers

  • Ignacio Arganda-Carreras (Howard Hughes Medical Institute, Department of Brain and Cognitive Sciences, Massachusetts Institute of Technology, Cambridge, MA, USA)
  • Sebastian Seung (Howard Hughes Medical Institute, Department of Brain and Cognitive Sciences, Massachusetts Institute of Technology, Cambridge, MA, USA)
  • Albert Cardona (Institute of Neuroinformatics, Uni/ETH Zurich, Switzerland)
  • Johannes Schindelin (University of Wisconsin, WI, USA)