ImageJ

{{Warning | '''''Please Note: This version of the software is outdated. We highly recommend using the new [[Automated workflow for parallel Multiview Reconstruction|automated workflow]]. It is much more user friendly, efficient and versatile using full automation of the processing'''''}}

{{Infobox
| name                   = Selective Plane Illumination Microscopy Registration on a cluster
| software               = Fiji
| author                 = Stephan Saalfeld, Christopher Schmied, Pavel Tomancak
| maintainer             = Pavel Tomancak
| filename               = n.a.
| released               = May 2013
| latest version         = May 2013
| category               = [[:Category:Registration|Registration]]
| website                = [http://www.mpi-cbg.de/research/research-groups/pavel-tomancak.html Pavel Tomancak's homepage]
}}= Introduction =

Light sheet microscopy such as SPIM produces enormous amounts of data especially when used in long-term time-lapse mode. In order to view and in some cases analyze the data it is necessary to process them which involves registration of the views within time-points, correction of sample drift across the time-lapse registration, fusion of data into single 3d image per time-point which may require multiview deconvolution and 3d rendering of the fused volumes. Here we describe how to perform such processing in parallel on a cluster computer.

We will use data derived from the [http://microscopy.zeiss.com/microscopy/en_de/products/imaging-systems/lightsheet-z-1.html Lightsheet Z.1] a commercial realisation of SPIM offered by Zeiss. The Lightsheet Z.1 data can be truly massive and cluster computing may well be the only way to deal with the data deluge coming of the microscope.

Every cluster is different both in terms of the used hardware and the software running on it, particularly the scheduling system. Here we use a cluster computer at the MPI-CBG that consists of '''44''' nodes each with '''12''' Intel Xeon E5-2640 cores running @ 2.50 GHz and enjoying '''128GB''' of memory. The cluster nodes have access to 200TB of data storage provided by a dedicated Lustre Server architecture. For more info on Lustre see [[wikipedia:Lustre (file system)|here]], suffice to say that it is optimised for high performance input/output (read/write) operations which is crucial for the SPIM data volumes.

Each node of this cluster runs CentOS 6.3 Linux distribution. The queuing system running on the MPI-CBG cluster is '''LSF''' - [[wikipedia:Platform LSF|Load Sharing Facility]]. The basic principles of job submission are the same across queuing systems, but the exact syntax will of course differ.

'''Note on versions'''

The SPIM registration is a piece of software that undergoes ongoing development. The original version gathered under plugins ''SPIM registration'' has been replaced in 2014 by new set of plugins gathered under ''Multiview reconstruction''. Moreover, the cluster pipeline has been changed to use a centralised Linux style master file. In 2015 this pipeline was reimplemented as '''[[Automated workflow for parallel Multiview Reconstruction|automated workflow]]''' using the workflow manager '''[https://bitbucket.org/johanneskoester/snakemake/wiki/Home. Snakemake]'''. Which allows to map and dispatch the workflow logic automatically either on a single maschine or on a HPC cluster. Therefore there are 4 versions available. We highly recommend using the latest version:

* [[#Original SPIM registration pipeline|Original SPIM registration pipeline]] - contains the most detailed description of the cluster pipeline using ''SPIM registration'' plugins. If you do not have much HPC/Linux experience start here.

* [[#NEW PIPELINE|NEW PIPELINE]] - also uses ''SPIM registration'' plugins and introduces the master file, less verbose requiring some experience with command line and HPC.

*  [[#New Multiview Reconstruction pipeline|New Multiview Reconstruction pipeline]] - builds on ''Multiview Reconstruction'' plugins, uses master file, written for experts

* '''[[Automated workflow for parallel Multiview Reconstruction|Automated workflow for parallel Multiview Reconstruction]]''' - automated workflow using the workflow manager '''[https://bitbucket.org/johanneskoester/snakemake/wiki/Home. Snakemake]'''

So, if you are new read a bit of the chapter 1 (original pipeline) to get familiar and then skip to chapter 3 (Multiview reconstruction pipeline) which is more up-to-date. To understand how the master file works refer to chapter 2 (NEW PIPELINE).