Skip to content, Skip to search

Changes

SPIM Workflow Manager For HPC

376 bytes removed, 09:41, 31 October 2018
HPC Cluster
= HPC Cluster =
Execution of the ''Snakemake'' pipeline from the implemented Fiji plugin was tested on the [https://docs.it4i.cz/salomon/introduction/ Salomon] supercomputer, at IT4Innovations in Ostrava, Czech Republic, which consists of 1 008 compute nodes, each of which is equipped with 2x12-core Intel Haswell processors and 128 GB RAM, providing a total of 24 192 compute cores of x86-64 architecture and 129 TB RAM. Furthermore, 432 nodes are accelerated by two Intel Xeon Phi 7110P accelerators with 16 GB RAM each, providing additional 52 704 cores and 15 TB RAM. The total theoretical peak performance reaches 2 000 TFLOPS. The system runs a Red Hat Linux.
The Using the developed plugin, we executed the pipeline was tested on the Salomon supercomputer. As the test data set we used 90 time-point SPIM acquisition of a dataset used in experiments run on ''Drosophila melanogaster'' embryo expressing FlyFos fluorescent GFP fusion reporter for the Madmax cluster at MPI-CBG [https://imagej.net/Automated_workflow_for_parallel_Multiview_Reconstruction]''nrv2'' gene. The Madmax cluster had 44 nodes embryo was imaged with two Intel Xeon E5-2640, 2Lightsheet Z.5 GHz CPUs with 6 cores each 1 SPIM microscope (average CPU PassMark 9 498Carl Zeiss Microscopy). In comparison, Salomon nodes are equipped with two Intel Xeon E5-2680v3, 2.from 5 GHz CPU with 12 cores each (average CPU PassMark 18 626)views every 15 minutes from cellular blastoderm stage until late stages of fruitfly embryogenesis. Salomon is running a newer generation The dataset consisted of Xeon processors (Haswell) providing double the performance 170 GB of the Sandy Bridge architecture used on Madmaximage data.
Using the developed plugin, we executed the pipeline on the Salomon supercomputer at IT4Innovations in Ostrava, Czech Republic. As the test data set we used 90 time-point SPIM acquisition of a ''Drosophila melanogaster'' embryo expressing FlyFos fluorescent GFP fusion reporter for the ''nrv2'' gene.The embryo was imaged with Lightsheet Z.1 SPIM microscope (Carl Zeiss Microscopy) from 5 views every 15 minutes from cellular blastoderm stage until late stages of fruitfly embryogenesis. The data transfer and pipeline execution on Salomon using 90 nodes took 6 9 hours 37 minutes. For comparison, processing of the same dataset on a common PC workstation took 44 23 hours and 8 56 minutes. The results show that despite the data transfer overhead, a significant speedup of SPIM image analysis has been achieved by employing HPC resources.
= Citation =
Please note that the plugin SPIM Workflow Manager for HPC available through Fiji is based on a publication. If you use it successfully for your research please be so kind to cite our work:
88
edits