The content of this page has not been vetted since shifting away from MediaWiki. If you’d like to help, check out the how to help guide!
|
|
|
| Category |
Registration |
SPIM allows for three-dimensional imaging of thick specimens. Among other approaches this can be exploited to analyse invasion of different cell types deep into tissues. Due to their potential role in cancer treatment the invasion of mesenchymal stem cells (MSC) into tumors is a medically relevant issue.
We developed an in-vitro screening tool to analyse the invasion potential of MSC under different conditions. Therefore, tumor model spheroids with invaded MSC are imaged and their invasion depths are quantified. We present here the computational approach to cope with the amount of data our screening invasion assay produces.
Image processing
Using light sheet microscopy for high-throughput screening results in a significant number of data sets with each data set requiring image processing that consists of a sequence of computational steps. Each of these steps is time consuming making the analysis of many data sets by hand ineffective. Inspired by the tutorial from Pavel Tomancak (Registration on cluster) we developed a collection of linux shell scripts for (semi)automation of the analysis. These scripts are not meant to analyse time lapse data. Please keep in mind that these scripts are written in a quick and dirty fashion and do therefore not contain any protection against incorrect entries or a data organisation that differs from the following description. Feel free to adapt these scripts to your needs. Please keep in mind that there is no guarantee for error free function of these scripts and macros generated thereby.
Preparation
The shell scripts were tested on a linux server (not a cluster) with a fiji installation containing the spim registration plugin. It is assumed that analysis of open spim data by hand works fine. The linux shell scripts must be stored on your linux server in a folder containing a subfolder called macros. In this macro folder the subfolders “converter”, “fuse” and “register” have to be present. Later you can find the respective generated fiji macros in these sub folders. Take care to delete macros that are no longer useful in order not to mix them up with newly generated macros. Generation and naming of the macros is done automatically. Therefore names may be identical and macros overwrite one another.
# Basic commands:
# change directory
cd /path/to/folder
# show content of current directory
ls
# create subfolders: */
mkdir converter fuse register
# delete all data within current directory
rm *
Data organisation
A prerequisite for effectively using these scripts is the following data organisation: Parent folder (e.g. one folder for each measuring day) containing several subfolders each containing original spim data of only one measurement (e.g. control1, condition1, …). These subfolders should be all on the same level (not further nested). In order not to mix original data with computed ones one has to start a shell script first that creates for each measurement a subfolder named “convert” (see below). This folder is located below the folder of the original spim data.
Usage
Beside the script generating the convert folder there are two other types of shell scripts: Ones that produce a list of fiji macros and others executing this list one after the other automatically.
Producer shell scripts
Scripts generating macros require two command line parameters: first the complete path and name of the parent data folder and as a second parameter a whole number (integer). The producer scripts will generate one fiji macro per data set named with an ascending order of numbers. This order of numbers start with the second parameter that has been added during the start sequence of the shell script. This allows to generate a list of macros containing more than the macros of only one parent directory. For the second parent directory one should use an integer as command line parameter that is greater than the integer naming the last macro of the first parent directory. All macros are stored in one of the folder “converter”, “fuse” or “register” (cf. Preparation).
#Example commands:
./1generate_convert_dir /path/to/data
./2convert_TIFFF_TIF_js /path/to/data 1
generates:
--- macros
| --- converter
| --- ...1.js
| --- ...2.js
| --- ...
Executer shell scripts
In order to keep control over data processing the macros generated by the producer scripts are not executed automatically making a pre-check possible. Executer scripts will start fiji in a headless mode and execute the macros one by one. Therefore executer macros do not require any parameter, they execute all macros stored in the respective folder “converter”, “fuse” or “register” (cf. Preparation).
#Example command:
./3execute_convert_macro_js
Detailed description and source code of the shell scripts
The scripts build on one another meaning that the output of one script is the input or a prerequisite for the next one. Therefore the scripts are marked with ascending numbers to simplify usage.
1generate_convert_dir
In order not to mix original data with computed ones we create for each measurement a subfolder named “convert”. This folder will be located in the subfolder containing the original spim data. 1generate_convert_dir automatically generates the “convert” folders in all folders below the parent folder given as a command line parameter. Specify complete path and name of the parent directory, but not a second parameter.
#.................................................................................
#generates a directory 'convert' in all subdirectories below the directroy given as the
#first parameter
#................................................................................
#!/bin/bash
echo "Start Macro"
Path="${1}"
echo ${Path}
ls -d ${Path}/* > dir
while read line
do
mkdir $line/convert
echo generated $line/convert
done < dir
#--------------------------------------------------------------------------------------
#executes the macros one after the other
# fiji --headless -macro /opt/Fiji.app/plugins/Macros/OpenSPIM_OME_TIFF_TIF_converter.ijm
# fiji --headless -macro /opt/Fiji.app/plugins/Macros/OpenSPIM_OME_TIFF_TIF_converter.ijm
# fiji --headless -macro /opt/Fiji.app/plugins/Macros/OpenSPIM_OME_TIFF_TIF_converter.ijm
#-------------------------------------------------------------------------------------
echo "End Macro"
2convert_TIFFF_TIF_js
Shell script generates fiji macros doing the pre-processing of spim data. Specify complete path to data and an integer for each data directory as second parameter (cf. Usage). These macros are stored in the folder “./macros/converter”.
#!/bin/bash
echo "Start Macro"
Path="${1}"
resultmacro_number="${2}"
echo ${Path}
resultmacro=./macros/converter/
ls -d ${Path}/* > dirlist
while read line
do
echo 'importClass(Packages.ij.IJ);' > $resultmacro${resultmacro_number}.js
echo 'for (var a=0;a<=4;a++)' >> $resultmacro${resultmacro_number}.js
echo '{' >> $resultmacro${resultmacro_number}.js
echo 'var imp = IJ.openImage("'${line}'/spim_TL01_Angle" + a + ".ome.tiff");' >> $resultmacro${resultmacro_number}.js
echo 'imp.show();' >> $resultmacro${resultmacro_number}.js
echo 'IJ.saveAs("'Tiff'", "'${line}'/convert/spim_TL01_Angle" + a + ".tif");' >> $resultmacro${resultmacro_number}.js
echo 'imp.close();' >> $resultmacro${resultmacro_number}.js
echo '}' >> $resultmacro${resultmacro_number}.js
echo $resultmacro_number
let resultmacro_number=$resultmacro_number+1
done < dirlist
#.................................................................................
#generates a directory 'convert' in all subdirectories below the directroy given as the
#first parameter
#................................................................................
echo "End Macro"
3execute_convert_macro_js
Executes the macros generated by the 2convert_TIFFF_TIF_js script. Processed data will be stored in the “convert” subfolder located under the original data folder.
#!/bin/bash
clear
echo "Start Macro"
ls -d ./macros/converter/* > macrolist
while read line
do
echo "start converting $line"
fiji $line
echo "end converting $line"
done < macrolist
#--------------------------------------------------------------------------------------
#executes the macros one after the other
# fiji /home/hh/programs/macros/converter/1.js
# .
# .
# .
# fiji /home/hh/programs/macros/converter/n.js
#-------------------------------------------------------------------------------------
echo "End Macro"
4generate_register_macros
Shell script that generates fiji macros doing registration of spim data. Before using this script you have to adopt the registration parameters to the needs of the respective experiment. Specify complete path to data and an integer for each data directory as second parameter (cf. Usage). These macros are stored in the folder “./macros/register”.
#!/bin/bash
echo "Start Macro"
Path="${1}"
resultmacro_number="${2}"
echo ${Path}
resultmacro=./macros/register/
ls -d ${Path}/* > dirlist
while read line
do
echo $resultmacro_number
#generation of JavaScript macros starts here
echo "importClass (Packages.ij.IJ);" > $resultmacro${resultmacro_number}.js
echo "importClass (Packages.ij.ImagePlus);" >> $resultmacro${resultmacro_number}.js
echo "IJ.run('Bead-based registration', 'select_type_of_registration=Single-channel' + ' ' +" >> $resultmacro${resultmacro_number}.js
echo "'select_type_of_detection=[Difference-of-Mean (Integral image based)] ' + ' ' +" >> $resultmacro${resultmacro_number}.js
echo "'spim_data_directory=' +'"$line"'+'/convert/' + ' ' +" >> $resultmacro${resultmacro_number}.js
echo "'pattern_of_spim=spim_TL0{t}_Angle{a}.tif' + ' ' +" >> $resultmacro${resultmacro_number}.js
echo "'timepoints_to_process=' + '1' + ' ' +" >> $resultmacro${resultmacro_number}.js
echo "'angles_to_process=' + '0-4' + ' ' +" >> $resultmacro${resultmacro_number}.js
echo "'bead_brightness=[Advanced ...]' + ' ' +" >> $resultmacro${resultmacro_number}.js
echo "'subpixel_localization=[3-dimensional quadratic fit (all detections)]' + ' ' +" >> $resultmacro${resultmacro_number}.js
echo "'specify_calibration_manually xy_resolution=0.658 z_resolution=6.0' + ' ' +" >> $resultmacro${resultmacro_number}.js
echo "'transformation_model=Affine' + ' ' +" >> $resultmacro${resultmacro_number}.js
echo "'channel_0_radius_1=2' + ' ' +" >> $resultmacro${resultmacro_number}.js
echo "'channel_0_radius_2=3' + ' ' +" >> $resultmacro${resultmacro_number}.js
echo "'channel_0_threshold=0.004'" >> $resultmacro${resultmacro_number}.js
echo ");" >> $resultmacro${resultmacro_number}.js
#generation of JavaScript macro ends here
let resultmacro_number=$resultmacro_number+1
done < dirlist
#.................................................................................
#To run SPIM_registration type 'SPIM_registration Path_of_SPIM_Data'
#outpus are one Java Script macro pro subdirectory in 'Path_of_SPIM_Data' with name convention 'i.js' This macro register spim data in the rescpective directory
#start macros by typing fiji i.js
#................................................................................
5execute_register_macros
Executes the macros generated by the 4generate_register_macros.
#!/bin/bash
clear
echo "Start Macro"
ls -d ./macros/register/* > macrolist
while read line
do
echo start converting $line
fiji $line
echo end converting $line
done < macrolist
#--------------------------------------------------------------------------------------
#executes the register macros one after the other
#-------------------------------------------------------------------------------------
echo "End Macro"
6generate_fusion_macros
Shell script that generates fiji macros doing fusion of spim data. Before using this script you have to adopt the fusion parameters to the needs of the respective experiment. Specify complete path to data and an integer for each data directory as second parameter (cf. Usage). These macros are stored in the folder “./macros/fuse”.
#!/bin/bash
echo "Start Macro"
Path="${1}"
resultmacro_number="${2}"
echo ${Path}
resultmacro=./macros/fuse/
ls -d ${Path}/* > dirlist
while read line
do
echo $resultmacro_number
#generation of JavaScript macros starts here
echo "importClass (Packages.ij.IJ);" > $resultmacro${resultmacro_number}.js
echo "importClass (Packages.ij.ImagePlus);" >> $resultmacro${resultmacro_number}.js
echo "IJ.run('Multi-view fusion', 'select_channel=Single-channel' + ' ' +" >> $resultmacro${resultmacro_number}.js
echo "'spim_data_directory=' +'"$line"'+'/convert/' + ' ' +" >> $resultmacro${resultmacro_number}.js
echo "'pattern_of_spim=spim_TL0{t}_Angle{a}.tif timepoints_to_process=' + '1' + ' ' +" >> $resultmacro${resultmacro_number}.js
echo "'pattern_of_spim=spim_TL0{t}_Angle{a}.tif' + ' ' +" >> $resultmacro${resultmacro_number}.js
echo "'angles=' + '0-4' + ' ' +" >> $resultmacro${resultmacro_number}.js
echo "'fusion_method=[Fuse into a single image]' + ' ' +" >> $resultmacro${resultmacro_number}.js
echo "'process_views_in_paralell=All' + ' ' +" >> $resultmacro${resultmacro_number}.js
echo "'blending' + ' ' +" >> $resultmacro${resultmacro_number}.js
echo "'content_based_weights_(fast,' + ' ' +" >> $resultmacro${resultmacro_number}.js
echo "'downsample_output=1' + ' ' +" >> $resultmacro${resultmacro_number}.js
echo "'registration=[Individual registration of channel 0]' + ' ' +" >> $resultmacro${resultmacro_number}.js
echo "'crop_output_image_offset_x=' + '0' + ' ' +" >> $resultmacro${resultmacro_number}.js
echo "'crop_output_image_offset_y=' + '0' + ' ' +" >> $resultmacro${resultmacro_number}.js
echo "'crop_output_image_offset_z=' + '0' + ' ' +" >> $resultmacro${resultmacro_number}.js
echo "'crop_output_image_size_x=' + '0' + ' ' +" >> $resultmacro${resultmacro_number}.js
echo "'crop_output_image_size_y=' + '0' + ' ' +" >> $resultmacro${resultmacro_number}.js
echo "'crop_output_image_size_z=' + '0' + ' ' +" >> $resultmacro${resultmacro_number}.js
echo "'fused_image_output=[Save 2d-slices, all in one directory]');" >> $resultmacro${resultmacro_number}.js
#generation of JavaScript macro ends here
let resultmacro_number=$resultmacro_number+1
done < dirlist
#.................................................................................
#To run SPIM_registration type 'SPIM_registration Path_of_SPIM_Data'
#outpus are one Java Script macro pro subdirectory in 'Path_of_SPIM_Data' with name convention 'i.js' This macro register spim data in the rescpective directory
#start macros by typing fiji i.js
#................................................................................
7execute_fuse_macros
Executes the macros generated by the 7execute_fuse_macros.
#!/bin/bash
clear
echo "Start Macro"
ls -d ./macros/fuse/* > macrolist
while read line
do
echo "start fusing $line"
fiji $line
echo "end fusing $line"
done < macrolist
#--------------------------------------------------------------------------------------
#executes the register macros one after the other
#-------------------------------------------------------------------------------------
echo "End Macro"
Analysis of Invasion
The strategy implemented here involves segmentation, i.e. three-dimensional localisation, of both spheroids and invaded cells and measuring invasion depths. In contrast to the prior image processing steps the following macros are conducted directly within the fiji user interface. This allows firsthand control on the analysis.
8bitconversion_macro
Fiji macro converting fused spim data to 8-bit files required for further analysis. The macro imports image sequences out of each “convert” folder and stores 8-bit files into a new output directory. Before using this script you have to adopt the scale parameters to the needs of the respective experiment.
input = "path\\to\\data\\"
output = "path\\to\\converted\\data\\"
function convert(input, output, filename) {
run("Image Sequence...",
"open=" + input + filename + "\\convert\\output\\img_tl1_ch0_z000.tif sort");
// set properties
run("Properties...",
"channels=1 frames=1 unit=um pixel_width=2.632 pixel_height=2.632 voxel_depth=2.632");
// set conversion scale
run("Brightness/Contrast...");
setMinAndMax(0, 30000);
call("ij.ImagePlus.setDefault16bitRange", 8);
run("8-bit");
// save .tif
index = lastIndexOf(filename, "/");
if (index!=-1) filename = substring(filename, 0, index);
saveAs("Tiff", output + filename );
close();
}
list = getFileList(input);
for (i = 0; i < list.length; i++)
convert(input, output, list[i]);
InvasionAnalysis_macro
Fiji macro doing analysis of cell invasion depths. Both spheroids and invaded cells are segmented and the shortest distance of each invaded cell to the surface of the spheroid is displayed. Analysis is based on the 3D object counter and 3D manager fiji plugins. Before using this script you have to adopt the threshold parameters for segmentation to the needs of the respective experiment.
input = "path\\to\\data\\";
output = "path\\to\\results\\";
// Thresholds for segmentation:
// spheroidThreshold = ;
// mscThreshold = ;
function distances(input, output, filename) {
// segmentation
open (input + filename);
run("3D Objects Counter", "threshold=spheroidThreshold slice=100 min.=80000 max.=45925560 surfaces");
selectWindow(filename);
run("3D Objects Counter", "threshold=mscThreshold slice=100 min.=400 max.=45925560 objects");
// distances
Objects = "Objects map of " + filename ;
Sphero = "Surface map of " + filename ;
run("3D Manager");
selectWindow(Sphero);
Ext.Manager3D_AddImage();
selectWindow(Objects);
Ext.Manager3D_AddImage();
Ext.Manager3D_Count(nb_obj);
for (i = 2; i < nb_obj-1; i++) {
Ext.Manager3D_Dist2(0,i,"c2b1",dist);
index = lastIndexOf(filename, ".tif");
if (index!=-1) filename = substring(filename, 0, index);
setResult(filename,i-2,dist);
}
// save results
run("Set Measurements...", " mean limit redirect=None decimal=0");
saveAs ("Results", output + filename + ".txt");
Ext.Manager3D_Close();
close(filename);
close(Sphero);
close(Objects);
}
list = getFileList(input);
for (i = 0; i < list.length; i++)
distances(input, output, list[i]);
