Linux command line tutorial

Connecting to the cluster

We will use SSH (secure shell) to connect to the cluster. On macOS we need to simply open a terminal window (under Applications › Utilities), on PC we may need to download an ssh client such as Putty.

In the terminal type:

ssh madmax.mpi-cbg.de

and enter your password:

tomancak@madmax.mpi-cbg.de's password:

Now you are on the cluster!

Last login: Sun May 19 09:01:24 2013 from 10.1.7.16  
"Platform HPC" 3.1 (build 7211) Management Node  
Welcome to the MPI-CBG cluster masternode.  
[tomancak@madmax ~]$

Moving around

pwd - print working directory

[tomancak@madmax ~]$ pwd  
/home/tomancak

cd - change directory

[tomancak@madmax ~]$ cd /projects  
[tomancak@madmax projects]$ pwd  
/projects

~ - tilda means home directory

[tomancak@madmax projects]$ cd ~  
[tomancak@madmax ~]$ pwd  
/home/tomancak

/ - slash demarcates directories and by itself means the root directory

[tomancak@madmax ~]$ cd /  
[tomancak@madmax /]$ pwd  
/

. - current directory, if we want to, for example, execute a script in the current directory we do it like this:

./executable_script

This tells the interpreter to look for the file executable_script in the current directory and not somewhere else.

.. - one directory up

[tomancak@madmax projects]$ cd ..  
[tomancak@madmax /]

pressing tab - auto-completion, we don’t need to type long names of directories, start typing the name and press tab and the linux system will either auto-complete or show you the options you have.

[tomancak@madmax ~]$ cd /projects/toman + press tab  
[tomancak@madmax ~]$ cd /projects/tomancak_lightsheet  
[tomancak@madmax tomancak_lightsheet]$

Manipulating files

more

tail

nano

scp

mkdir

rmdir

gunzip

tar

Permissions

chmod

sudo

Pipes and redirects

Misc

which

top

watch

grep

sort

find

man

Using it on the cluster

Here we put all of that goodness to work to make our life on a cluster easier.

Once we submit the jobs, we want to monitor the progress. For example bjobs -r gives us an output summarising our running jobs. If there are many, it’s not that useful. How about counting them?

bjobs -r | wc -l  
160

More usefully, we have 160 jobs running. Still how about the pending ones? We can do that:

bjobs -r | grep PEND | wc -l  
40

Ok 40. Now, how many jobs actually finished successfully? Let’s say they create some .tif files in ../../ouput:

ls ../../output/*.tif | wc -l  
200

Good, looks like 200 are finished. It’s tempting to keep pressing enter on those commands but it’s also tiresome. Using:

watch -n5 'ls ../../output/*.tif | wc -l'

outputs continuously:

Every 5.0s: ls ../../output/*.tif | wc -l                                                                                               
Mon May 20 21:16:37 2013  
180

i.e. we get the number of files automatically updated every 5 seconds (without the -n paramater, every 2s by default).

ImageJ Docs

Page contents