Connecting to the cluster
We will use SSH (secure shell) to connect to the cluster. On macOS we need to simply open a terminal window (under Applications › Utilities), on PC we may need to download an ssh client such as Putty.
In the terminal type:
and enter your password:
Now you are on the cluster!
pwd - print working directory
cd - change directory
~ - tilda means home directory
/ - slash demarcates directories and by itself means the root directory
. - current directory, if we want to, for example, execute a script in the current directory we do it like this:
This tells the interpreter to look for the file
executable_script in the current directory and not somewhere else.
.. - one directory up
pressing tab - auto-completion, we don’t need to type long names of directories, start typing the name and press tab and the linux system will either auto-complete or show you the options you have.
Pipes and redirects
Using it on the cluster
Here we put all of that goodness to work to make our life on a cluster easier.
Once we submit the jobs, we want to monitor the progress. For example bjobs -r gives us an output summarising our running jobs. If there are many, it’s not that useful. How about counting them?
More usefully, we have 160 jobs running. Still how about the pending ones? We can do that:
Ok 40. Now, how many jobs actually finished successfully? Let’s say they create some .tif files in
Good, looks like 200 are finished. It’s tempting to keep pressing enter on those commands but it’s also tiresome. Using:
i.e. we get the number of files automatically updated every 5 seconds (without the -n paramater, every 2s by default).