Skip to content, Skip to search

Changes

HPC Workflow Manager

1,335 bytes added, 06:27, 5 November 2019
m
Expanding on the greeting example
[[File:hpc-workflow-manager-create-job.png |300px|thumb|center| Figure 4: Example of a new Macro job configuration.]]
=== How to start a job upload the data and the Macro script ===
If you have created a new job, the main window should look roughly like Figure 5.
[[File:hpc-workflow-manager-created-job.png |300px|thumb|center| Figure 5: A new Macro job has been created.]]
Here Before you can start the job, you need to upload your script (“user.ijm”). To do this you must select the “Upload data” item from the context menu. If your script also needs data, they should also be located in the same directory that the user's script is in. The data will be uploaded along with the Macro script file. A timer will appear in the download column. When it has completed uploading the data and user's script the cell that corresponds to the job should indicate that it is “Done” (Figure 6). [[File:hpc-workflow-manager-upload-job.png |300px|thumb|center| Figure 6: Uploading files (in this case just the user’s script) is done.]] Now that the script file is uploaded the job can be started. === How to start a job ===Let's inspect figure 6 closer. In the figure you can see the following columns:
* “Job ID” - Job’s identification number;
* “Status” – The job’s current status which can be:
Right-click on the new job to display the context menu (of Figure 3). You will notice that there are new enabled items.
Before you can start the job, you need to upload your script (“user.ijm”). To do this you must select the “Upload data” item from the context menu. A timer will appear in the download column. When it has completed the uploading the cell that corresponds to the job should indicate that it is “Done” (Figure 6). [[File:hpc-workflow-manager-upload-job.png |300px|thumb|center| Figure 6: Uploading files (in this case just the user’s script) is done.]] Now that the script file is uploaded the job can be started. Right-click the row of the job and select “Start Job” from the context menu.
To make the source code of the user cleaner and easier to understand the special functions that make parallelism available to the user are appended to the user script on upload and a new file is created called “mpitest“parallelMacroWrappedScript.txt” ijm” which is the file that will be executed on the cluster.
To inspect the submitted file (for example for debugging) you can right-click the job and select “Open Macro in editor” where you can see the contents of the user script along with the appended function definitions that provide parallelism.
If you are new to Macro programming it is suggested to read [[Introduction into Macro Programming]] first. This will provide you with a sufficient introduction to the basics of Macro programming in Fiji.
If you opt to use the script editor you may also find [[Using the Script Editor]] useful. Using the script editor is suggested as it supports autocompletion for the functions added by HPC Workflow Manager.
You should also be familiar with the graphical user interface of the HPC Workflow Client.
[[File:hpc-workflow-manager-progress-no-task.png |300px|thumb|center| Figure 9: The job has finished and all the progress indicators are present. Note that the task "Get the cake" has a progress indicator only on node one (1) as expected. This is because this task was added in an <code>if</code> statement checking that the rank is one.]]
 
[[File:hpc-workflow-manager-side-by-side-example.png |800px|thumb|center| Figure 10: The red circle labeled zero (0) is a task performed by all nodes. It is first added, and then its progress is reported twice: once when it is zero and finally when it is done. The green circle labeled three (3) is first added and then its progress is reported as well, however, notice that all its related commands are inside the body of <code>if</code> statements. Notice that all calls of <code>parAddTask()</code> are before <code>parReportTasks()</code>.]]
=== Available functions (list) ===
Many of the functions have an MPI equivalent, this will also be listed in the table to aid people familiar with MPI. This is because the current implementation uses OpenMPI 4.0. Note however that this does not mean that this will be a wrapper for MPI for Fiji Macro and the underlying implementation may and probably will change.
{| class="wikitable"
| parScatter || An array to split and send, number of elements to send, number of elements to receive and the rank of the node to send the elements. || An array. || This works like parScatterEqually but in this case the user is responsible for providing the parameters to split the array. || MPI_Scatter
|-
| Gather parGather || An array to send, number of items to send, number of items to receive, receiver's rank|| An array. || All ranks send an equal amount of array items to a single node of a given rank. Be careful, the receiveCount parameter should be the count of items received from each rank separately. This is the inverse operation of parScatter. || MPI_Gather
|-
| GatherEqually parGatherEqually || An array to send, number of elements in received array, receiver's rank || An array. || The given rank will receive an array send in parts by all ranks. This is the inverse operation of the parScatterEqually. || None
|}
| parReportTasks || None || None || Outputs all task ids with their descriptions.
|}
 
If you need help remembering the functions and what they do you may use autocompletion to get help. Just type "par" and a list of options will appear. There is a link to this page in the autocomplete help. An example is provided in figure 10.
 
[[File:hpc-workflow-manager-autocomplete.png |300px|thumb|center| Figure 11: Function autocompletion example.]]
== Installation ==
95
edits