Skip to content, Skip to search

Changes

Getting started with MaMuT

1,976 bytes removed, 10:02, 20 March 2018
m
Fix typo
This page contains a tutorial for the [[MaMuT]] plugin. It describes and document all its features using a publication related dataset.
 
== Publication. ==
If you find MaMuT useful for your research, please cite it:
''Reconstruction of cell lineages and behaviors underlying arthropod limb outgrowth with multi-view light-sheet imaging and tracking'', Carsten Wolff, Jean-Yves Tinevez, Tobias Pietzsch, Evangelia Stamataki, Benjamin Harich, Stephan Preibisch, Spencer Shorte, Philipp J Keller, Pavel Tomancak, Anastasios Pavlopoulos '''bioRxiv''' 112623; doi: https://doi.org/10.1101/112623{{Publication | MaMuT}}
MaMuT offers three kind of views:
* '''MaMuT Viewer''' is the main view that overlays the image data and the annotations using the physical coordinate system. This is where you will mainly interact with the data, and create, edit and move spots around. This view is based on the BigDataViewer.
* '''TrackScheme''' is the track browser, taken from TrackMate. It only shows the annotation data in a hierarchical way, discarding any physical location information. Tracks are laid out along time ranging from top to bottom, and arranged from left to right according to their name. This view is useful to make sense of the annotation data, as well as to edit links between spots.
* '''3D Viewer''' shows a 3D View of the annotation data in the physical coordinate system, without the image data. It is based on the [[3D_Viewer|'''ImageJ 3D Viewer''']].
=== The BigDataViewer framework and large, multi-view images. ===
The MaMuT viewer is really a BigDataViewer window, so if you are familiar with this plugin, everything you know applies there. If are not, here is a recapitulation of how the BigDataViewer works and how to interact with it.
A little word on the BigDataViewer: The BigDataViewer was made to deal with very large images of a sample possibly acquired from several different orientations (e.g. rotating the sample in a SPIM). These different orientations (or views, but here we use this word for another notion) of the data amount to a new dimension. This dimension if more complex to handle than e.g. multiple channels, because changing the acquisition orientation generates a data block which is not aligned with the other blocks (rotated, translated, and if you change the magnification, scaled), and does not have necessary the same size. As of today, only a limited numbers of image viewers can deal with multiple orientations, and the BigDataViewer is one of them.
The dataset we use for this tutorial is an excerpt from a long term time-lapse experiment done with a Zeiss LightSheet Z1, filming the development of a shrimp (\textit{''Parhyale hawaiensis}'') over several days. We have here the first 10 time-points, spreading over a bit more than 1 hour. Three views were acquired that generated a source each:
* Angle 325° is the ventral-right view.
* Angle 235° is the ventral-left view.
* Angle 0°, which is the first one in MaMuT, is the fused volume from all 3 input sources after spatiotemporal registration and multi-view deconvolution.
 
* {{key press|Mousewheel}} Move along the Z-axis of the view. Press {{key press|Shift}} or {{key press|Control}} to change the speed.
* {{key press|Alt-Mousewheel}} (Mac and Linux) or {{key press|Ctrl-Shift-Mousewheel}} (Windows) Zoom in and out.
* {{key press|\textuparrowup}} / {{key press|\textdownarrowdown}} Zoom in / out. With {{key press|Shift}} fast zoom. With {{key press|Control}} slow zoom.
* {{key press|Left Drag}} Rotate around the point where the mouse was clicked.
* {{key press|X}} / {{key press|Y}} / {{key press|Z}} Select rotation axis.
* {{key press|\textleftarrowup}} / {{key press|\textrightarrowdown}} Rotate clockwise / counter-clockwise around the chosen rotation axis.
* {{key press|,}} / {{key press|.}} Move forward / backward along the z-axis.
* {{key press|Shift|X}} Rotate to the ZY-plane of the current source (look along the x-axis of the current source).
There is a fantastic feature in the BigDataViewer that you will find here called \textbf{'''bookmarks}'''. They let you store a position and orientation in space as bookmarks. You can later call them again. To use them:
* First press {{key press|Shift|B}} then any other key to bookmark the current view position.
* Pressing {{key press|B}} then the bookmark's key to restore the view position.
* {{key press|O}} does the same things, but only restore the bookmark orientation, and does not translate to its position.
\end{myitemize}
You can many bookmarks, all identified by the key you press after the bookmark command.
=== Configuring the source visibility and display. ===
Switching from one source to another is done with the numeric keys {{key press|1}} ... {{key press|0}} for up to 10 views. Pressing {{key press|F}} switches to the fused mode, where all sources are overlaid. You can add and remove sources from the fused view by pressing {{key press|Shift|1}} - etc. The color and brightness of each source are defined in the \textbf{'''brightness and color} ''' panel, brought by pressing the {{key press|S}} key.
[[Image:bdv-MaMuT_ViewerBrightnessColor.png|500px700px]]
On this screenshot, we used the fused mode, toggle the first and third sources off (the deconvolved source and the angle 280°), and looked at the data in a XZ plane.
The MaMuT viewer also overlays some useful information:
[[Image:bdv-MaMuT_ViewerOverlaysMaMuT_ViewerOverlays2.pdfpng|500px700px]]
The MaMuT viewer only displays a slice of the current source(s). It fetches the pixel values it needs to generate a single slice through the data. By default, pixel values are interpolated using the nearest neighbor, which might generate a pixelated look for high level of zoom. By pressing {{key press|I}} you can toggle between nearest-neighbor interpolation, and tri-linear interpolation, which smoothes the display.
Move the mouse pointer over the cell you want to annotation, and press {{key press|A}}. A magenta circle should appear, representing a cell or more generally, a spot. Spots are created, edited and removed using the following default key bindings:
\begin{myitemize} \item * {{key press|A}} to add a spot at the mouse location. \item * {{key press|D}} to delete the spot under the mouse location. \item * {{key press|Q}} / {{key press|E}} to decrease / increase the spot radius. Use {{key press|Shift}} / {{key press|Control}} to change the radius by a greater / smaller amount. \item * {{key press|Space}} is used to move a spot in the XY view plane: put the mouse pointer inside the spot you want to move, then press and hold space while moving the mouse. The spot will follow your mouse until you release the {{key press|Space}} key.\end{myitemize}
Spots represents point of interest (in our case, cells) under the shape of a sphere. You can change the sphere time-point, location and radius, but that's it. There is no object contour in MaMuT. Spots are drawn as circles in the MaMuT viewer, with the radius of their intersection with the view XY plane. If they are not intersecting with the view plane, they are drawn as small dots, as you can see by moving along Z a bit.
=== Using multiple viewers to position spots. ===
Accurately placing a spot in 3D can be difficult. This is where using multiple views in MaMut can help. On the main GUI window, click on the \smallimg{mammouth-32x32.png} \textbf{'''MaMuT Viewer} ''' button again to open a new view. This new view might not be showing the spot you just added. Fortunately, all views can be brought in sync by clicking inside a spot.
In the first view, click on the spot you added. The second view is translated so that this very spot is brought at the center of the view window. Rotate around this point so as to show the YZ plane of the source (press {{key press|Shift|X}}) and zoom close to the spot, to have a view pair resembling this:
[[Image:bdv-MaMuT_MultipleViews.png|500px]]
Just to ensure we are looking at the same spot in the two views, we checked the \textbf{'''Display spots names} ''' button in the main GUI. Use the {{key press|Space}} key to move the spot around in a plane until you are happy with its location. This view combination is useful to place properly spots in 3D. You can open as many views as you want. Other views can be used e.g. to have an overview of the data using a dezoomed view.
Spots can also be created with a double-click of the mouse. So if we recapitulate:
\begin{myitemize}
\item {{key press|Left Click}} inside a spot to select it and center all views to this spot.
\item {{key press|Double Left Click}} outside any spot to create a new one at the mouse location.
\end{myitemize}
* {{key press|Left Click}} inside a spot to select it and center all views to this spot.
* {{key press|Double Left Click}} outside any spot to create a new one at the mouse location.
=== Display settings for spots. ===
Now is a good time to talk a little bit on how we control the look of spots on the MaMuT viewer. All the MaMuT are in sync and they share common display settings. It is not possible to separately tune the display of each view. Display settings for spots can be tuned on the main GUI window, in the top part of the \textbf{Views} tab. Here you can toggle the visibility of all spots, make their name appear. The display radius simple change their apparent radius on the MaMuT viewer. This display setting has no impact on subsequent analysis and apart from visibility tidiness, it will have its utility later.
The spot coloring uses the notion of ''numerical features''. In MaMuT, and as in TrackMate, each annotation object can have several numerical, scalar features associated. For instance a spot can have features like X, Y, Z for its position, etc. The drop-box menu \textbf{Set color by} lets you choose the feature you want to use for the spot color. The color range below the menu shows you the min and max value for the feature you picked over all the dataset and interpolate from blue to red with a jet color-map. If you scroll through the menu, you can see that the features available are sorted in three categories: <tt style="font-size: 1.25em;">spot features</tt>, <tt style="font-size: 1.25em;">default</tt> and <tt style="font-size: 1Display settings for spots.25em;">track features</tt>. <tt style="font-size: 1.25em;">Spot features</tt> is the category where you can find all the numerical features that relate to single spots, like their position, radius, etc. In <tt style="font-size: 1.25em;">default</tt>, colors are not picked from a numerical feature, but either all the same (uniform color) or set manually (we will see later how). The <tt style="font-size: 1.25em;">track feature category</tt> is special: it gives to spots the color taken from the feature of the ''track'' they belong to.
[[Image:bdv-MaMuT_ConfigureSpotDisplayNow is a good time to talk a little bit on how we control the look of spots on the MaMuT viewer. All the MaMuT are in sync and they share common display settings. It is not possible to separately tune the display of each view. Display settings for spots can be tuned on the main GUI window, in the top part of the '''Views''' tab. Here you can toggle the visibility of all spots, make their name appear. The display radius simple change their apparent radius on the MaMuT viewer. This display setting has no impact on subsequent analysis and apart from visibility tidiness, it will have its utility later.pdf|500px]]
By defaultThe spot colouring uses the notion of ''numerical features''. In MaMuT, and as in TrackMate, each annotation object can have several numerical, scalar features associated. For instance a spot can have features like X, Y, Z for its position, etc. The drop-box menu '''Set color by''' lets you choose the feature you want to use for the spot color. The color range of below the color scale is taken from menu shows you the minimal to min and max value for the maximal feature valueyou picked over all the dataset and interpolate from blue to red with a jet color-map. This If you scroll through the menu, you can be changed by doublesee that the features available are sorted in three categories: <tt style="font-size: 1.25em;">spot features</tt>, <tt style="font-size: 1.25em;">default</tt> and <tt style="font-size: 1.25em;">track features</tt>. <tt style="font-clicking on size: 1.25em;">Spot features</tt> is the \textbf{Set color by}, which will bring a window category where you can find all the numerical features that relate to single spots, like their position, radius, ''etc.'' In <tt style="font-size: 1.25em;">default</tt>, colours are not picked from a numerical feature, but either all the same (uniform color) or set manually (we will see later how). The <tt style="font-size: 1.25em;">track feature category</tt> is special: it gives to spots the min and max manuallycolor taken from the feature of the ''track'' they belong to.
[[Image:bdv-MaMuT_ConfigureFeatureRangeMaMuT_ConfigureSpotDisplay.pdf|500pxpng]]
By default, the range of the color scale is taken from the minimal to the maximal feature value. This can be changed by double-clicking on the '''Set color by''', which will bring a window where you can set the min and max manually.
[[Image:bdv-MaMuT_ConfigureFeatureRange.png]]
== Linking cells across time. ==
We propagate the identity of a cell over time using several spots in several time-points, connected from one to its successor by ''links''. Particle-linking algorithms are algorithms that find automatically what are the right links from one time-point to another given a set of spots spread over several time-points. Of course, the definition of "right" depends on the specificity of the algorithm. Tracking is the process of finding automatically all spots in an 2D+T or 3D+T image, and linking them. MaMuT does not do fully automatic tracking. It is a tool that specializes in manual or semi-automatic tracking, privileging data exploration and manual annotation on very large images. However, we will show in another tutorial how to import the results of automated tracking algorithms in MaMuT, so that they can be verified and curated. Here, we limit ourselves to manual and semi-automatic tracking.
A link is a directed relationship from a spot, called the source, to another spot, called the target. The source spot is always the predecessor in time, so links always point towards increasing time. All the spots that can be reached from a spot by crossing links form what is called a ''track''. In our case, a track represents a lineage from a single starting (founder) cell lineage.
The actual data structure behind the annotations in MaMuT (and TrackMate) is a ''simple directed graph''. A graph is a mathematical structure made of vertices (in our case, spots representing cells) connected by edges (in our case, links representing propagation of identify identity across several time-points). The graph is ''simple'' because it is forbidden to have more than one link between two spots, and that a link cannot link a spot to itself. It is ''directed'' because links have a direction, following increasing time. In MaMuT, we added an extra constraint that forbids the existence of a link between two spots belonging to the same time-point, but otherwise this data structure is very standard.
So a link can be created between any two spots, provided they do not belong to the same time-point. There is no other restriction. For instance, a spot can be the source of several links. If there is two outgoing links from a spot, they may represent a cell division (mother cell linked to its two daughters). Several source spots can be linked to a single common target spot, though the biological meaning of this is less evident. MaMuT does not put constraints on the number of links going from or to a spot. We reasoned that MaMuT is to be used to build generic graph-like annotations, and that the subsequent analysis should be specific to the biological context. For instance, for cell lineaging there should be at most two outgoing links from a source spot, and at most one ingoing link on a target spot.
In the MaMuT Viewer, only spots are selectable with the mouse. To create or remove a link, you have to select exactly two spots.
\begin{myitemize}
\item Select a spot by clicking inside it.
\item Move to another time-point ({{key press|N}}, {{key press|M}}, {{key press|[}}, {{key press|]}}, the time slider).
\item Add a second spot to the selection by shift-clicking inside it.
\item Press {{key press|L}} to add a link between these two spots, or to remove it already exists.
\end{myitemize}
* Select a spot by clicking inside it.* Move to another time-point ({{key press|N}}, {{key press|M}}, {{key press|[}}, {{key press|]}}, the time slider).* Add a second spot to the selection by shift-clicking inside it. * Press {{key press|L}} to add a link between these two spots, or to remove it already exists. So here are the two new useful bindings for \textbf{'''editing links}''':\begin{myitemize} \item * {{key press|Shift-Left Click}} add and remove spots from the selection. \item * {{key press|L}} toggle a link between the two spots of the selection. This does nothing if there is not exactly two spots in the selection. \end{myitemize}
Now go back to your MaMuT session, with the data oriented as aid above, and try to annotate and link the cell that divides between the first and second time-point. The mother cell can be found around X=1095, Y=1020, Z=1820 and t=0. On the image below, we tracked the daughter cell that emerges from the division on the left part of the view.
[[Image:bdv-MaMuT_DesiredTracking_1.PNG|500px]]
Note that when you create a link with this method, after link creating the selection is set to be made of only the last spot added. To create the next link, you just have to add the target spot to the selection, the source spot is already in. But still, after creating links over only 10 time-points, lets recognize let us admit that this is probably not the best quickest way to quickly create a lineage. This method is probably best suited to edit an existing annotation. 
=== The auto-linking mode. ===
Press {{key press|Shift-L}} with a MaMuT viewer window active. A message should appear in the MaMuT viewer that states the auto-linking mode is now on. In this mode, a link is automatically created when you create a new spot between this spot and the last one in the selection. Then the selection is changed to be the last spot added. Using this, you can quickly create lineage by moving forward in time with the keyboard and creating spots by typing {{key press|A}}. You can also use {{key press|Double Click}} to create spots, but because a simple click would clear the selection, you have to hold the {{key press|Shift}} key down to use auto-linking with mouse clicks.
Try to use the auto-linking mode to create the cell lineage of the dividing cell above, this time following the other daughter cell. Move back to the first time-point, select the mother cell, move to the second time-point {{key press|M}} and add a spot {{key press|A}} or {{key press|Shift-Double Click}} on the cell location. Repeat by following the right daughter cell. You should end up with an annotation that resembles the following:
MaMuT has step-wise time browsing commands to just do that. The keyboard shortcut to do this are:
\begin{myitemize} \item * {{key press|[}} Jump to the previous time step. \item * {{key press|]}} Jump to the next time step.\end{myitemize}By default, the viewer will jump to time-points multiples of 5: 0, 5, 10, etc. You can set what is the time step in the \smallimg{Icon_TrackMate.png} \textbf{'''Annotation} ''' tab of the main GUI window, under the \textbf{'''Stepwise time browsing} ''' field.
If you are on a time-point not a multiple of the time-step, the next key-press will move to the closest multiple. The goal of these commands is to quickly generate a lineage that extends deeply in time by skipping some frames. But the time-points we reach with these commands are always the same multiples, so the annotated frames are all the same when using the stepwise time browsing.
Below is an example from an actual annotation, peeking ahead the lineage visualizer we will describe in the next section. Inn In this view , cells are arranged by lineage, time running from top to bottom. Time-points are lines of alternating color. You can see that except when cells are dividing or when there is an ambiguity, this user only annotated time-points multiples of 5. This leads to a sparser annotation, generated faster.
[[Image:bdv-MaMuT_TrackSchemeStepwiseTimeBrowsing.png|500px]]
 
 
 
=== Display settings for tracks. ===
Notice that the look of tracks (represented by a straight line for each link) can be tuned in the same way spots are.
[[Image:bdv-MaMuT_ConfigureTrackDisplay.pdf|500pxpng]]
Track coloring uses the same feature system than for spots. There are scalar numerical features associated to tracks and they are used to generate a color from a jet colormap. However, there is two kind of features for tracks:
\begin{myitemize} \item * There are <tt style="font-size: 1.25em;">track features</tt>, defined for whole tracks. For instance: the number of spots in a track, the displacement from the start to the end of track, etc. If you pick a track feature, all the links of a track will share the same color. \item * There are <tt style="font-size: 1.25em;">edge features</tt>, defined for a single link. These are features that are simply defined by a link between two spots, for instance, the instantaneous velocity for the link. Then each link may be of a different color.\end{myitemize}
There is also a <tt style="font-size: 1.25em;">default</tt> category, like for spots, that allow for using a uniform color or to use link colors set manually elsewhere.
Tracks extend in time, and they are displayed as lines over the image data. By default, the whole tracks are displayed from their start to their end. You can control how they appear with the \textbf{'''Track display mode} menu: \begin{myitemize} \item <tt style="font-size: 1.25em;">Show all entire tracks</tt> is the default and display all the whole tracks with no time cue. \item <tt style="font-size: 1.25em;">Show local tracks</tt> only shows a portion of the track which extend shortly before and after the current time-point. As the time distance increases, the track display fades in the image using transparency. You can control how much time context you want to display using the \textbf{Limit frame depth} checkbox and the \textbf{Frame depth} field. \item <tt style="font-size: 1.25em;">Show local tracks, backward</tt> is similar, but only extends tracks backward in time (this mode is sometimes nicknamed 'dragon tail' in other softwares). \item <tt style="font-size: 1.25em;">Show local tracks, forward</tt> is the same thing, but forward in time. \item The same three modes are repeated with the <tt style="font-size: 1.25em;">fast</tt> label added. They are identical to their counterparts, except that optimizations are made in favor of drawing speed, sacrificing frivolities such as transparency and anti-aliasing. \item The last mode is <tt style="font-size' menu: 1.25em;">Show selection only</tt>, and only draws what is currently selected. It is very useful to make sense of a lineage amongst a possibly very large dataset. Careful: the content of the selection model can still be edited in this mode. For instance, if you click outside a spot, the selection is cleared and nothing will be displayed in this mode. This mode is best used in conjunction with TrackScheme, which we describe in the next section.\end{myitemize} The \textbf{Limit drawing depth} field is used to restrict the Z neighborhood in which tracks are drawn. If activated, only part of the tracks that are close to the current Z plane of the view will be drawn. The extend of this Z section is set by the numerical value in the field.
* <tt style="font-size: 1.25em;">Show all entire tracks</tt> is the default and display all the whole tracks with no time cue.
* <tt style="font-size: 1.25em;">Show local tracks</tt> only shows a portion of the track which extend shortly before and after the current time-point. As the time distance increases, the track display fades in the image using transparency. You can control how much time context you want to display using the '''Limit frame depth''' checkbox and the '''Frame depth''' field.
* <tt style="font-size: 1.25em;">Show local tracks, backward</tt> is similar, but only extends tracks backward in time (this mode is sometimes nicknamed 'dragon tail' in other softwares).
* <tt style="font-size: 1.25em;">Show local tracks, forward</tt> is the same thing, but forward in time.
* The same three modes are repeated with the <tt style="font-size: 1.25em;">fast</tt> label added. They are identical to their counterparts, except that optimizations are made in favor of drawing speed, sacrificing frivolities such as transparency and anti-aliasing.
* The last mode is <tt style="font-size: 1.25em;">Show selection only</tt>, and only draws what is currently selected. It is very useful to make sense of a lineage amongst a possibly very large dataset. Careful: the content of the selection model can still be edited in this mode. For instance, if you click outside a spot, the selection is cleared and nothing will be displayed in this mode. This mode is best used in conjunction with TrackScheme, which we describe in the next section.
The '''Limit drawing depth''' field is used to restrict the Z neighborhood in which tracks are drawn. If activated, only part of the tracks that are close to the current Z plane of the view will be drawn. The extend of this Z section is set by the numerical value in the field.
== TrackScheme. ==
TrackScheme is the second view offered in MaMuT. It can be seen as a lineage visualizer or editor. In TrackScheme, the image data as well as the spatial location of spots are completely discarded in favor of a hierarchical layout that highlights how cells divide in time.
In the main MaMuT GUI window, click on the \smallimg{Icon3a_print_transparency.png} \textbf{'''TrackScheme} ''' button. A new window should appear, with the following content.
\screenshotC{[[Image:bdv-MaMuT_TrackSchemeStart.png}|500px]]
In TrackScheme, tracks are arranged from left to right and time runs from top to bottom. At this time we just have a single track, with two branches. The cell we tracked divides immediately after the first time-point, which is represented in TrackScheme by a fork going down. Each branch below this fork represents the annotation of a daughter cell. However, all the spots and links for these two daughter cells still belong to the same track, as they are connected ''via'' the mother cell.
Moving around is done classically with the mouse, and the panning is triggered by holding down the {{key press|Space}} key:
\begin{myitemize} \item * {{key press|Mousewheel}} scrolls up and down. \item * {{key press|Shift|Mousewheel}} scrolls left and right. \item * {{key press|Space|Mousedrag}} pans the view, à la ImageJ. If you pull the mouse out of the TrackScheme window, it will scroll in the direction of the mouse cursor. \item * {{key press|Space|Mousewheel}} is used for zooming.\end{myitemize}
The keyboard can also be used:
\begin{myitemize} \item * The numeric keypad numbers {{key press|6}}, {{key press|9}}, {{key press|8}}, {{key press|7}}, {{key press|4}}, {{key press|1}}, {{key press|2}} and {{key press|3}} are used to move as on a compass. \item * {{key press|+}} zoom in. \item * {{key press|{-}}} zoom out. \item * {{key press|{=}equals}} restores the zoom to its default level.\end{myitemize}
The top-left part of the TrackScheme window shows the outline of the graph. The blue square represents the current view and can be resized and moved around.
\screenshotC{[[Image:bdv-MaMuT_TrackSchemeOutline.pdf}  png |500px]]
=== Configuring TrackScheme look. ===
Though TrackScheme is a view of the annotation data like the MaMuT viewer, it completely and purposely ignores some the display settings you can set on the main GUI window, such as the track display mode and the global visibility of spots and tracks. The color it chooses for the links and spots representation is also peculiar: The spot color by feature mode is ignored, even for the circles that represent spot. They take their color from the track color mode, and use the color of the incident link. For instance, if you pick the <tt style="font-size: 1.25em;">Displacement</tt> feature in the \textbf{'''track color mode}''', you will get this:
[[Image:bdv-MaMuT_TrackSchemeTrackDisplayColor.png|500px]]
Tracks have a name, and are arranged in columns, separated by a vertical black line.
\TrackScheme arranges the annotations line by line, and each line represents a time-point. The row header tells you what time-point you are looking at. The background color of each row alternates to highlight different frames. If you find the background too crowded, you can disable the alternating color by clicking on the \smallimg{Icon_application_view_columns.png} \textbf{'''Display decoration button} ''' on the toolbar. The second mode disables track columns and rows alternating colors; the third mode re-enables track columns.
[[Image:bdv-MaMuT_TrackSchemeDecorationButton.png|500px]]
Finally, there is two \textbf{'''Styles} ''' for the spot schemes. The <tt style="font-size: 1.25em;">simple</tt> style sonly displays them as round spots. The <tt style="font-size: 1.25em;">full</tt> style displays them as rounded boxes, with each spot name apparent. In the <tt style="font-size: 1.25em;">full</tt> style, small thumbnails can be captured and displayed in TrackScheme for all spots. Just next to the menu, there is a thumbnail button. If activated, thumbnails are collected from all spots, using the image source they were created on. Thumbnails are captured around the spot location, using their radius plus a tolerance factor. Interestingly, the \textbf{'''Spot display radius ratio} ''' is used to define the size of the thumbnail. For instance, with a display factor of 2, you can obtain the layout below. Notice that the spot boxes can be resized manually to better display thumbnails. [[Image:bdv-MaMuT_TrackSchemeThumbnails.png|500px]]
\screenshotC{MaMuT_TrackSchemeThumbnails.pdf}
The hierarchical layout of the lineages provided by TrackScheme can be useful for communications. It can be exported using the three export buttons in the toolbar.
[[Image:bdv-MaMuT_TrackSchemeExportButtons.pdfpng|500px]]
\begin{myitemize}
 
\item The \smallimg{Icon_camera_go.png} \textbf{Capture undecorated TrackScheme} button will generate a view of \TrackScheme and open it in Fiji. The background is set to white and the zoom level is set to the default, regardless of what the actual zoom is in TrackScheme. Once this image is in Fiji, you can modify it, save it, etc using the tools in Fiji.
\item The \smallimg{Icon_camera_edit.png} \textbf{Capture TrackScheme with decorations} button does the converse. It captures a snapshot of the TrackScheme window as is, and uses the current zoom level to generate an image.
\item The \smallimg{Icon_camera_export.png} \textbf{Export to...} opens file browser on which you can pick the export file format and its location. Many file formats are supported:
\begin{myitemize}
\item PNG image file with/without transparent background.
\item PDF or SVG file, that can later be edited with e.g. Illustrator.
\item As a HTML page, though the layout is somewhat simplified.
\item The now deprecated VML file format (replaced by SVG).
\item As text, but this only saves a minimal amount of information.
\item The MXE file format is a specialized XML format, that can be parsed with classical XML parsers.
\item And all the common image formats (PNG, JPEG, GIF, BMP).
\end{myitemize}
\end{myitemize}
* The '''Capture undecorated TrackScheme''' button will generate a view of [[TrackScheme]] and open it in Fiji. The background is set to white and the zoom level is set to the default, regardless of what the actual zoom is in TrackScheme. Once this image is in Fiji, you can modify it, save it, etc using the tools in Fiji.
* The '''Capture TrackScheme with decorations''' button does the converse. It captures a snapshot of the TrackScheme window as is, and uses the current zoom level to generate an image.
* The '''Export to...''' opens file browser on which you can pick the export file format and its location. Many file formats are supported:
** PNG image file with/without transparent background.
** PDF or SVG file, that can later be edited with e.g. Illustrator.
** As a HTML page, though the layout is somewhat simplified.
** The now deprecated VML file format (replaced by SVG).
** As text, but this only saves a minimal amount of information.
** The MXE file format is a specialized XML format, that can be parsed with classical XML parsers.
** And all the common image formats (PNG, JPEG, GIF, BMP).
=== Managing a selection in TrackScheme. ===
\[[TrackScheme ]] is useful to build a selection and query its properties. As we said above, \TrackScheme does not abide any visibility setting. Spots and links are always visible, which is useful to build a selection. Spots and links are added to the current selection in a classical way: \begin{myitemize}
\item * {{key press|Left-Click}} on a spot or link to set the selection with this spot or link. The selection is cleared before. \item * {{key press|Left-Click}} outside a spot to clear the selection.   \item * {{key press|Shift|Left-Click}} on a spot or link to add or remove this spot or link to the selection. \item * {{key press|Mousedrag}} to select multiple spots and links in a selection box. Hold {{key press|Shift}} to add them to the current selection. \end{myitemize}
Adding to this, several items in the {{key press|Right-click}} popup menu help selecting part of tracks. If you {{key press|Right-click}} on a spot or {{key press|Right-click}} outside a spot with a non-empty selection, you can:
\begin{myitemize}* <tt style="font-size: 1.25em;">Select whole track</tt> will include all the spots and the links of the tracks the selection belongs to.* <tt style="font-size: 1.25em;">Select track downwards</tt> walks from the spots and links in the selection, and add the spots and links that connect from them, forward in time (downward in TrackScheme).* <tt style="font-size: 1.25em;">Select track upwards</tt> does the same, but backward in time.
\item \smallimg{Icon_arrow_updown.png} <tt style="font-size: 1.25em;">Select whole track</tt> will include all the spots and the links of the tracks the selection belongs to.  \item \smallimg{Icon_arrow_down.png} <tt style="font-size: 1.25em;">Select track downwards</tt> walks from the spots and links in the selection, and add the spots and links that connect from them, forward in time (downward in \TrackScheme).  \item \smallimg{Icon_arrow_up.png} <tt style="font-size: 1.25em;">Select track upwards</tt> does the same, but backward in time. \end{myitemize} The same tools exist in the second tab of the main GUI window (\smallimg{Icon_TrackMate.png} \textbf{'''Annotation}'''), but we will speak of this tab later.
Selections are very useful for visualization within a crowded annotation. For instance, select one of the two branches in our single track. With the default track display mode, the selection is drawn on the MaMuT viewer as a thick green light that extends fully in time. The eighth track display mode is called <tt style="font-size: 1.25em;">Show selection only</tt> and just does that. It displays in the MaMuT viewer only the spots and links in the selection, with their proper color settings, and abide to the frame and depth limit settings.
For instance, you can use it to only display a series of disjoint parts of a tracks:
[[Image:bdv-MaMuT_TrackSchemeSelectionOnly.png|500px700px]]  
=== TrackScheme info-pane and feature plots. ===
The info pane in the middle left takes the shape of a table, that displays the numerical feature values of the spot selection as a table. Spots are arranged as columns and feature as lines. This table can be exported to an ImageJ table with the {{key press|Right-click}} popup menu.
The bottom left part if the spot feature plotter. The \textbf{'''Feature for X axis} ''' drop down menu lets you choose what will be the feature used for the X axis. \textbf{'''Feature for Y axis} ''' menus work the same way. Y-axis features can be added and removed using the \smallimg{Icon_add.png} add and \smallimg{Icon_delete.png} remove buttons.
To generate the plot, click the \smallimg{Icon_plots.png} \textbf{'''Plot features} ''' button.
A graph should appear on which you can interact a bit. {{key press|Mousedrag}} towards the bottom right direction will zoom the plot, and {{key press|Mousedrag}} towards to up right direction will reset the zoom. The {{key press|Right-click}} menu lets you configure the plot, save it to an image file and export it as an ImageJ table.
[[Image:bdv-MaMuT_TrackSchemeSideBar.pdfpng|500px800px]]  
== Editing annotations with TrackScheme. ==
Make sure you have a TrackScheme window open, a MaMuT viewer window open, and move the later close to the cells we tracked previously. Make sure the auto-linking mode is off {{key press|Shift|L}}, and start creating spots over a cell close to the first one. Try to follow it over time. You should see spots appearing in TrackScheme, under a special column on the right called <tt style="font-size: 1.25em;">Unlaid spots</tt>. The TrackScheme window should resembles this:
[[Image:bdv-MaMuT_TrackSchemeUnlaidSpots.png|500px700px]]
Normally, TrackScheme only displays the spots that belong in a track. Lonely spots that are not linked to anything when you launch TrackScheme are not shown. The spots you create after TrackScheme are however stacked under this special column. From there, you can attach them to an existing track or create a new one.
Notice that the TrackScheme display of this new track is somewhat unsatisfactory. The first track may have changed color in the MaMuT viewer, but this change did not happen in TrackScheme. Plus, the new track does not have its own column, and the color of some of its spots might be wrong. The reasons for this are:
\begin{myitemize}
\item * We changed the annotation and these changes affected the numerical features that color the tracks. For instance, if you picked the <tt style="font-size: 1.25em;">Track index</tt> numerical feature for track coloring, there is now two tracks instead of one. The feature update is seen immediately in the MaMuT viewer, but for performance reason, TrackScheme as well as the color line on the main GUI window have to be refreshed manually. To do so, click on the \smallimg{Icon_theme.png} \textbf{'''Style} ''' button in the TrackScheme toolbar, and directly on the track color line on the main GUI window.
\item * The changes we made affected the track hierarchy, but the re-layout is not triggered automatically by such changes. To do so, press the \smallimg{Icon_refresh.png} \textbf{'''Layout} ''' button in TrackScheme toolbar. This will reorganize TrackScheme with a proper layout. Since in TrackScheme, spots can be moved around at will, this is also a good way to reorder things.
\end{myitemize}
 
 
=== Linking spots with drag and drop. ===
Another way to create single links is to enable the drag-and-drop linking mode. In the \TrackScheme toolbar, click on the grayedgrated-out \smallimg{Icon_connect.png} \textbf{'''Toggle linking} ''' button.
Now move over any cell in one track. As you do, the cell gets highlighted with a green square. If you click and drag from this cell, a new link (in magenta) will emerge. Release it on any cells to create a link between the source and the target.
Tracks are ordered from left to right alphanumerically with their name. To change a track name, {{key press|Double-click}} on it in the column header part. Track names should be made of a single line with a combination of any character.
Try for instance to change the track order by changing their name. Let's call the first one 'B' and the second one 'A'. Click the \textbf{'''Layout} ''' button. Your TrackScheme window should look like this:
[[Image:bdv-MaMuT_TrackSchemeTrackNames.png|500px]]
=== Editing spot names and imposing branch order. ===
Spots also have a name, that you can see either in the MaMuT viewer by checking the \textbf{'''Display spot names}''', either in TrackScheme by using the <tt style="font-size: 1.25em;">full</tt> display style. They are all called <tt style="font-size: 1.25em;">ID\#\#</tt> by default, which is not very informative.
To edit a spot name in TrackScheme, {{key press|Double-click}} on the spot. It should be replaced by an orange box in which you can type the spot name. Press {{key press|Shift|Enter}} to validate the new name, or {{key press|Escape}} to cancel the change. Spot names may be several lines long, but their display might then not be very pleasing.
[[Image:bdv-MaMuT_TrackSchemeSpotNames.png|500px]]
 
 
== Semi-automated tracking in MaMuT. ==
MaMuT does not ship fully automatic detection and particle-linking algorithms. It was build as a tool for the manual annotation and inspection of large movies. Typically, you would use MaMuT just after the acquisition and image registration process, to check whether the data you acquired is usable, and quickly generate an annotation that will fuel your first scientific conclusions. Nonetheless, manual annotation can be very cumbersome. The semi-automatic tracking can alleviate this a bit.
Let us track one of the cell close to the two we just annotated. Go back to the first time-point and create a spot above a cell, with the right radius and location. In the main GUI window, click on the \smallimg{Icon_TrackMate.png} \textbf{'''Annotation} ''' tab. You will notice that there is a \textbf{'''Semi-automatic tracking} ''' panel. There are several parameters we will describe later. For now, simply change the <tt style="font-size: 1.25em;">Max nFrames</tt> value to 10, so that you get roughly this configuration for your TrackMate session:
[[Image:bdv-MaMuT_SemiAutoTracking_1.png|500px]]
Click on the spot you just added to add it to the selection, and click on the \smallimg{Icon1_cropped.png} \textbf{'''Semi-automatic tracking} ''' button. The tracking initiates and processes iteratively. Cells are discovered one time-point after the other, and added to track consecutively.
If you follow the case depicted above, the semi-automatic tracking does a mistake at frame 7. It captures a brighter, smaller cell further from its predecessor rather than the right one.
\screenshotC{[[Image:bdv-MaMuT_SemiAutoTracking_2.png}]]
This gives us an opportunity to explain how does the semi-automatic tracking works and what are its limitations.
The semi-automatic tracking starts from a single cell in the selection. It inspects a neighborhood centered on this cell, but in the next time-point. The neighborhood is filtered by a Laplacian-of-Gaussian filter, tuned with the initial cell radius, and candidate cells are generated from the maxima in the filtered image. The filtered pixel-value of these maxima are used as a quality measure for candidate cell. The quality value is higher if the cell in the raw image is bright, and of radius similar to that of the initial cell.
Amongst these candidates, only those who have a quality higher than a fraction of the initial cell are retained. You set this parameter using the <tt style="font-size: 1.25em;">Quality threshold</tt> parameter in the \textbf{'''Semi-automatic tracking panel}'''. Only candidates with a quality larger than the quality of the initial cell times this threshold will be retained. Cells that are placed manually do not have a quality value (it shows up as -1 if you query it), so all the candidates cells are automatically accepted.
Candidates are then filtered by distance to the initial cell. Candidates that are found further away than <tt style="font-size: 1.25em;">Distance tolerance</tt> times the initial cell radius are discarded.
As we just saw, the semi-automatic tracking is very local and does not use nor derive any prior information on cell movements. It simple tries to find the best successor of a single cell, neglecting all the other cells. As such, it must be considered as a strongly suboptimal tracking method, whose goal is strictly to assist manual annotation. Alternating between semi-automatic tracking and manual annotation (e.g. using the auto-linking mode) when the semi-automatic tracking fails can generate deep lineages quickly.
 
 
 
== Exporting images and reslicing data. ==
=== Exporting the MaMuT viewer display. ===
Notice that each MaMuT viewer window has a menu bar in which a few items are listed. You will find shortcuts to call for the help page, brings up the brightness and visibility dialogs, and in the <tt style="font-size: 1.25em;">Tools</tt> menu, the \textbf{'''Record Movie} ''' and \textbf{'''Record Max-Projection Movie} ''' commands. They are similar to their BigDataViewer counterpart, and were adapted so that the annotations are also visible on the exported images.
The \textbf{'''Record Movie} ''' command will export a PNG capture of the viewer "as is". All the source visibility and brightness settings are used, as well as the display settings for the annotation. Even the image size is taken from the current viewer window size. As only the current slice is exported, you can only configure the time-points to export.
\textbf{'''Record Max-Projection Movie} ''' command performs a Maximum Intensity Projection of the data, in a direction orthogonal to the view plane. The annotations are projected as well, and they are drawn if they were located in the central slice. On the dialog of this command, you can configure how far from the current slice you want the projection to run, and how many slices to skip when doing the projection. Careful, the units used for this settings are in voxels, not in physical units.
Below is an example obtained on the last time-point of the demo dataset. Notice that the exported image a flattened view of the data, as RGB images.
\screenshotC{[[Image:bdv-MaMuTRecordMaxProjection.png}]]
First, select exactly two cells in TrackScheme or in a MaMuT viewer. These two cells must belong to the same track. We will export a volume that follows the cell over time, from the first to the second in selection by walking along a path that joins them in the track. For instance, select the first and last cell in the track now named 'A'.
Once you have them, go the third tab in the main GUI window, called \smallimg{Icon_cog.png} \textbf{'''Actions}'''. This tab contains only actions, that are MaMuT commands requiring special interaction with the data. Select the <tt style="font-size: 1.25em;">Export track stack</tt> action in the menu, and click the \smallimg{Icon_control_play_blue.png} \textbf{'''Execute} ''' button. A dialog shows up that allow configuring the export.
[[Image:bdv-MaMuT_ExportTrackStack.PNG]]
This command really reads into the raw data, and therefore generate ImageJ stacks that have the same data representation (8-bit, 16-bit, etc) that of the raw data. This can be very useful for subsequent analysis.
 
 
 
 
== Numerical features. ==
We already skimed over numerical features when we discussed the display settings for spots and tracks. Numerical features are scalar values associated to a spot, a link or a track that measures some useful value. For instance, the X position of a spot, the displacement across a link, and the number of split events in a track. The MaMuT feature system is direcly imported from \[[TrackMate]], but only contains a subset of feature definitions from \TrackMate and a few ones related to cell lineaging.
Numerical features are automatically kept in sync with the annotation. As soon as you add an annotation or modify and delete and existing one, the recalculation of features is triggered. As mentioned above however, the range display on the main GUI window is not, and you need to click over it to refresh the displayed colormap and its range.
|-
| style="padding: 5px;" | <tt style="font-size: 1.25em;">Manual spot color</tt>
| style="padding: 5px;" | This is a special feature, that is not automatically calculated. You can assign manually a color to a set of spots. To do so, select some spots and in \TrackScheme, right-click to show the pop-up menu. The menu item <tt style="font-size: 1.25em;">Manual color for X spots</tt> will let you set the selected spot color. This color will then be used in the views if you select the <tt style="font-size: 1.25em;">Manual color</tt> item in the \textbf{'''Set color by ''' menu in the spot panel of the main GUI window. Manual colors are saved to and retrieved from a MaMuT file.}
|-
{| border="1" style="border-collapse:collapse;"
| style="padding: 5px;" | <tt style="font-size: 1.25em;">Spot source ID</tt> & <tt style="font-size: 1.25em;">Spot target ID</tt>
| style="padding: 5px;" | The ID of the spot that is the source, respectively the target, of this link. In MaMuT and \TrackMate, links have a direction, that follows time.
|-
| style="padding: 5px;" | <tt style="font-size: 1.25em;">Link cost</tt>
|}
 
 
 
 
 
== Customizing MaMuT viewer key-bindings. ==
=== The <tt style="font-size: 1.25em;">mamut.properties === file.</tt>file. ===
The key-bindings used in the MaMuT viewer can be customized through a text file, that allow you to remap any command to any key. To so, create a text file named <tt style="font-size: 1.25em;">mamut.properties</tt> in the <tt style="font-size: 1.25em;">Fiji.app</tt> folder (where the <tt style="font-size: 1.25em;">plugins</tt> and <tt style="font-size: 1.25em;">jars</tt> folders are). This file must follow this syntax example:
\begin{verbatim}<pre>
A=add spot
ENTER=add spot
control\ Q=decrease spot radius a bit
...
\end{verbatim}</pre>
It follows the syntax <tt style="font-size: 1.25em;">key=command</tt>, one per line. Modifier keys such as {{key press|Control}} and {{key press|Shift}} are specified by prepending the key with their name, separated by a space escaped with a backslash '\textbackslash\ '. Spaces in commands do not need to be escaped. The dash \# charachter character at the beginning of a line is used to insert comments.
An example of such a file can be found [[Example mamut.properties file|'''here''']].
Here is a list of all available commands.
 
{| border="1" style="border-collapse:collapse;"
| style="padding: 5px;" | Add a spot at mouse location.
If the <tt style="font-size: 1.25em;">auto-linking</tt> mode is on, also link it with the spot previously in the selection, and set the newly added spot to be the current selection.
|-
| style="padding: 5px;" | <tt style="font-size: 1.25em;">delete spot</tt>
Here we recapitulate the default key-bindings for the MaMuT viewer. This image is also included in the help window of the MaMuT viewer.
[[Image:bdv-MaMuTKeyboardLayout.pdf|500pxpng]]   
== \TrackScheme key-bindings. ==
\TrackScheme key-bindings cannot be remapped like for the MaMuT viewer. We list them here.
| style="padding: 5px;" | Center view on the last spot in current selection.
|-
| style="padding: 5px;" | {{key press|+}} & {{key press|=equals}}
| style="padding: 5px;" | Zoom in.
|-
| style="padding: 5px;" | Zoom out.
|-
| style="padding: 5px;" | {{key press|Shift|=equals}}
| style="padding: 5px;" | Reset zoom.
|-
| style="padding: 5px;" | Clear selection.
|-
| style="padding: 5px;" | {{key press|\textuparrowup}} / {{key press|\textdownarrowdown}}
| style="padding: 5px;" | Move to the previous / next spot in time, within the current track.
|-
| style="padding: 5px;" | {{key press|\textleftarrowleft}} / {{key press|\textrightarrowright}}
| style="padding: 5px;" | Move to the previous / next sibling, within the current track. Sibling are spots that belong to the same track and to the same time-point. For instance the two spots of two sister daughter cells.
|-
160
edits