The problem with tracking algorithms is that they always give an answer.
This answer can be completely irrelevant, even non-physical, and there is no built-in flags that would indicate something wrong. The best way to avoid basing your downstream analysis on faulty tracking results is to know in what situation the tracker works the best, and what are its limitations. This is the aim of this page for the trackers and detectors shipped with TrackMate.
The ISBI 2012 single particle challenge.
In 2011-2012, an ISBI Grand Challenge was organized for the Single-Particle Tracking algorithms. Though TrackMate does not offer a completely new algorithm, product of an original Research work, we took the chance and participated in the challenge. The results and the methodology to compute the accuracy of a tracking algorithms were published thereafter.
Unsurprisingly, we did not score amongst the best. At the time, TrackMate was in version 1.1, and ship a stripped down version of the better performing Jaqaman et al. LAP framework. See the LAP trackers section for algorithm details. Plus, TrackMate was was young at the time, and some bugs did not help.
TrackMate v2.7.x series accuracy against the ISBI dataset.
From v2.7.x, TrackMate ships a new tracker that can deal specifically with linear motion. We though it was the right time to re-run the accuracy assessment with the ISBI challenge data. The people behind Icy offered the website to host the challenge data, and it is still available today for download.
The figures below shows the comparison of accuracy for the 3 classes of tracking algorithms available in TrackMate:
- The LAP framework derived from Jaqaman et al..
- The linear motion tracker based on Kalman filter.
- The plain Nearest neighbor tracker for reference.
It's best to directly read the paper to know what is behind these measures, but here is a brief survey of how they are done. The ISBI dataset covers four scenarios:
- MICROTUBULE: Particles have a slightly elongated shape to mimic MT tip staining. They undergo a roughly constant velocity motion.
- RECEPTOR: Particles are spherical and undergo a tethered motion.