[ImageJ-devel] ImageJ2 development
bnorthan at gmail.com
Tue May 13 16:24:55 CDT 2014
After seeing Pawel's e-mails, I also wanted to introduce myself and perhaps
become more involved with ImageJ2.
Like Pawel I've been exploring the ImageJ2 ecosystem recently (well for
over a year actually). I'm coming at it from a developer's point of view.
I used to work in the commercial realm until 2011 (Media Cybernetics,
mostly on Autoquant). After leaving that I did some unrelated things for a
couple of years. I did have a non-compete agreement, but that is long
expired so I am thinking about getting back into microscopy.
I'm interested mainly in high performance applications. A good example of
the type of project I'm interested in is GPU Deconvolution. I've seen buzz
about a few GPU deconvolution apps recently (I've been able to explore
Stephan Preibisch's and Bob Pepin's work on Github).
When evaluating a GPU implementation, one would want to understand
state-of-the-art CPU implementations. But how does one even determine what
is state-of-the-art?? How do you get the algorithms and test them in a
consistent fashion? (I recently attended the ISBI Deconvolution Grand
Challenge addressing some aspects of this issue). Further, it may not be
possible to produce the same results as CPU on a GPU implementation (memory
limitations, limited math libraries, etc.). It becomes necessary to run
many tests and comparisons. So to evaluate the benefits and trade-offs of
GPU deconvolution (or newish algorithms like wavelet approaches) it really
becomes important to have access to the older implementations.
For any new algorithm development, 90% of the problem (in my experience) is
understanding and evaluating what has been done before. With open source
the great thing is that understanding the past is no longer just a
literature search; it also involves getting the algorithms and running
So I see "uber-wrapper" projects (a means to run and test multiple existing
implementations under the same framework) being very useful, and I've been
developing one for deconvolution with ImageJ2 (
https://github.com/bnorthan/projects). Right now it wraps YacuDecu GPU
deconvolution , PSF generation in COSMOS  and an algorithm from ITK
deconvolution. I've also ported a Total Variation implementation from
IOCBIOS  and non periodic boundary handling from EPFL  to imglib2
based implementations (similar design to imglib2 FFTConvolution ).
These few algorithms barely scratch the surface of what is out there.
As I've gotten multiple algorithms and components wrapped or ported as
ImageJ2 commands (or more recently a few ops) it has become simple to run
test scripts and to combine components in interesting and useful ways.
I'm not entirely sure how far I will be able to develop this. Since I am
using some native components, it could become difficult to maintain and
distribute for multiple operating systems. But at the very least I hope to
polish it and get it on a personal update site. (Maybe isolating the java
only components, possibly snagging the PSF code from Icy).
The Github link will be out of date soon as I need to clean and refactor
things a bit. As I do this there may be an opportunity to contribute to
the Imagej-ops project or Imglib2. One thing I need to do is take
inventory of all the little math functions I've written. Where I've
re-invented the wheel I need to re-use the proper imagej-ops or imglib2
function instead. If some math functions I have written are not in
imagej-ops/imgib I could take a look at implementing them.
On Tue, May 13, 2014 at 3:41 AM, Pawel Niewiadomski <
pawelthebiologist at gmail.com> wrote:
> Hi Curtis,
> Thanks for your comprehensive reply.
> Hi Pawel,
>> > I decided to start work on the project by playing around with the
>> > ImageJ2 interface in some routine tasks I usually do in ImageJ. After
>> > having done some extensive testing, I just wanted to give you some,
>> > hopefully constructive, input. I don't want to sound too negative, but
>> > I really think that June 1st is a bit early to be releasing a
>> > production version of ImageJ2.
>> With respect to the ImageJ2 Swing user interface and commands, you are
>> absolutely right. However, the initial 2.0.0 release of ImageJ2 is going
>> to continue using the classic ImageJ 1.x user interface by default. The
>> new UI will still be included (Help > Switch to Modern Mode) but will
>> still be very much in beta.
> I was actually under the impression that ImageJ2 IS the modern interface
> and that it would be the default for the 2.0.0 release. I must have not
> found the relevant blog/discussion posts. My bad.
> See this blog post for details:
>> ImageJ 2.0.0 will be fundamentally the same as ImageJ1, but:
>> * Will be bundled with the Updater component which supports ImageJ
>> update sites and automatic update checking.
> * Will be bundled with the native ImageJ Launcher with quite a few nice
>> command line features.
> The imagej-win32.exe doesn't work in the build from about a week ago on my
> machine (Just the basic command that launches the GUI). I'll submit a bug
>> * Will be bundled with the ImageJ Script Editor including support for
>> several scripting languages.
> I am guessing it's the same as the FiJi script editor? In that case the
> legacy UI should have a menu option File->New->Script, like FiJi does. In
> addition, the plugins->new->plugin throws some sort of exception on first
> use, again I'll submit a bug report.
> * Will support parameterized ImageJ modules, including commands and
>> scripts, so that existing plugins can begin incremental migration toward
>> this new approach, which is more headless friendly and more
>> interoperable with tools such as CellProfiler, KNIME and OMERO.
> Sounds good - I need to look into it more closely.
> * Will come with an option to use the SCIFIO library when opening image
>> files (e.g., using File > Open). This will fix ImageJ's TIFF support to
>> be more robust, and add extensible support for additional file formats
>> without needing to hack the HandleExtraFileTypes source.
> * Will ship with all the new ImageJ2 APIs, but all these components will
>> still be in beta. We will bring each component out of beta after it has
>> been thoroughly vetted over time. These components include:
>> - imagej-common: The ImgLib2-based image data model and core
>> - imagej-ops: A framework for reusable algorithms; see
>> - imagej-ui-swing: The "pure ImageJ2" Swing user interface
>> - imagej-plugins-*, scijava-plugins-*: Core plugins for ImageJ2
>> including many commands
> Note that the Fiji distribution of ImageJ has been shipping of all of
>> these components, and operating in this way, for years now, and is a
>> well vetted system. But it is time for these components to be officially
>> available as part of ImageJ's core, rather than only from a specific
>> life-sciences-focused ImageJ distribution.
> Sounds good. I've been using FiJi for quite some time now, so I didn't see
> the "newness" of the features.
>> > Please don't take it the wrong way - I am in the process of analysing
>> > the codebase and I think it is a real software engineering feat.
>> No offense taken at all; again, it is absolutely true that the ImageJ2
>> UI needs more time in the oven. But meanwhile, it has already been over
>> four years since we launched the ImageJ2 project, and there are several
>> very mature components that need to get into the hands of users: the
>> Updater, the Launcher, the Script Editor, parameterized modules, and
>> everything else that is now part of the "SciJava Common" component
>> Because ImageJ2 consists of several pieces at various stages of
>> development, it needs to migrate out of beta piece by piece.
>> > It shows that there has been really serious thought put into
>> > architerctural design. However, at this point, and I am saying that as
>> > a daily ImageJ user, the architectural brilliance is not showing on
>> > the surface.
>> I am glad you like the design. And I agree that much more needs to be
>> done in terms of leveraging that design for the benefit of end users.
>> This is an area where your contributions could be really beneficial.
>> > I am going to submit some bug reports in a few days, but basically the
>> > interface is highly unpredictable and in many ways incompatible with
>> > ImageJ1.
>> Indeed. Please note that there are many bugs about such problems already
>> logged in the ImageJ Trac: http://trac.imagej.net/. We are actively in
>> the process of migrating away from the Trac system though, with
>> individual tickets being moved to GitHub Issues of the most relevant
>> repository. Unfortunately, since we are in the middle of that migration,
>> it may be difficult to verify whether an issue has already been filed
>> for any particular concern. When in doubt, file away and we can close
>> any duplicate issues accordingly.
>> > Let me just point to a few basic things before I submit detailed bug
>> > reports:
>> > 1. The brightness/contrast dialog sometimes sticks around when you
>> > open a new image and close the old one - you end up with multiple
>> > brightness/contrast dialogs and a single image. Moreover, I've had the
>> > dialog stay open even after I closed the application.
>> The IJ2 version of B&C has been the subject of frequent debate. In
>> short, it needs a lot more work. See http://trac.imagej.net/ticket/1100
>> and all its blocking tickets (those listed in "Blocked by").
> > 2. The color picker behavior is hectic and I couldn't figure out how
>> > and why it randomly changes color. Especially in 16-bit per channel
>> > images it is totally unpredictable.
>> > 3. With 48-bit 16-bit per channel composite images the drawing command
>> > with the white color selected basically draws random grey colors.
>> IJ2's current approach to foreground and background "colors" differs
>> from IJ1. And there are likely bugs, too.
>> * http://trac.imagej.net/ticket/965
>> * http://trac.imagej.net/ticket/1292
> > 4. The interface is extremely slow to the point of being unusable for
>> > things like looking through time-course stacks or stacks of
>> > medium-sized multicolor images.
>> That is not a problem we have noticed, unless image planes become very
>> large. How large are your image planes? >2Kx2K?
> These are not large images (1kx1k/48-bit/10 z-planes), but mind you, I am
> not using the latest hardware. However, I think many of ImageJ users will
> not be on modern hardware, and my computer can run Photoshop CS4 with ease,
> which is an indication that it is not a piece of junk, either.
> > 5. Shape selection keeps old selections after you make a new one,
>> > which is inconsistent with ImageJ1 behavior and quite maddening for
>> > someone who is used to it.
>> Yes. We will probably need an option for it, because for many new users,
>> it is maddening to have one ROI disappear when creating another. But the
>> main reason IJ2's UI works that way right now is technical: it currently
>> uses the JHotDraw library which works that way by default.
> > 6. Missing magic wand and text tool functionality.
> > 7. Missing custom toolbars.
>> Known, but no explicit issue for it yet. Low priority, given all the
>> other things the Swing UI needs first. Note that IJ2 is not
>> intrinsically limited to 8 tools like ImageJ1 is, so it is less urgent
>> to support customization. All available tools will be present in the bar
>> by default.
> OK. It would be nice if the toolbar were easily customizable by the user
> the way the old style custom toolbars were. Also, having custom
> floating/docking toolbars would be nice. I am thinking
> photoshop/illustrator here, or at least Office <2003. I don't think you
> necessarily need to implement an RCP-style interface to have those.
> > 8. In multicolor images there seems to be no way of adjusting
>> > brightness/contrast of each channel individually.
>> Hmm, you're right. This might be a relatively new bug.
> That's an absolutely essential feature for any microscopy work. I'll
> submit the bug report.
> > 9. LIF format import doesn't work - it doesn't present the usual
>> > BioFormats dialog and instead just imports the first image in the
>> > series with some random channel separation.
>> LIF format does not work with vanilla ImageJ1 either. It is handled by
>> the Bio-Formats plugin. We do not ship Bio-Formats with ImageJ2 because:
>> A) ImageJ2 is BSD-2 licensed, and the Bio-Formats proprietary file
>> format readers have an incompatible GPL license; and B) ImageJ2 is
>> supposed to be a "discipline-agnostic" piece of software, while
>> Bio-Formats is focused on life sciences file formats.
> OK. Understood.
> However, LIF should work if you download Fiji, choose Help > Switch to
>> Modern Mode, and then File > Open your LIF file. This is thanks to the
>> SCIFIO Bio-Formats compatibility component
>> (https://github.com/scifio/scifio-bf-compat) which is bundled with Fiji.
> It does work, but not with File->Open. It doesn't present the right dialog
> for file import. You can go to File->Import->Bio-Formats. When you go to
> file-open (even in modern mode in FiJi), it just imports the first image of
> the bundle with each channel on a separate plane instead of creating a
> composite. It doesn't offer you the choices the way Bio-Formats does.
> Alternately, you can install Bio-Formats by turning on the Fiji and/or
>> Bio-Formats update sites shown in ImageJ2's Help > Update "Manage Update
>> Sites" dialog.
>> > This is just a sample, but there is a bunch more. My prediction is
>> > that the users will not migrate to the new version if you ship it as
>> > is. They frankly need a reason to migrate and ImageJ2 is not offering
>> > them any.
>> Agreed; "regular users" should not be switching to the new interface
>> yet. There are too many bugs and not enough advantages.
>> That is why decided to keep the ImageJ2 releases using the 1.x
>> interface, for the time being. This keeps 100% backwards compatibility
>> while also providing many advantages:
>> * the ImageJ Updater
>> * the Script Editor
>> * user-facing improvements made possible by our ImageJ 1.x patching
>> mechanism, such as File > Open using the SCIFIO library to read TIFFs
>> and other formats more robustly
>> * new developer-facing APIs (esp.,parameterized modules)
> Sounds good.
>> There are downsides though:
>> * IJ1 UI is limited to XYZCT (though we may later patch in support
>> for additional dimensions)
>> * IJ1 UI cannot handle tiled huge image planes (a feature planned for
>> the IJ2 UI)
>> * Lack of separation of concerns; IJ1 UI is fundamentally tied to the
>> IJ1 data model
> I think that and the fact that FiJi already has many of these features is
> why I was convinced that ImageJ2 is the modern interface. The data model
> limitation is a serious one, because people cannot just migrate their
> plugins in one fell swoop. Rather, they have to first play with the new
> APIs, then start working on the data model conversion, etc.
> * Further reading: http://dev.imagej.net/rationale,
>> > Right now ImageJ1 is a mature platform with few bugs and a plethora of
>> > mostly seamlessly working plugins.
>> In many ways. But due to ImageJ1's protracted incremental development,
>> its API how grown organically far beyond its original design goals, so
>> it now contains a plethora of limitations and edge cases.
>> > The biggest gripe most users have with ImageJ1 is its antiquated UI
>> With that insight in mind, we recently decided to allow the ImageJ2
>> Swing UI to begin diverging much more from the ImageJ 1.x design. It
>> will be nice to take more liberties and create something that behaves in
>> way more standardized with other modern applications. See Icy for
>> inspiration (http://icy.bioimageanalysis.org/).
> I agree
>> > and I would wait with the release of the final version until you (we?)
>> > (1) have ironed out all the bugs and inconsistencies,
>> From experience, that goal will never occur. The ImageJ2 design
>> fundamentally cannot be 100% consistent with ImageJ 1.x. Not even ImageJ
>> 1.x is 100% consistent with previous versions of ImageJ 1.x. And no
>> software of this magnitude is bug-free, either.
> Yes. I may have overstated that goal. It would be a more realistic goal to
> have it bug-free enough that it is not super-annoying to the casual user.
> > (2) have good end-user documentation so they know how to do things the
>> > new way,
>> Indeed, we had a whole milestone dedicated to documentation in our
>> original release plan:
> > and (3) have provided users with at least one "killer" feature that
>> > they have longed for.
>> Many such "killer" feature ideas were discussed on the ImageJX list when
>> ImageJ2 was first launched:
>> * https://groups.google.com/d/msg/imagejx/gz7cgytSRuA/emlJLp8o7XYJ
>> * https://groups.google.com/d/msg/imagejx/_yaczl4UWK4/_w6dCnGcJ1QJ
>> * https://groups.google.com/d/msg/imagejx/lD4s32M5als/HZiEA02LhXsJ
>> * https://groups.google.com/d/msg/imagejx/ox2ooizORA4/scMd4P0cRZAJ
>> * https://groups.google.com/d/msg/imagejx/F3gWc_Ndz_U/UTKiut-HuQMJ
>> * https://groups.google.com/d/msg/imagejx/79rryiWqFno/Ne--gMjkSX8J
>> * https://groups.google.com/d/msg/imagejx/pL6ipxHkAk8/26u5MF41YGsJ
>> There are many others in the issue tracker:
>> * Coherent I/O (complete): http://trac.imagej.net/ticket/9
>> * Separation of concerns (complete): http://trac.imagej.net/ticket/10
>> * Better plugin framework (complete): http://trac.imagej.net/ticket/11
>> * Better scripting (complete): http://trac.imagej.net/ticket/12
>> * Better event handling (complete): http://trac.imagej.net/ticket/14
>> * N-dimensional images (complete): http://trac.imagej.net/ticket/17
>> * Very large image data (complete): http://trac.imagej.net/ticket/20
>> * CellProfiler interoperability (complete):
>> * KNIME interoperability (complete): http://trac.imagej.net/ticket/1004
>> * OMERO interoperability: http://trac.imagej.net/ticket/1003
>> * Very large image planes: http://trac.imagej.net/ticket/19
>> * Better undo/redo: http://trac.imagej.net/ticket/13
> Absolutely a killer feature, but I am guessing difficult to implement in a
> plugin-based framework.
> * Coordinate systems: http://trac.imagej.net/ticket/40
>> * Metadata: http://trac.imagej.net/ticket/8
>> I know a lot of those are in some sense architectural, but many of them
>> have huge impact for users, too. Happy to elaborate on any specific
>> aspects of these.
>> > I think there is one thing that can be done in terms of point (3) that
>> > will make many users happy is a "pin" button in each image
>> > window/dialog. If the "pin" is activated then the window/dialog will
>> > be brought to the foreground every time the user brings any other
>> > "pinned" window or the main imagej bar to the foreground. This solves
>> > the perrenial usability problem of ImageJ1 where if you have multiple
>> > images open, you have to hunt for the right image, then hunt for the
>> > brightness/contrast dialog, then hunt for the channels dialog in the
>> > taskbar/dock.
>> Note that there are shortcuts for many of ImageJ's windows, which reduce
>> the need to hunt through the taskbar. E.g.: Shift+C for the B&C window,
>> Shift+Z for channels, Enter for the main ImageJ window.
> Yes, I've been using those shortcuts, but it is still annoying to have to
> find the image, and then press shift-Z and shift-C every time you switch
> back and forth between applications. Most annoyingly, the ImageJ toolbar
> disappears (is there a shortcut to bring it to foreground???) unless you
> set it always on top, which in turn makes it get in the way of other
> applications. What many users do is switch back and forth between ImageJ
> and other apps that allow them to put figures together
> (Illustrator/Photoshop/PowerPoint/Word) or analyze the data
> (Excel/R/Statistica). Right now I spend half my time fishing for the right
>> > I don't think this should be a difficult feature to implement and I
>> > can try to do that, but I will need time to plow through the codebase
>> > and take it all in.
>> This feature could also be implemented for the ImageJ 1.x UI, which
>> would get it into the hands of users more immediately. And you could
>> easily distribute it via an ImageJ update site; see
> Sounds good. I'll give it a stab.
>> > I hope that my comments will help with the development. I am hoping to
>> > contribute to the actual work soon.
>> Development of the core ImageJ system is a substantially different
>> endeavor from feature ideas like better window management, analysis
>> plugins, etc. The question is: which sort of project are you more
>> interested in working on?
> I think I will be interested in features development more than the core,
> but that may change as I get more familiar with the core.
>> > Please let me know your thoughts. Also, I thought that I might want to
>> > send these comments to you personally rather than to the ImageJ-devel
>> > list, since I don't want to step on anyones toes. I imagine there is
>> > more to your decision to ship on June 1st than just software
>> > excellence - things like funding, publications, etc. Please feel free
>> > to forward my letter to the imagej-devel list if you think this will
>> > be constructive.
>> Thanks. As I said before, the imagej-devel list is the best place to
>> discuss these matters. ImageJ is an open source project, and as such is
>> best discussed in public to keep the community informed of the current
>> development directions, invite feedback and constructive criticism from
>> interested parties, etc.
> Got it.
> Paweł Niewiadomski
> e-mail: pawelthebiologist at gmail.com
> website: www.pawelthebiologist.com
> ImageJ-devel mailing list
> ImageJ-devel at imagej.net
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the ImageJ-devel