[ImageJ-devel] ImageJ2 development
Brian Northan
bnorthan at gmail.com
Tue May 13 16:24:55 CDT 2014
Hi List
After seeing Pawel's e-mails, I also wanted to introduce myself and perhaps
become more involved with ImageJ2.
Like Pawel I've been exploring the ImageJ2 ecosystem recently (well for
over a year actually). I'm coming at it from a developer's point of view.
I used to work in the commercial realm until 2011 (Media Cybernetics,
mostly on Autoquant). After leaving that I did some unrelated things for a
couple of years. I did have a non-compete agreement, but that is long
expired so I am thinking about getting back into microscopy.
I'm interested mainly in high performance applications. A good example of
the type of project I'm interested in is GPU Deconvolution. I've seen buzz
about a few GPU deconvolution apps recently (I've been able to explore
Stephan Preibisch's and Bob Pepin's work on Github).
When evaluating a GPU implementation, one would want to understand
state-of-the-art CPU implementations. But how does one even determine what
is state-of-the-art?? How do you get the algorithms and test them in a
consistent fashion? (I recently attended the ISBI Deconvolution Grand
Challenge addressing some aspects of this issue). Further, it may not be
possible to produce the same results as CPU on a GPU implementation (memory
limitations, limited math libraries, etc.). It becomes necessary to run
many tests and comparisons. So to evaluate the benefits and trade-offs of
GPU deconvolution (or newish algorithms like wavelet approaches) it really
becomes important to have access to the older implementations.
For any new algorithm development, 90% of the problem (in my experience) is
understanding and evaluating what has been done before. With open source
the great thing is that understanding the past is no longer just a
literature search; it also involves getting the algorithms and running
them.
So I see "uber-wrapper" projects (a means to run and test multiple existing
implementations under the same framework) being very useful, and I've been
developing one for deconvolution with ImageJ2 (
https://github.com/bnorthan/projects). Right now it wraps YacuDecu GPU
deconvolution [1], PSF generation in COSMOS [2] and an algorithm from ITK
deconvolution. I've also ported a Total Variation implementation from
IOCBIOS [3] and non periodic boundary handling from EPFL [4] to imglib2
based implementations (similar design to imglib2 FFTConvolution [5]).
These few algorithms barely scratch the surface of what is out there.
As I've gotten multiple algorithms and components wrapped or ported as
ImageJ2 commands (or more recently a few ops) it has become simple to run
test scripts and to combine components in interesting and useful ways.
I'm not entirely sure how far I will be able to develop this. Since I am
using some native components, it could become difficult to maintain and
distribute for multiple operating systems. But at the very least I hope to
polish it and get it on a personal update site. (Maybe isolating the java
only components, possibly snagging the PSF code from Icy[6]).
The Github link will be out of date soon as I need to clean and refactor
things a bit. As I do this there may be an opportunity to contribute to
the Imagej-ops project or Imglib2. One thing I need to do is take
inventory of all the little math functions I've written. Where I've
re-invented the wheel I need to re-use the proper imagej-ops or imglib2
function instead. If some math functions I have written are not in
imagej-ops/imgib I could take a look at implementing them.
Regards,
Brian
[1] https://github.com/bobpepin/YacuDecu
[2] http://cirl.memphis.edu/cosmos.php
[3] http://code.google.com/p/iocbio/wiki/IOCBioMicroscope
[4]
http://bigwww.epfl.ch/deconvolution/challenge/index.html?p=documentation/theory/richardsonlucy
[5]
https://github.com/imglib/imglib/blob/master/algorithms/gpl/src/main/java/net/imglib2/algorithm/fft2/FFTConvolution.java
[6]
http://icy.bioimageanalysis.org/plugin/Widefield_Fluorescence_Microscope_PSF
On Tue, May 13, 2014 at 3:41 AM, Pawel Niewiadomski <
pawelthebiologist at gmail.com> wrote:
> Hi Curtis,
>
> Thanks for your comprehensive reply.
>
>
> Hi Pawel,
>>
>> > I decided to start work on the project by playing around with the
>> > ImageJ2 interface in some routine tasks I usually do in ImageJ. After
>> > having done some extensive testing, I just wanted to give you some,
>> > hopefully constructive, input. I don't want to sound too negative, but
>> > I really think that June 1st is a bit early to be releasing a
>> > production version of ImageJ2.
>>
>> With respect to the ImageJ2 Swing user interface and commands, you are
>> absolutely right. However, the initial 2.0.0 release of ImageJ2 is going
>> to continue using the classic ImageJ 1.x user interface by default. The
>> new UI will still be included (Help > Switch to Modern Mode) but will
>> still be very much in beta.
>>
>>
> I was actually under the impression that ImageJ2 IS the modern interface
> and that it would be the default for the 2.0.0 release. I must have not
> found the relevant blog/discussion posts. My bad.
>
>
> See this blog post for details:
>> http://developer.imagej.net/2014/04/01/imagej-200-stable-
>> release-coming-spring
>>
>> ImageJ 2.0.0 will be fundamentally the same as ImageJ1, but:
>>
>> * Will be bundled with the Updater component which supports ImageJ
>> update sites and automatic update checking.
>>
>>
> OK
>
>
> * Will be bundled with the native ImageJ Launcher with quite a few nice
>> command line features.
>>
>
> The imagej-win32.exe doesn't work in the build from about a week ago on my
> machine (Just the basic command that launches the GUI). I'll submit a bug
> report.
>
>
>
>> * Will be bundled with the ImageJ Script Editor including support for
>> several scripting languages.
>>
>>
> I am guessing it's the same as the FiJi script editor? In that case the
> legacy UI should have a menu option File->New->Script, like FiJi does. In
> addition, the plugins->new->plugin throws some sort of exception on first
> use, again I'll submit a bug report.
>
>
> * Will support parameterized ImageJ modules, including commands and
>> scripts, so that existing plugins can begin incremental migration toward
>> this new approach, which is more headless friendly and more
>> interoperable with tools such as CellProfiler, KNIME and OMERO.
>>
>>
> Sounds good - I need to look into it more closely.
>
>
> * Will come with an option to use the SCIFIO library when opening image
>> files (e.g., using File > Open). This will fix ImageJ's TIFF support to
>> be more robust, and add extensible support for additional file formats
>> without needing to hack the HandleExtraFileTypes source.
>>
>>
> OK.
>
>
> * Will ship with all the new ImageJ2 APIs, but all these components will
>> still be in beta. We will bring each component out of beta after it has
>> been thoroughly vetted over time. These components include:
>> - imagej-common: The ImgLib2-based image data model and core
>> - imagej-ops: A framework for reusable algorithms; see
>> http://developer.imagej.net/2014/04/04/announcing-imagej-ops
>> - imagej-ui-swing: The "pure ImageJ2" Swing user interface
>> - imagej-plugins-*, scijava-plugins-*: Core plugins for ImageJ2
>> including many commands
>>
>>
> OK
>
>
> Note that the Fiji distribution of ImageJ has been shipping of all of
>> these components, and operating in this way, for years now, and is a
>> well vetted system. But it is time for these components to be officially
>> available as part of ImageJ's core, rather than only from a specific
>> life-sciences-focused ImageJ distribution.
>>
>
> Sounds good. I've been using FiJi for quite some time now, so I didn't see
> the "newness" of the features.
>
>
>
>> > Please don't take it the wrong way - I am in the process of analysing
>> > the codebase and I think it is a real software engineering feat.
>>
>> No offense taken at all; again, it is absolutely true that the ImageJ2
>> UI needs more time in the oven. But meanwhile, it has already been over
>> four years since we launched the ImageJ2 project, and there are several
>> very mature components that need to get into the hands of users: the
>> Updater, the Launcher, the Script Editor, parameterized modules, and
>> everything else that is now part of the "SciJava Common" component
>> (https://github.com/scijava/scijava-common).
>>
>> Because ImageJ2 consists of several pieces at various stages of
>> development, it needs to migrate out of beta piece by piece.
>>
>> > It shows that there has been really serious thought put into
>> > architerctural design. However, at this point, and I am saying that as
>> > a daily ImageJ user, the architectural brilliance is not showing on
>> > the surface.
>>
>> I am glad you like the design. And I agree that much more needs to be
>> done in terms of leveraging that design for the benefit of end users.
>> This is an area where your contributions could be really beneficial.
>>
>> > I am going to submit some bug reports in a few days, but basically the
>> > interface is highly unpredictable and in many ways incompatible with
>> > ImageJ1.
>>
>> Indeed. Please note that there are many bugs about such problems already
>> logged in the ImageJ Trac: http://trac.imagej.net/. We are actively in
>> the process of migrating away from the Trac system though, with
>> individual tickets being moved to GitHub Issues of the most relevant
>> repository. Unfortunately, since we are in the middle of that migration,
>> it may be difficult to verify whether an issue has already been filed
>> for any particular concern. When in doubt, file away and we can close
>> any duplicate issues accordingly.
>>
>> > Let me just point to a few basic things before I submit detailed bug
>> > reports:
>> >
>> > 1. The brightness/contrast dialog sometimes sticks around when you
>> > open a new image and close the old one - you end up with multiple
>> > brightness/contrast dialogs and a single image. Moreover, I've had the
>> > dialog stay open even after I closed the application.
>>
>> The IJ2 version of B&C has been the subject of frequent debate. In
>> short, it needs a lot more work. See http://trac.imagej.net/ticket/1100
>> and all its blocking tickets (those listed in "Blocked by").
>>
>>
> OK
>
>
> > 2. The color picker behavior is hectic and I couldn't figure out how
>> > and why it randomly changes color. Especially in 16-bit per channel
>> > images it is totally unpredictable.
>> >
>> > 3. With 48-bit 16-bit per channel composite images the drawing command
>> > with the white color selected basically draws random grey colors.
>>
>> IJ2's current approach to foreground and background "colors" differs
>> from IJ1. And there are likely bugs, too.
>> * http://trac.imagej.net/ticket/965
>> * http://trac.imagej.net/ticket/1292
>>
>>
> OK
>
>
> > 4. The interface is extremely slow to the point of being unusable for
>> > things like looking through time-course stacks or stacks of
>> > medium-sized multicolor images.
>>
>> That is not a problem we have noticed, unless image planes become very
>> large. How large are your image planes? >2Kx2K?
>>
>>
> These are not large images (1kx1k/48-bit/10 z-planes), but mind you, I am
> not using the latest hardware. However, I think many of ImageJ users will
> not be on modern hardware, and my computer can run Photoshop CS4 with ease,
> which is an indication that it is not a piece of junk, either.
>
>
> > 5. Shape selection keeps old selections after you make a new one,
>> > which is inconsistent with ImageJ1 behavior and quite maddening for
>> > someone who is used to it.
>>
>> Yes. We will probably need an option for it, because for many new users,
>> it is maddening to have one ROI disappear when creating another. But the
>> main reason IJ2's UI works that way right now is technical: it currently
>> uses the JHotDraw library which works that way by default.
>>
>>
> OK
>
>
> > 6. Missing magic wand and text tool functionality.
>>
>> https://github.com/imagej/imagej-plugins-tools/issues/8
>> https://github.com/imagej/imagej-plugins-tools/issues/9
>>
>>
> OK
>
>
> > 7. Missing custom toolbars.
>>
>> Known, but no explicit issue for it yet. Low priority, given all the
>> other things the Swing UI needs first. Note that IJ2 is not
>> intrinsically limited to 8 tools like ImageJ1 is, so it is less urgent
>> to support customization. All available tools will be present in the bar
>> by default.
>>
>>
> OK. It would be nice if the toolbar were easily customizable by the user
> the way the old style custom toolbars were. Also, having custom
> floating/docking toolbars would be nice. I am thinking
> photoshop/illustrator here, or at least Office <2003. I don't think you
> necessarily need to implement an RCP-style interface to have those.
>
>
> > 8. In multicolor images there seems to be no way of adjusting
>> > brightness/contrast of each channel individually.
>>
>> Hmm, you're right. This might be a relatively new bug.
>>
>>
> That's an absolutely essential feature for any microscopy work. I'll
> submit the bug report.
>
>
> > 9. LIF format import doesn't work - it doesn't present the usual
>> > BioFormats dialog and instead just imports the first image in the
>> > series with some random channel separation.
>>
>> LIF format does not work with vanilla ImageJ1 either. It is handled by
>> the Bio-Formats plugin. We do not ship Bio-Formats with ImageJ2 because:
>> A) ImageJ2 is BSD-2 licensed, and the Bio-Formats proprietary file
>> format readers have an incompatible GPL license; and B) ImageJ2 is
>> supposed to be a "discipline-agnostic" piece of software, while
>> Bio-Formats is focused on life sciences file formats.
>>
>>
> OK. Understood.
>
>
> However, LIF should work if you download Fiji, choose Help > Switch to
>> Modern Mode, and then File > Open your LIF file. This is thanks to the
>> SCIFIO Bio-Formats compatibility component
>> (https://github.com/scifio/scifio-bf-compat) which is bundled with Fiji.
>>
>>
> It does work, but not with File->Open. It doesn't present the right dialog
> for file import. You can go to File->Import->Bio-Formats. When you go to
> file-open (even in modern mode in FiJi), it just imports the first image of
> the bundle with each channel on a separate plane instead of creating a
> composite. It doesn't offer you the choices the way Bio-Formats does.
>
>
> Alternately, you can install Bio-Formats by turning on the Fiji and/or
>> Bio-Formats update sites shown in ImageJ2's Help > Update "Manage Update
>> Sites" dialog.
>>
>> > This is just a sample, but there is a bunch more. My prediction is
>> > that the users will not migrate to the new version if you ship it as
>> > is. They frankly need a reason to migrate and ImageJ2 is not offering
>> > them any.
>>
>> Agreed; "regular users" should not be switching to the new interface
>> yet. There are too many bugs and not enough advantages.
>>
>> That is why decided to keep the ImageJ2 releases using the 1.x
>> interface, for the time being. This keeps 100% backwards compatibility
>> while also providing many advantages:
>> * the ImageJ Updater
>> * the Script Editor
>> * user-facing improvements made possible by our ImageJ 1.x patching
>> mechanism, such as File > Open using the SCIFIO library to read TIFFs
>> and other formats more robustly
>> * new developer-facing APIs (esp.,parameterized modules)
>>
>
> Sounds good.
>
>
>
>> There are downsides though:
>> * IJ1 UI is limited to XYZCT (though we may later patch in support
>> for additional dimensions)
>> * IJ1 UI cannot handle tiled huge image planes (a feature planned for
>> the IJ2 UI)
>> * Lack of separation of concerns; IJ1 UI is fundamentally tied to the
>> IJ1 data model
>>
>
> I think that and the fact that FiJi already has many of these features is
> why I was convinced that ImageJ2 is the modern interface. The data model
> limitation is a serious one, because people cannot just migrate their
> plugins in one fell swoop. Rather, they have to first play with the new
> APIs, then start working on the data model conversion, etc.
>
>
> * Further reading: http://dev.imagej.net/rationale,
>> http://dev.imagej.net/proposal
>>
>> > Right now ImageJ1 is a mature platform with few bugs and a plethora of
>> > mostly seamlessly working plugins.
>>
>> In many ways. But due to ImageJ1's protracted incremental development,
>> its API how grown organically far beyond its original design goals, so
>> it now contains a plethora of limitations and edge cases.
>>
>> > The biggest gripe most users have with ImageJ1 is its antiquated UI
>>
>> With that insight in mind, we recently decided to allow the ImageJ2
>> Swing UI to begin diverging much more from the ImageJ 1.x design. It
>> will be nice to take more liberties and create something that behaves in
>> way more standardized with other modern applications. See Icy for
>> inspiration (http://icy.bioimageanalysis.org/).
>>
>
> I agree
>
>
>
>> > and I would wait with the release of the final version until you (we?)
>> > (1) have ironed out all the bugs and inconsistencies,
>>
>> From experience, that goal will never occur. The ImageJ2 design
>> fundamentally cannot be 100% consistent with ImageJ 1.x. Not even ImageJ
>> 1.x is 100% consistent with previous versions of ImageJ 1.x. And no
>> software of this magnitude is bug-free, either.
>>
>
> Yes. I may have overstated that goal. It would be a more realistic goal to
> have it bug-free enough that it is not super-annoying to the casual user.
>
>
> > (2) have good end-user documentation so they know how to do things the
>> > new way,
>>
>> Indeed, we had a whole milestone dedicated to documentation in our
>> original release plan:
>> http://trac.imagej.net/milestone/imagej2-b11-docs
>>
>>
> OK.
>
>
> > and (3) have provided users with at least one "killer" feature that
>> > they have longed for.
>>
>> Many such "killer" feature ideas were discussed on the ImageJX list when
>> ImageJ2 was first launched:
>> * https://groups.google.com/d/msg/imagejx/gz7cgytSRuA/emlJLp8o7XYJ
>> * https://groups.google.com/d/msg/imagejx/_yaczl4UWK4/_w6dCnGcJ1QJ
>> * https://groups.google.com/d/msg/imagejx/lD4s32M5als/HZiEA02LhXsJ
>> * https://groups.google.com/d/msg/imagejx/ox2ooizORA4/scMd4P0cRZAJ
>> * https://groups.google.com/d/msg/imagejx/F3gWc_Ndz_U/UTKiut-HuQMJ
>> * https://groups.google.com/d/msg/imagejx/79rryiWqFno/Ne--gMjkSX8J
>> * https://groups.google.com/d/msg/imagejx/pL6ipxHkAk8/26u5MF41YGsJ
>>
>> There are many others in the issue tracker:
>> * Coherent I/O (complete): http://trac.imagej.net/ticket/9
>> * Separation of concerns (complete): http://trac.imagej.net/ticket/10
>> * Better plugin framework (complete): http://trac.imagej.net/ticket/11
>> * Better scripting (complete): http://trac.imagej.net/ticket/12
>> * Better event handling (complete): http://trac.imagej.net/ticket/14
>> * N-dimensional images (complete): http://trac.imagej.net/ticket/17
>> * Very large image data (complete): http://trac.imagej.net/ticket/20
>> * CellProfiler interoperability (complete):
>> http://trac.imagej.net/ticket/1002
>> * KNIME interoperability (complete): http://trac.imagej.net/ticket/1004
>> * OMERO interoperability: http://trac.imagej.net/ticket/1003
>> * Very large image planes: http://trac.imagej.net/ticket/19
>> * Better undo/redo: http://trac.imagej.net/ticket/13
>>
>
> Absolutely a killer feature, but I am guessing difficult to implement in a
> plugin-based framework.
>
>
> * Coordinate systems: http://trac.imagej.net/ticket/40
>> * Metadata: http://trac.imagej.net/ticket/8
>>
>> I know a lot of those are in some sense architectural, but many of them
>> have huge impact for users, too. Happy to elaborate on any specific
>> aspects of these.
>>
>> > I think there is one thing that can be done in terms of point (3) that
>> > will make many users happy is a "pin" button in each image
>> > window/dialog. If the "pin" is activated then the window/dialog will
>> > be brought to the foreground every time the user brings any other
>> > "pinned" window or the main imagej bar to the foreground. This solves
>> > the perrenial usability problem of ImageJ1 where if you have multiple
>> > images open, you have to hunt for the right image, then hunt for the
>> > brightness/contrast dialog, then hunt for the channels dialog in the
>> > taskbar/dock.
>>
>> Note that there are shortcuts for many of ImageJ's windows, which reduce
>> the need to hunt through the taskbar. E.g.: Shift+C for the B&C window,
>> Shift+Z for channels, Enter for the main ImageJ window.
>>
>
> Yes, I've been using those shortcuts, but it is still annoying to have to
> find the image, and then press shift-Z and shift-C every time you switch
> back and forth between applications. Most annoyingly, the ImageJ toolbar
> disappears (is there a shortcut to bring it to foreground???) unless you
> set it always on top, which in turn makes it get in the way of other
> applications. What many users do is switch back and forth between ImageJ
> and other apps that allow them to put figures together
> (Illustrator/Photoshop/PowerPoint/Word) or analyze the data
> (Excel/R/Statistica). Right now I spend half my time fishing for the right
> image/dialog/button.
>
>
>
>> > I don't think this should be a difficult feature to implement and I
>> > can try to do that, but I will need time to plow through the codebase
>> > and take it all in.
>>
>> This feature could also be implemented for the ImageJ 1.x UI, which
>> would get it into the hands of users more immediately. And you could
>> easily distribute it via an ImageJ update site; see
>> http://wiki.imagej.net/Update_Sites.
>>
>
> Sounds good. I'll give it a stab.
>
>
>
>> > I hope that my comments will help with the development. I am hoping to
>> > contribute to the actual work soon.
>>
>> Development of the core ImageJ system is a substantially different
>> endeavor from feature ideas like better window management, analysis
>> plugins, etc. The question is: which sort of project are you more
>> interested in working on?
>>
>
> I think I will be interested in features development more than the core,
> but that may change as I get more familiar with the core.
>
>
>
>> > Please let me know your thoughts. Also, I thought that I might want to
>> > send these comments to you personally rather than to the ImageJ-devel
>> > list, since I don't want to step on anyones toes. I imagine there is
>> > more to your decision to ship on June 1st than just software
>> > excellence - things like funding, publications, etc. Please feel free
>> > to forward my letter to the imagej-devel list if you think this will
>> > be constructive.
>>
>> Thanks. As I said before, the imagej-devel list is the best place to
>> discuss these matters. ImageJ is an open source project, and as such is
>> best discussed in public to keep the community informed of the current
>> development directions, invite feedback and constructive criticism from
>> interested parties, etc.
>>
>
> Got it.
>
>
>> Regards,
>> Curtis
>>
>>
> Regards,
>
> Pawel
>
> --
> Paweł Niewiadomski
> e-mail: pawelthebiologist at gmail.com
> website: www.pawelthebiologist.com
>
> _______________________________________________
> ImageJ-devel mailing list
> ImageJ-devel at imagej.net
> http://imagej.net/mailman/listinfo/imagej-devel
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://imagej.net/pipermail/imagej-devel/attachments/20140513/8d68bd9b/attachment-0001.html>
More information about the ImageJ-devel
mailing list