[ImageJ-devel] ImageJ2 development

Curtis Rueden ctrueden at wisc.edu
Tue May 13 15:16:03 CDT 2014


Hi Pawel,

> I was actually under the impression that ImageJ2 IS the modern
> interface and that it would be the default for the 2.0.0 release.

The vision of the ImageJ2 project has substantially evolved since its 2009
inception.

The main takeaway is that "ImageJ2" is actually several different
components. Documenting these components clearly but succinctly is
certainly a challenge.

> The imagej-win32.exe doesn't work in the build from about a week ago
> on my machine (Just the basic command that launches the GUI). I'll
> submit a bug report.

OK, thanks. I suggest these resources for how to write a good bug report:
* http://stackoverflow.com/help/how-to-ask
* http://whathaveyoutried.com/

> I am guessing it's the same as the FiJi script editor? In that case
> the legacy UI should have a menu option File->New->Script, like FiJi
> does. In addition, the plugins->new->plugin throws some sort of
> exception on first use, again I'll submit a bug report.

The Fiji Script Editor came first. Then it was ported to the ImageJ2 Script
Editor, which at the moment is a separate codebase. We are on the cusp of
updating Fiji to use the IJ2 Script Editor, and retiring the old Fiji
codebase. But we cannot do it until the IJ2 Script Editor has all the
features that the current Fiji Script Editor has. It is a goal for the June
release.

> > * Will support parameterized ImageJ modules
>
> Sounds good - I need to look into it more closely.

https://github.com/imagej/imagej-tutorials/tree/master/simple-commands/src/main/java

> I've been using FiJi for quite some time now, so I didn't see the
> "newness" of the features.

Fiji (small J, BTW) incubated several of the features now being slated for
ImageJ 2.0.0:
* http://developer.imagej.net/2012/11/19/road-fiji-2
* http://developer.imagej.net/2014/03/27/imagej2-and-fiji-recent-milestones

> It does work, but not with File->Open. It doesn't present the right
> dialog for file import. You can go to File->Import->Bio-Formats. When
> you go to file-open (even in modern mode in FiJi), it just imports the
> first image of the bundle with each channel on a separate plane
> instead of creating a composite. It doesn't offer you the choices the
> way Bio-Formats does.

The "big honkin' dialog" of Bio-Formats is indeed specific to Bio-Formats.
File > Open in ImageJ2 will not use the Bio-Formats dialogs. After all,
that dialog was built using an IJ1 GenericDialog, which ImageJ2 eschews.
Eventually we will have some "advanced options" which will be very similar
to (some of) those options of the BF dialog, but we have not implemented
that yet.

> These are not large images (1kx1k/48-bit/10 z-planes), but mind you, I
> am not using the latest hardware. However, I think many of ImageJ
> users will not be on modern hardware, and my computer can run
> Photoshop CS4 with ease, which is an indication that it is not a piece
> of junk, either.

Browsing images of that size is supposed to be smooth. Maybe it is a CPU
issue as you say. In the long term, we are planning to overhaul IJ2's
display mechanism which should substantially help improve performance.

> OK. It would be nice if the toolbar were easily customizable by the
> user the way the old style custom toolbars were.

Customization of toolbars is planned but will come later.

> Also, having custom floating/docking toolbars would be nice. I am
> thinking photoshop/illustrator here, or at least Office <2003. I don't
> think you necessarily need to implement an RCP-style interface to have
> those.

The IJ2 toolbar is already dockable, so can be made to float. Try grabbing
the grippers on the left and dragging.

And we will definitely not need to use an RCP framework for the
customization.

> I think that and the fact that FiJi already has many of these features
> is why I was convinced that ImageJ2 is the modern interface. The data
> model limitation is a serious one, because people cannot just migrate
> their plugins in one fell swoop. Rather, they have to first play with
> the new APIs, then start working on the data model conversion, etc.

The new data model will be available in ImageJ 2.0.0, but still in beta --
meaning anyone who converts their plugins at this point might need to tweak
their code again in the future as the API continues to evolve before final
release.

But the parameterized module framework is out of beta and we will do our
best to ensure backwards compatibility. So the recommended next step for
migration of plugins would be to first switch the plugin to an ImageJ2
command using parameterized inputs and outputs, but with ImageJ1 data
structures (e.g., ImagePlus). Then switch to ImageJ2 data structures
later... unless the plugin really needs an IJ2-only feature immediately.

> Yes. I may have overstated that goal. It would be a more realistic
> goal to have it bug-free enough that it is not super-annoying to the
> casual user.

Sure. That is a gray area though, since it is subjective depending on the
user. But I agree with the sentiment.

> > * Better undo/redo: http://trac.imagej.net/ticket/13
>
> Absolutely a killer feature, but I am guessing difficult to implement
> in a plugin-based framework.

Difficult but doable. We just never quite had time to work on it
intensively, yet. The ticket explains what needs to be done.

> Most annoyingly, the ImageJ toolbar disappears (is there a shortcut to
> bring it to foreground???)

Yes, like I said: Enter.

> > This feature could also be implemented for the ImageJ 1.x UI,
>
> Sounds good. I'll give it a stab.

Awesome, definitely keep us posted on your progress!

> I think I will be interested in features development more than the
> core, but that may change as I get more familiar with the core.

That is understandable, and certainly very reasonable given ImageJ's
extensible plugin mechanism. The core is "good enough" for a great variety
of things. I think it takes a certain architectural obsession to want to
work on ImageJ2 itself, which is one reason there are so few external
contributors.

Then, the best way for you to improve ImageJ, and give back to the ImageJ
community, will be to create your own personal update site and start adding
useful plugins to it, which work in the ImageJ 1.x UI. That said, you could
still be an early adopter of the parameterized ImageJ2 command mechanism
and other ImageJ2-only features, which would be really nice for a number of
reasons.

Regards,
Curtis


On Tue, May 13, 2014 at 2:41 AM, Pawel Niewiadomski <
pawelthebiologist at gmail.com> wrote:

> Hi Curtis,
>
> Thanks for your comprehensive reply.
>
>
>  Hi Pawel,
>>
>>  > I decided to start work on the project by playing around with the
>>  > ImageJ2 interface in some routine tasks I usually do in ImageJ. After
>>  > having done some extensive testing, I just wanted to give you some,
>>  > hopefully constructive, input. I don't want to sound too negative, but
>>  > I really think that June 1st is a bit early to be releasing a
>>  > production version of ImageJ2.
>>
>> With respect to the ImageJ2 Swing user interface and commands, you are
>> absolutely right. However, the initial 2.0.0 release of ImageJ2 is going
>> to continue using the classic ImageJ 1.x user interface by default. The
>> new UI will still be included (Help > Switch to Modern Mode) but will
>> still be very much in beta.
>>
>>
> I was actually under the impression that ImageJ2 IS the modern interface
> and that it would be the default for the 2.0.0 release. I must have not
> found the relevant blog/discussion posts. My bad.
>
>
>  See this blog post for details:
>> http://developer.imagej.net/2014/04/01/imagej-200-stable-
>> release-coming-spring
>>
>> ImageJ 2.0.0 will be fundamentally the same as ImageJ1, but:
>>
>> * Will be bundled with the Updater component which supports ImageJ
>> update sites and automatic update checking.
>>
>>
> OK
>
>
>  * Will be bundled with the native ImageJ Launcher with quite a few nice
>> command line features.
>>
>
> The imagej-win32.exe doesn't work in the build from about a week ago on my
> machine (Just the basic command that launches the GUI). I'll submit a bug
> report.
>
>
>
>> * Will be bundled with the ImageJ Script Editor including support for
>> several scripting languages.
>>
>>
> I am guessing it's the same as the FiJi script editor? In that case the
> legacy UI should have a menu option File->New->Script, like FiJi does. In
> addition, the plugins->new->plugin throws some sort of exception on first
> use, again I'll submit a bug report.
>
>
>  * Will support parameterized ImageJ modules, including commands and
>> scripts, so that existing plugins can begin incremental migration toward
>> this new approach, which is more headless friendly and more
>> interoperable with tools such as CellProfiler, KNIME and OMERO.
>>
>>
> Sounds good - I need to look into it more closely.
>
>
>  * Will come with an option to use the SCIFIO library when opening image
>> files (e.g., using File > Open). This will fix ImageJ's TIFF support to
>> be more robust, and add extensible support for additional file formats
>> without needing to hack the HandleExtraFileTypes source.
>>
>>
> OK.
>
>
>  * Will ship with all the new ImageJ2 APIs, but all these components will
>> still be in beta. We will bring each component out of beta after it has
>> been thoroughly vetted over time. These components include:
>>    - imagej-common: The ImgLib2-based image data model and core
>>    - imagej-ops: A framework for reusable algorithms; see
>> http://developer.imagej.net/2014/04/04/announcing-imagej-ops
>>    - imagej-ui-swing: The "pure ImageJ2" Swing user interface
>>    - imagej-plugins-*, scijava-plugins-*: Core plugins for ImageJ2
>> including many commands
>>
>>
> OK
>
>
>  Note that the Fiji distribution of ImageJ has been shipping of all of
>> these components, and operating in this way, for years now, and is a
>> well vetted system. But it is time for these components to be officially
>> available as part of ImageJ's core, rather than only from a specific
>> life-sciences-focused ImageJ distribution.
>>
>
> Sounds good. I've been using FiJi for quite some time now, so I didn't see
> the "newness" of the features.
>
>
>
>>  > Please don't take it the wrong way - I am in the process of analysing
>>  > the codebase and I think it is a real software engineering feat.
>>
>> No offense taken at all; again, it is absolutely true that the ImageJ2
>> UI needs more time in the oven. But meanwhile, it has already been over
>> four years since we launched the ImageJ2 project, and there are several
>> very mature components that need to get into the hands of users: the
>> Updater, the Launcher, the Script Editor, parameterized modules, and
>> everything else that is now part of the "SciJava Common" component
>> (https://github.com/scijava/scijava-common).
>>
>> Because ImageJ2 consists of several pieces at various stages of
>> development, it needs to migrate out of beta piece by piece.
>>
>>  > It shows that there has been really serious thought put into
>>  > architerctural design. However, at this point, and I am saying that as
>>  > a daily ImageJ user, the architectural brilliance is not showing on
>>  > the surface.
>>
>> I am glad you like the design. And I agree that much more needs to be
>> done in terms of leveraging that design for the benefit of end users.
>> This is an area where your contributions could be really beneficial.
>>
>>  > I am going to submit some bug reports in a few days, but basically the
>>  > interface is highly unpredictable and in many ways incompatible with
>>  > ImageJ1.
>>
>> Indeed. Please note that there are many bugs about such problems already
>> logged in the ImageJ Trac: http://trac.imagej.net/. We are actively in
>> the process of migrating away from the Trac system though, with
>> individual tickets being moved to GitHub Issues of the most relevant
>> repository. Unfortunately, since we are in the middle of that migration,
>> it may be difficult to verify whether an issue has already been filed
>> for any particular concern. When in doubt, file away and we can close
>> any duplicate issues accordingly.
>>
>>  > Let me just point to a few basic things before I submit detailed bug
>>  > reports:
>>  >
>>  > 1. The brightness/contrast dialog sometimes sticks around when you
>>  > open a new image and close the old one - you end up with multiple
>>  > brightness/contrast dialogs and a single image. Moreover, I've had the
>>  > dialog stay open even after I closed the application.
>>
>> The IJ2 version of B&C has been the subject of frequent debate. In
>> short, it needs a lot more work. See http://trac.imagej.net/ticket/1100
>> and all its blocking tickets (those listed in "Blocked by").
>>
>>
> OK
>
>
>   > 2. The color picker behavior is hectic and I couldn't figure out how
>>  > and why it randomly changes color. Especially in 16-bit per channel
>>  > images it is totally unpredictable.
>>  >
>>  > 3. With 48-bit 16-bit per channel composite images the drawing command
>>  > with the white color selected basically draws random grey colors.
>>
>> IJ2's current approach to foreground and background "colors" differs
>> from IJ1. And there are likely bugs, too.
>> * http://trac.imagej.net/ticket/965
>> * http://trac.imagej.net/ticket/1292
>>
>>
> OK
>
>
>   > 4. The interface is extremely slow to the point of being unusable for
>>  > things like looking through time-course stacks or stacks of
>>  > medium-sized multicolor images.
>>
>> That is not a problem we have noticed, unless image planes become very
>> large. How large are your image planes? >2Kx2K?
>>
>>
> These are not large images (1kx1k/48-bit/10 z-planes), but mind you, I am
> not using the latest hardware. However, I think many of ImageJ users will
> not be on modern hardware, and my computer can run Photoshop CS4 with ease,
> which is an indication that it is not a piece of junk, either.
>
>
>   > 5. Shape selection keeps old selections after you make a new one,
>>  > which is inconsistent with ImageJ1 behavior and quite maddening for
>>  > someone who is used to it.
>>
>> Yes. We will probably need an option for it, because for many new users,
>> it is maddening to have one ROI disappear when creating another. But the
>> main reason IJ2's UI works that way right now is technical: it currently
>> uses the JHotDraw library which works that way by default.
>>
>>
> OK
>
>
>   > 6. Missing magic wand and text tool functionality.
>>
>> https://github.com/imagej/imagej-plugins-tools/issues/8
>> https://github.com/imagej/imagej-plugins-tools/issues/9
>>
>>
> OK
>
>
>   > 7. Missing custom toolbars.
>>
>> Known, but no explicit issue for it yet. Low priority, given all the
>> other things the Swing UI needs first. Note that IJ2 is not
>> intrinsically limited to 8 tools like ImageJ1 is, so it is less urgent
>> to support customization. All available tools will be present in the bar
>> by default.
>>
>>
> OK. It would be nice if the toolbar were easily customizable by the user
> the way the old style custom toolbars were. Also, having custom
> floating/docking toolbars would be nice. I am thinking
> photoshop/illustrator here, or at least Office <2003. I don't think you
> necessarily need to implement an RCP-style interface to have those.
>
>
>   > 8. In multicolor images there seems to be no way of adjusting
>>  > brightness/contrast of each channel individually.
>>
>> Hmm, you're right. This might be a relatively new bug.
>>
>>
> That's an absolutely essential feature for any microscopy work. I'll
> submit the bug report.
>
>
>   > 9. LIF format import doesn't work - it doesn't present the usual
>>  > BioFormats dialog and instead just imports the first image in the
>>  > series with some random channel separation.
>>
>> LIF format does not work with vanilla ImageJ1 either. It is handled by
>> the Bio-Formats plugin. We do not ship Bio-Formats with ImageJ2 because:
>> A) ImageJ2 is BSD-2 licensed, and the Bio-Formats proprietary file
>> format readers have an incompatible GPL license; and B) ImageJ2 is
>> supposed to be a "discipline-agnostic" piece of software, while
>> Bio-Formats is focused on life sciences file formats.
>>
>>
> OK. Understood.
>
>
>  However, LIF should work if you download Fiji, choose Help > Switch to
>> Modern Mode, and then File > Open your LIF file. This is thanks to the
>> SCIFIO Bio-Formats compatibility component
>> (https://github.com/scifio/scifio-bf-compat) which is bundled with Fiji.
>>
>>
> It does work, but not with File->Open. It doesn't present the right dialog
> for file import. You can go to File->Import->Bio-Formats. When you go to
> file-open (even in modern mode in FiJi), it just imports the first image of
> the bundle with each channel on a separate plane instead of creating a
> composite. It doesn't offer you the choices the way Bio-Formats does.
>
>
>  Alternately, you can install Bio-Formats by turning on the Fiji and/or
>> Bio-Formats update sites shown in ImageJ2's Help > Update "Manage Update
>> Sites" dialog.
>>
>>  > This is just a sample, but there is a bunch more. My prediction is
>>  > that the users will not migrate to the new version if you ship it as
>>  > is. They frankly need a reason to migrate and ImageJ2 is not offering
>>  > them any.
>>
>> Agreed; "regular users" should not be switching to the new interface
>> yet. There are too many bugs and not enough advantages.
>>
>> That is why decided to keep the ImageJ2 releases using the 1.x
>> interface, for the time being. This keeps 100% backwards compatibility
>> while also providing many advantages:
>>    * the ImageJ Updater
>>    * the Script Editor
>>    * user-facing improvements made possible by our ImageJ 1.x patching
>> mechanism, such as File > Open using the SCIFIO library to read TIFFs
>> and other formats more robustly
>>    * new developer-facing APIs (esp.,parameterized modules)
>>
>
> Sounds good.
>
>
>
>> There are downsides though:
>>    * IJ1 UI is limited to XYZCT (though we may later patch in support
>> for additional dimensions)
>>    * IJ1 UI cannot handle tiled huge image planes (a feature planned for
>> the IJ2 UI)
>>    * Lack of separation of concerns; IJ1 UI is fundamentally tied to the
>> IJ1 data model
>>
>
> I think that and the fact that FiJi already has many of these features is
> why I was convinced that ImageJ2 is the modern interface. The data model
> limitation is a serious one, because people cannot just migrate their
> plugins in one fell swoop. Rather, they have to first play with the new
> APIs, then start working on the data model conversion, etc.
>
>
>     * Further reading: http://dev.imagej.net/rationale,
>> http://dev.imagej.net/proposal
>>
>>  > Right now ImageJ1 is a mature platform with few bugs and a plethora of
>>  > mostly seamlessly working plugins.
>>
>> In many ways. But due to ImageJ1's protracted incremental development,
>> its API how grown organically far beyond its original design goals, so
>> it now contains a plethora of limitations and edge cases.
>>
>>  > The biggest gripe most users have with ImageJ1 is its antiquated UI
>>
>> With that insight in mind, we recently decided to allow the ImageJ2
>> Swing UI to begin diverging much more from the ImageJ 1.x design. It
>> will be nice to take more liberties and create something that behaves in
>> way more standardized with other modern applications. See Icy for
>> inspiration (http://icy.bioimageanalysis.org/).
>>
>
> I agree
>
>
>
>>  > and I would wait with the release of the final version until you (we?)
>>  > (1) have ironed out all the bugs and inconsistencies,
>>
>>  From experience, that goal will never occur. The ImageJ2 design
>> fundamentally cannot be 100% consistent with ImageJ 1.x. Not even ImageJ
>> 1.x is 100% consistent with previous versions of ImageJ 1.x. And no
>> software of this magnitude is bug-free, either.
>>
>
> Yes. I may have overstated that goal. It would be a more realistic goal to
> have it bug-free enough that it is not super-annoying to the casual user.
>
>
>   > (2) have good end-user documentation so they know how to do things the
>>  > new way,
>>
>> Indeed, we had a whole milestone dedicated to documentation in our
>> original release plan:
>> http://trac.imagej.net/milestone/imagej2-b11-docs
>>
>>
> OK.
>
>
>   > and (3) have provided users with at least one "killer" feature that
>>  > they have longed for.
>>
>> Many such "killer" feature ideas were discussed on the ImageJX list when
>> ImageJ2 was first launched:
>> * https://groups.google.com/d/msg/imagejx/gz7cgytSRuA/emlJLp8o7XYJ
>> * https://groups.google.com/d/msg/imagejx/_yaczl4UWK4/_w6dCnGcJ1QJ
>> * https://groups.google.com/d/msg/imagejx/lD4s32M5als/HZiEA02LhXsJ
>> * https://groups.google.com/d/msg/imagejx/ox2ooizORA4/scMd4P0cRZAJ
>> * https://groups.google.com/d/msg/imagejx/F3gWc_Ndz_U/UTKiut-HuQMJ
>> * https://groups.google.com/d/msg/imagejx/79rryiWqFno/Ne--gMjkSX8J
>> * https://groups.google.com/d/msg/imagejx/pL6ipxHkAk8/26u5MF41YGsJ
>>
>> There are many others in the issue tracker:
>> * Coherent I/O (complete): http://trac.imagej.net/ticket/9
>> * Separation of concerns (complete): http://trac.imagej.net/ticket/10
>> * Better plugin framework (complete): http://trac.imagej.net/ticket/11
>> * Better scripting (complete): http://trac.imagej.net/ticket/12
>> * Better event handling (complete): http://trac.imagej.net/ticket/14
>> * N-dimensional images (complete): http://trac.imagej.net/ticket/17
>> * Very large image data (complete): http://trac.imagej.net/ticket/20
>> * CellProfiler interoperability (complete):
>> http://trac.imagej.net/ticket/1002
>> * KNIME interoperability (complete): http://trac.imagej.net/ticket/1004
>> * OMERO interoperability: http://trac.imagej.net/ticket/1003
>> * Very large image planes: http://trac.imagej.net/ticket/19
>> * Better undo/redo: http://trac.imagej.net/ticket/13
>>
>
> Absolutely a killer feature, but I am guessing difficult to implement in a
> plugin-based framework.
>
>
>  * Coordinate systems: http://trac.imagej.net/ticket/40
>> * Metadata: http://trac.imagej.net/ticket/8
>>
>> I know a lot of those are in some sense architectural, but many of them
>> have huge impact for users, too. Happy to elaborate on any specific
>> aspects of these.
>>
>>  > I think there is one thing that can be done in terms of point (3) that
>>  > will make many users happy is a "pin" button in each image
>>  > window/dialog. If the "pin" is activated then the window/dialog will
>>  > be brought to the foreground every time the user brings any other
>>  > "pinned" window or the main imagej bar to the foreground. This solves
>>  > the perrenial usability problem of ImageJ1 where if you have multiple
>>  > images open, you have to hunt for the right image, then hunt for the
>>  > brightness/contrast dialog, then hunt for the channels dialog in the
>>  > taskbar/dock.
>>
>> Note that there are shortcuts for many of ImageJ's windows, which reduce
>> the need to hunt through the taskbar. E.g.: Shift+C for the B&C window,
>> Shift+Z for channels, Enter for the main ImageJ window.
>>
>
> Yes, I've been using those shortcuts, but it is still annoying to have to
> find the image, and then press shift-Z and shift-C every time you switch
> back and forth between applications. Most annoyingly, the ImageJ toolbar
> disappears (is there a shortcut to bring it to foreground???) unless you
> set it always on top, which in turn makes it get in the way of other
> applications. What many users do is switch back and forth between ImageJ
> and other apps that allow them to put figures together
> (Illustrator/Photoshop/PowerPoint/Word) or analyze the data
> (Excel/R/Statistica). Right now I spend half my time fishing for the right
> image/dialog/button.
>
>
>
>>  > I don't think this should be a difficult feature to implement and I
>>  > can try to do that, but I will need time to plow through the codebase
>>  > and take it all in.
>>
>> This feature could also be implemented for the ImageJ 1.x UI, which
>> would get it into the hands of users more immediately. And you could
>> easily distribute it via an ImageJ update site; see
>> http://wiki.imagej.net/Update_Sites.
>>
>
> Sounds good. I'll give it a stab.
>
>
>
>>  > I hope that my comments will help with the development. I am hoping to
>>  > contribute to the actual work soon.
>>
>> Development of the core ImageJ system is a substantially different
>> endeavor from feature ideas like better window management, analysis
>> plugins, etc. The question is: which sort of project are you more
>> interested in working on?
>>
>
> I think I will be interested in features development more than the core,
> but that may change as I get more familiar with the core.
>
>
>
>>  > Please let me know your thoughts. Also, I thought that I might want to
>>  > send these comments to you personally rather than to the ImageJ-devel
>>  > list, since I don't want to step on anyones toes. I imagine there is
>>  > more to your decision to ship on June 1st than just software
>>  > excellence - things like funding, publications, etc. Please feel free
>>  > to forward my letter to the imagej-devel list if you think this will
>>  > be constructive.
>>
>> Thanks. As I said before, the imagej-devel list is the best place to
>> discuss these matters. ImageJ is an open source project, and as such is
>> best discussed in public to keep the community informed of the current
>> development directions, invite feedback and constructive criticism from
>> interested parties, etc.
>>
>
> Got it.
>
>
>> Regards,
>> Curtis
>>
>>
> Regards,
>
> Pawel
>
> --
> Paweł Niewiadomski
> e-mail: pawelthebiologist at gmail.com
> website: www.pawelthebiologist.com
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://imagej.net/pipermail/imagej-devel/attachments/20140513/e5f98f28/attachment-0001.html>


More information about the ImageJ-devel mailing list