NOTICE! This is a static HTML version of a legacy ImageJ Trac ticket.

The ImageJ project now uses GitHub Issues for issue tracking.

Please file all new issues there.

Ticket #20 (closed feature: fixed)

Opened 2010-02-17T15:12:48-06:00

Last modified 2013-06-07T15:19:52-05:00

Efficient support for very large image data

Reported by: curtis Owned by: curtis
Priority: critical Milestone: imagej2-b7-ndim-data
Component: ImgLib2 Version:
Severity: major Keywords:
Cc: Blocked By: #216, #583, #1669, #1872, #1885
Blocking:

Description

Datasets are growing in both size and complexity: we are now seeing acquisition systems capable of producing datasets larger than available memory (e.g., 1024x1024x16-bit x 500 timepoints x 50 focal planes = 49GB). The current version of ImageJ provides a mechanism, the virtual stack, for handling such large datasets one image plane at a time.

With imglib it should also be possible to work with such data using a clever storage strategy, but it would be nice to have at least one concrete example in code demonstrating how this should work.

Change History

comment:1 Changed 2011-02-25T10:49:37-06:00 by aivar

  • Status changed from new to accepted

See #20.

comment:2 Changed 2011-02-25T11:17:06-06:00 by aivar

  • Severity changed from non-issue to major
  • Milestone changed from progress-report to biweekly-2011: Mar-28 to Apr-08

comment:3 Changed 2011-04-25T11:33:43-05:00 by curtis

  • Milestone changed from biweekly-2011: Apr-11 to Apr-22 to biweekly-2011: Jun-06 to Jun-17

This sort of testing makes sense to do as part of the 2.0-beta1 development in June.

comment:4 Changed 2011-08-01T10:02:10-05:00 by curtis

  • Status changed from accepted to closed
  • Resolution set to duplicate

We are working on new ImgLib2 container types for this. See ticket #216.

comment:5 Changed 2012-02-23T11:22:14-06:00 by curtis

  • Type changed from task to story

comment:6 Changed 2012-02-23T11:24:21-06:00 by curtis

  • Status changed from closed to reopened
  • Summary changed from Verify very large image data can be handled efficiently to Efficient support for very large image data
  • Resolution duplicate deleted
  • Blocked By 216 added
  • Milestone changed from biweekly-2011: Jul-18 to Jul-29 to imagej-2.0-beta2

This ticket is now a story describing the end goal of supporting large image data. Ticket #216 (and likely others to be added later) will indicate the actual approaches and tasks used to accomplish this goal.

comment:7 Changed 2012-02-23T11:24:46-06:00 by curtis

  • Blocked By 216 removed

comment:8 Changed 2012-02-23T11:25:01-06:00 by curtis

  • Blocked By 216 added

comment:9 Changed 2012-02-26T21:19:36-06:00 by curtis

  • Blocked By 583 added

comment:10 Changed 2012-05-15T15:51:09-05:00 by curtis

  • Owner changed from aivar to curtis
  • Status changed from reopened to assigned
  • Milestone changed from imagej-2.0.0-beta3 to imagej-2.0.0-beta4

comment:11 Changed 2012-08-14T09:51:09-05:00 by curtis

  • Milestone changed from imagej-2.0.0-beta4 to imagej-2.0.0-beta5

The theme of beta5 will be more flexible data visualization and processing.

comment:12 Changed 2012-12-12T09:40:43-06:00 by bdezonia

Just a note that there are two test implementations of a file backed Img in imlgib2 branches: fileimg (by Tobias) and toy-file-backed-img (by Barry)

comment:13 Changed 2013-03-07T12:12:36-06:00 by bdezonia

  • Blocked By 1669 added

comment:14 Changed 2013-04-03T12:44:42-05:00 by curtis

  • Priority changed from major to critical

comment:15 Changed 2013-04-15T11:26:47-05:00 by dscho

  • Blocked By 1819 added

comment:16 Changed 2013-05-15T13:42:32-05:00 by bdezonia

  • Blocked By 1872 added

comment:17 Changed 2013-05-22T11:06:28-05:00 by bdezonia

  • Blocked By 1885 added

comment:18 Changed 2013-06-07T15:19:00-05:00 by curtis

  • Blocked By 1819 removed

(In #1819) Dscho: Where can I find such data? I'd like to load it into beta7 and take a screenshot for the blog post. (But maybe it's not possible yet without some sort of multi-angle registration feature? If so, we can bump this to beta8.)

I am changing this ticket to block the user story, because it is more about the blog post than any technical coding issue.

comment:19 Changed 2013-06-07T15:19:52-05:00 by curtis

  • Status changed from assigned to closed
  • Resolution set to fixed

Thanks to SCIFIOCellImg, we have initial support for huge image data. And this ticket only had to stay open for three and a half years!