2017-12-04 - Fiji + KNIP hackathon

Revision as of 03:41, 7 December 2017 by Chaubold (talk | contribs) (add more notes to H5, not HDF5)

From Monday, December 4, 2017 through Friday, December 15, 2017, the Max Planck Institute of Molecular Cell Biology and Genetics hosts ~50 developers at the Center for Systems Biology in Dresden, Germany for a hackathon to develop ImageJ2 and Fiji core infrastructure and plugins.


Voluntary hackathon calendar


Hackathon google doc


Hot Topics (Technical Discussions)

H5, not HDF5

For more info on H5, check out the github repository here.

Perhaps make it versioned? Because a duck is not always a duck...

  • Philip Hanslovsky presented it
  • feels like HDF5, but stores chunks(blocks) in separate files in the file system.
  • is a Java library, but Constantin Pape already wrote a C++/python version of it: z5 (also matches "zarr" library), https://github.com/constantinpape/z5
  • there is a special type for label blocks
  • blocks can have a halo
  • the block grid does not need to be filled dense, some blocks could be missing
  • attributes are stored in an additional JSON file
  • Discussion: should we define standard now as to how data should be stored in there to prevent an emergence of a zoo of different flavors as there is for HDF5?
   * how to do time series where each timestep / angle could have different image size
   * if we want a general "N5 viewer" for images, we'd have to add calibration data
   * put this information around the N5 dataset, because it behaves more like a dataset within an H5 file.
   * at least add a version number!
  • why another file format?
   * parallel writes (awesome for clusters with shared filesystem)
   * provides 
   * couldn't this just be another flavor of HDF5?

Hackathon Progress