Difference between revisions of "Bug reporting best practices"

m (Environment information)
(Refactor point of view to be less Fiji-centric (general ImageJ))
Line 1: Line 1:
 
= Why put effort into bug reports =
 
= Why put effort into bug reports =
  
For light reading, there are numerous guides and essays on [http://www.chiark.greenend.org.uk/~sgtatham/bugs.html how and why to write excellent bug reports]. In the scope of an open-source development community like Fiji's, key points include:
+
For light reading, there are numerous guides and essays on [http://www.chiark.greenend.org.uk/~sgtatham/bugs.html how and why to write excellent bug reports]. ImageJ users should be aware that the development community is diverse: from publicly-funded individuals and teams to scientists and user contributors. For a largely open-source community like this, there are several key points to consider when submitting a bug:
  
* Open-source teams are typically smaller and lack dedicated testing teams. Development tends to be complaint-driven, relying on an active user base to provide feedback.
+
* Teams developing open-source code are typically smaller and lack dedicated testing teams. Developers thus rely on an active and vocal community to provide feedback and guide the development process by identifying the areas in need of attention (complaint-driven development).
* If you encounter a bug, you probably want it to be fixed quickly. The better the bug report, the faster a developer will be able to reproduce and address the issue. Poorly written bug reports are more likely to sit unanswered - not because developers see the issue as unimportant, but the time required to understand what went wrong presents a significant barrier when setting priorities within an overflowing schedule.
+
* If you encounter a bug, it is likely interfering with your desired workflow and needs to be resolved quickly. The better the bug report, the faster a developer will be able to reproduce and address the issue. Poorly written bug reports are more likely to sit unanswered - not because developers see the issue as unimportant, rather that the time required to clarify the bug report itself presents a significant barrier when setting priorities within an overflowing schedule.
 +
* When you find a bug, it is unlikely that you are the only individual affected by it. By reporting a bug in a way that developers can understand, identify and resolve the issue, you are performing a necessary and valuable service to the entire ImageJ community.
  
 
= Components of a complete bug report =
 
= Components of a complete bug report =
  
There are three critical components in a bug report for Fiji. If any of these are missing, the usefulness of that report may be limited.
+
There are three critical components in an ImageJ bug report. If a report is missing any of these components, its usefulness may be limited.
  
 
== Environment information ==
 
== Environment information ==
  
Fiji is just ImageJ... but it's also a whole lot more. Reporting the ImageJ 1.x version (e.g. 1.49e) in which a bug appears is helpful, but understand that Fiji ships with layers upon layers of software to provide rich features and plugins. Furthermore, Fiji/ImageJ are extensible via the update sites and manually distributed plugins.
+
ImageJ is a flexible and extensible platform, so the actual "ImageJ environment" can vary from user to user. A common misunderstanding in bug reports is to just report the version of the base ImageJ component (e.g. 1.49e). This is helpful, but says nothing about what plugins, update sites, etc... are in use.
  
Errors can appear in any component of the software, thus a complete view of a user's working environment is imperative. For your convenience, Fiji is capable of reporting its own environment. If you use the ''Help>Report a Bug'' command in Fiji, this information will automatically be populated and included in the [http://fiji.sc/bugzilla/ bugzilla report].
+
Errors can appear in any component of the software, and in some cases two plugins might work individually but have some negative interaction with each other. Thus a complete view of the ImageJ environment is imperative. Fiji currently ships with a ''Help>Report a Bug'' command; this will automatically populate the complete environment information and include it in the [http://fiji.sc/bugzilla/ bugzilla report].
  
Note that if you file a bug manually or are enquiring to the mailing list, you can still copy and paste the contents of the ''"Useful information about your system"'' text box. It will be greatly appreciated.
+
<div style="background:#fCC; padding: 10px 10px 0 10px; border: 1px solid black;">This bug-reporting functionality will be migrated to core ImageJ.</div>
 +
 
 +
 
 +
Note that if you file a bug manually or are enquiring to a [[Mailing_Lists|mailing list]], you can still copy and paste the contents of the ''"Useful information about your system"'' text box from the ''Report a Bug'' dialog. It will be greatly appreciated.
  
 
== Minimal and precise steps to reproduce ==
 
== Minimal and precise steps to reproduce ==
Line 22: Line 26:
 
When we're in a hurry, it's easy to provide a brief overview a bug without actually describing how to reproduce the error. We are also prone to providing too much information, which can confuse the issue and discourage thorough reading.
 
When we're in a hurry, it's easy to provide a brief overview a bug without actually describing how to reproduce the error. We are also prone to providing too much information, which can confuse the issue and discourage thorough reading.
  
The actual text of your bug report should simply describe the fewest steps possible to reproduce your issue. Additional information is typically unnecessary... if a developer can reproduce the problem, they will do their best to fix it.
+
The actual text of your bug report should succinctly describe the fewest steps possible to reproduce your issue. For example:
 +
 
 +
:1) Open sample image "blobs"
 +
:2) Run auto-threshold command
 +
:3) Run subtract background command
 +
:At this point, an evil kraken appears and sinks my hard drive.
 +
 
 +
Additional information is typically unnecessary... if a developer can reproduce the problem, they will do their best to fix it.
  
 
== Sample data ==
 
== Sample data ==
Line 28: Line 39:
 
Developers typically have a cache of sample data for testing their application. That said, we are still striving to reproduce the original environment of the error. Having the original image that caused the error is the best possible way to test.
 
Developers typically have a cache of sample data for testing their application. That said, we are still striving to reproduce the original environment of the error. Having the original image that caused the error is the best possible way to test.
  
The easiest and safest way to provide sample data is via the ''Help>Upload Sample Image'' command in Fiji. This sends the image to a website restricted to Fiji wiki users with explicit authentication (reserved for trusted developers). This allows proprietary data to be tested without fear of public distribution.
+
If your test data is small and public, you can typically attach it to the bug report.
 +
 
 +
If you are using a Fiji distribution, the easiest and safest way to provide sample data is via the ''Help>Upload Sample Image'' command. This sends the image to a website restricted to Fiji wiki users with explicit authentication (reserved for trusted developers). This allows proprietary data to be tested without fear of public distribution.
  
Note: if you do upload data, you should mention the name of the dataset in your bug report. This will help developers find your data quickly.
+
Note: if you do upload sample data via Fiji, you should mention the name of the dataset in your bug report. This will help developers find your image(s) quickly.
  
 
For bugs working with excessively large datasets, you may need to communicate with a developer to determine the best way to test on your data.
 
For bugs working with excessively large datasets, you may need to communicate with a developer to determine the best way to test on your data.
Line 36: Line 49:
 
= While you're waiting... =
 
= While you're waiting... =
  
If a showstopping bug comes up, you've reported it, but need to continue your work, you still have options available.
+
If you have encountered and reported a bug that is completely blocking your work, you still have options available while waiting for the issue to be resolved.
  
 
== Disable SCIFIO ==
 
== Disable SCIFIO ==
  
[http://imagej.net/ImageJ2 ImageJ2] is a recent addition to Fiji. It represents the next steps in extensible image analysis, but due to its vast scope there are inevitably issues remaining. If you run into an issue of the style where a dataset that used to open correctly for you is broken after updating, you can try disabling SCIFIO via the ''Edit>Options>ImageJ2'' command. This will allow you to continue to work with Fiji/ImageJ while waiting for the fix in ImageJ2.
+
In recent months we have begun to provide an alternative to the hard-coded case logic of ImageJ 1.x's image I/O to a new, highly extensible paradigm: [http://imagej.net/ImageJ2 ImageJ2]. It represents the next steps in scientific image analysis, but due to the vast scope of the overhaul there are inevitably issues remaining. For example, if you run into an issue of the style where a dataset that used to open correctly for you is broken after updating, you can check if you have a ''Edit>Options>ImageJ2'' command and '''un'''check the option to use SCIFIO. This will allow you to revert to the class image I/O until the new I/O is fixed or improved.
  
Note: even if disabling SCIFIO fixes the issue for you, '''please''' still report the discovered bug. The long-term goal is to migrate completely to SCIFIO, so if there are problems we need to know about them.
+
Note: even if disabling SCIFIO fixes the issue for you, '''please''' still report the discovered bug. The long-term vision for ImageJ is to migrate completely to the new image I/O paradigm, so if there are problems we need to know about them.
  
 
== Catastrophic failure ==
 
== Catastrophic failure ==
  
Check out the lifeline versions on the [[Downloads|Downloads page]].
+
We maintain lifeline ImageJ distributions on the [[Downloads|Downloads page]]. You can use these until any outstanding issues are resolved.

Revision as of 09:58, 10 July 2014

Why put effort into bug reports

For light reading, there are numerous guides and essays on how and why to write excellent bug reports. ImageJ users should be aware that the development community is diverse: from publicly-funded individuals and teams to scientists and user contributors. For a largely open-source community like this, there are several key points to consider when submitting a bug:

  • Teams developing open-source code are typically smaller and lack dedicated testing teams. Developers thus rely on an active and vocal community to provide feedback and guide the development process by identifying the areas in need of attention (complaint-driven development).
  • If you encounter a bug, it is likely interfering with your desired workflow and needs to be resolved quickly. The better the bug report, the faster a developer will be able to reproduce and address the issue. Poorly written bug reports are more likely to sit unanswered - not because developers see the issue as unimportant, rather that the time required to clarify the bug report itself presents a significant barrier when setting priorities within an overflowing schedule.
  • When you find a bug, it is unlikely that you are the only individual affected by it. By reporting a bug in a way that developers can understand, identify and resolve the issue, you are performing a necessary and valuable service to the entire ImageJ community.

Components of a complete bug report

There are three critical components in an ImageJ bug report. If a report is missing any of these components, its usefulness may be limited.

Environment information

ImageJ is a flexible and extensible platform, so the actual "ImageJ environment" can vary from user to user. A common misunderstanding in bug reports is to just report the version of the base ImageJ component (e.g. 1.49e). This is helpful, but says nothing about what plugins, update sites, etc... are in use.

Errors can appear in any component of the software, and in some cases two plugins might work individually but have some negative interaction with each other. Thus a complete view of the ImageJ environment is imperative. Fiji currently ships with a Help>Report a Bug command; this will automatically populate the complete environment information and include it in the bugzilla report.

This bug-reporting functionality will be migrated to core ImageJ.


Note that if you file a bug manually or are enquiring to a mailing list, you can still copy and paste the contents of the "Useful information about your system" text box from the Report a Bug dialog. It will be greatly appreciated.

Minimal and precise steps to reproduce

When we're in a hurry, it's easy to provide a brief overview a bug without actually describing how to reproduce the error. We are also prone to providing too much information, which can confuse the issue and discourage thorough reading.

The actual text of your bug report should succinctly describe the fewest steps possible to reproduce your issue. For example:

1) Open sample image "blobs"
2) Run auto-threshold command
3) Run subtract background command
At this point, an evil kraken appears and sinks my hard drive.

Additional information is typically unnecessary... if a developer can reproduce the problem, they will do their best to fix it.

Sample data

Developers typically have a cache of sample data for testing their application. That said, we are still striving to reproduce the original environment of the error. Having the original image that caused the error is the best possible way to test.

If your test data is small and public, you can typically attach it to the bug report.

If you are using a Fiji distribution, the easiest and safest way to provide sample data is via the Help>Upload Sample Image command. This sends the image to a website restricted to Fiji wiki users with explicit authentication (reserved for trusted developers). This allows proprietary data to be tested without fear of public distribution.

Note: if you do upload sample data via Fiji, you should mention the name of the dataset in your bug report. This will help developers find your image(s) quickly.

For bugs working with excessively large datasets, you may need to communicate with a developer to determine the best way to test on your data.

While you're waiting...

If you have encountered and reported a bug that is completely blocking your work, you still have options available while waiting for the issue to be resolved.

Disable SCIFIO

In recent months we have begun to provide an alternative to the hard-coded case logic of ImageJ 1.x's image I/O to a new, highly extensible paradigm: ImageJ2. It represents the next steps in scientific image analysis, but due to the vast scope of the overhaul there are inevitably issues remaining. For example, if you run into an issue of the style where a dataset that used to open correctly for you is broken after updating, you can check if you have a Edit>Options>ImageJ2 command and uncheck the option to use SCIFIO. This will allow you to revert to the class image I/O until the new I/O is fixed or improved.

Note: even if disabling SCIFIO fixes the issue for you, please still report the discovered bug. The long-term vision for ImageJ is to migrate completely to the new image I/O paradigm, so if there are problems we need to know about them.

Catastrophic failure

We maintain lifeline ImageJ distributions on the Downloads page. You can use these until any outstanding issues are resolved.