[ImageJ-devel] SemVer policies for ImageJ2 and Fiji

[Stephan Saalfeld]

Hi Mark,

>[...] but not the traditional definition of public API[1,2].

This is what worries me because for most things that are less trivial
than addition, `traditional' definitions are not available, i.e.
`unintended' is not well defined.

> Just to make sure we're on the same page - when you say "changes the
> output", I assume you mean "changes the behavior but not the API - i.e.
> return type is unchanged"

In a typed language yes, what about Python?  What I mean is indeed
change in behavior---correct.  I proposed to treat change in behavior as
change of API because that would enable to reason about compatibility.
If we enable PATCH upgrades to change behavior but only require `compile
time' compatibility (which limits the scope of PATCH to statically typed
and compiled languages which, I think, was not intended), then all we
can reason from seeing a PATCH upgrade is that the stuff will compile,
and somehow run, no guarantees about behavior.  That's not very helpful
because compiling and running stuff isn't expensive to just try.
Behavior, however, is critical and complicated to test, having a
contract that makes guarantees here would indeed be useful.

> In that case, I think it would be cleaner to just eliminate the PATCH
> number - because every bug fix necessarily changes behavior, right? So with
> this scheme, MAJOR increases = "existing behavior has changed", and MINOR
> increases = "new behavior is available".
> 

Not necessarily.  Performance improvements, crash-fixes, compatibility
adjustments to upgraded dependencies (!) would fall into PATCH.  But I
see that even crashes could be used by dependents as part of the API...
we're doomed.

> Some problems with creating this versioning scheme:
> 1. MAJOR versions will increase rapidly. This is aesthetic, but one that
> people can react very negatively to - and can certainly be confusing if
> people don't expect MAJOR version bumps to cover bug fixes.

Right---but SemVer is already counterintuitive compared to romantic
versioning where a MAYOR upgrade is associated with a lot of new
features, not breakage.

> 2. It's one more thing for external developers to learn. We can't just say
> "we use SemVer".

Very true.

> 3. Since this is an internal versioning scheme, it may not be easy to
> compare our versions with external project versions that use SemVer.

Why would we want to do this?

> 4. There will be false negatives for MAJOR version compatibility
> comparisons (instead of what could be considered false positives with
> SemVer)

I wouldn't call them false but conservative or overly cautious.  It
would tell you when compatibility is not guaranteed which is useful.

> 
> Unfortunately, #1 above alone makes it very unlikely that we would want to
> adopt this use of version numbers.

I understand and fully agree.  But SemVer with PATCHES of `unintended
behavior' or other sentimental or romantic versioning schemes (I am
stealing these terms from

https://gist.github.com/jashkenas/cbd2b088e20279ae2c8e

and 

http://sentimentalversioning.org/

which are both exciting reads) aren't useful to reason about
compatibility.  We should therefore not use them to reason about
compatibility which you correctly state in the following...

> But what you're trying to do here -
> ensure compatibility - is fantastic and something that would be great to
> have.
> 
> So let's take a step back and look at what guarantees we do and do not have
> right now:
> 
> + We have reproducible builds (release couplings, which requires -some-
> versioning scheme to be used)
> + We have API compatibility (SemVer)
> 

No---there is no strict API compatibility when using PATCH to fix
`unintended behavior'.  Happened with ImageJ many times over the last
years: API compile-time compatible that would qualify as PATCH upgrade,
behavior different, plugins delivering crazy output, discovered only
much later through bug reports.

> - We do not have strict behavior compatibility
> - We do not have dependency compatibility
> 
> Behavior and dependency compatibility are very closely related - if they
> were covered by a versioning scheme, we could automatically answer the
> question "is it safe to drop in version X to replace version Y?".
> 

Yes.

> However, I do not think we should conflate these concerns with SemVer.

I could not agree more.

> Instead, two potential options would be:
> 
> 1) Continue to use SemVer, accept its limitations, be content with
> reproducibility.

Or any other versioning scheme that people like.  My current impression
is that SemVer doesn't help much.  I will keep using it, but it isn't
any better than any other scheme that assigns keys to versions and
enables to order them.

> 2) Create a separate versioning scheme that covers behavior and dependency
> compatibility. Use it in tandem with SemVer.
> 

You're absolutely right, but---phew---that's probably asking too much of
people that are already unwilling to adopt to something as simple as
SemVer.

My current thinking is that we should require contributors to use a
versioning scheme that can be sorted correctly by

sort -V

Anything else, i.e. compatibility guarantees, aren't possible to express
with a simple number and require runtime testing (e.g. unit-tests
promise this int an ideal world).  We should therefore never drop any
upgrade without significant attempt to test.  We could assume that a
maven build includes sufficient [unit-]testing to enable deployment if
it succeeds (this is not true in reality but it is a meaningful
contract).  I.e. artifacts can be deployed if they were explicitly built
against upgraded dependencies.  In our heterogeneous environment,
however, it is unrealistic to expect all contributors to adapt their
code in time.  Instead, we should try to build pom-fiji including all
downstream modules *overriding* or *upgrading* their dependency versions
to the highest possible.  Do you have something like this already or do
you have an idea how that could be done?

Thanks for all the energy that you're putting into this!

Cheers,
Stephan



> If anyone can think of other examples of versioned guarantees that would be
> useful to have, or counter-examples to any claims made here - please share!
> 
> Also, please let me know if any of this is confusing and/or additional
> examples would be useful.
> 
> Thanks again for the continued discussion,
> Mark
> 
> [1]
> http://en.wikipedia.org/wiki/Application_programming_interface#API_in_object-oriented_languages
> [2]
> http://stackoverflow.com/questions/2954372/difference-between-spi-and-api